Running pywb using mod_wsgi?

Help!
1

I know the documentation says use uwsgi for producation:
https://pywb.readthedocs.io/en/latest/manual/usage.html#deployment

But as uwsgi is no longer developed and for the sake of simple setup (for us that is simpler) is possible to run wayback using mod_wsgi (directly with Apache) instead?

We have tried just using it but we get no reploy, nothing in logs and finally a timeout.
I guess we need to configure something or it it not work.

Can someone shed some light on this?
Is it supposed to work? DO you have a working example setup?
Is it not working and maybe is it planned?
Or will it simply not work?

2

Hi @erikmelkersson I can definitely see why a mod_wsgi setup would be simpler since there isn’t a uwsgi process to manage. We use apache + mod_proxy_uwsgi + uwsgi in swap.stanford.edu since Apache is used for other web projects and there is an example in the pywb repository. I had initially been hoping to use mod_wsgi too, so I’ll be interested to hear how you proceed.

I think in principle it should work? What does your Apache and wsgi configuration look like? It might be useful to come up with a simple example using Docker’s httpd image, which we could put on Github and experiment with?

Also, FWIW, I wouldn’t read “maintenance mode” for uwsgi as necessarily a bad thing. I think it mostly means that the project has stabilized and only bug fixes will be addressed?

3

Tried this:

In apache virtual host:

  WSGIDaemonProcess pywb user=apache group=apache processes=3 threads=7
  WSGIScriptAlias / /usr/local/lib/python3.6/site-packages/pywb/apps/wayback.py
  WSGIScriptReloading On
  WSGIProcessGroup pywb
  WSGIApplicationGroup pywb
  • added some logging.

And in the log:

From the error-log:

 Wed Apr 05 09:36:45.542618 2023] [ssl:info] [pid 587140] [client IPHERE:36364] AH01964: Connection to child 3 established (server rulle.it.liu.se:443)
[Wed Apr 05 09:36:45.601458 2023] [ssl:info] [pid 587140] (70014)End of file found: [client 130.236.2.189:36364] AH01991: SSL input filter read failed.
[Wed Apr 05 09:37:08.577779 2023] [ssl:info] [pid 587144] [client IPHERE:53290] AH01964: Connection to child 7 established (server rulle.it.liu.se:443)
[Wed Apr 05 09:37:08.821992 2023] [wsgi:info] [pid 587135] mod_wsgi (pid=587135): Create interpreter 'pywb'.
[Wed Apr 05 09:37:08.830055 2023] [wsgi:info] [pid 587135] [remote IPHERE:53290] mod_wsgi (pid=587135, process='pywb', application='pywb'): Loading Python script file '/usr/local/lib/python3.6/site-packages/pywb/apps/wayback.py'.

An strace on the process gives repeating:

[pid 587147] <... futex resumed>)       = -1 ETIMEDOUT (Connection timed out)
[pid 587146] <... futex resumed>)       = -1 ETIMEDOUT (Connection timed out)
[pid 587147] futex(0x7ff18c7c0be0, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
[pid 587146] futex(0x7ff18c7c0be0, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
[pid 587147] <... futex resumed>)       = 0
[pid 587146] <... futex resumed>)       = 0
[pid 587147] futex(0x7ff18c7c0b88, FUTEX_WAIT_PRIVATE, 0, {tv_sec=0, tv_nsec=4901607} <unfinished ...>
[pid 587146] futex(0x7ff18c7c0b88, FUTEX_WAIT_PRIVATE, 0, {tv_sec=0, tv_nsec=4905277}^Cstrace: Process 587135 detached

From the error-log:

[Wed Apr 05 09:38:19.865078 2023] [ssl:info] [pid 587138] [client IPHERE:49880] AH01964: Connection to child 1 established (server rulle.it.liu.se:443)
[Wed Apr 05 09:38:19.924277 2023] [ssl:info] [pid 587138] (70014)End of file found: [client IPHERE:49880] AH01991: SSL input filter read failed.
[Wed Apr 05 09:39:09.022505 2023] [wsgi:error] [pid 587144] [client IPHERE:53290] Timeout when reading response headers from daemon process 'pywb': /usr/local/lib/python3.6/site-packages/pywb/apps/wayback.py, referer: https://HOSTHERE/

Later in error log: (restarded wsgi?)

[Wed Apr 05 09:41:45.540994 2023] [ssl:info] [pid 587141] [client IPHERE:28710] AH01964: Connection to child 4 established (server rulle.it.liu.se:443)
[Wed Apr 05 09:41:45.601391 2023] [ssl:info] [pid 587141] (70014)End of file found: [client IPHERE:28710] AH01991: SSL input filter read failed.
[Wed Apr 05 09:42:08.510014 2023] [wsgi:info] [pid 587135] mod_wsgi (pid=587135): Daemon process deadlock timer expired, stopping process 'pywb'.
[Wed Apr 05 09:42:08.510136 2023] [wsgi:info] [pid 587135] mod_wsgi (pid=587135): Shutdown requested 'pywb'.
[Wed Apr 05 09:42:13.510404 2023] [wsgi:info] [pid 587135] mod_wsgi (pid=587135): Aborting process 'pywb'.
[Wed Apr 05 09:42:13.510447 2023] [wsgi:info] [pid 587135] mod_wsgi (pid=587135): Exiting process 'pywb'.
[Wed Apr 05 09:42:14.465426 2023] [wsgi:info] [pid 589353] mod_wsgi (pid=589353): Attach interpreter ''.
[Wed Apr 05 09:42:14.468682 2023] [wsgi:info] [pid 589353] mod_wsgi (pid=589353): Imported 'mod_wsgi'.
[Wed Apr 05 09:42:14.468983 2023] [wsgi:debug] [pid 589353] src/server/mod_wsgi.c(9073): mod_wsgi (pid=589353): Started thread 0 in daemon process 'pywb'.
4

This sometimes work, but mostly breaks and gets timeouts etc. (no load on the server)
So something is not correct. Suggestions are very welcome.

<VirtualHost *:443>
  ServerName ...
  ## Alias declarations for resources outside the DocumentRoot
  Alias /static "/...path.../pywb/lib/python3.6/site-packages/pywb/static"

  ## Logging 
  ...
  ## SSL directives
 ...

  ## WSGI configuration
  WSGIApplicationGroup %{GLOBAL}
  WSGIDaemonProcess pywb display-name=%{GROUP} processes=2 python-home=/...path.../pywb threads=15
  WSGIProcessGroup pywb
  WSGIScriptAlias / "/...path.../pywb/lib/python3.6/site-packages/pywb/apps/wayback.py"
</VirtualHost>
5

Hmmm… I seem to be onto something.
If I use

 WSGIDaemonProcess pywb display-name=%{GROUP} processes=2 python-home=/...path.../pywb threads=1

it works fine. Note the “threads=1”,

I interpret this as pywb is not thread safe.