Warning: Can't synchronize with repository "(default)" (Unsupported version control system "svn": No module named svn). Look in the Trac log for more information.

Ticket #2031 (closed defect: fixed)

Opened 11 years ago

Last modified 11 years ago

[PATCH] Catwalk can't create "/catwalk-session" when deploying using mod_wsgi

Reported by: llucax Owned by: Chris Arndt
Priority: normal Milestone: 1.1
Component: Toolbox.Catwalk Version: 1.0.4.4
Severity: critical Keywords: patch
Cc:

Description

Hi. I've just upgraded to Ubuntu Intrepid (8.10) which comes with TG 1.0.4.4 from an Ubuntu Hardy (8.04) which came with TG 1.0.2.2 and I think I've hit a regression.

I've deployed my app using apache's mod_wsgi and I'm "mounting" Catwalk at my root controller using something like this:

admin = identity.SecureObject(CatWalk(model), identity.has_permission('admin'))

This worked perfectly in 1.0.2.2 (and still works perfectly in 1.0.4.4 if I use the built-in cherrypy server running start-myproj.py) but now I get this error:

[Wed Nov 12 20:44:36 2008] [error] [client 190.226.108.220] mod_wsgi (pid=6351): Target WSGI script '/var/www/entregas/index.wsgi' cannot be loaded as Python module.
[Wed Nov 12 20:44:36 2008] [error] [client 190.226.108.220] mod_wsgi (pid=6351): Exception occurred processing WSGI script '/var/www/entregas/index.wsgi'.
[Wed Nov 12 20:44:36 2008] [error] [client 190.226.108.220] Traceback (most recent call last):
[Wed Nov 12 20:44:36 2008] [error] [client 190.226.108.220]   File "/var/www/entregas/index.wsgi", line 25, in <module>
[Wed Nov 12 20:44:36 2008] [error] [client 190.226.108.220]     import sercom.controllers
[Wed Nov 12 20:44:36 2008] [error] [client 190.226.108.220]   File "/var/lib/sercom/sercom/sercom/controllers.py", line 83, in <module>
[Wed Nov 12 20:44:36 2008] [error] [client 190.226.108.220]     class Root(controllers.RootController):
[Wed Nov 12 20:44:36 2008] [error] [client 190.226.108.220]   File "/var/lib/sercom/sercom/sercom/controllers.py", line 251, in Root
[Wed Nov 12 20:44:36 2008] [error] [client 190.226.108.220]     admin = identity.SecureObject(CatWalk(model), identity.has_permission('admin'))
[Wed Nov 12 20:44:36 2008] [error] [client 190.226.108.220]   File "/var/lib/python-support/python2.5/turbogears/toolbox/catwalk/__init__.py", line 133, in __init__
[Wed Nov 12 20:44:36 2008] [error] [client 190.226.108.220]     "CatWalk failed to load your model file.\\n" + str(e))
[Wed Nov 12 20:44:36 2008] [error] [client 190.226.108.220] ImportError: CatWalk failed to load your model file.
[Wed Nov 12 20:44:36 2008] [error] [client 190.226.108.220] [Errno 13] Permission denied: '/catwalk-session'

I digged a little and found that this exception was masked by this code (on catwalk/init.py arround line 133 like the traceback says):

       except Exception, e:
            raise ImportError, (
                "CatWalk failed to load your model file.\n" + str(e))

So I temporarily remove that try/except block and found the original exception was thrown in the same file, arround line 600 in the state_path() method):

 597         catwalk_session_dir = os.path.join(turbogears.util.get_package_name()
 598             or '', 'catwalk-session')
 599         catwalk_session_dir = os.path.abspath(catwalk_session_dir)
 600         if not os.path.exists(catwalk_session_dir):
 601             try:
 602                 os.mkdir(catwalk_session_dir)
 603             except IOError, e:
 604                 msg = 'Fail to create session directory %s' % e
 605                 raise cherrypy.HTTPRedirect(turbogears.url('error', msg=msg))
 606         return os.path.join(catwalk_session_dir, 'session.pkl')

There are several problems here:

  1. turbogears.util.get_package_name() returns nothing, so catwalk_session_dir is "/catwalk-session" in line 597. I made turbogears.util.get_package_name() return "myproj" adding a package = "myproj" in the config file, so I could get catwalk_session_dir being "myproj/catwalk-session". But this doesn't fix the problem.
  2. line 599 makes catwalk_session_dir be "/myproj/catwalk-session", because it looks like for mod_wsgi the current working directory is "/".
  3. In line 603, OSError should be catched too (permission denied is an OSError).

Since I didn't knew how to fix this properly, I've made a simple change to be able to specify via config file where the catwalk-session directory should be:

--- /tmp/catwalk.__init__.py.orig       2008-11-12 21:08:38.000000000 -0200
+++ /var/lib/python-support/python2.5/turbogears/toolbox/catwalk/__init__.py    2008-11-12 21:38:05.000000000 -0200
@@ -594,13 +594,15 @@ class CatWalk(turbogears.controllers.Con
         Create a session directory if nescesary.
 
         """
-        catwalk_session_dir = os.path.join(turbogears.util.get_package_name()
-            or '', 'catwalk-session')
+        catwalk_session_dir = turbogears.config.get('catwalk.session-dir', None)
+        if catwalk_session_dir is None:
+            catwalk_session_dir = os.path.join(turbogears.util.get_package_name()
+                or '', 'catwalk-session')
         catwalk_session_dir = os.path.abspath(catwalk_session_dir)
         if not os.path.exists(catwalk_session_dir):
             try:
                 os.mkdir(catwalk_session_dir)
-            except IOError, e:
+            except (IOError, OSError), e:
                 msg = 'Fail to create session directory %s' % e
                 raise cherrypy.HTTPRedirect(turbogears.url('error', msg=msg))
         return os.path.join(catwalk_session_dir, 'session.pkl')

Then I add:

catwalk.session-dir = "/path/to/my/catwalk-session"

And all works as expected.

Attachments

catwalk-session.patch Download (1.8 KB) - added by llucax 11 years ago.

Change History

comment:1 Changed 11 years ago by faide

Could you test with 1.0.7 which is the latest stable version out there ?

comment:2 Changed 11 years ago by llucax

Is there any easy way to use mod_wsgi from a VirtualEnv?? Because I only have a Debian and Ubuntu box and both have 1.0.4.4 and I don't want to "messy_install" my instalation =)

comment:3 Changed 11 years ago by faide

Could you attach a patch to this ticket? Your proposed solution seems correct.

see  http://docs.turbogears.org/patching_guidelines for more info on how to do so.

comment:4 Changed 11 years ago by llucax

Patch attached. I couldn't run the tests because I'm having problems with setuptools. I guess some documentation should be added about the new catwalk.session-dir configuration variable, but I didn't know where to put it.

The patch is against 1.0 branch.

comment:5 Changed 11 years ago by faide

Nice. Could you propose also a patch against the QSTemplate so that each new quickstarted application would have an app.cfg file that contain the config key.

Ideally the config key would just be commented out and would be preceded by a 2 line explanation like: "If you deploy behind mod-wsgi you'll need to set this to an absolute path if you want to use catwalk mounted in your own controllers..."

Thanks for the help!

Changed 11 years ago by llucax

comment:6 Changed 11 years ago by llucax

Patch updated.

comment:7 Changed 11 years ago by faide

Thanks!

comment:8 Changed 11 years ago by Chris Arndt

Documentation should go here:

 http://docs.turbogears.org/1.1/Configuration

(in section "TurboGears dev.cfg and prod.cfg Configuration Reference")

This page can only be changed by editors though. If you could provide a patch against the page source that would be great! If not, let me know and just post an unpolished description here and I'll write up something.

comment:9 Changed 11 years ago by llucax

I guess a small paragraph like the one in the app.cfg comment should be enough. Like

catwalk.session-dir
    This is the Catwalk session directory. It's only needed if you deploy behind
    mod_wsgi and you want to use catwalk mounted in your own controllers.

    For example:

    catwalk.session-dir = "/var/cache/myapp/catwalk-session"

    It should be an absolute path to a directory that exists and is writeable by
    the server.

comment:10 Changed 11 years ago by Chris Arndt

Thanks, I'll add this to the page.

comment:11 Changed 11 years ago by Chris Arndt

  • Owner changed from anonymous to Chris Arndt

comment:12 follow-up: ↓ 13 Changed 11 years ago by Chris Arndt

  • Status changed from new to closed
  • Resolution set to fixed

Applied in r5733 with modifications. I changed the name of the config option to catwalk.session_dir (with underscore) to be more consistent with other config settings. Then I fixed a another bug in the old code where the path prefix would be the root directory (/) when the package directory could not be determined. It now uses the current working directory instead.

I also added more accurate descriptions of how the session dir is determined to the app.cfg template and the CatWalk docstrings.

BTW, the first exception you cited in the ticket summary, IMHO indicates the root of the problem, namely, that the server could not write to the /catwalk-session dir. That indicates that turbogears.util.get_package_name() could not determine your application's package name. Maybe you should set the package config option.

Finally, I added a description of the option to  http://docs.turbogears.org/1.1/Configuration.

comment:13 in reply to: ↑ 12 Changed 11 years ago by llucax

Replying to Chris Arndt:

BTW, the first exception you cited in the ticket summary, IMHO indicates the root of the problem, namely, that the server could not write to the /catwalk-session dir. That indicates that turbogears.util.get_package_name() could not determine your application's package name. Maybe you should set the package config option.

I did, see the ticket description. The problem was mod_wsgi have as current working directory /, so catwalk tried to store the sessions in /myproj/catwalk-session.

Thanks for you corrections and improvements.

Note: See TracTickets for help on using tickets.