Dispatcher regression in TG2.1:

[lmacken]

I'm experiencing some dispatcher regression in TurboGears 2.1dev when returning a PylonsApp? from WSGI middleware with a custom resolve method that returns a BaseController? which does a lookup to another Controller.

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/nose/case.py", line 182, in runTest
    self.test(*self.arg)
  File "/srv/moksha/moksha/tests/quickstarts/test_livewidget.py", line 66, in test_livewidget
    resp = self.app.get('/widgets/mokshatest')
  File "/usr/lib/python2.6/site-packages/webtest/__init__.py", line 153, in get
    expect_errors=expect_errors)
  File "/usr/lib/python2.6/site-packages/webtest/__init__.py", line 310, in do_request
    res = req.get_response(app, catch_exc_info=True)
  File "/usr/lib/python2.6/site-packages/webob/__init__.py", line 1321, in get_response
    application, catch_exc_info=True)
  File "/usr/lib/python2.6/site-packages/webob/__init__.py", line 1293, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.6/site-packages/webtest/lint.py", line 170, in lint_app
    iterator = application(environ, start_response_wrapper)
  File "/usr/lib/python2.6/site-packages/paste/cascade.py", line 130, in __call__
    return self.apps[-1](environ, start_response)
  File "/usr/lib/python2.6/site-packages/paste/registry.py", line 350, in __call__
    app_iter = self.application(environ, start_response)
  File "/usr/lib/python2.6/site-packages/pylons/middleware.py", line 201, in __call__
    self.app, environ, catch_exc_info=True)
  File "/usr/lib/python2.6/site-packages/pylons/util.py", line 94, in call_wsgi_application
    app_iter = application(environ, start_response)
  File "/usr/lib/python2.6/site-packages/weberror/evalexception.py", line 235, in __call__
    return self.respond(environ, start_response)
  File "/usr/lib/python2.6/site-packages/weberror/evalexception.py", line 418, in respond
    return self.application(environ, start_response)
  File "/usr/lib/python2.6/site-packages/tg/configuration.py", line 550, in remover
    return app(environ, start_response)
  File "/usr/lib/python2.6/site-packages/repoze/tm/__init__.py", line 19, in __call__
    result = self.application(environ, save_status_and_headers)
  File "/usr/lib/python2.6/site-packages/repoze/who/middleware.py", line 107, in __call__
    app_iter = app(environ, wrapper.wrap_start_response)
  File "/usr/lib/python2.6/site-packages/tw/core/middleware.py", line 43, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python2.6/site-packages/tw/core/middleware.py", line 68, in wsgi_app
    resp = req.get_response(self.application)
  File "/usr/lib/python2.6/site-packages/webob/__init__.py", line 1325, in get_response
    application, catch_exc_info=False)
  File "/usr/lib/python2.6/site-packages/webob/__init__.py", line 1293, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.6/site-packages/tw/core/resource_injector.py", line 68, in _injector
    resp = req.get_response(app)
  File "/usr/lib/python2.6/site-packages/webob/__init__.py", line 1325, in get_response
    application, catch_exc_info=False)
  File "/usr/lib/python2.6/site-packages/webob/__init__.py", line 1293, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.6/site-packages/beaker/middleware.py", line 73, in __call__
    return self.app(environ, start_response)
  File "/usr/lib/python2.6/site-packages/beaker/middleware.py", line 152, in __call__
    return self.wrap_app(environ, session_start_response)
  File "/usr/lib/python2.6/site-packages/routes/middleware.py", line 130, in __call__
    response = self.app(environ, start_response)
  File "/srv/moksha/moksha/middleware/csrf.py", line 188, in __call__
    response = request.get_response(self.application)
  File "/usr/lib/python2.6/site-packages/webob/__init__.py", line 1325, in get_response
    application, catch_exc_info=False)
  File "/usr/lib/python2.6/site-packages/webob/__init__.py", line 1293, in call_application
    app_iter = application(self.environ, start_response)
  File "/srv/moksha/moksha/middleware/middleware.py", line 85, in __call__
    return self.mokshaapp(environ, start_response)
  File "/usr/lib/python2.6/site-packages/pylons/wsgiapp.py", line 125, in __call__
    response = self.dispatch(controller, environ, start_response)
  File "/usr/lib/python2.6/site-packages/pylons/wsgiapp.py", line 324, in dispatch
    return controller(environ, start_response)
  File "/srv/moksha/moksha/lib/base.py", line 63, in __call__
    return TGController.__call__(self, environ, start_response)
  File "/usr/lib/python2.6/site-packages/pylons/controllers/core.py", line 221, in __call__
    response = self._dispatch_call()
  File "/usr/lib/python2.6/site-packages/pylons/controllers/core.py", line 172, in _dispatch_call
    response = self._inspect_call(func)
  File "/usr/lib/python2.6/site-packages/pylons/controllers/core.py", line 107, in _inspect_call
    result = self._perform_call(func, args)
  File "/usr/lib/python2.6/site-packages/tg/controllers/dispatcher.py", line 231, in _perform_call
    r = self._call(func, params, remainder=remainder)
  File "/usr/lib/python2.6/site-packages/tg/controllers/decoratedcontroller.py", line 98, in _call
    params, remainder = self._remove_argspec_params_from_params(controller, params, remainder)
  File "/usr/lib/python2.6/site-packages/tg/controllers/dispatcher.py", line 142, in _remove_argspec_params_from_params
    remainder[i] = params[var]
IndexError: list assignment index out of range

[lmacken]

This patch seems to fix the bug, however, I'm currently having issues getting the tg-dev test suite running at the moment, so I can't see if this breaks anything else.

--- ./controllers/decoratedcontroller.py	2009-09-27 07:55:43.000000000 -0400
+++ /usr/lib/python2.6/site-packages/tg/controllers/decoratedcontroller.py	2009-09-27 08:31:53.213842831 -0400
@@ -81,7 +81,7 @@
             controller.decoration.run_hooks('before_validate', remainder,
                                             params)
 
-            validate_params = self._get_params_with_argspec(controller, params, remainder)
+            validate_params, remainder = self._get_params_with_argspec(controller, params, remainder)
 
             for ignore in config.get('ignore_parameters', []):
                 if params.get(ignore):
--- ./controllers/dispatcher.py	2009-09-27 07:55:43.000000000 -0400
+++ /usr/lib/python2.6/site-packages/tg/controllers/dispatcher.py	2009-09-27 08:34:42.911905225 -0400
@@ -105,8 +105,8 @@
             for i, var in enumerate(argvars):
                 if i >= len(remainder):
                     break
-                params[var] = remainder[i]
-        return params
+                params[var] = remainder.pop(i)
+        return params, remainder
 
     def _remove_argspec_params_from_params(self, func, params, remainder):
         """Remove parameters from the argument list that are

[percious]

Can you provide a testcase that reproduces this result?

[lmacken]

I still have been unable to duplicate this issue in TG's test suite. However, I did instrument the dispatcher to get some potentially useful output...

DecoratedController._call({'params': {}, 'controller': <bound method WidgetController.default of <moksha.controllers.widgets.WidgetController object at 0x2dbc990>>, 'remainder': ['mokshatest'], 'self': <moksha.controllers.root.RootController object at 0x30ba9d0>})
_get_params_with_argspec(<bound method WidgetController.default of <moksha.controllers.widgets.WidgetController object at 0x2dbc990>>, {}, ['mokshatest'])
Returning {'widget': 'mokshatest'}
_perform_validate(<bound method WidgetController.default of <moksha.controllers.widgets.WidgetController object at 0x2dbc990>>, {'widget': 'mokshatest'})
Returning {'chrome': None, 'source': u'', 'widget': u'mokshatest', 'live': None, 'module': None}
_remove_argspec_params_from_params(<bound method WidgetController.default of <moksha.controllers.widgets.WidgetController object at 0x2dbc990>>, {'chrome': None, 'source': u'', 'widget': u'mokshatest', 'live': None, 'module': None}, ['mokshatest'])
EXCEPTION: list assignment index out of range

So it looks like it _get_params_with_argspec associates the 'mokshatest' argument with the 'widget' argument, which is fine. However, it still keeps 'mokshatest' as the remainder when passing it to _remove_argspec_params_from_params

[lmacken]

Also, this is the call stack that is exploding.

 https://fedorahosted.org/moksha/browser/moksha/middleware/middleware.py#L83  https://fedorahosted.org/moksha/browser/moksha/wsgiapp.py#L51  https://fedorahosted.org/moksha/browser/moksha/controllers/root.py#L35  https://fedorahosted.org/moksha/browser/moksha/controllers/widgets.py#L33

[lmacken]

My patch seems to cause 1 problem with the test suite

======================================================================
FAIL: test_defalt_with_args_a_b (tg.tests.test_tg_controller_dispatch.TestTGController)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/code/bitbucket.org/tg-dev/tg/tests/test_tg_controller_dispatch.py", line 405, in test_defalt_with_args_a_b
    assert "DEFAULT WITH ARGS a b" in r.body, r
AssertionError: Response: 200 OK
Cache-Control: no-cache
Content-Type: text/html; charset=utf-8
Pragma: no-cache
Set-Cookie:  beaker.session.id=0da707369d40489e887767be385268ac; Path=/
DEFAULT WITH ARGS b None
    r =self.app.get('/sub4/default_with_args/a/b')
>>  assert "DEFAULT WITH ARGS a b" in r.body, r

[percious]

I have determined the problem has to do with the way luke has his own dispatch mechanism, not how TG2 works. TG2 works as advertised.

[lmacken]

This seems to be fixed with the latest 2.1 code! Percious is my hero.