Warning: Can't synchronize with repository "(default)" (Unsupported version control system "svn": No module named svn). Look in the Trac log for more information.

Ticket #1508 (closed defect: fixed)

Opened 10 years ago

Last modified 10 years ago

[PATCH] Paginate may cause SA to raise an exception in a specific scenario

Reported by: roger.demetrescu Owned by: roger.demetrescu
Priority: normal Milestone: 1.0.4
Component: TurboGears Version: 1.0.4b1
Severity: major Keywords: paginate sqlalchemy query
Cc:

Description

Steps to reproduce

  1. Apply vhata's patch from #1434 (required)
  2. Unzip and run the demo project (attached)
  3. Click on the column titles, in the following order: AAA, BBB, CCC, DDD, CCC
  4. Enjoy the exception: ... :)
 500 Internal error

The server encountered an unexpected condition which prevented it from fulfilling the request.

Page handler: <bound method Root.index of <demo.controllers.Root object at 0x01785870>>
Traceback (most recent call last):
  File "c:\python25\lib\site-packages\cherrypy-2.2.1-py2.5.egg\cherrypy\_cphttptools.py", line 105, in _run
    self.main()
  File "c:\python25\lib\site-packages\cherrypy-2.2.1-py2.5.egg\cherrypy\_cphttptools.py", line 254, in main
    body = page_handler(*virtual_path, **self.params)
  File "<string>", line 3, in index
  File "C:\Turbogears1.0\turbogears\controllers.py", line 344, in expose
    *args, **kw)
  File "<string>", line 5, in run_with_transaction
  File "C:\Turbogears1.0\turbogears\database.py", line 366, in sa_rwt
    retval = func(*args, **kw)
  File "<string>", line 5, in _expose
  File "C:\Turbogears1.0\turbogears\controllers.py", line 359, in <lambda>
    mapping, fragment, args, kw)))
  File "C:\Turbogears1.0\turbogears\controllers.py", line 399, in _execute_func
    return _process_output(output, template, format, content_type, mapping, fragment)
  File "C:\Turbogears1.0\turbogears\controllers.py", line 86, in _process_output
    fragment=fragment)
  File "c:\turbogears1.0\turbogears\view\base.py", line 129, in render
    return engine.render(**kw)
  File "c:\python25\lib\site-packages\TurboKid-1.0.3-py2.5.egg\turbokid\kidsupport.py", line 198, in render
    output=output, format=format)
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\__init__.py", line 301, in serialize
    raise_template_error(module=self.__module__)
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\__init__.py", line 299, in serialize
    return serializer.serialize(self, encoding, fragment, format)
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\serialization.py", line 107, in serialize
    text = ''.join(self.generate(stream, encoding, fragment, format))
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\serialization.py", line 629, in generate
    for ev, item in self.apply_filters(stream, format):
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\serialization.py", line 165, in format_stream
    for ev, item in stream:
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\parser.py", line 221, in _coalesce
    for ev, item in stream:
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\serialization.py", line 477, in inject_meta_tags
    for ev, item in stream:
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\parser.py", line 179, in _track
    for p in stream:
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\filter.py", line 32, in apply_matches
    item = stream.expand()
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\parser.py", line 108, in expand
    for ev, item in self._iter:
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\parser.py", line 179, in _track
    for p in stream:
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\parser.py", line 221, in _coalesce
    for ev, item in stream:
  File "C:\tmp\demo\demo\templates\welcome.py", line 58, in _pull
  File "C:\Turbogears1.0\turbogears\widgets\base.py", line 268, in display
    return view.engines.get('kid').transform(params, self.template_c)
  File "c:\python25\lib\site-packages\TurboKid-1.0.3-py2.5.egg\turbokid\kidsupport.py", line 218, in transform
    return kid.ElementStream(t.transform()).expand()
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\parser.py", line 108, in expand
    for ev, item in self._iter:
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\parser.py", line 179, in _track
    for p in stream:
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\parser.py", line 179, in _track
    for p in stream:
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\filter.py", line 26, in apply_matches
    for ev, item in stream:
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\parser.py", line 179, in _track
    for p in stream:
  File "c:\python25\lib\site-packages\kid-0.9.6-py2.5.egg\kid\parser.py", line 221, in _coalesce
    for ev, item in stream:
  File "c:\turbogears1.0\turbogears\widgets\templates\paginate_datagrid.py", line 205, in _pull
  File "c:\python25\lib\site-packages\sqlalchemy-0.3.10-py2.5.egg\sqlalchemy\orm\query.py", line 958, in __iter__
    return iter(self.select_whereclause())
  File "c:\python25\lib\site-packages\sqlalchemy-0.3.10-py2.5.egg\sqlalchemy\orm\query.py", line 358, in select_whereclause
    statement = self.compile(whereclause, **kwargs)
  File "c:\python25\lib\site-packages\sqlalchemy-0.3.10-py2.5.egg\sqlalchemy\orm\query.py", line 1161, in compile
    cf.traverse(o)
  File "c:\python25\lib\site-packages\sqlalchemy-0.3.10-py2.5.egg\sqlalchemy\sql.py", line 901, in traverse
    for c in t.get_children(**self.__traverse_options__):
AttributeError: 'int' object has no attribute 'get_children'
Error location in template file 'C:\\tmp\\demo\\demo\\templates\\welcome.kid'
between line 8 and line 9:
<body>
    ${grid.display(data)}

Powered by CherryPy 2.2.1

Explanation

There is a bug in the paginate.sort_ordering function, which manifests when you have 4 or more "columns" to reorder:

>>> from turbogears.paginate import sort_ordering
>>> d = dict()
>>> sort_ordering(d, 'a')   # user clicks on 'a' column
>>> sort_ordering(d, 'b')   # user clicks on 'b' column
>>> sort_ordering(d, 'c')   # user clicks on 'c' column
>>> sort_ordering(d, 'd')   # user clicks on 'd' column
>>> print d
{'a': [3, True], 'c': [1, True], 'b': [2, True], 'd': [0, True]}
>>> # as you can see, 'd' column has index 0 (zero), because it was the last column to be clicked
>>> # while 'a' has the highest index
>>> 
>>> # now, lets emulates the click on 'c' column again
>>> sort_ordering(d, 'c')   # user clicks on 'c' column
>>> print d
{'a': [3, True], 'c': [0, True], 'b': [3, True], 'd': [1, True]}
>>> 
>>> # oops... wheres the column with index = 2 ??
>>> # 'a' and 'b' column have the same index (3) !!
>>> #
>>> # Expected result:
>>> #
>>> # {'a': [3, True], 'c': [0, True], 'b': [2, True], 'd': [1, True]}
>>>

This gap can make SA generate this SQL:

SELECT 
    sampletable.aaa AS sampletable_aaa, 
    sampletable.bbb AS sampletable_bbb, 
    sampletable.id AS sampletable_id, 
    sampletable.ccc AS sampletable_ccc, 
    sampletable.ddd AS sampletable_ddd
FROM 
    sampletable 
ORDER BY 
    sampletable.ccc ASC, 
    sampletable.ddd ASC, 
    2,                       -- weird column
    sampletable.bbb ASC, 
    sampletable.id ASC
 LIMIT 10 OFFSET 0

This SQL is usually inoffensive when the Query object loads simple classes instances. However, when we have at least 1 non-lazy relation(...) in our Query, this SQL will make SA raise the following exception:

AttributeError: 'int' object has no attribute 'get_children'

So, the best thing to do is to make sure sort_ordering() function won't allow missing indexes in the ordering dictionary...

The following patch does this job... It also includes the 2nd patch from #1434.

Cheers,

Roger

Attachments

demo.zip Download (88.4 KB) - added by roger.demetrescu 10 years ago.
The bug demo project
sort_ordering.patch Download (1.8 KB) - added by roger.demetrescu 10 years ago.
sort_ordering_2.patch Download (1.1 KB) - added by roger.demetrescu 10 years ago.
Same patch, but without the changes on "def sql_get_column(colname, var_data):", which may be addressed in a separate changeset

Change History

Changed 10 years ago by roger.demetrescu

The bug demo project

Changed 10 years ago by roger.demetrescu

comment:1 Changed 10 years ago by roger.demetrescu

  • Summary changed from Paginate may cause SA to raise an exception in a specific scenario to [PATCH] Paginate may cause SA to raise an exception in a specific scenario

Changed 10 years ago by roger.demetrescu

Same patch, but without the changes on "def sql_get_column(colname, var_data):", which may be addressed in a separate changeset

comment:2 Changed 10 years ago by roger.demetrescu

  • Owner changed from anonymous to roger.demetrescu

comment:3 Changed 10 years ago by roger.demetrescu

  • Status changed from new to closed
  • Resolution set to fixed

Fixed in [3510] and [3511]

Note: See TracTickets for help on using tickets.