Ticket #817 (new defect)

Opened 2 years ago

Last modified 4 months ago

Upgrading past SVN 1081 breaks external modules' datababse connectivity.

Reported by: alice@gothcandy.com Assigned to: anonymous
Priority: high Milestone: __unclassified__
Component: SQLObject Version: 0.9a5
Severity: major Keywords:
Cc:

Description (Last modified by khorn)

There was some change around 1081 (as reported by Matthew Bevan in this post) that now causes a class (in an external module configured using TG's startup/shutdown notification entry point) inheriting from an InheritableSQLObject to throw the following traceback: 

Traceback (most recent call last):
  File "/usr/bin/tg-admin", line 7, in ?
    sys.exit(
  File "/usr/lib/python2.4/site-packages/setuptools-0.6a11-py2.4.egg/pkg_resources.py", line 236, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.4/site-packages/setuptools-0.6a11-py2.4.egg/pkg_resources.py", line 2097, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.4/site-packages/setuptools-0.6a11-py2.4.egg/pkg_resources.py", line 1830, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/home/mbevan/Projects/turbogears/turbogears/__init__.py", line 18, in ?
    ext = entrypoint.load()
  File "/usr/lib/python2.4/site-packages/setuptools-0.6a11-py2.4.egg/pkg_resources.py", line 1830, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/home/mbevan/Projects/TG-Content/content/__init__.py", line 23, in ?
    from content import controllers, model, widgets
  File "/home/mbevan/Projects/TG-Content/content/model.py", line 16, in ?
    class Bob(Test):
  File "/home/mbevan/Projects/turbogears/thirdparty/sqlobject/sqlobject/declarative.py", line 108, in __new__
    cls.__classinit__(cls, new_attrs)
  File "/home/mbevan/Projects/turbogears/thirdparty/sqlobject/sqlobject/main.py", line 748, in __classinit__
    if not connection and not getattr(cls, '_connection', None):
  File "/home/mbevan/Projects/turbogears/turbogears/database.py", line 148, in __get__
    self.set_hub()
  File "/home/mbevan/Projects/turbogears/turbogears/database.py", line 166, in set_hub
    raise KeyError, "No database configuration found!"
KeyError: 'No database configuration found!'

The above is thrown during TG initialization - applications no longer start up, and the tg-admin tool can no longer run.

I have been able to reproduce this error with a very simple (~60 lines of code including setup.py) set-up, and this error would even work as a system-wide DOS attack against the current version of TG. This is a show-stopping problem for the project Matthew and I are working on.

The simple case is attached. To reproduce:

  1. Extract the contents of the attached file (SimpleTest-1.0.0.tar.bz2) into a temporary directory.
  2. Enter the SimpleTest directory.
  3. Run sudo ./setup.py develop to install.
  4. Attempt to run tg-admin or any TurboGears application.
  • Alice McGregor

Attachments

SimpleTest-1.0.0.tar.bz2 (1.1 kB) - added by alice@gothcandy.com on 05/02/06 15:30:23.
Example code demonstrating the above flaw/exploit.

Change History

05/02/06 15:30:23 changed by alice@gothcandy.com

  • attachment SimpleTest-1.0.0.tar.bz2 added.

Example code demonstrating the above flaw/exploit.

05/02/06 15:37:47 changed by alice@gothcandy.com

  • priority changed from high to normal.

(Set the wrong field - sorry.)

05/02/06 17:00:13 changed by alice@gothcandy.com

I have narrowed the breaking change down to the following: 

--- turbogears/__init__.py (revision 1076)
+++ turbogears/__init__.py (revision 1087)
@@ -1,5 +1,7 @@
 "TurboGears Front-to-Back Web Framework"

+import pkg_resources
+
 from turbogears import config
 from turbogears.controllers import expose, flash, validate, redirect, \
                                    error_handler, exception_handler, url
@@ -11,6 +13,12 @@
 from turbogears.widgets import mochikit
 from turbogears.config import update_config

+extensions = pkg_resources.iter_entry_points("turbogears.extensions")
+for entrypoint in extensions:
+    ext = entrypoint.load()
+    if hasattr(ext, "tgsymbols"):
+        globals().update(ext.tgsymbols())
+
 i18n.install() # adds _ (gettext) to builtins namespace

Which hardly makes sense. Steps to reproduce the error:

  1. Follow the above instructions on installing SimpleTest?.
  2. Run the following: 
    svn co -r 1076 http://www.turbogears.org/svn/turbogears/trunk turbogears
  3. Install TurboGears in development mode: sudo python setup.py develop.
  4. Run tg-admin - successfully.
  5. Navigate to the turbogears/turbogears directory and execute the following: 
    svn up -r 1087 __init__.py
  6. Run tg-admin - with the error.

If I svn up to the latest revision and reverse this patch by hand (removing the import line and extensions block) it works like a charm. Over to you, Kevin! *grin*

(As of SVN 1076 there are no plugins using this tgsymbols feature. In the current SVN, plugins/fastdata/tgfastdata does.)

05/02/06 18:15:10 changed by alice@gothcandy.com

After some strong coffee and logical thinking I now understand the true problem:

  1. SQLObject should not be attempting to communicate with the database;
    1. At all during package initialization; and,
    2. Not before there is a database connection or full startup has been achieved.
  2. There should be some way to determine if a property, function, or method of an entry point is available without calling and thus initializing the whole package containing that entry point.

Temporarily, at least, the problem can be made to go away by removing or commenting out the tgsymbols related code from turbogears/__init__.py. This solution is, however, not optimal as being able to add properties to the turbogears namespace is useful. The only true fix is to change SQLObject's behavior to not query the database when initializing the child of an InheritableSQLObject. Unfortunately that operation is above -my- head. I'll post a link to this ticket on the SQLObject trac with a description of the main problem.

  • Alice

09/21/06 12:53:33 changed by khorn

  • description changed.
  • milestone changed from 0.8 to 1.0b3.

verified that this problem still exists using TG 1.0b1

assigning to milestone 1.0b3

01/11/07 14:52:14 changed by alberto

  • milestone changed from 1.0b3 to 1.1.

01/31/07 16:39:48 changed by jorge.vargas

  • priority changed from normal to high.

I can't see why they are not more reports of this in the mailing list.

03/28/07 12:42:01 changed by alberto

  • milestone changed from 1.1 to __unclassified__.

Batch moved into unclassified from 1.1 to properly track progress on the later

12/19/07 03:08:10 changed by Chris Arndt

  • milestone set to __unclassified__.

02/01/08 22:47:23 changed by khorn

verified that this problem still occurs in 1.0.1