Warning: Can't synchronize with repository "(default)" (Unsupported version control system "svn": No module named svn). Look in the Trac log for more information.

Ticket #9 (closed defect: wontfix)

Opened 14 years ago

Last modified 12 years ago

cherrypy autoreload magic causes headaches

Reported by: Attila Vágvölgyi Owned by: kevin
Priority: normal Milestone: 0.9
Component: Documentation Version: 0.9a6
Severity: normal Keywords:
Cc:

Description

If I embrace the whole file contents in two prints like

#!/usr/bin/env python
print "*** Begin ***"
import pkg_resources
pkg_resources.require("TurboGears")

...

cherrypy.root = Root()
cherrypy.server.start()
print "*** End ***"

it can be seen clearly from the output that the script runs twice recursively somehow:

*** Begin ***
*** Begin ***
2005/10/04 00:35:35 CONFIG INFO Server parameters:
2005/10/04 00:35:35 CONFIG INFO   server.environment: development
2005/10/04 00:35:35 CONFIG INFO   server.logToScreen: True
2005/10/04 00:35:35 CONFIG INFO   server.logFile:
2005/10/04 00:35:35 CONFIG INFO   server.protocolVersion: HTTP/1.0
2005/10/04 00:35:35 CONFIG INFO   server.socketHost:
2005/10/04 00:35:35 CONFIG INFO   server.socketPort: 8080
2005/10/04 00:35:35 CONFIG INFO   server.socketFile:
2005/10/04 00:35:35 CONFIG INFO   server.reverseDNS: False
2005/10/04 00:35:35 CONFIG INFO   server.socketQueueSize: 5
2005/10/04 00:35:35 CONFIG INFO   server.threadPool: 0
2005/10/04 00:35:35 HTTP INFO Serving HTTP on http://localhost:8080/
*** End ***
2005/10/04 00:35:38 HTTP INFO <Ctrl-C> hit: shutting down autoreloader
*** End ***

As far as I see, this is magic (something the developer does not expect to happen), and is not documented well.

I tried to edit the startup script in my project, and it caused real headache to find out, why it could not find a database connection using sqlite. Could be this more documented or could the execution flow be more regular (nearer to the expected one)?

Change History

comment:1 Changed 14 years ago by kevin

  • Status changed from new to assigned
  • Milestone set to 0.9

This looks like a function of how the autoreloader works. It actually forks the process and runs the everyhing in a subprocess. At least, that's my guess. It is worth looking into and documenting, though.

comment:2 Changed 14 years ago by Attila Vágvölgyi

  • Summary changed from Undocumented magic in project-start.py.source to cherrypy autoreload magic causes headaches

It seems, that turbogears.startup.reloader_thread() has nothing to do with the problem.

I found out, how cherrypy.lib.autoreload works, and cried. It is really effective and simple, but surprising for the uninitiated.

If autoreload.on is True or in development mode, it spawns a completely new interpreter with the same settings from cherrypy.server.start() with a new enviroment variable RUN_MAIN=true.

If a file called *.py is added/refreshed/deleted the spawned interpreter with the whole running server exits with error code 3 (breaking any running transactions?), and the whole server is respawned until it is killed with keyboard or runs on a sys.exit() with an error code different than 3.

Consequences:

  • Anything you do in *-start.py before calling cherrypy.server.start() will be done once for initializing the autoreloader, plus once for each reload.
  • I did not test, whether cherrypy.server.onStopServerList items are called on autoreload, but I bet they are not.

comment:3 Changed 14 years ago by ianb@…

Ideally when a reload is required you'd send a HUP signal or somesuch to the current process, which would trigger a reload/restart, all clean and pretty like. This doesn't necessarily "fix" the issues, but at least consolidates them into a single interface for development and production environments.

comment:4 Changed 14 years ago by kevin

This is how reloading works and, as pointed out, it's simple, effective and possibly confusing. So, I'm going to try to document this somewhere.

comment:5 Changed 14 years ago by kevin

  • Component changed from TurboGears to Docs

comment:6 Changed 14 years ago by kevin

  • Owner changed from anonymous to kevin
  • Status changed from assigned to new

comment:7 Changed 14 years ago by SuperJared

This can be overridden by using the CherryPy? method of doing this:

if os.environ.get("RUN_MAIN") == 'true':
    // This is where the cherrypy server actually starts
    cherrypy.server.start()
else:
    // And when you initialize here, it's really just going to trigger the autoreloader
    cherrypy.server.start()

See the applicant cherrypy code from lib/autoreload.py:

def restart_with_reloader():
    while True:
        args = [sys.executable] + sys.argv
        if sys.platform == "win32":
            args = ['"%s"' % arg for arg in args]
        new_environ = os.environ.copy()
        new_environ["RUN_MAIN"] = 'true'
        exit_code = os.spawnve(os.P_WAIT, sys.executable, args, new_environ)
        if exit_code != 3:
            return exit_code

def main(main_func, args=None, kwargs=None):
    if os.environ.get("RUN_MAIN") == "true":

        if args is None:
            args = ()
        if kwargs is None:
            kwargs = {}
        thread.start_new_thread(main_func, args, kwargs)

        # If KeyboardInterrupt is raised within reloader_thread,
        # let it propagate out to the caller.
        reloader_thread()
    else:
        # If KeyboardInterrupt is raised within restart_with_reloader,
        # let it propagate out to the caller.
        sys.exit(restart_with_reloader())

comment:8 Changed 13 years ago by michele

Is this still a problem?

comment:9 Changed 13 years ago by godoy

  • Status changed from new to closed
  • Resolution set to wontfix

I am closing this since there were lots of fixes in autoreload and a 2 months "silence" period since Michele asked for updates on this subject.

If you feel this is still a problem, please reopen this ticket and update the information so the we can work on it again.

Note: See TracTickets for help on using tickets.