wiki:TurboGearsOnDreamHost
Warning: Can't synchronize with repository "(default)" (Unsupported version control system "svn": No module named svn). Look in the Trac log for more information.

Version 19 (modified by anonymous, 9 years ago) (diff)

--

Introduction

Users of the DreamHost?.com (DH) hosting service are now able to run TurboGears (TG) applications behind Apache using FastCGI. This page describes the steps required for enabling TG as a FastCGI application on the DH servers.

Note: If you do happen to sign up for DreamHost?, please use this link to support TurboGears development:  http://www.dreamhost.com/r.cgi?turbogears

Prerequisites

FastCGI must be enabled on your DH account. It is recommended that you enable FastCGI before you begin building your TG/DH environment. To enable FastCGI, do the following:

  1. Login to your account using the DH panel ( https://panel.dreamhost.com/)
  2. Click on the “Domains” section (on the left) and then click on “Manage Domains”.
  3. Click on the “Edit” link under the “Web Hosting” column next to the domain that you would like to enable.
  4. Click on the “FastCGI Support?” option, and then save your settings.

Installation

Download Installation Files

A few files will need to be downloaded before starting the installation. To make it easier to clean up later, you should create a new directory to store these files (e.g. /$HOME/download/turbogears). Switch to the download directory and issue the following commands:

# Used for TurboGears Download/Installation
wget http://www.turbogears.org/download/ez_setup.py

# Current Python Download
wget http://www.python.org/ftp/python/2.4.2/Python-2.4.2.tgz

Create a Custom Python Installation

Because of DH’s current Python setup (the python-dev package is not installed), you will need to create a custom Python installation for your account. Issue the following commands from the directory where the Python-X.X.X.tgz file is located:

#Unpack Python
tar -zxvf Python-2.4.2.tgz

#Change to the Python directory
cd Python-2.4.2

#Run configure and enable Unicode
./configure --prefix=$HOME/lib --enable-unicode=ucs4
make
make install

Python is now installed, however, you will need to make your installation the default version:

# Create a home directory for Python
mkdir $HOME/bin
mkdir $HOME/bin/python
mkdir $HOME/bin/python/bin

# Create a link to the python executable.
ln -s $HOME/lib/bin/python $HOME/bin/python/bin/python

# Edit your bash_profile
cd ~
vim .bash_profile

# Add the following line then save the file: 
export PATH=$HOME/bin/python/bin:$HOME/lib/bin:$PATH

# Reload the file
source ~/.bash_profile

Type 'python'. You should see output similar to the following:

Python 2.4.2 (#3, Dec 10 2005, 10:22:59)
[GCC 3.3.5 (Debian 1:3.3.5-13)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

TurboGears Installation

Navigate to the directory where the ez_setup.py (downloaded earlier) file is located. Issue the following command to install TurboGears:

python ez_setup.py -f http://www.turbogears.org/download/index.html TurboGears

TurboGears should now be installed. Typing ‘tg-admin’ should display output similar to the following:

$ tg-admin

TurboGears 0.8a5 command line interface

Usage: /home/cubano/lib/bin/tg-admin [command] [options]

Available commands:
quickstart  Create a new TurboGears project
     shell  Start a Python prompt with your database available
       sql  Run the SQLObject manager

If you are testing your installation with the wiki20 example, you will also need to install the docutils package:

easy_install docutils

If you are using sqlite, you will need to install the pysqlite package:

easy_install pysqlite

For MySQL users, do the following to install the Python/MySQL library package:

# Download the file - Check for newer versions at http://sourceforge.net/project/showfiles.php?group_id=22307  
# This is the current version as of Jan 03, 2006
wget http://easynews.dl.sourceforge.net/sourceforge/mysql-python/MySQL-python-1.2.0.tar.gz

# Unpack the archive
tar -xvvzf MySQL-python-1.2.0.tar.gz

# Install the archive
cd MySQL-python-1.2.0
python setup.py install

FastCGI Installation

Download the tg_fastcgi.fcgi file. This file should be placed in the root folder of the TurboGears Application site. For example, if your application will be located at www.mysite.com, then this file should be placed in the corresponding /home/<user>/mysite.com/ directory.

TurboGears v.9:

wget http://trac.turbogears.org/turbogears/attachment/wiki/TurboGearsOnDreamHost/tg_fastcgi.fcgi.v9?format=raw
mv tg_fastcgi.fcgi.v9\?format\=raw tg_fastcgi.fcgi
chmod 755 tg_fastcgi.fcgi

TurboGears v.8:

wget http://trac.turbogears.org/turbogears/attachment/wiki/TurboGearsOnDreamHost/tg_fastcgi.fcgi.v8?format=raw
mv tg_fastcgi.fcgi.v8\?format\=raw tg_fastcgi.fcgi
chmod 755 tg_fastcgi.fcgi

Download the Python Fast-CGI code:

wget http://www.saddi.com/software/py-lib/py-lib/fcgi.py

This file will need to be placed in the Python path. An easy way to do this is to place it in the same directory as tg_fastcgi.fcgi.

Application Configuration

You now have everything that you need to run TG behind Apache using FastCGI. The final steps are to configure tg_fastcgi.fcgi and Apache.

tg_fastcgi.fcgi Configuration

Edit the top of tg_fastcgi.fcgi with your account-specific Python installation information:

#!/home/<user>/bin/python/bin/python

Edit the 'USER EDIT SECTION' with the correct configuration information:

# -- START USER EDIT SECTION
# -- Users must edit this section -- 
code_dir = ''           # (Required) The base directory of the TG app code where 
                           the *.cfg file is located.
root_class_name = ''    # (Required) The fully qualified Root class name 
                          (e.g. wiki20.controllers.Root)
log_dir = ''            # (Optional) The log directory. Default = code_dir.
# -- END USER EDIT SECTION

If you are not running your TurboGears application at the root of your server, you'll also have to change the virtualPathFilter.prefix to the path to tg_fastcgi.fcgi. For instance, if you are running your site from http://<yoursite>.com/wiki20/ then you need to change the prefix to "/wiki20/tg_fastcgi.fcgi"

To test the file, issue the following command from the directory where it is located:

./tg_fastcgi.fcgi

It may take a minute for the application to respond. After the application returns, navigate to the log directory that was specified in the 'USER EDIT SECTION' (if no directory was specified, then navigate to the code directory). If you are using dev.cfg, then the stdout.log file should contain the application's output; otherwise, the log file specified in prod.cfg should contain the relevant output. If there is no output in either log file, be sure to check stderr.log for more information.

Note: Your TG application code should not be placed in your web server's root directory. The TG application code should be placed in a directory that is not directly accessible from the Internet (e.g. /$HOME/apps/code/). The only files that need to exist in the web server's root directory are 'tg_fastcgi.fcgi' and 'fcgi.py'.

Apache Configuration

Create a .htaccess file in the same directory where the tg_fastcgi.fcgi file is located (web site root folder). Add the following lines to the file:

RewriteEngine On
RewriteBase /
RewriteRule ^(tg_fastcgi\.fcgi/.*)$ - [L]
RewriteRule !^stat - [C]
RewriteRule ^(.*)$ tg_fastcgi.fcgi/$1 [L]

These directives rewrite all of the incoming request URLs so that they are directed to the tg_fastcgi.fcgi code.

For additional non-TG sub-directories of your website, you can add lines like this after the stat line, but before the final line:

RewriteRule ^!(blog.*)$ - [C]

This is useful for referring to dreamhost apps such as Wordpress or Gallery that you may have as sub-dirs of the main site.

Make sure that the .htaccess file has the proper permissions set:

chmod 755 .htaccess

Conclusion

That's it! Your code should now be up and running.

Note: It may take some time (between 30 and 60 seconds) to initialize the application the first time it is accessed. After that, the application should respond normally.

Troubleshooting

If your application is not running correctly, and you can't figure out why, try running the fcgi code directly:

$ python tg_fastcgi.fcgi

If the code is running correctly, then you should receive output from the root of your application. Otherwise, an error should appear in the output.

If you make an error in the tg_fastcgi.fcgi module, you may have to kill all of the FastCGI processes before your changes will appear. To find all of the FastCGI processes, use the 'ps -A' command:

$ ps -A
  PID TTY          TIME CMD
 4903 ?        00:00:00 sshd
13966 pts/0    00:00:00 bash
29664 ?        00:00:01 tg_fastcgi.fcgi
 1409 ?        00:00:01 tg_fastcgi.fcgi
26362 ?        00:00:01 tg_fastcgi.fcgi
 8369 ?        00:00:01 tg_fastcgi.fcgi
30766 ?        00:00:00 tg_fastcgi.fcgi
27674 ?        00:00:00 tg_fastcgi.fcgi
  238 ?        00:00:00 tg_fastcgi.fcgi
17218 ?        00:00:00 tg_fastcgi.fcgi
  429 pts/0    00:00:00 ps

To kill all of the FastCGI processes, use the following command:

pkill tg_fastcgi.fcgi

These respawn quickly. Most of the time, your changes will take effect after one "kill" - no need to repeat the command, and sometimes repeating the command will lock up fcgi support for your site for a good while. This can be a source of great consternation as to why your site isn't working, even though running "python tg_fastcgi.fcgi" will dump proper output to stdout.log

References

  1. Django on Dreamhost -  http://wiki.dreamhost.com/index.php/Django
  2. Installing Python on a Shared Server -  http://www.wombatnation.com/2004/12/installing-python-on-shared-server
  3. Cherrypy FastCGI WSGI -  http://www.cherrypy.org/wiki/FastCGIWSGI
  4. Official TurboGears Linux Installation Instructions -  http://www.turbogears.org/download/nix.html

tg_fastcgi.fcgi Code for TG 0.8

#!/home/<user>/bin/python/bin/python
#
# File name: tg_fastcgi.fcgi
#
# This module provides the glue for running TurboGears applications behind 
# FastCGI-enabled web servers. The code in this module depends on the fastcgi
# module downloadable from here:
#
# http://www.saddi.com/software/py-lib/py-lib/fcgi.py
#
# NOTE: The fcgi.py file needs to be placed in a location that is on the
# system path, such as the same the directory as the tg_fastcgi.py file
# or in the base directory of the TG app code.
#
# To configure this module, please edit the three variables in the "USER EDIT
# SECTION" before starting the TG application.  Also remember to edit the
# top of this file with the correct Python installation information.

import cherrypy
import sys
import os
from os.path import *
import pkg_resources

pkg_resources.require("TurboGears")

# -- START USER EDIT SECTION
# -- Users must edit this section -- 
code_dir = ''           # (Required) The base directory of the TG app code.
root_class_name = ''    # (Required) The fully qualified Root class name.
log_dir = ''            # (Optional) The log directory. Default = code_dir.
# -- END USER EDIT SECTION

class VirtualPathFilter(object):
    """Custom class used to remove the fastcgi prefix before
       the main TG app code is called."""
    
    def beforeRequestBody(self):
        """Strips out the virual path prefix before the data is sent to
           Cherrypy."""
        
        if cherrypy.config.get('virtualPathFilter.on', False):
            prefix = cherrypy.config.get('virtualPathFilter.prefix', '')
            if prefix:
                path = cherrypy.request.path
            if path == prefix:
                path = '/'
            elif path.startswith(prefix):
                path = path[len(prefix):]
            cherrypy.request.path = path

def tg_init():
    """ Checks for the required data and initializes the application. """

    global code_dir
    global root_class_name
    global log_dir
    last_mark = 0

    # Input checks
    if not code_dir or not isdir(code_dir):
        raise ValueError("""The code directory setting is missing.
                            The fastcgi code will be unable to find
                            the TG code without this setting.""")

    if not root_class_name:
        raise ValueError("""The fully qualified root class name must
                            be provided.""")

    last_mark = root_class_name.rfind('.')
    
    if (last_mark < 1) or (last_mark + 1) == len(root_class_name):
        raise ValueError("""The user-defined class name is invalid.
                            Please make sure to include a fully
                            qualified class name for the root_class
                            value (e.g. wiki20.controllers.Root).""")    

    sys.path.append(code_dir)

    # Change the directory so the TG log file will not be written to the
    # web app root.
    if log_dir and isdir(log_dir):
        os.chdir(log_dir)
    else:
        os.chdir(code_dir)
        log_dir = code_dir

    sys.stdout = open(join(log_dir, 'stdout.log'),'a')
    sys.stderr = open(join(log_dir, 'stderr.log'),'a')        

    if exists(join(code_dir, "setup.py")):
        cherrypy.config.update(file=join(code_dir, "dev.cfg"))
    else:
        cherrypy.config.update(file=join(code_dir, "prod.cfg"))

    # Set environment to production to disable auto-reload and
    # add virutal path information.
    cherrypy.config.update({
        'global': {'server.environment': 'production'},
        '/': {'virtualPathFilter.on': True,
              'virtualPathFilter.prefix': "/tg_fastcgi.fcgi"
          }})

    # Parse out the root class information for Cherrypy Root class.
    package_name = root_class_name[:last_mark]
    class_name = root_class_name[last_mark+1:]
    exec('from %s import %s as Root' % (package_name, class_name))
    Root._cpFilterList = [VirtualPathFilter()]
    cherrypy.root = Root()

# Main section -
# Initialize the application, then start the server.
tg_init()

from fcgi import WSGIServer
cherrypy.server.start(initOnly=True, serverClass=None)

from cherrypy._cpwsgi import wsgiApp
WSGIServer(application=wsgiApp).run()

tg_fastcgi.fcgi Code for TG 0.9/SVN

#!/home/<user>/bin/python/bin/python
#
# File name: tg_fastcgi.fcgi
#
# This module provides the glue for running TurboGears applications behind 
# FastCGI-enabled web servers. The code in this module depends on the fastcgi
# module downloadable from here:
#
# http://www.saddi.com/software/py-lib/py-lib/fcgi.py
#
# NOTE: The fcgi.py file needs to be placed in a location that is on the
# system path, such as the same the directory as the tg_fastcgi.py file
# or in the base directory of the TG app code.
#
# To configure this module, please edit the three variables in the "USER EDIT
# SECTION" before starting the TG application.  Also remember to edit the
# top of this file with the correct Python installation information.

import cherrypy
import sys
import os
from os.path import *
import pkg_resources
import turbogears

pkg_resources.require("TurboGears")

# -- START USER EDIT SECTION
# -- Users must edit this section -- 
code_dir = ''                               # (Required) The base directory of the TG app code.
root_class_name = ''                        # (Required) The fully qualified Root class name.
project_module_name = 'PROJECTNAME.config'  # (Required) The config module name. Replace
                                            # PROJECTNAME with your project's name (e.g. wiki20).
log_dir = ''                                # (Optional) The log directory. Default = code_dir.
# -- END USER EDIT SECTION


def tg_init():
    """ Checks for the required data and initializes the application. """

    global code_dir
    global root_class_name
    global log_dir
    global project_module_name
    last_mark = 0

    # Input checks
    if not code_dir or not isdir(code_dir):
        raise ValueError("""The code directory setting is missing.
                            The fastcgi code will be unable to find
                            the TG code without this setting.""")

    if not root_class_name:
        raise ValueError("""The fully qualified root class name must
                            be provided.""")

    last_mark = root_class_name.rfind('.')
    
    if (last_mark < 1) or (last_mark + 1) == len(root_class_name):
        raise ValueError("""The user-defined class name is invalid.
                            Please make sure to include a fully
                            qualified class name for the root_class
                            value (e.g. wiki20.controllers.Root).""")    

    sys.path.append(code_dir)

    # Change the directory so the TG log file will not be written to the
    # web app root.
    if log_dir and isdir(log_dir):
        os.chdir(log_dir)
    else:
        os.chdir(code_dir)
        log_dir = code_dir

    sys.stdout = open(join(log_dir, 'stdout.log'),'a')
    sys.stderr = open(join(log_dir, 'stderr.log'),'a')        

    if exists(join(code_dir, "setup.py")):
        turbogears.update_config(configfile=join(code_dir, "devcfg.py"),modulename=project_module_name)
    else:
        turbogears.update_config(configfile=join(code_dir, "prodcfg.py"),modulename=project_module_name)

    # Set environment to production to disable auto-reload and
    # add virutal path information.
    cherrypy.config.update({
        'global': {'server.environment': 'production'}})

    # Parse out the root class information for Cherrypy Root class.
    package_name = root_class_name[:last_mark]
    class_name = root_class_name[last_mark+1:]
    exec('from %s import %s as Root' % (package_name, class_name))
    cherrypy.root = Root()

# Main section -
# Initialize the application, then start the server.
tg_init()

from fcgi import WSGIServer
cherrypy.server.start(initOnly=True, serverClass=None)

from cherrypy._cpwsgi import wsgiApp
WSGIServer(application=wsgiApp).run()

Attachments