Warning: Can't synchronize with repository "(default)" (Unsupported version control system "svn": No module named svn). Look in the Trac log for more information.

Ticket #29 (closed defect: fixed)

Opened 14 years ago

Last modified 12 years ago

HTML entities cause non-obvious errors in Kid.

Reported by: matt.bevan@… Owned by: anonymous
Priority: high Milestone: 0.9
Component: Kid Version: 0.5
Severity: normal Keywords:
Cc:

Description

Note: This ticket covers a problem well known and discussed on the IRC channel and mailing lists. It is being submitted here for tracking purposes.

Cause

Adding standard XHTML character entities to your templates, especially your master template.

Symptoms

Pages do not display and instead show the following backtrace (development mode only?):

Traceback (most recent call last):
  File "/usr/lib64/python2.4/site-packages/CherryPy-2.1.0_betadev-py2.4.egg/cherrypy/_cphttptools.py", line 276, in run
    main()
  File "/usr/lib64/python2.4/site-packages/CherryPy-2.1.0_betadev-py2.4.egg/cherrypy/_cphttptools.py", line 498, in main
    body = page_handler(*args, **cherrypy.request.paramMap)
  File "/usr/lib64/python2.4/site-packages/TurboGears-0.5.1-py2.4.egg/turbogears/controllers.py", line 94, in newfunc
    return controllers._process_output(turbogearsfmt, output, html)
  File "/usr/lib64/python2.4/site-packages/TurboGears-0.5.1-py2.4.egg/turbogears/controllers.py", line 31, in _process_output
    output = view.render(output, turbogearsfmt, html=html)
  File "/usr/lib64/python2.4/site-packages/TurboGears-0.5.1-py2.4.egg/turbogears/view.py", line 75, in render
    tclass = lookupTemplate(html)
  File "/usr/lib64/python2.4/site-packages/TurboGears-0.5.1-py2.4.egg/turbogears/view.py", line 53, in lookupTemplate
    mod = kid.load_template(tfile, name=classname)
  File "/usr/lib64/python2.4/site-packages/kid-0.7a-py2.4.egg/kid/__init__.py", line 118, in load_template
    mod = importer._create_module(code, name, filename, store=cache)
  File "/usr/lib64/python2.4/site-packages/kid-0.7a-py2.4.egg/kid/importer.py", line 87, in _create_module
    exec code in mod.__dict__
  File "/home/mbevan/Projects/infilm/infilm/templates/welcome.py", line 15, in ?
  File "/usr/lib64/python2.4/site-packages/kid-0.7a-py2.4.egg/kid/template_util.py", line 38, in get_base_class
    cls = kid.load_template(path).Template
  File "/usr/lib64/python2.4/site-packages/kid-0.7a-py2.4.egg/kid/__init__.py", line 116, in load_template
    code = template.compile(dump_code=cache)
  File "/usr/lib64/python2.4/site-packages/kid-0.7a-py2.4.egg/kid/compiler.py", line 87, in compile
    code = self.code
  File "/usr/lib64/python2.4/site-packages/kid-0.7a-py2.4.egg/kid/compiler.py", line 101, in code
    self._code = py_compile(self.python, pyfile, 'exec')
  File "/usr/lib64/python2.4/site-packages/kid-0.7a-py2.4.egg/kid/compiler.py", line 108, in python
    py = kid.parser.parse_file(self.kid_file, self.encoding)
  File "/usr/lib64/python2.4/site-packages/kid-0.7a-py2.4.egg/kid/parser.py", line 55, in parse_file
    return parse(source, encoding, filename=filename)
  File "/usr/lib64/python2.4/site-packages/kid-0.7a-py2.4.egg/kid/parser.py", line 44, in parse
    return parser.parse()
  File "/usr/lib64/python2.4/site-packages/kid-0.7a-py2.4.egg/kid/parser.py", line 71, in parse
    self.proc_stream(self.module_code)
  File "/usr/lib64/python2.4/site-packages/kid-0.7a-py2.4.egg/kid/parser.py", line 126, in proc_stream
    for (ev, item) in self.stream:
  File "/usr/lib64/python2.4/site-packages/kid-0.7a-py2.4.egg/kid/pull.py", line 152, in _track
    for p in stream:
  File "/usr/lib64/python2.4/site-packages/kid-0.7a-py2.4.egg/kid/pull.py", line 179, in _coalesce
    for ev, item in stream:
  File "/usr/lib64/python2.4/site-packages/kid-0.7a-py2.4.egg/kid/pull.py", line 321, in __iter__
    for (ev, stuff) in self._expat_stream():
  File "/usr/lib64/python2.4/site-packages/kid-0.7a-py2.4.egg/kid/pull.py", line 293, in _expat_stream
    feed(data)
  File "/usr/lib64/python2.4/site-packages/kid-0.7a-py2.4.egg/kid/pull.py", line 361, in feed
    raise e
ExpatError: undefined entity —: line 35, column 216

After the initial error is shown, subsequent refreshes will only display the following, possibly leading to confusion:

Traceback (most recent call last):
  File "/usr/lib64/python2.4/site-packages/CherryPy-2.1.0_betadev-py2.4.egg/cherrypy/_cphttptools.py", line 276, in run
    main()
  File "/usr/lib64/python2.4/site-packages/CherryPy-2.1.0_betadev-py2.4.egg/cherrypy/_cphttptools.py", line 498, in main
    body = page_handler(*args, **cherrypy.request.paramMap)
  File "/usr/lib64/python2.4/site-packages/TurboGears-0.5.1-py2.4.egg/turbogears/controllers.py", line 94, in newfunc
    return controllers._process_output(turbogearsfmt, output, html)
  File "/usr/lib64/python2.4/site-packages/TurboGears-0.5.1-py2.4.egg/turbogears/controllers.py", line 31, in _process_output
    output = view.render(output, turbogearsfmt, html=html)
  File "/usr/lib64/python2.4/site-packages/TurboGears-0.5.1-py2.4.egg/turbogears/view.py", line 75, in render
    tclass = lookupTemplate(html)
  File "/usr/lib64/python2.4/site-packages/TurboGears-0.5.1-py2.4.egg/turbogears/view.py", line 57, in lookupTemplate
    tempclass = mod.Template
AttributeError: 'module' object has no attribute 'Template'

Restarting the server process will re-generate the first error, once.

Effected Packages

This is both a Kid issue and a more general one of TurboGears being unable to handle broken templating.

Change History

comment:1 Changed 14 years ago by mrbmahoney@…

Here is code for Kid pull.py which will recognize HTML entities. This would go in class ExpatParser? at the end of init


      
        self.entity = {}  ## new code follows this existing line of __init__
        
        self.setForeignDTD()

    # Two new methods, setForeignDTD and _buildForeign.
    # Now methods _start and _start_list will receive an attrib_in
    # string with the html entities replaced by the  ASCII/unicode character
    # for the entity value. " " will be replaced by '\x0a' (hex(160))
    # "€" with value of "€" will be replaced by '\u20ac' (hex(8364))
    #
    # Method _default will receive the original entity when element pcdata is
    # parsed.  _default has code to replace that entity with the value in
    # self.entity
    #
    #
    # Anything set here can be changed before parsing data ( before feed() )
    def setForeignDTD(self):
        
        from xml.parsers import expat
        from htmlentitydefs import entitydefs
        
        self.entity = entitydefs

        # If you want entity values other than "&#nnn;" and "&#nnnn;"
        # then change self.entity dict before parsing data
        for k,v in self.entity.items():
            if 1 == len(v):
                self.entity[k] = "&#%d;" % ord(v) 
            # else, entity value already in form "&#nnnn;"


        # This 'foreign' DTD will be only entity declarations
        # Again, if you want something else, change before parsing data
        self.foreignDTD = ""
        for k,v in self.entity.items():
            self.foreignDTD += '\n<!ENTITY %s "%s">' % (k, v) 

        # This code will set things so that when it is time
        # to parse a DTD when there is no DTD in the input, expat 
        # will call the ExternalEntityRefHandler ( aka _buildForeign  )
        # with all args of None

        self._parser.SetParamEntityParsing(
            expat.XML_PARAM_ENTITY_PARSING_ALWAYS)
            # No really a param entity, but necessary

        self._parser.ExternalEntityRefHandler = self._buildForeign
        self._parser.UseForeignDTD()
 
        
    def _buildForeign(self, context, base, systemId, publicId):
        assert ( None == (systemId and publicId) ), \
            ( ("Did not expect any external entity. "\
               +"System id '%s', Public Id '%s'") % ( systemId, publicId ) )
                        
        import StringIO
        parseableFile = StringIO.StringIO(self.foreignDTD)

        # Standard code for creating new parser for an external file. 
        original_parser = self._parser
        self._parser = self._parser.ExternalEntityParserCreate(context)

        self._parser.ParseFile( parseableFile )
        
        self._parser = original_parser

        return 1   # Important !

-- end of modification -- Brian Mahoney mrbmahoney@…

comment:2 Changed 14 years ago by kevin

  • Milestone set to 0.9

comment:3 Changed 14 years ago by SuperJared

comment:4 Changed 14 years ago by kevin

  • Priority changed from normal to high
  • Component changed from TurboGears to Kid

comment:5 Changed 14 years ago by anonymous

The first error also occurs if an entity appears in the argument passed to the XML function. I discovered this when an RSS feed included an &nbsp;.

comment:6 Changed 14 years ago by rtomayko@…

  • Status changed from new to closed
  • Resolution set to fixed

I've committed a fix for this to Kid trunk. See  http://lesscode.org/projects/kid/ticket/86 for more information.

Note: See TracTickets for help on using tickets.