Warning: Can't synchronize with repository "(default)" (Unsupported version control system "svn": No module named svn). Look in the Trac log for more information.

Ticket #178 (closed defect: fixed)

Opened 14 years ago

Last modified 12 years ago

[PATCH] unicode charachters are not properly handled by CatWalk

Reported by: ronald Jaramillo Owned by: anonymous
Priority: high Milestone: 0.9
Component: Toolbox Version:
Severity: blocker Keywords: CatWalk JavaScript develix
Cc:

Description

CatWalk doesn't handle unicode characters properly. The root of the problem is the javascript escape function. CW should use the encodeURIComponet function instead, because escape does some weird things with the encoding.

Attachments

catwalk.patch Download (1.3 KB) - added by joeysmith@… 13 years ago.
should close both outstanding issues with this bug

Change History

comment:1 Changed 14 years ago by anonymous

  • Severity changed from major to blocker

comment:2 Changed 14 years ago by anonymous

  • Priority changed from normal to high

It also doesn't let you edit a UnicodeCol?, the fields just come up blank even if there is data there, and if you enter something and hit 'update', it's lost.

comment:3 Changed 14 years ago by jpaulofarias at gmail dot com

Try creating a sitecustomize.py under lib/site-packages with the following content:

import sys
sys.setdefaultencoding('utf8') # put your encoding here

This works for me.

comment:4 Changed 13 years ago by anonymous

jpaulofarias:

I read somewhere that it's bad to do that.

Also, it may not be possible to access that file in a shared hosting environment. Better to fix the problem than rely on a solution that only works if you can have root.

comment:5 Changed 13 years ago by nerkles@…

jpaulofarias:

that didn't solve the issue here.

I messed around with the catwalk code a little and got halfway there by adding code for UnicodeCol? wherever I found StringCol?. The values were then visible when browsing or clicking 'edit' with catwalk, but it still wouldn't save any changes.

comment:6 Changed 13 years ago by florian

here on r438 and the issue's still there.

My problem I have with it runs like this.

  • I got a product table with a UnicodeCol? title, containing some non ascii data (as in automatically encoded as utf-8 upon insert by sqlobject)
  • I got another table participates that references by ForeignKey?('product')
  • viewing either product and participates is fine
  • Then I go to the participates structure view in catwalk and select title of product to be the display of the ForeignKey? relation instead of id I get his:
2006/01/01 12:29:54 HTTP INFO 192.168.1.16 - GET /catwalk/columns?objectName=participates&foreignObjectName=product&foreignKeyName=product HTTP/1.1
2006-01-01 12:29:54,848 turbogears.controllers DEBUG Returning JSON
192.168.1.16 - - [2006/01/01 12:29:54] "GET /catwalk/columns?objectName=participates&foreignObjectName=product&foreignKeyName=product HTTP/1.1" 200 3500
2006/01/01 12:29:55 HTTP INFO 192.168.1.16 - GET /catwalk/columnsForLabel?objectName=participates&foreignObjectName=product&foreignKeyName=product HTTP/1.1
2006-01-01 12:29:55,048 turbogears.controllers DEBUG Returning JSON
192.168.1.16 - - [2006/01/01 12:29:55] "GET /catwalk/columnsForLabel?objectName=participates&foreignObjectName=product&foreignKeyName=product HTTP/1.1" 200 415
2006/01/01 12:29:57 HTTP INFO 192.168.1.16 - GET /catwalk/setColumnForLabel?objectName=participates&foreignObjectName=product&foreignKeyName=product&columnName=title HTTP/1.1
2006/01/01 12:29:58  INFO Traceback (most recent call last):
  File "/usr/local/lib/python2.4/site-packages/CherryPy-2.1.0-py2.4.egg/cherrypy/_cphttptools.py", line 271, in run
    main()
  File "/usr/local/lib/python2.4/site-packages/CherryPy-2.1.0-py2.4.egg/cherrypy/_cphttptools.py", line 502, in main
    body = page_handler(*args, **cherrypy.request.paramMap)
  File "turbogears/controllers.py", line 242, in newfunc
  File "turbogears/database.py", line 189, in run_with_transaction
  File "turbogears/controllers.py", line 259, in _execute_func
  File "/projects/turbogears/turbogears/toolbox/catwalk/__init__.py", line 995, in setColumnForLabel
    return self.columns(objectName)
  File "turbogears/controllers.py", line 245, in newfunc
  File "turbogears/controllers.py", line 259, in _execute_func
  File "/projects/turbogears/turbogears/toolbox/catwalk/__init__.py", line 1006, in columns
    columns=self.object_columns(objectName),
  File "/projects/turbogears/turbogears/toolbox/catwalk/__init__.py", line 515, in object_columns
    cols = self.get_columns_for_object(object)
  File "/projects/turbogears/turbogears/toolbox/catwalk/__init__.py", line 527, in get_columns_for_object
    cols.append(self.get_column_properties(column_name,column))
  File "/projects/turbogears/turbogears/toolbox/catwalk/__init__.py", line 489, in get_column_properties
    props = self.get_foreign_key_properties(column,props)
  File "/projects/turbogears/turbogears/toolbox/catwalk/__init__.py", line 504, in get_foreign_key_properties
    properties['labelColumn'])
  File "/projects/turbogears/turbogears/toolbox/catwalk/__init__.py", line 364, in foreign_key_alternatives
    return [{'id':type(x.id)(x.id),
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf1' in position 5: ordinal not in range(128)

192.168.1.16 - - [2006/01/01 12:29:58] "GET /catwalk/setColumnForLabel?objectName=participates&foreignObjectName=product&foreignKeyName=product&columnName=title HTTP/1.1" 500 2496

From this point forth I can't structure view or browse participates anymore with the MsgBox?

Fail to load object instance: 'ascii' codec can't encode character u'\xf1' in position5: ordinal not in range(128)

popping up.

comment:7 Changed 13 years ago by kevin

  • Keywords develix added

comment:8 Changed 13 years ago by Florian

I tried this and it is still broken on r532

Changed 13 years ago by joeysmith@…

should close both outstanding issues with this bug

comment:9 Changed 13 years ago by joeysmith@…

  • Status changed from new to assigned

Patch attached that should fix both outstanding issues.

comment:10 Changed 13 years ago by joeysmith@…

  • Summary changed from unicode charachters are not properly handled by CatWalk to [PATCH] unicode charachters are not properly handled by CatWalk

Forgot to change summary

comment:11 Changed 13 years ago by ronald@…

  • Status changed from assigned to closed
  • Resolution set to fixed

Changeset r546 applied the attached patch, thanks to joeysmith@… for it. Changeset r550 fixed handling of unicode characters for StringCol?

Note: See TracTickets for help on using tickets.