Warning: Can't synchronize with repository "(default)" (Unsupported version control system "svn": No module named svn). Look in the Trac log for more information.

Ticket #297 (closed defect: fixed)

Opened 13 years ago

Last modified 12 years ago

[PATCH] i18n and turbogears.flash

Reported by: mte Owned by: kevin
Priority: normal Milestone: 0.9
Component: TurboGears Version: 0.9a6
Severity: normal Keywords: develix
Cc:

Description

Here I have two statements (i use the svn version of tg):

print _("MyString?", "sl") turbogears.flash(_("MyString?"), "sl)

When only ascii characters are present in the translated string, both statements work. However, if I try to use a special character, the first statement executes with no problems, but the second one raises this error: Traceback (most recent call last):

File "c:\python24\lib\site-packages\CherryPy?-2.1.0-py2.4.egg\cherrypy\_cphttptools.py", line 271, in run

main()

File "c:\python24\lib\site-packages\CherryPy?-2.1.0-py2.4.egg\cherrypy\_cphttptools.py", line 502, in main

body = page_handler(*args, cherrypy.request.paramMap)

File "c:\python24\lib\site-packages\TurboGears-0.9a0dev_r410-py2.4.egg\turbogears\controllers.py", line 242, in newfunc

html, *args, kw)

File "c:\python24\lib\site-packages\TurboGears-0.9a0dev_r410-py2.4.egg\turbogears\database.py", line 175, in run_with_transaction

retval = func(*args, kw)

File "c:\python24\lib\site-packages\TurboGears-0.9a0dev_r410-py2.4.egg\turbogears\controllers.py", line 259, in _execute_func

output = func(self, *args, kw)

File "c:\python24\lib\site-packages\TurboGears-0.9a0dev_r410-py2.4.egg\turbogears\controllers.py", line 147, in newfunc

output = func(self, *args, kw)

File "c:\python24\Scripts\MyProject?\myproject\controllers.py", line 29, in save

turbogears.flash(_("MyString?", "sl"))

File "c:\python24\lib\site-packages\TurboGears-0.9a0dev_r410-py2.4.egg\turbogears\controllers.py", line 266, in flash

cherrypy.response.simpleCookietg_flash? = message

File "c:\python24\lib\Cookie.py", line 580, in setitem

rval, cval = self.value_encode(value)

File "c:\python24\lib\Cookie.py", line 667, in value_encode

strval = str(val)

UnicodeEncodeError?: 'ascii' codec can't encode character u'\u0161' in position 13: ordinal not in range(128)

The .po file has correctly set the encoding to utf-8, so I don't understand why it wants to take ascii here...

Attachments

flash.patch Download (923 bytes) - added by ischenko@… 13 years ago.
the same patch I had posted inline
#297tests.diff Download (2.9 KB) - added by oefe <turbogears at oefelein.de> 13 years ago.
unit test for flash
#297patch.diff Download (1.5 KB) - added by oefe <turbogears at oefelein.de> 13 years ago.
a better patch that avoids cookies
flash-tests.patch Download (3.8 KB) - added by oefe 13 years ago.
improved tests which also check the redirect

Change History

comment:1 Changed 13 years ago by mte

  • Component changed from Kid to TurboGears

for my needs I just added this line to turbogearsdir/kidsupport.py:

from i18n.kidutils import *

and now it works. So if someone could fix the svn version it would be nice...

comment:2 Changed 13 years ago by mte

oops..the above solution does not apply here... it was meant for ticket #298

comment:3 Changed 13 years ago by kevin

  • Keywords develix added

comment:4 Changed 13 years ago by Dan Jacob

I fixed the problem by having a sitecustomize.py file in the project root directory (i.e. where your myproject-start.py script is). There may be issues with using sitecustomize.py and other packages, so I don't know if it is an optimal solution.

sitecustomize.py:

import sys
sys.setdefaultencoding("UTF-8")

Your TG app should now use UTF-8 instead of ascii as the default encoding.

comment:5 Changed 13 years ago by anonymous

  • Status changed from new to assigned

comment:6 Changed 13 years ago by ischenko@…

Here is my take on this: gettext() returns a Unicode string (or at least it should, ticket #387 fixes that for lazy_gettext()). Kid template accepts Unicode string and would render it fine, encoding to UTF-8. BUT the cookie library uses str() so it should take already encoded string and not Unicoded one.

And here lies the problem. How would you know _which_ charset to use to encode cookie string? For kid that's easy, but for cookie?

The following patch just uses utf-8 to encode cookie and then decode it back to unicode. It may or may not be a good idea to use some configuration option for that instead:

Index: controllers.py
===================================================================
--- controllers.py	(revision 519)
+++ controllers.py	(working copy)
@@ -263,6 +261,12 @@
 
 def flash(message):
     """Set a message to be displayed in the browser on next page display"""
+    if hasattr(message, '__unicode__'):
+        message = unicode(message)
+    if isinstance(message, unicode):
+        message = message.encode('utf-8')
     cherrypy.response.simpleCookie['tg_flash'] = message
     cherrypy.response.simpleCookie['tg_flash']['path'] = '/'
     
@@ -283,6 +287,8 @@
         response_cookie.pop("tg_flash")
     else:
         message = None
+    if message:
+        message = unicode(message, 'utf-8')
     return message
 class Controller(object):
     """Base class for a web application's controller.

Note that it probably will not work unless you apply my other patch (#387). I.e. if you are using lazy_gettext() vs. simply gettext().

HTH, Max.  maxischenko.in.ua

Changed 13 years ago by ischenko@…

the same patch I had posted inline

comment:7 Changed 13 years ago by kevin

  • Summary changed from i18n and turbogears.flash to [PATCH] i18n and turbogears.flash

Changed 13 years ago by oefe <turbogears at oefelein.de>

unit test for flash

Changed 13 years ago by oefe <turbogears at oefelein.de>

a better patch that avoids cookies

comment:8 Changed 13 years ago by oefe <turbogears at oefelein.de>

  • Status changed from assigned to new
  • Owner changed from anonymous to oefe <turbogears at oefelein.de>

Why is turbogears.flash using a cookie at all? The cookie is normally cleared before the call returns, so the client won't see it. However, using a cookie gives clients a possibility to manipulate tg_flash, which has probably unwanted results, and is potentially dangerous (if someone tries to execute it). test_bogus_flash_cookie tests for this scenario.

I attached some unit tests, and a patch that avoids cookies. Instead, it sets (and clears) an attribute in the response object. (Clearing might actually not be necessary.) This means, that flash can now be an arbitrary Python object, including unicode or complex nested data structures.

comment:9 Changed 13 years ago by kevin

note for posterity: cookies are required in the flash function because the message will often be displayed after a redirect.

Changed 13 years ago by oefe

improved tests which also check the redirect

comment:10 Changed 13 years ago by oefe

  • Owner changed from oefe <turbogears at oefelein.de> to kevin

OK, so forget my previous patches. I attached updated tests (this time with a proper .patch suffix, so that trac recognizes it). In particular, I added a test that flash works with redirects. With Max Ischenko's patch, all tests pass; without, the patches which use unicode fail.

comment:11 Changed 13 years ago by kevin

  • Status changed from new to closed
  • Resolution set to fixed

thanks for the patch and the tests! committed in [560].

Note: See TracTickets for help on using tickets.