Warning: Can't synchronize with repository "(default)" (Unsupported version control system "svn": No module named svn). Look in the Trac log for more information.

Ticket #744 (closed defect: fixed)

Opened 13 years ago

Last modified 12 years ago

admi18n wrong dealing with non-english utf-8 characters

Reported by: matej Owned by: anonymous
Priority: normal Milestone: 1.0b1
Component: Toolbox Version:
Severity: normal Keywords:
Cc: ronald@…, ischenko@…

Description

I translated strings with utf-8 characters č, š, ž, Č, Š, Ž from browser but tg obviously doesn't know what to do with them so after compilation those strings became blank. My workaround:

  • edit the messages.po manually in a text editor in utf-8 mode
  • convert the file to ANSI and save that
  • compile again
  • Success!

A strange thing to me is that the file needs to be in ANSI, if I save it as utf-8 I get some strange errors.

Change History

comment:1 Changed 13 years ago by anonymous

  • Version 0.9a4 deleted
  • Milestone set to 0.9a5

comment:2 Changed 13 years ago by godoy

What errors? po files can be in UTF-8. Are you on Windows? I don't see that here with Firefox on Linux...

comment:3 Changed 13 years ago by matej

Yes, I am on Windows. I get the following as soon as I encode the file to utf-8. After reencoding to ansi, I don't get the error anymore:

Page handler: <bound method Internationalization.po_view of <turbogears.toolbox.admi18n.Internationalization object at 0x0146E490>> Traceback (most recent call last):

File "c:\python24\lib\site-packages\cherrypy-2.2.0-py2.4.egg\cherrypy\_cphttptools.py", line 106, in _run

self.main()

File "c:\python24\lib\site-packages\cherrypy-2.2.0-py2.4.egg\cherrypy\_cphttptools.py", line 255, in main

body = page_handler(*virtual_path, self.params)

File "<string>", line 3, in po_view File "c:\turbogears\turbogears\controllers.py", line 200, in expose

output = database.run_with_transaction(expose._expose,func, accept, allow_json, allow_json_from_config,*args, kw)

File "c:\turbogears\turbogears\database.py", line 216, in run_with_transaction

retval = func(*args, kw)

File "<string>", line 5, in _expose File "c:\turbogears\turbogears\controllers.py", line 220, in <lambda>

expose._expose.when(rule)(lambda _func, accept, allow_json, allow_json_from_config,*args,kw: _execute_func(

File "c:\turbogears\turbogears\controllers.py", line 241, in _execute_func

output = errorhandling.try_call(func, *args, kw)

File "c:\turbogears\turbogears\errorhandling.py", line 68, in try_call

return func(self, *args, kw)

File "c:\turbogears\turbogears\toolbox\admi18n\init.py", line 125, in po_view

return dict(code=code, catalog=catalog.items(path,sort_by,dir),visible_checkbox=visible_checkbox )

File "c:\turbogears\turbogears\toolbox\admi18n\catalog.py", line 24, in items

po = parse(path)

File "c:\turbogears\turbogears\toolbox\admi18n\catalog.py", line 172, in parse

l = eval(l)

File "<string>", line 1

# SOME DESCRIPTIVE TITLE.

SyntaxError?: unexpected EOF while parsing

comment:4 Changed 13 years ago by matej

I forgot to mention that if I save the file as UTF-8 without BOM, it works, too. So it seems the admi18n doesn't like the BOM, at least on Windows.

But still, editing .po files manually isn't the right way if the web interface is provided and it looks like a bug.

comment:5 Changed 13 years ago by kevin

  • Cc ronald@… added
  • Milestone changed from 0.9a5 to 1.0b1

Bumping to next version

comment:6 Changed 13 years ago by max

  • Cc ischenko@… added
  • Status changed from new to closed
  • Resolution set to fixed

I made major changes to i18n implementation that should fix this bug among others.

Please re-test (from SVN or when new stable release arrive) and re-open this ticket or create new one if needed.

comment:7 Changed 13 years ago by matej

  • Status changed from closed to reopened
  • Resolution fixed deleted

Nope, it's the same with current svn. The only thing being different is that I don't get the visible error when saving the .po file as utf-8 with bom, and instead the whole server crashes with error message "Escape error on C:\Python24\Scripts\testni\locales\sl\LC_MESSAGES\messages.po:1 before: u'\ufeff# SOME DESCRIPTIVE TITLE.'" The rest is the same as before.

comment:8 Changed 13 years ago by max

  • Status changed from reopened to closed
  • Resolution set to fixed

Indeed. Please retest with r1504.

Note: See TracTickets for help on using tickets.