Warning: Can't synchronize with repository "(default)" (Unsupported version control system "svn": No module named svn). Look in the Trac log for more information.

Ticket #1695 (closed defect: fixed)

Opened 10 years ago

Last modified 9 years ago

i18n - translations for text with leading/trailing whitespace not found

Reported by: amit Owned by: Chris Arndt
Priority: normal Milestone: 1.0.x bugfix
Component: TurboGears Version: 1.0.4b6
Severity: critical Keywords:
Cc: diegobz

Description

I have seen that the TG 1.0.4b6, collects strings with trailing or leading space, for example:

<span>Welcome to <a href="http://mysite.com">My Site</a>.</span>

We get:

msgid "Welcome to "

instead of:

msgid "Welcome to"

And TG fails to get translation for "Welcome to ". If I manually remove that trailing whitespace, it just works fine...

Attachments

admi18n.patch Download (968 bytes) - added by amit 10 years ago.
I don't know whether this is the correct way to fix the problem, but it works for me…
i18n-kid-strip-msg.diff Download (1.3 KB) - added by Chris Arndt 9 years ago.

Change History

comment:1 Changed 10 years ago by amit

Another strange behavior:

<td>
    Total:
</td>

on tg-admin i18n collect, I have:

#: templates/test.kid:td
msgid ""
"\n"
"    Total:"
msgstr ""

instead of:

#: templates/test.kid:td
msgid "Total:"
msgstr ""

That is, all leading and trailing whitespace needs to be stripped out from the gathered strings.

comment:2 Changed 10 years ago by amit

  • Summary changed from i18n - can't translate text with trailing whitespace (v1.0.4b6) to i18n - can't translate text with leading/trailing whitespace (v1.0.4b6)

Changed 10 years ago by amit

I don't know whether this is the correct way to fix the problem, but it works for me...

comment:3 Changed 10 years ago by Chris Arndt

  • Version changed from 1.0.4b3 to 1.0.4b6

comment:4 Changed 10 years ago by Chris Arndt

  • Summary changed from i18n - can't translate text with leading/trailing whitespace (v1.0.4b6) to i18n - translations for text with leading/trailing whitespace not found
  • Milestone changed from 1.0.4 to 1.0.x bugfix

comment:5 Changed 9 years ago by diegobz

  • Cc diegobz added

comment:6 Changed 9 years ago by diegobz

Unfortunate this patch didn't work here.

Any other idea?

My TG version currently is: TurboGears-1.0.4.3-2.fc8

comment:7 Changed 9 years ago by amit

It works, I have tested with  http://svn.turbogears.org/branches/1.0 (rev 4328).

comment:8 Changed 9 years ago by diegobz

I have added the "s = s.strip()" for both files and compiled them with:

[root@localhost admi18n]# pwd
/usr/lib/python2.5/site-packages/turbogears/toolbox/admi18n
[root@localhost admi18n]# python -O -m compileall ./
Listing ./ ...
Compiling ./catalog.py ...
Compiling ./pygettext.py ...

After that, I have tried to run "tg-admin i18n collect" into my project directory, but I got the following error:

[diego@localhost transifex]$ tg-admin i18n collect
Use po/ as a locale directory
Use transifex as a message domain
Scanning source directory transifex
Working on transifex/people.py
Working on transifex/model.py
Working on transifex/release.py
Working on transifex/admin.py
Working on transifex/repo.py
Working on transifex/util.py
Working on transifex/module.py
Working on transifex/__init__.py
Working on transifex/controllers.py
Working on transifex/config/__init__.py
Working on transifex/templates/__init__.py
Working on transifex/tests/test_controllers.py
Working on transifex/tests/test_model.py
Working on transifex/tests/__init__.py
Traceback (most recent call last):
  File "/usr/bin/tg-admin", line 8, in <module>
    load_entry_point('TurboGears==1.0.4.3', 'console_scripts', 'tg-admin')()
  File "/usr/lib/python2.5/site-packages/turbogears/command/base.py", line 371, in main
    command.run()
  File "/usr/lib/python2.5/site-packages/turbogears/command/i18n.py", line 142, in run
    self.scan_source_files()
  File "/usr/lib/python2.5/site-packages/turbogears/command/i18n.py", line 302, in scan_source_files
    pygettext.main()
  File "/usr/lib/python2.5/site-packages/turbogears/toolbox/admi18n/pygettext.py", line 751, in main
    eater.write(fp)
  File "/usr/lib/python2.5/site-packages/turbogears/toolbox/admi18n/pygettext.py", line 593, in write
    print >> fp, 'msgid', normalize(k, options.escape)
  File "/usr/lib/python2.5/site-packages/turbogears/toolbox/admi18n/pygettext.py", line 269, in normalize
    lines = s.split('\n')
AttributeError: 'list' object has no attribute 'split'

Any idea?

comment:9 Changed 9 years ago by diegobz

Never mind. My mistake. It just worked.

However, after I've applied the patch, when I have a situation like:

in info.kid

<li>Click <a>here</a>.</li>

in other_file.kid

<li>Hey, click <a>here</a>.</li>

with a DOT (or whatever 'single' character like: ')' or ':' ) between the last two TAGS, I'm having problems with msgid duplicated like:

msgid "." 

into my POT file

comment:10 Changed 9 years ago by Chris Arndt

  • Status changed from new to assigned
  • Owner changed from anonymous to Chris Arndt

comment:11 Changed 9 years ago by Chris Arndt

Unfortunately, stripping the message string in the normalize function is too late. The unstripped message is used before to determine the uniqueness of the message, so messages with different whitespace on the left or right end would be doubled in the .pot file as reported by diogobz.

My attached patch fixes the problem for kid files. I'll look into Python source files later. Does the problem even exist there?

Changed 9 years ago by Chris Arndt

comment:12 Changed 9 years ago by amit

Thanks Chris,

It has fixed the problem for kid, but the problem still exists for Python code. You are right, the patch I have attached is not good (duplicate message ids in pot).

comment:13 Changed 9 years ago by Chris Arndt

In what way does this bug affect translatable strings in Python source code files? If you have a translatable string in Python source code, you wrap it into the _ function. There is no need to surround the string inside with white space nor should it be stripped off, IMHO. So if you do this:

foo = _(u' *** Eat more spamm! *** ')

... the message key will (and should) be exactly " *** Eat more spamm! *** "., e.g.:

#:mypkg/controllers.py:nn
msgid " *** Eat more spamm! *** "
msgstr " *** Esst mehr Spamm! ** "

This will be translated normally by gettext.

comment:14 Changed 9 years ago by Chris Arndt

  • Status changed from assigned to closed
  • Resolution set to fixed

Fixed in r5367 by applying my attached patch in 1.0, 1.1 and 1.5 branch.

Note: See TracTickets for help on using tickets.