i18n - translations for text with leading/trailing whitespace not found

[amit]

I have seen that the TG 1.0.4b6, collects strings with trailing or leading space, for example:

<span>Welcome to <a href="http://mysite.com">My Site</a>.</span>

We get:

msgid "Welcome to "

instead of:

msgid "Welcome to"

And TG fails to get translation for "Welcome to ". If I manually remove that trailing whitespace, it just works fine...

[amit]

Another strange behavior:

<td>
    Total:
</td>

on tg-admin i18n collect, I have:

#: templates/test.kid:td
msgid ""
"\n"
"    Total:"
msgstr ""

instead of:

#: templates/test.kid:td
msgid "Total:"
msgstr ""

That is, all leading and trailing whitespace needs to be stripped out from the gathered strings.

[amit]

I don't know whether this is the correct way to fix the problem, but it works for me...

[diegobz]

Unfortunate this patch didn't work here.

Any other idea?

My TG version currently is: TurboGears-1.0.4.3-2.fc8

[amit]

It works, I have tested with  http://svn.turbogears.org/branches/1.0 (rev 4328).

[diegobz]

I have added the "s = s.strip()" for both files and compiled them with:

[root@localhost admi18n]# pwd
/usr/lib/python2.5/site-packages/turbogears/toolbox/admi18n
[root@localhost admi18n]# python -O -m compileall ./
Listing ./ ...
Compiling ./catalog.py ...
Compiling ./pygettext.py ...

After that, I have tried to run "tg-admin i18n collect" into my project directory, but I got the following error:

[diego@localhost transifex]$ tg-admin i18n collect
Use po/ as a locale directory
Use transifex as a message domain
Scanning source directory transifex
Working on transifex/people.py
Working on transifex/model.py
Working on transifex/release.py
Working on transifex/admin.py
Working on transifex/repo.py
Working on transifex/util.py
Working on transifex/module.py
Working on transifex/__init__.py
Working on transifex/controllers.py
Working on transifex/config/__init__.py
Working on transifex/templates/__init__.py
Working on transifex/tests/test_controllers.py
Working on transifex/tests/test_model.py
Working on transifex/tests/__init__.py
Traceback (most recent call last):
  File "/usr/bin/tg-admin", line 8, in <module>
    load_entry_point('TurboGears==1.0.4.3', 'console_scripts', 'tg-admin')()
  File "/usr/lib/python2.5/site-packages/turbogears/command/base.py", line 371, in main
    command.run()
  File "/usr/lib/python2.5/site-packages/turbogears/command/i18n.py", line 142, in run
    self.scan_source_files()
  File "/usr/lib/python2.5/site-packages/turbogears/command/i18n.py", line 302, in scan_source_files
    pygettext.main()
  File "/usr/lib/python2.5/site-packages/turbogears/toolbox/admi18n/pygettext.py", line 751, in main
    eater.write(fp)
  File "/usr/lib/python2.5/site-packages/turbogears/toolbox/admi18n/pygettext.py", line 593, in write
    print >> fp, 'msgid', normalize(k, options.escape)
  File "/usr/lib/python2.5/site-packages/turbogears/toolbox/admi18n/pygettext.py", line 269, in normalize
    lines = s.split('\n')
AttributeError: 'list' object has no attribute 'split'

Any idea?

[diegobz]

Never mind. My mistake. It just worked.

However, after I've applied the patch, when I have a situation like:

in info.kid

<li>Click <a>here</a>.</li>

in other_file.kid

<li>Hey, click <a>here</a>.</li>

with a DOT (or whatever 'single' character like: ')' or ':' ) between the last two TAGS, I'm having problems with msgid duplicated like:

msgid "." 

into my POT file

[Chris Arndt]

Unfortunately, stripping the message string in the normalize function is too late. The unstripped message is used before to determine the uniqueness of the message, so messages with different whitespace on the left or right end would be doubled in the .pot file as reported by diogobz.

My attached patch fixes the problem for kid files. I'll look into Python source files later. Does the problem even exist there?

[amit]

Thanks Chris,

It has fixed the problem for kid, but the problem still exists for Python code. You are right, the patch I have attached is not good (duplicate message ids in pot).

[Chris Arndt]

In what way does this bug affect translatable strings in Python source code files? If you have a translatable string in Python source code, you wrap it into the _ function. There is no need to surround the string inside with white space nor should it be stripped off, IMHO. So if you do this:

foo = _(u' *** Eat more spamm! *** ')

... the message key will (and should) be exactly " *** Eat more spamm! *** "., e.g.:

#:mypkg/controllers.py:nn
msgid " *** Eat more spamm! *** "
msgstr " *** Esst mehr Spamm! ** "

This will be translated normally by gettext.

[Chris Arndt]

Fixed in r5367 by applying my attached patch in 1.0, 1.1 and 1.5 branch.