launchpadblog

Now grows together what belongs together. Introducing a cool new feature: importing translation templates from Bazaar branches.

You can (and should) host your source code on code.launchpad.net using Bazaar branches. You also have a great translation tool in translations.launchpad.net that can help you to publish your project in many different languages. But so far, these two great applications lived seperate lives. This is beginning to change now!

How do you start translating your project into foreign languages? You start by generating a template file from your source code and uploading this to Launchpad. The translation template contains the English strings that translators are meant to translate into their language. What if the source code is hosted on Launchpad? Until now, you still had to upload that template manually to Launchpad Translations and wait for it to be imported.

Here is how it works now: generate the template file in your source tree using gettext tools, as you would have done before. But instead of uploading it, simply commit it to your Bazaar branch and push that branch to Launchpad. Now it just takes a one-time configuration step and your template file or files will be automatically imported into Translations. What’s more, this happens every time you push a new version of that file without you having to do anything.

To get started with importing translation templates from a branch, navigate to the “Overview” page of the release series of you project that you want to import translations into (usually “trunk”). Click on “Link to branch” to link this series to a Bazaar branch. If this has already been done, use this step to verify that this is really where your template file can be found. Next, go to the “Translations” page of that release series. You’ll probably see the message “No translatable templates available” but you are about to change that by clicking on “Settings” in the menu bar. Here you change the import mode to “Import translation files”. Once you click on “Save settings” an initial import of your translation templates will be triggered and you should soon find it in the import queue and shortly after that the template is available for translation work. Now, whenever you update the template file in your branch you just have to wait some 15 minutes or so and the change will appear in your translation template on Launchpad.

Read all the gory details on the help page and watch out for how in the future Bazaar branches and translations will grow even closer together.

I’m in Buenos Aires this week with the Launchpad Translations team — Danilo, Jeroen and Henning — along with Launchpad UI guy Martin Albisetti.

We’re working on making Launchpad cleverer at giving people the information they need where they need it. Each day we’ve been sitting with a white board, a list of existing pages that we want to improve and one simple question: what do translators want from that page?

And that’s a question that we can answer through the feedback that we’ve had on how translation can work better and also the translations that we’ve made ourselves.

We’re half way through the week and have some great plans that you can expect to see in place before our 3.0 release in July.

A few days ago a fix for bug #286359 landed: provide translation import notifications to Ubuntu packagers. This means that all packagers will start getting emails about translation files (basically, *.pot and *.po files) from their packages as soon as they are imported (or the import fails).

This should help packagers detect and fix l10n-related problems more easily, since they’ll have most of the debugging information they need right inside their inboxes. If you want to filter these emails, look for a sender of rosetta@launchpad.net.

The grey area of where to send notifications for automatically synced package uploads from Debian remains. I am planning on solving that with a public mailing list where anyone interested can go in and check for problems, and also fix them if they want to. Alternative is to rely on the recent improvement by Jeroen which keeps any failure messages inside the import queue itself, and simply drop all of these messages altogether.

Due to some caching problems between our machines, some large translations have failed to import, generating strange errors such as:

• “String not terminated” (the most common one, at a big line number)
• “Got a truncated message!”
• “String is not quoted”
• “Invalid content u’msg’” (or other similar instances)

You’d see the error message in import failure emails. Retrying (sometimes a few times) would push the file through.

This morning we’ve disabled caching between the importing and file storing servers, and it seems to have fixed all the problems. However, the fix we used means that everyone will now get a broken URL in translation import failure emails. If the email mentions a URL like

http://mizuho.canonical.com:8000/SOME-NUMBER/filename

replace mizuho.canonical.com with launchpadlibrarian.net and the link should work.

If you still see one of the above errors on your imports, and you are sure it’s not a msgfmt syntax error (or bug #88831), please add a comment to the bug.

The week of February 2nd – 6th has been the inaugural Launchpad Performance Week: a week where we concentrate on the most pressing performance issues and do anything we can to improve them. During that week, we also establish a hard goal that we enforce by lowering the timeout limit by 5 seconds.

The Launchpad Translations team has been concentrating on improving performance of two most problematic pages we are in charge of: POFile +translate pages (where translators do most of their work) and Person +translations pages, which display history for a person’s translation contributions inside Launchpad.

Jeroen has done a very good job improving Person:+translations pages (pages like Yannick’s translation history which used to timeout occasionally). We have since seen no timeouts on that page.

Henning has spent some time on writing a script to remove obsolete Ubuntu translations (translations for Warthy, Hoary, Breezy, Edgy and Feisty) from the database, before joining Ubuntu guys for the Jaunty sprint in Berlin. This should remove around one third of our entire data set, which should help with rendering of POFile:+translate pages. Unfortunately, he was unable to complete that — due to not being around — so I picked it up and have completed and tested it this week. We should have this done by the end of the month, and that should bring large improvements to the rendering of +translate page.

Additionally, I’ve disabled global suggestions on edge (translations coming from different PO templates in Launchpad for the same English string) to confirm this is indeed our biggest problem. Edge has practically been flawless since, and we’ve discovered more areas for improvement. We are getting some new hardware to help with competition for resources between different jobs on the server as well (should be in by the end of the month), and as soon as we do, we’ll re-enable global suggestions (which are one of Launchpad’s essential features).

Just as the week was ending, we started getting a bunch of timeouts on DistroSeriesLanguage pages like the Brazilian Portuguese one for Jaunty: anonymous users started getting it frequently using a script, along with a large batch size of 300, and that started putting a lot of load on the server. Jeroen already has an idea how to improve the situation significantly, but that will have to wait until a later date.

I am looking forward to February release of Launchpad which should bring performance improvements for everyone to notice.

Finding the right balance between open access and quality control is an ongoing challenge both for translations groups and the Launchpad Translations developers.

Launchpad already gives projects the flexibility to choose what level of openness versus control that they want for their projects, through different permissions policies. While that allows you to decide who can translate and who should review new translations, until now it hasn’t been particularly easy to introduce new and drive-by translators to your way of doing things.

Now, if you’re in a translation team, you can set a link to an externally hosted translations style guide. Simply go to the translations tab of your team’s page to set the link.

Once set, a link to your style guide will appear on each translation page for which your team is responsible, meaning you have a greater chance of getting suitable strings from new translators.

Providing translation work back upstream is now greatly simplified!

There is a lot of translation work going on in Launchpad, for Ubuntu as well as for other projects. There is also a lot of translation work going on for the projects, that is not done in Launchpad. This is especially true for many of the Ubuntu packages that have their own translation effort “upstream”.

For various reasons, translations imported from upstream projects may be altered in Launchpad. One possible scenario is that an error is detected in the upstream translation with no time to fix that upstream and import again, because the next Ubuntu release is imminent. The Ubuntu translator will then fix it in Launchpad and Ubuntu will have the corrected version. But now it is a matter of good community citizenship to provide that change back upstream.

So far, the only option here was to either communicate the changes manually or to download the whole translation file and provide that to the upstream project. Unfortunately this may not be easy to merge into the upstream translations which may have progressed in the meantime. This step is now simplified by a new feature that only exports those translation strings from Launchpad that were changed from what was originally imported from upstream. This export also includes translations of strings that were not translated at all before.

You can find information about the feature on the help page.

I really hope that this feature will find good use and that the upstream translations can profit much more from the translation work done in Launchpad. It is as easy as clicking on “export” and then forwarding the exported file to upstream.

Watch for more enhancements on the import/export front in the next releases.

If you translate using Launchpad, you’ll almost certainly be delighted to read that you can now search for translations!

Take a look at the English (UK) translations for Wordpress. There are two search boxes. One at the very top-right of the page: that’s Launchpad’s new site-wide search. Below that, still on the right, is the translation search box.

Using the translations search box, you can search for a word or phrase as it appears in either the original English template or the language you’re currently viewing.

I had a chat with Jeroen Vermuelen, leader of the Launchpad Translations team, to ask about the work that has gone into the feature. He told me:

“This is something that people have wanted for a long time and the new translations search is the result of work we’ve been doing for the past year. We could have done it sooner but we wanted to do it right.

“When you’re searching as much data as we have in Launchpad Translations – around 16 million strings – you have to take the right approach to ensure results are accurate and timely.

“So, this is just what the doctor ordered for translators who are managing a particular translation and want to be able to tweak a specific message quickly.”

The next Ubuntu release – Hardy Heron – is now open for translation!

If you want to help ensure it’s accurately translated into your language, you can get going at:

https://translations.launchpad.net/ubuntu/hardy/+translations

If you’re new to translating Ubuntu and want to find out more, take a look at the Translating Ubuntu.

Frequent lang pack updates

If you’re running Hardy Heron during its development cycle, you’ll receive twice weekly language pack updates reflecting the latest work in Launchpad Translations. These updates should start later this week.

If you’re planning to work on Hardy translations, this is a great way to see your work in place before the official release.

If you have any questions about using Launchpad to translate, ask on launchpad-users or come join us in #launchpad on Freenode.

Translating using Launchpad is now faster and you’ll see next to no timeouts.

Over the past month, Launchpad’s Translations developers – Carlos, Danilo and Jeroen – have worked exclusively on simplifying the database and code used by Launchpad Translations. Danilo told me a bit about what they’ve done:

“This is a very welcome change of data model which has enabled us to cut the code size by 30%, improve speed and keep the same set of features. Even if this doesn’t solve all the timeout problems, it provides us with the infrastructure to do so in the near future.”

If you use Launchpad Translations, you can expect exports and imports to take roughly half as long as before and everyday use of Launchpad Translations to be noticeably faster.