Feeling the heat
Published by Matthew Revell January 27, 2010 in Bug Tracking
Ascertaining the importance of a bug can be one of the harder parts of bug triage, right?
It’s no problem if the project’s relatively small and you’re intimately involved with each bug but if you’re helping to triage for a larger project, it’s not always that easy.
To help, we’ve introduced a new indicator of how hot a particular bug is. A bug’s heat is a calculated estimate of its importance, using factors such as how many people are subscribed, whether it’s a security issue, how many people have marked the bug as affecting them, and so on. Note that it is not a replacement for the bug’s human-set importance but rather an aid to the people deciding what the importance should be.
You can see the full details of how bug heat is calculated on our wiki page.
A bug’s heat is shown on the bug report page as four flame icons. The hotter the bug, the more of the icons are lit. You can also sort bug search results by bug heat.
There’s a bug at the moment that means there’s a fairly high barrier for a bug report to jump before any of its flames light up but take a look at the Ubuntu bug list, sorted by bug heat, and you’ll see the new bug heat icons in action.
Let us know how you’d like to use bug heat.
Improved Bug Patch Notifications
Published by Deryck Hodge in Bug Tracking
There are a couple of new features related to patch handling in Launchpad bugs this month.
Building on the work we did in December to better distinguish patches in bug pages, we now use an icon to show if a bug has a patch attached in bug listings. Any search on Launchpad will now indicate if a bug has a patch attached. Look for the band aid icons, and you’ll know that a bug has a patch attached.
Also, bug mail notifications have been updated to distinguish patches from any other attachment. Now when a patch is added or removed from a bug the email notification will read “Patch added” or “Patch removed” to make spotting patches easier in email.
These are small improvements to our handling of patches to help patches become more easily spotted on Launchpad. Combined with our work on sorting bugs by a heat number, the Launchpad bugs app is doing more to let users know about the state and quality of a bug report.
ACTION: Back up old sources from PPAs
Published by Jonathan Lange January 26, 2010 in General
Now that I have your attention…
We’ve been overwhelmed by the popularity of PPAs on Launchpad. In fact, according to our sysadmins, they are a little too popular and now our disks are full.
Full disks mean no more PPAs, and no more uploads to PPAs. We’d like to add some more disks, but we can’t actually do that soon enough for a bunch of complicated reasons.
Instead, we’ve decided that we’re going to remove all of the source files for any uploads that are:
- in PPAs
- not published, that is, deleted or superseded
- have been not published for over seven days
Note that we already delete the binaries for such uploads.
We are going to delete these old files this Wednesday, January 27th. We’re really sorry that we are announcing it so close to the actual event — we know it’s a hassle.
If you want to keep any of these files, you are going to have to download them right now. Here’s how to do it.
- Go to your PPA’s web page on Launchpad and click on “View package details”.
- Change the filter to search for “Any status”. Click “Filter”.
- For each superseded or deleted upload with files you want to save, expand the upload and manually save all the files under the “Package files” heading.
If it’s a busy PPA like the example one, then there will be a lot of old versions to download. If you aren’t sure, you probably won’t need all of them. Ask on #launchpad on Freenode or the launchpad-users mailing list if you are unsure.
Launchpad 10.1 roll-out 09.00-11.30 27th January 2010
Published by Matthew Revell January 22, 2010 in Notifications
Launchpad’s web interface will be read-only, and all other parts of Launchpad offline, for roughly two and a half hours from 09.00 UTC on the 27th January 2010.
Starts: 09.00 UTC 27th January 2010
Expected back: 11.30 UTC 27th January 2010
This is for the roll-out of our Launchpad 10.01 code. We’ll post details of what’s new after the release.
Anonymous Access to the Launchpad Web Service API
Published by Leonard Richardson January 19, 2010 in General
You asked for it and we’ve delivered. Your launchpadlib scripts can now get read-only access to the Launchpad web service API without going through any authorization process. Previously, to authorize your script, you had to open a web page in the end-user’s web browser and get the user to click a button. Now, you only have to go through that process if you want to access someone’s private data or modify the Launchpad dataset on their behalf.
If you’re using the latest version of launchpadlib (1.5.4), you can log in anonymously with the login_anonymously helper method. You just have to pass in a string identifying your client, and the name of the Launchpad server you want to access.
from launchpadlib.launchpad import Launchpad
launchpad = Launchpad.login_anonymously("my client", "edge")
print launchpad.bugs[1].title
# Microsoft has a majority market share'
(That code also illustrates how the latest version of Launchpadlib lets you specify a short string like “edge” instead of having to importing a constant like EDGE_SERVICE_ROOT.)
If you’re using an older version of launchpadlib, you can log in anonymously using the login helper method. You can get anonymous access by passing in empty strings for the token string and access secret:
from launchpadlib.launchpad import Launchpad, EDGE_SERVICE_ROOT
launchpad = Launchpad.login("my client", "", "", EDGE_SERVICE_ROOT)
print launchpad.bugs[1].title
# Microsoft has a majority market share'
Right now, anonymous access only works on Launchpad’s ‘edge’ and ‘staging’ instances. It’ll be installed on production at the end of the month, with the release of Launchpad 10.01.
Bug Watch Updating Temporarily Down
Published by Deryck Hodge January 15, 2010 in Bug Tracking, General
Some Launchpad users may have noticed that we are not syncing bug watches with external trackers as we should. We have had to turn off syncing with external trackers while we work on a problem we have had with hammering bug trackers with large numbers of bug watches in Launchpad.
The Launchpad Bugs team has been doing a lot of work recently to improve our handling of bug syncing between Launchpad and external trackers. We’ve fixed several bugs. Unfortunately, we cannot re-enable syncing with external trackers until we are certain we will not hammer an external tracker.
We expect bug syncing to be down through the weekend and fixed early next week. Interested developers can follow progress at the bug linked above.
Launchpad maintenance 20th January 2010
Published by Matthew Revell January 13, 2010 in Notifications
Launchpad may be unavailable for up to 30 minutes from 11.00 UTC on Wednesday the 20th January, for planned hardware maintenance.
Starting: 11.00 UTC 20th January 2010
Expected back before: 11.30 UTC 20th January 2010
We are, though, hopeful that we can complete the maintenance without taking Launchpad off-line, in which case you will not notice any effect to your service.
The Road Ahead
Published by Jonathan Lange January 8, 2010 in Coming changes
It’s my pleasure to introduce to you the single greatest Launchpad planning achievement for 2010: the roadmap.
For the last few months we’ve been working on bridging the gap between the Ubuntu distribution and the upstreams that it’s made from: making it easier for patches, translations, and bug reports to flow between Ubuntu users, Ubuntu developers, and upstream developers.
We’ve been asking users what they want and trying really hard to listen to them. And, of course, since we’re Free Software now, all of our discussion, development and planning is out in the open.
Still, there are a lot of people who care a lot about Launchpad but don’t have time to follow our mailing list or dev wiki. In particular, Ubuntu contributors are generally way too busy making Ubuntu better to keep up with every thread there. If people who care about Launchpad can’t keep track of what we are doing, they can’t tell us how we can do it better, and they can’t cheer us on when we’re doing it right.
We needed something to keep everyone in the loop. But it needed to be something simple, since we don’t want to spend all of our time telling people what we are going to do instead of actually doing it.
At Kiko’s urgings (perhaps inspired by Jono Bacon’s roadmaps), we’ve prepared a roadmap for the next few months of Launchpad development, in which the highlights are:
- Guiding developers to the most important (hottest) bugs
- Getting patches attached to bugs into the code review system
- Automatic imports into bzr from hg (we already do git)
- Ubuntu packages built fresh each day from the latest code
And of course, there’s more on the roadmap.
We intend to keep the page up-to-date and to keep the URL constant. This means that any time you want to know what we’re planning, you can look at that page and know for sure.
Remember, it’s an expression of intent, and not an actual commitment. If we can figure out a way to better connect the world of open source software, we’ll change the roadmap in a heartbeat.
For those who aren’t content with a high-level overview, you can also look at the 10.5 series on Bugs, Soyuz, Code and so forth. See the page for our current development cycle for a full listing. And, of course, you can keep following this blog for major announcements and items of interest.
I hope you enjoy the Launchpad roadmap, and I can’t wait to hear back from you. Have a great 2010.
2010 Launchpad Release Calendar
Published by Brian Fromme December 22, 2009 in Releases
We have published the 2010 Launchpad release calendar. You’ll see that there is a new Launchpad release naming scheme as well. We have moved to a YY.MM naming, which matches the Ubuntu model. You’ll find the calendar on the Launchpad development wiki, as:
https://dev.launchpad.net/Releases/2010Calendar
Having a public release calendar is particularly useful for anyone wanting to contribute to our open source project. It is also helpful for users to know when we’re likely to have down-time or read-only time.
The calendar doesn’t cover maintenance or system status. To keep up on that, please subscribe to:
http://identi.ca/launchpadstatus
Trying Out Launchpad Translations
Published by Jeroen T. Vermeulen in General, Translations
You may already know this: if you want to try out Launchpad, there’s a “playground” version of the site at https://staging.launchpad.net/
This site gets refreshed daily (or almost daily) with the latest copy of the database that’s behind the real, production Launchpad site. That makes it a good place to mess around: it’s pretty realistic but you won’t damage anything, and you won’t pollute Launchpad’s real database with fake information. It also runs newer code than the production site, so the Launchpad developers themselves use it for testing their latest changes.
Until recently, you couldn’t really test Launchpad Translations on the staging site: you needed help from an admin to get your templates imported, and if you got that done before they were overwritten by the daily refresh, you couldn’t export your translations. But today things are a lot better thanks to translation imports from, and exports to, bzr branches.
You will need:
- A Launchpad login. You can only get this on the real Launchpad; but it will also be valid on staging.
- At least one gettext translation template (“.pot file”) with strings from your program.
The translation template should have a filename in all lower-case letters and end in “.pot” as a filename extension. It should contain English “msgid” strings and empty “msgstr” strings as usual in the gettext format. (The msgstr strings don’t actually need to be empty, but in a template their contents will be ignored.)
You will be setting up:
- A bzr “development branch” on staging. This is how your templates get into the system.
- A bzr “translations branch” on staging. The system will write your translations into this branch.
Here’s how you set up and test translations on the staging site. With each of these steps, always make sure you’re working on staging, not on the production site! The staging site has the word “demo” scrawled diagonally across the page background. You may need to log in separately on this server; your account and password are the same as on the real Launchpad site.
1. Set up a project
You can do that on the project registration page. Or if you already had your project registered on the production site, it will show up in staging as well. In that case, just work with the project you have.
On the project’s main page in staging, select “Change details.” This takes you to the project’s setting page. Enable translations: check the box labeled “Translations for this project are done in Launchpad” and submit using the Change button at the bottom.
Your project should have a “trunk” release series. This is where you will be doing most of your work. Another release series will work just as well, but trunk is the default.
2. Set up a development branch
You may already have a development branch set up on the real Launchpad. But staging is a separate environment that does not have copies of your production branches, so you’ll have to set up a branch on staging anyway. Like everything else on staging, this branch will disappear when the staging site is refreshed.
The first thing to do is to make sure your bzr program is logged into Launchpad:
bzr launchpad-login mylogin
(Where mylogin stands for your Launchpad login name). Next you’ll push a branch to the staging server. How to do this depends on whether your project already has a bzr branch on Launchpad:
a. You have a bzr branch on your local system
You’ll have to push a copy of the branch to the staging server. Open a command-line shell and go into your branch. Then, supplying your own Launchpad login and project name in the obvious places, push the branch:
bzr push –use-existing-dir lp://staging/~mylogin/acme-zyzzyx/testbranch
This may complain:
bzr: ERROR: Target directory lp://staging/~mylogin/acme-zyzzyx/testbranch
already exists, but does not have a valid .bzr directory. Supply –use-existing-dir
to push there anyway.
If it does that, just run the command again and it should work.
b. You have a bzr branch on Launchpad
Go to the regular Launchpad page for the branch. The page will show how to access the branch using the bzr command-line tool. For example,
Get this branch:
bzr branch lp:acme-zyzzyx
(That is assuming that your branch is the main development branch for a project called “acme-zyzzyx.” Yours probably has a different name; it could also be of another form such as “lp:~mylogin/acme-zyzzyx/trunk”).
On your local system, go to some empty directory where you can create temporary data. Execute the “bzr branch” command line there. This creates a directory with your branch contents. Go into that directory, and push the branch to staging. You do this using the “bzr push” command:
bzr push –use-existing-dir lp://staging/~mylogin/acme-zyzzyx/testbranch
(Of course you use your own login and project name instead of “mylogin” and “acme-zyzzyx”). The “bzr push” command creates a real branch on staging. It may complain:
bzr: ERROR: Target directory lp://staging/~mylogin/acme-zyzzyx/testbranch
already exists, but does not have a valid .bzr directory. Supply –use-existing-dir
to push there anyway.
If it does that, just run the command again and it should work.
c. You don’t have a bzr branch
It’s okay if your branch is actually in some other revision control system such as subversion or git. Create a temporary copy of your source tree, so you can play without damaging anything. From the command line, go into the copy and run:
bzr init
Now use “bzr add” to add all the files that you want to see appear in the staging branch. For testing the only part that really matters is your translation templates:
bzr add po/messages.pot
Your directory structure and filenames may be different of course, but the filename should end in .pot. Commit your changes:
bzr commit -m “Setting up test branch.”
Congratulations—you have now set up a bzr branch for your project. Push a copy to the staging server. Supplying your own Launchpad login and project names in the obvious places, type:
bzr push –use-existing-dir lp://staging/~mylogin/acme-zyzzyx/testbranch
This may complain:
bzr: ERROR: Target directory lp://staging/~mylogin/acme-zyzzyx/testbranch
already exists, but does not have a valid .bzr directory. Supply –use-existing-dir
to push there anyway.
If it does that, just run the command again and it should work.
Make a note of that “bzr push” command line. You can make changes to the template later, commit them, and push them to the same location.
3. Set up a translations export branch
Besides import your template from a bzr branch, Launchpad can also export the template’s translations to a bzr branch. The real Launchpad site does this once a day, but since staging doesn’t do any of the heavy lifting, and data on staging doesn’t have so long to live, it does this every hour.
Let’s start with an empty branch for this that will contain just the translations. There can be good reasons to do it differently in your real project, but exporting to an empty branch is always a good way to start. That way you can make sure that the exports really do what you want them to before making Launchpad write to a real branch.
In some temporary directory on your system, run this from the command line:
mkdir translations-export-test
cd translations-export-test
bzr init
bzr commit –unchanged -m “Setting up translations branch.”
bzr push lp://staging/~mylogin/acme-zyzzyx/translations-export-test
(Here you probably see another complaint about “Target directory […] already exists, but does not have a valid .bzr directory.” Just run that last command line again to get past it.)
Make a note of the exact URL in that “bzr push” command. You’ll be pulling the exported translations from there into your local branch.
4. Configuring imports and exports
We now have two branches, one to import templates from and another to export translations to. Next we configure imports from your development branch and exports to your translations branch.
For the imports, tie your staging development branch to your project’s trunk release series. Go to the project’s main page on the staging server, and from there, click on to the “trunk” release series. The page you arrive at should have a section “Code for this series.” In that section, you may see a link to your real branch on production. The production branch won’t work on staging, so replace it with your development branch on staging. Click on the little “edit” icon next to the branch URL to go to the settings form. If no branch was tied to release series yet, click “link the branch to this series” instead.
You can now ask the staging server to start importing and exporting. Back on the “trunk” page, select the Translations tab at the top of the page. Look for a link “Set up branch synchronization,” and follow it.
This takes you to the page where translations synchronization with your branches is configured. It should show your chosen development branch (the one on staging!) as the import branch. Under “Import settings” select “Import template files” for now, and click “Save settings.” From now on, every time you commit a change to your translation template and push it to your development branch on staging, the latest version of the template will be imported to the staging server automatically. It usually happens within the hour.
Important: on the real Launchpad site, the development branch can also be a mirrored version of a branch you keep elsewhere. You can register an external branch (kept in CVS, Subversion, git, or bzr) and let Launchpad mirror it to a local branch. That automates the full transport from committing your template changes to your own repository on another server, all the way to having the changes show up in Launchpad Translations.
We’ll also want to set up translation exports on this page. Under “Export translations to branch,” you’ll see a link “Choose a target branch.” Selecting a branch will enable the exports immediately. (If an export branch has already been set, you’ll see its URL with an “edit” icon next to it. Clicking the icon will take you to the page where you set the branch.) Use the link to set your translations export branch.
Important: the translations export branch must be one that you own, and it must have been pushed to the server—just registering a branch is not enough. In the future it’ll also be possible to use a branch that is owned by a team that you are a member of. That plan is tracked in Launchpad under bug 407260.
At this point you have a viable working setup. As changes to your template (or even completely new templates) appear in your development branch, they are imported automatically. Snapshots of your translations are periodically exported to your translations branch. You may want to refine this further; I’ll go into that later.
5. Check up on the imports
Go to the translations import queue for your project. You’ll see links to the queue from the Translations pages for your project and for the trunk series. Within the hour you’ll see an entry for your template appear there. Its initial status will be Approved, which means it’s sitting in the queue waiting to be processed. Pending imports are processed several times an hour. Once your template has had its turn, if you refresh the queue page, its status will show as Imported (or Failed if there were errors).
Once your template has been imported, it will show up in the Translations page for your trunk series. You will see red bars for each of the languages you have set up in your preferences, standing for untranslated messages. The number of untranslated messages is indicated next to each. Until your template is imported, the red bars will show but not the numbers.
6. Translate!
Try clicking on one of the languages in the translations list with the red bars. This will take you to the template’s translation page: the UI for entering translations. Try translating one or more messages. Remember to hit “Save & continue” at the bottom to submit your changes.
Your changes are going to show up magically in your export branch. Remember the location you pushed your translations branch to on staging? Go back into your local copy of that branch and run, with that same URL:
bzr pull lp://staging/~mylogin/acme-zyzzyx/translations-export-test
Most likely, nothing will happen and the command will just say “no revisions to pull.” But keep trying! Sometime within the next hour, your latest translations will be written into the branch. When you set things up on production, you may want to set up an automated job (e.g. using “cron”) to pull snapshots of your latest translations to wherever you like to keep your translations.
7. Update the template
As your program changes, so will your template. You’ll probably run the xgettext command to extract the latest message strings. But today, since we’re only trying things out, you can edit the .pot file in a regular text editor.
So edit the template in your staging development branch. Add a new message:
msgid “Launchpad translations test message.”
msgstr “”
Now commit the change, and push it back onto staging using the same URL where you pushed your development branch before:
bzr commit -m “Template update.”
bzr push lp://staging/~mylogin/acme-zyzzyx/testbranch
Your template’s entry on the import queue will go back to being Approved, and soon after that, will be imported again. Now go back to the translation page. The new message, “Launchpad translations test message” will appear in the same relative position where you inserted it into the template.
This is how your project’s translations in Launchpad will keep up with development. From time to time you’ll want to start work on a big new release while translators continue to work on the existing release. That’s where you create a new release series. Each series has its own branches and its own translations. If the same English string occurs unchanged between two release series, the two will share their translations by default; but the old release series will be less of a “moving target” for the translators as the English strings probably won’t change so much between updates as you focus on the new release series.
8. Import translations
Remember how we set up imports of templates only? You can auto-import translations (“.po” files) as well. Go back to the branch synchronization options (on the trunk page, under the Translations tab) and set the import option to “Import template and translation files.”
Now, go back to your development branch. Did you have any PO files in there? If not, copy the one that was created in your translations export branch to the same directory where your template is. Translate a previously untranslated message so that you can see the changes happening. Add the file to the branch with “bzr add,” commit the change with “bzr commit,” and finally re-run the “bzr push” command.
If you already had PO files in the branch, all you need to do is use the one-time import option on the branch synchronization page. Each PO file should be in the same directory as its template, and be named after its language code with the “.po” extension: de.po for German, pt.po for Portuguese, pt_BR.po for Brazilian Portuguese, el.po for Greek, zh_CN.po for Simplified Chinese and zh_TW.po for Traditional Chinese, es.po for Spanish, and so on.
Now wait for up to one hour (but probably less) for your PO file to appear on the import queue for your project. This time it will appear as “Needs review”; a script that runs about once or twice an hour will approve it automatically. (If it doesn’t, check that you followed the naming rules). It will be imported soon after that.
If you go back to the translation pages, you’ll now see that the changes you pushed into the development branch have become visible in the translation UI. In the translations overview for trunk you’ll see translated messages show up as green (translated) bars instead of red (untranslated).
Note: if you made a translations change in the branch for a message that you had already translated in the Launchpad UI, you may not see the change appear. Translation changes made locally in Launchpad override translations that are imported from elsewhere; you’ll see such changes show up as light blue in the status bars. This allows you to keep track of changes made in Launchpad, and if the project’s translations are centralized elsewhere, feed the changes back “upstream.” You should try to keep the differences minimal by making sure that improvements are fed back to the origin, and that changes made in Launchpad are real improvements.
9. Going full-circle
By now you’re probably thinking that it’s not very useful to import translations from one branch and export them to another. Things get much more interesting when you export the Launchpad translations back to the branch they came from!
This is actually possible if your development branch is hosted on Launchpad. There’s nothing stopping you from using the development branch as the translations export branch as well. If you do that, you’ve got full two-way synchronization between your branch and the Launchpad user interface. Your translators can work in the Launchpad UI, or they can edit PO files from the branch and push them right back using bzr.
We normally recommend that you import only your templates from a branch, and export your translations to a branch (which may be the same one). But depending on how you choose to use Launchpad, two-way synchronization of translation files can be useful as well. There are a few important things to keep in mind though.
First, the exported translations may not always be exactly what you expect. If you name your PO files after invalid language codes, like “de_DE.po” (should be “de.po”) or “zh.po” (should be either “zh_CN.po” or “zh_TW.po” depending on which language it really is), they will normally not be imported. The same can happen if you include other things than the language code in the filenames, such as the template name. But if these files ever do get imported, they’ll be exported with their normalized, proper names. So it’s possible to end up with both a de.po and a de_DE.po in your branch. Of these, only the correctly-named one will be updated and re-imported—de.po in this case—even though your translators may still try to work on the other one.
Second, the translations export will simply overwrite any files that were already there. There is no intelligent merging, and “merge conflicts” are ignored. If you set up translations export to your branch before the first import has happened, the export can overwrite translations that were in the branch. Make sure this is really okay before enabling the exports!
There’s also the risk of “crossing” translations: translator A makes a change in the UI, around the same time translator B pushes an updated PO file to the branch. If there are pending changes in the branch when the export is scheduled to run, the export will not happen so as to avoid overwriting the changed files. Of course this does mean that frequent changes to a large branch can stop the exports from happening.
Third, an exported translation file will not be completely identical to the original, even if there were no changes in the Launchpad UI. Messages can appear in a different order, and the file’s header is updated. You may want to try merging Launchpad’s exported files into the original branch manually to get past the initial changes.
So, before you set up two-way synchronization, always test the exports on an empty branch. The staging server is the ideal environment for this. Or if you do it on the real Launchpad site, remember to save space by deleting the experimental branch once you’re done with it.
10. Help! Staging refreshed and now everything’s gone!
This happens once a day, or depending on the circumstances, once every few days. If you still have your branches on your local system, and your project exists on the real Launchpad site, it’s not hard to set things up on staging again for another experiment. Just re-run the “bzr push” command lines for the branches and set up the imports and exports again.
Once you’re satisfied, we hope you’ll decide to set up translations permanently on the real Launchpad.
