launchpadblog

It’s incredible to think that more than 57,000 people have used Launchpad to translate software from English into their own language.

Many of them have worked directly on upstream projects, such as the OpenShot video editor. Others have helped translate Ubuntu packages of software. And then there’s a whole group of people who translate upstream software outside of Launchpad.

Today we’ve taken another step in bringing those efforts closer together by making it far easier to get upstream translations directly into Ubuntu.

We want the strings produced by translators working directly on software projects, whether in Launchpad or elsewhere, to be easily available to the Ubuntu translators and we believe it should be just as easy for software projects to take advantage of the work of Ubuntu translators.

How it works

Translation sharing between different releases of a project, or Ubuntu, has been available in Launchpad for some time now. Also, sharing translations between an upstream project translated in Launchpad and its Ubuntu package has been helping to bring those two communities of translators closer together.

What’s changed today is that strings from upstream projects who make their translations outside Launchpad are now just as easily imported and ready for use by Ubuntu.

Now, so long as the upstream project is set up in Launchpad to do this, a change made in an upstream project’s source code — whether hosted directly in Launchpad or elsewhere in Bazaar, Git, Subversion of CVS — will be available to Ubuntu translators just a few hours later.

Previously, Ubuntu took translation templates and files from the source packages as they were uploaded. There was no automated route for those new upstream translations to get into Ubuntu after that initial import. In effect, this allowed Ubuntu translations to diverge from upstream during the six month Ubuntu cycle.

This change has a nice side benefit of making it easier for upstream projects to make use of translation work done for Ubuntu, because the English strings will diverge far less and it will be easier to spot where the Ubuntu community has done new translation work, rather than their being a divergence due to the two efforts drifting apart.

To start with, this is available for projects that use intltools, which includes all of GNOME. To get your project’s translations automatically imported into Launchpad, see our help guide.

Photo by Kino Praxis. Licence: CC BY 2.0

Here’s a quick pub quiz:

Question: How do you make packages for Ubuntu?

You can choose from the following answers:

1. learn Debian packaging through hours of study and practice
2. borrow existing packaging from elsewhere, throw a couple of Bazaar branches together and let Launchpad handle the rest
3. Uruguay in both 1930 and 1950.

If you selected either of the first answers you’d be right.

Okay, so, if you want to do it for real — i.e. become an Ubuntu MOTU or otherwise create Debian-style packages from scratch — then you still need to go through the hard work.

However, for everyone else who really just needs to get something out there and working for, say, a group of beta testers, we now have Launchpad’s source package recipes.

How it works, in three steps

It’s almost ridiculously easy to set up a source package build:

And that’s it. Within a few minutes you can set up a daily build direct from your trunk or any other buildable branch in Launchpad.

Watch how it works in our screencast:

An example

Let’s say you’re the developer of a home finance application called Alvin. You track your project’s code using Git and host it on your own server. For the past couple of years Alvin has been packaged in the Ubuntu universe and your trunk has also been imported from Git to a Bazaar branch in Launchpad at lp:alvin.

Just as you’re approaching Alvin’s next release, you want to get some wider testing. In the past, you’ve published a nightly tarball and provided instructions on manual installation. That’s given you a handful of dedicated beta testers but you’re worried that you’re asking too much of people.

With Launchpad’s source package recipes, you write a short recipe that pulls in your trunk branch, adds the packaging from Alvin’s existing Ubuntu package and then builds an installable Ubuntu package in the PPA of your choice:


# bzr-builder format 0.3 deb-version 2.0beta+{revno}
lp:alvin
nest-part packaging lp:ubuntu/alvin debian debian


Paste the recipe into Launchpad and with a couple of clicks you have a daily build of your trunk, that’s published as an Ubuntu package in your PPA.

Now you can ask people to test the latest Alvin code by doing no more than adding your PPA to their system. Launchpad will build a new version of the package on each day it spots a change in your trunk (or the Ubuntu packaging). For your beta testers, any changes will show up just like any other Ubuntu update.

Simple as that!

Here’s a quick recap of how it works: you can take any buildable branch — whether hosted in directly Launchpad or imported from Git, Subversion, CVS or Bazaar hosted elsewhere — merge or nest other branches, add packaging and then leave it to Launchpad to create a daily build that it publishes in your chosen PPA.

Seeing it in action

During the beta, people added a whole range of source package recipes, with a list of more than 350 daily builds as I write this.

Daily builds on Launchpad right now include Project Neon, who have around sixty recipes providing daily builds of KDE and Amarok. There are also daily builds of the Scribus DTP app, Audacity and the scriptable screen reader Gnome Orca.

Try it out

It’s easy to get your own source package recipes and daily builds up and running.

Start at our Getting Started guide and screencast.

I’ll leave the last word to Luke Benstead, who has been using source package recipes while developing a set of game libraries:

I’ve been using LP to develop some small open source game libraries. Because there are quite a few of them, packaging them all is a pain, so the package builds have worked out pretty well for them.

Now I get nightly builds delivered to a PPA, so I know that if I fix a bug it’s reflected to all my machines. And my recipes are only a single line so they’ve been really easy to use. I’m not really sure how they could be easier.

Images:
Beer photo by dearbarbie. CC-BY-SA.
Alvin Hall photo by Phil Guest. CC-BY-SA.

A few weeks ago the Yellow Squad made another change to increase the relevance of the email you get from Launchpad by eliminating some noisy ones.  A while back, Matthew Paul Thomas noticed that a change to a bug that was subsequently reverted could be deemed a mistake and was an action no one really needed to know about.  As is his habit, mpt opened a bug (164196) with the suggestion that corrected actions not generate email.

So if Alice is assigned to a bug by mistake and then unassigned within five minutes no email is generated.  Likewise, if a tag is added but then quickly removed the action does not cause any email to be sent.

Note that avoiding email requires that the action be undone, not just fixed.  By that I mean the bug must be returned to the original state to be recognized as an undoing.  So if you assign Alice to a bug by mistake and then change that assignment to Bob then the action will not be seen as a mistake.  Email will be sent notifying about both assignee changes to Alice and then to Bob.

Thanks to Gary and Danilo for the fix.

Short story: http://pad.lv/12345 takes you to bug 12345, and pad.lv describes more abbreviations.

Sometimes you’d like to point people to an interesting bug in a project that uses Launchpad, like bug 685380 (that ‘1’ and ‘l’ may need to be more distinct in the new Ubuntu Font).

Typing out https://launchpad.net/bugs/685380 is a bit tedious, and it uses up a fair bit of space in a microblog entry. You can use any of innumerable URL-shortening services, but then the URL’s opaque; which is a shame since it really just wants to represent a 6-digit number.

Therefore: pad.lv (pad love), transparent short URLs for bugs, and other things including projects, people, bug-filing forms, packages, and more.

Maybe someone would like to make bookmarklets that generate these links, or add them into the Launchpad UI?

Thanks to Latvia for letting us use a fraction of their domain name space!

Photo by Anton Lindqvist. Licence: CC BY 2.0.

Launchpad has an anonymous FTP server that people use to upload source packages to Ubuntu and their PPAs. When Launchpad processes the upload it does a huge number of checks, one of which is verifying the GPG signature on the upload’s .changes file.

One of the problems with this is that if there’s a problem with the signature, or there is no signature at all, Launchpad simply throws the upload away as it cannot be sure who uploaded the package. If it tried to send an email it would also quickly become a spam vector! (See bug 374019)

In today’s release, there is a brand new FTP server that will do preliminary GPG signature checks right in the FTP session itself. If you upload a package that is not signed properly, you’ll see a message that looks like this:

Uploading to launchpad (via ftp to ppa.launchpad.net):
Uploading hello_2.5-1ubuntu1.dsc: done.
Uploading hello_2.5-1ubuntu1.diff.gz: done.
Uploading hello_2.5-1ubuntu1_source.changes: 1k/2k550 (‘Changes file must be signed with a valid GPG signature: Verification failed 3 times: [“(7, 8, u\’Bad signature\’)”, “(7, 8, u\’Bad signature\’)”, “(7, 8, u\’Bad signature\’)”] ‘,): Permission denied.
Note: This error might indicate a problem with your passive_ftp setting.
Please consult dput.cf(5) for details on this configuration option.

which means you get immediate feedback instead of your upload disappearing entirely!

As a bonus, this new FTP server also fixes another long-standing bug where uploads would hang with 1k left to go.

This checking is not available on the SFTP service yet, but we hope to implement that in the near future.

Launchpad’s bug mail can be a bit chatty sometimes as I’m sure you’ve noticed.  This cycle the Yellow Squad is working to give you more control about the bug mail Launchpad sends to you.

One problem that we’ve known about for a really long time was reported as bug 548 and has recently been closed thanks to the effort of  Данило Шеган and Gary Poster.  Their fix allows you to globally specify whether you want to receive email about actions you did.

Most people probably do not need to be reminded of something they did a few minutes ago and will want to turn off those emails.  But, since this is new functionality we’ve preserved the old behavior unless a user changes the setting.  If you do nothing you’ll continue to get email about actions you instigate on bugs.

Opting out of those messages is easy.  Simply go to https://launchpad.net/people/+me/+edit and uncheck the box as shown below by the big red arrow.

As mentioned, this is just the first of many features and refinements that we’re working on to help you customize the stream of bug mail coming from Launchpad to suit your needs.

Launchpad has a REST API that exposes almost every object within Launchpad. Most of them have API URLs that closely match the human-readable URL: for instance, https://bugs.launchpad.net/bugs/316694 can also be obtained in machine-oriented JSON or XML form from http://api.launchpad.net/1.0/bugs/316694. (The “1.0” in that URL means this is in the 1.0 API namespace.)

Previously, many Launchpadlib clients used string transformation to get from the API URL to something they could show a human in a web browser.

We’ve just added a web_link property on all Launchpad objects, so client applications can (and should) now just use that instead. Because this is just sent in the object representation you don’t need to upgrade launchpadlib to see it.

An long-requested feature in Launchpad is to let people see who’s using a PPA. Finally, we’ve implemented this!

Initially, the stats are only available on Launchpad’s webservice API. but we aim to show something useful in the web UI at some point.

If you are already familiar with the webservice API, then you can use the following binary_package_publishing_history object methods to retrieve the information:

• getDailyDownloadTotals
• getDownloadCount
• getDownloadCounts

Fabien Tassin is already using the stats to see how many people are using his daily build PPAs, and wrote an interesting blog post about it.

Continuing with the recent improvements to the build farm – Jelmer has made another massive one.

The last major scalability problem that we had was one where the whole farm was blocked when a single builder was ready to upload a build. In the case of large packages, like the kernel, the manager process could block for over a minute while it waited for the upload processor to unpack the package and verify its contents.

Jelmer’s work has decoupled the upload processing from the build farm manager process. What happens now is that the files collected from the builder are thrown into a staging queue area and then the manager process immediately continues with polling the rest of the builders, unblocked. A cron job will then process the builder upload queue at 1 minute intervals.

You can see the dramatic effect this has had on the overall queue for the PPA builders here:

This is quite an incredible improvement as you can see! But we’re not stopping there, we’re currently doing a massive refactoring of the builder dispatching code so it’s all fully asynchronous. When this is all done we’re going to be in superb shape to support an increase in load that’s anticipated from the increasing number of people using the package recipes.

Some bugs get reported more than once. That’s why we’ve got the dupe finder.

Some duplicate bugs slip through the dupe finder. Really common issues get quite a few dupes and someone from the relevant project usually goes through and marks them as duplicates of the master bug where the actual discussion and tracking is taking place.

There has been a really annoying bug in the way Launchpad has handled all this, though, and Deryck‘s just fixed it 🙂

Let’s say you’ve got a bug report that has a few duplicates attached to it. You then discover that, actually, there’s an older bug with a more mature discussion and that, really, that’s the master bug. Until now, before you marked your bug as a duplicate of the master, you’d first have to take all the dupes of your bug and manually make them dupes of the master.

Still with me? 🙂

For a busy bug with many dupes, some of which have their own dupes, that’s a real disincentive to clearing up the multiple duplicates and gathering everything together on the one true master bug.

Now, though, simply mark your bug as a duplicate and Launchpad will automatically transfer your bug’s dupes to the new master bug.

Simple 🙂