Good news if you run a project’s translation effort in Launchpad!
Until today, when you imported a template or translation file into Launchpad for the first time, you’d have to wait for a member of the Canonical Launchpad team to review and then approve that file before your project’s translation community could make use of it.
Now, if you’re a project maintainer, you can manage your project’s translations import queue yourself. All you need do is follow the “import queue” link on your project’s translations overview page and you’ll see something like this:
Once you’ve approved a file, and it has been imported, subsequent changes will go through Launchpad’s automatic approval process.
I’m thrilled to be writing this blog post just over a year after starting as Launchpad’s technical architect. During that year we have been steadily improving our ability to deploy changes to Launchpad without causing downtime (of any or all services). Our ability to do this directly impacts our ability to deliver bug fixes and new functionality – our users are very sensitive to downtime.
There has been one particularly tricky holdout though – our monthly 90 minute downtime window where we apply schema changes, do DB server maintenance and so forth.
Starting very soon we will instead have very short windows – approximately 60 seconds long – where we perform schema changes, database server failover (in order to permit DB maintenance on the master server) and so forth.
We expect to do these about 6 times a month based on our historical rate of schema patches, and we are – for now – planning on doing these at 0800 UTC consistently.
This will deliver much less total downtime – 6 minutes a month rather than 90 – at the cost of more frequent interruptions.
If you have API scripts running against Launchpad, you may want to build in a retry mechanism to deal with up to a few minutes of downtime.
We cannot remove downtime entirely for purely technical reasons: Our primary database (postgresql) blocks new readers (or writers) when a schema change is being executed, and the schema change blocks on existing readers (or writers) to complete – it needs an exclusive lock on each relation being altered.
What we can do is automate the process of disconnecting and interrupting existing database connections to let the schema change execute rapidly, and make our schema changes as minimal as possible. Previously, we shut down all the application servers (via a script, but shutting down gracefully takes time), and then ran schema changes which did data migration and so forth. In this new process we will leave the appservers running and just interrupt their connections for the time it take to apply the schema change. That, combined with moving data migration to a background job rather than doing it during the schema change, gives us the short downtimes we’re about to start doing.
Users of Launchpad Answers will see that asking a question, editing it, or posting a comment to it is faster. Email about question changes is sent a few minutes latter. Many bugs relating to question emails were fixed as we moved the work of sending emails to the new process.
Users and answer contacts saw slow pages or time out errors when working with questions in large projects. Simple actions like asking a question or providing an answer would fail. It was common to see errors converting bugs into questions. A few weeks ago, we saw that 8 of the top 10 kinds of time outs involved questions; though this ratio was caused in part by the tremendous work of making other parts of Launchpad faster.
The root cause of the slow question pages was sending email to all the subscribers before showing the next page. The solution was to queue the the event to notify subscribers, and send the emails later. While updating the code, there were many opportunities to fix related Answers bugs. I am particularly pleased with the changes to the rules to create a question. There were four lines of code, and while I intended to fix one line, I realised there was a bug related to each line of code. In a matter of minutes I had fixed four bugs. The most obvious change you will see is that question emails will now state that you received the email because you asked the question, where previously you were merely described as a subscriber.
lpad is based on a two-layered design. The top layer offers a static API which allows a more comfortable interaction with the API with static checks, better documentation, and more. The bottom layer is fully dynamic and enables the developer to access all the features of Launchpad, even those not supported by the top static layer.
There’s still work to do but the library is pretty much complete and it’s well tested, including integration tests which communicate with the real production servers.
It’s incredible to think that more than 57,000 people have used Launchpad to translate software from English into their own language.
Many of them have worked directly on upstream projects, such as the OpenShot video editor. Others have helped translate Ubuntu packages of software. And then there’s a whole group of people who translate upstream software outside of Launchpad.
Today we’ve taken another step in bringing those efforts closer together by making it far easier to get upstream translations directly into Ubuntu.
We want the strings produced by translators working directly on software projects, whether in Launchpad or elsewhere, to be easily available to the Ubuntu translators and we believe it should be just as easy for software projects to take advantage of the work of Ubuntu translators.
How it works
Translation sharing between different releases of a project, or Ubuntu, has been available in Launchpad for some time now. Also, sharing translations between an upstream project translated in Launchpad and its Ubuntu package has been helping to bring those two communities of translators closer together.
What’s changed today is that strings from upstream projects who make their translations outside Launchpad are now just as easily imported and ready for use by Ubuntu.
Now, so long as the upstream project is set up in Launchpad to do this, a change made in an upstream project’s source code — whether hosted directly in Launchpad or elsewhere in Bazaar, Git, Subversion of CVS — will be available to Ubuntu translators just a few hours later.
Previously, Ubuntu took translation templates and files from the source packages as they were uploaded. There was no automated route for those new upstream translations to get into Ubuntu after that initial import. In effect, this allowed Ubuntu translations to diverge from upstream during the six month Ubuntu cycle.
This change has a nice side benefit of making it easier for upstream projects to make use of translation work done for Ubuntu, because the English strings will diverge far less and it will be easier to spot where the Ubuntu community has done new translation work, rather than their being a divergence due to the two efforts drifting apart.
To start with, this is available for projects that use intltools, which includes all of GNOME. To get your project’s translations automatically imported into Launchpad, see our help guide.
Recently, Ursula and I have been improving the way that we test new Launchpad features.
Already, Launchpad has an extensive test suite but there are some things that automated tests can’t look out for. Rather than just testing the quality of our code, we also want to test the quality of the experience.
To do that, we’ve been doing more exploratory testing. Now, when a feature is getting close to deployment we will try out every part of the feature and make notes of anything in the experience that needs to be fixed before we release it. In particular, we’re interested in the feature’s:
ease of use and discoverability
quality of implementation
suitability to the problem it is solving
conformity to Launchpad’s principles and UI guidelines.
We’re also aiming to timebox the testing; something that takes too long to explore is likely too complex. For now, we’re using a 25 minute limit, as borrowed from The Pomodoro Technique, as it seems like a good starting point.
If you’re interested in what we’re doing, you can follow our progress both on the launchpad-dev mailing list and on the Launchpad dev wiki. Also, I’m hoping that we can get your help in testing beta features. I’ll write more about that soon.
learn Debian packaging through hours of study and practice
borrow existing packaging from elsewhere, throw a couple of Bazaar branches together and let Launchpad handle the rest
Uruguay in both 1930 and 1950.
If you selected either of the first answers you’d be right.
Okay, so, if you want to do it for real — i.e. become an Ubuntu MOTU or otherwise create Debian-style packages from scratch — then you still need to go through the hard work.
However, for everyone else who really just needs to get something out there and working for, say, a group of beta testers, we now have Launchpad’s source package recipes.
How it works, in three steps
It’s almost ridiculously easy to set up a source package build:
Choose a branch in Launchpad, whether hosted directly or imported.
Write a short recipe that tells Launchpad which other branches to pull in, for example to provide packaging or make the code buildable.
Paste your recipe into Launchpad.
And that’s it. Within a few minutes you can set up a daily build direct from your trunk or any other buildable branch in Launchpad.
Watch how it works in our screencast:
Let’s say you’re the developer of a home finance application called Alvin. You track your project’s code using Git and host it on your own server. For the past couple of years Alvin has been packaged in the Ubuntu universe and your trunk has also been imported from Git to a Bazaar branch in Launchpad at lp:alvin.
Just as you’re approaching Alvin’s next release, you want to get some wider testing. In the past, you’ve published a nightly tarball and provided instructions on manual installation. That’s given you a handful of dedicated beta testers but you’re worried that you’re asking too much of people.
With Launchpad’s source package recipes, you write a short recipe that pulls in your trunk branch, adds the packaging from Alvin’s existing Ubuntu package and then builds an installable Ubuntu package in the PPA of your choice:
Paste the recipe into Launchpad and with a couple of clicks you have a daily build of your trunk, that’s published as an Ubuntu package in your PPA.
Now you can ask people to test the latest Alvin code by doing no more than adding your PPA to their system. Launchpad will build a new version of the package on each day it spots a change in your trunk (or the Ubuntu packaging). For your beta testers, any changes will show up just like any other Ubuntu update.
Simple as that!
Here’s a quick recap of how it works: you can take any buildable branch — whether hosted in directly Launchpad or imported from Git, Subversion, CVS or Bazaar hosted elsewhere — merge or nest other branches, add packaging and then leave it to Launchpad to create a daily build that it publishes in your chosen PPA.
I’ll leave the last word to Luke Benstead, who has been using source package recipes while developing a set of game libraries:
I’ve been using LP to develop some small open source game libraries. Because there are quite a few of them, packaging them all is a pain, so the package builds have worked out pretty well for them.
Now I get nightly builds delivered to a PPA, so I know that if I fix a bug it’s reflected to all my machines. And my recipes are only a single line so they’ve been really easy to use. I’m not really sure how they could be easier.
There are also a bunch of client programs, command line and graphical, that talk to Launchpad to do various things.
What we don’t yet have, and what I think would be great, is a systematic client that lets you manipulate
everything from the command line. There’s some code that starts towards this in Hydrazine, lptools and others, but I think having just a single tool that eventually does everything would be more discoverable and avoid unnecessary fragmentation or duplication.
(That’s not to say there’s not room for others that are guis, that are specialized to particular projects or that encapsulate a lot of policy or opinion about what they’re doing.)
So dobey and I have agreed to gradually merge hydrazine into lptools, and with other people to work towards making lptools cover everything you can do through the web UI or the API. If you have scripts you’ve written yourself, perhaps you’d like to merge them in.
Sometimes you’d like to point people to an interesting bug in a project that uses Launchpad, like bug 685380 (that ‘1’ and ‘l’ may need to be more distinct in the new Ubuntu Font).
Typing out https://launchpad.net/bugs/685380 is a bit tedious, and it uses up a fair bit of space in a microblog entry. You can use any of innumerable URL-shortening services, but then the URL’s opaque; which is a shame since it really just wants to represent a 6-digit number.
Therefore: pad.lv (pad love), transparent short URLs for bugs, and other things including projects, people, bug-filing forms, packages, and more.