launchpadblog

Bjorn Tillenius recently took the role of Launchpad Technical Architect. That, and the release of Launchpad 3.0, seemed like a good opportunity to learn a bit more about his new role.

Matthew: You’ve just taken a new role in the Launchpad team of Launchpad Technical Architect. That sounds like a big job: how is it going so far?

Bjorn: Yes, it’s indeed a big job. There are a lof of things that are in need of attention, so a big part of my work so far have been to talk to people, and get a list of the most pressing tasks written down. But I’ve also managed to do something concrete, like integrating our Windmill (a Javascript integration test framework) test suite into our zope.testing test runner, which we use for all our other tests.

Matthew: Is your role mostly concerned with the code of Launchpad or also the servers etc?

Bjorn: My role is mostly concerned with the code of Launchpad. My main objective is to make it easier to develop Launchpad, which includes designing and refactoring code to make it easier to use. I’m also concerned about ensuring that the quality of the code is high, and that we have good automated tests.

Matthew: What are your top three priorities for your new role?

Bjorn:

1. Make it quicker to develop new pieces of code.
2. Code reuse.
3. Make sure Launchpad code is scalable.

Matthew: What challenges does being a large Python/Zope application throw up?

Bjorn: Complexity. There are a lot of sub-systems that need to interact, and it’s hard to keep track of everything in the code base. If you work on one part of the system, you might not know that another part of the system has already solved part of your problem.

Another big challenge is that Python is big on readability. It should be easy to read and understand your code. This is great, except when you get more users, and realise that the beautiful code you wrote is really slow. Producing code that is both beautiful and fast is hard, and sometimes impossible. You have to make a lot of compromises.

Matthew: What coming technologies look as though they might be interesting for Launchpad?

Bjorn: We’re currently looking into using Grok and memcached.

Grok is interesting, since it allows us to more easily replace our ZCML code with Python code. This makes it easier to both write, and to understand, since ZCML adds another language the developer has to learn, and it adds an indirection which can be hard to follow.

Memcached is interesting, since we currently render a big part of the HTML page on-the-fly, which is quite expensive. Memcached allows us to cache part of the page, so that our pages will render faster.

Matthew: How do you ensure a consistent approach to development and architecture of Launchpad with a distributed team?

Bjorn: Being in a distributed team certainly makes things harder, when it comes to making sure everyone does the same thing. I’d say our biggest asset here is the code review that we require before any code can land in the mainline. With another pair of eyes looking at the code, the chance of spotting areas of improvement increases. Of course, spotting mistakes at this point isn’t ideal, since it’s quite late in the process, the developer has already spent time writing the code. That’s why we also are pro-active, and use mailing lists and IRC to inform the developers about things they should be aware of when coding. We also aim to have a call with another developer, before the code is written, so that there are at least two people thinking about the design.

Most of these discussions are made on public mailing lists, so that all the developers can see and be part of the discussion.

Thanks Bjorn!

This is a last call for user experience data with Launchpad’s “Mentoring” feature for bugs and blueprints.

We’ve seen only limited usage of this feature, and haven’t gotten much positive feedback on it. Right now, it’s dropped from our 3.0 redesign; but before 3.0 is finalized, we’re interested in hearing people’s experiences with mentoring.

If you’ve used the Mentoring feature, either as a mentor or a mentee, please leave your feedback as a comment here. We’re particularly interested in answers to these questions:

1. Would you say that Mentoring caused any difference in the rate of resolution between mentored and non-mentored bugs? (It could be a positive or a negative difference; both are interesting to us.)

2. For people who offered mentoring: in your experience, did you have responses on your mentoring offers? Were you successful in integrating the mentee’s changes?

3. Do you have any thoughts on how to improve the feature?

Please note: this is about Launchpad’s specific implementation of Mentoring, not about mentoring features in the abstract. We’re interested in your experiences with what’s actually been available in Launchpad. If a few people found the Mentoring feature useful, but most who tried it did not, then we will probably leave it out of Launchpad 3.0. On the other hand, if we hear something very unexpected from the feedback, that could affect our decision of course.

Our first Launchpad meet-up is happening on Monday the 28th September in London. Come along to meet some of Canonical’s Launchpad development team and other members of the Launchpad community.

Come drink a beer/coffee/mineral water with us and get your hands on a Launchpad t-shirt — if you arrive before we run out 🙂

If you came to the London release party for Ubuntu Jaunty you’ll know the venue, which is The Warwick just off Regent Street in London. Google Maps reckons it’s a three minute walk from Picadilly Circus Tube Station. We’ll be there from around 7.30pm, in the upstairs bar. You’ll be able to tell who we are from the snazzy Launchpad t-shirts we’ll be wearing 🙂

When: 7.30pm onwards, Monday 28th September 2009
Where: Upstairs at The Warwick, 1-3 Warwick Street, London, W1B 5LR
Nearest tube: Picadilly Circus

If you’re definitely coming, mail me with your t-shirt size — XS (female only) S, M, L, XL, XXL (male only).

Want an easy way to direct people to your pages in Launchpad? Whether it’s for yourself or your project, you can pop one of our new badges on your website:

You can choose badges from 160px wide to 250px and we even host the badge for you, so all you need is to copy and paste the image URL.

Take a look at our badge kit page for the legal details and also to see the image URLs.

You’ll need a PGP key registered in your Launchpad account if you want to use the bug tracker’s email interface, sign a Code of Conduct or upload a package.

There’s a guide in the help wiki and also now this screencast!

On the 28th September, the Canonical Launchpad team leads will be in London, along with myself, community guy Karl Fogel and UI guy Martin Albisetti.

I’m organising a Launchpad community meet-up for the evening of the 28th, which will be a chance to meet other Launchpad types, including several of Canonical’s Launchpad engineers.

We’re considering a couple of different venues at the moment but it’ll most likely be a pub in central London and fairly informal.

If you think you’ll be able to make it along, let me know. More details soon.

There used to be only one way of exporting translations from Launchpad—by requesting your files in the Launchpad UI and waiting for an email with the download URL.  It works, but it’s not very convenient if you’re trying to automate things.  It’s not that easy to make the request automatically, and then right in the middle of your script you also have to wait for the email, catch it, and parse it to get the file.

Now there’s another option: automatic exports to a Bazaar branch.  If you set up this option, Launchpad will regularly produce a snapshot of your translations and commit it to a Launchpad-hosted branch of your choice.  Now you can always find a reasonably fresh export of your translations in the same place, and download it automatically, without any asynchronous requests.

It also means that large numbers of developers or translators can get regular, fresh updates of the translation files.

How to use it

Let’s say you have a project called bzamqi. It’s set up for translation in Launchpad, and you want regular translation exports of the 1.0 release series.

The first thing you need is a branch.  It doesn’t have to be a branch for the bzamqi project, but you do have to be the owner.  (The branch can belong to a team, but then you do need to be a direct or indirect member of that team.)  The Launchpad help wiki describes various ways to create a branch:

• https://help.launchpad.net/Code/UploadingABranch
• https://help.launchpad.net/Code/TeamBranches
• https://help.launchpad.net/Code/PersonalBranches

Just remember that the branch must be Hosted, i.e. the master branch must be stored on Launchpad where we can commit to it.

Update: at the moment, team-owned branches don’t work. To work around this, make yourself the owner of the branch; set it as the translations branch; and then make the team the owner of the branch again.

Now, go to the page for bzamqi’s 1.0 release series.  You’ll be wanting the Translations tab.  Once there, pick the Settings option.

On the page that takes you to, near the bottom, there’s a section “Export translations to branch.”  You can set a branch there, and translations for the series will be exported to that branch.

If you want to disable the exports, go back (use the “pencil” icon next to the branch name to select a different branch) and select no branch at all.  Just clear the input field.  The settings page will go back to how it was before you selected a branch.

How it works

The main thing to grok is that this is not Launchpad “editing” text files for you.  The export brutally overwrites any previous versions of the files in your branch.  In the file that Launchpad writes, translations may have changed—but messages may also appear in a different order, or with different comments, and so on.

So do not expect files that you can merge back into the originals without any further work!  The Launchpad database and bzr have completely different views of this data.  There is a trick though; see the “Advanced” bit below.  Also, it will help to keep the messages in translation files in the same order in which they appear in the template.

Directory layout

In the branch, each translation file will have a normalized name (de.po for a German translation, zh_CN.po for Simplified Chinese, eo.po for Esperanto, and so on).  Each translation sits in the same directory where its template was on import.

If you have multiple templates, make sure that they have different directory paths, or the translation files for one will overwrite those for the other!  It was already recommended practice to give each template its own directory.

Templates are not exported.

Advanced: going two-way

It is possible to import translations from and export translations to the same branch.  If you push changes to a translation file into the branch, the export will not overwrite the changed file unless your updated file has already been copied onto the translations import queue.

Of course it’s still possible for the import to fail for whatever reason, and in that case your file will be overwritten.  This hopefully won’t hurt; the file that’s being exported is “better” in the sense that it shouldn’t cause any import errors.  But it may not always be what you want, so be careful if you do choose this setup.

(Your templates will not be touched, since the export doesn’t write them.)

Don’t use this “two-way” setup lightly, since it complicates things.  Complications are where accidents happen.  Either wait until there’s more practical experience with this, or make sure in some way that you won’t lose data if something goes horribly wrong.

A bit less adventurous but still useful is to have only your templates imported from a branch, and have your translations committed to the same branch.

Questions

How often do the exports happen?

For now, once a day.  We may tweak that later depending on how much this feature is used and how heavily it loads the servers.

If the export runs at a moment when you’ve pushed changes into the branch that Launchpad hasn’t processed yet, the export will not happen.  This is a deliberate protection against conflicting updates.  Unless you update your branch all the time or just at the wrong time of day, the next day’s export should catch up with the latest changes.

Why do I need to own the branch?

Otherwise you would be making Launchpad write data into someone else’s branch.  Not a nice thing to do to someone who isn’t expecting it!

If you want someone else to own the branch, you can change who owns it later, using the normal branch settings UI.  You only need to be the owner when you set it as the translations export branch.

What happens if I leave the project?

The exports keep happening to the same branch.  If that is not what you want, it’s up to the remaining project owners to choose a different branch or disable the exports.

William Grant was the first person outside Canonical to submit a patch to the newly open sourced Launchpad. I asked him a few questions to see how he got on.

Matthew: What did your patch change?

William: Launchpad users may elect to receive the bug’s description and status in the footer of each bug change notification email. This helps keep track of context when dealing with lots of different bugs.

Previously, bug emails that got to you through a team wouldn’t respect the setting — email coming through some teams would have the description, but for other teams it would be omitted. My fix ensures that the user’s selection is respected even for team emails.

Matthew: Why did you take the time to contribute to the Launchpad codebase?

William: I use Launchpad Bugs a lot, and get a lot of bug mail. This is an issue that has been irritating me for some time, and the fix seemed simple enough, so I decided it might be a good first task.

Matthew: How was it finding your way around the source tree?

William: It was easier than I expected. Most of the Bugs code now conveniently lives in lp.bugs, so I wasn’t overwhelmed with having to deal with the whole tree at first. The extensive test suite (particularly some good doctests) pointed me in the right direction, and developers and grep showed me the rest.

Matthew: How easy/hard was it to make your changes and stick to the Launchpad coding guidelines?

William: I didn’t get it quite right at first, but once pointed at the guidelines, it was simple to make my code compliant. It ended up looking a lot cleaner.

Matthew: What about the review process?

William: Remarkably easy. I found a reviewer (Graham Binns) on IRC, and submitted a merge proposal for my branch. A couple of hours and a few test changes later, my branch was approved. Graham landed it as soon as PQM reopened.

Matthew: What advice do you have for other people considering writing a patch for Launchpad?

William: Do it — it’s great to be able to fix those issues that have been bugging you for ages, and it’s not as hard as it looks. The test suite is a great guide through much of the codebase, and there are always developers on IRC to point you in the right direction if you get lost. The reviews are quick, developers are friendly, code style guidelines are clear, releases are frequent… all up it’s a very pleasant experience.

Matthew: Thanks William!

Last month we made a small change to the series page as a commitment to making a distinction between the driver for a project and the driver of a series. The drivers of a project have the power to make the decision of what features go into a release. But the driver of a series is special. Often a select number of individuals are delegated the awesome responsibility to define the intent of a series, manage all the milestones necessary to meet the goals, and to create the release. Series drivers are release managers.

When developers talk about the trusted persons who are making the release happen, they use the term release manager. While Launchpad recognised the role, it required the person also be a project owner, which is not always suitable for large projects. Release managers do not need the power to edit the project information, they need the power to edit the series information, create milestones, and release them. That is the power they now have.

A project owner can set a user, or a team as the series release manager from the series page. Project drivers also have the power to create a series; they have the power to start planning to be make themselves the release manger of the series they create.

What’s next

Distribution have release managers too, they have the same responsibilities as project release managers, but giving them power is a bit tricky. For distributions other than Ubuntu, release managers will be able to create series and milestones, and edit their details just as a project release manager.

Ubuntu must be handled as an exception; the gift of Soyuz and Translations is also a curse. Many special tasks must be performed before an Ubuntu series can be register a series in Launchpad. The release managers for an Ubuntu series will have more power to edit the series details and milestones.

Launchpad has supported assigning questions to users for several years, but the privilege was limited to project owners. This meant the feature was rarely used. Since the feature was also not visible, answer contacts often requested that we develop the feature. Question listing now include the assignee column. Answer contacts can assign a question to a user via the edit page. The assigned user will receive a notification about the assigned question. An assigned question will never expire; the assignee is obligated to answer the question.

The launchpad team had considered removing this feature two year ago because it was not popular. There were a few users who explained the need to assign questions to knowledgeable or privileged answer contacts. Simply put, the problem was not that assignment was unwanted, but that it could not be used by the people who needed the feature. The Launchpad team did not really understand how users were trying to use Launchpad until we decided to take turns answering every question asked to the launchpad project. We soon understood the need to assign questions to users. There are many questions that can only be answered by one or two people. The assignment must be visible to everyone, otherwise you would spend an hour reviewing open questions that were already assigned to someone. The cruelest part of assignment that that the assignee was never told that he has a task to complete.

This situation was especially frustrating for me because Answers is the application I started working on when I joined the Launchpad team. I knew that I could fix the issues in a few hours of work. Answers however, is not an application we are developing at the moment, I could not work on it during work hours. So I decided to fix the assignee feature on a Saturday. I could do this because the Launchpad on-call reviewer cannot easily say no to a merge request for a branch. I was also pretty certain my branch would be accepted because the reviewer is also answer contact who has experienced the assignee problems too.

The on-call reviewer rule is in place to ensure every patch is reviewed and given an opportunity to merge. This rule applies to everyone. Some Launchpad users have submitted patches for Launchpad pages and scripts and we have reviewed and merged them. Launchpad is now open source. You too can submit a branch knowing that someone must review it. Many of the Launchpad team members hack on Launchpad on our own time because we love Launchpad. Yet we still cannot fix every bug, or implement every feature. There are a lot of bugs that can be solved in a few hours work. If you want help to close a issue that you care about, we can help. Again the on-call reviewer is obligated try to advise on implementation issues. Also, Launchpad developers prefer to have pre-implementation plans to ensure that when someone decides to start a branch, it can be completed in less than two days and will be accepted by a reviewer. Someone on the #launchpad-dev channel on freenode can help.