Archive for the ‘General’ Category

Deployment reports are now public

Friday, September 30th, 2011

Steve Kowalik writes:

For a while now Launchpad has been using deployment reports that tell us what state our QA is in, which revisions are safe to deploy to production, or which revisions are not safe to deploy since they failed QA.

Some time ago, I started the process to make these reports public, and I’m proud to announce that today, they are!

If you’re waiting to see when your code in Launchpad is likely to go live, take a look at the new public deployment report.

Speeding up development

Friday, September 9th, 2011

SpeedometerToday we reached a significant milestone, we completed our first fast down-time deployment. Two obvious reasons for doing this were already mentioned in the announcement and our technical architect”s post describing the change:

  1. We’ll have less downtime per month (at the cost of more frequent but short interruption).
  2. We’ll be able to deploy fixes and changes involving DB schema more frequently.

But from my perspective, the most important benefit I think we’ll get from this is a speed up in our rate of development, particularly, in terms of completing feature projects. It’s not a secret, our feature squads spends a lot of time to complete their projects. There are multiple reasons for this, but in the end, there usually fall under two broad cateries: the time it takes to actually make the change, and the delays in getting feedback on the change itself.

To help with the first category, you’ll want better and more powerful libraries, better architecture, developers’ training, etc. Think the time difference between developping a database web application in Django vs as a CGI C application using only the standard C library. Launchpad isn’t using the most modern libraries and toolkits, and we could still make a lot of improvements there.  But the costs of making changes in this space are compounded by the problems of the other category.

Once you wrote the change, you are far from done. There are lots of hoops you still have to jump through before saying “done-done“: you’ll need to make sure the tests pass, to get your changes reviewed, merged, QAed and then deployed. And finally, you’ll probably want to make sure that it matches the user’s expectations, but until it’s in production, this is hard to assess reliably. All of these steps takes time and introduce delays, some bigger than others. The Launchpad team is always on the look-out to cut in these delays and the new “fast down-time deployments” cut on of the biggest one we had.

Cycle time distribution

Since a picture is worth at least a thousand words, have a look at the chart above to have a better idea of what I’m talking about. It shows the distribution of the time it takes to complete a “change”. (What this plots is the cycle time from coding to deployment of our Kanban cards which roughly map to one logical change.) You’ll see that 50% of our changes are deployed to production in about a week. And the next 45% takes between 1 and 5 weeks. Now, our feature projects are composed of many many of these smaller changes. If those are all  relatively small changes, why do they take so long?

One of the big bottleneck was the batch size of our DB deployment. If a change required a DB schema it waited until the next downtime deployment which happened once every month. In theory, that means that on average a change involving the database would wait 2 weeks in the queue before deployment. In practice, it’s more complex than that, because squad leads would often plan around these. So a database change would be hold off onto because it was deemed that it couldn’t be safely completely to be part of the next downtime deployment. So it might be put on hold in favour of other work, and delayed to the next downtime deployment. It’s also frequent to have other changes building on the first also queued up waiting for the next deployment window. Add on top of that, that it’s common for the completion of a feature to require several iterations of DB change based on feedback and you quickly understand how you can be working months on a feature project!

But this major bottleneck is now gone! We’ll be able to land and deploy DB changes reliably within days, giving us much more rapid feedback. I’m looking forward to the change in the cycle time distribution in the coming months. The whole distribution should move toward the left. I’ll write a follow-up in two months to see if this prediction comes true.
Photo by Nathan E Photography. Licence: CC BY 2.0.



Matthew Revell is the new Launchpad Product Manager

Friday, August 26th, 2011

Matt RevellIt’s my pleasure to announce that as of today, Matthew Revell is the new
Launchpad Product Manager replacing Jonathan Lange.

We were seduced by his bold vision for Launchpad along with his data-centric approach that he intends to bring to the role. He also has an already extensive experience interacting with Launchpad developers and users. If you read this blog, you probably read something written by him! Or you might have interacted with him in one of the many user-research sessions he ran. The introduction of user-research helped us release better designed feature. Building on this experience, we hope that his leadership will bring Launchpad to the next level.

Matt will communicate more about his plans for Launchpad shortly. In the
mean time, let’s give him our warmest congratulations!



Users can now move bugs between projects and distros

Wednesday, August 17th, 2011

Users can use the affects form on the bug page to change which project or distribution the bug affects. You can also select the affected package. Lp API users can assign a project, distribution, or package to the BugTask target property to change the affected bug target. The behaviour is similar to the way questions can be retargeted between projects and distributions. Affected series cannot be changed, though the affected series package can be.

Retargeting a bug to a distribution, package, or project

Retargeting a bug to a distribution, package, or project

Previously, users had to mark a bug affecting a project or distribution as invalid, then add a new affected project or distribution. This cluttered the UI, caused excessive emails, and made pages slower.

How the Launchpad project does code review

Tuesday, July 19th, 2011

Not long back, Sumana Harihareswara, who is Volunteer Development Coordinator at the Wikimedia Foundation, contacted some members of the Launchpad community to ask how we handle code review.

She’s written a summary of her research for the Wikimedia community.

It makes interesting reading for someone, such as me, who’s close to the process. If you’re interested in contributing to Launchpad, Sumana’s report is a really useful overview of how we review and deploy code.

Echoes from the Dublin Thunderdome

Friday, July 15th, 2011

ThunderdomeI can’t believe it’s been two weeks already! From June 25th to July 1st, the whole Launchpad team was gathered in Dublin for our semi-annual all-hands event. Like usual, we also invited the Bazaar team and  one of our friendly sysadmin­.

The theme this time around was “UI, UI, UI”. Since the January squad reorganisation, it became evident that a big stumbling block for squads working on features was UI work. Our team is still inexperienced with JavaScript and the YUI3 framework. So each squad has encountered similar problems and sometime found different solutions. We wanted to take advantage of the face-time to come to a common understanding of the best patterns to use.

In the end, it was a week of intense hacking on UI infrastructure, with part of the mornings dedicated on presentation where people shared what they learnt in the past 6 months.

Projects that were worked on during the week:

  • An asynchronous notification system (extracted from Landscape) which will allow us to update pages as soon as tasks complete on the backend.
  • We ditched our buggy Windmill-based Javascript integration tests, in favour of a yuitest-based suite. We already use yuitest for JS unit tests, but now we’ll be able to use it  for integration tests where  we need to make XHR requests to a live server. We’ll even be able to set server-side fixtures from within the test!
  • Investigate the use of selenium2 for acceptance testing. These are tests ensuring a complete workflow works and that ideally will be able to run against both the staging server as well as within tree for regression testing.
  • Integrating loggerhead-backed branch data on some Launchpad pages. You should soon be able to get the diff of a particular revision directly on the merge proposal page.
  • Infrastructure to refresh all the client-side representation of objects related to a page in one request.
  • Many UI bugs were fixed.

This was also the week we said farewell to old friend: Jonathan our Product Strategist  and Ursula Junque who is now working as QA analyst for the Ubuntu Server team. In the end, Jono didn’t get pied as we still had a huge number of Critical bugs left open. We did rejoice in achieving our performance improvements target. Given the trend, I estimated that we have still 6 months before emptying the Critical bugs list. The new target is to reduce to a 100 for October. Let’s see if I was more realistic this time around.

And we also took the traditional team picture.

Launchpad Team Photo

Great fun it was, and I can’t wait for the next one in January.

Photo by Miss_Colleen. Licence: CC BY-NC-SA 2.0.


Thursday, July 14th, 2011

Bags of Tilda riceVisit this URL:

Cool, eh?

Same works for, as well as code, translations, answers and blueprint.

Actually, it’ll work anywhere that you’d normally put /people/+me or your own Launchpad id — so long as you’re logged in.

For example: visit to see a list of all your team memberships.

Thanks to Martin Pool.

Photo based on an original by Michael Francis McCarthy. Licence: CC BY 2.0

How to tell who has what role

Wednesday, July 13th, 2011

So, you’re about to select someone in Launchpad — maybe you’re assigning a bug — and you’re not exactly sure you’ve got the right person. One way to confirm would be for Launchpad to tell you what relation that person has to the project you’re currently dealing with.

Sound like something that might help you?

If you’ve got 25 minutes today or tomorrow, you can help us choose the best way to show those affiliations. Sign up to let us know if you’re interested. There may even be an Amazon voucher in it for you 🙂

In for the long poll

Monday, July 11th, 2011

A long pollLaunchpad has quite a few background tasks that update things outside of your browser request, and the only way you know that they finished is to reload a page to see if it changes.

Updating the preview diff on a merge proposal is one example.

At our Launchpad sprint in Dublin a couple of weeks ago, we added a so-called “long poll” where the browser opens a long-running connection using XHR (a background Javascript connection) to a new piece of infrastructure called the “long poll server”

This server is something that the Landscape team originally wrote and we’ve now open-sourced it so we can use it in Launchpad. The server connects to a RabbitMQ instance and relays information back to the browser when the background task completes so that the user doesn’t need to keep reloading the page – the page will automatically update thanks to the magic of Javascript.

The proof-of-concept is done but we now need to make it work in our production environment, which means making it more reliable, configuring the deployment and add some stats collection so we can see how it behaves. We’ll start off by only letting a small subset of people use it and grow it over time until we consider it stable enough for all users.

Photo by Jay Gooby. Licence: CC BY 2.0

Consistent display of tangential links

Friday, June 24th, 2011

On Launchpad we have a lot of links that take your focus away from the current task. Usually, these exist to let you get more information on Launchpad or take you off Launchpad to do something like look at upstream information.

These links are usually represented one of two ways–a blue link with a globe for something that takes you to a new website, or a blue or green link for an internal link that opens a new window.

There’s a problem with this though. Green links mean an action will not take you away from your current view. Using a green link to take you to another window is a contradiction in our UI. Blue links alone aren’t ideal either, because blue links are scary. Everyone’s been bitten by clicking a link while in the process of filling out a form and losing the data they’ve spend five minutes filling out.

Right now, on our enhanced person-pickers, we have a green “Details…” link to let you see more information about a person. This link opens in a new tab, and we need to come up with a better way to indicate that action. We’ve put together a survey showing some rough ideas of how we might indicate a link that opens in a new window. It’s a very short survey, and we would love to get your input.

Please, take the “link opens a new window” survey.