launchpadblog

Last week, members of the Launchpad team got together with members of other teams across Canonical to sprint on lazr-js.  lazr-js is the library the Launchpad team created on top of the Yahoo User Interface 3.0 javascript library to provide a richer user experience.  For instance, the “widgets” in lazr-js are used in bug pages to help users change bug status and importance without leaving the bug page, and allowing users to subscribe to a branch without leaving the branch page.

One of the big things we focused on with lazr-js last week was integration into apps other than Launchpad.  We worked with members of the internal Canonical webapp teams, the Ubuntu One team, and the Landscape team to get lazr-js integrated into those apps, and find the issues that anyone would have while trying to integrate lazr-js into their own app.

As we focused on integration, we were able to see where we could make lazr-js more generic and easier to use.  We were able to find ways to automate the tests we have written for the code in lazr-js.  We also focused on fixing the bugs in lazr-js that would prevent Internet Explorer users from benefiting from lazr-js enhancements to an application, but we still have much to do in the way of cross-browser compatibility.  We’ll be working towards browser compatibility in the next few months, and welcome any contributions from the community.

The sprint was a huge success.  At the beginning of the week, lazr-js was a Launchpad library, and at the end, it became a library that can be used by anyone that wants to have a rich browser experience for their webapp.

A couple of weeks ago, the Launchpad team leads at Canonical gathered in Millbank Tower to talk about what we’ll be doing over the next six months. We talked with each other, we talked with Martin Pool from Bazaar, we talked with people on the Ubuntu Platform team, we talked with Mark Shuttleworth, we talked a lot.

Over the week, two very important things slowly began to dawn on us. I’ll talk about one of them now, and leave the other one to hang tantalizingly in the air like some forbidden fruit that’s learned how to hover.

The first important thing we realized is that Launchpad was originally conceived as a way of helping better connect the Ubuntu operating system to the upstream projects on which it depends. We further realized that could do that much better than we are right now.

A flood of bugs

Zillions of bugs get filed against Ubuntu every day. While some of them are introduced when the Ubuntu community packages software, many are really bugs in the underlying upstream code.^{[citation needed]} And quite often they’re already fixed in the latest upstream version — it’s just that the Ubuntu package doesn’t have the fix yet.

Yet even though Ubuntu is drowning in this sea of bugs, it can’t simply forward them upstream indiscriminately. Upstreams shouldn’t be bothered with old bugs; they only want to hear about bugs that are still in their code. And Ubuntu needs to know when such a bug has been found, both to tell users that a fix is coming and to help plan packaging updates.

Package of the day

Launchpad should be doing much more to help rescue Ubuntu from this deluge. With PPAs and source package branches, Launchpad ought to be able to make it really easy to create a packaged version of the tip of any upstream, to test against, and to file bugs and provide patches directly to that upstream. That is,
Launchpad needs to make Ubuntu Daily Builds rock.

That’s going to be our overall focus now. At the same time, we’re also aware that we need to spend time polishing what we already have. So, for this month and for UDS, we’re going to be focusing only on reducing technical debt, fixing OOPSes and cleaning up the UI.

Where to now?

The Canonical Launchpad team are going to be focused on “bridging the gap” between Ubuntu and its upstreams. We’ll focus on better, faster bug triage, on making it really easy to get upstream tip on the Ubuntu desktop and really tight translations integration between Ubuntu & its upstreams. Early next week, we’ll email out a high-level roadmap of where we want to go.

We are interested in getting real-user feedback about our solution to better integrating upstreams and Ubuntu developers. If you are an upstream or Ubuntu developer interested by that problem, please contact us.

PS. If you’ve read this far, you are probably wondering what the second Very Important Thing was. I’m afraid you’ll just have to wait.

Martin posts on his blog:

We’re trying to improve the icons we have in Launchpad so they’re more usable across different cultures and types of users, and our first step is to do some user testing on our current icons.

The Canonical User Experience team has set up a survey to gather information on how users see our icons, so if you have a few spare minutes (it’s very quick!), please take the survey and pass it on to other people, especially if they don’t use Launchpad, as they will be less biased.

Here’s the link: http://www.surveymonkey.com/s.aspx?sm=6iwthaIT4FwPCsMPa1EDEA_3d_3d

Launchpad’s Code Hosting service (code.launchpad.net and all Bazaar access) will be offline from 10:00am-10:30am UTC, Thursday, 1 October 2009. Sorry for the short notice; we need to do some unexpected hardware maintenance.

     Going offline: 10am UTC, 1 Oct 2009
     Expected back: 10:30am UTC, 1 Oct 2009

That’s 11am London (BST), 12 noon Western Europe, 2pm Moscow, 6pm Beijing, 8pm Sydney, 3am California, 6am New York.

The Soyuz team was very, very busy over the last year fixing lots of bugs and adding plenty of new features. These are the highlights that I’d like to mention!

New features

• Multiple PPAs per person — split your packages into different repositories without the hassle of creating new LP users.
• Karma for uploads! Distro and PPA uploaders (and the package creator if different) will be recognised for their work and get karma.
• Massively improved webservice APIs to control various operations, such as copying packages, manipulating builds, inspecting PPAs etc. Allows script-based control of many soyuz-related operations.
• Hugely faster build farm scanner, builds are dispatched a LOT quicker now. That means there’s less waiting around for your packages to get built.
• Private PPA subscriptions / tokens. People can now control who is able to download their PPA software.
• Package sets and their upload ACLs implemented for Ubuntu. Karmic and onwards will be using package sets for upload permissioning.
• Security in Soyuz. Complete support for the Canonical security team to use a private PPA and directly unembargo packages from it.

3.0 UI

We also did a lot of complete page redesigns for 3.0:

• PPA page split into two pages; one user-focused, one developer-focused.
• New build page, with a cleaner look.
• New global /builders page; sortable table data and a time-based estimate of the queue sizes.
• New distribution source package release page; much better presentation of data.
• New distribution source package page; makes more use of upstream package description/logo etc.
• Distro series source package release page gets a new layout.
• Builder +admin and builder +edit collapsed into a single page.
• New builder page.

But the work doesn’t stop here. We’re already thinking about 4.0!

Launchpad 3.0’s most noticeable feature is a new user interface. Martin Albisetti led the development of that new interface, so I asked him how the work had gone.

Matthew: What problem were you and the rest of the Launchpad team trying to solve with this UI redesign?

Martin: When I started working on Launchpad, I was presented with the challenge of bringing the project into the 2.0 era, where everything is editable inline, nice to use, fast and slick.

Mark Shuttleworth created a new team dedicated to making sure that Canonical shipped amazing user experiences in every single project, and Launchpad has been the pioneer project.

There were long-standing issues with the UI we wanted to solve, and there where many new features being worked on, the redesign was taken to make sure that the new and old worked well together. There where also a lot of infrastructure changes that needed to be made to enable us to continue developing in an Agile way and keep quality high, while we upped the bar in the experience our users had.

It’s been a great challenge, but with a team of super-stars like Launchpad it almost felt easy.

Matthew: Would you consider using Flash or other similar technologies in the Launchpad UI?

Martin: We did. Being an open source project sponsored by an open source company with an open source culture, proprietary tools are usually the last option we consider. We use Flash to create mockup movies of what we want the interaction and experience to look like, so developers can see it over and over again while they’re making javascript magic.

With browsers supporting new open technologies every single day, it didn’t make sense for us to force our users to install proprietary software to user our service. AJAX, and especially the canvas element allow us to do whatever we want. I can’t tell you enough about how grateful us web developers are for the healthy browser competition that has spring up the the last few years.

Matthew: How do you produce and maintain the templates used for Launchpad pages?

Martin: Well, it can happen in a few different ways. A common path would be:

1. Find out what the intention of that page is.
2. Research how users are currently using it (it’s amazing how people use pages in unexpected ways).
3. Create a low quality mockup of what the page could look like, something easy to change.
4. Throw it out in the public, the more feedback you can get, the less likely you will miss out on something important.
5. Hand it over to a developer, work with them on implementing it and solving problems you didn’t predict would come up.

Launchpad uses Zope, so our templates are Zope templates. Like any technology, it has it’s advantages and it’s limitations. Sometimes it limits us on what we can ship, but rarely significantly thanks to our incredibly creative developers.

Matthew: You introduced a system of UI reviews to the Launchpad project: how did that work? Has it been a success?

Martin: This to me has been the most significant change. We have 30 developers working at the same time delivering improvements and new features to our edge users on a daily basis, and it was hard to keep the interface consistent and clean.

UI reviews where an extension of an existing process, code reviews. Not one line of code was changed or added to the project without someone else looking at it first. So I started reviewing the UI for all changes before they landed, working together to change anything that would be inconsistent or hard to understand. This had an immediate impact. Developers shifted to thinking about UI before they started programming, had the opportunity to learn in the review process and ask questions.

We got to a stage where some people had understood this so well I started building a team of UI reviewers, people who where especially interested in user interfaces and had learned from the reviewing experience. Very quickly, more than half the UI changes where being reviewed by developers. There’s a graduation process where they can approve changes with no supervision, and I already have a few candidates close to graduation day!

A wonderful side effect of inverting the roles was that the developers improved their own work, fully knowing what was required to land a change. I feel it’s been the biggest success since I joined the company over a year ago.

Matthew: How do you teach an understanding of what makes good UI?

Martin: In the process of including more people in the UI review process (and eventually, we’re aiming at including everyone) there where many hours of conversations around changes, explaining why some things needed changing and figuring out together how to do it. These conversations slowly grew into a wiki page with tips and tricks, easy to follow and understand. All this together with experience is what I feel teaches understanding of good UI.

Matthew: Many of the UI changes have involved Ajax: what have been the biggest challenges in retrofitting the web application for Ajax and also for introducing Ajax to the team?

Martin: The first challenge was picking the right tools. The rich number of options out there made this a non-trivial task. Since Launchpad was spearheading many of the changes that will happen to the rest of the web applications in Canonical, it was an exhaustive process. In the end, we gambled on using YUI3, Yahoo’s Javascript library currently in beta but in early alpha back then. YUI2 had a great reputation, well tested with a vibrant community, and YUI3 was building a new library from the ground up with all the lessons they learned from previous versions. It was the right choice.

Testing infrastructure was also a big thing for us. We’re a quality-oriented company, so we weren’t about to get into hundreds of thousands of lines of Javascript and CSS without being able to have integration and regression tests. This seems to be an area that still needs maturing. Windmill seemed like the most reasonable choice. I hear their community has been very valuable in our adoption.

Once we had made the big decisions, all the team flew into London for two weeks of javascript training to kick off the work. It was a very fun sprint.

Retrofitting I feel was easy due to the way the code base had been developed. We had HTML fallbacks for every single page already available, so we just needed to tack on a click handler to a link and we where ready to go, allowing us to use what is commonly called “progressive enhancement”.

Matthew: How have coding standards changed as a result of the UI changes?

Martin: New processes were added to bring UI to the front, we standardized on a Javascript-specific style of coding, and “trivial” changes rarely became trivial.

Matthew: Some browsers are less well supported by the Launchpad interface when it comes to Ajax. Why is that?

Martin: YUI3 grades browsers based on how well they are supported, so we inherit those standards as a base. CSS comes into play as well, so based on our user base we adapt that list. MS Internet Explorer is usually the biggest headache for us and usually gets less support as it’s a small user base.

Matthew: Thanks Martin!

Bjorn Tillenius recently took the role of Launchpad Technical Architect. That, and the release of Launchpad 3.0, seemed like a good opportunity to learn a bit more about his new role.

Matthew: You’ve just taken a new role in the Launchpad team of Launchpad Technical Architect. That sounds like a big job: how is it going so far?

Bjorn: Yes, it’s indeed a big job. There are a lof of things that are in need of attention, so a big part of my work so far have been to talk to people, and get a list of the most pressing tasks written down. But I’ve also managed to do something concrete, like integrating our Windmill (a Javascript integration test framework) test suite into our zope.testing test runner, which we use for all our other tests.

Matthew: Is your role mostly concerned with the code of Launchpad or also the servers etc?

Bjorn: My role is mostly concerned with the code of Launchpad. My main objective is to make it easier to develop Launchpad, which includes designing and refactoring code to make it easier to use. I’m also concerned about ensuring that the quality of the code is high, and that we have good automated tests.

Matthew: What are your top three priorities for your new role?

Bjorn:

1. Make it quicker to develop new pieces of code.
2. Code reuse.
3. Make sure Launchpad code is scalable.

Matthew: What challenges does being a large Python/Zope application throw up?

Bjorn: Complexity. There are a lot of sub-systems that need to interact, and it’s hard to keep track of everything in the code base. If you work on one part of the system, you might not know that another part of the system has already solved part of your problem.

Another big challenge is that Python is big on readability. It should be easy to read and understand your code. This is great, except when you get more users, and realise that the beautiful code you wrote is really slow. Producing code that is both beautiful and fast is hard, and sometimes impossible. You have to make a lot of compromises.

Matthew: What coming technologies look as though they might be interesting for Launchpad?

Bjorn: We’re currently looking into using Grok and memcached.

Grok is interesting, since it allows us to more easily replace our ZCML code with Python code. This makes it easier to both write, and to understand, since ZCML adds another language the developer has to learn, and it adds an indirection which can be hard to follow.

Memcached is interesting, since we currently render a big part of the HTML page on-the-fly, which is quite expensive. Memcached allows us to cache part of the page, so that our pages will render faster.

Matthew: How do you ensure a consistent approach to development and architecture of Launchpad with a distributed team?

Bjorn: Being in a distributed team certainly makes things harder, when it comes to making sure everyone does the same thing. I’d say our biggest asset here is the code review that we require before any code can land in the mainline. With another pair of eyes looking at the code, the chance of spotting areas of improvement increases. Of course, spotting mistakes at this point isn’t ideal, since it’s quite late in the process, the developer has already spent time writing the code. That’s why we also are pro-active, and use mailing lists and IRC to inform the developers about things they should be aware of when coding. We also aim to have a call with another developer, before the code is written, so that there are at least two people thinking about the design.

Most of these discussions are made on public mailing lists, so that all the developers can see and be part of the discussion.

Thanks Bjorn!

This is a last call for user experience data with Launchpad’s “Mentoring” feature for bugs and blueprints.

We’ve seen only limited usage of this feature, and haven’t gotten much positive feedback on it. Right now, it’s dropped from our 3.0 redesign; but before 3.0 is finalized, we’re interested in hearing people’s experiences with mentoring.

If you’ve used the Mentoring feature, either as a mentor or a mentee, please leave your feedback as a comment here. We’re particularly interested in answers to these questions:

1. Would you say that Mentoring caused any difference in the rate of resolution between mentored and non-mentored bugs? (It could be a positive or a negative difference; both are interesting to us.)

2. For people who offered mentoring: in your experience, did you have responses on your mentoring offers? Were you successful in integrating the mentee’s changes?

3. Do you have any thoughts on how to improve the feature?

Please note: this is about Launchpad’s specific implementation of Mentoring, not about mentoring features in the abstract. We’re interested in your experiences with what’s actually been available in Launchpad. If a few people found the Mentoring feature useful, but most who tried it did not, then we will probably leave it out of Launchpad 3.0. On the other hand, if we hear something very unexpected from the feedback, that could affect our decision of course.

Our first Launchpad meet-up is happening on Monday the 28th September in London. Come along to meet some of Canonical’s Launchpad development team and other members of the Launchpad community.

Come drink a beer/coffee/mineral water with us and get your hands on a Launchpad t-shirt — if you arrive before we run out 🙂

If you came to the London release party for Ubuntu Jaunty you’ll know the venue, which is The Warwick just off Regent Street in London. Google Maps reckons it’s a three minute walk from Picadilly Circus Tube Station. We’ll be there from around 7.30pm, in the upstairs bar. You’ll be able to tell who we are from the snazzy Launchpad t-shirts we’ll be wearing 🙂

When: 7.30pm onwards, Monday 28th September 2009
Where: Upstairs at The Warwick, 1-3 Warwick Street, London, W1B 5LR
Nearest tube: Picadilly Circus

If you’re definitely coming, mail me with your t-shirt size — XS (female only) S, M, L, XL, XXL (male only).

Want an easy way to direct people to your pages in Launchpad? Whether it’s for yourself or your project, you can pop one of our new badges on your website:

You can choose badges from 160px wide to 250px and we even host the badge for you, so all you need is to copy and paste the image URL.

Take a look at our badge kit page for the legal details and also to see the image URLs.