Roundup of the first Launchpad Performance Week
Published by Matthew Revell February 26, 2009 in Launchpad Performance Week
Back at the beginning of February, we ran our first Launchpad Performance Week.
Our aim was to find pages that, although they weren’t slow enough to trigger our internal warning, were nonetheless taking too long load. The first thing we did was to get our internal warning system to generate a report when a page takes 25 seconds to load, rather than 30 seconds.
So, armed with a list of sloth-like pages, each part of the Launchpad team went away with one aim: to fix whatever was preventing those pages from loading in a reasonable time.
Those fixed pages have been around on Launchpad’s edge environment for some time now and hit production with our 2.2.2 release.
So, let’s see what the team have been working on.
Faster bug pages
As a bug in Launchpad gets more attention — whether you measure that by the number of comments or subscribers — it takes more work to display it. That has led to particularly popular bugs, including the oft-linked-to bug #1, taking too long to load.
Two of the big pain points, on bug pages, were comments and the box listing the bug’s subscribers. For most bugs, they present no problem. However, bugs with many comments/subscribers were nudging Launchpad’s hard timeout limit.
Gavin tackled the slow loading of multiple comments by changing the bug page to display only 80 comments. Where the 81st comment would be there’s now a link to view the rest of the comments or post a new comment.
Tom‘s approach to the subscriber list was to load it asynchronously, preventing it from acting as a bottle-neck to loading the rest of the page. So far, there’s very little wait between the rest of the page loading and the subscriber list appearing.
Personal Package Archive overview pages
The Soyuz team worked on PPA overview pages. Take a look at the Exaile development team’s PPA as an example.
Two hold-ups were caused on this page, particularly for PPAs with many packages: the disk usage information and the detail for each package. Both are now loaded asynchronously, the package detail only being loaded when you click its header.
There’s more detail from Julian in his blog post.
Translation pages
The translations pages are some of the most data-intesnive in all of Launchpad. Danilo talks about the tranlsations team’s Launchpad Performance Week work in his blog post.
Code project cloud
Launchpad Code developer, Jonathan, wrote on his own blog about his work on the project cloud page:
The Launchpad Code home page is now a little bit nicer and a little bit faster.
For a while now, the page has had a tag cloud showing some projects that host branches on Launchpad. The tag cloud is drawn based on these two rules:
- the more branches, the bigger the tag
- the more recent the last commit, the less faded the tag.
Up until recently, the home page only showed some projects starting with ‘A’ — which is a bit silly. My recent patch changed the query to show the most branch-y projects on Launchpad. It looks a lot nicer, IMO.
We also got rid of the green / blue distinction. No one knew what it meant and it made the page more visually muddled. There are still some tweaks we can make, but I think this is definitely an improvement.
And elsewhere
We made a number of smaller changes right across Launchpad, as part of the Performance Week. We’re also working on introducing much more AJAX into Launchpad’s web interface, which should make the experience of using Launchpad feel much quicker.
We’ll be holding another Launchpad Performance Week in April, this time cutting our definition of slow page by a further five seconds. Again, we’ll be posting details of what we did right here.
New privacy features for commercial subscribers
Published by Matthew Revell February 25, 2009 in General
Launchpad is set up to make it easy for groups to come together spontaneously around tasks big or small. Take this bug as an example: it affects three projects that share the same code. Launchpad brings them together to find a fix.
But there are times when collaboration needs to be done privately. It could be because of security concerns, or confidentiality obligations to a client, or to coordinate the timing of events for marketing purposes, etc.
Launchpad already enables individual bugs to be marked as private but, until now, that’s been all. However, we’re running an interesting beta program at the moment that allows projects to default to both private bugs and private branches. As this will be of most interest to projects who aren’t ready to share their code, we’ll offer it by commercial subscription.
We’ve already been offering basic commercial subscriptions for a few months now. While Launchpad is always free of charge for free software projects, it also hosts some projects with non-free licenses, and charges them a fee to offset the costs of running the site. Until now, these commercial subscriptions have just meant access to Launchpad with no extra features. But following our beta of private branches and private bugs, we hope to offer a wider range of features to commercial subscribers, as they are likely to have different privacy needs from free software projects. If you’re a commercial subscriber and want to take part in the beta, or if you’re considering a commercial subscription, please contact us.
Of course, commercial subscriptions don’t take anything away from the free software projects that use Launchpad. They remain the focus of our development work, particularly as we move to open sourcing Launchpad itself.
Meet Tim Penhey
Published by Matthew Revell in Meet the devs
We’re back in New Zealand for today’s Launchpad developer interview with Tim Penhey.
Matthew: What do you do on the Launchpad team?
Tim: I lead the team that works on the Bazaar code-hosting side of Launchpad. This includes all the pages that are under the “Code” tab, the linking of branches to other parts of the Launchpad system, the services and scripts behind the scenes that allow the hosting of branches, and the branch browser (Loggerhead).
Matthew: Can we see something in Launchpad that you’ve worked on?
Tim: I’ve touched probably every page visible under the “Code” tab. A large amount of work is going into the work flows around code reviews, and that is where I’ve been spending a lot of time recently.
Matthew: Where do you work?
Tim: I work from home, and home for me is currently Dunedin, New Zealand. I turned one of the bedrooms in the house into an office, which contains my desk, a nice comfortable leather chair, and too much computer equipment.
Matthew: What can you see from your office window?
Tim: Our deck, and beyond that our back yard which is mostly grass with a trampoline for the girls and part of the garden which currently has several pumpkin plants doing extremely well.
Matthew: What did you do before working at Canonical?
Tim: I worked as a contract developer in London, UK. I primarily worked at investment banks working on their back office risk systems. I had worked for Lehman Brothers (RIP), Royal Bank of Scotland, Barclays Capital, and Cantor Fitzgerald. Prior to that I had a stint at Reuters working on their stock exchange connectivity software.
Matthew: How did you get into free software?
Tim: Sometimes I wonder if I am “into” free software. As far as using free software, I’ve had an aversion to paying for Microsoft software when perfectly good free alternatives were around for a while. I toyed with Linux a couple of times back with Red Had 7.1, and some SUSE version shortly after that, but I didn’t make the switch until I came across the Hoary Hedgehog release candidate. I’ve been using Ubuntu since then.
I’ve been meaning to get into actually contributing to free software for so long. Every time I looked into it I got put off by the massive leaps into a group’s culture and code base. I use KDE and have it as my main desktop almost exclusively, and I’d love to contribute to KDE, but it seems that there are always the “KDE” or “QT” ways of doing things. I had a background in C++, but they don’t use what I’d consider “pure” C++, so there is always a huge learning curve, and I don’t have the time right now to invest in that. I have a three girls, 8, 6, and 3 and a half (don’t forget the half), and they take up a lot of my non-work time. I’ve had the opportunity to contribute a few small patches to Bazaar, and I’d still like to contribute to other projects.
Matthew: What’s more important? Principle or pragmatism?
Tim: That’s like saying: What is more important? Food or water? They are both important. Particularly when developing code if you don’t have one tempered
with the other you end up unreadable spaghetti or gold plated code. I lean towards more principle as it tends to give cleaner code, but sometimes pragmatism wins out to actually get code out and useful. Code that isn’t used might as well not be written.
Matthew: Do you/have you contribute(d) to any free software projects?
Tim: I’ve got a couple of small patches into Bazaar. I use aliases a lot with bash, and also with Bazaar, but didn’t like that I had to edit the configuration file to add aliases, so scratching my own itch, I added the ‘alias’ command to Bazaar. I’ve also done some work on the PQM project, but most of my work is on Launchpad.
Matthew: Tell us something really cool about Launchpad that not enough people know about.
Tim: Code reviews. We are actively working to make them better, but the coolest thing about code reviews in Launchpad, is that you can do them entirely through email. Not only can you go to the web page and add a comment, and that comment gets emailed out to all interested parties, but you can reply to the emails using your mail client, and those replies are parsed, added to the review, and emailed out to the interested parties just as if you had entered them through the web interface.
A lot of work has also gone in recently to expose the code review functionality through launchpadlib as well. This will allow people to integrate Launchpad’s review functionality into other tools.
Matthew: Is the film Eagle v Shark an accurate picture of life in New Zealand?
Tim: To be honest, I have no idea as I haven’t seen the film. It has been on my list of something to get on DVD at some stage, but I don’t have it yet.
Matthew: Okay, Kiko‘s special question! You’re at your computer, you reach for your wallet: what are you most likely to be doing?
Tim: Getting my credit card to pay for books from Amazon. Books are very expensive in NZ. On my coffee scale, it is almost twice as expensive as the UK. Three coffees in the UK for a paperback, whereas it is closer to five or six coffees in NZ.
Matthew: Thanks Tim!
Launchpad offline 00.00 – 01.00 UTC 26th February
Published by Matthew Revell February 24, 2009 in Notifications
Launchpad will be offline for one hour from 00.00 UTC on Thursday the 26th February while we roll-out the code for our latest release, Launchpad 2.2.2.
Going offline: 00.00 UTC 26th February 2009
Expected back: 01.00 UTC 26th February 2009
We’ll post details of the release to this blog!
Meet Brad Crittenden
Published by Matthew Revell in Meet the devs
Today it’s Brad Crittenden‘s turn in Meet the devs.
Matthew: What do you do on the Launchpad team?
Brad: I work on the Registry team. We handle everything to do with people and teams, projects, series, milestones, and user communications, such as mailing lists.
Matthew: Can we see something in Launchpad that you’ve worked on?
Brad: I do a lot of behind-the-scenes work. The most prominent new feature I did, with my colleagues Edwin Grubbs and Elliot Murphy, was implement the Atom feeds. Trivially, I also set up the launchpadbugs account on Twitter which a lot of folks have found very handy.
Matthew: Where do you work?
Brad: I work from my home near Chapel Hill, North Carolina.
Matthew: What can you see from your office window?
Brad: I have a nice office on the second floor of my house and have my desk in front of a large window looking out onto the woods. I rigged up a pulley system so I have a large bird feeder in a tree about 25 feet off the ground where I get a lot of gold finches and other small birds. Lately a pileated woodpecker has been visiting almost daily. A while back I was surprised to see this guy lurking near the feeder.
Matthew: What did you do before working at Canonical?
Brad: I worked for a company that made a cryptography acceleration chip for doing crazy fast random number generation and making SSL and IPSec negotiations and bulk encryption speedy. I spent a lot of my time deep in the OpenSSL code, which is a dark, twisty place.
Matthew: How did you get into free software?
Brad: All of my previous work has depended on Linux, open source libraries, and the GNU toolchain so I knew a lot about it and was able to move into the world quite easily when I started with Canonical.
Matthew: What’s more important? Principle or pragmatism?
Brad: It’s not an either/or question for me. You’ve got to get the job done (pragmatism) but within a reasonable framework that will best serve yourself and your users in the long run (principle). In some situations the stakes are so high you can’t afford to compromise on the principles even slightly.
Matthew: Do you/have you contribute(d) to any free software projects?
Brad: I’ve contributed through the years to various projects I use by sending in code patches, filing bugs, and doing some documentation work. I have not yet gotten directly involved in any major projects but hope to do so soon.
Matthew: Tell us something really cool about Launchpad that not enough people know about.
Brad: Launchpad was built to support the development of Ubuntu and the open source ecosystem — everyone knows that. What a lot of people don’t know is that commercial or closed source projects can also use Launchpad. Since they aren’t directly giving back to the open source world we do ask they contribute to our operating costs by purchasing an annual commercial-use subscription. People can email me if they have questions or need help getting started.
Matthew: Which lens would you save from a house fire?
Brad: Of all of my lenses my favorite is actually the least expensive — a Nikkor 50mm/f1.4 — it’s incredibly sharp and produces an amazing bokeh. But in the event of a fire I’d ignore the equipment, it’s all insured and replaceable, and grab my 2Tb RAID drive that has my photos. (This reminds me I need to flesh out my off-site backup solution…)
Matthew: Okay, Kiko‘s special question! You’re at your computer, you reach for your wallet: what are you most likely to be doing?
Paying to run our test suite on ec2, of course!
Matthew: Ah, of course 🙂 Thanks Brad!
Meet Michael Hudson
Published by Matthew Revell February 23, 2009 in Meet the devs
Michael Hudson is the latest member of the Launchpad team to face a friendly grilling.
Matthew: What do you do on the Launchpad team?
Michael: I work on the Launchpad Code team, which means I work on the side of Launchpad to do with hosting, finding, viewing, importing and reviewing source code. What I do personally seems to mostly involve backend stuff — I did a lot of work on the infrastructure around the code import system that allows importing CVS and Subversion branches to Bazaar, and if you’ve used stacked branches on Launchpad, you’ll have used some of my code.
I also help maintain Loggerhead, the web viewer for Bazaar branches, which we run on Launchpad but also try to keep easy to install and use for every user of Bazaar.
Matthew: Can we see something in Launchpad that you’ve worked on?
Michael: Actually seeing something I’ve done is a little hard, for reasons I’ve explained above 🙂 Loggerhead is a counter-example, and if you’ve used codehosting in any way you will have “seen” my code 🙂
Matthew: Where do you work?
Michael: Palmerston North, New Zealand.
Matthew: What can you see from your office window?
Michael: The back of the garage. It’s not a very exciting view. I can see the Tararua mountains from the end of my drive on a clear day though.
Matthew: What did you do before working at Canonical?
Michael: I worked for the Heinrich Heine Universität Düsseldorf as part of the EU project on PyPy, the Python implementation of Python.
Matthew: How did you get into free software?
Michael: A fairly standard way, I think… I was always interested in computersand programming as a kid, and then when I got to uni and found the internet and all this activity, I also found that bugs were pretty easy both to find and (sometimes) to fix 🙂 After getting over the nervousness of sending my first patch off and having it gratefully received, it just grew from there.
Matthew: What’s more important? Principle or pragmatism?
Michael: Well… probably pragmatism, but… way back in 1999 Eric Raymond said “I want to live in a world where software doesn’t suck”, and for all that I don’t agree with much else ESR says, I think this is a good rallying cry. But I also think the free software approach has the best chance of producing software that doesn’t suck, so I don’t know that the question is really fair 🙂
Matthew: Do you/have you contribute(d) to any free software projects?
Michael: I was pretty heavily involved in CPython development from around 2000 to 2004, where I did a bit of everything, reviewing and applying patches,
fixing bugs, keeping an eye on the state of trunk, being a release manager, editing bits of the website etc.
These days, aside from maintaining Loggerhead on work time, I contribute the occasional fix to Bazaar and try to do contribute a bit to Twisted — usually code reviews of late.
Matthew: Tell us something really cool about Launchpad that not enough people know about.
Michael: I think the display of unmerged revisions on the branch merge proposal page is pretty cool — it gives you a feel for what’s been going on in the branch that you’re about to review.
Matthew: What’s been most unexpected about your move from the UK to New Zealand?
Michael: Superlatives like “most” always scare me, so here’s two answers:
1) I was surprised by how much of New Zealand has been affected by human occupation, mostly farming and forestry, but also
2) I knew New Zealand had many famously beautiful sights, but I’ve been surprised by how much beauty there is to be found down random side roads. Kaitoke Park, Mount Damper falls, the Mount Messenger pass, Whanganui Bay — these aren’t famous places, but they’re almost as amazing as the places that are.
Matthew: Okay, Kiko’s special question! You’re at your computer, you reach for your wallet: what are you most likely to be doing?
Michael: Trying to find the card I need to get past my bank’s challenge-response log in system.
Matthew: Thanks Michael!
Translation import notifications for Ubuntu
Published by Данило Шеган in Coming changes, Notifications, Translations
A few days ago a fix for bug #286359 landed: provide translation import notifications to Ubuntu packagers. This means that all packagers will start getting emails about translation files (basically, *.pot and *.po files) from their packages as soon as they are imported (or the import fails).
This should help packagers detect and fix l10n-related problems more easily, since they’ll have most of the debugging information they need right inside their inboxes. If you want to filter these emails, look for a sender of rosetta@launchpad.net.
The grey area of where to send notifications for automatically synced package uploads from Debian remains. I am planning on solving that with a public mailing list where anyone interested can go in and check for problems, and also fix them if they want to. Alternative is to rely on the recent improvement by Jeroen which keeps any failure messages inside the import queue itself, and simply drop all of these messages altogether.
Why Launchpad for Mailman?
Published by barry February 20, 2009 in General
Over the last 18 months, I’ve moved GNU Mailman development from SourceForge to Launchpad. The reasons are varied.
Mailman was one of the first projects hosted on SourceForge ages ago. I think our project id is a pride-inducing low 103, and we were even highlighted as its Project of the Month at one point. Of course SF itself uses Mailman to serve its own mailing lists. While the SF guys have always been great (including providing assistance during the migration to LP), I just became increasingly roadblocked by it.
The first major motivation for moving was Bazaar. This is of course the GPL’d distributed version control system developed by Canonical and used for code hosting on Launchpad. Having come from decades of SCCS/RCS/CVS/SVN use, distributed version control systems in general and Bazaar in particular have been an enlightenment on the order of learning Python. I mean, who’d have thunk a version control system could be fun?
After we moved code hosting to Bazaar on LP, evaluating the other benefits of Launchpad became easier. Truth be told, there was (and still is) some resistance in the community to moving to LP because Mailman is a GNU project but LP is not open source. That’s being fixed. The next service to migrate was the tracker, and thanks to the excellent assistance of my colleague Graham Binns, we were able to migrate the SF issues to LP. For years, even before I joined Canonical, I let the Mailman tracker languish because I found it so difficult to use. The simplicity and power of Launchpad’s tracker really shines for me here, especially with its ability to link across projects and artifacts (e.g. branches linked to bugs).
The next major service to consider is translations. While being one of the first Python applicatons to be internationalized and translated, we really have a pretty crufty process for updating translations. Some language champions have commit privileges, but others have to email or upload po-file diffs, or even entire po-files. This really sucks for a number of reasons, probably most of which is that the code developers are too tightly coupled with the translators. Our releases are held up for quite a while as we gather updates to more than two dozen languages.
Currently Mailman has an experimental Pootle service. Despite the gallant efforts of its maintainer, I don’t think it’s working out too well, mostly because it’s not well known and not well integrated with the rest of Mailman’s processes. Again, here’s where Launchpad integration would really shine for us, but until Launchpad is open sourced we won’t be moving translations here.
Meet Martin Albisetti
Published by Matthew Revell in Meet the devs
Next in the chair for our Meet the devs series is Martin Albisetti, also know as beuno.
Matthew: What do you do on the Launchpad team?
Martin: I’m in charge of the user interface, making sure that new (and existing!) features and pages are easy to use and slick. I have been on board for about six months, so I haven’t managed to fully sink my teeth into everything yet, but we’re getting there, and the next few months will be an amazing succession of roll outs of AJAX enhancements everywhere, which should improve the overall experience dramatically.
Matthew: Can we see something in Launchpad that you’ve worked on?
Martin: Well, if you use Launchpad for a few minutes, you will inevitably encouter pages that have gone through me before landing. I don’t get to actually do much coding, although I have recently done a few improvements to the Launchpad home page, and gave shape to the in-line text editing that’s already in the wild (although it was a team work effort with Mark).
Matthew: Where do you work?
Martin: From my office in sunny Buenos Aires.
Matthew: What can you see from your office window?
Martin: A nice garden with very green grass, flowers and palm trees.
Matthew: What did you do before working at Canonical?
Martin: I have a web development company (Pentacorp), so I’ve been doing almost everything related to web development (php, mysql, design, server admin) for the past 6 or 7 years. The web fascinates me 🙂
Matthew: How did you get into free software?
Martin: Free software just had better tools for web development, and was a much nicer and saner enviroment to work in, so I started moving over until one day I woke up with no proprietary software on my laptop.
Matthew: What’s more important? Principle or pragmatism?
Martin: Oh, you and your hard questions… I’m inclined towards pragmatism, I like to get things done more than I like being right!
Matthew: Do you/have you contribute(d) to any free software projects?
Martin: I have, it’s almost impossible not to once you start using it. I’ve done some work on Bazaar, I’ve been co-maintaining Loggerhead for 8 or 9 months, done some bzr-gtk, and have spread around patches on dozen of projects out there. You have to love the “scratch your own itches” properties of free software.
Matthew: Tell us something really cool about Launchpad that not enough people know about.
Martin: Launchpad developers really care deeply about users and user experience, I was amazed as to how much when I joined Canonical.
Also, merge proposals for code branches. They’re like the best thing ever.
Matthew: Is Argentine steak really the best in the world?
Martin: Yes, and probably in the known universe.
Matthew: Okay, Kiko‘s special question! You’re at your computer, you reach for your wallet: what are you most likely to be doing?
Martin: Moving it to my front pocket, my leg gets numb if I sit down for too long with my wallet in my back pocket.
Matthew: Thanks very much Martin!
AJAX in Launchpad
Published by Martin Albisetti in Coming changes, Cool new stuff
After a few months of working on all the infrastructure changes needed to start deploying AJAX on Launchpad, we are now ready to start developing the mockups we’ve been working on for the past months in the User Experience team.
To make sure we coordinated the work of 35 distributed developers, and really get the ball rolling 10 of us gathered for a sprint in Berlin and went into full ninja-hacking mode for a week, while defining standards and best practices as we went along.
The outcome of the sprint has been amazing (it’s actually still going on!), and we’re very well prepared to roll out all kind of AJAX goodies like “in-line status editing”, “multi-line editing” and “person pickers” in the next few months. It’s going to be awesome 😉
We have decided on YUI3 as our javascript library, and while it is still not a final version, we are very happy with our decision and it has allowed us to do some really cutting edge work, while maintaining clean and re-usable code.
I’m also very excited about in-line text editing having landed last week in Launchpad’s edge version (used by beta testers), enabling in-line editing of bug titles and project names, which landed thanks to the amazing work of Māris Fogels and Francis Lacoste. For all you non-beta testers, you will see it rolled out late next week.
Since it got rolled out, every time I edit a bug title I get the greatest feeling when presented with this:
A lot more coming soon, so stay tuned!
