Roundup of the first Launchpad Performance Week

Back at the beginning of February, we ran our first Launchpad Performance Week.

Our aim was to find pages that, although they weren’t slow enough to trigger our internal warning, were nonetheless taking too long load. The first thing we did was to get our internal warning system to generate a report when a page takes 25 seconds to load, rather than 30 seconds.

So, armed with a list of sloth-like pages, each part of the Launchpad team went away with one aim: to fix whatever was preventing those pages from loading in a reasonable time.

Those fixed pages have been around on Launchpad’s edge environment for some time now and hit production with our 2.2.2 release.

So, let’s see what the team have been working on.

Faster bug pages

As a bug in Launchpad gets more attention — whether you measure that by the number of comments or subscribers — it takes more work to display it. That has led to particularly popular bugs, including the oft-linked-to bug #1, taking too long to load.

Two of the big pain points, on bug pages, were comments and the box listing the bug’s subscribers. For most bugs, they present no problem. However, bugs with many comments/subscribers were nudging Launchpad’s hard timeout limit.

Gavin tackled the slow loading of multiple comments by changing the bug page to display only 80 comments. Where the 81st comment would be there’s now a link to view the rest of the comments or post a new comment.

Launchpad now asks if you want to Show more than 80 comments

Tom‘s approach to the subscriber list was to load it asynchronously, preventing it from acting as a bottle-neck to loading the rest of the page. So far, there’s very little wait between the rest of the page loading and the subscriber list appearing.

Personal Package Archive overview pages

The Soyuz team worked on PPA overview pages. Take a look at the Exaile development team’s PPA as an example.

Two hold-ups were caused on this page, particularly for PPAs with many packages: the disk usage information and the detail for each package. Both are now loaded asynchronously, the package detail only being loaded when you click its header.

The PPA storage information is now fetched asynchronously

There’s more detail from Julian in his blog post.

Translation pages

The translations pages are some of the most data-intesnive in all of Launchpad. Danilo talks about the tranlsations team’s Launchpad Performance Week work in his blog post.

Code project cloud

Launchpad Code developer, Jonathan, wrote on his own blog about his work on the project cloud page:

The Launchpad Code home page is now a little bit nicer and a little bit faster.

For a while now, the page has had a tag cloud showing some projects that host branches on Launchpad. The tag cloud is drawn based on these two rules:

  • the more branches, the bigger the tag
  • the more recent the last commit, the less faded the tag.

Up until recently, the home page only showed some projects starting with ‘A’ — which is a bit silly. My recent patch changed the query to show the most branch-y projects on Launchpad. It looks a lot nicer, IMO.

We also got rid of the green / blue distinction. No one knew what it meant and it made the page more visually muddled. There are still some tweaks we can make, but I think this is definitely an improvement.

And elsewhere

We made a number of smaller changes right across Launchpad, as part of the Performance Week. We’re also working on introducing much more AJAX into Launchpad’s web interface, which should make the experience of using Launchpad feel much quicker.

We’ll be holding another Launchpad Performance Week in April, this time cutting our definition of slow page by a further five seconds. Again, we’ll be posting details of what we did right here.

8 Responses to “Roundup of the first Launchpad Performance Week”

  1. Vadim P. Says:

    “elsehwere”

  2. Daniel Says:

    And tranlsations :)

  3. Markus Korn Says:

    I wonder if it is a good thing to increase speed by hiding information, especially by hiding the last comments on bugreports with more than 80 comments.
    The theory might be that bugreports with more than 80 don’t have much value at all, but in practice the last comment are often the most important ones (e.g they show when a bug was fixed, or why a bug was closed, or they give a workaround, or they give feedback on a fix etc.)
    As a result, bugpages do not become faster for the end user, because he has to open another page to see all comments.
    I don’t think that restructuring bugreports with a lot of comments is a bad idea, but it has to be done the right way. One solution could be to add voting for comments, so the comment with the most votes is shown first. Or add proper paging. Or allow the bug supervisor to close a bugreport so that it is impossible to add further comments (“this bugreport is getting unclear, if this is still an issue for you, please open a new bugreport”)

  4. Stefano Says:

    Hi,
    very interesting performance work!

    I was wondering which kind of tools did you used in your bottleneck analisys. I work in the performance area, we are used to use some commercial tools to find out which part of a page is actually the bottleneck.

    Can you describe which kind of tools did you use in the process?

    Thanks!

  5. Julian Edwards Says:

    Stefano,

    We use a combination of Python profiling tools and Launchpad-specific tools that measure the number and elapsed time of database queries.

    The latter is usually the larger culprit!

  6. Matthew Revell Says:

    Thanks Vadim, fixed :)

  7. Marco Rodrigues Says:

    I think add pagination for comments pages would be the best solution and add some AJAX to handle the pages when user click in the 1, 2, 3, will make it very user friendly.

  8. Marco Rodrigues Says:

    “data-intesnive” ??

Leave a Reply