Posts Tagged ‘front-page’

Bug search no longer does substring matching of source package names

Wednesday, February 16th, 2011

As part of improving performance we have disabled the substring matching of source package names. This fixes bug 268508 and bug 607960. However its a slightly contentious issue – opinions vary about whether bug 268508 is a valid bug or not.

So we have only disabled it – the code is still present and when we have more leeway on the performance of bug searching we’ll revisit this and look into some design and UI analysis to decide whether substring matching of this sort should be done or not.

For now though, there should be less timeouts in bug searches.

Silencing bug notifications for stuff you did

Friday, February 11th, 2011

Launchpad’s bug mail can be a bit chatty sometimes as I’m sure you’ve noticed.  This cycle the Yellow Squad is working to give you more control about the bug mail Launchpad sends to you.

One problem that we’ve known about for a really long time was reported as bug 548 and has recently been closed thanks to the effort of  Данило Шеган and Gary Poster.  Their fix allows you to globally specify whether you want to receive email about actions you did.

Most people probably do not need to be reminded of something they did a few minutes ago and will want to turn off those emails.  But, since this is new functionality we’ve preserved the old behavior unless a user changes the setting.  If you do nothing you’ll continue to get email about actions you instigate on bugs.

Opting out of those messages is easy.  Simply go to https://launchpad.net/people/+me/+edit and uncheck the box as shown below by the big red arrow.

opt out of bug mail

As mentioned, this is just the first of many features and refinements that we’re working on to help you customize the stream of bug mail coming from Launchpad to suit your needs.

Should bug search match target names?

Monday, February 7th, 2011

We have a small quandry on the Launchpad development team at the moment. As bug 268508 discusses, when one searches for a bug on Launchpad we do a substring search on the names of bug targets.

For instance, searching in Ubuntu for ‘gcc’ will return all bugs on the packages ‘gcc’, ‘gcc-4.4’, ‘gcc-4.3’, ‘gcc-3.3’ and so forth. Likewise search for bugs in a project group will do a similar substring search on each of the individual projects in the project group.

It turns out that doing this search is itself expensive. I asked on the Ubuntu devel list about turning it off. We would close bug 268508 and also significantly improve search performance.

However this is a possibly contentious change – there was one mail strongly in favour of the current behaviour – so I’d like to get this change proposed to a wider community.

If you’ve got a strong opinion – that the current behaviour is good, or like bug  268508 describes, that its a poor behaviour and we would be better off without it, then I’d love to hear from you. Just leave a comment on this post, drop me an email – robert at canonical.com – or post to the launchpad-users mailing list.

Thanks,
Rob (LP technical architect)

Changing how we track Launchpad’s bugs, questions and blueprints

Wednesday, December 15th, 2010

From today, all Launchpad bugs, code, questions and blueprints are tracked under the one launchpad project.

We’ve already moved everything from the individual projects over to the parent launchpad project. All you need do differently is search/file bugs, questions and blueprints under that parent Launchpad project, rather than Rosetta, for example.

Don’t worry, though, there are redirects in place so that old links will still work.

There are also a couple of one-time steps you may need to take:

  • Update your bug subscriptions: if you’re subscribed to individual bugs, you need do nothing. If you’re subscribed to all bugs for a particular project, Malone for example, you’ll now need to subscribe to all Launchpad bugs.
  • Check your answer contact status: if you’re an answer contact for one particular application in Launchpad, and want to continue as such, you’ll need to become an answer contact for all of Launchpad.

To start with, bugs that we’ve merged in from one of the old sub-projects will have a tag that shows which project it came from. However, we’re planning to drop those tags once everyone’s settled into using just the one project.

Our code hosting won’t change at all as we’ve always hosted code under the parent Launchpad project.

This new approach will better reflect that Launchpad is one codebase but will also have a big practical benefit: it’ll be easier to find bugs and dupes because everything will be under the same project.

Why we’re doing this

For almost four years now, Canonical’s Launchpad team has been divided along application lines: i.e. we have sub-teams who each look after a particular part of Launchpad. So, Deryck, Abel, Gavin and Graham are currently the Launchpad Bugs team and work on nothing other than Launchpad’s bug tracker.

Reflecting this team structure in our Launchpad projects has made it easier for those sub-teams to plan their work.

It has worked pretty well but we’re about to change the structure of Canonical’s Launchpad team for a couple of reasons:

  • we want to focus on releasing features, and fixing problems, wherever they are
  • we want all Canonical Launchpad developers to be familiar with the full Launchpad codebase, rather than focusing only on one part.

So, as of February 17th the Canonical Launchpad team will have five squads. At any one time, three of those squads will each be working on a particular feature and the other two will be working on maintenance. Once a feature squad finishes its feature, it’ll switch places with one of the maintenance squads.

This will mean that there’ll always be ten Canonical Launchpad developers dedicated to fixing bugs, dealing with critical issues and generally making sure that Launchpad is serving you well. And of course there’ll be fifteen developers working on new features.

Rather than make this post even longer, I’ll write more soon and in the meantime point you to Rob Collins’ rousing launchpad-developers post in favour of the new structure.

As ever, if you have questions then please join us on the launchpad-developers mailing list or feel free to contact me directly.

How do you want to get Launchpad service status updates?

Tuesday, November 30th, 2010

I’m running a short survey — four questions — to find out how people want to get system status updates about Launchpad.

If you’ve got an opinion on how you’d prefer to get info about pending and unplanned Launchpad service interruptions, take the survey 🙂

Launchpad edge site deprecated

Wednesday, November 24th, 2010

I previously posted about our continuous deployment efforts in Launchpad. Since then the project has come a long way. We can deploy to nearly all our services without downtime. The remaining services are a bit trickier – but we are working on them.

As part of the project we are consolidating the ‘edge’ domain – https://edge.launchpad.net/, https://bugs.edge.launchpad.net/ and other similar domains – into the main launchpad UI. These domains are now deprecated.

The most important thing this means for you is that for members of our beta test program, we will no longer redirect you to https://edge.launchpad.net/ – instead we are serving our beta UI directly from the main website. The edge site is now running exactly the same code as the main Launchpad cluster and is updated at exactly the same time.

We have done this to deliver new features to our users more efficiently and at the same time simplify our production environment. So far the project has been very successful from our perspective – as I write this we have 5 days of inventory – code we’ve written but not deployed. This is down from an average of 2 weeks prior to this initiative starting, and we often sit lower – 1 to 2 days worth.

In the coming months as we refine this process and project we want to remove the edge cluster. As part of this we will start redirecting browser requests to ‘edge’ domains to the main Launchpad domain.

API clients cannot be redirected in this way, so we also ask that anyone writing or using Launchpad API scripts update them to use the primary cluster. We will slowly decrease the cluster size and disable it completely once we see no traffic on it. The main cluster is currently 3 times the size and should perform better for nearly any API script. To do this, use LPNET_SERVICE_ROOT rather than EDGE_SERVICE_ROOT. To get the LPNET_SERVICE_ROOT symbol, import it from launchpadlib.uris:

from launchpadlib.uris import LPNET_SERVICE_ROOT

If you have any questions about any of this we’d be delighted to hear from you – here, on IRC or the launchpad-user mailing list.

Rob Collins
Technical Architect

Launchpad Bug Jam December 2010

Tuesday, November 9th, 2010

Between December 13th and 24th we’re holding Launchpad’s first bug jam!

For two weeks, Canonical’s Launchpad team will focus solely on closing bugs and we’d love it if you’d join us.

Right now, there are more than 6,500 open bug reports for the Launchpad project. During the bug jam we want to close as many of these as we possibly can.

Closing a bug doesn’t necessarily mean fixing it: it may be something that can’t be fixed or even that’s already been fixed.

If you want to take part, or track progress, join the launchpad-dev team and mailing list. You should also take a look at the bug jam page of the Launchpad dev wiki.

New features for bug supervisors

Thursday, October 28th, 2010

We are starting to rollout features more rapidly on Launchpad as we move to a continuous deployment model.  There are some fixes being deployed today that I want to give Launchpad users a heads up about.  These are fixes meant to make the life of a bug supervisor easier.

Bug 114766, Only bug supervisor should be able to nominate a bug for release

Nominating to release has in the past been used as a mechanism to request that the bug be fixed, which is not what the feature is for, and we’ve now made it where only the bug supervisor for a project can nominate a bug for a series.

Bug 347218, Allow bug supervisors to make tags official

Until now, only project maintainers could make a tag an official bug tag.  Now bug supervisors have this ability, too.

Bug 664096, Fix Released should be locked against reopening

Bug supervisors waste time when they have to fix the status of a bug that had been marked fixed but later changed by someone else who mistakenly thinks they have the same bug or has the same problem in an older release.  The option to change a fixed bug to another status is now limited to bug supervisors.

There’s even more goodness to come as we get close to daily updates of Launchpad.  Stay tuned!

More Build Farm Improvements

Monday, October 4th, 2010

Continuing with the recent improvements to the build farm – Jelmer has made another massive one.

The last major scalability problem that we had was one where the whole farm was blocked when a single builder was ready to upload a build. In the case of large packages, like the kernel, the manager process could block for over a minute while it waited for the upload processor to unpack the package and verify its contents.

Jelmer’s work has decoupled the upload processing from the build farm manager process. What happens now is that the files collected from the builder are thrown into a staging queue area and then the manager process immediately continues with polling the rest of the builders, unblocked. A cron job will then process the builder upload queue at 1 minute intervals.

You can see the dramatic effect this has had on the overall queue for the PPA builders here:
Build Farm Queue Size

This is quite an incredible improvement as you can see! But we’re not stopping there, we’re currently doing a massive refactoring of the builder dispatching code so it’s all fully asynchronous. When this is all done we’re going to be in superb shape to support an increase in load that’s anticipated from the increasing number of people using the package recipes.

Meet Rob Collins

Monday, September 20th, 2010

Rob Collins joined the Launchpad team recently. Here’s a bit about him.

Matthew: What do you do on the Launchpad team?

Rob: I’m the Architect for Launchpad – I’m responsible for guiding the development and technology choices we make so that Launchpad can meet the needs of our stakeholders, users, developers and system administrators.

Practically speaking this means that I do a bit of everything: if I don’t, I can’t really feel what’s going on across the system, and being well informed is key to making good decisions / giving good advice.

Matthew: Can we see something that you’ve worked on in Launchpad?

Rob: A few recent examples of my personal code are: email list moderation pages are now batched, so should time out much less often if a list gets a lot of mail to moderate; I’ve been poking at bugs and bug API performance quite a bit, and the /attachments API is significantly faster now; the /messages API is going to be faster soon — and the main bugs page has had several seconds of overhead shaved off it for many bugs. I’ve set up a page with some of the things I think will help us with building and running Launchpad as we move forward.

Matthew: Where do you work?

Rob: I work from home like so many of us do; I recently moved to Rangiora, NZ. Yes, we were 40km or so from the epicentre.

Matthew: What can you see from your office window?

Rob: Grass, fence, roofs and sky; shorly to be grass, hedge and sky 😉

Matthew: What did you do before joining the Launchpad team?

Rob: Most recently I was one of the developers of Bazaar at Canonical, working mainly on the deep guts of the system.

Matthew: You’ve dedicated your Tuesdays to performance issues in Launchpad. Tell us about some of the improvements you’ve made.

Rob: So the ones I’ve listed above are pretty nice.

Here’s a list of specific timeout bug fixes I’ve done. More importantly though, we’ve been working on better diagnostics gathering for timeouts, and that work is making things easier and easier to analyse.

Matthew: What else are you working on right now?

Rob: Release features when they are done — this change to how we develop and rollout launchpad is a key change to speeding up the delivery of features. It’s not the full story, because we don’t (yet) have a plan for dealing with schema changes without downtime — that’s in the slightly longer term pipeline.

Matthew: What’s the next big thing you want to tackle in Launchpad?

Rob: Downtime-free schema changes; this should permit us to deliver new functionality much more rapidly.

Matthew: What experiences from your time in the Bazaar team will help with your new-ish role?

Rob: Ah! I spot a leading question. One of the interesting differences between Bazaar and Launchpad is that nearly every operation in Bazaar exercises an entire stack from scratch: there’s no active DB server with loaded caches ready and primed to go. That really got me thinking, continually, about the overheads involved in the entire stack : I think we need that same thought and consideration in Launchpad as we work on performance and deployment.

Matthew: How can community contributors help with what you’re working on?

Rob: Many, many ways!

Firstly, there’s the current timeout bugs. All of these can be fixed by patches … the exact patch and how hard it is to write will of course vary 🙂

Many timeouts are actually pretty shallow issues — but others will require schema changes and so forth. I’m delighted to help mentor folk working on any part of Launchpad.

The process and deployment changes we’re doing are mainly a sysadmin/deployment issue and not as amenable to contribution (by anyone not one of our sysadmins).

Matthew: How did you get into free software?

Rob: Thats a long story 🙂 Gradual love affair starting with demo code back on an Amiga A500.

Matthew: What’s more important? Principle or pragmatism?

Rob: Case by case basis 😉

Matthew: What free software projects do you contribute to?

Rob: Currently I’m still swimming for the surface in Launchpad 😉

Recently I’ve contributed to:

  • Launchpad
  • Bazaar
  • A variety of python testing projects:
    • testtools,testresources,testscenarios,fixtures,testrepository,
      unittest in Python itself
  • Storm
  • Ubuntu and Debian

The list going way back is longer … but also mainly of historical interest.

Matthew: Tell us something really cool about Launchpad that not enough people know about.

Rob: Recipes. Recipes are cool.