launchpadblog

The frequency of these posts has fallen off recently because I’ve been working on a Javascript client for the Launchpad web service, which we’ll be using to add some Ajax functionality to Launchpad. But it’s time for another one, featuring progress that’s visible to you.

First, an announcement: on Tuesday, the 20th of January, I’ll be doing an IRC chat on the Launchpad web service as part of Ubuntu Developer Week. See the schedule for details.

Newly published objects

The Bugs team has published CVEs through the web service. There’s a cves collection associated with every bug, and you can link and unlink CVEs from bugs.

The Soyuz team has published archives. You can use the web service to inspect a distribution archive or PPA, copy packages from one archive to another, and manage the permissions of who’s allowed to upload to an archive.

Merge proposals have also been published, but they’re read-only for now, so not very useful yet. But pretty soon you should be able to integrate them into an external code review tool.

Modify fields directly instead of through named operations

Consider the bug task object. Up to this point it’s had a number of read-only fields like ‘status’, ‘importance’, and ‘assignee’. These fields aren’t really read-only: you can modify them, but you have to know the secret. Each has a corresponding named operation: transitionToStatus, transitionToImportance, and transitionToAssignee. To modify ‘status’ you need to invoke transitionToStatus.

So you can’t modify a bug task’s status the way you can modify a bug’s description. This launchpadlib code works:

>>> bug.description = "A new description"
>>> bug.lp_save()

But this code doesn’t (except now, on staging):

>>> task.status = "New"
>>> task.lp_save()

You have to do this instead:

>>> task.transitionToStatus("New")

I’ve long felt that this transitionTo stuff is an internal detail you shouldn’t have to worry about, and judging from the bugs you’re filing, some of you agree. So I’ve made ‘status’, ‘importance’, and ‘assignee’ look like regular read-write fields.

The transitionTo methods are still available, so your old code will still work. In fact, the new code just calls the server-side transitionTo methods behind the scenes. The validation errors you used to get from passing a bad value into transitionToStatus will happen the same way if you set ‘status’ to a bad value.

There are other named operations that could be replaced by making the read-only field they modify into a read-write field, but I only changed those three bug task fields. I’ve talked to other Launchpad programmers about this new tool, and they’ll start using it according to their own schedules.

http_etag

Finally, there’s one part of the Javascript work that’s generally useful if you’re writing code against the web service on the level of HTTP requests rather than using launchpadlib. That’s the http_etag field now associated with entry objects such as people and bugs. It’s the same information provided in the HTTP header “ETag” when you get the entry object. So why send it again?

Well, imagine that you get a whole collection of bugtasks, and then decide you want to edit one of them. Ideally you’d use the value of “ETag” to make a conditional PUT or PATCH request, so that you wouldn’t accidentally overwrite someone else’s changes.

But you never got the ETag for that bugtask, because you got it as part of a collection. The http_etag field comes to the rescue here, allowing you to make a conditional PUT or PATCH without having to send a separate GET just to get the bugtask’s ETag.

Free software development is an essentially social activity and we’ve built Launchpad to help those interactions — both between people and projects — along the way.

By working on free software through Launchpad you build a picture not only of what you’ve done — reporting bugs, creating branches of code, etc. — but your team memberships can also show what roles you have in different free software communities.

Part of what we’ve been doing recently is to open Launchpad to make it far easier to use that information elsewhere; in particular, the web services API, the bug tracker plugin API and Launchpad becoming an OpenID provider.

Today we’re releasing two modules for Drupal 5.x under the AGPL:

• openid-launchpad: delegate your Drupal site’s user authentication to Launchpad
• openid-teams: assign Drupal roles to logged-in users based on their membership of specific Launchpad teams.

Using these modules, you can create a Drupal site that makes use of each person’s participation in your community, as reflected in Launchpad. For example, if you want to allow only members of a core development team to post release announcements to your project website all you need do is create a Drupal role with those permissions and then assign it to the Launchpad team of your project’s developers.

The Ubuntu Fridge news site is one of the first sites to use the modules. It passes user authentication to Launchpad and also grants an editor role to members of the Ubuntu Fridge Editors team in Launchpad.

To use the modules you’ll need to be running Drupal 5.x and also be using our modified version of the Drupal OpenID module (GPL). Full setup details are available in our help guide.

There’s more about both modules and you can report any bugs in Launchpad (openid-teams and openid-launchpad). If you use either of the modules, let us know how you get on!

Update: We’re working on support for Drupal 6.x. You can follow the progress by watching the openid-teams and openid-launchpad branches.

A while ago we announced the beta versions of the Launchpad plugins for Trac and Bugzilla. These allow Launchpad to sync not just bug statuses but also comments and, eventually, whole bug reports with the remote bug trackers that have them installed, bi-directionally. The practical upshot of this is that upstream projects that don’t use Launchpad for bug tracking can install one of these plugins and never again have to worry about not seeing bugs that people file using Launchpad but don’t forward upstream.

We in the Launchpad team realise, of course, that Bugzilla and Trac aren’t the only two bug trackers in the world. A project that uses Mantis, for example, may want its bug tracker to interact with Launchpad in the same way that Bugzilla and Trac now can. Unfortunately we can’t develop a plugin for every bug tracker out there as we don’t have the resources or the in-depth knowledge of those trackers’ internals necessary to be able to do so.

To make it possible for anyone to develop a plugin for any bug tracker, we’ve now opened up our Plugin API spec. The spec details all the APIs that need to be implemented for Launchpad to be able to sync bi-directionally with a remote bug tracker. It’s pretty detailed, and we’ve rolled into it our experiences in developing the Bugzilla and Trac plugins so as to make sure that new plugins don’t repeat the issues that we came across when we started interacting with the first bug trackers to install those plugins.

Once you’ve written an API plugin for your bug tracker of choice you can contact us by filing a question against Launchpad Bugs to let us know about it. We’ll then work to write the code on the Launchpad side that will interact with the API you’ve written. Once that’s done we’ll continue to work with you to iron out any bugs that we may come across.

I’m looking forward to seeing your plugins!

As of today, you can use your Launchpad identity to log into any website that supports OpenID. Now, you need remember only your Launchpad username and password for thousands of websites.

Here’s how it works:

1. You visit a website that’s an OpenID receiver and it asks you to log in.
2. You give that website your Launchpad profile’s URL – e.g. https://launchpad.net/~fred-bloggs
3. Launchpad asks you to confirm that you want to log in to the other website.
4. Once you’ve confirmed, you’re logged into the website and can use it just like normal.

That works both for sites that support OpenID 2.0 and 1.1. If you ever need your explicit OpenID URL, that’s no problem: you’ll find it on your profile page just below your contact details.

There’s one thing to bear in mind: once you start using your Launchpad profile to log into OpenID sites you shouldn’t rename your Launchpad account. That’s not to say you can’t change the display name but rather the system name that appears in your Launchpad URLs. In my case, the system name is matthew.revell.

Want to get started and not sure where to find sites that support OpenID? Take a look at the OpenID site’s Where page. There’s more on our OpenID help page.

You can now push your branches to Launchpad much more quickly. That’s because Launchpad now supports stacked branches: Bazaar branches that have only a little bit of their history.

By pushing up a stacked branch to Launchpad, you push up only that little bit, rather than the full history of your project. For the code that we Launchpad developers work on – i.e. Launchpad itself – it now takes less than two minutes to push up a branch. It used to take an hour and a half.

Upgrade to Bazaar 1.7 or better

To start using stacked branches, you need to upgrade everything to Bazaar format 1.6. Upgrade your trunk, all of your branches, your local shared repositories — everything. To do this you’ll need Bazaar 1.7 or better. You can get this from the Bazaar website or from the Bazaar PPA.

Set your development focus

You’ll also need to set a development focus for your project. The development focus is the mainline branch, often called “trunk” or “devel”. You want to set it to a branch that people want to branch from a lot. If your project doesn’t have a development focus yet, there’ll be a prominent link to set it on the Code tab for that project.

Push your branch just like before and whooosh!

OK. You’ve got a dev focus and all your branches are in the new Bazaar format. All you need to do now to make a stacked branch is:

bzr push lp:~foo/bar/baz

That is, you just push the way you always do and then watch the speed.

What happens is that Launchpad hints to Bazaar that new branches should be stacked on the development focus. Bazaar follows that hint and stacks the new branch appropriately. It then only sends up the revisions that are in the new branch but not in trunk.

This is why you want to set your development focus to a branch that’s going to be branched from and merged to a lot: the time it takes to push up a new branch is proportional to how different that branch is from your development focus.

A lot of work has gone into this feature (particular thanks to Michael Hudson and basically all of the Bazaar core devs), I hope you enjoy it!

If you’ve ever come to file a bug and found that it’s already been reported, you may have wanted to let the project know that you too have been affected.

Trouble is: many projects find “me too” comments unhelpful because they don’t add much to the discussion.

Launchpad’s new “This bug affects me too” feature lets you record just that but without the guilt! Give it a go in our staging environment.

We’ll use the “me too” data in future to help projects identify “hot” bugs.

This status report was put off until today because the edge site was frozen. This update is huge. We’ve made one change to launchpadlib, exposed a whole lot of Launchpad’s project registry, and published branches for the first time.

Since the development cycle is over, the next update will probably be in two weeks.

(more…)

Something I love about Launchpad is that I can quickly get a feeling for the people whose work I come across. Some of what you see on a Launchpad profile page is more geared towards knowing the person than their work.

We’ve added a bit more to that “getting to know you” part of the profile page with our new Google Maps mash-up. Now you’ve got the option of showing other people where in the world you are by flagging your location.

This is pretty cool for a bit of fun but when it comes to organising team meetings – in person or online – it becomes really useful. Take a look at the profile page for the Launchpad Developers team. On our team map you get a quick-glance feeling for where in the world members of the Launchpad team are.

Visit your profile page to set your location.

The guys in the Launchpad-Bazaar integration team added code review to Launchpad earlier this year.

Now, when you take part in a code review it’ll count towards your Launchpad karma!

If you’re new to code review, take a look at my previous post.

Last week the Launchpad web service didn’t change, because the source tree was frozen pending a Launchpad release. But this week we landed the branches we wrote last week, as well as some new ones.

The biggest change this week is client-side caching. Update your copy of launchpadlib to revision 17 and change your scripts to pass a directory name into the Launchpad constructor. That directory will be used as a cache.


>>> cachedir = "/home/me/.launchpadlib/cache/"
>>> launchpad = Launchpad(credentials, STAGING_SERVICE_ROOT, cachedir)


Without the cache, it takes several seconds just to start up launchpadlib. Every time you create a Launchpad object, the library has to download information about how Launchpad
works. With the cache in place, launchpadlib will download that information once and reuse it for every Launchpad object you create. The cache will also store the bugs, people, and other objects you retrieve as you use launchpadlib. The more often you use the same Launchpad objects, the more time the cache will save you. When we change Launchpad, launchpadlib will notice and will download the documents that have changed.

Second, Edwin Grubbs continued his work on publishing the Launchpad registry. launchpad.projects is now open for business. You can get basic information about Launchpad projects, their milestones and branches.

>>> bzr = launchpad.projects['bzr']
>>> bzr.title
u'Bazaar Version Control System'
>>> bzr.programming_language
u'Python'
>>> [milestone.name for milestone in bzr.all_milestones]
[u'0.9a', u'1.3.1rc1', u'1.4rc2', u'1.5', u'1.6', ...]


Finally, I’ve given launchpadlib some code that uses the client-side cache to avoid the “lost update problem.” This happens when two people modify the same object without coordinating, and the second one unknowingly overwrites the first person’s changes. If you use launchpadlib to make a change and it turns out someone else has changed the same object, you’ll get a chance to look at the new version of the object and see if your changes are still relevant.

One thing we haven’t done yet is gotten the API HTML reference to be regenerated whenever we change the web service. I planned to work on this yesterday, but instead spent the day fixing bugs. I’m working on it now and we should have it up early next week.

Next week we’ll see also more of the Launchpad registry exposed, and some long-awaited aspects of Launchpad’s bug tracker. And on Tuesday, Barry Warsaw and I will be talking about Launchpad’s web service API as part of Ubuntu Developer Week. That’s at 1900 UTC, in #ubuntu-classroom on irc.freenode.net.