launchpadblog

Last week the Launchpad web service didn’t change, because the source tree was frozen pending a Launchpad release. But this week we landed the branches we wrote last week, as well as some new ones.

The biggest change this week is client-side caching. Update your copy of launchpadlib to revision 17 and change your scripts to pass a directory name into the Launchpad constructor. That directory will be used as a cache.


>>> cachedir = "/home/me/.launchpadlib/cache/"
>>> launchpad = Launchpad(credentials, STAGING_SERVICE_ROOT, cachedir)


Without the cache, it takes several seconds just to start up launchpadlib. Every time you create a Launchpad object, the library has to download information about how Launchpad
works. With the cache in place, launchpadlib will download that information once and reuse it for every Launchpad object you create. The cache will also store the bugs, people, and other objects you retrieve as you use launchpadlib. The more often you use the same Launchpad objects, the more time the cache will save you. When we change Launchpad, launchpadlib will notice and will download the documents that have changed.

Second, Edwin Grubbs continued his work on publishing the Launchpad registry. launchpad.projects is now open for business. You can get basic information about Launchpad projects, their milestones and branches.

>>> bzr = launchpad.projects['bzr']
>>> bzr.title
u'Bazaar Version Control System'
>>> bzr.programming_language
u'Python'
>>> [milestone.name for milestone in bzr.all_milestones]
[u'0.9a', u'1.3.1rc1', u'1.4rc2', u'1.5', u'1.6', ...]


Finally, I’ve given launchpadlib some code that uses the client-side cache to avoid the “lost update problem.” This happens when two people modify the same object without coordinating, and the second one unknowingly overwrites the first person’s changes. If you use launchpadlib to make a change and it turns out someone else has changed the same object, you’ll get a chance to look at the new version of the object and see if your changes are still relevant.

One thing we haven’t done yet is gotten the API HTML reference to be regenerated whenever we change the web service. I planned to work on this yesterday, but instead spent the day fixing bugs. I’m working on it now and we should have it up early next week.

Next week we’ll see also more of the Launchpad registry exposed, and some long-awaited aspects of Launchpad’s bug tracker. And on Tuesday, Barry Warsaw and I will be talking about Launchpad’s web service API as part of Ubuntu Developer Week. That’s at 1900 UTC, in #ubuntu-classroom on irc.freenode.net.

Now that the Launchpad web service API has entered beta, I’ll be posting an update every week about the improvements we make to it. This way you’ll always know what’s going on, and you’ll see when something you want to do becomes possible.

Web service improvements

This week I made one major change to the web service itself. There’s now a special top-level resource that’s an alias to “you” on Launchpad. This is a nice convenience when you’re writing scripts for yourself, but it’s practically essential if other people are going to be running your program.

Here’s how to get a reference to the person who’s running the script, using launchpadlib, our Python interface to the web service.

>>> me = launchpad.me
>>> print me.name
leonardr


If you’re not using launchpadlib, GET the service root at https://api.staging.launchpad.net/beta/, and you’ll see the URL to this resource as ‘me_link’. When you GET that URL you’ll be redirected to your own user account.

launchpadlib improvements

I spent most of my time working on launchpadlib. Apart from some bugfixes and performance improvements that you won’t even notice, I made three big improvements.

Introspection

Previously, to see what capabilities a launchpadlib object had, you had to check the reference documentation. (That documentation is two weeks out of date, by the way; we’ll be fixing that next week.) Now you can just call dir() on an object, and all of its Launchpad-derived attributes and methods will show up in the list. If you only want to see the Launchpad attributes or methods and not all the internal launchpadlib stuff, you can check lp_attributes or lp_operations. This code shows what you can do with a launchpadlib person object:

>>> me.lp_attributes
['self_link', 'resource_type_link', 'longitude', ... 'homepage_content']

>>> me.lp_operations
['addMember', ... 'setLocation']


Slices

The second new feature is the ability to slice Launchpad collections as though they were Python lists. Here’s some code that gets the 10 most recently filed bugs in Launchpad.

>>> recent_bugs = launchpad.bugs[:10]
>>> [bug.id for bug in recent_bugs]
[258042, 258041, 258040, 258039, 258038, 258037, 258036, 258033, 258032, 258030]


Previously, the only way to do this was to iterate over launchpad.bugs and insert a break statement when you’d had enough, which was very awkward.

Loading from bookmarks

The third new feature is the ability to bookmark launchpadlib objects and go back to them later, the way you can bookmark web pages in your browser. Here’s launchpadlib code to acquire a bug.

>>> a_bug = launchpad.bugs[251497]
>>> print a_bug.title
Make it possible to instantiate a resource from a URL


I can play around with that bug all I want within a Python session, but if I exit the Python session the bug will disappear. If I want to get the bug back later, I’ll need to find it again from scratch. Unless I store the bug’s unique ID (also known as its URL).

>>> print str(a_bug)
https://api.staging.launchpad.net/beta/bugs/251497


At this point I can write that string to a file or database. Later on, a different process might come online and load the string back into memory. That process can get the bug object back by passing the bug’s URL into launchpad.load().

>>> a_bug = launchpad.load("https://api.staging.launchpad.net/beta/bugs/251497")
>>> print a_bug.title
Make it possible to instantiate a resource from a URL


Pretty simple stuff–people have been saving URLs and passing them around to each other for over fifteen years. The advantage of our web service’s design is that it gives you the same power in a scripted environment.

Upcoming work

Next week I plan to spend most of my time on behind-the-scenes performance improvements. You won’t notice anything if you use launchpadlib. If you’re writing your own client, you’ll know what I’m talking about when I say “ETag support.”

Meanwhile, Edwin Grubbs will be working to expose Launchpad’s project registry through the web service.

See you next week!

If you’ve been wanting to integrate Launchpad into your development tools, or create scripts that read and write Launchpad’s dataset, your wait is over. We’ve released the initial version of our RESTful web service API to the beta testing team. Now you can integrate with Launchpad using our Python library or by making simple HTTP requests.

No longer will you need to screen-scrape or write scripts that pretend to be a web browser; your programs will be able to communicate directly with Launchpad. Our support for OAuth means your users can delegate a subset of their Launchpad privileges to your program or website, without handing over their Launchpad pasword.

Right now, the web service provides basic access to Launchpad’s people and bugs. We’re working now to expose more of Launchpad’s data–projects, milestones, and so on–and to improve the usability of the Python client. Once we’ve stabilized the framework, we’ll open up the web service API to everyone. In the meantime, you can try it out by joining the Launchpad Beta Testers team. Once you’re on the team, you can get started by visiting the help homepage.

Our Python library, launchpadlib, works as a “web service browser”. You can use it to navigate the web service from a Python script, the way you surf the Launchpad website from your web browser. You don’t need any special knowledge of web services to use launchpadlib; you can write natural-looking code like this:


>>> me = launchpad.people['my-username']
>>> me.display_name = 'My new display name'
>>> me.lp_save()


We’ll be improving launchpadlib and the web service simultaneously; this is just the beginning.