Three tips for faster launchpadlib api clients
Three tips from Leonard’s lightning talk in Prague about writing faster Launchpadlib API clients:
1. Use the latest launchpadlib. It gets faster from one release to the next. (The versions in the current Ubuntu release should be fine; otherwise run from the branch or the latest tarball.)
2. Profile:
import httplib2
httplib2.debuglevel = 1
will show each http request and response, so that you can see what’s taking time.
3. Fetch objects only once:
Don’t do this:
if bug.person is not None:
print bug.person.name
instead
p = bug.person
if p is not None:
print p.name
In the first case, the client may fetch the Person object twice. (We may fix this in future.)
Tags: API, clients, front-page, Performance, tip
July 14th, 2010 at 2:28 pm
Another trick is to use Uris instead of objects in method arguments, for example
In [6]: timeit zeitgeist.searchTasks(assignee=str(launchpad._root_uri) + “~thekorn”)
1 loops, best of 3: 337 ms per loop
In [7]: timeit zeitgeist.searchTasks(assignee=launchpad.people[“thekorn”])
1 loops, best of 3: 537 ms per loop
The api doc at [0] tells you how the uri of a certain object has to llok like.
[0] https://launchpad.net/+apidoc/
July 14th, 2010 at 2:41 pm
For a read-only script of mine trawling bugs, I saw a roughly 5-times speed increase by wrapping my bug-task objects in this below – a very blunt caching mechanism:
class LPWrap: "Simple wrapper Cache-Proxy for LP objects" def __init__(self, lpObj): self.lpObj = lpObj def __getattr__(self, attr): result = getattr(self.lpObj, attr) if result.__class__.__name__ == "Entry": result = LPWrap(result) setattr(self, attr, result) return result[edited to fix formatting — mbp]
July 14th, 2010 at 2:43 pm
oh dear – indentation not preserved :(. the if … “Entry” only acts on the single succeeding line. I think everything else is relatively unambiguous
July 14th, 2010 at 9:40 pm
“May fix”
It should be the API’s responsibility to handle caching. Else you are coding around the core problem instead of fixing it. Passing references all around the place to try limit your http calls is silly. The API is supposed to be the interface to LP not just a dumb layer on top of http.
It should at the very least be an option that is enabled by default. Since most people that use the API will be small time fish and the bigger fish would simple know the API well enough to turn off the caching and handle it internally.
Just my 2cents.
July 15th, 2010 at 9:21 am
@drubin I agree we should have at least a sensible minimal level of caching, and we need the API to at least take the concept of caching into account. I can imagine over-aggressive caching also being confusing to clients.
July 20th, 2010 at 7:58 pm
Latest version of launchpadlib is rather vague – talking with Leonard about this he really meant the version of launchpadlib in Maverick which is still under active development.
July 23rd, 2010 at 7:15 pm
Working with Leonard I’ve included the performance improvement patches into a Lucid version of the package which can be found in my PPA – https://edge.launchpad.net/~brian-murray/+archive/ppa. This should also make its way into Lucid updates and you can follow bug 607947 regarding that.