launchpadblog

We are reimagining the nature of privacy in Launchpad. The goal of the disclosure feature is to introduce true private projects, and we are reconciling the contradictory implementations of privacy in bugs and branches.

We are adding a new kind of privacy called “Proprietary” which will work differently than the current forms of privacy.

The information in proprietary data is not shared between projects. The conversations, client, customer, partner, company, and organisation data are held in confidence. proprietary information is unlikely to every be made public.

Many projects currently have private bugs and branches because they contain proprietary information. We expect to change these bugs from generic private to proprietary. We know that private bugs and branches that belong to projects that have only a proprietary license are intended to be proprietary. We will not change bugs that are public, described as security, or are shared with another project.

This point is a subtle change from what I have spoken and written previously. We are not changing the current forms of privacy. We do not assume that all private things are proprietary. We are adding a new kind of privacy that cannot be shared with other projects to ensure the information is not disclosed.

Launchpad currently permits projects to have default private bugs and branches. These features exist for proprietary projects. We will change the APIs to clarify this. eg:

    project.private_bugs = True  => project.default_proprietary_bugs = True
    project.setBranchVisibilityTeamPolicy(FORBIDDEN) => project.default_proprietary_branches = True

Projects with commercial subscriptions will get the “proprietary” classification. Project contributors will be able to classify their bugs and branches as proprietary. The maintainers will be able to enable default proprietary bugs and branches.

Next part: Launchpad will use policies instead of roles to govern who has access to a kind of privacy.

Launchpad will soon permit you to say your project is not affected by a bug shared with another project — you can delete the spurious bug task. This action can be done from the bug’s web page, and using Launchpad API.

You can remove a project from a bug if you are the project maintainer, bug supervisor, or are the person who added the project to the bug. This action can only be performed when the bug affects more than one project — you cannot delete an entire bug. This feature permits you to undo mistakes.

Launchpad beta testers will see the remove action next to the affected project name in the affects table.

The delete() method was added to the bug_task entry in the Launchpad API. There is a example API script, delete_bugtasks.py, that can remove a project from many bugs. There is also a split action to create a separate bug just for the specified project to track separate conversations in bug comments.

Usage: delete_bugtasks.py [options] bug_id:project_name [bug_id:project_name]

Options:
  -h, --help            show this help message and exit
  -v, --verbose
  -q, --quiet
  -f, --force           Delete or split bug tasks that belong to a public bug
  -d, --delete          Delete a spurious project bug task from a bug
  -s, --split           Split a project bug task from an existing bug so that
                        the issue can be tracked separately
  -w WEBSITE, --website=WEBSITE
                        The URI of Launchpad web site.
                        Default: https://api.launchpad.net;
                        Alternates: https://api.staging.launchpad.net,
                        https://api.qastating.launchpad.net

Previously, you could not remove spurious bug reports about your project. Many were cause by poor bug target management; you could not move a bug between projects and distributions. You can now move bugs between projects and distributions, but thousands of bugs still wrongly claim to affect a project or distribution. This causes clutter on bug pages and searches, and it causes Launchpad performance problems.

This change is a part of a super-feature called Disclosure. To ensure that confidential data is not accidentally disclosed, Launchpad will only permit private bugs to affect a single project. Soon, you may need to remove a project from a bug before marking the bug private.

I recently posted about Initializing page JavaScript from the JSONCache. Now I’m pleased to announce that you can also get updated copies of the IJSONRequestCache, to make it easier to update your page.

Brad Crittenden and I started work on this at the Dublin Thunderdome, and it’s finally been deployed. What this means is that for basically any page on Launchpad, you can append /++model++ to the URL, to get a fresh copy of the IJSONRequestCache. With ++model++, a change will typically require only two roundtrips; one to make a change, and one to retrieve an updated model. Future work may reduce this to a single roundtrip.

Why ++model++, not ++cache++? Cache is a really poor name for what the IJSONRequestCache is. Rather than providing fast access to whatever data has been previously retrieved, it is a complete collection of all the relevant data.

In Launchpad, the IJSonRequestCache is associated with the view, so we’re trying to rebrand it as the “view model”. This may seem strange from an MVC (Model, View Controller) perspective, but MVC can be recursive. A view may use a model to render itself.

I’m not a developer. I’ve played around — various flavours of BASIC as a kid, Python nowadays — but it rarely goes beyond a little experimentation.

Lately, though, my lack of Python-fu has been increasingly frustrating. In particular, there are times when I want to know something that I’m pretty sure Launchpad can tell me but there’s not an easy way to get at that information through the web UI.

For just those occasions we have the Launchpad web services API and its Python library, launchpadlib.

So, I’m going to learn my way around launchpadlib and write about my experience here. Bear in mind, I’m writing this from the perspective of someone who isn’t a developer. That may seem odd but I’m willing to be there are plenty of people out there who want to do the same as me: get more out of Launchpad and the data it hosts.

Getting started

You can install launchpadlib straight from the primary Ubuntu archive in recent releases. That’s what I’ve done so, unless I say otherwise, assume I’m writing about the version of launchpadlib that’s available in Ubuntu 11.04 (Natty).

Installation takes nothing more than a simple:


sudo apt-get install python-launchpadlib


Authenticating with Launchpad

When you access Launchpad through launchpadlib you can opt to either:

• access Launchpad anonymously
• log in with a user account and grant that launchpadlib instance a certain level of access to the account’s data.

If all you need is read-only access to public data, you may as well access Launchpad anonymously. For anything where you need to write data or access data that’s available only when logged in, each person who uses your launchpadlib application will need to grant their instance of the app access to their Launchpad account.

The Launchpad API, and so launchpadlib, lets OAuth take care of the authentication. It’s pretty simple really: when you run your app it opens up a Launchpad page in your browser. On that page you can choose what level of access that particular application has to your account. That log-in should be a one-time thing, as launchpadlib will cache your credentials.

And you can check/change what access different API apps have at any time.

Accessing Launchpad data

launchpadlib lets you access Launchpad data and functionality just like they were any other Python object. That’s great for someone like me because it means I don’t have to care about REST APIs, HTTP or any of that.

In my next post I’ll start actually getting hold of Launchpad data and maybe trying to do something with it.

Photo by Sébastien Bertrand. Licence: CC BY 2.0

There’s a new Launchpad client library, called lpad, for the Go programming language.

Gustavo writes:

lpad is based on a two-layered design. The top layer offers a static API which allows a more comfortable interaction with the API with static checks, better documentation, and more. The bottom layer is fully dynamic and enables the developer to access all the features of Launchpad, even those not supported by the top static layer.

There’s still work to do but the library is pretty much complete and it’s well tested, including integration tests which communicate with the real production servers.

You can get hold of lpad with a simple:

bzr branch lp:lpad

Check out the full API documentation.

Photo by Iain Farrell. Licence: CC BY-ND 2.0.

Launchpad API client developers: have your say in bug 714043 on whether launchpadlib should connect by default to Launchpad’s staging server (so data is discarded), or to real Launchpad.

Launchpad has a REST API that exposes almost every object within Launchpad. Most of them have API URLs that closely match the human-readable URL: for instance, https://bugs.launchpad.net/bugs/316694 can also be obtained in machine-oriented JSON or XML form from http://api.launchpad.net/1.0/bugs/316694. (The “1.0” in that URL means this is in the 1.0 API namespace.)

Previously, many Launchpadlib clients used string transformation to get from the API URL to something they could show a human in a web browser.

We’ve just added a web_link property on all Launchpad objects, so client applications can (and should) now just use that instead. Because this is just sent in the object representation you don’t need to upgrade launchpadlib to see it.

An long-requested feature in Launchpad is to let people see who’s using a PPA. Finally, we’ve implemented this!

Initially, the stats are only available on Launchpad’s webservice API. but we aim to show something useful in the web UI at some point.

If you are already familiar with the webservice API, then you can use the following binary_package_publishing_history object methods to retrieve the information:

• getDailyDownloadTotals
• getDownloadCount
• getDownloadCounts

Fabien Tassin is already using the stats to see how many people are using his daily build PPAs, and wrote an interesting blog post about it.

Three tips from Leonard’s lightning talk in Prague about writing faster Launchpadlib API clients:

1. Use the latest launchpadlib. It gets faster from one release to the next. (The versions in the current Ubuntu release should be fine; otherwise run from the branch or the latest tarball.)

2. Profile:

    import httplib2
    httplib2.debuglevel = 1

will show each http request and response, so that you can see what’s taking time.

3. Fetch objects only once:

Don’t do this:

    if bug.person is not None:
        print bug.person.name

instead

    p = bug.person
    if p is not None:
        print p.name

In the first case, the client may fetch the Person object twice. (We may fix this in future.)

Launchpad’s strategist, Jonathan Lange, has started a series of blog posts on getting started with Launchpad’s Python library, launchpadlib:

launchpadlib is the Python client-side library that talks to Launchpad’s own REST API. It turns out that customize scripted control of a bug-tracker-code-hosting-translation-distribution-building-cross-project-collaboration thing is actually quite handy.

Catch-up with the first two posts in the series:

• Get started with launchpadlib
• launchpadlib powerup.

And you can subscribe to Jonathan’s blog or follow blogs from both Jonathan and other members of the Launchpad community on Planet Launchpad.