API on Launchpad Blog

Make fetch service opt-in

Vaishnavi Asawale — Tue, 28 Oct 2025 11:31:12 +0000

Launchpad Builders do not have direct access to the Internet. To reach external resources, they must acquire an authentication token that allows access to a restricted set of URLs via a proxy. This can either be a custom authenticated builder proxy or the fetch service.

The fetch service is a custom sophisticated context-aware forward proxy. Whereas the builder proxy allows requests to allowlisted URLs, the fetch service also keeps track of requests and dependencies for a build.

Users can now opt-in to use the fetch service while building snaps, charms, rocks and sourcecraft packages. You can read more about the fetch service here.

Why is the fetch service important?

To achieve traceability and reproducibility, artifact dependencies retrieved during a build must be identified. The fetch service mediates network access between the build host and the outside world, examining the request protocol, creating a manifest of the downloaded artifacts, and keeping a copy of the artifacts for archival and metadata extraction for each package build.

How to use the fetch service?

To be able to use the fetch service, users must opt-in. For snaps, charms, rocks and sourcecraft packages, the use_fetch_service flag should be set to true in the API. For snaps and charms, this setting is also available in the Edit Recipe UI page.

The fetch service can be run in two modes, “strict” and “permissive”, where it defaults to the former. Both modes only allow certain resources and formats, as defined by inspectors which are responsible for inspecting the requests and the various downloads that are made during the build, ensuring that the requests are permitted.

The “strict” mode errors out if any restrictions are violated. The “permissive” mode works similarly, but only logs a warning when encountering any violations. The mode can be configured using the fetch_service_policy option via the API. For snaps and charms, the mode can also be selected from a dropdown on the Edit Recipe UI page.

When to use the fetch service?

Use the fetch service when you need to keep track of requests and dependencies for a build, e.g., when you need to verify that the artifacts belong to secure, trusted sources.

Information sharing is now in beta for everyone

Curtis Hovey — Tue, 28 Aug 2012 13:18:41 +0000

Launchpad’s bug and branch privacy features are being replaced by information sharing that permits project maintainers to share kinds of information with people at the project level. No one needs to manage bug and branch subscriptions to ensure trusted users have access to confidential information.

Project maintainers and drivers can see the “Sharing” link on their project’s front page. The page lists every user and team that the project shares with. During the transition period of the beta, you might see many users with “Some” access to “Private Security” or “Private” user information. They have this access because they are subscribed to bugs and branches. Maintainers can unshare with users who do not need access to any confidential information, or just unshare a bug or branch with a user. Maintainers can share share with a team to give them full access to one or more kinds of confidential information.

I have prepared a video that demonstrates the features (my apologies for the flickering)

Commercial projects can set bug and branch policies

Projects with commercial subscriptions can also change bug and branch sharing policies to set the default information type of a bug or branch, and control what types they may be changed to. Maintainers can set policies that ensure that bugs and branches are proprietary, and only proprietary, to ensure confidential information is never disclosed.

I maintain many project which have a lot of private bugs and branches. The sharing page lists a lot of people, too many to read quickly. I know most work for my organisation, but I don’t even know everyone in my organisation. So I wrote a Launchpad API script that can be run by any project maintainer to share the project with a team, then unshare with the team members. The members still have access to the bugs and branches and their subscriptions still work, but they will lose access to my project when they leave the team. This arrangement makes it very easy to manage who has access to my projects. share-projects-with-team.py is run with the name of the team and a list of projects to share with it.

./share-projects-with-team.py my-team project1 project2

Beta test: asynchronous PPA package copies

Colin Watson — Wed, 18 Jul 2012 18:12:36 +0000

The Ubuntu Foundations team has sponsored work on various improvements to Launchpad’s archive handling lately, mainly to expose various new facilities on the API where we were previously using privileged scripts. This has involved cleaning up a substantial amount of old code along the way, and it has become possible to fix some other old bugs as spin-offs.

One of these old bugs is “Archive:+copy-packages nearly unusable due to timeouts”. The +copy-packages page allows anyone who can upload to a PPA to instead copy packages from another PPA. This saves effort, and in the “Copy existing binaries” mode it can save a substantial amount of build time as well. For example, the LibreOffice packaging team uses this to deliver packages to different sets of users after they have passed various levels of testing.

Unfortunately, the very cases where this is most useful, namely large and complex packages, are also the cases where it is most likely to break. Copying large numbers of binary packages involves large numbers of database queries and can quite easily overrun the timeout for a single request to the Launchpad web application. Doing this for several series at once, a common case which seems reasonable, is proportionally less likely to work. Various attempts have been made to optimise the database interactions here, but ultimately doing lots of complex synchronous work in time for a single web request is doomed to failure.

The solution to all this is to copy packages asynchronously. For some time Launchpad has had the ability to schedule “package copy jobs” which run very shortly after the request (typically within a minute) but not immediately. For example, the Ubuntu team uses these when copying new versions of packages from Debian unstable in cases where there are no Ubuntu-specific modifications, and when releasing proposed updates to stable releases for general use after verification. A similar facility has been present in the code for the +copy-packages page for some time, but not exposed due to various bugs. We believe that these bugs have been fixed now, and so we would like to start copying packages asynchronously when requested via the web UI.

We have exposed this to beta testers first. The effect is that, if you are a beta tester when you ask for packages to be copied, you will be told something like “Requested sync of 2 packages. Please allow some time for these to be processed.” The processing should normally happen within a minute or two, and you will be able to see it in progress on the +packages page for the target archive. If it succeeds, the in-progress notification will be removed and you will be able to see the changes in the target archive. Otherwise, you will see a failure notification along these lines:

If beta-testing goes well, then we will enable this for all users, and remove the old synchronous copying code shortly afterwards; so please do report any problems you see.

If you are relying on package copies in the web UI happening immediately rather than within a few minutes, firstly, please contact us (e.g. #launchpad-dev on freenode IRC, or launchpad-users@lists.launchpad.net) as we would like to understand your requirements in more detail; secondly, you may be able to use the Archive.syncSource API method instead, which also has timeout constraints but is at least guaranteed to remain synchronous. However, we hope that most people will not have such a requirement.

Launchpad needs a command line

Martin Pool — Fri, 18 Mar 2011 08:51:11 +0000

Launchpad has a web UI, an email interface, and a ReST API that exposes every object in the database.

There are also a bunch of client programs, command line and graphical, that talk to Launchpad to do various things.

What we don’t yet have, and what I think would be great, is a systematic client that lets you manipulate
everything from the command line. There’s some code that starts towards this in Hydrazine, lptools and others, but I think having just a single tool that eventually does everything would be more discoverable and avoid unnecessary fragmentation or duplication.

(That’s not to say there’s not room for others that are guis, that are specialized to particular projects or that encapsulate a lot of policy or opinion about what they’re doing.)

So dobey and I have agreed to gradually merge hydrazine into lptools, and with other people to work towards making lptools cover everything you can do through the web UI or the API. If you have scripts you’ve written yourself, perhaps you’d like to merge them in.

New 'web_link' property on API objects

Martin Pool — Mon, 07 Feb 2011 06:06:46 +0000

Launchpad has a REST API that exposes almost every object within Launchpad. Most of them have API URLs that closely match the human-readable URL: for instance, https://bugs.launchpad.net/bugs/316694 can also be obtained in machine-oriented JSON ~~or XML~~ form from http://api.launchpad.net/1.0/bugs/316694. (The “1.0” in that URL means this is in the 1.0 API namespace.)

Previously, many Launchpadlib clients used string transformation to get from the API URL to something they could show a human in a web browser.

We’ve just added a web_link property on all Launchpad objects, so client applications can (and should) now just use that instead. Because this is just sent in the object representation you don’t need to upgrade launchpadlib to see it.

Three tips for faster launchpadlib api clients

Martin Pool — Wed, 14 Jul 2010 13:30:04 +0000

Three tips from Leonard’s lightning talk in Prague about writing faster Launchpadlib API clients:

1. Use the latest launchpadlib. It gets faster from one release to the next. (The versions in the current Ubuntu release should be fine; otherwise run from the branch or the latest tarball.)

2. Profile:

    import httplib2
    httplib2.debuglevel = 1

will show each http request and response, so that you can see what’s taking time.

3. Fetch objects only once:

Don’t do this:

    if bug.person is not None:
        print bug.person.name

instead

    p = bug.person
    if p is not None:
        print p.name

In the first case, the client may fetch the Person object twice. (We may fix this in future.)

Anonymous Access to the Launchpad Web Service API

Leonard Richardson — Tue, 19 Jan 2010 14:46:14 +0000

You asked for it and we’ve delivered. Your launchpadlib scripts can now get read-only access to the Launchpad web service API without going through any authorization process. Previously, to authorize your script, you had to open a web page in the end-user’s web browser and get the user to click a button. Now, you only have to go through that process if you want to access someone’s private data or modify the Launchpad dataset on their behalf.

If you’re using the latest version of launchpadlib (1.5.4), you can log in anonymously with the login_anonymously helper method. You just have to pass in a string identifying your client, and the name of the Launchpad server you want to access.

from launchpadlib.launchpad import Launchpad
launchpad = Launchpad.login_anonymously("my client", "edge")
print launchpad.bugs[1].title
# Microsoft has a majority market share'

(That code also illustrates how the latest version of Launchpadlib lets you specify a short string like “edge” instead of having to importing a constant like EDGE_SERVICE_ROOT.)

If you’re using an older version of launchpadlib, you can log in anonymously using the login helper method. You can get anonymous access by passing in empty strings for the token string and access secret:

from launchpadlib.launchpad import Launchpad, EDGE_SERVICE_ROOT
launchpad = Launchpad.login("my client", "", "", EDGE_SERVICE_ROOT)
print launchpad.bugs[1].title
# Microsoft has a majority market share'

Right now, anonymous access only works on Launchpad’s ’edge’ and ‘staging’ instances. It’ll be installed on production at the end of the month, with the release of Launchpad 10.01.