Introducing AutoPPA!

Jamu Kakar — Wed, 07 Nov 2007 19:04:27 +0000

Launchpad’s Personal Package Archives (PPA) service takes a great deal of pain out of the packaging process. The days of maintaining system images for package builds are thankfully behind us, but preparing sources to upload to PPA can still be a tedious task. I’m primarily responsible for building packages for the Landscape client. There are several factors which make handling source uploads manually problematic:

Subtle differences between the various releases of Ubuntu we support require slightly different debian/control files to build packages for different releases.
The pool-based APT repository that PPA publishes requires release-specific package version numbers in debian/changelog. Also, the changelog needs to include the name of the Ubuntu release the build is targeted at.
I need to build packages often, weekly or bi-weekly and need to be able to roll out bug fix packages as quickly as possible.
The process needs to be sufficiently simple that someone else can easily step in and take care of it if I’m not around.
I need to produce packages for all officially supported Ubuntu releases. At present this includes Dapper through Gutsy. In a few weeks we’ll start building packages for hardy. As time goes on we’ll have even more releases to support.

Automation to the rescue! AutoPPA is an application that automates the process of preparing and uploading signed sources to PPA. A project using AutoPPA for automated source uploads needs to meet some criteria:

It must be stored in a local Bazaar branch.
It must include a debian directory with valid packaging files.

Setting up AutoPPA

The first thing we need to do is install and configure AutoPPA. Add the following deb lines to your /etc/apt/sources.list (replace the Ubuntu release name with whatever is appropriate for your system):

deb http://ppa.launchpad.net/autoppa/ubuntu gutsy universe
deb-src http://ppa.launchpad.net/autoppa/ubuntu gutsy universe

Then, update and install autoppa.

sudo apt-get update
sudo apt-get install autoppa

Now, let’s assume we are building packages for the fictitious supertool Python application. For the sake of this example, we’ll assume that the branch to build from is stored at ~/src/supertool/trunk and has Debian package files already in place.

Let’s start by setting up ~/.autoppa.conf. This configuration file contains per-project configuration settings that are unlikely to change often. Here’s a sample:

[supertool]
email = Jamshed Kakar <jamshed.kakar@canonical.com>
branch = /home/jkakar/src/supertool/trunk
repository = /home/jkakar/src/supertool
ppa = supertool-ppa
releases = dapper edgy feisty gutsy

The ’email’ field specifies the e-mail address to use in the changelog. The ‘branch’ field specifies the location of the Bazaar branch that code will be built from. The ‘repository’ field specifies the location where temporary branches will be created. The ‘ppa’ field specifies the name of the PPA to upload to, as defined in ~/.dput.cf. Finally, the ‘releases’ field is a space-separated list of Ubuntu releases to prepare packages for. The configuration file can contain as many stanzas as you have projects to build with AutoPPA. Note that the name of the stanza must be the same as the name of the source package, as specified in debian/control.

We’re not quite ready to generate and upload signed sources. There are two more things we need to deal with first. The first is dealing with subtle differences in debian/control because of differences between Ubuntu releases. For a Python application, one of these is the reliance on python-support on dapper and python-central for edgy and newer. AutoPPA provides a simple file-template replacement mechanism to deal with this. Any files that end in .autoppa will be specially processed with output written to a file without the .autoppa extension. There is currently a single special directive, AUTOPPA_INCLUDE(...):, that is handled specially when encountered. If the following lines are defined in debian/control.autoppa:

AUTOPPA_INCLUDE(dapper):Build-Depends: debhelper, python-support,
lsb-release
AUTOPPA_INCLUDE(edgy,feisty,gutsy):Build-Depends: debhelper,
python-central, lsb-release

Then the following will be included in debian/control for Dapper:

Build-Depends: debhelper, python-support, lsb-release

and for Edgy, Feisty and Gutsy:

Build-Depends: debhelper, python-central, lsb-release

This allows us to have a single set of sources despite requiring minor differences for different Ubuntu releases. The obvious question is, why not just use an OR in the Build-Depends list? The answer is that sbuild, used by PPA to perform package builds, doesn’t handle ORs. There are also other cases where this kind of simple processing is handy.

The second thing we may want is version replacement. Any file, not just those that end in .autoppa, that include the special AUTOPPA_VERSION() string will be replaced with the version generated for the build. In Python code the following idiom can be used to assign the version to a local, for example:

VERSION = "AUTOPPA_VERSION(devel)"[len("AUTOPPA_VERSION("):-1]

When AutoPPA is run it will replace the ‘devel’ portion of that magic symbol with whatever version has been generated for the build.

Finally, Seahorse and devscripts needs to be configured to handle automatic GPG signing. You’ll need to import your GPG key into Seahorse and enable password caching. In order for Debian’s devscripts to work with Seahorse you’ll also need to do the following:

cp /usr/share/devscripts/conf.default ~/.devscripts.

Edit ~/.devscripts, find the ‘# DEBUILD_PRESERVE_ENVVARS=""’ line and
replace it with:

DEBUILD_PRESERVE_ENVVARS="DISPLAY"

Phew. Thankfully all that is initial setup that you’ll hopefully never have to think about it again.

Using AutoPPA

Now that everything is set up for our fictitious project, we can use AutoPPA to generate signed sources and upload them for building by PPA. It’s accomplished with a single command:

autoppa supertool-1.0.0-supertool1

The first thing that will happen is that $EDITOR will be opened to collect changelog details. Once you’ve provided details for the changelog everything else should Just Work(tm). AutoPPA will do the following:

Create a build branch based on the source branch you specified in ~/.autoppa.conf
For each Ubuntu release you’ve specified to build for:
- Files will be processed such that any .autoppa files are processed and AUTOPPA_VERSION() symbols are updated. The version generated, using the sample above will be, supertool-1.0.0-<release>1-supertool1. This ensure a unique per-release version string.
- The modifications made for the release will be committed as a revision in the temporary build branch and signed sources will be generated.
When signed sources have been generated for all the specified Ubuntu releases:
- They will all be uploaded to the specified PPA.
- The build branch will be merged back to the source branch, with the changelog as the commit message.
- The merged revision will be tagged with the version, 1.0.0-supertool1 in this case.
- Signed sources and the build branch will be deleted.

Now all we need to do is wait for PPA to build our packages! If you have more than one project setup in ~/.autoppa.conf you can specify multiple builds using a single command, such as:

autoppa supertool-1.0.0-supertool1 awesometool-1.0.0-awesometool1

That’s it! I hope AutoPPA turns out to be useful to others!

Jamu on Launchpad Blog

Introducing AutoPPA!

Setting up AutoPPA

Using AutoPPA