Automatically import files to Launchpad using product release finder

Launchpad has a little known feature that is getting better. The product release finder is a process that runs everyday to locate new releases and import them to Launchpad. The process uses the series’ Release file pattern to locate files and import them to the appropriate release. It can even create releases for series. The process is robust and worth consideration if you want to upload large release files for your project.

The project owner and series release manager can set the Release URL pattern by the series edit page. The pattern is an ftp, http, or https URL with a glob (*) in the part of the file name that varies with each release for a series. For example:

http://widgets.dom/downloads/widget-2.*

describes all files that start with ‘widget-2.’. This might be the source for two different releases, widget-2.1.tar.gz and widget-2.2.3.tar.gz. The pattern will also match multiple files that belong to a single release, such as widget-2.1.tar.gz, widget-2.1.zip, widget-2.1.changelog.

Many projects choose to group files in series in a directory of their own, in which case the Release URL pattern would look something like:

http://widgets.dom/downloads/2.8/*

You can tell the product release finder to search multiple directories by using a glob for a directory. For example, if your project separates release files in directories for each OS then you can try

http://widgets.dom/downloads/*/widget-2.*

to scan downloads/ubuntu/widget-2.* and downloads/mac/widget-2.*.

Be careful to include the common part of the series in the URL, otherwise files from different series will be imported to the wrong series. Do not do something like:

http://widgets.dom/all-releases/*

because any file that looks like it has version information in it will be imported to one series.

In all cases, the product release finder will extract the version from the file name, and match it to a milestone name. It will create the milestone and release it if necessary. If a version cannot be extracted, the file is ignored.

The version numbers extracted from file names conform to Launchpad URL name rules. So if your release files have underscores or pluses in their version names, dashes will be substituted. Flavour information is also ignored in the file name. For example these file names yield these versions:


emacs-21.10.tar.gz => 21.10
vpnc-0.2-rm+zomb-pre1.tar.gz => 0.2-rm-zomb-pre1
warzone2100-2.0.5_rc1.tar.bz2 => 2.0.5-rc1
furiusisomount-0.8.1.0_de_DE.tar.gz => 0.8.1.0
glow-0.2.1_i386.deb => 0.2.1
Bazaar-1.16.1.win32-py2.5.exe => 1.16.1

What’s Next

The product release finder should notify owners and release managers when there are problems with imports. A lot of problems were fixed recently, but there are two issues we are still seeing in the logs that indicate the Release url pattern must be updated for some projects. The product release finder cannot access the server or directory in some of the URLs. There are also a few URLs that have no glob. They appear to be the URL to a single file, where a glob is needed so that the series can have many releases. If the product release finder does not find your files after a few days, review the Release url pattern and check the remote site to verify they are fine.

We will update the UI to make the Release url pattern more prominent, and easier to set for each series.

3 Responses to “Automatically import files to Launchpad using product release finder”

  1. pwolanin Says:

    Is there any more detailed documentation for this? I’d like to try it, but I would like to know if it import all historical files or only new files? Also, can I apply more complex patterns in the rules – for example so I can add releases greater than an exiting version to an existing series using something like 6.(1[5-9]|[3-9]*). to start importing at release 6.15?

  2. Curtis Hovey Says:

    Yes you can release greater than the existing version. That is why the match rule is on the series, so that all future releases are found and added.

    Sorry there is no other documentation. This post constitutes the start of the documentation. We do intend to add the documentation to both the help site and the pages that related to the product-release-finder in launchpad after the 3.0 UI is completed in a few weeks.

    The glob pattern is not an regular expression. The OR condition is not supported:
    http://docs.python.org/library/glob.html

    I do not see how to exclude releases before .15. Before .2 is easy:
    6.[2-9]*

  3. pwolanin Says:

    I tried a test of this, and existing tarballs do not seem to be imported. Does this functionality depend on a certain format for the index listing of the directory? Or am I misunderstanding your comments and shoudl only expect new files to be imported? i.e. Files that are created after the rule is created?

Leave a Reply