That Juju that you do (Part II: A magical balm to sooth your ills)
In my previous post I talked about the pain of having to set up a testing environment for our parallelised test work, and how there were an awful lot of hoops to jump through in order to get something usable up and running. Now, dear reader, let me tell you a tale of strangeness and charms.
Enter Juju
If you’re not familiar with Juju, I’d urge you to pay a visit to the Juju website to learn more, but in brief, I’ll explain: Juju is an orchestration service for Ubuntu. Using Juju allows you to deploy services rapidly, scaling up or down as you need. Each service is contained within a Charm, which is at it simplest a set of scripts that ensure that a given Juju unit does what it’s supposed to do at the appointed time (for example: install and config_changed are two of the most common hook scripts for a charm to have). We realised that in order to make our life simpler when testing our parallelisation work we could develop a pair of Buildbot Charms (one for the master, one for the slave) which when deployed through Juju, and given the right set of configuration options, would give us a working Buildbot setup on which to test Launchpad.
More about the charms…
The charms need to be able to automatically configure themselves to talk to each other (this is usually managed in Buildbot by static configuration files). Luckily, Juju provides for exactly that situation with the notion of “relations”; one charm can declare that it provides a particular interface as part of a relation and another can say that it requires that interface in order to be able to be a part of that relation. For our Buildbot charms, we have the following in the master’s metadata.yaml:
provides:
buildbot:
interface: master
And in the slave:
provides:
buildbot:
interface: slave
requires:
buildbot:
interface: master
Each charm has a couple of hooks that deal with relation, named in the form buildbot-relation-* where * is joined, changed or broken. These are run by Juju at the appropriate point in the process of connecting one instance to another. With all this in place, then, we can set up a working Buildbot environment by doing something like this:
$ juju bootstrap # create the Juju environment $ juju deploy buildbot-master —config=/path/to/master/config.yaml # deploy the master charm $ juju deploy buildbot-slave —config=/path/to/slave/config.yaml # deploy the slave charm $ juju add-relation buildbot-slave buildbot-master
The last line – juju add-relation buildbot-slave buildbot-master– tells Juju to connect the buildbot slave node to the master node. The two then do a bit of a dance to configure each other properly (in fact, it’s mostly a case of the slave saying: “Hey, I’m here, what do you want me to do?” and the master passing back configuration instructions). Once this is all done, you have a working Buildbot master and slave, ready to accept work to build.
What have we discovered about Juju?
First and foremost, we’ve learned just how powerful Juju actually is. We’ve taken a fairly complex-to-configure build environment, for which we normally use dedicated machinery whose configuration is not to be touched without sysadmin blessing on pain of pain, and turned it into something that we can deploy with four or five commands and a couple of configuration files. Sure, Juju has its quirks and oddnesses, but when we’ve run across them the Juju development team has been amazingly helpful with workarounds or, more usually, bug fixes. The current version of Juju is implemented in Python, too, so we find it pretty easy to contribute fixes of our own if we need to.
Where can I find out more?
As I said above, if you want to know more about Juju, you can check out the Juju website. If you want to take a look at our Buildbot charms and how we’ve built our hooks (they’re written in Python because that happens to be our language of choice, but in fact they can be written in anything so long as they’re executable), you can grab our code from Launchpad:
- For the master:
bzr branch lp:~yellow/charms/oneiric/buildbot-master/trunk buildbot-master - For the slave:
bzr branch lp:~yellow/charms/oneiric/buildbot-slave/trunk buildbot-slave
If you’ve got questions about Juju in general, the folks in #juju on Freenode are always tremendously helpful. If you’ve got any questions about our charms, ask them in the comments here and I’ll do my best to answer them.
((Image by http://www.samcatchesides.com/ under a Creative Commons license)
