Visualizing Ubuntu Differences Against Debian
Published by Julian Edwards June 16, 2011 in Beta, Cool new stuff
Over the past few months, my team have been working to make it easier to create and manage derivative distributions in Launchpad.
Although most of this will be of interest only to distro admins and contributors, we recently released a beta of a new feature that lets you visualize the differences between Ubuntu’s Oneiric series and Debian Sid.
There’s a new portlet on the distroseries page that will list the numbers of packages in both distributions, numbers only in Sid and numbers only in Oneiric. Clicking on the links takes you to pages that show the differences in more detail and allows you to request debdiffs and add comments on the packages.
Please let me know what you think of this change. We’re tracking bugs using the ‘derivation’ bug tag. Note that this is not meant to be a replacement for MoM – yet. We have a long way to go for that.
Coming soon, actual syncing from Debian to Ubuntu. Watch this space!
Using visibility for rudimentary JavaScript templating
Published by Aaron Bentley June 14, 2011 in General
DRY is one of my watchwords. That’s “Don’t Repeat Yourself“, of course.
If you’re designing your page to work with-and-without JavaScript enabled, then you’ll likely be tempted to repeat some of the
user-interface text in the template and in the JavaScript file. Especially for text that appears only some of the time.
One alternative is to render that text all the time, and only show it some of the time. In Launchpad, you add the “unseen” class to text in order to hide it.
In YUI3, it’s easy to add or remove classes using Node.toggleClass(). For example:
foo.toggleClass('unseen', baz);
This adds the ‘unseen‘ class to foo when baz is true, and removes it when baz is false.
This approach means you don’t need to repeat the text. It means your HTML stays in an XHTML file, with all the syntax-highlighting and other tool support that implies. It makes it easier to live with the fact that Launchpad don’t have a hardcore templating system in JavaScript.
It doesn’t eliminate all issues. If the page needs to have more than one instance of the text, this approach doesn’t directly apply. However, it’s easy to imagine extending it by using a hidden copy as a prototype and copying it as needed.
One disadvantage is that testing becomes harder. With conditional rendering, you can just check whether the text is rendered. With conditional visibility, you need to check whether the text has the unseen class, which is harder. If plan to apply this technique, it’s best to start out using it, or else you’ll have to rewrite all your tests when you switch approaches.
Another issue is that you will probably need to leave placeholders. For example, one of our conditionally-rendered strings included a link to a branch. When there’s no assigned branch, that text must be hidden. When someone assigns the branch via AJAX, we need to have a placeholder for the branch link, so that we can set it to the correct value with Javascript. At present, there’s no way to generate placeholders for arbitrary types, so it must be done manually.
Initializing page JavaScript from the JSONCache
Published by Aaron Bentley June 10, 2011 in General
Launchpad has a nice feature for initialising on-page JavaScript, called the JSONCache. I recommend considering it as your first choice when writing JavaScript that needs to be initialised. The one downside is that due to bug #740208, it’s not available for users who aren’t logged in. But then, users who aren’t logged in don’t need a lot of JavaScript, because they can’t write to most of Launchpad.
I found out about the JSONCache when we were working on he sourcepackage +sharing-details pages recently. This JavaScript has model objects, but I didn’t know how we should initialise them. We considered initialising them from the page HTML. We considered using the web service. But it turned out there’s already a great facility for this: the JSON cache.
If you’ve hacked on Launchpad’s HTML, you probably already know that every you can get LP.cache.context and LP.cache.me from any Launchpad page. These come for free, but with a little more work, you can get any object you need:
cache = IJSONRequestCache(self.request)
cache.objects.update({
'productseries': self.context.productseries,
'upstream_branch': self.upstream_branch,
'product': self.product,
})
These entries will then appear in the LP.cache with the names specified in the dict.
However, they are provided as plain JavaScript mappings, unlike the values returned by lp.client.Launchpad. You can use convert_cache to create a copy of the cache containing lp.client.Entry values. This allows you to intermix values from the lp.cache and the web service freely.
convert_cache currently lives in translations/javascript/sourcepackage_sharing_details.js, but should probably be moved into lp.client.
One extension I’d love to see would be a way to retrieve an updated copy of just the JSONRequestCache for a page. That way, we could reduce our number of round-trips, and also be confident that everything was up-to-date.
Getting started with launchpadlib
Published by Matthew Revell June 8, 2011 in API
I’m not a developer. I’ve played around — various flavours of BASIC as a kid, Python nowadays — but it rarely goes beyond a little experimentation.
Lately, though, my lack of Python-fu has been increasingly frustrating. In particular, there are times when I want to know something that I’m pretty sure Launchpad can tell me but there’s not an easy way to get at that information through the web UI.
For just those occasions we have the Launchpad web services API and its Python library, launchpadlib.
So, I’m going to learn my way around launchpadlib and write about my experience here. Bear in mind, I’m writing this from the perspective of someone who isn’t a developer. That may seem odd but I’m willing to be there are plenty of people out there who want to do the same as me: get more out of Launchpad and the data it hosts.
Getting started
You can install launchpadlib straight from the primary Ubuntu archive in recent releases. That’s what I’ve done so, unless I say otherwise, assume I’m writing about the version of launchpadlib that’s available in Ubuntu 11.04 (Natty).
Installation takes nothing more than a simple:
sudo apt-get install python-launchpadlib
Authenticating with Launchpad
When you access Launchpad through launchpadlib you can opt to either:
- access Launchpad anonymously
- log in with a user account and grant that launchpadlib instance a certain level of access to the account’s data.
If all you need is read-only access to public data, you may as well access Launchpad anonymously. For anything where you need to write data or access data that’s available only when logged in, each person who uses your launchpadlib application will need to grant their instance of the app access to their Launchpad account.
The Launchpad API, and so launchpadlib, lets OAuth take care of the authentication. It’s pretty simple really: when you run your app it opens up a Launchpad page in your browser. On that page you can choose what level of access that particular application has to your account. That log-in should be a one-time thing, as launchpadlib will cache your credentials.
And you can check/change what access different API apps have at any time.
Accessing Launchpad data
launchpadlib lets you access Launchpad data and functionality just like they were any other Python object. That’s great for someone like me because it means I don’t have to care about REST APIs, HTTP or any of that.
In my next post I’ll start actually getting hold of Launchpad data and maybe trying to do something with it.
Photo by Sébastien Bertrand. Licence: CC BY 2.0
JavaScript development: YUI testing
Published by Aaron Bentley June 6, 2011 in General
I recently came back to JavaScript, having largely ignored it for a couple of years. Development in JavaScript is much nicer than I remember. I thought I should share what I’ve learned.
The first thing I’ll mention is YUI tests. If you’re a TDD developer (and if not, why not?), YUI unit tests are the way TDD should be. The change/test cycle is very fast (to run the tests, hit “reload” in your browser). And since they’re proper unit tests, they feel like a safety net when you start to make changes.
They are a good way to test your model code and HTML changes. Other members of the Launchpad team have used them to do more integration-style testing, as well. It’s somewhat difficult to set up I/O dependent tests, but that will encourage you to make most of your code I/O-independent.
For information on how to get started with them, see: Developing with YUI.Test
Bryce Harrington appointed Ubuntu Launchpad stakeholder representative
Published by Francis J. Lacoste June 4, 2011 in General
When I explained how we determined what feature to work on next, I mentioned that we were looking to add a stakeholders representative for the Ubuntu project. Well, it’s now done. The Ubuntu Technical Board delegated Bryce Harrington to that position.
So if you are part of the Ubuntu community and need something from Launchpad, you can now talk to Bryce.
I’m really glad to have him on that position. Not only is he a respected member of the Ubuntu community, but he also did a 6 months rotation within the Launchpad team, so has understanding of how our project operates.
He’ll now be in a position to escalate important bugs affecting the Ubuntu community, as well as present the community’s view on what new features should be prioritized. It won’t be an easy job, since the Ubuntu project is big and varied. The people who cares about bugs are often not the same that cares about translations nor the same that cares about package publishing or the Ubuntu distribution development toolchain. And the stakeholder representative job is to help prioritize these needs to make sure that the Launchpad team works on the most important work. Plus then he’ll have to find common grounds with the other stakeholder representatives!
Thanks Bryce for stepping up to these responsibilities!
Photo by Wally Gobetz. Licence: CC BY-NC-ND 2.0.
What does rotation to maintenance really means?
Published by Francis J. Lacoste June 3, 2011 in General
I’ve been asked this question privately a few times already. What does it mean for a squad to rotate from feature-mode to maintenance-mode? The question arises as a number of folks noticed that some members of the squad continue working on feature-related bugs after they are on maintenance shift. That happened with the former Blue squad when they completed daily builds, with the Orange squad when they completed sharing translations, and with the Yellow squad and better bug subscriptions. So let me clarify what that boundary means.
In a nutshell, it means that the squad starts new work from a different queue. After the switch, new work will come from the general maintenance queue. Because we do the switch aligned with the calendar week, there is always work related to the feature that is still in-process (code started but not reviewed, or landed but not deployed, etc.) That’s the work that continues after the transition. After the rotation, the squad also becomes on the hook to cater for the user support duties related to maintenance mode.
What are the trade-off related to this boundary definition? The main impact is that we get a dip in maintenance related bug fixes around the rotation. That’s because the new maintenance squad has still some members not ready to start new maintenance-related work. It could be argued that because of the interrupt nature of the maintenance responsibilities, it would take longer to finish off the work related to the feature. I’m not sure that happens in practice, but I make note to check the data…
The alternative would be to only rotate after all the feature-related bugs are done-done. The rest of the squad could fix some lower priority bugs related to the feature or clean-up tech-debt accumulated during their rotation. It would preserve the number of people effectively working on maintenance around the transitions. But it would mean that it would take longer before we start the next feature. And given the poor state of our feature cycle time, it’s not a trade-off that I (and I guess our stakeholders) would be willing to make at this time. We might revisit this in the future though.
Photo by Camilia Hoel. Licence: CC BY-NC-SA 2.0.
Launchpad read-only June 8th 22.00 UTC
Published by Matthew Revell June 2, 2011 in Notifications
Launchpad’s web interface will be read-only, with other aspects offline, for around 90 minutes from 22.00 UTC on the 8th June 2011. This is to allow for our monthly database update.
Starts: 22.00 UTC 2011-06-08
Expected back by: 23.30 UTC 2011-06-08
Guessing relevant test modules with Fault Line
Published by Aaron Bentley in General
Launchpad’s test suite takes a long time to run. Far too long to wait for when you’ve made a change. And the likelihood that you’ve broken any given test is pretty small. So you probably want to start by running the tests that were likely to break.
You can usually guess which tests to run; if you’ve changed “archive.py“, you should probably run “test_archive“. But some connections are easier to miss, so I’ve hacked up a Fault Line, a bzr plugin that uses past changes to guess which test files correlate to the files you’ve changed. You can run it like so:
bin/test -m $(bzr fault-line --module-regex)
This will look at all the files you’ve changed, look at their recent history, and see which files tended to change in the last 100 revisions where you changed the specified files. The --module-regex option causes it to output a regular expression, assuming that the test files are Python modules. Otherwise, it would just output a list of the test files it found.
Thanks to Jelmer Vernoij, this is even easier to achieve for testing bzr. You just need to run “bzr selftest --auto“.
To install Fault Line, just run:
bzr branch lp:fault-line ~/.bazaar/plugins/faultline
It currently requires bzr 2.3 or later (e.g. stock Natty bzr).
Fault line is pretty limited right now. For example, the way it guesses what’s a test file is hard-coded. But a few days ago, it wasn’t even a proof of concept. Who knows what the future holds?
team owner no longer implies team member
Published by Robert Collins May 31, 2011 in Coming changes
A short headsup about an upcoming change.
A very long time ago the team owner was always a team member. This was changed to make team owners optionally members (sometime before 2008!). However the change was incomplete – there has been an inconsistency in the codebase ever since. For the details see bug 227494.
I wanted to let everyone know about us actually finishing this change though, because for a small number of teams (about 400) their administrators may be surprised when they cannot do things.
The inconsistency was this: if a team owner leaves the team, so they just own it, then they are not listed as a team member. But if they try to exercise a privilege the team grants – e.g. if the team is a bug supervisor – the team owners were able to do this. This setup made it impossible for users to accurately determine who can carry out the responsibilities of a team : the Launchpad web UI incorrectly reported team members.
The fix which will be deployed in the next day or so corrects this inconsistency: Team ownership will no longer grant access to anything that team membership grants.
For clarity, these are the rules around team owners:
- When a team owner is assigned (or a team made) the owner defaults to being an administrator-member.
- If a team owner deactivates their team membership then they are not considered a team member anymore: resources and access that team membership grants will not be available to the owner at this point.
- Team owners can always perform adminstrative tasks on the team: creating new administrators, edit the team description, rename the team etc.
- Point 3 allows an owner to add themself to the team they own even if they deactivated their membership previously.
