Work items in blueprints
Published by Matthew Revell March 15, 2012 in Cool new stuff
At its most basic, a blueprint in Launchpad is a statement of intent: by creating a blueprint, you’re saying, “Here’s an idea for how this project should progress.”
How do you track the detailed steps to get from having the idea to seeing it implemented?
The Ubuntu development community came up with a solution: they break a blueprint down into items of work that can be assigned to an individual. That makes it easier to track who’s doing what and how they’re getting on, while the blueprint itself tracks the status of the big idea.
Until now, these work items weren’t recognised by Launchpad. Instead, Ubuntu and Linaro tracked them in the blueprint’s whiteboard. Using the Launchpad API, they pulled the whiteboard contents out and used them to generate burn-down charts for status.ubuntu.com and status.linaro.org.
Now, thanks to work by some of the team at Linaro, each blueprint has a text box where you can add, update and delete its related work items.
How you make use of work items will depend on the project you’re working on. Both Ubuntu and Linaro already make extensive use of work items and, if you’re working as part of those communities, you should follow their lead.
Using work items
Work items are pretty simple to use.
Here’s an example of how a set of work items might look:
Design the user interaction: DONE
Visual design: INPROGRESS
Test the UI: TODO
Decide on an API format: POSTPONED
Bootstrap the dev environment: TODO
Set up Jenkins CI: TODO
As you can see, each work item sits on its own line and you indicate its status after a colon.
As you can see, each work item sits on its own line and you indicate its status after a colon. The statuses you can use are:
- TODO
- INPROGRESS
- POSTPONEDDONE
Sometimes a blueprint can be targeted to one milestone but you’ll have a individual work items that need to be targeted to a different milestone. That’s fine, you can tell Launchpad that a bunch of work items are for some other milestone.
In the work items text box, simply use a header like this:
Work items for <milestone>:
If you enter an invalid milestone you’ll get an error message and the opportunity to correct the text field.
A milestone picker will of course be more useful and we’ve submitted a bug to keep track of that.
What’s next?
Soon, you’ll be able get a view of all the work items and bugs assigned to a particular individual or team for a particular milestone.
Of course, if you have other ideas for how Launchpad can make use of work items, we’d love to hear from you.
Photo by Courtney Dirks. Licence: CC BY 2.0
Reimagining the nature of privacy in Launchpad (part 3)
Published by Curtis Hovey February 21, 2012 in Bug Tracking, Code, Coming changes
We are reimagining the nature of privacy in Launchpad. The goal of the disclosure feature is to introduce true private projects, and we are reconciling the contradictory implementations of privacy in bugs and branches.
Launchpad will use policies instead of roles to govern who has access to a kind of privacy
We are implementing three kinds of policies, proprietary, embargoed security, and user data. The maintainer is the default member of these policies. The maintainer can share a kind or private data by adding a user or team to a policy.
For proprietary projects, the maintainer can add their organisational teams to the proprietary policy to share all the project information with the team members.
For Ubuntu, the maintainer will set the apport bot to be the only user in the user data policy; user data is only shared with a bot that removes personal data so that the bug can be made public. The Ubuntu security team will be the only users in the security policy.
Most maintainers will want to add project drivers to the policies if they use drivers. Bug supervisors can be added as well, though the team must be exclusive (moderated or restricted).
You can still subscribe a user or team to a private bug or branch and Launchpad will also permit the user to access it without sharing everything with the user. The existing behaviour will continue to work but it will be an exception to the normal rules.
Polices replace the bug-subscription-on-privacy-change rules. If you have every had to change the bug supervisor for a project with many private bugs, you can rejoice. You will not need to manually update the subscriptions to the private bugs to do what Launchpad implied would happen. Subscriptions are just about notification. You will not be notified of proprietary changes is proprietary information is not shared with you. Sharing kinds or information via policy means that many existing private bugs without subscribers will finally be visible to project members who can fix the issue.
Reimagining the nature of privacy in Launchpad (part 2)
Published by Curtis Hovey February 15, 2012 in API, Bug Tracking, Code, Coming changes, General
We are reimagining the nature of privacy in Launchpad. The goal of the disclosure feature is to introduce true private projects, and we are reconciling the contradictory implementations of privacy in bugs and branches.
We are adding a new kind of privacy called “Proprietary” which will work differently than the current forms of privacy.
The information in proprietary data is not shared between projects. The conversations, client, customer, partner, company, and organisation data are held in confidence. proprietary information is unlikely to every be made public.
Many projects currently have private bugs and branches because they contain proprietary information. We expect to change these bugs from generic private to proprietary. We know that private bugs and branches that belong to projects that have only a proprietary license are intended to be proprietary. We will not change bugs that are public, described as security, or are shared with another project.
This point is a subtle change from what I have spoken and written previously. We are not changing the current forms of privacy. We do not assume that all private things are proprietary. We are adding a new kind of privacy that cannot be shared with other projects to ensure the information is not disclosed.
Launchpad currently permits projects to have default private bugs and branches. These features exist for proprietary projects. We will change the APIs to clarify this. eg:
project.private_bugs = True => project.default_proprietary_bugs = True
project.setBranchVisibilityTeamPolicy(FORBIDDEN) => project.default_proprietary_branches = True
Projects with commercial subscriptions will get the “proprietary” classification. Project contributors will be able to classify their bugs and branches as proprietary. The maintainers will be able to enable default proprietary bugs and branches.
Next part: Launchpad will use policies instead of roles to govern who has access to a kind of privacy.
Contacting teams is easier and more reliable
Published by Curtis Hovey February 14, 2012 in Cool new stuff
Two changes to the contact team email feature were recently released that make communication more reliable.
Non-team members now see the “Contact this team’s admins” link. Previously, non-members saw a link to contact the team owner. The owner is often one person, and some team owners delegate the running of the teams to the team admins. There are often more admins then there are owners. So the message is more likely to reach someone who is involved in the day-to-day team management.
Team members see the “Contact this team’s members” link. Previously, members might have seen a link to contact the team, but that email when to the team email address that the team might not respond to. Many teams still use bogus email addresses to avoid emails from the days before Launchpad had great bug subscription filters. Launchpad has an anti-feature that prevents team mailing-lists from contacts all the team members. Team admins found it impossible to notify the member about issue ranging from policy changes, polls, meetings, to security issues. So members can now trust that their messages will be sent to the team members.
The feature is a convenient way to contact a user or team. Sometimes the feature is the only way you can contact a user or team that does not have a public email address. A user may use the contact user/team feature 3 times each day. The limit ensures no one can spam Launchpad users. The limit also means the feature is not a substitute for mailing lists.
Reimagining the nature of privacy in Launchpad (part 1)
Published by Curtis Hovey February 13, 2012 in Bug Tracking, Code, Coming changes, General
We are reimagining the nature of privacy in Launchpad. The goal of the disclosure feature is to introduce true private projects, and we are reconciling the contradictory implementations of privacy in bugs and branches.
We must change the UI to accommodate the a kind of privacy, and we must change some existing terms because to avoid confusion.
We currently have two checkboxes, Private and Security that create 4 combined states:
- Public
- Public Security
- Private Security
- Private something else
Most private bugs in Launchpad are private because they contain user data. You might think at first that something that is just private is proprietary. This is not the so. Ubuntu took advantage of defects in Launchpad’s conflation of subscription and access to address a kind of privacy we did not plan for. Most private bugs in Launchpad are owned by Ubuntu. They were created by the apport bug reporting process and may contain personal user data. These bugs cannot be made public until they are redacted or purged of user data. We reviewed a sample of private bugs that belong to public projects and discovered more than 90% were made private because they contained user data. Since project contributors cannot hide or edit bug comments, they chose to make the bug private to protect the user. Well done. Launchpad needs to clarify when something contains user data so that everyone knows that it cannot
be made public without removing the personal information.
Public and private security bugs represent two states in a workflow. The goal of every security bug is to be resolved, then made public so that users are informed. People who work on these issues do not use “public” and “private”, they use “unembargoed” and “embargoed”.
Also, when I view something that is private, Launchpad needs to tell me why. The red privacy banner shown on Launchpad pages must tell me why something is private. Is it because the page contains user data, proprietary information, or an embargoed security issue? This informs me if the thing could become public.
When I want to change somethings visibility, I expect Launchpad to show me a choice that clearly states my options. Launchpad’s pickers currently shows me a term without an explanation, yet Launchpad’s code does contain the term’s definition. Instead of making me search help.launchpad.net (in vain), the picker must inform me. Given the risks of disclosing personal user data or proprietary information, I think an informative picker is essential. I expect to see something like this when I open the visibility picker for a bug:
Branches require a similar, if not identical way of describing their kind of information. I am not certain branches contain user data, but if one did, it would be clear that the branch should not be visible to everyone and should not be merged until the user data is removed.
Next post: We are adding a new kind of privacy called “Proprietary” which will work differently than the current forms of privacy.
Faster deployments
Published by Francis J. Lacoste February 3, 2012 in General
Back in September, we announced our first fastdowntime deployment. That was a new way to do deployment involving DB changes. This meannt less downtime for you the user, but we were also hoping that it would speed up our development by allowing us to deliver changes more often.
How can we evaluate if we sped up development using this change? The most important metric we look at when making this evaluation is cycle time. That’s the time it takes to go from starting to make a change to having this change live in production. So before fastdowntime, our cycle time was about 10 days, and it is now about 9 days. So along the introduction of this new deployment process, we cut 1 day off the average, or a 10% improvement. That’s not bad.
But comparing the cumulative frequency distribution of the cycle time with the old process and the new will give us a better idea of the improvement.
On this chart, the gap between the orange (fastdowntime deployment) and blue (original process) lines shows the improvement to us. We can see that more changes were completed sooner. For example, under the old process about 60% of the changes were completed in less than 9 days whereas about 70% were completed under the same time in the new process. It’s interesting to note that for changes that took less than 4 days to complete or that took more than 3 weeks to complete, there is no practical difference between the two distributions. We can explain that by the fact that things that were fast before are still fast, and things that takes more than 3 weeks would usually have also encountered a deployment point in the past.
That’s looking at the big picture. Looking at the overall cycle time is what gives us confidence that the process as a whole was improved. For example, the gain in deployment could have been lost by increased development time. But the closer picture is more telling.
The cycle time charted in this case is from the time a change is ready to be deployed until it’s actually live. It basically excludes the time to code, review, merge and test the changes. In this case, we can see that 95% of the changes had to wait less than 9 days to go live under the new process whereas it would take 19 days previously to get the same ratio. So an
improvement of 10 days! That’s way more nice.
Our next step on improving our cycle time is to parallelize our test suite. This is another major bottleneck in our process. In the best case, it usually takes about half a day between the time a developer submits their branch for merging until it is ready for QA on qastaging. The time in between is passed waiting and running the test suite. It takes about 6 hours to our buildbot to validate a set of revisions. We have a project underway to run the tests in parallel. We hope to reduce the test suite time to under an hour with it. This means that it now would be possible for a developer to merge and QA a change on the same day! With this we expect to shave another day maybe two from the global cycle time.
Unfortunately, there are no easy silver bullets to make a dent in the time it takes to code a change. The only way to be faster there would be to make the Launchpad code base simpler. That’s also under way with the services oriented architecture project. But that will take some time to complete.
Photo by Martin Heigan. Licence: CC BY NC ND 2.0.
How to do Juju – Charming oops-tools
Published by Diogo Matsubara February 2, 2012 in General
Recently the Launchpad Red Squad and Product Team started working on a new cloud project. As part of that project we’ll be using Juju, a tool that helps you easily deploy services on the cloud.
As an opportunity to learn more about how Juju works, I wrote a charm to deploy oops-tools, an open source Django application that helps visualize and aggregate error reports from Launchpad, on Amazon’s EC2 and Canonical’s private Openstack cloud.
You might be asking, what’s a charm? Charms are basically instructions on how to deploy services, that can be shared and re-used.
Assuming you already have a bootstrapped juju environment, deploying oops-tools using this charm is as easy as:
$ juju deploy --repository=. local:oops-tools
$ juju deploy --repository=. local:postgresql
$ juju add-relation postgresql:db oops-tools
$ juju expose oops-tools
That’s it! With just a few commands, I have an instance of oops-tools up and running in a few minutes.
Under the hood, the oops-tools charm:
- starts two Ubuntu instances in the chosen cloud provider, one for the webserver and another for the database server
- downloads the latest trunk version of oops-tools and its dependencies from Launchpad
- configures oops-tools to run under Apache’s mod_wsgi
- configures oops-tools to use the database server
There’s still work to do, like add support for RabbitMQ (oops-tools uses rabbit to provide real-time error reports), but this initial iteration proved useful to learn about Juju and how to write a charm. As it is, it can be used by developers who want to hack on oops-tools and can be easily changed to deploy oops-tools in a production environment.
If you’d like to give it a try, you can get the charm here: https://code.launchpad.net/~charmers/charms/oneiric/oops-tools/trunk
Enjoy!
| (“Harry Potter’s The Standard Book of Spells” by Craig Grobler, licensed under a CC BY-NC-ND license) |
Fighting fire with fire – Changes to bug heat
Published by Dan Harrop-Griffiths January 30, 2012 in General
We’re making changes to the way that bug heat is calculated and displayed in Launchpad. From 6th February, bug heat will no longer age/degrade, and the active flame counter symbol will be replaced by a number, next to a static flame. Here’s why.
Bug heat ageing is the cause of a wide range of timeouts in Launchpad. Every bug’s heat has to be recalculated and updated every day using a series of complex calculations, and when there are around 1 million bugs reports to track, that’s a lot of pressure on the system, consuming a significant chunk of resources. Turning off bug aging is the simplest way to solve this issue.
Display
The flame counter symbol, although adding some visual flair (and flare), also needs to update every time the bug age recalculations are made. The continual stream of updates to the bug rows also results in poor search index performance.
We’ll still have a flame symbol, however it’ll be static, with the bug heat number next to it. Although not as visually dynamic, it’ll be easier to work out bug heat scores more exactly, at a glance.
Although I’m sure some of us will miss this little Launchpad feature, less timeouts is good news for everyone.
| (“Happy and safe birthday” by Stéfan, licensed under a CC:BY-NC-SA license) |
Social private teams
Published by Curtis Hovey January 26, 2012 in Cool new stuff, Projects
The title may sound like a contradiction, but I assure you that it is not. You can now use private teams in social contexts in Launchpad. Private teams can collaborate with public projects and other teams if they choose to reveal the existence of the private team.
- Private teams can me members of other teams.
- Other teams can be members of private teams.
- Private teams can subscribe to bugs, blueprints, branches, and merge proposals
- Private teams can be in public project roles, such as maintainers, drivers, and bug supervisors.
- You can change the branch owner to be a private team.
- Private team personal branches (+junk) are always private.
When a member places the private team in a social situation, a the member is asked to confirm that it is okay to reveal the team’s identifying information. This information is the team’s Launchpad Id used in URLs, the displayname, icon, and logo. A user can visit the private team’s Launchpad page and will only see that information. The rest of the page is not shared. Anonymous users cannot see a private team’s page because that user is not being social; logged in users can see the private team’s page
Private team page seen by a non-member
Launchpad did not permit these interactions previously because it was not clear who should know about the team. Someone has to know. If Launchpad permitted private teams to subscribe to bugs or be members of teams without anyone knowing of them, they would be unaccountable. Private teams could spy on organisation, or learn about security vulnerabilities to exploit. Launchpad will not ever permit such asocial behaviour. The resolution for social interactions was to permit other parties to know enough of the private team to make an informed decision. For example, when I choose to make a bug private, I want to know who was already seen the bug through a subscription. I may choose to unsubscribe the private team if I do not trust them.
Private teams may invite exclusive teams to be members. Exclusive teams (moderated or restricted teams) review all members so they are accountable. If a team admin trusts the admins of another team, and that team is accountable, Launchpad permits the other team to be a member. This is actually a rule that applied to all exclusive teams. private teams are always exclusive (restricted membership policy). The only nuance with private teams is when it is a member of another team; the super team may know the members of the private sub team because the super team has the right to audit all its members so that it too can be accountable.
New feature – Customise your bug listings
Published by Dan Harrop-Griffiths January 23, 2012 in Bug Tracking, General
Custom Bug Listings
Over the past few months the Launchpad Orange Squad has been working to make it easier to get the information that matters to you from bug listings.
A lot of you have said in the past that you’d like to be able to filter bugs in a way that works best for you. Hopefully this new feature, with its customisable functionality should help with this goal, filling your screen with more of what you want to see.
Features
You can now sort bugs by criteria such as name, importance, status and age. You can switch on the criteria that you use most and turn off criteria that you don’t use. So if you always like to see bug age, but aren’t interested in heat, you can switch on age and switch off heat, and so on.
Display
We’ve also redesigned how bug listings are displayed – fitting more information into each bug listing, and adding sort options such as bug age, assignee, reporter, and tags.
You can put your results into ascending or descending order without having to reload the page, and you’ll be able to save your preferred layout, so your settings will be saved for the next time you need to look over your bugs.
User research
This was my first main project since joining the Launchpad team back in November as the new Usability & Communications Specialist. User research has played an important part in how we’ve defined the feature and the decisions the team has made to improve the display, wording and functionality.
A number of you took part in one to one interviews, at group sessions at UDS-P and by taking part in an online survey. Thanks to everyone involved – what you told us has really helped to make this feature a more user-friendly experience. Some of our user research results (link) are already available online, with more being added soon. We’ll be carrying out some further tests in the weeks ahead, so please get in touch if you’d like to get involved.
Bugs
Every new feature has teething problems, and custom bug listings is no different. We still have a number of bugs that need tweaking, so please bear with us, and file any bugs if you spot anything that’s still out there.
