Archive for the ‘General’ Category

Project maintainers can see private bugs

Monday, July 23rd, 2012

Project maintainers can now see all the private bugs in their project. While Launchpad tried to ensure the proper people could see private bugs in the past, the old subscription mechanism was brittle. Users could unsubscribe themselves and lose access, or retarget a bug to another projects which does not update bug subscriptions. The Purple squad migrated project configurations to project sharing so that all private information was shared with project maintainers. Project sharing ensures that confidential information is disclosed to the proper people.

If you are a project maintainer, you might be surprised to find old private bugs that you have never seen before. This happened to me. Some ancient private bugs were in the “New” listing of bugs, other were buried in search results. You can search for just private bugs to review all private bugs.

advanced search for private information types

Privacy terminology is restored

We reverted the information type terminology changes introduced a few months ago.

  • User data ➙ Private
  • Embargoed Security ➙ Private Security
  • Unembargoed Security ➙ Public Security

While the jargon-laden terms helped the small number of people who work with confidential information, the people who report bugs were confused. The most common reason for unwanted disclosure is that people enter confidential information, and cannot see how to make it private. Sometimes a user may not notice the mistake until a few minutes later. We also revised the descriptions of the information types to help new users quickly select the correct information type.

change information type

You can hide your bug and question comments

Monday, July 23rd, 2012

You can now hide your own bug and question comments. If you want to hide a comment made in error, you can use the “Hide comment” action.

hide your comment

You can see it, and even unhide it if you choose. The project’s maintainer or the trusted people delegated to work with private information can still see your comment.

your hidden comment

This allows you, or the people the project shares private information with, to hide just the comments that contain personal information. The bug does not need to be made private if the comment can be hidden. Project maintainers can also hide comments because they contain spam or abuse.

 

Beta test: asynchronous PPA package copies

Wednesday, July 18th, 2012

The Ubuntu Foundations team has sponsored work on various improvements to Launchpad’s archive handling lately, mainly to expose various new facilities on the API where we were previously using privileged scripts.  This has involved cleaning up a substantial amount of old code along the way, and it has become possible to fix some other old bugs as spin-offs.

One of these old bugs is “Archive:+copy-packages nearly unusable due to timeouts”.  The +copy-packages page allows anyone who can upload to a PPA to instead copy packages from another PPA.  This saves effort, and in the “Copy existing binaries” mode it can save a substantial amount of build time as well.  For example, the LibreOffice packaging team uses this to deliver packages to different sets of users after they have passed various levels of testing.

Unfortunately, the very cases where this is most useful, namely large and complex packages, are also the cases where it is most likely to break.  Copying large numbers of binary packages involves large numbers of database queries and can quite easily overrun the timeout for a single request to the Launchpad web application.  Doing this for several series at once, a common case which seems reasonable, is proportionally less likely to work.  Various attempts have been made to optimise the database interactions here, but ultimately doing lots of complex synchronous work in time for a single web request is doomed to failure.

The solution to all this is to copy packages asynchronously.  For some time Launchpad has had the ability to schedule “package copy jobs” which run very shortly after the request (typically within a minute) but not immediately.  For example, the Ubuntu team uses these when copying new versions of packages from Debian unstable in cases where there are no Ubuntu-specific modifications, and when releasing proposed updates to stable releases for general use after verification.  A similar facility has been present in the code for the +copy-packages page for some time, but not exposed due to various bugs.  We believe that these bugs have been fixed now, and so we would like to start copying packages asynchronously when requested via the web UI.

We have exposed this to beta testers first.  The effect is that, if you are a beta tester when you ask for packages to be copied, you will be told something like “Requested sync of 2 packages.  Please allow some time for these to be processed.”  The processing should normally happen within a minute or two, and you will be able to see it in progress on the +packages page for the target archive.  If it succeeds, the in-progress notification will be removed and you will be able to see the changes in the target archive.  Otherwise, you will see a failure notification along these lines:

A notification of a failed copy to a PPA.

If beta-testing goes well, then we will enable this for all users, and remove the old synchronous copying code shortly afterwards; so please do report any problems you see.

If you are relying on package copies in the web UI happening immediately rather than within a few minutes, firstly, please contact us (e.g. #launchpad-dev on freenode IRC, or launchpad-users@lists.launchpad.net) as we would like to understand your requirements in more detail; secondly, you may be able to use the Archive.syncSource API method instead, which also has timeout constraints but is at least guaranteed to remain synchronous.  However, we hope that most people will not have such a requirement.

Bug reporting and search knows about privacy

Monday, July 16th, 2012

The Purple squad recently updated bug reporting and searching to understand the new privacy rules. Some of the changes were requirements to support sharing, others were opportunities we took advantage of.

Improvements to bug reporting and forms

The Purple squad updated the bug reporting UI to make it consistent with the bug pages. We choose to develop one consistent and tested UI rather than update the many kinds of widgets used in bug forms.

  • Project maintainers, drivers, and bug supervisors can report private bugs.
  • Autocomplete works with bug tags
  • The status and importance controls show their definitions.
  • Undecided is the first importance because it is the default importance.

Improvements to bug searching

Advanced bug search was updated after we discovered that recent changes made it possible fix some long standing issues with a few additional lines of code.

  • Anyone can search for Private or Embargoed Security bugs that are shared with them.
  • Autocomplete works with bug tags.

Usability and Accessibility fixes

We discovered that the popups that show bug status, importance and information type did not work with keyboards. It was possible to tab out of every other kind of popup by accident. We made deep fixes to the code so that all launchpad popups work with keyboard.

  • You can use the tab key to move between the items in popups.
  • You cannot accidentally tab out of any popup.

Launchpad does not have private projects…yet.

Friday, July 13th, 2012

Nothing breaks my heart quite like a request to make a project private–make it invisible to everyone except to the people the project trusts. I am utterly crushed when someone who works for Canonical or on Launchpad asks for one. I have been planning this feature for more than two years, and the Purple squad has been working on it for 13 months. I blog about this, I send emails about this, I present reports on this, but the people who most need private projects don’t know what the Purple squad is doing. I think the problem here is that Launchpad squads no longer use Launchpad to plan and execute work. There is no place for any interested party to see what the goals of Disclosure is and gauge how we are progressing.

I present my first draft of a report that states the simple goals of that the Disclosure feature wants to achieve . The report provides some summaries of the work that allows anyone to see what the Purple squad is doing, recently done, and will do next. There is also some analysis that provides insight into the amount of work remaining. This report complements the Purple squad’s kanban board. While kanban is excellent for tracking branches of code and technical tasks, the level of detail is unsuitable for non-Launchpad developers. The kanban board is also only accessible to a small number of people. I want a report that anyone interested in private projects or managing the disclosure of private information can see and understand. Mostly, I want everyone to see that the Purple squad is delivering valuable features and know when we will be done.

I based the report on the intended reporting UI for Launchpad series and milestones. I really miss using series and milestones to plan releases. For every milestone, I wrote our goals in the summary, and targeted bugs to the milestone. Though we abandoned the analytics because of performance concerns, I could reliably judge  the contributors’ velocity, and see when I needed to retarget work to another milestone because the remaining effort exceeded the milestone’s work capacity. Though I didn’t provide a burn down chart of the work, I could sort the milestone to see the colour change. I could confidently see and predict 3 months of work.

This report replaces the canvas-based chart I planned for series and milestones with a YUI 3 chart. The listing of bugs are split into categories so that I can focus on scheduling or provide Diogo with a list of bugs that need exploratory testing. Though this report thinks it is talking to Launchpad, it is actually using JSON for the 500+ bugs that I pulled using a trivial Launchpad API script. Since the data is cheap to retrieve, I can load the chart multiple times, each looking a different set of bug tags so that I can see specific themes of work.

The report shows that there is more than 60 days of work to complete the features needed by private projects. The Orange squad will work on private projects while the Purple squad finishes the prerequisites.

 

Webinar on Ubuntu MAAS: Metal as a Service

Wednesday, July 11th, 2012

Some of us on the Launchpad team have been working on Ubuntu’s MAAS (Metal as a Service) over the past few months.

MAAS is all about giving your physical servers the flexibility and ease of deployment you’ve come to expect from the cloud. With MAAS your cluster of ten, one thousand or a hundred thousand servers becomes a single, reusable, pool of computing resource that you can pull up and tear down just like VMs in the cloud.

Tomorrow, Matthew from the Launchpad team is hosting a couple of webinars introducing MAAS. They’re both the same content but at different times.

If you’re interested, sign up for the 15,00 UTC talk or the later one at 18.00 UTC.

Creating teams on demand

Monday, June 25th, 2012

teams

We want the project maintainer to be the default party that the project shares private information with. The problem at the moment is that Launchpad does not know how to set a team as the project maintainer during setup. Improper project setup is the root cause of most cases where information is disclosed to the wrong people. We need to improve project registration and setup to ensure users can ensure private information is managed properly. This issue is complicated by a very old issue, it is not possible to register a team at the moment you discover you need one. Launchpad must let me register a new team that will maintain my project when I first setup my project.

The Purple Squad discussed what we can do to simplify team registration and perform the registration in any page that allows you to set a team. We discovered several areas where we can make improvements.

  • Do not ask for non-essential information like contact address.
  • We can simplify the team membership policy language.
  • We can fix the confusion about membership renewal.
  • Launchpad can pre-fill the form with sensible defaults when the team will be used in a role.

Ian put together a demonstration to prove we could extend the person picker to also permit you to register a team.

When you want to set the project maintainer to a new team, Launchpad will ask you to confirm its suggestions for the Launchpad Id, display name, and membership policy. You can change the values, but most of the time you will choose to continue, and Launchpad will register the team and place it in the role.

 

Fixing a simple bug in Launchpad – A screencast

Wednesday, June 20th, 2012

When I ran the Launchpad Development Clinics at the last UDS, I was asked if I would give a presentation on how to fix a simple bug in Launchpad. This sounded like a great idea – after all there’s a lot of parts to the Launchpad development process that apply to every single branch you ever write, so it seemed well worth the effort. Rather than a presentation, though, I figured a screencast would be of more use, so here is just such a screencast. If you’ve got any questions or comments, don’t hesitate to get in touch.

Meet John Meinel – Blue Squad leader and Papa Smurf

Wednesday, June 20th, 2012

Papa Smurf

Laura: What do you do on the Launchpad team?
John: I’m the team lead for the Blue squad. Right now our squad is on Maintenance, so I generally do the coordination work of our team with other teams and the system administrators.

Laura: Can we see something that you’ve worked on?
John: Before we switched to maintenance, our team was focusing on doing Bazaar work. Within the more recent time, we’ve done stuff like fixing up the Ubuntu Distributed Development package importer, and getting translations for Quantal started for Launchpad.

Laura:Where do you work?
John: I work from home in the Netherlands.

Laura: What can you see from your office window?
John: We have a forest near our house, and some of the other neighbors houses.

Laura: What did you do before working on the Launchpad team?
John: As mentioned, we were focused on development of Bazaar, though arguably that was still part of the Launchpad group (not officially, but in spirit).

Laura: What did you do before working at Canonical?
John: I worked for a company developing medical imaging algorithms, mostly for detection and visualization of disease.

Laura: How did you get into free software?
John: Our team wanted to use something better than CVS for development. At the time SVN was pretty hard to set up, and there was just the beginnings of a couple of tools for distributed version control. I got into tla at the beginning, and was happy when Canonical started Bazaar, and I was able to hack on it.

Laura: What’s more important? Principle or pragmatism?
John: I’m a fairly pragmatic engineer. I think it is good to use principle as a guideline, but in the end if the work isn’t in the hands of people using it, it is providing no benefit and is arguably wasted effort.

Laura: Do you/have you contribute(d) to any free software projects?
John: Well, Launchpad and Bazaar are both pretty clear things (and tla a little bit before that). I also developed some other tools while here at Canonical. Such as Meliae for profiling python memory.

Laura: Tell us something really cool about Launchpad that not enough people know about.
John: The UDD package importer turns the changes from debian packages into real VCS branches that you can do lots of nice stuff on. (annotate the history, log the history, see the graph over time, etc.)

I think we are at about 90% coverage, and you can do “bzr log ubuntu:package” to find out the recent history for a package in ubuntu.

Laura: Is there anything in particular that you want to change in Launchpad?
John: My own personal project with launchpad is improving the connection handling when accessing Bazaar branches. Right now it is approximately 3s just to do the ssh handshake and start talking to codehosting. I have some improvements that should decrease that significantly, but we encounter some strange hanging bugs that are only reproducible in production. And LP’s commitment to having minimal user-visible downtime is particularly problematic for SSH connections. A single HTTP request is less than 5s, but an SSH connection can legitimately be active for an hour if accessing a large project.

Privacy and security replaced by information type

Monday, June 18th, 2012

Bug and branch information types

All users can now see the information type section that replaces the privacy and security section shown on bug and branch pages. This change allows users to clearly state the kind of information a bug or branch contains. Launchpad will soon permit project maintainers to share information types instead of managing individual bug and branch subscriptions. Project maintainers can see a link on their project’s front page to the  “Sharing” page. Sharing lists all the users and teams their project shares some private bugs and branches with. This list might be surprising. Launchpad Beta testers will soon be able share and unshare kinds of information to simplify management of whom the project discloses private information to.

See Reimaging the nature of privacy in Launchpad for more details.