Self-service riscv64 builds
Launchpad has supported building for riscv64 for a while, since it was a requirement to get Ubuntu’s riscv64 port going. We don’t actually have riscv64 hardware in our datacentre, since we’d need server-class hardware with the hypervisor extension and that’s still in its infancy; instead, we do full-system emulation of riscv64 on beefy amd64 hardware using qemu. This has worked well enough for a while, although it isn’t exactly fast.
The biggest problem with our setup wasn’t so much performance, though; it was that we were just using a bunch of manually-provisioned virtual machines, and they weren’t being reset to a clean state between builds. As a result, it would have been possible for a malicious build to compromise future builds on the same builder: it would only need a chroot or container escape. This violated our standard security model for builders, in which each build runs in an isolated ephemeral VM, and each VM is destroyed and restarted from a clean image at the end of every build. As a result, we had to limit the set of people who were allowed to have riscv64 builds on Launchpad, and we had to restrict things like snap recipes to only use very tightly-pinned parts from elsewhere on the internet (pinning is often a good idea anyway, but at an infrastructural level it isn’t something we need to require on other architectures).
We’ve wanted to bring this onto the same footing as our other architectures for some time. In Canonical’s most recent product development cycle, we worked with the OpenStack team to get riscv64 emulation support into nova, and installed a backport of this on our newest internal cloud region. This almost took care of the problem. However, Launchpad builder images start out as standard Ubuntu cloud images, which on riscv64 are only available from Ubuntu 22.04 LTS onwards; in testing 22.04-based VMs on other relatively slow architectures we already knew that we were seeing some mysterious hangs in snap recipe builds. Figuring this out blocked us for some time, and involved some pretty intensive debugging of the “strace absolutely everything in sight and see if anything sensible falls out” variety. We eventually narrowed this down to a LXD bug and were at least able to provide a workaround, at which point bringing up new builders was easy.
As a result, you can now enable riscv64 builds for yourself in your PPAs or snap recipes. Visit the PPA and follow the “Change details” link, or visit the snap recipe and follow the “Edit snap package” link; you’ll see a list of checkboxes under “Processors”, and you can enable or disable any that aren’t greyed out, including riscv64. This now means that all Ubuntu architectures are fully virtualized and unrestricted in Launchpad, making it easier for developers to experiment.
Tags: front-page, PPA, soyuz
