The Appliance Builder That Isn’t
November 10th, 2009 @ 2:19 am UTC by evanI’ve said it before – there are a lot of appliance builders out there. With virtualization and the cloud being the hot ticket items of the day, everybody wants to try their hand at writing the software to provision those VMs.
Unfortunately, they all seem to suck. At least, the Debian/Ubuntu ones do. I haven’t found a VM or appliance builder application that I like, mostly because they all seem to be bad knock-offs of the actual debian-installer or ubuntu-installer.
The appliance builder I want has four key features:
- It should run unattended.
This one is kind of obvious, but rules out options like just running the debian-installer by hand and answering the questions as they come up. I do a lot of repetitive installs, and it’s important that I can hand my appliance builder a pre-crafted config file and get a customized, but totally unattended install.
- It should run trivially in a virtual environment, and seamlessly supports multiple hypervisors.
All of the appliance builders that anybody uses, or at least the ones I’ve attempted to use (VMBuilder and xen-create-image) run in the hypervisor. This is anywhere from an inconvenience to an actual security threat.
I want to be able to offer users a high degree of customizability, but my users are generally untrusted, and you simply can’t allow any flexibility when the appliance installer runs as root on your hypervisor. You certainly can’t allow your users to install packages out of their own apt repositories, including PPAs – a targeted attacker can easily break out of the chroot they’re put into when their package installs, and any package can include code that runs as root. Even if you don’t allow your users to customize appliances, the principal of least privilege says you shouldn’t be running the installs as root when you can run them as not-root, and you pretty clearly can.
Therefore, being able to run the appliance builder in a VM is an absolute must, regardless of the performance hit. We were able to adapt xen-create-image to do this for Invirt, but it wasn’t pretty, it took a lot of shoehorning, and it’s still pretty fragile.
Not only do I want to be able to install my appliances in a guest, but I also want to be able to run that guest under various virtualization environments. Many of my deployments are still heavily dependent on Xen. I have other deployments using KVM. Ideally, I’d like my appliance builder to work fairly transparently with multiple virtualization environments, although it’s probably OK for me if the resulting appliance image only works with the particular hypervisor that created it.
- It should use the distributions installer mechanism instead of jerry-rigging its own.
All of the appliance building applications I know of use their own installation code. For Debian/Ubuntu installers, this means running debootstrap and then frobbing the output. Even kiwi, the software behind the very shiny SUSE Studio effectively starts by unpacking a list of RPMs by hand.
There’s a lot of complexity in the Debian/Ubuntu installers. When you try to duplicate it, you will get it wrong. The resulting system will not be equivalent to the same system installed using a CD. I’ve certainly seen cases before where an installer-built image was different than an appliance-builder-built image, and it’s incredibly frustrating. Maybe this is something that could be fixed by actively developing the appliance builder (Ubuntu’s VMBuilder seems to be getting help from the ubuntu-installer developers), but it inherently seems like a waste of time to have this kind of code duplication.
- It should have a layer of abstraction that keeps me from repeating myself.
Simply booting the debian-installer or ubuntu-installer with a preseed file would certainly address the first three points. However, the preseed file needed simply to get an unattended Ubuntu install with no other bells and whistles is more than 20 lines long. Even if I have a template I can copy around, it’s gross from a DRY perspective.
I want my appliance builder to be configured through a config format that abstracts that away. I only want to specify that which can’t be reasonably guessed, not everything that I might want to have a say about.
All of the virtualization projects I’m involved in right now – Invirt, Virtigo, and some smaller personal projects – could really benefit from this kind of infrastructure piece, which means I’m likely to attempt to write it if it doesn’t exist. And as far as I know, this kind of appliance building application doesn’t exist for Debian and Ubuntu, at the very least. I’ll admit that I know almost nothing about other Linux distributions. Do any of them get this more right?
Tags: appliance, autoinstaller, cloud, install, virtualization
You say “Simply booting the debian-installer or ubuntu-installer with a preseed file would certainly address the first two points.”
Certainly it will perfectly address the third point as well.
So what’s left? The fourth point is that you don’t want to write 20 lines of preseed. That hardly seems like an insurmountable problem — you could e.g. make a script that generates the preseed from some other configuration file that you design.
> Certainly it will perfectly address the third point as well.
Yeah, good catch. I went back and added the first point at the end, because I wanted to be explicit about that. I meant that it addressed the first three, not the first two.
Some script that generates the preseed from other configuration is certainly a big piece of what’s missing. I think ideally I want the appliance builder to also take responsibility for spinning up the VMs that actually run the install as well.
I’m not saying this is a particularly challenging problem, just that nobody’s solved it yet. “Small matter of writing code”, and so forth
[...] « The Appliance Builder That Isn’t [...]
For the uninitiated, what’s an appliance builder?
I can’t really define an appliance builder without just saying that, “it’s a software app for building appliances.”
Appliance is something of a buzz word in the virtualization space these days. It basically refers to some application (like a database or web server) bundled together with an (usually minimal) operating system. They’re designed for fast deployment, etc, etc, and they usually run as VMs.
The application distribution side of the appliance buzz doesn’t really interest me, but in my mind, for appliances to be useful, they have to be quick and easy to assemble and so forth.
What I’m really looking for is a generic VM autoinstaller, but “appliance builder” fits the lingo of the moment a little better.