Virtastic: A Survey of Virtualization Naming

planet sipb, posts 1 Response »
Mar 252010

For today’s post, I thought I’d survey what software projects related to virtualization name themselves.

It turns out that most projects have cute, but relatively unoriginal, names, formed by finding a word with “vert” and changing it to “virt”.

To collect today’s list, I started with sed -ne 's/vert/virt/p' /usr/share/dict/words. That gives me 433 words.

To pare down the list a bit, I used Google’s web search API to ignore words that had no Google results. That got the list down to 125 words.

Finally, I filtered the list down to things that I actually thought were reasonable project names (for instance, “advirt” might be a reasonable project; “incontrovirtible” and “ovirtalkative”, not so much). This admittedly subjective sampling got me down to 27 words that seemed worth exploring further.

And so, without further ado, here is a sample of names for today’s vertvirtualization projects:

  • ConVirt: ConVirt has seen a lot of evolution. It was originally a Linux desktop graphical Xen management application, a la virt-manager, and was originally called XenMan (you can still see some evidence of this at their old website. Since then, it’s been renamed to ConVirt, enhanced to support KVM (using libvirt), and moved into a rich web application. It’s still available under the GPL, but Convirture, the company supporting ConVirt development, is selling some advanced features along with their paid support.
  • Divirt: Divirt is a project to create virtual networks, allowing geographically disparate virtual machines to act as if they’re all connected by a LAN. Doesn’t really look like it ever got off the ground, though.
  • ExtraVirt: ExtraVirt was a project from UMich to detect processor errors by running the same system synchronously in multiple VMs, regulating non-deterministic inputs, and comparing the output. All I can find is a single two-page brief on the project.
  • IntroVirt: Another project from the same group at UMich, IntroVirt was a system for both active intrusion detection and post-facto intrusion analysis. It used a bunch of tests from the host to monitor suspicious activity.
  • Invirt: The super awesome project from the MIT SIPB. Invirt is a full-stack, multi-host, Xen-based management platform targeted at semi-public deployments with a web-based control interface. It has per-machine access control and quotas, and supports creation, deletion, installation, and general administration through the web interface, including an autoinstaller for Debian and Ubuntu. The primary deployment of Invirt, the XVM service, provides free VMs for the MIT community. XVM is currently running 246 separate VMs on 4 physical servers.
  • oVirt: Similar to ConVirt (at least in its current form), oVirt is a RedHat-sponsored web-based virtualization management platform. In contrast to something like Invirt that’s designed for building a “public cloud”, oVirt is designed for building more of a “private cloud”, where all of the VMs are managed by the same person. oVirt was one of the first projects to heavily utilize libvirt, and both projects come from the same group in RedHat.
  • ReVirt: From the group that brought you ExtraVirt and IntroVirt, ReVirt is a trustworthy execution logger. It logs a VM’s execution for later replay. Like IntroVirt, it seems to be primarily designed for intrusion detection and analysis.
  • SubVirt: Yet another paper from UMich, this paper was somewhat groundbreaking research into malicious virtualization technology. The SubVirt project developed proof-of-concept “VMBRs” (virtual-machine based rootkits), which installed themselves as a hypervisor on machines, transparently turning the OS previously running on bare metal into a virtual machine.
  • TransVirtual Systems: These guys sell a compatibility layer that lets you run ancient software from Wang VS on a modern UNIX platform.
  • Xilinx Virtex: While not virtualizing the same layer as the other projects and products here, FPGA’s are basically virtualization for silicon, letting you literally create new digital logic on the fly, and Xilinx’s Virtex series of FPGAs is the top-of-the-line.
  • Virtigo: I’m sort of cheating here, because Google won’t help you find this one. When I left my internship at Google, I decided to pull the virtualization testing framework I was working on out from the larger body of work it was originally included in, and Virtigo is what I decided to call it. If I ever have the time to pull the project back together, it’ll live at virtigo.org.

In addition to those names, though, there are actually some names that haven’t yet been taken:

  • advirt
  • ambivirt
  • antevirt
  • chetvirt
  • controvirt
  • covirt
  • culvirt
  • discovirt
  • evirt
  • obvirt
  • pervirt
  • povirty
  • retrovirt
  • virtebra
  • virtical

So – what are you going to name your next virtualization-related project or product?

Posted by evan at 6:23 pm Tagged with: , ,

Making Xen Suck Less: Part 4

planet sipb, posts No Responses »
Mar 212010

Magic SysRqs

One of the most powerful ways to debug or recover Linux is through the Magic SysRq keys.

Normally, if you’re using a serial console, you send BREAK, followed by the command you want. However, this doesn’t work with the Xen paravirtualized serial console driver. Instead, you have to use Ctrl+o, followed by the command you want.

For instance, to do an emergency sync, press Ctrl+o, then s.

Note that you use Ctrl+o instead of BREAK for both dom0 and domU serial consoles – they both are using the Xen paravirtualized driver.

Posted by evan at 4:34 pm Tagged with: , , , , ,

Making Xen Suck Less: Part 3

planet sipb, posts No Responses »
Feb 212010

Paravirtualized Clocks

In theory, Xen dom0′s are supposed to forcibly sync their system clock to the domU’s. In practice, due to some incompatibility between either Ubuntu’s version of the dom0 or domU patches, that doesn’t work, even though the feature is enabled, which leads to clock drift and occasionally weird clock lockup bugs.

The easiest way to fix this is to disable the Xen clock syncing entirely, and rely on the standard Linux clock mechanism. You can do that by adding these two lines just before exit 0 in /etc/rc.local:

echo '1' > /proc/sys/xen/independent_wallclock
echo 'jiffies' > /sys/devices/system/clocksource/clocksource0/current_clocksource

You’ll want to be sure to run NTP or some other service to keep your clock in sync.

Posted by evan at 11:23 am Tagged with: , , , , ,

Making Xen Suck Less: Part 2

planet sipb, posts No Responses »
Feb 102010

Now for part two in my ongoing series on making Xen suck less. Last time we looked at making networking work for hardware virtualized machines. Networking for paravirtualized VMs does work out of the box, but this hint might help if you’re running into performance problems.

Paravirtualized Networking Performance

If you’re running a web server or some other server that’s sending large files (or sometimes small files), you may find that your VM seems to hang inexplicably on those transfers.

For some reason, the paravirtualized Xen networking drivers advertise that they support on-board TCP segmentation. In fact, they seem to pass the packets onto the wire un-segmented, which frequently will cause the packets to be dropped for going over the MTU.

If you’re using xen-create-image, there’s a commented out line in /etc/network/interfaces that runs ethtool -K eth0 tx off. That’s close to the right issue. You actually want to add a line to your /etc/network/interfaces so that it looks something like this:

auto eth0
iface eth0 inet static
 address 18.181.0.80
 gateway 18.181.0.1
 netmask 255.255.0.0

 post-up ethtool -K eth0 tso off

Stay tuned for more hints, including how to deal with clock issues and magic sysrqs. I’ll also be pulling walkthroughs together on converting paravirtualized to hardware virtualized VMs, and how to upgrade older Ubuntu releases to more recent ones safely.

Posted by evan at 9:32 pm Tagged with: , , , , ,

Making Xen Suck Less: Part 1

planet sipb, posts No Responses »
Feb 072010

Right now, all of my Xen dom0′s run Ubuntu Hardy with Xen 3.3 from hardy-backports. Before we even talk about making Xen work, that statement bears some looking at.

I use Xen for a variety of reasons. Some are historical – the Invirt Project was built on top of Xen, and migrating away from Xen to a solution like KVM or VMWare would require working with users that are running paravirtualized operating systems. Some are circumstantial – I still have hardware that doesn’t support hardware virtualization.

My reasons for using Ubuntu are far less logical – I know how to use it, and don’t want to learn anything else if I don’t have to.

And I use Xen 3.3 because it’s way more stable than Xen 3.2.

In any case, if you find yourself using Xen 3.3 on Ubuntu Hardy as a dom0, there are a lot of tricks I’ve picked up for making it work better. Over the next few weeks, I’ll be working my way through them. I’ll be tagging them all with xen-tips for easy retrieval later.

As a disclaimer, I have no idea if these problems have been fixed in later versions of Xen or Linux, or if they’re specific to the Xen and/or kernel shipped by Ubuntu. For me, there’s a lot of value in getting all of my software from my distribution, so these instructions are designed to help do that.

HVM Networking

I have no idea whose fault this is, but HVM networking just doesn’t seem to work out of the box. qemu-dm, which emulates the VM’s devices, hooks the VM to a tap net device, while Xen sets up networking for a vifN.0 device. As far as I can tell, the intent was to connect the tap and vif devices, but nothing does.

For Invirt, we worked around this by writing a wrapper script around qemu-dm to make sure everything was setup correctly. If you want to use this script, you can drop qemu-dm-invirt in /usr/sbin and qemu-ifup in /etc/xen. (You’ll probably want to replace vif-invirtroute in qemu-ifup with vif-bridge or vif-route or whatever networking script you’re using).

/usr/lib/xen/bin/qemu-dm is hard-coded to run /etc/xen/qemu-ifup, if it exists. Without the qemu-dm-invirt wrapper, though, qemu-ifup doesn’t have any access to the domain ID for the domain it’s setting up. qemu-ifup then sets up and triggers the normal Xen networking script, which repeats the same setup it did for the vifN.0 interface.

Then, in your Xen config file, be sure to set device_model = '/usr/sbin/qemu-dm-invirt'.

Posted by evan at 12:55 pm Tagged with: , , , , ,

Fast Computing in the Kernel

planet sipb, posts No Responses »
Jan 022010

I’m kind of inspired by Geoffrey’s speculative write-up on Linux seccomp to do a speculative write-up of my own. Most of the SIPB people around here will recognize this discussion, as we’ve had it a couple of times. My 6.UAT TA will recognize it as well, since I presented on this as a “representative M.Eng. thesis”—that is, something that I could do, but have no intention of actually doing, for my M.Eng. thesis.

To setup the premise here, programs that do a lot of number crunching tend to run fast regardless of how they’re running, whether that’s natively, under virtualization, or whatever. They’re generally allowed to do almost everything they need to without any help from the operating system, or any other layers sitting on top of them.

On the other hand, any program that needs to interact with the outside world at all does so using a system call, which is basically a special function that causes the program to jump into the operating system itself. Because you don’t want random processes to have unfiltered access to raw hardware, a surprising amount of functionality is exposed through system calls, including read, write, send, recv. This means that applications such as, say, apache, spend almost all of their time doing system calls, since all a web server really does is read a file from disk, and then send it over the network.

The problem comes in when you consider the context switch between userspace applications and the kernelspace operating system needed to execute a system call. As it turns out, this context switch is slooow. How slow is it? Well, we can look at a paper from Microsoft Research. Their highly experimental operating system, Singularity, is flexible enough that it can run applications either with or without the context switch required in a traditional operating system. Here’s what they found:

  Cost (CPU Cycles)
  ABI call[1] Yield[2] PSR[3] Create Proc[4]
Singularity
SIP-Phys[5]		 80 365 1,041 388,162
Singularity
HIP-R3[6]		 304 638 2,580 830,999
FreeBSD 878 911 13,304 1,032,254
Linux 437 906 5,797 719,447
Windows 627 753 6,344 5,375,735
[1] Their terminology for a system call. On each operating system tested, they specifically chose a system call that could always return very quickly.
[2] Surrender remaining time in the current thread of execution and schedule another thread.
[3] “Process-Send-Receive” – their term for an IPC benchmark that sends a byte of data back and forth between two separate processes.
[4] Create a new process. Equivalent to a fork+exec in UNIX terminology.
[5] Singularity running without the hardware context switch.
[6] Singularity running with a hardware context switch.

What’s the take-away here? There are two. First, using hardware isolation to Singularity adds almost a factor of 4 on the time to execute a system call. Second, Singularity is way faster than other operating systems, all of which use a hardware context switch (of course, they’re also much more featureful than Singularity).

So that’s our problem. To try and solve it, we look to the techniques pioneered by VMWare for total machine virtualization.

When running an operating system under virtualization, we need some way to simulate what would otherwise be privileged operations on raw hardware. There are a lot of approaches to solving this problem, but VMWare primarily uses just-in-time binary translation (or BT). With BT, VMWare’s Virtual Machine Monitor (VMM) examines instructions just before they’re executed. If there are any unsafe instructions, they’re replaced with calls into functions in the VMM that emulate those instructions.

That on its own doesn’t make anything fast, but VMWare takes this a step further. In order to minimize the overhead of this emulation, VMWare’s VMM runs the translated code within the kernel (ring 0). It turns out that, because of this, VMWare’s VMM has an average slowdown of only 4% (see A Comparison of Software and Hardware Techniques for x86 Virtualization for detailed analysis).

Here’s the question: can we take the binary translation techniques from VMWare’s VMM and adapt them to run otherwise unmodified processes instead of operating systems within the kernel? And if we do, what is the performance impact?

If we can bypass the context switch expense measured by the Singularity team, it could easily more than compensate for the relatively small overhead of running applications under binary translation. I would go so far as to say that I expect syscall-heavy applications to run faster.

Putting the Singularity and VMWare papers right next to each other, this is a pretty obvious next step. But as far as I know, nobody’s done it yet. Does anybody else know of an implementation of this idea for a real operating system? Maybe a Linux kernel module that lets you run certain apps in-kernel? If it’s out there, I haven’t found it yet.

Posted by evan at 8:32 pm Tagged with: , , , , ,

Continuing the Search for Appliance Builders

planet sipb, posts No Responses »
Dec 182009

(Is there a non-ambiguous abbreviation for “appliance”? I don’t want to use “app builders”, because people would obviously get the wrong impression…)

I’m still looking for an appliance builder that has everything I want. Right now the three software packages on my list are Cobbler+koan, Thincrust, or maybe Kiwi.

I started looking into Kiwi a while back, but backed off because they seem to have DRY problems. Not to mention it’s written in Perl.

Cobbler and Thincrust look a little more promising, at least on the surface, but it’s hard to get a good sense of the kind of flexibility I can get out of them. It certainly doesn’t look like either of them have the ability to install a Debian/Ubuntu system without being handed the 20 lines of required pre-seed, but I could be wrong.

Does anybody have experience with these? Does anybody know if they fit the 4 features from last time, or could be hammered into fitting them?

Posted by evan at 12:33 am Tagged with: , , , , ,

The Appliance Builder That Isn't

planet sipb, posts 5 Responses »
Nov 102009

I’ve said it before – there are a lot of appliance builders out there. With virtualization and the cloud being the hot ticket items of the day, everybody wants to try their hand at writing the software to provision those VMs.

Unfortunately, they all seem to suck. At least, the Debian/Ubuntu ones do. I haven’t found a VM or appliance builder application that I like, mostly because they all seem to be bad knock-offs of the actual debian-installer or ubuntu-installer.

The appliance builder I want has four key features:

  1. It should run unattended.

    This one is kind of obvious, but rules out options like just running the debian-installer by hand and answering the questions as they come up. I do a lot of repetitive installs, and it’s important that I can hand my appliance builder a pre-crafted config file and get a customized, but totally unattended install.

  2. It should run trivially in a virtual environment, and seamlessly supports multiple hypervisors.

    All of the appliance builders that anybody uses, or at least the ones I’ve attempted to use (VMBuilder and xen-create-image) run in the hypervisor. This is anywhere from an inconvenience to an actual security threat.

    I want to be able to offer users a high degree of customizability, but my users are generally untrusted, and you simply can’t allow any flexibility when the appliance installer runs as root on your hypervisor. You certainly can’t allow your users to install packages out of their own apt repositories, including PPAs – a targeted attacker can easily break out of the chroot they’re put into when their package installs, and any package can include code that runs as root. Even if you don’t allow your users to customize appliances, the principal of least privilege says you shouldn’t be running the installs as root when you can run them as not-root, and you pretty clearly can.

    Therefore, being able to run the appliance builder in a VM is an absolute must, regardless of the performance hit. We were able to adapt xen-create-image to do this for Invirt, but it wasn’t pretty, it took a lot of shoehorning, and it’s still pretty fragile.

    Not only do I want to be able to install my appliances in a guest, but I also want to be able to run that guest under various virtualization environments. Many of my deployments are still heavily dependent on Xen. I have other deployments using KVM. Ideally, I’d like my appliance builder to work fairly transparently with multiple virtualization environments, although it’s probably OK for me if the resulting appliance image only works with the particular hypervisor that created it.

  3. It should use the distributions installer mechanism instead of jerry-rigging its own.

    All of the appliance building applications I know of use their own installation code. For Debian/Ubuntu installers, this means running debootstrap and then frobbing the output. Even kiwi, the software behind the very shiny SUSE Studio effectively starts by unpacking a list of RPMs by hand.

    There’s a lot of complexity in the Debian/Ubuntu installers. When you try to duplicate it, you will get it wrong. The resulting system will not be equivalent to the same system installed using a CD. I’ve certainly seen cases before where an installer-built image was different than an appliance-builder-built image, and it’s incredibly frustrating. Maybe this is something that could be fixed by actively developing the appliance builder (Ubuntu’s VMBuilder seems to be getting help from the ubuntu-installer developers), but it inherently seems like a waste of time to have this kind of code duplication.

  4. It should have a layer of abstraction that keeps me from repeating myself.

    Simply booting the debian-installer or ubuntu-installer with a preseed file would certainly address the first three points. However, the preseed file needed simply to get an unattended Ubuntu install with no other bells and whistles is more than 20 lines long. Even if I have a template I can copy around, it’s gross from a DRY perspective.

    I want my appliance builder to be configured through a config format that abstracts that away. I only want to specify that which can’t be reasonably guessed, not everything that I might want to have a say about.

All of the virtualization projects I’m involved in right now – Invirt, Virtigo, and some smaller personal projects – could really benefit from this kind of infrastructure piece, which means I’m likely to attempt to write it if it doesn’t exist. And as far as I know, this kind of appliance building application doesn’t exist for Debian and Ubuntu, at the very least. I’ll admit that I know almost nothing about other Linux distributions. Do any of them get this more right?

Posted by evan at 2:19 am Tagged with: , , , ,

Installing GRUB onto a Disk Image

planet sipb, posts 9 Responses »
Aug 042009

As part of my summer internship, I needed to write an installer for VMs. For various reasons, I wasn’t able to use the multitude of VM installers already out there, but one thing I noticed is that most of them don’t actually install a bootloader. They create a /boot/grub/menu.lst, but never run grub-install.

Turns out this is because it’s hard to do. grub-install is very complicated and seems to be pretty explicitly designed for the case of running in an installer environment, where all of the disks and block devices are laid out the same way as they will be the next time you boot. When you’re installing in a host into a loop mount or something, that’s definitely not the case.

In trying to make this work, I discovered a few core issues:

  • grub-install assumes that the block device you’re installing onto “looks like” the sort of device you’d normally install GRUB onto (i.e. is named like a hard disk or floppy – hda, sda, fd0, etc.)
  • grub-install uses df to determine the block device a given file or directory’s filesystem is on. That works really poorly when you’re already chrooting into your loop mount.

If you read my wording carefully, you might see where I’m going with this. In order to get grub-install to work, I needed to convince it it’s installing onto a hard drive, and I needed to run it outside of the loop mount.

The former is obviously a bit more challenging, and to accomplish that, I used the device-mapper to create a node named something like /dev/mapper/hda.

I’ve only tested this on an Ubuntu Jaunty host so far, so I can’t guarantee that it works on Debian or even other Ubuntu versions, but I think it should. I’d love to hear if you have good or bad experiences on other Linux versions.

Here’s roughly how it works (you’ve probably performed some of these steps already in the process of running an installer):

  1. Loop mount your partitioned disk image:
    mathias:~ evan$ sudo losetup --show --find disk.img
/dev/loop0
  2. To setup the device map, you’ll need the major and minor numbers of the loop device, and the size (in bytes) of the disk. The latter is easiest to get from the disk image file, instead of from the loop device (emphasis mine):
    mathias:~ evan$ ls -l /dev/loop0
brw-rw---- 1 root disk 7, 0 2009-07-18 11:27 /dev/loop0
mathias:~ evan$ ls -l disk.img
-rw-r--r-- 1 evan evan 10737418240 2009-08-04 15:28 disk.img
  3. Create a device-mapper node. Any name of the form hd[a-z], sd[a-z], or vd[a-z] will work. Others might as well. The size of the disk should be converted to 512-byte sectors, and the device numbers for the loop device should be in the form major:minor. This will create a new device node in /dev/mapper:
    mathias:~ evan$ echo '0 20971520 linear 7:0 0' | sudo dmsetup create hda
mathias:~ evan$ ls -l /dev/mapper/hda
brw-rw---- 1 root disk 252, 4 2009-08-04 15:36 /dev/mapper/hda
  4. Use kpartx to create device-mapper nodes for the partitions on the disk image:
    mathias:~ evan$ sudo kpartx -a /dev/mapper/hda
mathias:~ evan$ ls -l /dev/mapper/hda*
brw-rw---- 1 root disk 252, 4 2009-08-04 15:36 /dev/mapper/hda
brw-rw---- 1 root disk 252, 5 2009-08-04 15:38 /dev/mapper/hda1
brw-rw---- 1 root disk 252, 6 2009-08-04 15:38 /dev/mapper/hda2
  5. Mount the root partition onto a tempdir (note: this is not a loop mount, because the kernel already thinks this is a real block device):
    mathias:~ evan$ mktemp -d
/tmp/tmp.MPUXeJWqpn
mathias:~ evan$ sudo mount /dev/mapper/hda1 /tmp/tmp.MPUXeJWqpn
  6. Create a fake device.map for grub-install to use (yeah, this is a bad use of tee, but I’m trying to be clear about what I’m doing):
    mathias:~ evan$ echo '(hd0) /dev/mapper/hda' | sudo tee /tmp/tmp.MPUXeJWqpn/boot/grub/device.map
(hd0) /dev/mapper/hda
  7. And now, for the grand finale, actually install GRUB from outside the chroot:
    mathias:~ evan$ sudo grub-install --root-directory=/tmp/tmp.MPUXeJWqpn /dev/mapper/hda
grub-probe: error: no mapping exists for `hda1'
[: 494: =: unexpected operator
Installing GRUB to /dev/mapper/hda as (hd0)...
Installation finished. No error reported.
This is the contents of the device map /tmp/tmp.MPUXeJWqpn/boot/grub/device.map.
Check if this is correct or not. If any of the lines is incorrect,
fix it and re-run the script `grub-install'.

(hd0) /dev/mapper/hda

    (You don’t need to worry about those two errors at the beginning of the output – it’s some logic specialized for XFS filesystems)

  8. Cleanup the mess you made:
    mathias:~ evan$ sudo umount /tmp/tmp.MPUXeJWqpn
mathias:~ evan$ sudo rm -rf /tmp/tmp.MPUXeJWqpn
mathias:~ evan$ sudo kpartx -d /dev/mapper/hda
mathias:~ evan$ sudo dmsetup remove hda
mathias:~ evan$ sudo losetup -d /dev/loop0
  9. Finally, examine your disk image, and see that it definitely has GRUB installed:
    mathias:~ evan$ file disk.img
disk.img: x86 boot sector; GRand Unified Bootloader, stage1 version 0x3, 1st sector stage2 0x884009; partition 1: ID=0x83, active, starthead 0, startsector 1, 18876374 sectors; partition 2: ID=0x82, starthead 254, startsector 18876375, 2088450 sectors

And there you have it! You will, of course, still need to write out GRUB’s menu.lst through some other means (such as Debian/Ubuntu’s update-grub).

Posted by evan at 6:22 pm Tagged with: , , , ,

Another Summer in California

posts 1 Response »
Jul 162009

I haven’t updated in a while. That seems to be the norm. I may stop commenting on it every time I don’t update for 6 months. It turns out that daily updates on my life aren’t usually that interesting, so it really seems better to have a lot of news to share at once.

But now that I’ve just finished my mid-term evaluations for my summer job, it seems like a great time to say what I’m doing this year!

This summer, I’m working at Google, in Corporate Engineering, which is totally less glamorous than Engineering proper, but because most of Corp Eng uses more traditional technologies, it’s also more directly applicable to the world outside of Google.

I can thank Tim Abbott for referring me for this job. Since I also blame him for dragging me into SIPB in the first place (which subsequently absorbed my life), I figure that puts us at about even.

Working for Google really is a lot more fun than I expected it to be. I think a lot of that is that (and this is the great irony, really), in spite of working for the biggest web company in the world, I’m not doing web development. The environment is very casual, my supervisor is cool, the food is great (I totally underestimated how much the food contributes to making things pleasant). Before coming here, I don’t think I would have considered trying to get a full-time job here, but I’m definitely giving it serious consideration now.

As for what I’m doing, I can actually talk about that. I’m spending the summer writing a new type of test framework. Because of how traditional software testing works, it can be incredibly difficult to test software when it wasn’t written from the ground up to be testable. This is a real problem for a lot of common open-source projects. And even if the code is kind of testable, it frequently is so thoroughly stubbed and mocked and neutered that it’s hard to draw any conclusions about the functionality of the code from whether it passes the tests. The moral is that there is just no substitute for actually running code in situ.

So that’s what I’m trying to do – make it easier to run code in situ. The goal of my summer project is to let developers build the entire environment under test in a series of virtual machines. These machines have the OS installed on demand, and are network isolated from the host and the outside world. At the end of the test, the resulting state of the VMs is destroyed, so tests can be arbitrarily destructive. They’re repeatable and safe, even if the test itself isn’t trusted. Most importantly, they let you come pretty arbitrarily close to running your code live.

The project is open source (GPL2), and currently being hosted on code.google.com. I think this is way better than working on something internal to Google. I can point people at what I’ve actually done, I can try to get it picked up outside of here (as well as within), and I can keep contributing once I’ve left (which I plan to do, because I’m convinced that it’s worth spending time on).

The one catch is the name. For various reasons, we’re grouping my project under Debmarshal, which was originally designed for managing repositories of Debian packages. But both my supervisor and I want to switch this to another name…we just don’t have one to switch to. So this is where you, my two readers, come in – what’s a good name for a virtualization-based testing framework? It needs to be something relatively unused. I tried the common trick for virtualization projects: taking a word with “vert” in it and replacing that with “virt”, but all the reasonable names seem to be taken (divirt, convirt, revirt, Invirt, and even IntroVirt). So…any suggestions? Come up with something we use and I’ll buy you lunch the next time I see you, or something.

So that’s work. I’m living right in the same area as last summer (within a few mile radius of Google), although more in the Palo Alto area than Mountain View. I’m living with friends from school, which always trumps random roommates off of Craigslist. We have a giant house, with a pool. It’s a pretty good setup.

And…looks like it’s dinner time here, so I’m out.

Posted by evan at 8:39 pm Tagged with: , , , ,
Older Entries
© 2012 No Name Blog Suffusion theme by Sayontan Sinha