Right now, all of my Xen dom0′s run Ubuntu Hardy with Xen 3.3 from hardy-backports. Before we even talk about making Xen work, that statement bears some looking at.
I use Xen for a variety of reasons. Some are historical – the Invirt Project was built on top of Xen, and migrating away from Xen to a solution like KVM or VMWare would require working with users that are running paravirtualized operating systems. Some are circumstantial – I still have hardware that doesn’t support hardware virtualization.
My reasons for using Ubuntu are far less logical – I know how to use it, and don’t want to learn anything else if I don’t have to.
And I use Xen 3.3 because it’s way more stable than Xen 3.2.
In any case, if you find yourself using Xen 3.3 on Ubuntu Hardy as a dom0, there are a lot of tricks I’ve picked up for making it work better. Over the next few weeks, I’ll be working my way through them. I’ll be tagging them all with xen-tips for easy retrieval later.
As a disclaimer, I have no idea if these problems have been fixed in later versions of Xen or Linux, or if they’re specific to the Xen and/or kernel shipped by Ubuntu. For me, there’s a lot of value in getting all of my software from my distribution, so these instructions are designed to help do that.
I have no idea whose fault this is, but HVM networking just doesn’t seem to work out of the box. qemu-dm, which emulates the VM’s devices, hooks the VM to a tap net device, while Xen sets up networking for a vifN.0 device. As far as I can tell, the intent was to connect the tap and vif devices, but nothing does.
For Invirt, we worked around this by writing a wrapper script around qemu-dm to make sure everything was setup correctly. If you want to use this script, you can drop qemu-dm-invirt in /usr/sbin and qemu-ifup in /etc/xen. (You’ll probably want to replace vif-invirtroute in qemu-ifup with vif-bridge or vif-route or whatever networking script you’re using).
/usr/lib/xen/bin/qemu-dm is hard-coded to run /etc/xen/qemu-ifup, if it exists. Without the qemu-dm-invirt wrapper, though, qemu-ifup doesn’t have any access to the domain ID for the domain it’s setting up. qemu-ifup then sets up and triggers the normal Xen networking script, which repeats the same setup it did for the vifN.0 interface.
Then, in your Xen config file, be sure to set device_model = '/usr/sbin/qemu-dm-invirt'.