Installing Xen on Ubuntu 9.10
Recently I switched to Ubuntu as the primary operating system on my desktop machine. As part of the switch, I wanted to install Xen to virtualize some additional operating systems. Xen provides very good performance when virtualizing Linux distributions due to paravirtualization. The latest versions can also virtualize certain unmodified guest operating systems on processors that support virtualization.
In a Xen setup, the Xen hypervisor runs directly on the hardware (bare-metal). The first guest operating system (dom0) runs “on top” of Xen and has full access to the underlying hardware. Additional guests (domU) also run on top of the Xen hypervisor, but with limited access to the underlying hardware.
Converting an existing Ubuntu install to a Xen dom0 install requires installing the Xen hypervisor. In previous versions of Ubuntu this could easily be done using apt. Unfortunately no packages exist for Ubuntu 9.10 (Karmic Koala). In this post I’ll describe the steps I took to install Xen from source on Karmic. If you’d rather install via binary packages you’ll need to find a third-party repository.
Install Xen hypervisor and tools
# In order to compile Xen, you must first install some build dependencies:
$ sudo apt-get install bin86 bcc iasl uuid-dev libsdl-dev
$ cd /usr/src
# See http://www.xen.org/products/xen_source.html for latest downloads
$ wget http://bits.xensource.com/oss-xen/release/3.4.2/xen-3.4.2.tar.gz
$ tar -xzf xen-3.4.2.tar.gz
# I created this small patch to adapt Xen to Ubuntu’s layout
$ wget http://www.bltweb.net/patches/xen-3.4.2-ubuntu.patch
# or if you are using Xen 4.0:
$ wget http://www.bltweb.net/patches/xen-4.0.0-ubuntu.patch
$ cd xen-3.4.2
$ patch -p1 < ../xen-3.4.2-ubuntu.patch
$ make dist-xen dist-tools dist-stubdom
$ sudo make install-xen install-tools install-stubdom
Install Xen Kernel Sources
The stock Karmic kernel will not work when running as dom0 in Xen. You must install a new kernel. I chose to compile my own kernel. Compiling a kernel is not for the faint of heart, but isn’t as hard as you might think.
You have two main choices when it comes to a dom0 kernel. The Xen kernel or a dom0 pv-ops kernel. Normally I would recommend using a pv-ops kernel. The same pv-ops kernel can run on bare-metal or under the Xen hypervisor. The pv-ops kernel is going to be the default in future versions of Xen and will most likely be included in the mainline Linux kernel soon.
Unfortunately, the pv-ops kernel will not work with binary graphics drivers provided by Nvidia. Since I have an Nvidia graphics card (and want to use the binary drivers) I need to use the standard Xen kernel. The standard Xen kernel is still version 2.6.18, however luckily Andrew Lyon maintains forward ported patches for Gentoo that we can use on our Ubuntu install.
First things first, let’s get the kernel sources downloaded and patched:
$ cd /usr/src
# Download the vanilla kernel sources from http://www.kernel.org/
$ wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.31.8.tar.bz2
$ tar -xjf linux-2.6.31.8.tar.bz2
$ mv linux-2.6.31.8 linux-2.6.31.8-xen0-amd64
# Download the forward ported xen patches from http://code.google.com/p/gentoo-xen-kernel/downloads/list
$ wget http://gentoo-xen-kernel.googlecode.com/files/xen-patches-2.6.31-10.tar.bz2
$ mkdir xenpatches-v10_2.6.31.8
$ cd xenpatches-v10_2.6.31.8
$ tar -xjf ../xen-patches-2.6.31-10.tar.bz2
$ cd ../
# Apply the xen patches
$ cd linux-2.6.31.8-xen0-amd64
$ for i in `ls ../xenpatches-v10_2.6.31.8/`; do patch -p1 < ../xenpatches-v10_2.6.31.8/"$i"; done
At this point I like to modify the kernel’s “extra version” to clearly identify that this is a dom0 kernel. To do this, modify the Makefile using your favorite text editor and change the EXTRAVERSION line to look like:
VERSION = 2 PATCHLEVEL = 6 SUBLEVEL = 31 EXTRAVERSION = .8-xen0-amd64 NAME = Man-Eating Seals of Antiquity ...
Compile Xen Kernel
Configuring your kernel is left as an exercise for the reader. There is a ton of existing documentation on how to do this. The easiest way to get started is to copy the configuration from the stock Ubuntu kernel and run make menuconfig.
$ make menuconfig
There are some options that you want to be sure you enable:
Processor type and features ---> [*] Enable Xen compatible kernel Networking support ---> Networking options ---> <*> 802.1d Ethernet Bridging Device Drivers ---> XEN ---> [*] Privileged Guest (domain 0) <M> Backend driver support <M> Block-device backend driver <M> Block-device tap backend driver <M> Block-device tap backend driver 2 <M> Network-device backend driver <M> PCI-device backend driver <M> SCSI backend driver <M> USB backend driver <*> Block-device frontend driver <*> Network-device frontend driver <M> SCSI frontend driver <M> USB frontend driver <*> Framebuffer-device frontend driver <*> Keyboard-device frontend driver [*] Disable serial port drivers <*> Export Xen attributes in sysfs Xen version compatibility (3.3.0 and later) ---> 3.3.0 and later
Now compile and install your kernel:
$ export CONCURRENCY_LEVEL=5
$ make
$ sudo make install
$ sudo make modules_install
# Make an initramfs
$ sudo mkinitramfs -o /boot/initrd.img-2.6.31.8-xen0-amd64 2.6.31.8-xen0-amd64
If you compiled the Xen net, block and usb backend drivers as modules, you will probably want to load them by default by adding them to your /etc/modules
file:
# /etc/modules ... # These modules are required for Xen DomUs to work netbk blkbk usbbk
Update Grub
Most existing Xen documentation on the web refers to the first version of Grub. Ubuntu 9.10 ships with Grub 2. Grub 2 works great with Xen, unless you use fakeraid. If you use fakeraid, I would suggest downgrading to Grub 1.
Ubuntu’s update-grub
command won’t recognize your new Xen kernel, but you can easily modify the /etc/grub.d/40_custom
script to manually insert it into your Grub configuration. Here’s what my 40_custom
file looks like:
#!/bin/sh exec tail -n +3 $0 # This file provides an easy way to add custom menu entries. Simply type the # menu entries you want to add after this comment. Be careful not to change # the 'exec tail' line above. menuentry "Ubuntu, Xen 3.4.2 dom0 2.6.31.8-xen0-amd64" { set quiet=1 insmod raid insmod mdraid insmod ext2 set root=(md1) search --no-floppy --fs-uuid --set 31d4873b-fed1-49d9-b37d-89d9f7a46566 multiboot (md1)/xen-3.4.2.gz vga=gfx-1440x900x32 quiet module (md1)/vmlinuz-2.6.31.8-xen0-amd64 root=/dev/mapper/vg0-root root=/dev/mapper/vg0-root ro quiet splash module (md1)/initrd.img-2.6.31.8-xen0-amd64 }
You will need to adjust this for your system. You can use the blkid
command to get the UUID for your disk partitions. You might notice that I passed the root=
parameter twice on the module
line. This may be a bug somewhere in Xen or Grub but I could not get anything to work when only including this parameter once.
You also need to update your default operating system. You may want to wait until after you’ve verified everything works, but eventually you should edit /etc/default/grub
. Mine looks like:
# If you change this file, run 'update-grub' afterwards to update # /boot/grub/grub.cfg. GRUB_DEFAULT="Ubuntu, Xen 3.4.2 dom0 2.6.31.8-xen0-amd64" #GRUB_DEFAULT="0" #GRUB_HIDDEN_TIMEOUT=1 GRUB_HIDDEN_TIMEOUT_QUIET=true GRUB_TIMEOUT="3" GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian` GRUB_CMDLINE_LINUX_DEFAULT="quiet splash" GRUB_CMDLINE_LINUX="" # The resolution used on graphical terminal GRUB_GFXMODE=1440x900x32 # Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux #GRUB_DISABLE_LINUX_UUID=true # Uncomment to disable generation of recovery mode menu entrys #GRUB_DISABLE_LINUX_RECOVERY="true" GRUB_DISABLE_OS_PROBER=true
Now run update-grub
:
Reboot
Cross your fingers, it’s time to reboot into the Xen hypervisor and your new Xen kernel. If you are lucky everything will work and you’ll soon be back to your desktop. You may run into driver issues, in which case you’ll need to boot your original kernel and try your hand at reconfiguring and recompiling your kernel.
If you use Nvidia or ATI binary drivers they won’t work with the Xen kernel (yet). Enabling these is a subject for another day, but it can be done. For now, let’s stay focused on getting Xen fully installed and running.
After rebooting, you can check a couple of things:
2.6.31.8-xen0-amd64
$ sudo cat /proc/xen/capabilities
control_d
Start Xen tools
Once you are running the Xen hypervisor and a Xen compatible dom0 kernel, you need to start the Xen daemon. This is done with:
You can start this automatically by establishing sysvinit links with:
If you want Xen domUs in /etc/xen/auto
to start automatically on boot, set the xendomains
init script to run on boot:
Compile a domU kernel
This step is optional. You may want to compile a kernel for use by your Linux domUs. Many distributions can run as a domU with their stock kernel however if that doesn’t work you can configure them to use a Xen domU kernel.
Configuring a Xen domU kernel is similar to configuring a Xen dom0 kernel:
$ cd /usr/src
$ tar -xjf linux-2.6.31.8.tar.bz2
$ mv linux-2.6.31.8 linux-2.6.31.8-xenU-amd64
# Modify Makefile so EXTRAVERSION = .8-xenU-amd64
$ cp /boot/config-2.6-31-17-generic .config
$ make menuconfig
$ sudo make install
$ sudo make modules_install
When configuring the domU kernel, you’ll want to make sure you enable:
Processor type and features ---> [*] Paravirtualized guest support ---> [*] Xen guest support Device drivers ---> [*] Xen /dev/xen/evtchn device <*> Xen filesystem [*] Create xen entries under /sys/hypervisor
Ready for domUs
Congratulations, your Ubuntu 9.10 Karmic Koala installation is now running as a domain 0 on top of the latest Xen hypervisor. You are ready to start installing domUs.
hmmm, how did you perform the inaltsl in reference to intsalling grub,, and replacing ntldr?, are you saying it is loading the windows media screen and then failing?, or grub is loading and failing? if the first u may have corrupted system files on the windows system partition possibly checkdsk or fixboot will repair,, if the second u should probably boot into ubuntu and fix the grub menu to properly point at the windows active sys partition =) becareful
ο ε ο ε ο ε ο εο ε ο background ου ? ο ε
Hey hey hey, take a gander at what’ you’ve done
Thinking like that shows an expert’s touch
Od jutra rozpoczynam odchudzanie, kto się odchudza ze mna?
Znalazlam w google dobry sposob na chudnięcie, wygoglujcie sobie – xxally mój sukces
w odchudzaniu
Oot ihana Hanna, oikea hyvän olon lähde! Sain tänään palautettua 9 kk ahertamani diplomityöni, huhhuh mikä urakka, onhan se kuitenkin itse kirjoittamani "kirja". Osaat varmaan samaistua tunteisiini
On yhtäaikaa tyhjä, iloinen, helpottunut, ja niin kovin outo…olo. Nyt on pitkästä aikaa mahdollisuus antaa aikaa blogeillekin, tulin tietty hakemaan täältä iloista mieltä! Saat jotenkin vangittua elämän kauneuden kuviisi! -Marissa
Thank You for this clear & honest sharing….. it seems we have a lot in common, including your ankle and me my left foot this June. All experiences, stepping stones on our journey to heal.Good Luck and Blessings to you. Keep well, Brigitte
"The statistics speak for themselves, and so has its chilling effect on America's status as the worlds #1 jailer"…NOT thinking clearly are you?People in this country get the government they deserve by what they do (or don't do) at the polling booth…