Re: How to build XEN dom0 and domU kernels based on 2.6.25?
- To: debian-devel <firstname.lastname@example.org>
- Subject: Re: How to build XEN dom0 and domU kernels based on 2.6.25?
- From: Daniel Widenfalk <Daniel@lilla-graen.c.se>
- Date: Mon, 23 Jun 2008 19:53:56 +0200
- Message-id: <[🔎] 485FE334.email@example.com>
- In-reply-to: <firstname.lastname@example.org>
- References: <email@example.com> <20080507073444.GG20425@edu.joroinen.fi> <firstname.lastname@example.org> <email@example.com> <4826BD12.firstname.lastname@example.org> <email@example.com>
Ian Campbell wrote:
On Sun, 2008-05-11 at 11:32 +0200, Daniel Widenfalk wrote:
Ian Campbell wrote:
On Thu, 2008-05-08 at 14:16 +0200, Daniel Widenfalk wrote:
Ok, so dropping back a step. Let's assume that I build the 3.2.0 XEN
hypervisor and dom0 kernel using 2.6.18 as base. I should then be able to
build domU kernel(s) using the linux-source-2.6.25 files? How?
I can't seem to get CONFIG_XEN set in my .config file. Running "make
menuconfig" does not show any xen-specific options.
Have you enabled CONFIG_PARAVIRT? You might also need to select a 686
kernel flavour for the option to appear. I've attached my config in case
it is useful, it's PAE+XEN.
Also that that kernel only has i386 support for Xen, no 64 bit yet.
Many thanks all,
I've decided to do the following:
Upgrade to Xen 3.2.0 using the 2.6.18 kernel (debian tree) for
my dom0 machines and build 2.6.25 kernels for my domUs. I can't
see any major howlers with this as the domUs won't rely on
anything linux/version specific in the dom0 kernel?
No that should be perfectly fine and is pretty much what I do (I use the
XenSource 2.6.18-xen tree though).
My major problem with 2.6.25 was that I was trying to build a
64-bit kernel which caused the config system to disable Xen.
Quite explainable if xen is not yet available for 64-bit in the
current linux tree...
Yes, there is a tree at
http://git.et.redhat.com/?p=xen-pvops-64.git;a=summary which is the
in-progress tree but it needs pretty specific configuration settings:
I'd just like to come back with some good news! I've (finally)
had some success with a 64-bit 2.6.18 dom0 together with a
32-bit 2.6.25 domU.
I started on an Amd64 machine running (an old) Xen 3.0 with
custom built kernels. Here is how I did it:
0) Save all changes to the network initialization scripts used
1) Shut down all domU machines before updating Xen from 3.0 to
3.2. The reason for this is that Xen 3.0 is bound to Python
2.4 whereas Xen 3.2 is bound to Python 2.5.
If you don't shut down your domU guests before updating Xen,
then you won't be able to shut them down gracefully afterwards!
(Learned that the hard way ;-) If you forget, then you have
to go in and redirect the "python" soft link in /usr/bin to
point at python2.4 instead, then run "xm shutdown -wa" and then
point is back at python2.5.
2) Port your changes to the network initialization scripts!
It seems that the default bridge name is no longer
"xenbr0", it will be "eth0". I had some trouble with this
as I'm running a DMZ-bridge for use by the web and email
servers. Had to update my guest configuration files to
refer to eth0 instead of xenbr0...
3) Reboot your machine using your old dom0 kernel. This should
work without a hitch as long as you're got the network
scripts fixed OK. You may run into problems if your old
kernel had cpu frequency adaption enabled. This shouldn't
be the case as Xen 3.0 doesn't support cpu frequence adaption
but I had applied a patch to allow this which has caused me
a lot of problems.
4) Download the Xen distribution and also the latest 2.6.18
kernel from xensource. Configure and build a new dom0
kernel using the Xen sources.
IMPORTANT: Don't use gcc-4.3 for this! This is what tripped
me up. The kernels I built using gcc-4.3 crashed on boot.
> make linux-2.6-xen-config CONFIGMODE=menuconfig (or xconfig)
> make linux-2.6-xen-build CC=gcc-4.2
> sudo cp dist/install/boot/config* /boot
> sudo cp dist/install/boot/System.map* /boot
> sudo cp dist/install/vmlinuz* /boot
Update your grub-menu with an additional entry using your
newly built kernel.
5) Reboot your machine with your new dom0 kernel. This should,
hopefully, work like a charm. You're now running Xen 3.2
together with a dom0 kernel based on Xen 3.2.
6) Download the linux-tree-2.6.25 (and build dependencies, of
course) package and build/configure your domU kernel:
> MAKEFLAGS="CC=gcc-4.2" make-kpkg --rootcmd fakeroot \
--revision 1 --added-patches debian --append-to-version \
my-32-xenu --cross-compile - --arch i386 \
IMPORTANT: Note the use of gcc-4.2!
IMPORTANT: Enable PAE and 64Gbyte memory or it won't work!
The important point here is to remember:
"--cross-compile - --arch i386"
This tells make-kpkg to build a 32-bit image using the native
amd64 gcc. You'll need to install the "gcc-multilib" package!
7) Install your newly created image
8) Create disk images using, e.g., xen-create-image. Remember to
specify "--arch i386" or you'll get an Amd64 image and that
9) Mount your root image and edit the fstab file. The virtual
block devices are named xvda1, xvda2 etc and not sda1, sda2!
You probably also want to make all other important changes
in etc now before booting your machine for the first time. I
find I usually have to update the network configuration to
get it right the first time.
Remember to unmount your image!
10) Open your Xen domain config file and add a line:
extra = 'console=hvc0'
I've also added a line:
pae = 1
which I'm not 100% sure if it's actually needed.
Make sure your root entry uses xvda1 instead of sda1!
11) Launch your 32-bit guest!
This seems to work fine for me. I haven't yet transitioned
any of my production servers so I can't say anything about
stability and such, but it seems to run fine.
I hope this helps someone!