Hi,
I am highly interested in kfreebsd port, and even few times
tested it, it is going better and better every day.
Thanks.
I wanted to ask if anybody is going to look at kfreeBSD Xen [1]
support in Squeuzee? (like xen-tools cross-installer,
or easly available kernels for i386 and amd64 for Linux [as .deb]
so they can be easly booted, network setuped, and so on).
Two weeks ago i managed to run such setup:
dom0:
* 2x Intel Xeon II 460 MHZ with 2GB
* Debian GNU/Linux i386 Lenny
* Linux 2.6.26-2-xen-686
* Xen version 3.2-1
domU:
* FreeBSD-current from May 2009 (prebuild bziped 100MB image [2a])
* swap, root and usr disk image as files on ext3 partition
* booting working correclty
* working xen console
* bridged networking working
* SSH server workin
* portsnap working and compiling few things
========================================================================
========================================================================
========================================================================
root@noisy:~# cat /etc/xen/freebsd-current.cfg
bootloader = "/usr/lib/xen-3.2-1/bin/pygrub"
memory = 128
name = "freebsd_current"
#vif = [ 'mac=00:bd:c4:13:00:01,bridge=xenbr0' ]
#vif = [ 'type=ioemu, mac=00:16:3e:8a:c1:52, bridge=br0, model=ne2k_pci' ]
vif = [ 'mac=00:16:3e:8a:c1:52,bridge=br0' ]
disk = [
'file:/usr/xen-domains/freebsd_current/root-head-r192577.img,0xCA00,w',
'file:/usr/xen-domains/freebsd_current/usr2.img,0xCA20,w',
'file:/usr/xen-domains/freebsd_current/swap.img,0xCA10,w',
# 'phy:/dev/hosting_5_D1/XEN_freebsd_current_root,0xCA00,w'
# 'phy:/dev/hosting_5_D1/XEN_freebsd_current_swap,0xCA10,w'
]
#on_crash = 'destroy'
on_crash = 'preserve'
on_shutdown = 'destroy'
on_reboot = 'restart'
extra = ""
extra += ",vfs.root.mountfrom=ufs:xbd0,kern.hz=100,boot_verbose=1"
root@noisy:~#
========================================================================
========================================================================
========================================================================
After few experiments with finding proper vif parameters, and disk ids, and extra
flags freebsd is very stable. (Othere configurations panics kfreebsd,
on boot or on network setup):
========================================================================
========================================================================
========================================================================
root@noisy:~# xm create freebsd_current.cfg -c
pyGRUB version 0.6
┌────────────────────────────────────────────────────────────────────────┐
│ FreeBSD │
│ │
│ │
│ │
│ │
│ │
│ │
│ │
└────────────────────────────────────────────────────────────────────────┘
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS. 'e' to edit the
commands before booting, 'a' to modify the kernel arguments
before booting, or 'c' for a command line.
Started domain freebsd_current
WARNING: loader(8) metadata is missing!
GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2009 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.0-CURRENT #0: Fri May 22 15:45:51 UTC 2009
adrian@agnus.home.cacheboy.net:/home/adrian/work/freebsd/xen/obj-head/home/adrian/work/freebsd/xen/svn/head/sys/XEN
WARNING: WITNESS option enabled, expect reduced performance.
Xen reported: 450.122 MHz processor.
Timecounter "ixen" frequency 1000000000 Hz quality 0
CPU: Pentium II/Pentium II Xeon/Celeron (450.12-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x653 Stepping = 3
Features=0x183fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR>
Instruction TLB: 4 KB pages, 4-way set associative, 32 entries
Instruction TLB: 4 MB pages, fully associative, 2 entries
Data TLB: 4 KB pages, 4-way set associative, 64 entries
2nd-level cache: 512 KB, 4-way set associative, 32 byte line size
1st-level instruction cache: 16 KB, 4-way set associative, 32 byte line size
Data TLB: 4 MB Pages, 4-way set associative, 8 entries
1st-level data cache: 16 KB, 4-way set associative, 32 byte line size
real memory = 134217728 (128 MB)
Physical memory chunk(s):
0x000000000064c000 - 0x0000000007d7dfff, 124985344 bytes (30514 pages)
avail memory = 123322368 (117 MB)
APIC: Using the MPTable enumerator.
SMP: Added CPU 0 (BSP)
ULE: setup cpu 0
cpu=0 irq=0 vector=0
cpu=0 irq=0 vector=1
Event-channel device installed.
random: <entropy source, Software, Yarrow>
kbd0 at kbdmux0
mem: <memory>
Pentium Pro MTRR support enabled
nfslock: pseudo-device
null: <null device, zero device>
io: <I/O>
Grant table initialized
xenbus0: <Xen Devices> on motherboard
xc0: <Xen Console> on motherboard
npx0: INT 16 interface
Device configuration finished.
procfs registered
Timecounters tick every 10.000 msec
lo0: bpf attached
xbd0: 512MB <Virtual Block Device> at device/vbd/51712 on xenbus0
GEOM: new disk xbd0
GEOM_LABEL: Label for provider xbd0 is ufsid/4a16ce286b8b4567.
xbd2: 2000MB <Virtual Block Device> at device/vbd/51744 on xenbus0
GEOM: new disk xbd2
xbd1: 512MB <Virtual Block Device> at device/vbd/51728 on xenbus0
xn0: <Virtual Network Interface> at device/vif/0 on xenbus0
xn0: bpf attached
xn0: Ethernet address: 00:16:3e:8a:c1:52
GEOM_LABEL: Label for provider xbd2 is ufsid/4aafb30fe1ddc483.
GEOM: new disk xbd1
WARNING: WITNESS option enabled, expect reduced performance.
flowtable cleaner started
Trying to mount root from ufs:xbd0
warning: no time-of-day clock registered, system time will not be set accurately
start_init: trying /sbin/init
No suitable dump device was found.
Entropy harvesting: interrupts ethernet point_to_point kickstart.
GEOM_LABEL: Label ufsid/4a16ce286b8b4567 removed.
/dev/xbd0: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/xbd0: clean, 320834 free (258 frags, 40072 blocks, 0.0% fragmentation)
GEOM_LABEL: Label for provider xbd0 is ufsid/4a16ce286b8b4567.
GEOM_LABEL: Label ufsid/4a16ce286b8b4567 removed.
GEOM_LABEL: Label ufsid/4aafb30fe1ddc483 removed.
kldload: can't load zfs: No such file or directory
/etc/rc: WARNING: Unable to load kernel module zfs
Starting Network: lo0 xn0.
add net default: gateway 10.0.1.1
/etc/rc: WARNING: Dump device does not exist. Savecore not run.
Tue Sep 15 15:37:10 UTC 2009
FreeBSD/i386 (freebsd-current) (xc0)
login: root
Password:
Sep 15 15:37:16 freebsd-current login: ROOT LOGIN (root) ON xc0
Last login: Tue Sep 15 15:36:52 on xc0
Copyright (c) 1992-2009 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD 8.0-CURRENT (XEN) #0: Fri May 22 15:45:51 UTC 2009
Welcome to FreeBSD!
Before seeking technical support, please use the following resources:
o Security advisories and updated errata information for all releases are
at http://www.FreeBSD.org/releases/ - always consult the ERRATA section
for your release first as it's updated frequently.
o The Handbook and FAQ documents are at http://www.FreeBSD.org/ and,
along with the mailing lists, can be searched by going to
http://www.FreeBSD.org/search/. If the doc distribution has
been installed, they're also available formatted in /usr/share/doc.
If you still have a question or problem, please take the output of
`uname -a', along with any relevant error messages, and email it
as a question to the questions@FreeBSD.org mailing list. If you are
unfamiliar with FreeBSD's directory layout, please refer to the hier(7)
manual page. If you are not familiar with manual pages, type `man man'.
You may also use sysinstall(8) to re-enter the installation and
configuration utility. Edit /etc/motd to change this login announcement.
freebsd-current# cat /etc/rc.conf
hostname="freebsd-current"
ifconfig_xn0="inet 10.0.1.147 netmask 255.255.255.0"
defaultrouter="10.0.1.1"
zfs_enable="YES"
freebsd-current# uptime
3:32PM up 8 days, 1:30, 1 user, load averages: 0.05, 0.01, 0.00
freebsd-current#
========================================================================
========================================================================
========================================================================
So i have also working minimal SSH server on FreeBSD on Xen on Debian/Linux.
Still there is few problems:
* It is FreeBSD, not Debian kFreeBSD
* I have no knowledge how to build image manually:
it will need: building ufs file system on i386 linux, mounting it,
and then performing (cross)debootstrap on different architecture,
something which can be very fragile.
mayby we would need some prebuild images, or delegate image
creation to speciall domU kfreebsd only for this purpose (building images),
so debootstrap inside it can be run without problem (including ufs, swap or zfs,
file system creation and data manipulation).
* It is not very well supported.
* Some bugs:
- after "poweroff" or "reboot" inside freebsd guest, virtual
machine doesn't really reboot or destroy,
I need to manually perform "xm destroy freebsd_current;
xm create freebsd_current.cfg"
- clock is constantly changing back to ~ "Tue Sep 15 15:31:59 UTC 2009",
even after changing it manually to some sane value, with
`date 200912090303`, it goes back to ~ "Tue Sep 15 15:31:42 UTC 2009",
after no more than one minute, in fact it goes back even without changing,
anything:
freebsd-current# date
Tue Sep 15 15:32:11 UTC 2009
freebsd-current# date
Tue Sep 15 15:32:11 UTC 2009
freebsd-current# date
Tue Sep 15 15:32:12 UTC 2009
freebsd-current# date
Tue Sep 15 15:32:12 UTC 2009
freebsd-current# date
Tue Sep 15 15:31:08 UTC 2009
sometimes clock will increse monotonically for few minutes (no more than 15:40),
and then for no reason go back. I haveno ntp client or server running.
This makes `make` compilation process very problematic.
dmesg says
warning: no time-of-day clock registered, system time will not be set accurately
but this happens only once. there is no informations about clock going backward.
(there are also broadcast messages from `date` to all consoles, after manual
changes).
- zfs is not working at all.
# zfs
internal error: failed to initialize ZFS library
(but /lib/lib{zfs,zpool}.so.1 exists)
# truss zfs
....
open("/dev/zfs",O_RDWR,05001302571) ERR#2 'No such file or directory'
modfind(0x280c8089,0x2,0x28058579,0x28092000,0x258,0x280c8089) ERR#2 'No such file or directory'
kldload("zfs") ERR#2 'No such file or directory'
open("/dev/zfs",O_RDWR,05001302571) ERR#2 'No such file or directory'
....
# find / -iname '*zfs*'
/boot/zfs
/boot/zfsboot
/boot/gptzfsboot
/etc/periodic/daily/404.status-zfs
/etc/rc.d/zfs
/etc/zfs
/lib/libzfs.so.1
/sbin/zfs
/usr/lib/libzfs.a
/usr/lib/libzfs_p.a
/usr/lib/libzfs.so
/usr/share/man/man8/zfs.8.gz
Where is kernel module?
- dtrace is also not working.
# dtrace -l
dtrace: failed to initialize dtrace: DTrace device not available on system
(again /lib/libdtrace.so.1 exists)
# truss dtrace -l
....
open("/dev/dtrace/dtrace",O_RDWR,00) ERR#2 'No such file or directory'
open("/dev/dtrace/fasttrap",O_RDWR,00) ERR#2 'No such file or directory'
....
Next week I would like to try running the same (or newer) image,
inside 2.6.33-git+pv_ops (it is new dom0 support from scratch for Linux),
(Xen for 2.6.26 in Lenny is really backported from 2.6.18 and pretty old).
Hopefully xen_dom0_pv_ops will be included in 2.6.34 :) [4]
Currently it isn't easy to run dom0 in testing or unstable, in
Lenny it is just working.
I will be also testing ZFS inside guest, and trying export it via NFS4 back to the
host dom0 :) [3] Then I would like to test performance, and Jails.
I'm also curious if live migration is working.
I would also appreciate any clues how to run kfreebsd directly
on Xen (based on mini.iso ?), not freebsd-current (additionally outdate).
BTW. I'm only interested in Xen paravirtualized setup,
i don't have hardware virtualization support, so KVM / HVM
is no go for me. I also like Xen approach, and for it have
very good performance and companion tools which makes
maintaining VM very easy.
PS. Apparently few mentioned problems, are already being solved:
o kern/140313 xen [xen] [panic] FreeBSD8 RC2 as PV domU crashes during c
o kern/135421 xen [xen] FreeBSD Xen PVM DomU network failure - netfronc.
o kern/135179 xen [xen] Xen domU does not properly reboot
o kern/135178 xen [xen] Xen domU outgoing data transfer stall when TSO i
o kern/135069 xen [xen] FreeBSD-current/Xen SMP doesn't function at all
o kern/135008 xen [xen] FreeBSD-current/Xen timecounter jumps
o kern/134926 xen [xen] [panic] FreeBSD-current Xen DomU networking pani
Usefull links:
[1] http://wiki.freebsd.org/FreeBSD/Xen
[2] http://wiki.freebsd.org/AdrianChadd/XenImages
[2a] http://people.freebsd.org/~adrian/xen/root-head-r192577.img.gz
[3] http://wiki.freebsd.org/AdrianChadd/XenHackery
[4] http://wiki.xensource.com/xenwiki/XenParavirtOps
[5] http://lists.freebsd.org/mailman/listinfo/freebsd-xen
--
Witold Baryluk
JID: witold.baryluk // jabster.pl
Attachment:
signature.asc
Description: Digital signature