Hallo Liste,
ich möchte an eine domU ein PCI-Gerät durchreichen, bekomme es aber
nicht hin. https://wiki.xenproject.org/wiki/Xen_PCI_Passthrough habe ich
gefunden, kann es aber nicht so umsetzen, das es auch funktioniert.
Hardware: Asrock j5040-ITX mit Intel(R) Pentium(R) Silver J5040 CPU @ 2.00GHz
~# lspci
00:00.0 Host bridge: Intel Corporation Gemini Lake Host Bridge (rev 06)
00:00.1 Signal processing controller: Intel Corporation Celeron/Pentium Silver Processor Dynamic Platform and Thermal Framework Processor Participant (rev 06)
00:02.0 VGA compatible controller: Intel Corporation GeminiLake [UHD Graphics 605] (rev 06)
00:0f.0 Communication controller: Intel Corporation Celeron/Pentium Silver Processor Trusted Execution Engine Interface (rev 06)
00:12.0 SATA controller: Intel Corporation Celeron/Pentium Silver Processor SATA Controller (rev 06)
00:13.0 PCI bridge: Intel Corporation Gemini Lake PCI Express Root Port (rev f6)
00:13.1 PCI bridge: Intel Corporation Gemini Lake PCI Express Root Port (rev f6)
00:13.2 PCI bridge: Intel Corporation Gemini Lake PCI Express Root Port (rev f6)
00:13.3 PCI bridge: Intel Corporation Gemini Lake PCI Express Root Port (rev f6)
00:15.0 USB controller: Intel Corporation Celeron/Pentium Silver Processor USB 3.0 xHCI Controller (rev 06)
00:1f.0 ISA bridge: Intel Corporation Celeron/Pentium Silver Processor LPC Controller (rev 06)
00:1f.1 SMBus: Intel Corporation Celeron/Pentium Silver Processor Gaussian Mixture Model (rev 06)
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)
04:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)
Den ASMedia SATA controller will ich an die domU durchreichen.
~# cat /etc/apt/sources.list
#
# bookworm (12.x) stable since 10.06.2023
#
deb http://deb.debian.org/debian/ bookworm main non-free-firmware contrib non-free
deb http://security.debian.org/debian-security bookworm-security main non-free-firmware contrib non-free
deb http://deb.debian.org/debian/ bookworm-updates main non-free-firmware contrib non-free
~# cat /etc/default/grub
...
GRUB_CMDLINE_LINUX_DEFAULT="quiet"
GRUB_CMDLINE_LINUX="console=tty0 console=hvc0 console=ttyS0,19200 ipv6.disable=1"
...
~# cat /etc/default/grub.d/xen.cfg
...
GRUB_CMDLINE_XEN_DEFAULT="dom0_mem=4G cpuidle dom0_max_vcpus=2"
GRUB_CMDLINE_XEN="com1=19200,8n1 console=com1,vga"
...
~# cat /proc/cmdline
placeholder root=/dev/mapper/server-root ro console=tty0 console=hvc0 console=ttyS0,19200 ipv6.disable=1 quiet
~# uname -a
Linux j5040 6.1.0-21-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.90-1 (2024-05-03) x86_64 GNU/Linux
~# cat /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
# Parameters can be specified after the module name.
xen_pciback
Dann habe ich mir noch einen eigenen systemd service gebastelt.
~# cat /etc/systemd/system/xenpciback.service
[Unit]
Description=make a device assignable to Xen PCI Passthrough
After=xen.service
Before=xendomains.service
[Service]
Type=oneshot
ExecStart=/usr/sbin/xl pci-assignable-add 04:00.0
[Install]
WantedBy=multi-user.target
Wenn ich dann prüfe, ob das device bereit für passthrough ist
~# xl pci-assignable-list
0000:04:00.0
scheint alles vorbereitet zu sein.
~# cat /etc/xen/omv.cfg
#
# Configuration file for the Xen instance omv, created
# by xen-tools 4.9.2 on Mon Apr 22 18:41:22 2024.
#
#
# Kernel + memory size
#
bootloader = 'pygrub'
vcpus = '2'
memory = '4096'
#
# Disk device(s).
#
root = '/dev/xvda2 ro'
disk = [
'phy:/dev/server/omv-disk,xvda2,w',
'phy:/dev/server/omv-swap,xvda1,w',
]
#
# Physical volumes
#
#
# Hostname
#
name = 'omv'
#
# Networking
#
dhcp = 'dhcp'
vif = [ 'mac=00:16:3E:55:CD:2D' ]
#
# Behaviour
#
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'destroy'
#
# PCI Passthrough
#
pci = [ '04:00.0' ]
Dann starte ich die domU
~# xl -v create /etc/xen/omv.cfg
Parsing config from /etc/xen/omv.cfg
libxl: detail: libxl_create.c:662:libxl__domain_make: passthrough: sync_pt
domainbuilder: detail: xc_dom_allocate: cmdline="root=/dev/xvda2 ro elevator=noop iommu=soft root=/dev/xvda2 ro", features=""
domainbuilder: detail: xc_dom_kernel_mem: called
domainbuilder: detail: xc_dom_module_mem: called
domainbuilder: detail: xc_dom_boot_xen_init: ver 4.17, caps xen-3.0-x86_64 hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
domainbuilder: detail: xc_dom_parse_image: called
domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ...
domainbuilder: detail: loader probe failed
domainbuilder: detail: xc_dom_find_loader: trying HVM-generic loader ...
domainbuilder: detail: loader probe failed
domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ...
domainbuilder: detail: XZ: Saw data stream end
domainbuilder: detail: _xc_try_lzma_decode: XZ decompress OK, 0x7ad1e0 -> 0x3ed8e54
domainbuilder: detail: loader probe OK
xc: detail: ELF: phdr: paddr=0x1000000 memsz=0x18d7068
xc: detail: ELF: phdr: paddr=0x2a00000 memsz=0x63b000
xc: detail: ELF: phdr: paddr=0x303b000 memsz=0x34000
xc: detail: ELF: phdr: paddr=0x306f000 memsz=0x1991000
xc: detail: ELF: memory: 0x1000000 -> 0x4a00000
xc: detail: ELF: note: GUEST_OS = "linux"
xc: detail: ELF: note: GUEST_VERSION = "2.6"
xc: detail: ELF: note: XEN_VERSION = "xen-3.0"
xc: detail: ELF: note: VIRT_BASE = 0xffffffff80000000
xc: detail: ELF: note: INIT_P2M = 0x8000000000
xc: detail: ELF: note: ENTRY = 0xffffffff8306f1c0
xc: detail: ELF: note: HYPERCALL_PAGE = 0xffffffff81a28000
xc: detail: ELF: note: FEATURES = "!writable_page_tables|pae_pgdir_above_4gb"
xc: detail: ELF: note: SUPPORTED_FEATURES = 0x8801
xc: detail: ELF: note: PAE_MODE = "yes"
xc: detail: ELF: note: LOADER = "generic"
xc: detail: ELF: note: unknown (0xd)
xc: detail: ELF: note: SUSPEND_CANCEL = 0x1
xc: detail: ELF: note: MOD_START_PFN = 0x1
xc: detail: ELF: note: HV_START_LOW = 0xffff800000000000
xc: detail: ELF: note: PADDR_OFFSET = 0
xc: detail: ELF: note: PHYS32_ENTRY = 0x1000860
xc: detail: ELF: addresses:
xc: detail: virt_base = 0xffffffff80000000
xc: detail: elf_paddr_offset = 0x0
xc: detail: virt_offset = 0xffffffff80000000
xc: detail: virt_kstart = 0xffffffff81000000
xc: detail: virt_kend = 0xffffffff84a00000
xc: detail: virt_entry = 0xffffffff8306f1c0
xc: detail: p2m_base = 0x8000000000
domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-x86_64: 0xffffffff81000000 -> 0xffffffff84a00000
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_64 <= matches
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32p
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_64
domainbuilder: detail: xc_dom_mem_init: mem 4096 MB, pages 0x100000 pages, 4k each
domainbuilder: detail: xc_dom_mem_init: 0x100000 pages
domainbuilder: detail: xc_dom_boot_mem_init: called
domainbuilder: detail: x86_compat: guest xen-3.0-x86_64, address size 64
domainbuilder: detail: xc_dom_malloc : 8192 kB
domainbuilder: detail: xc_dom_build_image: called
domainbuilder: detail: xc_dom_malloc : 348 kB
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x1000+0x3a00 at 0x7f68e2327000
domainbuilder: detail: xc_dom_alloc_segment: kernel : 0xffffffff81000000 -> 0xffffffff84a00000 (pfn 0x1000 + 0x3a00 pages)
xc: detail: ELF: phdr 0 at 0x7f68e2327000 -> 0x7f68e3bfe068
xc: detail: ELF: phdr 1 at 0x7f68e3d27000 -> 0x7f68e4362000
xc: detail: ELF: phdr 2 at 0x7f68e4362000 -> 0x7f68e4396000
xc: detail: ELF: phdr 3 at 0x7f68e4396000 -> 0x7f68e5d27000
domainbuilder: detail: xc_dom_alloc_page : start info : 0xffffffff84a00000 (pfn 0x4a00)
domainbuilder: detail: xc_dom_alloc_page : xenstore : 0xffffffff84a01000 (pfn 0x4a01)
domainbuilder: detail: xc_dom_alloc_page : console : 0xffffffff84a02000 (pfn 0x4a02)
domainbuilder: detail: count_pgtables: 0x0000007fffffffff/39: 0x0000ff8000000000 -> 0x0000ffffffffffff, 1 table(s)
domainbuilder: detail: count_pgtables: 0x000000003fffffff/30: 0x0000ffff80000000 -> 0x0000ffffbfffffff, 1 table(s)
domainbuilder: detail: count_pgtables: 0x00000000001fffff/21: 0x0000ffff80000000 -> 0x0000ffff84bfffff, 38 table(s)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x4a03+0x29 at 0x7f68f2283000
domainbuilder: detail: xc_dom_alloc_segment: page tables : 0xffffffff84a03000 -> 0xffffffff84a2c000 (pfn 0x4a03 + 0x29 pages)
domainbuilder: detail: xc_dom_alloc_page : boot stack : 0xffffffff84a2c000 (pfn 0x4a2c)
domainbuilder: detail: xc_dom_build_image : virt_alloc_end : 0xffffffff84a2d000
domainbuilder: detail: xc_dom_build_image : virt_pgtab_end : 0xffffffff84c00000
domainbuilder: detail: xc_dom_malloc : 742 kB
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x4c00+0x7bc6 at 0x7f68da761000
domainbuilder: detail: xc_dom_alloc_segment: module0 : 0xffffffff84c00000 -> 0xffffffff8c7c6000 (pfn 0x4c00 + 0x7bc6 pages)
domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x225e1f6 -> 0x7bc5210
domainbuilder: detail: count_pgtables: 0x0000007fffffffff/39: 0x0000008000000000 -> 0x000000ffffffffff, 1 table(s)
domainbuilder: detail: count_pgtables: 0x000000003fffffff/30: 0x0000008000000000 -> 0x000000803fffffff, 1 table(s)
domainbuilder: detail: count_pgtables: 0x00000000001fffff/21: 0x0000008000000000 -> 0x00000080007fffff, 4 table(s)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0xc7c6+0x806 at 0x7f68f093f000
domainbuilder: detail: xc_dom_alloc_segment: phys2mach : 0xffffffff8c7c6000 -> 0xffffffff8cfcc000 (pfn 0xc7c6 + 0x806 pages)
domainbuilder: detail: xc_dom_boot_image: called
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x4a00+0x1 at 0x7f68f27aa000
domainbuilder: detail: start_info_x86_64: called
domainbuilder: detail: setup_hypercall_page: vaddr=0xffffffff81a28000 pfn=0x1a28
domainbuilder: detail: domain builder memory footprint
domainbuilder: detail: allocated
domainbuilder: detail: malloc : 9335 kB
domainbuilder: detail: anon mmap : 62 MB
domainbuilder: detail: mapped
domainbuilder: detail: file mmap : 0 bytes
domainbuilder: detail: domU mmap : 189 MB
domainbuilder: detail: bootlate_pv: shared_info: pfn 0x0, mfn 0x61bc7
domainbuilder: detail: shared_info_x86_64: called
domainbuilder: detail: vcpu_x86_64: called
domainbuilder: detail: vcpu_x86_64: cr3: pfn 0x4a03 mfn 0x468403
domainbuilder: detail: xc_dom_set_gnttab_entry: d1 gnt[0] -> d0 0x468402
domainbuilder: detail: xc_dom_set_gnttab_entry: d1 gnt[1] -> d0 0x468401
domainbuilder: detail: xc_dom_release: called
Die domU stürzt gleich nach dem starten ab, und im log steht nur
~# cat /var/log/xen/xl-omv.log
Waiting for domain omv (domid 1) to die [pid 1385]
Domain 1 has shut down, reason code 3 0x3
Action for shutdown reason code 3 is destroy
Domain 1 needs to be cleaned up: destroying the domain
libxl: error: libxl_device.c:1452:libxl__wait_for_backend: Backend /local/domain/0/backend/pci/1/0 not ready
Done. Exiting now
Ohne PCI Passthrough startet die domU normal und ich kann auf sie
zugreifen.
Meine Frage lautet nun: Was mache ich falsch?
Frohe Pfingsten
Attachment:
signature.asc
Description: PGP signature