Re: KVM: GPU passthrough

[Christian Seiler <christian@iwakd.de>]

Hi there,

Am 2021-04-09 00:37, schrieb Gokan Atmaca:
> error:
> pci,host=0000:01:00.0,id=hostdev0,bus=pci.0,addr=0x9: vfio
> 0000:01:00.0: group 1 is not viable
> Please ensure all devices within the iommu_group are bound to their
> vfio bus driver.

This is a known issue with PCIe passthrough: depending on your
mainboard and CPU, some PCIe devices will be grouped together,
and you will either be able to forward _all_ devices in the
group to the VM or none at all.

(If you have a "server" GPU that supports SR-IOV you'd have
additional options, but that doesn't appear to be the case.)

This will highly depend on the PCIe slot the card is in, as well
as potentially some BIOS/UEFI settings on PCIe lane distribution.

First let's find out what devices are in the same IOMMU group.
From your kernel log:

[    0.592011] pci 0000:00:01.0: Adding to iommu group 1
[    0.594091] pci 0000:01:00.0: Adding to iommu group 1
[    0.594096] pci 0000:01:00.1: Adding to iommu group 1

Could you check with "lspci" what these devices are in your case?

If you are comfortable forwarding the other two devices into the
VM as well, just add that to the list of passthrough devices,
then this should work.

If you need the other two devices on the host, then you need to
either put the GPU into a different PCIe slot, put the other
devices into a different PCIe slot, or find some BIOS/UEFI setting
for PCIe lane management that separates the devices in question
into different IOMMU groups implicitly. (BIOS/UEFI settings will
typically not mention IOMMU groups at all, so look for "lane
management" or "lane distribution" or something along those
lines. You might need to drop some PCIe lanes from other devices
and give them directly to the GPU you want to pass through in
order for this to work, or vice-versa, depending on the specific
situation.)

Note: the GUI tool "lstopo" from the package "hwloc" is _very_
useful to identify how the PCIe devices are organized in your
system and may give you a clue as to why your system is grouped
together in the way it is.

Hope that helps.

Regards,
Christian