Ask Your Question
2

[Request] Modern Guide for KVM/QEMU/PCI pass-through

asked 2016-04-18 13:48:18 -0500

comjf gravatar image

Fellow Fedora enthusiasts,

I've recently become very interested in KVM after setting up an "Unraid" virtualization system to run Windows 10 (strictly for gaming) at the same time as Fedora 23 (for everything else)... and then docker instances on the unraid bland hypervisor.

I have a display port/USB KVM hub (physical switch as opposed to the virtualization technique) that plugs into both VMs. Each VM has it's dedicated GPU and USB controller (PCIe passthrough) which is how I make this all possible, however I find this is an sub-optimal solution for the following reasons:

  • I have to buy unraid (which hey, it's not a problem only 100ish dollars and it offers a great out of the box solution that I've been using the trial key for)
  • The bigger issue of using linux only to host a VM of linux... I'd rather just work on the bare metal and manage virtual instances myself. This also has the added benefit of being able to manage the docker agents from within Fedora.
  • A few audio issues that I believe are due to virtualization (or at least made worse by it). Likely IRQ/MSIx interrupt problems that I've largely eliminated but still are annoying.
  • unraid mainly focuses on being a great NAS solution. I already own a synology and don't need a custom NAS (yet?)
  • The biggest reason, I want to understand how these technologies work, their strengths/limitations/uses etc. I work in tech and staying generally knowledgable has helped me a lot in my career.

Attempt to use Fedora to act as a KVM/QEMU hypervisor with proper device passthrough. I'm running x99 MSI board with a beefy haswell-e chip that supports 40 PCIx lanes so I have a bit to play with. The best I've been able to find are either very old (but excellently written) fedora branded documentation, or help questions that focus on a very small part of the problem. So this can be seen as a generic request for a detailed guide on this kind of virtualization setup.

I spent a considerable amount of time this weekend attempting the project, but couldn't get beyond the very early task of stubbing my unused (by the host) PCI devises. The main way to do this is to boot with grub settings like pci-stub.id= special vendor/device id's but my specific problems is I'm running to GTX 980ti graphics cards with the same identifiers. I want the first to work on Fedora and the second to be stubbed for later use by my VM. I found a work-around using a systemd service that runs on boot and stubs things using a different method: https://gist.github.com/comjf/c6c9670...

However, this doesn't work :( I enable the systemd job that executes this, and the boot freezes. Nothing in the logs that I can find will shed any light (but I'm still getting used to systemd and ... (more)

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
1

answered 2016-04-19 08:25:40 -0500

pbogu gravatar image

As far as I can see all CPUs that are supported by X99 chipset support IOMMU (VT-d) but I don't have live fedora system at hand to check Fedora's 23 kernel does it have IOMMU support turned on by default so you may need to recompile your kernel (check with dmesg | grep -e DMAR -e IOMMU).

Anyway, great guide is provided by KVM themselves: How to assign devices with VT-d in KVM

edit flag offensive delete link more

Comments

1

$ cat /etc/fedora-release Fedora release 23 (Twenty Three)

[ 0.040991] DMAR-IR: IOAPIC id 2 under DRHD base 0xfed91000 IOMMU 1

Striker gravatar imageStriker ( 2016-04-19 09:14:04 -0500 )edit

Thanks for your reply!

In my case the IOMMU group isn't the problem, simply modifying the boot flags to include intel_iommu=on or whatever, did properly load the module as reported by dmesg. However, this is a very important step.

Thanks for the link to documentation on stubbing, it certainly is helpful an I might have missed a step here so I'll be looking into this shortly.

comjf gravatar imagecomjf ( 2016-04-21 12:23:36 -0500 )edit

Question Tools

2 followers

Stats

Asked: 2016-04-18 13:47:00 -0500

Seen: 1,050 times

Last updated: Apr 19 '16