Threadripper - Graphics Card PassthroughMon, Mar 5, 2018
Getting GPU passthrough to work with a host containing two NVIDIA GTX1080TI cards on Threadripper
Note, These are somewhat old and there may be better ways available. See level1techs.com for more information. This tutorial may work better for you. https://forum.level1techs.com/t/play-games-in-windows-on-linux-pci-passthrough-quick-guide/108981 Also here for the patched information. https://forum.level1techs.com/t/threadripper-reset-fixes/123937
HOST : System Hosting the VM’s VM : System running as a VM
System tutorial is for Fedora 27 Patching the kernel: This is the source for the patching instructions. https://forum.level1techs.com/t/threadripper-reset-fixes/123937
Install required packages
sudo dnf install fedpkg fedora-packager rpmdevtools ncurses-devel pesign sudo dnf install rpm-build flex perl-devel perl-generators openssl-devel hmaccalc elfutils-devel
Make a directory to work on kernel source within
mkdir src cd src
Anonymously clone the fedora kernel and go to that directory
fedpkg clone -a kernel cd kernel
Switch to the branch f27
fedpkg switch-branch f27
Within the kernel.spec define your own kernel name change the ‘# define buildid .local’ to ‘%define buildid .threadripper’
Adding the kernel patch Download the patch and save it as ‘tr.patch’ within the ‘src/kernel’ directory Threadripper iommu patch
Within the file ‘kernel.spec’ look for the line ‘# END OF PATCH DEFINITIONS’ and add the following right above it.
Patch999: tr.patch # END OF PATCH DEFINITIONS
Perform the compile and install of your custom kernel. Use htop to watch those threads work!!!
Picking the correct card.
If you already have the nvidia drivers installed on the HOST you may use nvidia settings to grab the pci numbers
All boot devices, but it doesn’t single out any of them.
‘find /sys/devices/pci* -name boot_vga’
command to list pci devices:
My script located in:
Change the permissions to make it executable
chmod 755 /usr/sbin/vfio-pci-override-vga.sh
#!/bin/sh echo "vfio-pci" > /sys/devices/pci0000:00/0000:00:03.1/0000:08:00.0/driver_override echo "vfio-pci" > /sys/devices/pci0000:00/0000:00:03.1/0000:08:00.1/driver_override DEVS=0000:00:08.0 for DEV in $DEVS; do echo "vfio-pci" > /sys/bus/pci/devices/$DEV/driver_override done modprobe -i vfio-pci
Now we need dracut to refernece the file and address the gpu hardware id’s in file /etc/dracut.conf.d/vfio.conf add the following
add_drivers+=“vfio vfio_iommu_type1 vfio_pci vfio_virqfd” install_items+=“/sbin/vfio-pci-override-vga.sh”
Verification: First off you will see your HOST desktop on boot up just as you always did. If not then you messed up.
If it worked then verify the secondary card worked correctly. Run the lspci command, you will see the card to be dedicated to the VM with the words ‘Kernel driver in use: vfio-pci’ under its listing.
Installing windows 8.1
Need to select UEFI during the ‘New VM’ creation process. On Step 5 of the ‘Create a new virtual machine’ dialog make sure to check the “Customize configuration before install”. After clicking finish Select the ‘Overview’ item on the left hand side. Under ‘Hypervisor Details’ ‘Firmware’ dropdown select a UEFI firmware.
dnf install edk2-ovmf edk2-ovmf-ia32 systemctl restart libvirtd
Setting up the VM, If you do not do this the NVIDIA driver will disabled the video card with an error 43. Use Device Manager to view this. Instructions from https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#.22Error_43:_Driver_failed_to_load.22_on_Nvidia_GPUs_passed_to_Windows_VMs
To get the virtio drivers, install the driver disk. Follow the instructions for installing the repo, virtio-win package, and enabling the virtio-win-latest https://docs.fedoraproject.org/quick-docs/en-US/creating-windows-virtual-machines-using-virtio-drivers.html it should allow for access to an ISO with the virtio drivers that you may then use on the VM.