https://bugs.freedesktop.org/show_bug.cgi?id=100399
--- Comment #5 from Michel Dänzer michel@daenzer.net --- (In reply to jimijames.bove from comment #4)
Before I switched to AMD, I was passing an NVidia GPU (GTX 660) into my virtual machine, and I could unbind and rebind it between nouveau and vfio-pci as much as I wanted. No trouble at all. Even while X was running, once DRI3 support came.
You can still do that with DRI3, you just have to prevent Xorg from using the secondary GPU, e.g. via
Section "ServerFlags" Option "AutoAddGPU" "off" EndSection
in /etc/X11/xorg.conf.