On Thu, 5 Aug 2021 22:18:56 -0300 Jason Gunthorpe jgg@nvidia.com wrote:
This is in support of Max's series to split vfio-pci. For that to work the reflck concept embedded in vfio-pci needs to be sharable across all of the new VFIO PCI drivers which motivated re-examining how this is implemented.
Another significant issue is how the VFIO PCI core includes code like:
if (pci_dev_driver(pdev) != &vfio_pci_driver)
Which is not scalable if there are going to be multiple different driver types.
This series takes the approach of moving the "reflck" mechanism into the core code as a "device set". Each vfio_device driver can specify how vfio_devices are grouped into the set using a key and the set comes along with a set-global mutex. The core code manages creating per-device set memory and associating it with each vfio_device.
In turn this allows the core code to provide an open/close_device() operation that is called only for the first/last FD, and is called under the global device set lock.
Review of all the drivers show that they are either already open coding the first/last semantic or are buggy and missing it. All drivers are migrated/fixed to the new open/close_device ops and the unused per-FD open()/release() ops are deleted.
The special behavior of PCI around the bus/slot "reset group" is recast in terms of the device set which conslidates the reflck, eliminates two touches of pci_dev_driver(), and allows the reset mechanism to share across all VFIO PCI drivers. PCI is changed to acquire devices directly from the device set instead of trying to work backwards from the struct pci_device.
Overall a few minor bugs are squashed and quite a bit of code is removed through consolidation.
v4:
- Fix use-after-free typo in mbochs error unwind
- Allow mdevs to work when they don't have open/release ops, for bisect-ability
- Redo the vfio_pci_try_bus_reset() patch, make it dev_set centric
- Change VFIO_DEVICE_PCI_HOT_RESET to align with the new vfio_pci_try_bus_reset() design
Applied to vfio next branch for v5.15 with Connie and Christoph's additional Reviewed-bys. Thanks,
Alex