The Virtual Dmabuf or Virtio based Dmabuf (Vdmabuf) driver can be used to "transfer" a page-backed dmabuf created in the Guest to the Host without making any copies. This is mostly accomplished by recreating the dmabuf on the Host using the PFNs and other meta-data shared by the guest. A use-case where this driver would be a good fit is a multi-GPU system (perhaps one discrete and one integrated) where one of the GPUs does not have access to the display/connectors/outputs. This could be an embedded system design decision or a restriction made at the firmware/BIOS level or perhaps the device is setup in UPT (Universal Passthrough) mode. When such a GPU is passthrough'd to a Guest OS, this driver can help in transferring the scanout buffer(s) (rendered using the native rendering stack) to the Host for the purpose of displaying them. Or, quite simply, this driver can be used to transfer a dmabuf created by an application running on the Guest to another application running on the Host.
The userspace component running in the Guest that transfers the dmabuf is referred to as the producer or exporter and its counterpart running in the Host is referred to as importer or consumer. For instance, a Wayland compositor would potentially be a producer and Qemu UI would be a consumer. It is the producer's responsibility to not reuse or destroy the shared buffer while it is still being used by the consumer. The consumer would send a release cmd indicating that it is done after which the shared buffer can be safely used again by the producer. One way the producer can prevent accidental re-use of the shared buffer is to lock the buffer when it exports it and unlock it after it gets a release cmd. As an example, the GBM API provides a simple way to lock and unlock a surface's buffers.
For each dmabuf that is to be shared with the Host, a 128-bit unique ID is generated that identifies this buffer across the whole system. This ID is a combination of the Qemu process ID, a counter and a randomizer. We could potentially use UUID API but we currently use the above mentioned combination to identify the source of the buffer at any given time for potential bookkeeping.
A typical cycle starts with the producer or exporter calling the alloc_fd IOCTL to get a new fd/dmabuf from Vdmabuf. It would then import and render to it and finally exports it by calling the export IOCTL. A new unique ID is generated for this buffer and it gets registered with the Host. The Host then alerts the consumer or importer by raising an event and shares the ID. In response, the consumer calls the import IOCTL using the ID and gets a newly created dmabuf fd in return. After it is done using the dmabuf, the consumer finally calls the release IOCTL and the Guest is notified which in turn notifies the producer letting it know that the buffer is now safe to reuse.
v2: - Added a notifier mechanism for getting the kvm pointer. - Added start and stop routines in the Vhost backend. - Augmented the cover letter and made some minor improvements.
v3: - Refactored the code to make it similar to vsock. - Used two virtqueues instead of one for efficient two-way communication.
v4: - Made Vdmabuf guest driver allocate the dma-buf and backing storage and exported it to Weston to be used as a render target. (Gerd)
The Vdmabuf driver was tested using Weston (headless) and Qemu from here: https://gitlab.freedesktop.org/Vivek/weston/-/blob/vdmabuf/libweston/backend...
https://lists.nongnu.org/archive/html/qemu-devel/2021-02/msg02976.html
TODO: - Use dma_fences to improve synchronization for multiple importers. - Ensure that a process other than Qemu can also be the importer on Host.
Other Considerations: - Should virtio-gpu be augmented to provide the same functionality as vdmabuf? - How can virtio-gpu/vdmabuf work with a Windows Guest? - Should there be a Vhost backend for virtio-gpu to reduce overhead? - Should a transfer of a dma-buf from Guest to Host be dependent on a DRM driver (virtio-gpu)?
Vivek Kasireddy (3): kvm: Add a notifier for create and destroy VM events virtio: Introduce Vdmabuf driver vhost: Add Vdmabuf backend
drivers/vhost/Kconfig | 9 + drivers/vhost/Makefile | 3 + drivers/vhost/vdmabuf.c | 1372 +++++++++++++++++++++++++++ drivers/virtio/Kconfig | 8 + drivers/virtio/Makefile | 1 + drivers/virtio/virtio_vdmabuf.c | 1105 +++++++++++++++++++++ include/linux/kvm_host.h | 5 + include/linux/virtio_vdmabuf.h | 287 ++++++ include/uapi/linux/vhost.h | 3 + include/uapi/linux/virtio_ids.h | 1 + include/uapi/linux/virtio_vdmabuf.h | 87 ++ virt/kvm/kvm_main.c | 20 +- 12 files changed, 2899 insertions(+), 2 deletions(-) create mode 100644 drivers/vhost/vdmabuf.c create mode 100644 drivers/virtio/virtio_vdmabuf.c create mode 100644 include/linux/virtio_vdmabuf.h create mode 100644 include/uapi/linux/virtio_vdmabuf.h