From: Pekka Paalanen pekka.paalanen@collabora.com
Set up the expectations on how hot-unplugging a DRM device should look like to userspace.
Written by Daniel Vetter's request and largely based on his comments in IRC and from https://lists.freedesktop.org/archives/dri-devel/2020-May/265484.html .
Signed-off-by: Pekka Paalanen pekka.paalanen@collabora.com Cc: Daniel Vetter daniel@ffwll.ch Cc: Andrey Grodzovsky andrey.grodzovsky@amd.com Cc: Dave Airlie airlied@redhat.com Cc: Sean Paul sean@poorly.run Cc: Simon Ser contact@emersion.fr
---
v2: - mmap reads/writes undefined (danvet) - make render ioctl behaviour driver-specific (danvet) - restructure the mmap paragraphs (danvet) - chardev minor notes (Simon) - open behaviour (danvet) - DRM leasing behaviour (danvet) - added links
Disclaimer: I am a userspace developer writing for other userspace developers. I took some liberties in defining what should happen without knowing what is actually possible or what existing drivers already implement. --- Documentation/gpu/drm-uapi.rst | 102 +++++++++++++++++++++++++++++++++ 1 file changed, 102 insertions(+)
diff --git a/Documentation/gpu/drm-uapi.rst b/Documentation/gpu/drm-uapi.rst index 56fec6ed1ad8..520b8e640ad1 100644 --- a/Documentation/gpu/drm-uapi.rst +++ b/Documentation/gpu/drm-uapi.rst @@ -1,3 +1,5 @@ +.. Copyright 2020 DisplayLink (UK) Ltd. + =================== Userland interfaces =================== @@ -162,6 +164,106 @@ other hand, a driver requires shared state between clients which is visible to user-space and accessible beyond open-file boundaries, they cannot support render nodes.
+Device Hot-Unplug +================= + +.. note:: + The following is the plan. Implementation is not there yet + (2020 May). + +Graphics devices (display and/or render) may be connected via USB (e.g. +display adapters or docking stations) or Thunderbolt (e.g. eGPU). An end +user is able to hot-unplug this kind of devices while they are being +used, and expects that the very least the machine does not crash. Any +damage from hot-unplugging a DRM device needs to be limited as much as +possible and userspace must be given the chance to handle it if it wants +to. Ideally, unplugging a DRM device still lets a desktop to continue +running, but that is going to need explicit support throughout the whole +graphics stack: from kernel and userspace drivers, through display +servers, via window system protocols, and in applications and libraries. + +Other scenarios that should lead to the same are: unrecoverable GPU +crash, PCI device disappearing off the bus, or forced unbind of a driver +from the physical device. + +In other words, from userspace perspective everything needs to keep on +working more or less, until userspace stops using the disappeared DRM +device and closes it completely. Userspace will learn of the device +disappearance from the device removed uevent or in some cases +driver-specific ioctls returning EIO. + +Only after userspace has closed all relevant DRM device and dmabuf file +descriptors and removed all mmaps, the DRM driver can tear down its +instance for the device that no longer exists. If the same physical +device somehow comes back in the mean time, it shall be a new DRM +device. + +Similar to PIDs, chardev minor numbers are not recycled immediately. A +new DRM device always picks the next free minor number compared to the +previous one allocated, and wraps around when minor numbers are +exhausted. + +Requirements for UAPI +--------------------- + +The goal raises at least the following requirements for the kernel and +drivers: + +- The kernel must not hang, crash or oops, no matter what userspace was + in the middle of doing when the device disappeared. + +- All GPU jobs that can no longer run must have their fences + force-signalled to avoid inflicting hangs to userspace. + +- KMS connectors must change their status to disconnected. + +- Legacy modesets and pageflips fake success. + +- Atomic commits, both real and TEST_ONLY, fake success. + +- Pending non-blocking KMS operations deliver the DRM events userspace + is expecting. + +- dmabuf which point to memory that has disappeared will continue to + be successfully imported if it would have succeeded before the + disappearance. + +- Attempting to import a dmabuf to a disappeared device will succeed if + it would have succeeded without the disappearance. + +- Some userspace APIs already define what should happen when the device + disappears (OpenGL, GL ES: `GL_KHR_robustness`_; `Vulkan`_: + VK_ERROR_DEVICE_LOST; etc.). DRM drivers are free to implement this + behaviour the way they see best, e.g. returning failures in + driver-specific ioctls and handling those in userspace drivers, or + rely on uevents, and so on. + +- open() on a device node whose underlying device has disappeared will + fail. + +- Attempting to create a DRM lease on a disappeared DRM device will + fail. Existing DRM leases remain. + +.. _GL_KHR_robustness: https://www.khronos.org/registry/OpenGL/extensions/KHR/KHR_robustness.txt +.. _Vulkan: https://www.khronos.org/vulkan/ + +Requirements for memory maps +---------------------------- + +Memory maps have further requirements. If the underlying memory +disappears, the mmap is modified such that reads and writes will still +complete successfully but the result is undefined. This applies to both +userspace mmap()'d memory and memory pointed to by dmabuf which might be +mapped to other devices. + +Raising SIGBUS is not an option, because userspace cannot realistically +handle it. Signal handlers are global, which makes them extremely +difficult to use correctly from libraries like those that Mesa produces. +Signal handlers are not composable, you can't have different handlers +for GPU1 and GPU2 from different vendors, and a third handler for +mmapped regular files. Threads cause additional pain with signal +handling as well. + .. _drm_driver_ioctl:
IOCTL Support on Device Nodes