On 5/19/20 6:06 AM, Pekka Paalanen wrote:
From: Pekka Paalanen pekka.paalanen@collabora.com
Set up the expectations on how hot-unplugging a DRM device should look like to userspace.
Written by Daniel Vetter's request and largely based on his comments in IRC and from https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.free... .
Signed-off-by: Pekka Paalanen pekka.paalanen@collabora.com Cc: Daniel Vetter daniel@ffwll.ch Cc: Andrey Grodzovsky andrey.grodzovsky@amd.com Cc: Dave Airlie airlied@redhat.com Cc: Sean Paul sean@poorly.run
Disclaimer: I am a userspace developer writing for other userspace developers. I took some liberties in defining what should happen without knowing what is actually possible or what existing drivers already implement.
Documentation/gpu/drm-uapi.rst | 75 ++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+)
diff --git a/Documentation/gpu/drm-uapi.rst b/Documentation/gpu/drm-uapi.rst index 56fec6ed1ad8..80db4abd2cbd 100644 --- a/Documentation/gpu/drm-uapi.rst +++ b/Documentation/gpu/drm-uapi.rst @@ -1,3 +1,5 @@ +.. Copyright 2020 DisplayLink (UK) Ltd.
- =================== Userland interfaces ===================
@@ -162,6 +164,79 @@ other hand, a driver requires shared state between clients which is visible to user-space and accessible beyond open-file boundaries, they cannot support render nodes.
+Device Hot-Unplug +=================
+.. note::
- The following is the plan. Implementation is not there yet
- (2020 May 13).
+Graphics devices (display and/or render) may be connected via USB (e.g. +display adapters or docking stations) or Thunderbolt (e.g. eGPU). An end +user is able to hot-unplug this kind of devices while they are being +used, and expects that the very least the machine does not crash. Any +damage from hot-unplugging a DRM device needs to be limited as much as +possible and userspace must be given the chance to handle it if it wants +to. Ideally, unplugging a DRM device still lets a desktop to continue +running, but that is going to need explicit support throughout the whole +graphics stack: from kernel and userspace drivers, through display +servers, via window system protocols, and in applications and libraries.
+Other scenarios that should lead to the same are: unrecoverable GPU +crash, PCI device disappearing off the bus, or forced unbind of a driver +from the physical device.
+In other words, from userspace perspective everything needs to keep on +working more or less, until userspace stops using the disappeared DRM +device and closes it completely. Userspace will learn of the device +disappearance from the device removed uevent or in some cases specific +ioctls returning EIO.
+This goal raises at least the following requirements for the kernel and +drivers:
+- The kernel must not hang, crash or oops, no matter what userspace was
- in the middle of doing when the device disappeared.
+- All GPU jobs that can no longer run must have their fences
- force-signalled to avoid inflicting hangs to userspace.
+- KMS connectors must change their status to disconnected.
+- Legacy modesets and pageflips fake success.
+- Atomic commits, both real and TEST_ONLY, fake success.
+- Pending non-blocking KMS operations deliver the DRM events userspace
- is expecting.
+- If underlying memory disappears, the mmaps are replaced with harmless
- zero pages where access does not raise SIGBUS. Reads return zeros,
- writes are ignored.
Regarding this paragraph - what about exiting mappings ? In the first patchset we would actively invalidate all the existing CPU mappings to device memory and i think we still should do it otherwise we will see random crashes in applications as was before. I guess it's because TLBs and page tables are not updated to reflect the fact the device is gone.
Andrey
+- dmabuf which point to memory that has disappeared are rewritten to
- point to harmless zero pages, similar to mmaps. Imports still succeed
- both ways: an existing device importing a dmabuf pointing to
- disappeared memory, and a disappeared device importing any dmabuf.
+- Render ioctls return EIO which is then handled in userspace drivers,
- e.g. Mesa, to have the device disappearance handled in the way
- specified for each API (OpenGL, GL ES: GL_KHR_robustness;
- Vulkan: VK_ERROR_DEVICE_LOST; etc.)
+Raising SIGBUS is not an option, because userspace cannot realistically +handle it. Signal handlers are global, which makes them extremely +difficult to use correctly from libraries like Mesa produces. Signal +handlers are not composable, you can't have different handlers for GPU1 +and GPU2 from different vendors, and a third handler for mmapped regular +files. Threads cause additional pain with signal handling as well.
+Only after userspace has closed all relevant DRM device and dmabuf file +descriptors and removed all mmaps, the DRM driver can tear down its +instance for the device that no longer exists. If the same physical +device somehow comes back in the mean time, it shall be a new DRM +device.
.. _drm_driver_ioctl:
IOCTL Support on Device Nodes