A number of hybrid GPU notebook computer designs with dual (integrated plus discrete) GPUs are equipped with multiplexers (muxes) that allow display panels to be driven by either the integrated GPU or the discrete GPU. Typically, this is a selection that can be made at boot time as a menu option in the system firmware's setup screen, and the mux selection stays fixed for as long as the system is running and persists across reboots until it is explicitly changed. However, some muxed hybrid GPU systems have dynamically switchable muxes which can be switched while the system is running.
NVIDIA is exploring the possibility of taking advantage of dynamically switchable muxes to enhance the experience of using a hybrid GPU system. For example, on a system configured for PRIME render offloading, it may be possible to keep the discrete GPU powered down and use the integrated GPU for rendering and displaying the desktop when no applications are using the discrete GPU, and dynamically switch the panel to be driven directly by the discrete GPU when render-offloading a fullscreen application.
We have been conducting some experiments on systems with dynamic muxes, and have found some limitations that would need to be addressed in order to support use cases like the one suggested above:
* In at least the i915 DRM-KMS driver, and likely in other DRM-KMS drivers as well, eDP panels are assumed to be always connected. This assumption is broken when the panel is muxed away, which can cause problems. A typical symptom is i915 repeatedly attempting to retrain the link, severely impacting system performance and printing messages like the following every five seconds or so:
[drm:intel_dp_start_link_train [i915]] *ERROR* failed to enable link training [drm] Reducing the compressed framebuffer size. This may lead to less power savings than a non-reduced-size. Try to increase stolen memory size if available in BIOS.
This symptom might occur if something causes the DRM-KMS driver to probe the display while it's muxed away, for example a modeset or DPMS state change.
* When switching the mux back to a GPU that was previously driving a mode, it is necessary to at the very least retrain DP links to restore the previously displayed image. In a proof of concept I have been experimenting with, I am able to accomplish this from userspace by triggering DPMS off and then back on again; however, it would be good to have an in-kernel API to request that an output owned by a DRM-KMS driver be refreshed to resume driving a mode on a disconnected and reconnected display. This API would need to be accessible from outside of the DRM-KMS driver handling the output. One reason it would be good to do this within the kernel, rather than rely on e.g. DPMS operations in the xf86-video-modesetting driver, is that it would be useful for restoring the console if X crashes or is forcefully killed while the mux is switched to a GPU other than the one which drives the console.
Basically, we'd like to be able to do the following:
1) Communicate to a DRM-KMS driver that an output is disconnected and can't be used. Ideally, DRI clients such as X should still see the output as being connected, so user applications don't need to keep track of the change. 2) Request that a mode that was previously driven on a disconnected output be driven again upon reconnection.
If APIs to do the above are already available, I wasn't able to find information about them. These could be handled as separate APIs, e.g., one to set connected/disconnected state and another to restore an output, or as a single API, e.g., signal a disconnect or reconnect, leaving it up to the driver receiving the signal to set the appropriate internal state and restore the reconnected output. Another possibility would be an API to disable and enable individual outputs from outside of the DRM-KMS driver that owns them. I'm curious to hear the thoughts of the DRM subsystem maintainers and contributors on what the best approach to this would be.
On Fri, 27 Mar 2020, Daniel Dadap ddadap@nvidia.com wrote:
A number of hybrid GPU notebook computer designs with dual (integrated plus discrete) GPUs are equipped with multiplexers (muxes) that allow display panels to be driven by either the integrated GPU or the discrete GPU. Typically, this is a selection that can be made at boot time as a menu option in the system firmware's setup screen, and the mux selection stays fixed for as long as the system is running and persists across reboots until it is explicitly changed. However, some muxed hybrid GPU systems have dynamically switchable muxes which can be switched while the system is running.
NVIDIA is exploring the possibility of taking advantage of dynamically switchable muxes to enhance the experience of using a hybrid GPU system. For example, on a system configured for PRIME render offloading, it may be possible to keep the discrete GPU powered down and use the integrated GPU for rendering and displaying the desktop when no applications are using the discrete GPU, and dynamically switch the panel to be driven directly by the discrete GPU when render-offloading a fullscreen application.
We have been conducting some experiments on systems with dynamic muxes, and have found some limitations that would need to be addressed in order to support use cases like the one suggested above:
- In at least the i915 DRM-KMS driver, and likely in other DRM-KMS
drivers as well, eDP panels are assumed to be always connected. This assumption is broken when the panel is muxed away, which can cause problems. A typical symptom is i915 repeatedly attempting to retrain the link, severely impacting system performance and printing messages like the following every five seconds or so:
[drm:intel_dp_start_link_train [i915]] *ERROR* failed to enable link training [drm] Reducing the compressed framebuffer size. This may lead to less power savings than a non-reduced-size. Try to increase stolen memory size if available in BIOS.
This symptom might occur if something causes the DRM-KMS driver to probe the display while it's muxed away, for example a modeset or DPMS state change.
- When switching the mux back to a GPU that was previously driving a
mode, it is necessary to at the very least retrain DP links to restore the previously displayed image. In a proof of concept I have been experimenting with, I am able to accomplish this from userspace by triggering DPMS off and then back on again; however, it would be good to have an in-kernel API to request that an output owned by a DRM-KMS driver be refreshed to resume driving a mode on a disconnected and reconnected display. This API would need to be accessible from outside of the DRM-KMS driver handling the output. One reason it would be good to do this within the kernel, rather than rely on e.g. DPMS operations in the xf86-video-modesetting driver, is that it would be useful for restoring the console if X crashes or is forcefully killed while the mux is switched to a GPU other than the one which drives the console.
Basically, we'd like to be able to do the following:
- Communicate to a DRM-KMS driver that an output is disconnected and
can't be used. Ideally, DRI clients such as X should still see the output as being connected, so user applications don't need to keep track of the change.
I think everything will be much easier if you provide a way for userspace to control the muxing using the KMS API, and not lie to the userspace about what's going on.
You're not actually saying what component you think should control the muxing.
Why should the drivers keep telling the userspace the output is connected when it's not? Obviously the userspace should also switch to using a different output on a different GPU, right? Or are you planning some proprietary behind the scenes hack for discrete?
BR, Jani.
- Request that a mode that was previously driven on a disconnected
output be driven again upon reconnection.
If APIs to do the above are already available, I wasn't able to find information about them. These could be handled as separate APIs, e.g., one to set connected/disconnected state and another to restore an output, or as a single API, e.g., signal a disconnect or reconnect, leaving it up to the driver receiving the signal to set the appropriate internal state and restore the reconnected output. Another possibility would be an API to disable and enable individual outputs from outside of the DRM-KMS driver that owns them. I'm curious to hear the thoughts of the DRM subsystem maintainers and contributors on what the best approach to this would be.
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
On 3/30/20 10:11 AM, Jani Nikula wrote:
On Fri, 27 Mar 2020, Daniel Dadap ddadap@nvidia.com wrote:
A number of hybrid GPU notebook computer designs with dual (integrated plus discrete) GPUs are equipped with multiplexers (muxes) that allow display panels to be driven by either the integrated GPU or the discrete GPU. Typically, this is a selection that can be made at boot time as a menu option in the system firmware's setup screen, and the mux selection stays fixed for as long as the system is running and persists across reboots until it is explicitly changed. However, some muxed hybrid GPU systems have dynamically switchable muxes which can be switched while the system is running.
NVIDIA is exploring the possibility of taking advantage of dynamically switchable muxes to enhance the experience of using a hybrid GPU system. For example, on a system configured for PRIME render offloading, it may be possible to keep the discrete GPU powered down and use the integrated GPU for rendering and displaying the desktop when no applications are using the discrete GPU, and dynamically switch the panel to be driven directly by the discrete GPU when render-offloading a fullscreen application.
We have been conducting some experiments on systems with dynamic muxes, and have found some limitations that would need to be addressed in order to support use cases like the one suggested above:
- In at least the i915 DRM-KMS driver, and likely in other DRM-KMS
drivers as well, eDP panels are assumed to be always connected. This assumption is broken when the panel is muxed away, which can cause problems. A typical symptom is i915 repeatedly attempting to retrain the link, severely impacting system performance and printing messages like the following every five seconds or so:
[drm:intel_dp_start_link_train [i915]] *ERROR* failed to enable link training [drm] Reducing the compressed framebuffer size. This may lead to less power savings than a non-reduced-size. Try to increase stolen memory size if available in BIOS.
This symptom might occur if something causes the DRM-KMS driver to probe the display while it's muxed away, for example a modeset or DPMS state change.
- When switching the mux back to a GPU that was previously driving a
mode, it is necessary to at the very least retrain DP links to restore the previously displayed image. In a proof of concept I have been experimenting with, I am able to accomplish this from userspace by triggering DPMS off and then back on again; however, it would be good to have an in-kernel API to request that an output owned by a DRM-KMS driver be refreshed to resume driving a mode on a disconnected and reconnected display. This API would need to be accessible from outside of the DRM-KMS driver handling the output. One reason it would be good to do this within the kernel, rather than rely on e.g. DPMS operations in the xf86-video-modesetting driver, is that it would be useful for restoring the console if X crashes or is forcefully killed while the mux is switched to a GPU other than the one which drives the console.
Basically, we'd like to be able to do the following:
- Communicate to a DRM-KMS driver that an output is disconnected and
can't be used. Ideally, DRI clients such as X should still see the output as being connected, so user applications don't need to keep track of the change.
I think everything will be much easier if you provide a way for userspace to control the muxing using the KMS API, and not lie to the userspace about what's going on.
You're not actually saying what component you think should control the muxing.
Why should the drivers keep telling the userspace the output is connected when it's not? Obviously the userspace should also switch to using a different output on a different GPU, right? Or are you planning some proprietary behind the scenes hack for discrete?
The desire to lie to userspace is driven mainly by trying to avoid interactions from desktop environments / window managers reacting to the display going away. Many desktops will do things like try to migrate windows in response to a change in the current display configuration, and updating all of them to avoid doing so when a display appears to disappear from one GPU and reappear on another GPU seems harder than allowing userspace to believe that nothing has changed. I wouldn't mind if e.g. X drivers were in on the lie, and the lie boundary shifts to RandR, but it would be nice to avoid having to deal with the fallout of desktop environments handling displays apparently vanishing and re-appearing.
The particular use case we're envisioning here is as follows:
* GPU A drives an X protocol screen which hosts a desktop session. Applications are rendered on GPU A by default. The mux is switched to GPU A by default. * GPU B drives a GPU screen that can be used as a PRIME render offload source. Applications rendered on GPU B can run in windows presented by GPU A via PRIME render offloading. * If an application rendered on GPU B and presented on GPU A becomes fullscreen, the mux can switch to GPU B and GPU B can present the application directly for as long as the application remains in the foreground and fullscreen. * The mux switches back to GPU A and the application presents via GPU A and render offloading if it transitions to a window or another window occludes it.
I think DRI3 render offload works a bit differently, but hopefully the high-level concept is somewhat applicable to that framework as well.
As for what should be controlling the muxing, I suppose that depends on what you mean by controlling:
If you mean controlling the mux device itself, that should be a platform driver that offers an API to execute the mux switch itself. The existing vga-switcheroo framework would be a natural fit, but it would need some substantial changes in order to support this sort of use case. I've described some of the challenges we've observed so far in my response to Daniel Vetter.
If you mean what should drive the policy of when automatic mux switches occur, it would have to be something that is aware of what at least one of the GPUs is displaying. It could be one of the GPU drivers, or a client of the GPU drivers, e.g. X11 or a Wayland compositor.
For the proof of concept experiments we are currently conducting, both of these roles are currently performed by components of the NVIDIA proprietary GPU stack, but the functionality could be moved to another component (e.g. vga-switcheroo, X11, server-side GLVND, ???) if the necessary functionality becomes supported in the future.
BR, Jani.
- Request that a mode that was previously driven on a disconnected
output be driven again upon reconnection.
If APIs to do the above are already available, I wasn't able to find information about them. These could be handled as separate APIs, e.g., one to set connected/disconnected state and another to restore an output, or as a single API, e.g., signal a disconnect or reconnect, leaving it up to the driver receiving the signal to set the appropriate internal state and restore the reconnected output. Another possibility would be an API to disable and enable individual outputs from outside of the DRM-KMS driver that owns them. I'm curious to hear the thoughts of the DRM subsystem maintainers and contributors on what the best approach to this would be.
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
-- Jani Nikula, Intel Open Source Graphics Center
On Tue, 31 Mar 2020 20:59:39 -0500 Daniel Dadap ddadap@nvidia.com wrote:
On 3/30/20 10:11 AM, Jani Nikula wrote:
On Fri, 27 Mar 2020, Daniel Dadap ddadap@nvidia.com wrote:
A number of hybrid GPU notebook computer designs with dual (integrated plus discrete) GPUs are equipped with multiplexers (muxes) that allow display panels to be driven by either the integrated GPU or the discrete GPU. Typically, this is a selection that can be made at boot time as a menu option in the system firmware's setup screen, and the mux selection stays fixed for as long as the system is running and persists across reboots until it is explicitly changed. However, some muxed hybrid GPU systems have dynamically switchable muxes which can be switched while the system is running.
NVIDIA is exploring the possibility of taking advantage of dynamically switchable muxes to enhance the experience of using a hybrid GPU system. For example, on a system configured for PRIME render offloading, it may be possible to keep the discrete GPU powered down and use the integrated GPU for rendering and displaying the desktop when no applications are using the discrete GPU, and dynamically switch the panel to be driven directly by the discrete GPU when render-offloading a fullscreen application.
We have been conducting some experiments on systems with dynamic muxes, and have found some limitations that would need to be addressed in order to support use cases like the one suggested above:
- In at least the i915 DRM-KMS driver, and likely in other DRM-KMS
drivers as well, eDP panels are assumed to be always connected. This assumption is broken when the panel is muxed away, which can cause problems. A typical symptom is i915 repeatedly attempting to retrain the link, severely impacting system performance and printing messages like the following every five seconds or so:
[drm:intel_dp_start_link_train [i915]] *ERROR* failed to enable link training [drm] Reducing the compressed framebuffer size. This may lead to less power savings than a non-reduced-size. Try to increase stolen memory size if available in BIOS.
This symptom might occur if something causes the DRM-KMS driver to probe the display while it's muxed away, for example a modeset or DPMS state change.
- When switching the mux back to a GPU that was previously driving a
mode, it is necessary to at the very least retrain DP links to restore the previously displayed image. In a proof of concept I have been experimenting with, I am able to accomplish this from userspace by triggering DPMS off and then back on again; however, it would be good to have an in-kernel API to request that an output owned by a DRM-KMS driver be refreshed to resume driving a mode on a disconnected and reconnected display. This API would need to be accessible from outside of the DRM-KMS driver handling the output. One reason it would be good to do this within the kernel, rather than rely on e.g. DPMS operations in the xf86-video-modesetting driver, is that it would be useful for restoring the console if X crashes or is forcefully killed while the mux is switched to a GPU other than the one which drives the console.
Basically, we'd like to be able to do the following:
- Communicate to a DRM-KMS driver that an output is disconnected and
can't be used. Ideally, DRI clients such as X should still see the output as being connected, so user applications don't need to keep track of the change.
I think everything will be much easier if you provide a way for userspace to control the muxing using the KMS API, and not lie to the userspace about what's going on.
You're not actually saying what component you think should control the muxing.
Why should the drivers keep telling the userspace the output is connected when it's not? Obviously the userspace should also switch to using a different output on a different GPU, right? Or are you planning some proprietary behind the scenes hack for discrete?
The desire to lie to userspace is driven mainly by trying to avoid interactions from desktop environments / window managers reacting to the display going away. Many desktops will do things like try to migrate windows in response to a change in the current display configuration, and updating all of them to avoid doing so when a display appears to disappear from one GPU and reappear on another GPU seems harder than allowing userspace to believe that nothing has changed. I wouldn't mind if e.g. X drivers were in on the lie, and the lie boundary shifts to RandR, but it would be nice to avoid having to deal with the fallout of desktop environments handling displays apparently vanishing and re-appearing.
Hi,
I love the general idea of using the mux to optimize hardware usage, but I really do not like the initial software design proposal.
I'm afraid that lying is going to lead to a disaster eventually, instead of needing to fix a more obvious shortcoming in a simpler way today - or rather, implementing a new feature taking advantage of the new capabilities. Lying would lock the whole graphics stack to the single design you thought of today, stopping any novel ways of using the feature from appearing later.
Bypassing the desktop or the display server is practically always the wrong design, whether it is this, color management, or whatever.
The particular use case we're envisioning here is as follows:
- GPU A drives an X protocol screen which hosts a desktop session.
Applications are rendered on GPU A by default. The mux is switched to GPU A by default.
- GPU B drives a GPU screen that can be used as a PRIME render offload
source. Applications rendered on GPU B can run in windows presented by GPU A via PRIME render offloading.
- If an application rendered on GPU B and presented on GPU A becomes
fullscreen, the mux can switch to GPU B and GPU B can present the application directly for as long as the application remains in the foreground and fullscreen.
- The mux switches back to GPU A and the application presents via GPU A
and render offloading if it transitions to a window or another window occludes it.
I do not see how you could ever pull that off without patching all display servers to specifically support that use case (think of Wayland architecture here). When the mux is switched, the userspace also needs to switch posting DRM FBs from DRM KMS device A to DRM KMS device B.
What you describe is totally fine for a Wayland display server to do automatically, and in the Wayland architecture there really is no other component that could even attempt to do it. In fact, a Wayland display server is the only component in the architecture that actually needs to do anything about it to make it work. Your use case is a perfect fit in the Wayland architecture, if the mux is controlled exclusively by the display server and no-one lies.
My suggestion is to just trigger the exact same hotplug actions that physically disconnecting a display cable from one card and plugging it into another card does already. It's not like that is going to ever happen beyond a display server's back, it is the display server itself toggling the mux, so it knows to handle it right. What the display server *does* need to know before-hand is exactly which connectors the mux affects.
So yeah, I am implying that having any side-band to access to the mux directly, bypassing the running display server, is a bad idea. Just like we do not allow non-DRM-master programs to set modes via KMS while a display server has DRM-master thinking it is controlling the displays.
If it is useful for Xorg to lie to the X11 RandR clients, then ok, I don't care about that. That's up to Xorg. RandR itself is already a kind of a side-band for setting modes and whatnot behind the X11 desktop environment's back. Wayland architecture does not have that problem, and I don't want that problem to appear either.
This optimization in general, not the mux toggling part, would be already extremely useful with eGPUs[1]. Assume you have a game rendering on eGPU and a display connected to the eGPU showing the game. A naive display server, who has a client rendering on an eGPU, will transfer the client frames to the iGPU for composition and then again back to the eGPU for display. If the client frame can be shown directly on the eGPU display, there is no need to transfer the frame back and forth over the bus. So I bet display servers will be gaining that optimization logic sooner or later (if they care about the eGPU use case), and it does not seem that making the same logic apply to mux switching would be too much work on top.
Thanks, pq
[1] https://gitlab.gnome.org/GNOME/mutter/-/issues/348
I think DRI3 render offload works a bit differently, but hopefully the high-level concept is somewhat applicable to that framework as well.
As for what should be controlling the muxing, I suppose that depends on what you mean by controlling:
If you mean controlling the mux device itself, that should be a platform driver that offers an API to execute the mux switch itself. The existing vga-switcheroo framework would be a natural fit, but it would need some substantial changes in order to support this sort of use case. I've described some of the challenges we've observed so far in my response to Daniel Vetter.
If you mean what should drive the policy of when automatic mux switches occur, it would have to be something that is aware of what at least one of the GPUs is displaying. It could be one of the GPU drivers, or a client of the GPU drivers, e.g. X11 or a Wayland compositor.
For the proof of concept experiments we are currently conducting, both of these roles are currently performed by components of the NVIDIA proprietary GPU stack, but the functionality could be moved to another component (e.g. vga-switcheroo, X11, server-side GLVND, ???) if the necessary functionality becomes supported in the future.
BR, Jani.
- Request that a mode that was previously driven on a disconnected
output be driven again upon reconnection.
If APIs to do the above are already available, I wasn't able to find information about them. These could be handled as separate APIs, e.g., one to set connected/disconnected state and another to restore an output, or as a single API, e.g., signal a disconnect or reconnect, leaving it up to the driver receiving the signal to set the appropriate internal state and restore the reconnected output. Another possibility would be an API to disable and enable individual outputs from outside of the DRM-KMS driver that owns them. I'm curious to hear the thoughts of the DRM subsystem maintainers and contributors on what the best approach to this would be.
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
-- Jani Nikula, Intel Open Source Graphics Center
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
On 4/1/20 3:14 AM, Pekka Paalanen wrote:
On Tue, 31 Mar 2020 20:59:39 -0500 Daniel Dadapddadap@nvidia.com wrote:
On 3/30/20 10:11 AM, Jani Nikula wrote:
On Fri, 27 Mar 2020, Daniel Dadapddadap@nvidia.com wrote:
A number of hybrid GPU notebook computer designs with dual (integrated plus discrete) GPUs are equipped with multiplexers (muxes) that allow display panels to be driven by either the integrated GPU or the discrete GPU. Typically, this is a selection that can be made at boot time as a menu option in the system firmware's setup screen, and the mux selection stays fixed for as long as the system is running and persists across reboots until it is explicitly changed. However, some muxed hybrid GPU systems have dynamically switchable muxes which can be switched while the system is running.
NVIDIA is exploring the possibility of taking advantage of dynamically switchable muxes to enhance the experience of using a hybrid GPU system. For example, on a system configured for PRIME render offloading, it may be possible to keep the discrete GPU powered down and use the integrated GPU for rendering and displaying the desktop when no applications are using the discrete GPU, and dynamically switch the panel to be driven directly by the discrete GPU when render-offloading a fullscreen application.
We have been conducting some experiments on systems with dynamic muxes, and have found some limitations that would need to be addressed in order to support use cases like the one suggested above:
- In at least the i915 DRM-KMS driver, and likely in other DRM-KMS
drivers as well, eDP panels are assumed to be always connected. This assumption is broken when the panel is muxed away, which can cause problems. A typical symptom is i915 repeatedly attempting to retrain the link, severely impacting system performance and printing messages like the following every five seconds or so:
[drm:intel_dp_start_link_train [i915]]*ERROR* failed to enable link training [drm] Reducing the compressed framebuffer size. This may lead to less power savings than a non-reduced-size. Try to increase stolen memory size if available in BIOS.
This symptom might occur if something causes the DRM-KMS driver to probe the display while it's muxed away, for example a modeset or DPMS state change.
- When switching the mux back to a GPU that was previously driving a
mode, it is necessary to at the very least retrain DP links to restore the previously displayed image. In a proof of concept I have been experimenting with, I am able to accomplish this from userspace by triggering DPMS off and then back on again; however, it would be good to have an in-kernel API to request that an output owned by a DRM-KMS driver be refreshed to resume driving a mode on a disconnected and reconnected display. This API would need to be accessible from outside of the DRM-KMS driver handling the output. One reason it would be good to do this within the kernel, rather than rely on e.g. DPMS operations in the xf86-video-modesetting driver, is that it would be useful for restoring the console if X crashes or is forcefully killed while the mux is switched to a GPU other than the one which drives the console.
Basically, we'd like to be able to do the following:
- Communicate to a DRM-KMS driver that an output is disconnected and
can't be used. Ideally, DRI clients such as X should still see the output as being connected, so user applications don't need to keep track of the change.
I think everything will be much easier if you provide a way for userspace to control the muxing using the KMS API, and not lie to the userspace about what's going on.
You're not actually saying what component you think should control the muxing.
Why should the drivers keep telling the userspace the output is connected when it's not? Obviously the userspace should also switch to using a different output on a different GPU, right? Or are you planning some proprietary behind the scenes hack for discrete?
The desire to lie to userspace is driven mainly by trying to avoid interactions from desktop environments / window managers reacting to the display going away. Many desktops will do things like try to migrate windows in response to a change in the current display configuration, and updating all of them to avoid doing so when a display appears to disappear from one GPU and reappear on another GPU seems harder than allowing userspace to believe that nothing has changed. I wouldn't mind if e.g. X drivers were in on the lie, and the lie boundary shifts to RandR, but it would be nice to avoid having to deal with the fallout of desktop environments handling displays apparently vanishing and re-appearing.
Hi,
I love the general idea of using the mux to optimize hardware usage, but I really do not like the initial software design proposal.
I'm afraid that lying is going to lead to a disaster eventually, instead of needing to fix a more obvious shortcoming in a simpler way today - or rather, implementing a new feature taking advantage of the new capabilities. Lying would lock the whole graphics stack to the single design you thought of today, stopping any novel ways of using the feature from appearing later.
Bypassing the desktop or the display server is practically always the wrong design, whether it is this, color management, or whatever.
That is fair. I regret expressing a lie to userspace as an ideal outcome in the initial message, as it's really secondary to the more important issues of needing to be able to disconnect the panel at a software level within the GPU driver and initiating a retrain when it's reconnected.
The particular use case we're envisioning here is as follows:
- GPU A drives an X protocol screen which hosts a desktop session.
Applications are rendered on GPU A by default. The mux is switched to GPU A by default.
- GPU B drives a GPU screen that can be used as a PRIME render offload
source. Applications rendered on GPU B can run in windows presented by GPU A via PRIME render offloading.
- If an application rendered on GPU B and presented on GPU A becomes
fullscreen, the mux can switch to GPU B and GPU B can present the application directly for as long as the application remains in the foreground and fullscreen.
- The mux switches back to GPU A and the application presents via GPU A
and render offloading if it transitions to a window or another window occludes it.
I do not see how you could ever pull that off without patching all display servers to specifically support that use case (think of Wayland architecture here). When the mux is switched, the userspace also needs to switch posting DRM FBs from DRM KMS device A to DRM KMS device B.
What you describe is totally fine for a Wayland display server to do automatically, and in the Wayland architecture there really is no other component that could even attempt to do it. In fact, a Wayland display server is the only component in the architecture that actually needs to do anything about it to make it work. Your use case is a perfect fit in the Wayland architecture, if the mux is controlled exclusively by the display server and no-one lies.
My suggestion is to just trigger the exact same hotplug actions that physically disconnecting a display cable from one card and plugging it into another card does already. It's not like that is going to ever happen beyond a display server's back, it is the display server itself toggling the mux, so it knows to handle it right. What the display server*does* need to know before-hand is exactly which connectors the mux affects.
That should be trivial for external displays, and in fact I have observed some designs which make the muxed-away from GPU see a hot unplug and the muxed-to GPU see a hotplug when mux switching an external display; however, since eDP is currently treated as always connected by at least i915, we would need to at least make it possible for an eDP display to be "unplugged".
So yeah, I am implying that having any side-band to access to the mux directly, bypassing the running display server, is a bad idea. Just like we do not allow non-DRM-master programs to set modes via KMS while a display server has DRM-master thinking it is controlling the displays.
If it is useful for Xorg to lie to the X11 RandR clients, then ok, I don't care about that. That's up to Xorg. RandR itself is already a kind of a side-band for setting modes and whatnot behind the X11 desktop environment's back. Wayland architecture does not have that problem, and I don't want that problem to appear either.
Yes, RandR is already rug that many things are swept under. Using RandR as the abstraction point if one is desired makes sense to me.
This optimization in general, not the mux toggling part, would be already extremely useful with eGPUs[1]. Assume you have a game rendering on eGPU and a display connected to the eGPU showing the game. A naive display server, who has a client rendering on an eGPU, will transfer the client frames to the iGPU for composition and then again back to the eGPU for display. If the client frame can be shown directly on the eGPU display, there is no need to transfer the frame back and forth over the bus. So I bet display servers will be gaining that optimization logic sooner or later (if they care about the eGPU use case), and it does not seem that making the same logic apply to mux switching would be too much work on top.
Thanks, pq
[1]https://gitlab.gnome.org/GNOME/mutter/-/issues/348
I think DRI3 render offload works a bit differently, but hopefully the high-level concept is somewhat applicable to that framework as well.
As for what should be controlling the muxing, I suppose that depends on what you mean by controlling:
If you mean controlling the mux device itself, that should be a platform driver that offers an API to execute the mux switch itself. The existing vga-switcheroo framework would be a natural fit, but it would need some substantial changes in order to support this sort of use case. I've described some of the challenges we've observed so far in my response to Daniel Vetter.
If you mean what should drive the policy of when automatic mux switches occur, it would have to be something that is aware of what at least one of the GPUs is displaying. It could be one of the GPU drivers, or a client of the GPU drivers, e.g. X11 or a Wayland compositor.
For the proof of concept experiments we are currently conducting, both of these roles are currently performed by components of the NVIDIA proprietary GPU stack, but the functionality could be moved to another component (e.g. vga-switcheroo, X11, server-side GLVND, ???) if the necessary functionality becomes supported in the future.
BR, Jani.
- Request that a mode that was previously driven on a disconnected
output be driven again upon reconnection.
If APIs to do the above are already available, I wasn't able to find information about them. These could be handled as separate APIs, e.g., one to set connected/disconnected state and another to restore an output, or as a single API, e.g., signal a disconnect or reconnect, leaving it up to the driver receiving the signal to set the appropriate internal state and restore the reconnected output. Another possibility would be an API to disable and enable individual outputs from outside of the DRM-KMS driver that owns them. I'm curious to hear the thoughts of the DRM subsystem maintainers and contributors on what the best approach to this would be.
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
-- Jani Nikula, Intel Open Source Graphics Center
On Tue, 31 Mar 2020, Daniel Dadap ddadap@nvidia.com wrote:
On 3/30/20 10:11 AM, Jani Nikula wrote:
On Fri, 27 Mar 2020, Daniel Dadap ddadap@nvidia.com wrote:
A number of hybrid GPU notebook computer designs with dual (integrated plus discrete) GPUs are equipped with multiplexers (muxes) that allow display panels to be driven by either the integrated GPU or the discrete GPU. Typically, this is a selection that can be made at boot time as a menu option in the system firmware's setup screen, and the mux selection stays fixed for as long as the system is running and persists across reboots until it is explicitly changed. However, some muxed hybrid GPU systems have dynamically switchable muxes which can be switched while the system is running.
NVIDIA is exploring the possibility of taking advantage of dynamically switchable muxes to enhance the experience of using a hybrid GPU system. For example, on a system configured for PRIME render offloading, it may be possible to keep the discrete GPU powered down and use the integrated GPU for rendering and displaying the desktop when no applications are using the discrete GPU, and dynamically switch the panel to be driven directly by the discrete GPU when render-offloading a fullscreen application.
We have been conducting some experiments on systems with dynamic muxes, and have found some limitations that would need to be addressed in order to support use cases like the one suggested above:
- In at least the i915 DRM-KMS driver, and likely in other DRM-KMS
drivers as well, eDP panels are assumed to be always connected. This assumption is broken when the panel is muxed away, which can cause problems. A typical symptom is i915 repeatedly attempting to retrain the link, severely impacting system performance and printing messages like the following every five seconds or so:
[drm:intel_dp_start_link_train [i915]] *ERROR* failed to enable link training [drm] Reducing the compressed framebuffer size. This may lead to less power savings than a non-reduced-size. Try to increase stolen memory size if available in BIOS.
This symptom might occur if something causes the DRM-KMS driver to probe the display while it's muxed away, for example a modeset or DPMS state change.
- When switching the mux back to a GPU that was previously driving a
mode, it is necessary to at the very least retrain DP links to restore the previously displayed image. In a proof of concept I have been experimenting with, I am able to accomplish this from userspace by triggering DPMS off and then back on again; however, it would be good to have an in-kernel API to request that an output owned by a DRM-KMS driver be refreshed to resume driving a mode on a disconnected and reconnected display. This API would need to be accessible from outside of the DRM-KMS driver handling the output. One reason it would be good to do this within the kernel, rather than rely on e.g. DPMS operations in the xf86-video-modesetting driver, is that it would be useful for restoring the console if X crashes or is forcefully killed while the mux is switched to a GPU other than the one which drives the console.
Basically, we'd like to be able to do the following:
- Communicate to a DRM-KMS driver that an output is disconnected and
can't be used. Ideally, DRI clients such as X should still see the output as being connected, so user applications don't need to keep track of the change.
I think everything will be much easier if you provide a way for userspace to control the muxing using the KMS API, and not lie to the userspace about what's going on.
You're not actually saying what component you think should control the muxing.
Why should the drivers keep telling the userspace the output is connected when it's not? Obviously the userspace should also switch to using a different output on a different GPU, right? Or are you planning some proprietary behind the scenes hack for discrete?
The desire to lie to userspace is driven mainly by trying to avoid interactions from desktop environments / window managers reacting to the display going away. Many desktops will do things like try to migrate windows in response to a change in the current display configuration, and updating all of them to avoid doing so when a display appears to disappear from one GPU and reappear on another GPU seems harder than allowing userspace to believe that nothing has changed. I wouldn't mind if e.g. X drivers were in on the lie, and the lie boundary shifts to RandR, but it would be nice to avoid having to deal with the fallout of desktop environments handling displays apparently vanishing and re-appearing.
If the change from one GPU to another was driven and initiated by userspace, it would not be responding to changes in display configuration to begin with.
I think lying to the userspace in this case is a hack to enable a very specific use case. Of course it's going to seem easier if you're only looking at enabling that narrowly defined feature. But usually the hacks end up being restrictions that make future work harder.
I also think you're downplaying the complexity of gracefully pretending a display is there when it's not. And realistically, a lot of that complexity is going to end up being forced on i915.
And then there are plenty of additional details, for example who controls panel power sequencing, how does brightness control work, is PWM pin muxed or not, and what if you have DPCD brightness control instead of PWM?
The particular use case we're envisioning here is as follows:
- GPU A drives an X protocol screen which hosts a desktop session.
Applications are rendered on GPU A by default. The mux is switched to GPU A by default.
- GPU B drives a GPU screen that can be used as a PRIME render offload
source. Applications rendered on GPU B can run in windows presented by GPU A via PRIME render offloading.
- If an application rendered on GPU B and presented on GPU A becomes
fullscreen, the mux can switch to GPU B and GPU B can present the application directly for as long as the application remains in the foreground and fullscreen.
- The mux switches back to GPU A and the application presents via GPU A
and render offloading if it transitions to a window or another window occludes it.
I think DRI3 render offload works a bit differently, but hopefully the high-level concept is somewhat applicable to that framework as well.
As for what should be controlling the muxing, I suppose that depends on what you mean by controlling:
If you mean controlling the mux device itself, that should be a platform driver that offers an API to execute the mux switch itself. The existing vga-switcheroo framework would be a natural fit, but it would need some substantial changes in order to support this sort of use case. I've described some of the challenges we've observed so far in my response to Daniel Vetter.
If you mean what should drive the policy of when automatic mux switches occur, it would have to be something that is aware of what at least one of the GPUs is displaying. It could be one of the GPU drivers, or a client of the GPU drivers, e.g. X11 or a Wayland compositor.
IMO the policy should obviously be in the userspace, and the kernel would only provide the mechanism.
For the proof of concept experiments we are currently conducting, both of these roles are currently performed by components of the NVIDIA proprietary GPU stack, but the functionality could be moved to another component (e.g. vga-switcheroo, X11, server-side GLVND, ???) if the necessary functionality becomes supported in the future.
Now, this brings us to the so far unspoken issue. It is highly unlikely that we'd start making changes in i915 to support a feature in a proprietary stack. I really don't think this is the kind of feature that can be done with open design only, it's going to have to be open source. It's basically handing over live peripheral hardware from one driver to another.
We can keep talking, of course, and I appreciate this discussion. But please keep the above in mind when you come up with ideas that impose complexity on i915 with no benefit to i915 or open source users.
BR, Jani.
BR, Jani.
- Request that a mode that was previously driven on a disconnected
output be driven again upon reconnection.
If APIs to do the above are already available, I wasn't able to find information about them. These could be handled as separate APIs, e.g., one to set connected/disconnected state and another to restore an output, or as a single API, e.g., signal a disconnect or reconnect, leaving it up to the driver receiving the signal to set the appropriate internal state and restore the reconnected output. Another possibility would be an API to disable and enable individual outputs from outside of the DRM-KMS driver that owns them. I'm curious to hear the thoughts of the DRM subsystem maintainers and contributors on what the best approach to this would be.
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
-- Jani Nikula, Intel Open Source Graphics Center
Since I see no mention of this anywhere in your mail ... have you tried looking at drivers/gpu/vga/vga_switcheroo.c? This also supports switching of just outputs, not just the old optimus way of switching out the entire gpu and having to disable the other one.
There's some rough corners (like the uapi doesn't exist, it's in debugfs), and the locking has an inversion problem (I have ideas), but generally what you want exists already. -Daniel
On Mon, Mar 30, 2020 at 9:12 AM Daniel Dadap ddadap@nvidia.com wrote:
A number of hybrid GPU notebook computer designs with dual (integrated plus discrete) GPUs are equipped with multiplexers (muxes) that allow display panels to be driven by either the integrated GPU or the discrete GPU. Typically, this is a selection that can be made at boot time as a menu option in the system firmware's setup screen, and the mux selection stays fixed for as long as the system is running and persists across reboots until it is explicitly changed. However, some muxed hybrid GPU systems have dynamically switchable muxes which can be switched while the system is running.
NVIDIA is exploring the possibility of taking advantage of dynamically switchable muxes to enhance the experience of using a hybrid GPU system. For example, on a system configured for PRIME render offloading, it may be possible to keep the discrete GPU powered down and use the integrated GPU for rendering and displaying the desktop when no applications are using the discrete GPU, and dynamically switch the panel to be driven directly by the discrete GPU when render-offloading a fullscreen application.
We have been conducting some experiments on systems with dynamic muxes, and have found some limitations that would need to be addressed in order to support use cases like the one suggested above:
- In at least the i915 DRM-KMS driver, and likely in other DRM-KMS
drivers as well, eDP panels are assumed to be always connected. This assumption is broken when the panel is muxed away, which can cause problems. A typical symptom is i915 repeatedly attempting to retrain the link, severely impacting system performance and printing messages like the following every five seconds or so:
[drm:intel_dp_start_link_train [i915]] *ERROR* failed to enable link training [drm] Reducing the compressed framebuffer size. This may lead to less power savings than a non-reduced-size. Try to increase stolen memory size if available in BIOS.
This symptom might occur if something causes the DRM-KMS driver to probe the display while it's muxed away, for example a modeset or DPMS state change.
- When switching the mux back to a GPU that was previously driving a
mode, it is necessary to at the very least retrain DP links to restore the previously displayed image. In a proof of concept I have been experimenting with, I am able to accomplish this from userspace by triggering DPMS off and then back on again; however, it would be good to have an in-kernel API to request that an output owned by a DRM-KMS driver be refreshed to resume driving a mode on a disconnected and reconnected display. This API would need to be accessible from outside of the DRM-KMS driver handling the output. One reason it would be good to do this within the kernel, rather than rely on e.g. DPMS operations in the xf86-video-modesetting driver, is that it would be useful for restoring the console if X crashes or is forcefully killed while the mux is switched to a GPU other than the one which drives the console.
Basically, we'd like to be able to do the following:
- Communicate to a DRM-KMS driver that an output is disconnected and
can't be used. Ideally, DRI clients such as X should still see the output as being connected, so user applications don't need to keep track of the change. 2) Request that a mode that was previously driven on a disconnected output be driven again upon reconnection.
If APIs to do the above are already available, I wasn't able to find information about them. These could be handled as separate APIs, e.g., one to set connected/disconnected state and another to restore an output, or as a single API, e.g., signal a disconnect or reconnect, leaving it up to the driver receiving the signal to set the appropriate internal state and restore the reconnected output. Another possibility would be an API to disable and enable individual outputs from outside of the DRM-KMS driver that owns them. I'm curious to hear the thoughts of the DRM subsystem maintainers and contributors on what the best approach to this would be.
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
On 3/31/20 2:32 AM, Daniel Vetter wrote:
Since I see no mention of this anywhere in your mail ... have you tried looking at drivers/gpu/vga/vga_switcheroo.c? This also supports switching of just outputs, not just the old optimus way of switching out the entire gpu and having to disable the other one.
We did look into vga-switcheroo while developing our PoC, but found it insufficient for supporting the use case we had in mind in its current form. Among the limitations we observed were that it didn't seem to be possible to switch with output-level granularity, only with gpu-level granularity, with the whole switched-away-to GPU being powered down. Your description suggests that this is indeed possible, but if that's the case, then the mechanism for doing so isn't obvious in what I can see of the API from the kernel source code, even in the drm-next tree. Do you have pointers to documentation on how the per-output switching is supposed to work?
Other limitations of vga-switcheroo included:
* The can_switch() callbacks for all current vga-switcheroo-capable GPU drivers don't seem to allow for a switch to occur while there are active KMS clients. This has the effect of making it so that non-deferred switches can only be initiated under the same circumstances that deferred switches wait for. * It's only possible to address one mux device. Some systems have separate muxes for internal and external displays. From what I could see in existing vga-switcheroo mux handlers, it seems that multi-muxed systems just switch all of the muxes simultaneously, which makes sense for the all-or-nothing "power down the GPU not in use" approach, but might not be desirable for fine-grained output-level switching. * On some systems, it's possible to put the panel into a self-refresh mode before switching the mux and exit self-refresh mode after switching, to hide any glitches that might otherwise appear while transitioning. Additional handler callbacks for pre-switch and post-switch operations would be useful.
If vga-switcheroo could be updated to address these limitatons, then it could make sense to handle the display disconnect/connect notifications and display refreshing as part of a vga_switcheroo client driver's set_gpu_state() callback (or a finer-grained per-output callback); however, it also seems that it should be possible to implement APIs within the DRM subsystem to accomplish the desired functionality independently of current or future vga-switcheroo design.
There's some rough corners (like the uapi doesn't exist, it's in debugfs), and the locking has an inversion problem (I have ideas), but generally what you want exists already. -Daniel
On Mon, Mar 30, 2020 at 9:12 AM Daniel Dadap ddadap@nvidia.com wrote:
A number of hybrid GPU notebook computer designs with dual (integrated plus discrete) GPUs are equipped with multiplexers (muxes) that allow display panels to be driven by either the integrated GPU or the discrete GPU. Typically, this is a selection that can be made at boot time as a menu option in the system firmware's setup screen, and the mux selection stays fixed for as long as the system is running and persists across reboots until it is explicitly changed. However, some muxed hybrid GPU systems have dynamically switchable muxes which can be switched while the system is running.
NVIDIA is exploring the possibility of taking advantage of dynamically switchable muxes to enhance the experience of using a hybrid GPU system. For example, on a system configured for PRIME render offloading, it may be possible to keep the discrete GPU powered down and use the integrated GPU for rendering and displaying the desktop when no applications are using the discrete GPU, and dynamically switch the panel to be driven directly by the discrete GPU when render-offloading a fullscreen application.
We have been conducting some experiments on systems with dynamic muxes, and have found some limitations that would need to be addressed in order to support use cases like the one suggested above:
- In at least the i915 DRM-KMS driver, and likely in other DRM-KMS
drivers as well, eDP panels are assumed to be always connected. This assumption is broken when the panel is muxed away, which can cause problems. A typical symptom is i915 repeatedly attempting to retrain the link, severely impacting system performance and printing messages like the following every five seconds or so:
[drm:intel_dp_start_link_train [i915]] *ERROR* failed to enable link training [drm] Reducing the compressed framebuffer size. This may lead to less power savings than a non-reduced-size. Try to increase stolen memory size if available in BIOS.
This symptom might occur if something causes the DRM-KMS driver to probe the display while it's muxed away, for example a modeset or DPMS state change.
- When switching the mux back to a GPU that was previously driving a
mode, it is necessary to at the very least retrain DP links to restore the previously displayed image. In a proof of concept I have been experimenting with, I am able to accomplish this from userspace by triggering DPMS off and then back on again; however, it would be good to have an in-kernel API to request that an output owned by a DRM-KMS driver be refreshed to resume driving a mode on a disconnected and reconnected display. This API would need to be accessible from outside of the DRM-KMS driver handling the output. One reason it would be good to do this within the kernel, rather than rely on e.g. DPMS operations in the xf86-video-modesetting driver, is that it would be useful for restoring the console if X crashes or is forcefully killed while the mux is switched to a GPU other than the one which drives the console.
Basically, we'd like to be able to do the following:
- Communicate to a DRM-KMS driver that an output is disconnected and
can't be used. Ideally, DRI clients such as X should still see the output as being connected, so user applications don't need to keep track of the change. 2) Request that a mode that was previously driven on a disconnected output be driven again upon reconnection.
If APIs to do the above are already available, I wasn't able to find information about them. These could be handled as separate APIs, e.g., one to set connected/disconnected state and another to restore an output, or as a single API, e.g., signal a disconnect or reconnect, leaving it up to the driver receiving the signal to set the appropriate internal state and restore the reconnected output. Another possibility would be an API to disable and enable individual outputs from outside of the DRM-KMS driver that owns them. I'm curious to hear the thoughts of the DRM subsystem maintainers and contributors on what the best approach to this would be.
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
-- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
On Wed, Apr 1, 2020 at 3:58 AM Daniel Dadap ddadap@nvidia.com wrote:
On 3/31/20 2:32 AM, Daniel Vetter wrote:
Since I see no mention of this anywhere in your mail ... have you tried looking at drivers/gpu/vga/vga_switcheroo.c? This also supports switching of just outputs, not just the old optimus way of switching out the entire gpu and having to disable the other one.
We did look into vga-switcheroo while developing our PoC, but found it insufficient for supporting the use case we had in mind in its current form. Among the limitations we observed were that it didn't seem to be possible to switch with output-level granularity, only with gpu-level granularity, with the whole switched-away-to GPU being powered down. Your description suggests that this is indeed possible, but if that's the case, then the mechanism for doing so isn't obvious in what I can see of the API from the kernel source code, even in the drm-next tree. Do you have pointers to documentation on how the per-output switching is supposed to work?
I think the per-output switch isn't support (yet), but otherwise there should be two modes: - power switch mode, other gpu has to be offlined. This also means switching needs to wait until all the clients are shut down. - output switching only. That seems to be roughly what you want, except you want to switch more than just the integrated panel. I think all models thus far simply wired all external outputs to the dgpu always.
I primarily asked about vgaswitcheroo since you didn't mention it at all.
Other limitations of vga-switcheroo included:
- The can_switch() callbacks for all current vga-switcheroo-capable GPU
drivers don't seem to allow for a switch to occur while there are active KMS clients. This has the effect of making it so that non-deferred switches can only be initiated under the same circumstances that deferred switches wait for.
- It's only possible to address one mux device. Some systems have
separate muxes for internal and external displays. From what I could see in existing vga-switcheroo mux handlers, it seems that multi-muxed systems just switch all of the muxes simultaneously, which makes sense for the all-or-nothing "power down the GPU not in use" approach, but might not be desirable for fine-grained output-level switching.
- On some systems, it's possible to put the panel into a self-refresh
mode before switching the mux and exit self-refresh mode after switching, to hide any glitches that might otherwise appear while transitioning. Additional handler callbacks for pre-switch and post-switch operations would be useful.
If vga-switcheroo could be updated to address these limitatons, then it could make sense to handle the display disconnect/connect notifications and display refreshing as part of a vga_switcheroo client driver's set_gpu_state() callback (or a finer-grained per-output callback); however, it also seems that it should be possible to implement APIs within the DRM subsystem to accomplish the desired functionality independently of current or future vga-switcheroo design.
vgaswitcheroo was written by Dave Airlie, doesn't get much more "within the gpu subsystem". I think we should look into extending vgaswitcheroo instead of inventing a completely new uapi. If we go with a simplified use-case of - only integrated panel switches - external output on the dgpu then that should match existing systems, so you get support on at least some desktops for free. Plus no headaches about userspace we need for merging new uapi. Once we have that (for your mux, and I guess with i915+nouveau all working) then we can look into how to extend it, or give the same underlying stuff an interface that's not in debugfs :-)
Also for your seamless switching with self-refresh this could be implemented without new userspace, greatly simplifying everything.
Cheers, Daniel
There's some rough corners (like the uapi doesn't exist, it's in debugfs), and the locking has an inversion problem (I have ideas), but generally what you want exists already. -Daniel
On Mon, Mar 30, 2020 at 9:12 AM Daniel Dadap ddadap@nvidia.com wrote:
A number of hybrid GPU notebook computer designs with dual (integrated plus discrete) GPUs are equipped with multiplexers (muxes) that allow display panels to be driven by either the integrated GPU or the discrete GPU. Typically, this is a selection that can be made at boot time as a menu option in the system firmware's setup screen, and the mux selection stays fixed for as long as the system is running and persists across reboots until it is explicitly changed. However, some muxed hybrid GPU systems have dynamically switchable muxes which can be switched while the system is running.
NVIDIA is exploring the possibility of taking advantage of dynamically switchable muxes to enhance the experience of using a hybrid GPU system. For example, on a system configured for PRIME render offloading, it may be possible to keep the discrete GPU powered down and use the integrated GPU for rendering and displaying the desktop when no applications are using the discrete GPU, and dynamically switch the panel to be driven directly by the discrete GPU when render-offloading a fullscreen application.
We have been conducting some experiments on systems with dynamic muxes, and have found some limitations that would need to be addressed in order to support use cases like the one suggested above:
- In at least the i915 DRM-KMS driver, and likely in other DRM-KMS
drivers as well, eDP panels are assumed to be always connected. This assumption is broken when the panel is muxed away, which can cause problems. A typical symptom is i915 repeatedly attempting to retrain the link, severely impacting system performance and printing messages like the following every five seconds or so:
[drm:intel_dp_start_link_train [i915]] *ERROR* failed to enable link training [drm] Reducing the compressed framebuffer size. This may lead to less power savings than a non-reduced-size. Try to increase stolen memory size if available in BIOS.
This symptom might occur if something causes the DRM-KMS driver to probe the display while it's muxed away, for example a modeset or DPMS state change.
- When switching the mux back to a GPU that was previously driving a
mode, it is necessary to at the very least retrain DP links to restore the previously displayed image. In a proof of concept I have been experimenting with, I am able to accomplish this from userspace by triggering DPMS off and then back on again; however, it would be good to have an in-kernel API to request that an output owned by a DRM-KMS driver be refreshed to resume driving a mode on a disconnected and reconnected display. This API would need to be accessible from outside of the DRM-KMS driver handling the output. One reason it would be good to do this within the kernel, rather than rely on e.g. DPMS operations in the xf86-video-modesetting driver, is that it would be useful for restoring the console if X crashes or is forcefully killed while the mux is switched to a GPU other than the one which drives the console.
Basically, we'd like to be able to do the following:
- Communicate to a DRM-KMS driver that an output is disconnected and
can't be used. Ideally, DRI clients such as X should still see the output as being connected, so user applications don't need to keep track of the change. 2) Request that a mode that was previously driven on a disconnected output be driven again upon reconnection.
If APIs to do the above are already available, I wasn't able to find information about them. These could be handled as separate APIs, e.g., one to set connected/disconnected state and another to restore an output, or as a single API, e.g., signal a disconnect or reconnect, leaving it up to the driver receiving the signal to set the appropriate internal state and restore the reconnected output. Another possibility would be an API to disable and enable individual outputs from outside of the DRM-KMS driver that owns them. I'm curious to hear the thoughts of the DRM subsystem maintainers and contributors on what the best approach to this would be.
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
-- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
On 4/1/20 1:46 AM, Daniel Vetter wrote
On Wed, Apr 1, 2020 at 3:58 AM Daniel Dadap ddadap@nvidia.com wrote:
On 3/31/20 2:32 AM, Daniel Vetter wrote:
Since I see no mention of this anywhere in your mail ... have you tried looking at drivers/gpu/vga/vga_switcheroo.c? This also supports switching of just outputs, not just the old optimus way of switching out the entire gpu and having to disable the other one.
We did look into vga-switcheroo while developing our PoC, but found it insufficient for supporting the use case we had in mind in its current form. Among the limitations we observed were that it didn't seem to be possible to switch with output-level granularity, only with gpu-level granularity, with the whole switched-away-to GPU being powered down. Your description suggests that this is indeed possible, but if that's the case, then the mechanism for doing so isn't obvious in what I can see of the API from the kernel source code, even in the drm-next tree. Do you have pointers to documentation on how the per-output switching is supposed to work?
I think the per-output switch isn't support (yet), but otherwise there should be two modes:
- power switch mode, other gpu has to be offlined. This also means
switching needs to wait until all the clients are shut down.
- output switching only. That seems to be roughly what you want,
except you want to switch more than just the integrated panel. I think all models thus far simply wired all external outputs to the dgpu always.
Is there some tree with an early version of the output switching implemented that you can point to for testing?
I primarily asked about vgaswitcheroo since you didn't mention it at all.
I had actually anticipated that vga-switcheroo would likely be suggested, and my first draft of my initial message had a lengthy explanation about why vga-switcheroo (in its current state) isn't suitable, but figured it would be better to simplify the problem statement and then discuss switcheroo's limitations if and when it came up in discussion.
Other limitations of vga-switcheroo included:
- The can_switch() callbacks for all current vga-switcheroo-capable GPU
drivers don't seem to allow for a switch to occur while there are active KMS clients. This has the effect of making it so that non-deferred switches can only be initiated under the same circumstances that deferred switches wait for.
- It's only possible to address one mux device. Some systems have
separate muxes for internal and external displays. From what I could see in existing vga-switcheroo mux handlers, it seems that multi-muxed systems just switch all of the muxes simultaneously, which makes sense for the all-or-nothing "power down the GPU not in use" approach, but might not be desirable for fine-grained output-level switching.
- On some systems, it's possible to put the panel into a self-refresh
mode before switching the mux and exit self-refresh mode after switching, to hide any glitches that might otherwise appear while transitioning. Additional handler callbacks for pre-switch and post-switch operations would be useful.
If vga-switcheroo could be updated to address these limitatons, then it could make sense to handle the display disconnect/connect notifications and display refreshing as part of a vga_switcheroo client driver's set_gpu_state() callback (or a finer-grained per-output callback); however, it also seems that it should be possible to implement APIs within the DRM subsystem to accomplish the desired functionality independently of current or future vga-switcheroo design.
vgaswitcheroo was written by Dave Airlie, doesn't get much more "within the gpu subsystem". I think we should look into extending vgaswitcheroo instead of inventing a completely new uapi. If we go with a simplified use-case of
- only integrated panel switches
- external output on the dgpu
then that should match existing systems, so you get support on at least some desktops for free. Plus no headaches about userspace we need for merging new uapi. Once we have that (for your mux, and I guess with i915+nouveau all working) then we can look into how to extend it, or give the same underlying stuff an interface that's not in debugfs :-)
If output-based switching is already in development, have you addressed the "permanently connected" assumption for eDP displays as part of this?
Also for your seamless switching with self-refresh this could be implemented without new userspace, greatly simplifying everything.
Cheers, Daniel
There's some rough corners (like the uapi doesn't exist, it's in debugfs), and the locking has an inversion problem (I have ideas), but generally what you want exists already. -Daniel
On Mon, Mar 30, 2020 at 9:12 AM Daniel Dadap ddadap@nvidia.com wrote:
A number of hybrid GPU notebook computer designs with dual (integrated plus discrete) GPUs are equipped with multiplexers (muxes) that allow display panels to be driven by either the integrated GPU or the discrete GPU. Typically, this is a selection that can be made at boot time as a menu option in the system firmware's setup screen, and the mux selection stays fixed for as long as the system is running and persists across reboots until it is explicitly changed. However, some muxed hybrid GPU systems have dynamically switchable muxes which can be switched while the system is running.
NVIDIA is exploring the possibility of taking advantage of dynamically switchable muxes to enhance the experience of using a hybrid GPU system. For example, on a system configured for PRIME render offloading, it may be possible to keep the discrete GPU powered down and use the integrated GPU for rendering and displaying the desktop when no applications are using the discrete GPU, and dynamically switch the panel to be driven directly by the discrete GPU when render-offloading a fullscreen application.
We have been conducting some experiments on systems with dynamic muxes, and have found some limitations that would need to be addressed in order to support use cases like the one suggested above:
- In at least the i915 DRM-KMS driver, and likely in other DRM-KMS
drivers as well, eDP panels are assumed to be always connected. This assumption is broken when the panel is muxed away, which can cause problems. A typical symptom is i915 repeatedly attempting to retrain the link, severely impacting system performance and printing messages like the following every five seconds or so:
[drm:intel_dp_start_link_train [i915]] *ERROR* failed to enable link training [drm] Reducing the compressed framebuffer size. This may lead to less power savings than a non-reduced-size. Try to increase stolen memory size if available in BIOS.
This symptom might occur if something causes the DRM-KMS driver to probe the display while it's muxed away, for example a modeset or DPMS state change.
- When switching the mux back to a GPU that was previously driving a
mode, it is necessary to at the very least retrain DP links to restore the previously displayed image. In a proof of concept I have been experimenting with, I am able to accomplish this from userspace by triggering DPMS off and then back on again; however, it would be good to have an in-kernel API to request that an output owned by a DRM-KMS driver be refreshed to resume driving a mode on a disconnected and reconnected display. This API would need to be accessible from outside of the DRM-KMS driver handling the output. One reason it would be good to do this within the kernel, rather than rely on e.g. DPMS operations in the xf86-video-modesetting driver, is that it would be useful for restoring the console if X crashes or is forcefully killed while the mux is switched to a GPU other than the one which drives the console.
Basically, we'd like to be able to do the following:
- Communicate to a DRM-KMS driver that an output is disconnected and
can't be used. Ideally, DRI clients such as X should still see the output as being connected, so user applications don't need to keep track of the change. 2) Request that a mode that was previously driven on a disconnected output be driven again upon reconnection.
If APIs to do the above are already available, I wasn't able to find information about them. These could be handled as separate APIs, e.g., one to set connected/disconnected state and another to restore an output, or as a single API, e.g., signal a disconnect or reconnect, leaving it up to the driver receiving the signal to set the appropriate internal state and restore the reconnected output. Another possibility would be an API to disable and enable individual outputs from outside of the DRM-KMS driver that owns them. I'm curious to hear the thoughts of the DRM subsystem maintainers and contributors on what the best approach to this would be.
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
-- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
-- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
Hi Daniel (another one!),
On Thu, 2 Apr 2020 at 08:18, Daniel Dadap ddadap@nvidia.com wrote:
I primarily asked about vgaswitcheroo since you didn't mention it at all.
I had actually anticipated that vga-switcheroo would likely be suggested, and my first draft of my initial message had a lengthy explanation about why vga-switcheroo (in its current state) isn't suitable, but figured it would be better to simplify the problem statement and then discuss switcheroo's limitations if and when it came up in discussion.
Luckily, it isn't immutable! cf. https://lwn.net/Articles/443531/
I'd like to reiterate this from Jani as well:
I think lying to the userspace in this case is a hack to enable a very specific use case. Of course it's going to seem easier if you're only looking at enabling that narrowly defined feature. But usually the hacks end up being restrictions that make future work harder.
I understand the motivation to keep things as tightly contained as possible, but we already have enough complexity that it's important to avoid magic functionality that is totally transparent and handled without anyone needing to know, except for when it isn't actually transparent and people do need to specifically handle it and then it's a really spiky corner of API.
Cheers, Daniel
On Fri, Mar 27, 2020 at 04:25:19PM -0500, Daniel Dadap wrote:
A number of hybrid GPU notebook computer designs with dual (integrated plus discrete) GPUs are equipped with multiplexers (muxes) that allow display panels to be driven by either the integrated GPU or the discrete GPU. Typically, this is a selection that can be made at boot time as a menu option in the system firmware's setup screen, and the mux selection stays fixed for as long as the system is running and persists across reboots until it is explicitly changed. However, some muxed hybrid GPU systems have dynamically switchable muxes which can be switched while the system is running.
As you may be aware, there's drivers/gpu/vga/vga_switcheroo.c (of which I'm listed as a reviewer in MAINTAINERS) to support such hardware.
It also supports muxed configurations, including those that support switching at runtime (and not only at boot) such as the MacBook Pro, which uses drivers/platform/x86/apple-gmux.c to interface between vga_switcheroo and the hardware mux.
However, so far switching only actually works on LVDS-based MacBook Pros, i.e. all pre-retina machines introduced between Late 2008 and Mid 2012, because that hardware is capable of switching the DDC pins separately from the display, so we lock and switch them when probing the EDID.
The retina machines introduced from Mid 2012 onward use eDP and run into the issues you're describing: The AUX channel cannot be switched separately from the display, so link training fails unless the entire display is switched. Nevertheless macOS can switch the panel seamlessly. So how are they doing it?
Well, I don't own a retina MacBook Pro, hence never got very far with supporting them, but I did some research and experiments in the 2015/2016 time frame which a colleague, Bruno Bierbaumer, tested on his machine:
First of all, there's DPCD byte 3 bit 6 (NO_AUX_HANDSHAKE_LINK_TRAINING) which is documented as follows:
Does not require AUX CH handshake when the link configuration is already known. [...] The known-good drive current and pre-emphasis level (or those used in the last "full" link training with AUX CH handshake) must be used when the link training is performed without AUX CH handshake.
That bit is set on the MacBook Pros in question.
So I think what we should be doing here is that the DRM driver which happens to be muxed to the panel on boot performs link training and informs vga_switcheroo of the drive current, pre-emph level, etc. The other DRM driver is notified when that information is available and uses it to set up its eDP output, skipping an actual AUX CH handshake.
At least i915 probes various capabilities in the DPCD without any consideration that the AUX channel may currently not be available. Back in the day I experimented with a read-only proxy mechanism to make that work, whereby the inactive DRM driver uses the active DRM driver to access the DPCD:
https://patchwork.kernel.org/patch/7000591/
An alternative would be to have the active DRM driver cache the required portions of the DPCD for use by the inactive DRM driver.
Note that vga_switcheroo is currently controlled via debugfs. That is a historic artefact. The kernel has since gained a mux subsystem in drivers/mux/ which could be used to represent the display mux in a standardized way in regular sysfs.
Thanks,
Lukas
On Thu, Apr 02, 2020 at 01:39:25PM +0200, Lukas Wunner wrote:
First of all, there's DPCD byte 3 bit 6 (NO_AUX_HANDSHAKE_LINK_TRAINING) which is documented as follows:
Does not require AUX CH handshake when the link configuration is already known. [...] The known-good drive current and pre-emphasis level (or those used in the last "full" link training with AUX CH handshake) must be used when the link training is performed without AUX CH handshake.That bit is set on the MacBook Pros in question.
So I think what we should be doing here is that the DRM driver which happens to be muxed to the panel on boot performs link training and informs vga_switcheroo of the drive current, pre-emph level, etc. The other DRM driver is notified when that information is available and uses it to set up its eDP output, skipping an actual AUX CH handshake.
Actually I now remember that I already prepared for this by adding a vga_switcheroo handler capability:
* @VGA_SWITCHEROO_NEEDS_EDP_CONFIG: whether the handler is unable to switch * the AUX channel separately. This signals to clients that the active * GPU needs to train the link and communicate the link parameters to the * inactive GPU (mediated by vga_switcheroo). The inactive GPU may then * skip the AUX handshake and set up its output with these pre-calibrated * values (DisplayPort specification v1.1a, section 2.5.3.3)
And the referenced DP spec section says:
For a closed, embedded connection, the DisplayPort transmitter and receiver may be set to pre-calibrated parameters without going through the full link training sequence. In this mode, the DisplayPort transmitter may start a normal operation following the transmission of the Clock Recovery Pattern with pre-calibrated drive current and pre-emphasis level, as shown with a dotted arrow in Figure 2-36.
Thanks,
Lukas
On Thu, 2 Apr 2020 13:39:25 +0200 Lukas Wunner lukas@wunner.de wrote:
Note that vga_switcheroo is currently controlled via debugfs. That is a historic artefact. The kernel has since gained a mux subsystem in drivers/mux/ which could be used to represent the display mux in a standardized way in regular sysfs.
Hi,
if mux control was in sysfs, then how would userspace figure out which mux state refers to which DRM device and its connector?
Maybe some DRM connector property referring to the mux and its state?
How would a display server running as a regular user gain access to sysfs to control the mux?
Thanks, pq
On Thu, Apr 2, 2020 at 2:58 PM Pekka Paalanen ppaalanen@gmail.com wrote:
On Thu, 2 Apr 2020 13:39:25 +0200 Lukas Wunner lukas@wunner.de wrote:
Note that vga_switcheroo is currently controlled via debugfs. That is a historic artefact. The kernel has since gained a mux subsystem in drivers/mux/ which could be used to represent the display mux in a standardized way in regular sysfs.
Hi,
if mux control was in sysfs, then how would userspace figure out which mux state refers to which DRM device and its connector?
Maybe some DRM connector property referring to the mux and its state?
How would a display server running as a regular user gain access to sysfs to control the mux?
I think a sysfs interface for vgaswitcheroo would need some kind of links from drm_device (and for the outpu-only mux from drm_connector sysfs files) to the corresponding sysfs directory for the vgaswitcheroo switch.
But actual change of state needs to indeed be done through some other interface, since sysfs interfaces suck for compositors. We have this problem already with backlight, and the solution there is a special backlight frobbing services which compositors can call through udev, and fun stuff like that. Definitely not an uapi pattern we want to repeat. So I think sysfs would be only for information links, and maybe the global switch for the old optimus designs where you can only switch the entire gpu, and need to delay the switch until all existing clients are closed. Aka 1) stop compositor 2) write to sysfs 3) start new compositor on new gpu. For that case compositors don't need to ever write to sysfs themselves, so I think that's ok. Also only relevant for rather old hw. -Daniel
On Thu, Apr 02, 2020 at 03:13:26PM +0200, Daniel Vetter wrote:
On Thu, Apr 2, 2020 at 2:58 PM Pekka Paalanen ppaalanen@gmail.com wrote:
On Thu, 2 Apr 2020 13:39:25 +0200 Lukas Wunner lukas@wunner.de wrote:
Note that vga_switcheroo is currently controlled via debugfs. That is a historic artefact. The kernel has since gained a mux subsystem in drivers/mux/ which could be used to represent the display mux in a standardized way in regular sysfs.
if mux control was in sysfs, then how would userspace figure out which mux state refers to which DRM device and its connector?
Maybe some DRM connector property referring to the mux and its state?
How would a display server running as a regular user gain access to sysfs to control the mux?
I think a sysfs interface for vgaswitcheroo would need some kind of links from drm_device (and for the outpu-only mux from drm_connector sysfs files) to the corresponding sysfs directory for the vgaswitcheroo switch.
Yes, that would be one way to do it.
But actual change of state needs to indeed be done through some other interface, since sysfs interfaces suck for compositors. We have this problem already with backlight, and the solution there is a special backlight frobbing services which compositors can call through udev, and fun stuff like that. Definitely not an uapi pattern we want to repeat. So I think sysfs would be only for information links, and maybe the global switch for the old optimus designs where you can only switch the entire gpu, and need to delay the switch until all existing clients are closed. Aka 1) stop compositor 2) write to sysfs 3) start new compositor on new gpu. For that case compositors don't need to ever write to sysfs themselves, so I think that's ok. Also only relevant for rather old hw.
There's an in-kernel interface in <linux/mux/consumer.h> which could be invoked from, say, an ioctl(), to control or query the mux.
I'm not really prepared to answer detailed questions how it should be done, my point is just that the existing building blocks (such as the mux subsystem) should be used instead of reinventing the wheel. If the mux subsystem lacks specific features needed for graphics muxes, I think adding them won't face much opposition from Peter Rosin.
Thanks,
Lukas
On Thu, Apr 2, 2020 at 4:31 PM Lukas Wunner lukas@wunner.de wrote:
On Thu, Apr 02, 2020 at 03:13:26PM +0200, Daniel Vetter wrote:
On Thu, Apr 2, 2020 at 2:58 PM Pekka Paalanen ppaalanen@gmail.com wrote:
On Thu, 2 Apr 2020 13:39:25 +0200 Lukas Wunner lukas@wunner.de wrote:
Note that vga_switcheroo is currently controlled via debugfs. That is a historic artefact. The kernel has since gained a mux subsystem in drivers/mux/ which could be used to represent the display mux in a standardized way in regular sysfs.
if mux control was in sysfs, then how would userspace figure out which mux state refers to which DRM device and its connector?
Maybe some DRM connector property referring to the mux and its state?
How would a display server running as a regular user gain access to sysfs to control the mux?
I think a sysfs interface for vgaswitcheroo would need some kind of links from drm_device (and for the outpu-only mux from drm_connector sysfs files) to the corresponding sysfs directory for the vgaswitcheroo switch.
Yes, that would be one way to do it.
But actual change of state needs to indeed be done through some other interface, since sysfs interfaces suck for compositors. We have this problem already with backlight, and the solution there is a special backlight frobbing services which compositors can call through udev, and fun stuff like that. Definitely not an uapi pattern we want to repeat. So I think sysfs would be only for information links, and maybe the global switch for the old optimus designs where you can only switch the entire gpu, and need to delay the switch until all existing clients are closed. Aka 1) stop compositor 2) write to sysfs 3) start new compositor on new gpu. For that case compositors don't need to ever write to sysfs themselves, so I think that's ok. Also only relevant for rather old hw.
There's an in-kernel interface in <linux/mux/consumer.h> which could be invoked from, say, an ioctl(), to control or query the mux.
I'm not really prepared to answer detailed questions how it should be done, my point is just that the existing building blocks (such as the mux subsystem) should be used instead of reinventing the wheel. If the mux subsystem lacks specific features needed for graphics muxes, I think adding them won't face much opposition from Peter Rosin.
I guess if someone is bored they could port the internal vgaswitcheroo mux drivers over to that mux subsystem. But we'll still need some amount of graphics specific glue, and that seems perfectly reasonable to have in vgaswitcheroo.c. That stuff isn't about a mux, but the hand-off between the involved gpu (and audio) drivers, to me it doesn't make sense to push all these kind of interactions into a generic mux subsystem.
That also applies to the extensions discussed here to better handle per-output muxing between drivers (like the self refresh trick for seamless muxing). That's all graphics, and in practice already possible with the hw vgaswitcheroo supports today (apple does that already in macos). No change on the mux driver side needed at all I think.
so tldr I guess if you want add a todo entry to look into porting vgaswitcheroo drivers to the mux framework. But imo that would be orthogonal to what we're discussing here. -Daniel
On 4/2/20 6:39 AM, Lukas Wunner wrote:
External email: Use caution opening links or attachments
On Fri, Mar 27, 2020 at 04:25:19PM -0500, Daniel Dadap wrote:
A number of hybrid GPU notebook computer designs with dual (integrated plus discrete) GPUs are equipped with multiplexers (muxes) that allow display panels to be driven by either the integrated GPU or the discrete GPU. Typically, this is a selection that can be made at boot time as a menu option in the system firmware's setup screen, and the mux selection stays fixed for as long as the system is running and persists across reboots until it is explicitly changed. However, some muxed hybrid GPU systems have dynamically switchable muxes which can be switched while the system is running.
As you may be aware, there's drivers/gpu/vga/vga_switcheroo.c (of which I'm listed as a reviewer in MAINTAINERS) to support such hardware.
It also supports muxed configurations, including those that support switching at runtime (and not only at boot) such as the MacBook Pro, which uses drivers/platform/x86/apple-gmux.c to interface between vga_switcheroo and the hardware mux.
However, so far switching only actually works on LVDS-based MacBook Pros, i.e. all pre-retina machines introduced between Late 2008 and Mid 2012, because that hardware is capable of switching the DDC pins separately from the display, so we lock and switch them when probing the EDID.
I have observed that on at least some systems, the EDID for the internal panel can be read via the ACPI _DDC method regardless of whether it's actively muxed in. I don't know whether that's true for all systems where the DDC line can't be switched independently, but maybe vga_switcheroo could also export an interface for GPU drivers to cache EDIDs so that a muxed-away GPU can read an EDID that was previously read by another GPU? I guess the utility of that would depend on how prevalent the combination of no DDC muxing + no ACPI EDID reads turns out to be.
The retina machines introduced from Mid 2012 onward use eDP and run into the issues you're describing: The AUX channel cannot be switched separately from the display, so link training fails unless the entire display is switched. Nevertheless macOS can switch the panel seamlessly. So how are they doing it?
Well, I don't own a retina MacBook Pro, hence never got very far with supporting them, but I did some research and experiments in the 2015/2016 time frame which a colleague, Bruno Bierbaumer, tested on his machine:
First of all, there's DPCD byte 3 bit 6 (NO_AUX_HANDSHAKE_LINK_TRAINING) which is documented as follows:
Does not require AUX CH handshake when the link configuration is already known. [...] The known-good drive current and pre-emphasis level (or those used in the last "full" link training with AUX CH handshake) must be used when the link training is performed without AUX CH handshake.That bit is set on the MacBook Pros in question.
I'll check one of the eDP-based systems I've been experimenting on to see if setting the VGA_SWITCHER_NEEDS_EDP_CONFIG capability in the handler is sufficient to make i915 avoid poking the AUX channel when it's mux-switched away. (This would be in addition to hacking the can_switch() callback in the GPU drivers to allow switching while there are still active KMS clients for the purposes of this experiment, unless somebody can point me to a tree with the WIP per-output switching Daniel Vetter mentioned.
So I think what we should be doing here is that the DRM driver which happens to be muxed to the panel on boot performs link training and informs vga_switcheroo of the drive current, pre-emph level, etc. The other DRM driver is notified when that information is available and uses it to set up its eDP output, skipping an actual AUX CH handshake.
At least i915 probes various capabilities in the DPCD without any consideration that the AUX channel may currently not be available. Back in the day I experimented with a read-only proxy mechanism to make that work, whereby the inactive DRM driver uses the active DRM driver to access the DPCD:
https://patchwork.kernel.org/patch/7000591/
An alternative would be to have the active DRM driver cache the required portions of the DPCD for use by the inactive DRM driver.
Note that vga_switcheroo is currently controlled via debugfs. That is a historic artefact. The kernel has since gained a mux subsystem in drivers/mux/ which could be used to represent the display mux in a standardized way in regular sysfs.
Thanks,
Lukas
On Thu, Apr 02, 2020 at 12:56:04PM -0500, Daniel Dadap wrote:
I'll check one of the eDP-based systems I've been experimenting on to see if setting the VGA_SWITCHER_NEEDS_EDP_CONFIG capability in the handler is sufficient to make i915 avoid poking the AUX channel when it's mux-switched away.
That will not be sufficient I'm afraid. The capability was added in preparation for taking advantage of pre-calibration by the active GPU as permitted by the DP spec, but that feature only ever existed in the experimental patches I linked to and which I never got around to completing (so far).
BTW, if the inactive GPU sets up the eDP output with the precalibrated values and the mux is switched without using the PSR trick you've mentioned, will the transition be glitchy?
If so, is there a chance to switch the mux in a vblank to avoid glitches?
(This would be in addition to hacking the can_switch() callback in the GPU drivers to allow switching while there are still active KMS clients for the purposes of this experiment, unless somebody can point me to a tree with the WIP per-output switching Daniel Vetter mentioned.
I'm not aware anyone ever bothered implementing per-output switching. Which hardware supports that anyway? I documented the situation on the MacBook Pro in apple-gmux.c:
* The external DP port is only fully switchable on the first two unibody * MacBook Pro generations, MBP5 2008/09 and MBP6 2010. * The following MacBook Pro generations replaced the external DP port with a * combined DP/Thunderbolt port and lost the ability to switch it between GPUs, * connecting it either to the discrete GPU or the Thunderbolt controller. So only very old machines can switch the external DP port separately. We just switch it in unison with the LVDS panel on those machines for simplicity.
I'm not aware of other contemporary machines besides MacBook Pros which allow switching the panel at runtime, let alone switch external ports separately. Are you at liberty to divulge what hardware you're working on? Just curious.
Thanks,
Lukas
On 4/2/20 1:25 PM, Lukas Wunner wrote:
On Thu, Apr 02, 2020 at 12:56:04PM -0500, Daniel Dadap wrote:
I'll check one of the eDP-based systems I've been experimenting on to see if setting the VGA_SWITCHER_NEEDS_EDP_CONFIG capability in the handler is sufficient to make i915 avoid poking the AUX channel when it's mux-switched away.
That will not be sufficient I'm afraid. The capability was added in preparation for taking advantage of pre-calibration by the active GPU as permitted by the DP spec, but that feature only ever existed in the experimental patches I linked to and which I never got around to completing (so far).
Ah, I see. I'll take a look at those patches and see how far they go, then.
I just remembered another issue that falls out from the assumption that eDP is always connected: in my test setup, i915 always sees and advertises a connected eDP panel, even if the mux is switched away from i915's GPU before e.g. starting X. Presumably the panel was probed when starting up i915 to drive the console, and since eDP can't ever be disconnected in i915, it just remains "connected" forever. So I'm not sure that just proxying the aux channel through the switched-to GPU's driver will be quite enough, either, if we're not going to be hiding the mux switch from clients.
BTW, if the inactive GPU sets up the eDP output with the precalibrated values and the mux is switched without using the PSR trick you've mentioned, will the transition be glitchy?
If so, is there a chance to switch the mux in a vblank to avoid glitches?
I don't know. In my current PoC I'm just using X11 DPMS to shut down the head when it's muxed away and bring it back up when it's muxed back in, as a way to trigger link training when muxing back in and to shut off the head when muxed away. This leads to a rather lengthy and conspicuous blanking glitch on systems without PSR, and a conspicuous hesitation on systems with PSR. If we can refresh the display without doing a full modeset (I think DPMS in X.org's modesetting_drv.so does a full modeset, but I didn't investigate particularly closely), then I expect the duration of the switch can be reduced substantially, but I don't know to what extent it would still glitch.
(This would be in addition to hacking the can_switch() callback in the GPU drivers to allow switching while there are still active KMS clients for the purposes of this experiment, unless somebody can point me to a tree with the WIP per-output switching Daniel Vetter mentioned.
I'm not aware anyone ever bothered implementing per-output switching. Which hardware supports that anyway? I documented the situation on the MacBook Pro in apple-gmux.c:
- The external DP port is only fully switchable on the first two unibody
- MacBook Pro generations, MBP5 2008/09 and MBP6 2010.
- The following MacBook Pro generations replaced the external DP port with a
- combined DP/Thunderbolt port and lost the ability to switch it between GPUs,
- connecting it either to the discrete GPU or the Thunderbolt controller.
So only very old machines can switch the external DP port separately. We just switch it in unison with the LVDS panel on those machines for simplicity.
Yes, I noticed that apple-gmux switches both muxes in unison when I was initially evaluating vga-switcheroo. Daniel Vetter mentioned per-output switching was possible, without shutting down the switched-away-from GPU, but I didn't see any evidence of such functionality in the torvalds/linux tree or the drm/drm-next tree. I didn't search anywhere beyond that. Daniel, could you point out where the per-output switching support you mentioned can be found?
I'm not aware of other contemporary machines besides MacBook Pros which allow switching the panel at runtime, let alone switch external ports separately. Are you at liberty to divulge what hardware you're working on? Just curious.
Sorry, not at this time.
On Fri, Apr 3, 2020 at 8:54 AM Daniel Dadap ddadap@nvidia.com wrote:
On 4/2/20 6:39 AM, Lukas Wunner wrote:
External email: Use caution opening links or attachments
On Fri, Mar 27, 2020 at 04:25:19PM -0500, Daniel Dadap wrote:
A number of hybrid GPU notebook computer designs with dual (integrated plus discrete) GPUs are equipped with multiplexers (muxes) that allow display panels to be driven by either the integrated GPU or the discrete GPU. Typically, this is a selection that can be made at boot time as a menu option in the system firmware's setup screen, and the mux selection stays fixed for as long as the system is running and persists across reboots until it is explicitly changed. However, some muxed hybrid GPU systems have dynamically switchable muxes which can be switched while the system is running.
As you may be aware, there's drivers/gpu/vga/vga_switcheroo.c (of which I'm listed as a reviewer in MAINTAINERS) to support such hardware.
It also supports muxed configurations, including those that support switching at runtime (and not only at boot) such as the MacBook Pro, which uses drivers/platform/x86/apple-gmux.c to interface between vga_switcheroo and the hardware mux.
However, so far switching only actually works on LVDS-based MacBook Pros, i.e. all pre-retina machines introduced between Late 2008 and Mid 2012, because that hardware is capable of switching the DDC pins separately from the display, so we lock and switch them when probing the EDID.
I have observed that on at least some systems, the EDID for the internal panel can be read via the ACPI _DDC method regardless of whether it's actively muxed in. I don't know whether that's true for all systems where the DDC line can't be switched independently, but maybe vga_switcheroo could also export an interface for GPU drivers to cache EDIDs so that a muxed-away GPU can read an EDID that was previously read by another GPU? I guess the utility of that would depend on how prevalent the combination of no DDC muxing + no ACPI EDID reads turns out to be.
The retina machines introduced from Mid 2012 onward use eDP and run into the issues you're describing: The AUX channel cannot be switched separately from the display, so link training fails unless the entire display is switched. Nevertheless macOS can switch the panel seamlessly. So how are they doing it?
Well, I don't own a retina MacBook Pro, hence never got very far with supporting them, but I did some research and experiments in the 2015/2016 time frame which a colleague, Bruno Bierbaumer, tested on his machine:
First of all, there's DPCD byte 3 bit 6 (NO_AUX_HANDSHAKE_LINK_TRAINING) which is documented as follows:
Does not require AUX CH handshake when the link configuration is already known. [...] The known-good drive current and pre-emphasis level (or those used in the last "full" link training with AUX CH handshake) must be used when the link training is performed without AUX CH handshake.That bit is set on the MacBook Pros in question.
I'll check one of the eDP-based systems I've been experimenting on to see if setting the VGA_SWITCHER_NEEDS_EDP_CONFIG capability in the handler is sufficient to make i915 avoid poking the AUX channel when it's mux-switched away. (This would be in addition to hacking the can_switch() callback in the GPU drivers to allow switching while there are still active KMS clients for the purposes of this experiment, unless somebody can point me to a tree with the WIP per-output switching Daniel Vetter mentioned.
Two things: I thought (but not sure) that for the output switching muxes we'd run vgaswitcheroo in a different mode, where it doesn't check whether whether the driver can be killed. Because it wont. On a quick search only thing I've found is the ddc-only switching done by vga_switcheroo_lock/unlock_ddc. Maybe misremembering, but I thought there was more. But been a while I last looked at this all in detail.
Wrt per-output switching WIP branch. That would be something you'd need to type ofc, I was just laying out what I think would make sense as a possible path to integrate this into upstream. -Daniel
So I think what we should be doing here is that the DRM driver which happens to be muxed to the panel on boot performs link training and informs vga_switcheroo of the drive current, pre-emph level, etc. The other DRM driver is notified when that information is available and uses it to set up its eDP output, skipping an actual AUX CH handshake.
At least i915 probes various capabilities in the DPCD without any consideration that the AUX channel may currently not be available. Back in the day I experimented with a read-only proxy mechanism to make that work, whereby the inactive DRM driver uses the active DRM driver to access the DPCD:
https://patchwork.kernel.org/patch/7000591/
An alternative would be to have the active DRM driver cache the required portions of the DPCD for use by the inactive DRM driver.
Note that vga_switcheroo is currently controlled via debugfs. That is a historic artefact. The kernel has since gained a mux subsystem in drivers/mux/ which could be used to represent the display mux in a standardized way in regular sysfs.
Thanks,
Lukas
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
On 4/3/20 2:16 AM, Daniel Vetter wrote:
On Fri, Apr 3, 2020 at 8:54 AM Daniel Dadap ddadap@nvidia.com wrote:
On 4/2/20 6:39 AM, Lukas Wunner wrote:
On Fri, Mar 27, 2020 at 04:25:19PM -0500, Daniel Dadap wrote:
A number of hybrid GPU notebook computer designs with dual (integrated plus discrete) GPUs are equipped with multiplexers (muxes) that allow display panels to be driven by either the integrated GPU or the discrete GPU. Typically, this is a selection that can be made at boot time as a menu option in the system firmware's setup screen, and the mux selection stays fixed for as long as the system is running and persists across reboots until it is explicitly changed. However, some muxed hybrid GPU systems have dynamically switchable muxes which can be switched while the system is running.
As you may be aware, there's drivers/gpu/vga/vga_switcheroo.c (of which I'm listed as a reviewer in MAINTAINERS) to support such hardware.
It also supports muxed configurations, including those that support switching at runtime (and not only at boot) such as the MacBook Pro, which uses drivers/platform/x86/apple-gmux.c to interface between vga_switcheroo and the hardware mux.
However, so far switching only actually works on LVDS-based MacBook Pros, i.e. all pre-retina machines introduced between Late 2008 and Mid 2012, because that hardware is capable of switching the DDC pins separately from the display, so we lock and switch them when probing the EDID.
I have observed that on at least some systems, the EDID for the internal panel can be read via the ACPI _DDC method regardless of whether it's actively muxed in. I don't know whether that's true for all systems where the DDC line can't be switched independently, but maybe vga_switcheroo could also export an interface for GPU drivers to cache EDIDs so that a muxed-away GPU can read an EDID that was previously read by another GPU? I guess the utility of that would depend on how prevalent the combination of no DDC muxing + no ACPI EDID reads turns out to be.
The retina machines introduced from Mid 2012 onward use eDP and run into the issues you're describing: The AUX channel cannot be switched separately from the display, so link training fails unless the entire display is switched. Nevertheless macOS can switch the panel seamlessly. So how are they doing it?
Well, I don't own a retina MacBook Pro, hence never got very far with supporting them, but I did some research and experiments in the 2015/2016 time frame which a colleague, Bruno Bierbaumer, tested on his machine:
First of all, there's DPCD byte 3 bit 6 (NO_AUX_HANDSHAKE_LINK_TRAINING) which is documented as follows:
Does not require AUX CH handshake when the link configuration is already known. [...] The known-good drive current and pre-emphasis level (or those used in the last "full" link training with AUX CH handshake) must be used when the link training is performed without AUX CH handshake.That bit is set on the MacBook Pros in question.
I'll check one of the eDP-based systems I've been experimenting on to see if setting the VGA_SWITCHER_NEEDS_EDP_CONFIG capability in the handler is sufficient to make i915 avoid poking the AUX channel when it's mux-switched away. (This would be in addition to hacking the can_switch() callback in the GPU drivers to allow switching while there are still active KMS clients for the purposes of this experiment, unless somebody can point me to a tree with the WIP per-output switching Daniel Vetter mentioned.
Two things: I thought (but not sure) that for the output switching muxes we'd run vgaswitcheroo in a different mode, where it doesn't check whether whether the driver can be killed. Because it wont. On a quick search only thing I've found is the ddc-only switching done by vga_switcheroo_lock/unlock_ddc. Maybe misremembering, but I thought there was more. But been a while I last looked at this all in detail.
Wrt per-output switching WIP branch. That would be something you'd need to type ofc, I was just laying out what I think would make sense as a possible path to integrate this into upstream. -Daniel
Okay. I misunderstood. When you said that vga-switcheroo could switch individual outputs and do so without powering down the switched-away-from GPU, I took that to mean that this feature had already been implemented somewhere, despite appearances to the contrary upstream. I agree that adding per-output switching support to vga-switcheroo would be a sensible path.
Does this sound like a sensible high-level design?
* vga-switcheroo-capable GPU drivers can register muxable outputs. * Each GPU driver must register each unique muxable output with the same identifier. The outputs will be registered together with individual devices they can be muxed to, in order to support e.g. muxing between different GPU devices driven by the same vendor. (I'm not aware of any designs that actually support this, but it seems reasonable to design for.) The output identifier may be platform-specific (e.g. ACPI handle, simple index). For example, two GPU drivers may each register an internal panel with identifier "0" and an external port with identifier "1". * For each output that was registered with more than one device, vga-switcheroo will expose a sysfs file listing the devices that output can be muxed to, and a sysfs file allowing the selection of a particular device. Perhaps these could be the same file (e.g., on reading it prints a list with an indicator to mark the currently selected device, write a device ID to switch to that device).
I think this would allow flexibility across N muxable outputs that can each be switched to one of M GPU devices, without requiring that each of the outputs be switchable to all M possible GPUs.
In addition to the above mechanism for advertising the switchability of individual outputs and providing an interface, the following would also be needed to support PSR:
* A new callback for GPU drivers so vga-switcheroo can tell them an output is being switched to a GPU. * vga-switcheroo can call this callback for all devices once before switching the mux, and then again after the mux switch is complete. A parameter of the callback would indicate whether the mux switch is pending, completed, or failed, allowing GPU drivers to do any necessary pre-switch setup, post-switch cleanup, or failed-switch recovery in response to events from vga-switcheroo. * Maybe a similar callback for mux handlers so vga-switcheroo can signal pre- and post-switch events to the handler driver. The PSR designs I'm aware of are GPU-driven, but I can imagine a design that is driven by some component other than a GPU, in which case it would make sense for the mux handler to drive PSR. I suppose this could be left out for now and then added later if such a design emerges. * It would probably be good to let userspace drive the pre/switch/post operations independently, e.g. so that userspace can prepare the first frame for presentation on the switched-to GPU before PSR is disabled.
Now, I'm not sure if it makes more sense to also implement the AUX channel proxying that Lukas had prototyped a while back, or to update i915 and any other vga-switcheroo-compatible drivers that assume that eDP is always connected to allow for the possibility of eDP being disconnected due to being muxed away. I personally think disconnecting the eDP output makes more sense, since it more accurately reflects reality than proxying the AUX channel through to the GPU that's muxed in, but if there's another reason the AUX proxying makes more sense, we could plan on making that work (on a per-output basis, since I think e.g. regular DP for external connectors wouldn't need this) as well.
So I think what we should be doing here is that the DRM driver which happens to be muxed to the panel on boot performs link training and informs vga_switcheroo of the drive current, pre-emph level, etc. The other DRM driver is notified when that information is available and uses it to set up its eDP output, skipping an actual AUX CH handshake.
At least i915 probes various capabilities in the DPCD without any consideration that the AUX channel may currently not be available. Back in the day I experimented with a read-only proxy mechanism to make that work, whereby the inactive DRM driver uses the active DRM driver to access the DPCD:
https://patchwork.kernel.org/patch/7000591/
An alternative would be to have the active DRM driver cache the required portions of the DPCD for use by the inactive DRM driver.
Note that vga_switcheroo is currently controlled via debugfs. That is a historic artefact. The kernel has since gained a mux subsystem in drivers/mux/ which could be used to represent the display mux in a standardized way in regular sysfs.
Thanks,
Lukas
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
-- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
On Fri, Apr 3, 2020 at 8:06 PM Daniel Dadap ddadap@nvidia.com wrote:
On 4/3/20 2:16 AM, Daniel Vetter wrote:
On Fri, Apr 3, 2020 at 8:54 AM Daniel Dadap ddadap@nvidia.com wrote:
On 4/2/20 6:39 AM, Lukas Wunner wrote:
On Fri, Mar 27, 2020 at 04:25:19PM -0500, Daniel Dadap wrote:
A number of hybrid GPU notebook computer designs with dual (integrated plus discrete) GPUs are equipped with multiplexers (muxes) that allow display panels to be driven by either the integrated GPU or the discrete GPU. Typically, this is a selection that can be made at boot time as a menu option in the system firmware's setup screen, and the mux selection stays fixed for as long as the system is running and persists across reboots until it is explicitly changed. However, some muxed hybrid GPU systems have dynamically switchable muxes which can be switched while the system is running.
As you may be aware, there's drivers/gpu/vga/vga_switcheroo.c (of which I'm listed as a reviewer in MAINTAINERS) to support such hardware.
It also supports muxed configurations, including those that support switching at runtime (and not only at boot) such as the MacBook Pro, which uses drivers/platform/x86/apple-gmux.c to interface between vga_switcheroo and the hardware mux.
However, so far switching only actually works on LVDS-based MacBook Pros, i.e. all pre-retina machines introduced between Late 2008 and Mid 2012, because that hardware is capable of switching the DDC pins separately from the display, so we lock and switch them when probing the EDID.
I have observed that on at least some systems, the EDID for the internal panel can be read via the ACPI _DDC method regardless of whether it's actively muxed in. I don't know whether that's true for all systems where the DDC line can't be switched independently, but maybe vga_switcheroo could also export an interface for GPU drivers to cache EDIDs so that a muxed-away GPU can read an EDID that was previously read by another GPU? I guess the utility of that would depend on how prevalent the combination of no DDC muxing + no ACPI EDID reads turns out to be.
The retina machines introduced from Mid 2012 onward use eDP and run into the issues you're describing: The AUX channel cannot be switched separately from the display, so link training fails unless the entire display is switched. Nevertheless macOS can switch the panel seamlessly. So how are they doing it?
Well, I don't own a retina MacBook Pro, hence never got very far with supporting them, but I did some research and experiments in the 2015/2016 time frame which a colleague, Bruno Bierbaumer, tested on his machine:
First of all, there's DPCD byte 3 bit 6 (NO_AUX_HANDSHAKE_LINK_TRAINING) which is documented as follows:
Does not require AUX CH handshake when the link configuration is already known. [...] The known-good drive current and pre-emphasis level (or those used in the last "full" link training with AUX CH handshake) must be used when the link training is performed without AUX CH handshake.That bit is set on the MacBook Pros in question.
I'll check one of the eDP-based systems I've been experimenting on to see if setting the VGA_SWITCHER_NEEDS_EDP_CONFIG capability in the handler is sufficient to make i915 avoid poking the AUX channel when it's mux-switched away. (This would be in addition to hacking the can_switch() callback in the GPU drivers to allow switching while there are still active KMS clients for the purposes of this experiment, unless somebody can point me to a tree with the WIP per-output switching Daniel Vetter mentioned.
Two things: I thought (but not sure) that for the output switching muxes we'd run vgaswitcheroo in a different mode, where it doesn't check whether whether the driver can be killed. Because it wont. On a quick search only thing I've found is the ddc-only switching done by vga_switcheroo_lock/unlock_ddc. Maybe misremembering, but I thought there was more. But been a while I last looked at this all in detail.
Wrt per-output switching WIP branch. That would be something you'd need to type ofc, I was just laying out what I think would make sense as a possible path to integrate this into upstream. -Daniel
Okay. I misunderstood. When you said that vga-switcheroo could switch individual outputs and do so without powering down the switched-away-from GPU, I took that to mean that this feature had already been implemented somewhere, despite appearances to the contrary upstream. I agree that adding per-output switching support to vga-switcheroo would be a sensible path.
So I looked again, and maybe another clarification. What I think is already there is the use-case of switching one integrated panel. I checked the code now, this is supported with DRM_SWITCH_POWER_DYNAMIC_OFF. Essentially this allows you to always open the gpu cdev node, no matter the vgaswitcheroo state. And the vgaswitcheroo only controls where the outputs are connected (and most designs only switch the integrated panel, external outputs go to the discrete gpu).
The ddc switching is so that the external gpu can still probe the panel when it's connected to the integrated gpu.
What's not there is switching for outputs indivually.
Note that's at least my understanding, I dont have a system supporting this.
So my suggestio would be: - mayke the nvidia blob work with the current stuff first. I assume you'll need that :-) - look into fixing the locking and refcounting bugs. There's FIXME's all over the code, and we kinda need to address this before we make everything worse.
That gets us a baseline. Next up I think enabling the seamless edp switching would be nice. For that we don't really need any per-output switching yet, the current vgaswitcheroo model still works. But what we probably need: - some coordination between drm drivers, probably going through userspace so we follow the sequence correctly - a new uapi which userspace can use without going into debugfs (which is a no-go really for shipping to enduser). Probably something that's tied to drm_connectors so we can extend it later on - the userspace which makes this work seamlessly, if you want this with X that probably means -modesetting.
Does this sound like a sensible high-level design?
- vga-switcheroo-capable GPU drivers can register muxable outputs.
- Each GPU driver must register each unique muxable output with the same
identifier. The outputs will be registered together with individual devices they can be muxed to, in order to support e.g. muxing between different GPU devices driven by the same vendor. (I'm not aware of any designs that actually support this, but it seems reasonable to design for.) The output identifier may be platform-specific (e.g. ACPI handle, simple index). For example, two GPU drivers may each register an internal panel with identifier "0" and an external port with identifier "1".
- For each output that was registered with more than one device,
vga-switcheroo will expose a sysfs file listing the devices that output can be muxed to, and a sysfs file allowing the selection of a particular device. Perhaps these could be the same file (e.g., on reading it prints a list with an indicator to mark the currently selected device, write a device ID to switch to that device).
With my plan the above isn't needed, at least not at first. Naming and identifying stuff is a really hard problem, so probably best we delay it so we have more time to figure this out.
I think this would allow flexibility across N muxable outputs that can each be switched to one of M GPU devices, without requiring that each of the outputs be switchable to all M possible GPUs.
In addition to the above mechanism for advertising the switchability of individual outputs and providing an interface, the following would also be needed to support PSR:
- A new callback for GPU drivers so vga-switcheroo can tell them an
output is being switched to a GPU.
- vga-switcheroo can call this callback for all devices once before
switching the mux, and then again after the mux switch is complete. A parameter of the callback would indicate whether the mux switch is pending, completed, or failed, allowing GPU drivers to do any necessary pre-switch setup, post-switch cleanup, or failed-switch recovery in response to events from vga-switcheroo.
- Maybe a similar callback for mux handlers so vga-switcheroo can signal
pre- and post-switch events to the handler driver. The PSR designs I'm aware of are GPU-driven, but I can imagine a design that is driven by some component other than a GPU, in which case it would make sense for the mux handler to drive PSR. I suppose this could be left out for now and then added later if such a design emerges.
- It would probably be good to let userspace drive the pre/switch/post
operations independently, e.g. so that userspace can prepare the first frame for presentation on the switched-to GPU before PSR is disabled.
This more or less matches what I'm suggesting too. Except no new callbacks just because we might need them, we're only going to add what the drivers actually need to orchestrate a given use case. The drivers = i915 + nouveau + whatever vgaswitcheroo mux driver you have, all in upstream.
Now, I'm not sure if it makes more sense to also implement the AUX channel proxying that Lukas had prototyped a while back, or to update i915 and any other vga-switcheroo-compatible drivers that assume that eDP is always connected to allow for the possibility of eDP being disconnected due to being muxed away. I personally think disconnecting the eDP output makes more sense, since it more accurately reflects reality than proxying the AUX channel through to the GPU that's muxed in, but if there's another reason the AUX proxying makes more sense, we could plan on making that work (on a per-output basis, since I think e.g. regular DP for external connectors wouldn't need this) as well.
I think if we expect userspace to drive the seamless edp transition, then probably a disconnected state that reflects reality is needed.
otoh, and I think that's the driver behind Lukas' work: Current drivers need the edp state to set everything up at driver load. If they can only fully probe the features of the integrated panel later on it's too late. That's also why we have the ddx mux, so that drivers can load as if they're connected to the panel.
So I think we still need Lukas' edp proxying patches most likely. What the userspace visible connection_status will be like needs to be figured out with how exactly we're going to drive the switchover. -Daniel
So I think what we should be doing here is that the DRM driver which happens to be muxed to the panel on boot performs link training and informs vga_switcheroo of the drive current, pre-emph level, etc. The other DRM driver is notified when that information is available and uses it to set up its eDP output, skipping an actual AUX CH handshake.
At least i915 probes various capabilities in the DPCD without any consideration that the AUX channel may currently not be available. Back in the day I experimented with a read-only proxy mechanism to make that work, whereby the inactive DRM driver uses the active DRM driver to access the DPCD:
https://patchwork.kernel.org/patch/7000591/
An alternative would be to have the active DRM driver cache the required portions of the DPCD for use by the inactive DRM driver.
Note that vga_switcheroo is currently controlled via debugfs. That is a historic artefact. The kernel has since gained a mux subsystem in drivers/mux/ which could be used to represent the display mux in a standardized way in regular sysfs.
Thanks,
Lukas
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
-- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
On 4/3/20 2:59 PM, Daniel Vetter wrote:
On Fri, Apr 3, 2020 at 8:06 PM Daniel Dadap ddadap@nvidia.com wrote:
On 4/3/20 2:16 AM, Daniel Vetter wrote:
On Fri, Apr 3, 2020 at 8:54 AM Daniel Dadap ddadap@nvidia.com wrote:
On 4/2/20 6:39 AM, Lukas Wunner wrote:
On Fri, Mar 27, 2020 at 04:25:19PM -0500, Daniel Dadap wrote:
A number of hybrid GPU notebook computer designs with dual (integrated plus discrete) GPUs are equipped with multiplexers (muxes) that allow display panels to be driven by either the integrated GPU or the discrete GPU. Typically, this is a selection that can be made at boot time as a menu option in the system firmware's setup screen, and the mux selection stays fixed for as long as the system is running and persists across reboots until it is explicitly changed. However, some muxed hybrid GPU systems have dynamically switchable muxes which can be switched while the system is running.
As you may be aware, there's drivers/gpu/vga/vga_switcheroo.c (of which I'm listed as a reviewer in MAINTAINERS) to support such hardware.
It also supports muxed configurations, including those that support switching at runtime (and not only at boot) such as the MacBook Pro, which uses drivers/platform/x86/apple-gmux.c to interface between vga_switcheroo and the hardware mux.
However, so far switching only actually works on LVDS-based MacBook Pros, i.e. all pre-retina machines introduced between Late 2008 and Mid 2012, because that hardware is capable of switching the DDC pins separately from the display, so we lock and switch them when probing the EDID.
I have observed that on at least some systems, the EDID for the internal panel can be read via the ACPI _DDC method regardless of whether it's actively muxed in. I don't know whether that's true for all systems where the DDC line can't be switched independently, but maybe vga_switcheroo could also export an interface for GPU drivers to cache EDIDs so that a muxed-away GPU can read an EDID that was previously read by another GPU? I guess the utility of that would depend on how prevalent the combination of no DDC muxing + no ACPI EDID reads turns out to be.
The retina machines introduced from Mid 2012 onward use eDP and run into the issues you're describing: The AUX channel cannot be switched separately from the display, so link training fails unless the entire display is switched. Nevertheless macOS can switch the panel seamlessly. So how are they doing it?
Well, I don't own a retina MacBook Pro, hence never got very far with supporting them, but I did some research and experiments in the 2015/2016 time frame which a colleague, Bruno Bierbaumer, tested on his machine:
First of all, there's DPCD byte 3 bit 6 (NO_AUX_HANDSHAKE_LINK_TRAINING) which is documented as follows:
Does not require AUX CH handshake when the link configuration is already known. [...] The known-good drive current and pre-emphasis level (or those used in the last "full" link training with AUX CH handshake) must be used when the link training is performed without AUX CH handshake.That bit is set on the MacBook Pros in question.
I'll check one of the eDP-based systems I've been experimenting on to see if setting the VGA_SWITCHER_NEEDS_EDP_CONFIG capability in the handler is sufficient to make i915 avoid poking the AUX channel when it's mux-switched away. (This would be in addition to hacking the can_switch() callback in the GPU drivers to allow switching while there are still active KMS clients for the purposes of this experiment, unless somebody can point me to a tree with the WIP per-output switching Daniel Vetter mentioned.
Two things: I thought (but not sure) that for the output switching muxes we'd run vgaswitcheroo in a different mode, where it doesn't check whether whether the driver can be killed. Because it wont. On a quick search only thing I've found is the ddc-only switching done by vga_switcheroo_lock/unlock_ddc. Maybe misremembering, but I thought there was more. But been a while I last looked at this all in detail.
Wrt per-output switching WIP branch. That would be something you'd need to type ofc, I was just laying out what I think would make sense as a possible path to integrate this into upstream. -Daniel
Okay. I misunderstood. When you said that vga-switcheroo could switch individual outputs and do so without powering down the switched-away-from GPU, I took that to mean that this feature had already been implemented somewhere, despite appearances to the contrary upstream. I agree that adding per-output switching support to vga-switcheroo would be a sensible path.
So I looked again, and maybe another clarification. What I think is already there is the use-case of switching one integrated panel. I checked the code now, this is supported with DRM_SWITCH_POWER_DYNAMIC_OFF. Essentially this allows you to always open the gpu cdev node, no matter the vgaswitcheroo state. And the vgaswitcheroo only controls where the outputs are connected (and most designs only switch the integrated panel, external outputs go to the discrete gpu).
Okay, it looks like you're talking about the support for "mux-only" switching (MIGD/MDIS). It seems that this doesn't touch any of the driver callbacks, though, and it would be nice to at least hit reprobe(). Would it make sense to implement another switch type that switches without checking can_switch(), but still hits the driver callbacks? I switched my PoC to do this and it seems to work so far.
The ddc switching is so that the external gpu can still probe the panel when it's connected to the integrated gpu.
What's not there is switching for outputs indivually.
Note that's at least my understanding, I dont have a system supporting this.
So my suggestio would be:
- mayke the nvidia blob work with the current stuff first. I assume
you'll need that :-)
- look into fixing the locking and refcounting bugs. There's FIXME's
all over the code, and we kinda need to address this before we make everything worse.
That gets us a baseline. Next up I think enabling the seamless edp switching would be nice. For that we don't really need any per-output switching yet, the current vgaswitcheroo model still works. But what we probably need:
- some coordination between drm drivers, probably going through
userspace so we follow the sequence correctly
- a new uapi which userspace can use without going into debugfs (which
is a no-go really for shipping to enduser). Probably something that's tied to drm_connectors so we can extend it later on
- the userspace which makes this work seamlessly, if you want this
with X that probably means -modesetting.
Does this sound like a sensible high-level design?
- vga-switcheroo-capable GPU drivers can register muxable outputs.
- Each GPU driver must register each unique muxable output with the same
identifier. The outputs will be registered together with individual devices they can be muxed to, in order to support e.g. muxing between different GPU devices driven by the same vendor. (I'm not aware of any designs that actually support this, but it seems reasonable to design for.) The output identifier may be platform-specific (e.g. ACPI handle, simple index). For example, two GPU drivers may each register an internal panel with identifier "0" and an external port with identifier "1".
- For each output that was registered with more than one device,
vga-switcheroo will expose a sysfs file listing the devices that output can be muxed to, and a sysfs file allowing the selection of a particular device. Perhaps these could be the same file (e.g., on reading it prints a list with an indicator to mark the currently selected device, write a device ID to switch to that device).
With my plan the above isn't needed, at least not at first. Naming and identifying stuff is a really hard problem, so probably best we delay it so we have more time to figure this out.
I think this would allow flexibility across N muxable outputs that can each be switched to one of M GPU devices, without requiring that each of the outputs be switchable to all M possible GPUs.
In addition to the above mechanism for advertising the switchability of individual outputs and providing an interface, the following would also be needed to support PSR:
- A new callback for GPU drivers so vga-switcheroo can tell them an
output is being switched to a GPU.
- vga-switcheroo can call this callback for all devices once before
switching the mux, and then again after the mux switch is complete. A parameter of the callback would indicate whether the mux switch is pending, completed, or failed, allowing GPU drivers to do any necessary pre-switch setup, post-switch cleanup, or failed-switch recovery in response to events from vga-switcheroo.
- Maybe a similar callback for mux handlers so vga-switcheroo can signal
pre- and post-switch events to the handler driver. The PSR designs I'm aware of are GPU-driven, but I can imagine a design that is driven by some component other than a GPU, in which case it would make sense for the mux handler to drive PSR. I suppose this could be left out for now and then added later if such a design emerges.
- It would probably be good to let userspace drive the pre/switch/post
operations independently, e.g. so that userspace can prepare the first frame for presentation on the switched-to GPU before PSR is disabled.
This more or less matches what I'm suggesting too. Except no new callbacks just because we might need them, we're only going to add what the drivers actually need to orchestrate a given use case. The drivers = i915 + nouveau + whatever vgaswitcheroo mux driver you have, all in upstream.
Now, I'm not sure if it makes more sense to also implement the AUX channel proxying that Lukas had prototyped a while back, or to update i915 and any other vga-switcheroo-compatible drivers that assume that eDP is always connected to allow for the possibility of eDP being disconnected due to being muxed away. I personally think disconnecting the eDP output makes more sense, since it more accurately reflects reality than proxying the AUX channel through to the GPU that's muxed in, but if there's another reason the AUX proxying makes more sense, we could plan on making that work (on a per-output basis, since I think e.g. regular DP for external connectors wouldn't need this) as well.
I think if we expect userspace to drive the seamless edp transition, then probably a disconnected state that reflects reality is needed.
otoh, and I think that's the driver behind Lukas' work: Current drivers need the edp state to set everything up at driver load. If they can only fully probe the features of the integrated panel later on it's too late. That's also why we have the ddx mux, so that drivers can load as if they're connected to the panel.
So I think we still need Lukas' edp proxying patches most likely. What the userspace visible connection_status will be like needs to be figured out with how exactly we're going to drive the switchover.
What exactly do the drivers need to probe? Is there more than just the panel's EDID involved? Many systems implement a _DDC method in ACPI which allows retrieving the internal panel's EDID via ACPI, which on muxed systems, is especially useful, as it will work regardless of which driver needs it or what the current mux state is.
Lukas suggested caching "the required portions of the DPCD", so it sounds like it might be more than just the EDID, though. I was looking at plumbing through our DP AUX code to the DRM DP helpers so I could try testing that, but it wasn't as trivial as I had hoped, so I want to be sure that it's really needed before going further down that path. I'm also a bit nervous about giving drivers access to other drivers' DP AUX channels. Limiting it to reading DPCD is possibly okay, but even then, there may be registers with read side effects.
-Daniel
So I think what we should be doing here is that the DRM driver which happens to be muxed to the panel on boot performs link training and informs vga_switcheroo of the drive current, pre-emph level, etc. The other DRM driver is notified when that information is available and uses it to set up its eDP output, skipping an actual AUX CH handshake.
At least i915 probes various capabilities in the DPCD without any consideration that the AUX channel may currently not be available. Back in the day I experimented with a read-only proxy mechanism to make that work, whereby the inactive DRM driver uses the active DRM driver to access the DPCD:
https://patchwork.kernel.org/patch/7000591/
An alternative would be to have the active DRM driver cache the required portions of the DPCD for use by the inactive DRM driver.
Note that vga_switcheroo is currently controlled via debugfs. That is a historic artefact. The kernel has since gained a mux subsystem in drivers/mux/ which could be used to represent the display mux in a standardized way in regular sysfs.
Thanks,
Lukas
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
-- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
-- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
On Wed, Apr 22, 2020 at 05:04:54PM -0500, Daniel Dadap wrote:
On 4/3/20 2:59 PM, Daniel Vetter wrote:
On Fri, Apr 3, 2020 at 8:06 PM Daniel Dadap ddadap@nvidia.com wrote:
On 4/3/20 2:16 AM, Daniel Vetter wrote:
On Fri, Apr 3, 2020 at 8:54 AM Daniel Dadap ddadap@nvidia.com wrote:
On 4/2/20 6:39 AM, Lukas Wunner wrote:
On Fri, Mar 27, 2020 at 04:25:19PM -0500, Daniel Dadap wrote:
> A number of hybrid GPU notebook computer designs with dual (integrated plus > discrete) GPUs are equipped with multiplexers (muxes) that allow display > panels to be driven by either the integrated GPU or the discrete GPU. > Typically, this is a selection that can be made at boot time as a menu > option in the system firmware's setup screen, and the mux selection stays > fixed for as long as the system is running and persists across reboots until > it is explicitly changed. However, some muxed hybrid GPU systems have > dynamically switchable muxes which can be switched while the system is > running. As you may be aware, there's drivers/gpu/vga/vga_switcheroo.c (of which I'm listed as a reviewer in MAINTAINERS) to support such hardware.
It also supports muxed configurations, including those that support switching at runtime (and not only at boot) such as the MacBook Pro, which uses drivers/platform/x86/apple-gmux.c to interface between vga_switcheroo and the hardware mux.
However, so far switching only actually works on LVDS-based MacBook Pros, i.e. all pre-retina machines introduced between Late 2008 and Mid 2012, because that hardware is capable of switching the DDC pins separately from the display, so we lock and switch them when probing the EDID.
I have observed that on at least some systems, the EDID for the internal panel can be read via the ACPI _DDC method regardless of whether it's actively muxed in. I don't know whether that's true for all systems where the DDC line can't be switched independently, but maybe vga_switcheroo could also export an interface for GPU drivers to cache EDIDs so that a muxed-away GPU can read an EDID that was previously read by another GPU? I guess the utility of that would depend on how prevalent the combination of no DDC muxing + no ACPI EDID reads turns out to be.
The retina machines introduced from Mid 2012 onward use eDP and run into the issues you're describing: The AUX channel cannot be switched separately from the display, so link training fails unless the entire display is switched. Nevertheless macOS can switch the panel seamlessly. So how are they doing it?
Well, I don't own a retina MacBook Pro, hence never got very far with supporting them, but I did some research and experiments in the 2015/2016 time frame which a colleague, Bruno Bierbaumer, tested on his machine:
First of all, there's DPCD byte 3 bit 6 (NO_AUX_HANDSHAKE_LINK_TRAINING) which is documented as follows:
Does not require AUX CH handshake when the link configuration is already known. [...] The known-good drive current and pre-emphasis level (or those used in the last "full" link training with AUX CH handshake) must be used when the link training is performed without AUX CH handshake.That bit is set on the MacBook Pros in question.
I'll check one of the eDP-based systems I've been experimenting on to see if setting the VGA_SWITCHER_NEEDS_EDP_CONFIG capability in the handler is sufficient to make i915 avoid poking the AUX channel when it's mux-switched away. (This would be in addition to hacking the can_switch() callback in the GPU drivers to allow switching while there are still active KMS clients for the purposes of this experiment, unless somebody can point me to a tree with the WIP per-output switching Daniel Vetter mentioned.
Two things: I thought (but not sure) that for the output switching muxes we'd run vgaswitcheroo in a different mode, where it doesn't check whether whether the driver can be killed. Because it wont. On a quick search only thing I've found is the ddc-only switching done by vga_switcheroo_lock/unlock_ddc. Maybe misremembering, but I thought there was more. But been a while I last looked at this all in detail.
Wrt per-output switching WIP branch. That would be something you'd need to type ofc, I was just laying out what I think would make sense as a possible path to integrate this into upstream. -Daniel
Okay. I misunderstood. When you said that vga-switcheroo could switch individual outputs and do so without powering down the switched-away-from GPU, I took that to mean that this feature had already been implemented somewhere, despite appearances to the contrary upstream. I agree that adding per-output switching support to vga-switcheroo would be a sensible path.
So I looked again, and maybe another clarification. What I think is already there is the use-case of switching one integrated panel. I checked the code now, this is supported with DRM_SWITCH_POWER_DYNAMIC_OFF. Essentially this allows you to always open the gpu cdev node, no matter the vgaswitcheroo state. And the vgaswitcheroo only controls where the outputs are connected (and most designs only switch the integrated panel, external outputs go to the discrete gpu).
Okay, it looks like you're talking about the support for "mux-only" switching (MIGD/MDIS). It seems that this doesn't touch any of the driver callbacks, though, and it would be nice to at least hit reprobe(). Would it make sense to implement another switch type that switches without checking can_switch(), but still hits the driver callbacks? I switched my PoC to do this and it seems to work so far.
->can_switch has the most questionable of locking, so reworking that sounds like a good idea I'd say. I thought for the dynamic case we already skip checking that ...
The ddc switching is so that the external gpu can still probe the panel when it's connected to the integrated gpu.
What's not there is switching for outputs indivually.
Note that's at least my understanding, I dont have a system supporting this.
So my suggestio would be:
- mayke the nvidia blob work with the current stuff first. I assume
you'll need that :-)
- look into fixing the locking and refcounting bugs. There's FIXME's
all over the code, and we kinda need to address this before we make everything worse.
That gets us a baseline. Next up I think enabling the seamless edp switching would be nice. For that we don't really need any per-output switching yet, the current vgaswitcheroo model still works. But what we probably need:
- some coordination between drm drivers, probably going through
userspace so we follow the sequence correctly
- a new uapi which userspace can use without going into debugfs (which
is a no-go really for shipping to enduser). Probably something that's tied to drm_connectors so we can extend it later on
- the userspace which makes this work seamlessly, if you want this
with X that probably means -modesetting.
Does this sound like a sensible high-level design?
- vga-switcheroo-capable GPU drivers can register muxable outputs.
- Each GPU driver must register each unique muxable output with the same
identifier. The outputs will be registered together with individual devices they can be muxed to, in order to support e.g. muxing between different GPU devices driven by the same vendor. (I'm not aware of any designs that actually support this, but it seems reasonable to design for.) The output identifier may be platform-specific (e.g. ACPI handle, simple index). For example, two GPU drivers may each register an internal panel with identifier "0" and an external port with identifier "1".
- For each output that was registered with more than one device,
vga-switcheroo will expose a sysfs file listing the devices that output can be muxed to, and a sysfs file allowing the selection of a particular device. Perhaps these could be the same file (e.g., on reading it prints a list with an indicator to mark the currently selected device, write a device ID to switch to that device).
With my plan the above isn't needed, at least not at first. Naming and identifying stuff is a really hard problem, so probably best we delay it so we have more time to figure this out.
I think this would allow flexibility across N muxable outputs that can each be switched to one of M GPU devices, without requiring that each of the outputs be switchable to all M possible GPUs.
In addition to the above mechanism for advertising the switchability of individual outputs and providing an interface, the following would also be needed to support PSR:
- A new callback for GPU drivers so vga-switcheroo can tell them an
output is being switched to a GPU.
- vga-switcheroo can call this callback for all devices once before
switching the mux, and then again after the mux switch is complete. A parameter of the callback would indicate whether the mux switch is pending, completed, or failed, allowing GPU drivers to do any necessary pre-switch setup, post-switch cleanup, or failed-switch recovery in response to events from vga-switcheroo.
- Maybe a similar callback for mux handlers so vga-switcheroo can signal
pre- and post-switch events to the handler driver. The PSR designs I'm aware of are GPU-driven, but I can imagine a design that is driven by some component other than a GPU, in which case it would make sense for the mux handler to drive PSR. I suppose this could be left out for now and then added later if such a design emerges.
- It would probably be good to let userspace drive the pre/switch/post
operations independently, e.g. so that userspace can prepare the first frame for presentation on the switched-to GPU before PSR is disabled.
This more or less matches what I'm suggesting too. Except no new callbacks just because we might need them, we're only going to add what the drivers actually need to orchestrate a given use case. The drivers = i915 + nouveau + whatever vgaswitcheroo mux driver you have, all in upstream.
Now, I'm not sure if it makes more sense to also implement the AUX channel proxying that Lukas had prototyped a while back, or to update i915 and any other vga-switcheroo-compatible drivers that assume that eDP is always connected to allow for the possibility of eDP being disconnected due to being muxed away. I personally think disconnecting the eDP output makes more sense, since it more accurately reflects reality than proxying the AUX channel through to the GPU that's muxed in, but if there's another reason the AUX proxying makes more sense, we could plan on making that work (on a per-output basis, since I think e.g. regular DP for external connectors wouldn't need this) as well.
I think if we expect userspace to drive the seamless edp transition, then probably a disconnected state that reflects reality is needed.
otoh, and I think that's the driver behind Lukas' work: Current drivers need the edp state to set everything up at driver load. If they can only fully probe the features of the integrated panel later on it's too late. That's also why we have the ddx mux, so that drivers can load as if they're connected to the panel.
So I think we still need Lukas' edp proxying patches most likely. What the userspace visible connection_status will be like needs to be figured out with how exactly we're going to drive the switchover.
What exactly do the drivers need to probe? Is there more than just the panel's EDID involved? Many systems implement a _DDC method in ACPI which allows retrieving the internal panel's EDID via ACPI, which on muxed systems, is especially useful, as it will work regardless of which driver needs it or what the current mux state is.
Edid, but also dp aux register values (since almost everything is edp nowadays, at least on laptops). Especially for edp panels there's a lot of dpcd for all the fancy features.
Lukas suggested caching "the required portions of the DPCD", so it sounds like it might be more than just the EDID, though. I was looking at plumbing through our DP AUX code to the DRM DP helpers so I could try testing that, but it wasn't as trivial as I had hoped, so I want to be sure that it's really needed before going further down that path. I'm also a bit nervous about giving drivers access to other drivers' DP AUX channels. Limiting it to reading DPCD is possibly okay, but even then, there may be registers with read side effects.
Yeah I think some dpcd cache could be all we need.
dp aux itself is exposed through debugfs, so handing out access like that to other parties should be fine. dp aux is also exposed through the i2c layer to userspace (and there's actually tools using it, for stuff like ddc), so if there's bugs in the locking for concurrent access, they're already a problem. Only thing to avoid here is a midlayer taste, but I think that should be doable. Unfortunately the current vga_switcheroo is quite a bit a midlayer, and e.g. locking rules around ->can_switch make that design mistake quite clear. -Daniel
-Daniel
So I think what we should be doing here is that the DRM driver which happens to be muxed to the panel on boot performs link training and informs vga_switcheroo of the drive current, pre-emph level, etc. The other DRM driver is notified when that information is available and uses it to set up its eDP output, skipping an actual AUX CH handshake.
At least i915 probes various capabilities in the DPCD without any consideration that the AUX channel may currently not be available. Back in the day I experimented with a read-only proxy mechanism to make that work, whereby the inactive DRM driver uses the active DRM driver to access the DPCD:
https://patchwork.kernel.org/patch/7000591/
An alternative would be to have the active DRM driver cache the required portions of the DPCD for use by the inactive DRM driver.
Note that vga_switcheroo is currently controlled via debugfs. That is a historic artefact. The kernel has since gained a mux subsystem in drivers/mux/ which could be used to represent the display mux in a standardized way in regular sysfs.
Thanks,
Lukas
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
-- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
-- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
Sorry for letting this stagnate. I have some patches ready for discussion which I'll send shortly, but wanted to follow up on some of the previous conversation:
On 4/28/20 5:14 AM, Daniel Vetter wrote:
On Wed, Apr 22, 2020 at 05:04:54PM -0500, Daniel Dadap wrote:
On 4/3/20 2:59 PM, Daniel Vetter wrote:
On Fri, Apr 3, 2020 at 8:06 PM Daniel Dadap ddadap@nvidia.com wrote:
On 4/3/20 2:16 AM, Daniel Vetter wrote:
On Fri, Apr 3, 2020 at 8:54 AM Daniel Dadap ddadap@nvidia.com wrote:
On 4/2/20 6:39 AM, Lukas Wunner wrote:
On Fri, Mar 27, 2020 at 04:25:19PM -0500, Daniel Dadap wrote: >> A number of hybrid GPU notebook computer designs with dual (integrated plus >> discrete) GPUs are equipped with multiplexers (muxes) that allow display >> panels to be driven by either the integrated GPU or the discrete GPU. >> Typically, this is a selection that can be made at boot time as a menu >> option in the system firmware's setup screen, and the mux selection stays >> fixed for as long as the system is running and persists across reboots until >> it is explicitly changed. However, some muxed hybrid GPU systems have >> dynamically switchable muxes which can be switched while the system is >> running. > As you may be aware, there's drivers/gpu/vga/vga_switcheroo.c (of which > I'm listed as a reviewer in MAINTAINERS) to support such hardware. > > It also supports muxed configurations, including those that support > switching at runtime (and not only at boot) such as the MacBook Pro, > which uses drivers/platform/x86/apple-gmux.c to interface between > vga_switcheroo and the hardware mux. > > However, so far switching only actually works on LVDS-based MacBook Pros, > i.e. all pre-retina machines introduced between Late 2008 and Mid 2012, > because that hardware is capable of switching the DDC pins separately > from the display, so we lock and switch them when probing the EDID. I have observed that on at least some systems, the EDID for the internal panel can be read via the ACPI _DDC method regardless of whether it's actively muxed in. I don't know whether that's true for all systems where the DDC line can't be switched independently, but maybe vga_switcheroo could also export an interface for GPU drivers to cache EDIDs so that a muxed-away GPU can read an EDID that was previously read by another GPU? I guess the utility of that would depend on how prevalent the combination of no DDC muxing + no ACPI EDID reads turns out to be.
> The retina machines introduced from Mid 2012 onward use eDP and run > into the issues you're describing: The AUX channel cannot be switched > separately from the display, so link training fails unless the entire > display is switched. Nevertheless macOS can switch the panel seamlessly. > So how are they doing it? > > Well, I don't own a retina MacBook Pro, hence never got very far with > supporting them, but I did some research and experiments in the 2015/2016 > time frame which a colleague, Bruno Bierbaumer, tested on his machine: > > First of all, there's DPCD byte 3 bit 6 (NO_AUX_HANDSHAKE_LINK_TRAINING) > which is documented as follows: > > Does not require AUX CH handshake when the link configuration is > already known. [...] The known-good drive current and pre-emphasis > level (or those used in the last "full" link training with AUX CH > handshake) must be used when the link training is performed without > AUX CH handshake. > > That bit is set on the MacBook Pros in question. I'll check one of the eDP-based systems I've been experimenting on to see if setting the VGA_SWITCHER_NEEDS_EDP_CONFIG capability in the handler is sufficient to make i915 avoid poking the AUX channel when it's mux-switched away. (This would be in addition to hacking the can_switch() callback in the GPU drivers to allow switching while there are still active KMS clients for the purposes of this experiment, unless somebody can point me to a tree with the WIP per-output switching Daniel Vetter mentioned.
Two things: I thought (but not sure) that for the output switching muxes we'd run vgaswitcheroo in a different mode, where it doesn't check whether whether the driver can be killed. Because it wont. On a quick search only thing I've found is the ddc-only switching done by vga_switcheroo_lock/unlock_ddc. Maybe misremembering, but I thought there was more. But been a while I last looked at this all in detail.
Wrt per-output switching WIP branch. That would be something you'd need to type ofc, I was just laying out what I think would make sense as a possible path to integrate this into upstream. -Daniel
Okay. I misunderstood. When you said that vga-switcheroo could switch individual outputs and do so without powering down the switched-away-from GPU, I took that to mean that this feature had already been implemented somewhere, despite appearances to the contrary upstream. I agree that adding per-output switching support to vga-switcheroo would be a sensible path.
So I looked again, and maybe another clarification. What I think is already there is the use-case of switching one integrated panel. I checked the code now, this is supported with DRM_SWITCH_POWER_DYNAMIC_OFF. Essentially this allows you to always open the gpu cdev node, no matter the vgaswitcheroo state. And the vgaswitcheroo only controls where the outputs are connected (and most designs only switch the integrated panel, external outputs go to the discrete gpu).
Okay, it looks like you're talking about the support for "mux-only" switching (MIGD/MDIS). It seems that this doesn't touch any of the driver callbacks, though, and it would be nice to at least hit reprobe(). Would it make sense to implement another switch type that switches without checking can_switch(), but still hits the driver callbacks? I switched my PoC to do this and it seems to work so far.
->can_switch has the most questionable of locking, so reworking that sounds like a good idea I'd say. I thought for the dynamic case we already skip checking that ...
So for "mux only" switches can_switch is indeed skipped, but so is reprobe.
The ddc switching is so that the external gpu can still probe the panel when it's connected to the integrated gpu.
What's not there is switching for outputs indivually.
Note that's at least my understanding, I dont have a system supporting this.
So my suggestio would be:
- mayke the nvidia blob work with the current stuff first. I assume
you'll need that :-)
- look into fixing the locking and refcounting bugs. There's FIXME's
all over the code, and we kinda need to address this before we make everything worse.
That gets us a baseline. Next up I think enabling the seamless edp switching would be nice. For that we don't really need any per-output switching yet, the current vgaswitcheroo model still works. But what we probably need:
- some coordination between drm drivers, probably going through
userspace so we follow the sequence correctly
- a new uapi which userspace can use without going into debugfs (which
is a no-go really for shipping to enduser). Probably something that's tied to drm_connectors so we can extend it later on
- the userspace which makes this work seamlessly, if you want this
with X that probably means -modesetting.
Does this sound like a sensible high-level design?
- vga-switcheroo-capable GPU drivers can register muxable outputs.
- Each GPU driver must register each unique muxable output with the same
identifier. The outputs will be registered together with individual devices they can be muxed to, in order to support e.g. muxing between different GPU devices driven by the same vendor. (I'm not aware of any designs that actually support this, but it seems reasonable to design for.) The output identifier may be platform-specific (e.g. ACPI handle, simple index). For example, two GPU drivers may each register an internal panel with identifier "0" and an external port with identifier "1".
- For each output that was registered with more than one device,
vga-switcheroo will expose a sysfs file listing the devices that output can be muxed to, and a sysfs file allowing the selection of a particular device. Perhaps these could be the same file (e.g., on reading it prints a list with an indicator to mark the currently selected device, write a device ID to switch to that device).
With my plan the above isn't needed, at least not at first. Naming and identifying stuff is a really hard problem, so probably best we delay it so we have more time to figure this out.
I think this would allow flexibility across N muxable outputs that can each be switched to one of M GPU devices, without requiring that each of the outputs be switchable to all M possible GPUs.
In addition to the above mechanism for advertising the switchability of individual outputs and providing an interface, the following would also be needed to support PSR:
- A new callback for GPU drivers so vga-switcheroo can tell them an
output is being switched to a GPU.
- vga-switcheroo can call this callback for all devices once before
switching the mux, and then again after the mux switch is complete. A parameter of the callback would indicate whether the mux switch is pending, completed, or failed, allowing GPU drivers to do any necessary pre-switch setup, post-switch cleanup, or failed-switch recovery in response to events from vga-switcheroo.
- Maybe a similar callback for mux handlers so vga-switcheroo can signal
pre- and post-switch events to the handler driver. The PSR designs I'm aware of are GPU-driven, but I can imagine a design that is driven by some component other than a GPU, in which case it would make sense for the mux handler to drive PSR. I suppose this could be left out for now and then added later if such a design emerges.
- It would probably be good to let userspace drive the pre/switch/post
operations independently, e.g. so that userspace can prepare the first frame for presentation on the switched-to GPU before PSR is disabled.
This more or less matches what I'm suggesting too. Except no new callbacks just because we might need them, we're only going to add what the drivers actually need to orchestrate a given use case. The drivers = i915 + nouveau + whatever vgaswitcheroo mux driver you have, all in upstream.
Now, I'm not sure if it makes more sense to also implement the AUX channel proxying that Lukas had prototyped a while back, or to update i915 and any other vga-switcheroo-compatible drivers that assume that eDP is always connected to allow for the possibility of eDP being disconnected due to being muxed away. I personally think disconnecting the eDP output makes more sense, since it more accurately reflects reality than proxying the AUX channel through to the GPU that's muxed in, but if there's another reason the AUX proxying makes more sense, we could plan on making that work (on a per-output basis, since I think e.g. regular DP for external connectors wouldn't need this) as well.
I think if we expect userspace to drive the seamless edp transition, then probably a disconnected state that reflects reality is needed.
otoh, and I think that's the driver behind Lukas' work: Current drivers need the edp state to set everything up at driver load. If they can only fully probe the features of the integrated panel later on it's too late. That's also why we have the ddx mux, so that drivers can load as if they're connected to the panel.
So I think we still need Lukas' edp proxying patches most likely. What the userspace visible connection_status will be like needs to be figured out with how exactly we're going to drive the switchover.
What exactly do the drivers need to probe? Is there more than just the panel's EDID involved? Many systems implement a _DDC method in ACPI which allows retrieving the internal panel's EDID via ACPI, which on muxed systems, is especially useful, as it will work regardless of which driver needs it or what the current mux state is.
Edid, but also dp aux register values (since almost everything is edp nowadays, at least on laptops). Especially for edp panels there's a lot of dpcd for all the fancy features.
Lukas suggested caching "the required portions of the DPCD", so it sounds like it might be more than just the EDID, though. I was looking at plumbing through our DP AUX code to the DRM DP helpers so I could try testing that, but it wasn't as trivial as I had hoped, so I want to be sure that it's really needed before going further down that path. I'm also a bit nervous about giving drivers access to other drivers' DP AUX channels. Limiting it to reading DPCD is possibly okay, but even then, there may be registers with read side effects.
Yeah I think some dpcd cache could be all we need.
I experimentally implemented a DPCD cache but found that it didn't seem to add any value, but perhaps I just wasn't hitting any code paths where DPCD access was required. I've omitted it from the current patch series because I didn't test it particularly heavily. With my current patches, I can get the mux to hand off successfully between the NVIDIA and modesetting X.org drivers, but not yet between two different DRM-KMS drivers both using the modesetting X driver. It seems the framebuffer doesn't get hande off, and I assume something will need to be done beyond the existing console remap code to make that happen. I'm not familiar enough with the DRM-KMS stack to know exactly what will be needed to get that working. I did add a helper to allow DRM drivers to access EDIDs from ACPI, which is useful for not only muxed designs, but a small number of non-hybrid and muxless hybrid systems also use an ACPI _DDC method to retrieve the EDID because the panel doesn't provide one via the usual ways.
dp aux itself is exposed through debugfs, so handing out access like that to other parties should be fine. dp aux is also exposed through the i2c layer to userspace (and there's actually tools using it, for stuff like ddc), so if there's bugs in the locking for concurrent access, they're already a problem. Only thing to avoid here is a midlayer taste, but I think that should be doable. Unfortunately the current vga_switcheroo is quite a bit a midlayer, and e.g. locking rules around ->can_switch make that design mistake quite clear. -Daniel
-Daniel
> So I think what we should be doing here is that the DRM driver which > happens to be muxed to the panel on boot performs link training and > informs vga_switcheroo of the drive current, pre-emph level, etc. > The other DRM driver is notified when that information is available > and uses it to set up its eDP output, skipping an actual AUX CH > handshake. > > At least i915 probes various capabilities in the DPCD without any > consideration that the AUX channel may currently not be available. > Back in the day I experimented with a read-only proxy mechanism > to make that work, whereby the inactive DRM driver uses the active > DRM driver to access the DPCD: > > https://patchwork.kernel.org/patch/7000591/ > > An alternative would be to have the active DRM driver cache the > required portions of the DPCD for use by the inactive DRM driver. > > Note that vga_switcheroo is currently controlled via debugfs. > That is a historic artefact. The kernel has since gained a > mux subsystem in drivers/mux/ which could be used to represent > the display mux in a standardized way in regular sysfs. > > Thanks, > > Lukas _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
-- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
-- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
-- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
dri-devel@lists.freedesktop.org
-
Daniel Dadap -
Daniel Stone -
Daniel Vetter -
Jani Nikula -
Lukas Wunner -
Pekka Paalanen