Hi Bastien,
On Fri, Mar 04, 2016 at 04:12:27PM +0000, Bastien Nocera wrote:
Lukas Wunner <lukas <at> wunner.de> writes:
Enable GPU switching on the pre-retina MacBook Pro (2008 - 2013), v5.
I've tested your patchset on a MacBookPro8,1, with an integrated Intel and discrete AMD/ATI GPUs.
Hm, it must be either an 8,2 or 8,3. The 8,1 was a 13" machine and only had an integrated GPU.
I've used the COPR repository here to cut down on my compilation time: https://copr.fedorainfracloud.org/coprs/firstyear/kernel-mbp/
I'm not certain how to test out your changes, or what the consequences should be on a stock Fedora 23/GNOME 3.18 installation. After booting (note that I did not change any command-line options in grub), a gnome-shell/gdm X11 session comes up (I disabled Wayland, to rule out behavioural changes), I'd log in to GNOME and gnome-shell (which starts another X11 session on another VT).
Switching and power control currently requires manual intervention by echoing commands to /sys/kernel/debug/vgaswitcheroo/switch as documented here: https://01.org/linuxgraphics/gfx-docs/drm/modes_of_use.html
As you've correctly observed, the machine is initially switched to the discrete GPU and both GPUs are turned on. By echoing "IGD" to the sysfs file, you'll switch to the integrated GPU and turn off the discrete GPU.
It's possible to let the EFI firmware switch to the integrated GPU on boot by using this tool: https://github.com/0xbb/gpu-switch However still both GPUs will be powered up, so you have to issue the "OFF" command to sysfs to power the discrete GPU down. Also, once you boot into OS X, the setting made by the gpu-switch tool will be overwritten and the machine will be switched to the discrete GPU again the next time you boot Linux.
Note that switching is only possible from the text console, with X11/Wayland shut down. Obviously this is not great in terms of UX. A few years ago there was a GSoC proposal to get hot GPU switching to work on Linux (akin to what OS X does) but nothing ever came of it: http://www.phoronix.com/scan.php?page=news_item&px=OTIyMQ https://lists.x.org/archives/xorg/2011-March/052522.html
Unfortunately this seems to be a low priority item for kernel graphics developers since nowadays most dual GPU notebooks no longer have a mux and cannot switch. The MacBook Pro seems to be the last one supporting this but I've witnessed a bit of an anti-Apple sentiment among kernel graphics developers since everything is non-standard there. Which is unfortunate because these machines have a large market share and Apple software quality is deteriorating rapidly so a lot of Mac users are ripe for converting to Linux.
Anyway, one short-term improvement will be to add runtime pm support (called "Driver power control" in the vga_switcheroo documentation linked above). That way it'll no longer be necessary to power the discrete GPU up and down manually, this will happen automatically as needed (when switching or using render offloading with DRI PRIME). I have patches to enable this for radeon but they're completely untested:
http://wunner.de/mbp_switcheroo_v5-4.5.tar.gz => gpu switching for 4.5 http://wunner.de/mbp_switcheroo_v5-4.5-runpm.tar.gz => runtime pm for radeon
I have an Nvidia based machine and runtime pm doesn't work there yet because of bugs in nouveau that I haven't had the time to look into.
I did not use any external screens to test this.
Since your machine has Thunderbolt, the external port is no longer switchable between GPUs, it can only be driven by the discrete GPU. So you need to power it up manually for this to work. You don't need to switch to it, but it's probably recommendable to save energy. (Otherwise both GPUs are on with the integrated GPU driving the panel and the discrete GPU driving the DP port.)
The runpm tarball linked above contains a patch to automatically wake the discrete GPU on hotplug.
I've heard that the AMD GPU is picky about external monitors and doesn't recognize them unless they're plugged in at exactly the right moment, so you may need to retry a couple of times until it works.
Best regards,
Lukas
On Sat, 2016-03-05 at 15:16 +0100, Lukas Wunner wrote:
Hi Bastien,
On Fri, Mar 04, 2016 at 04:12:27PM +0000, Bastien Nocera wrote:
Lukas Wunner <lukas <at> wunner.de> writes:
Enable GPU switching on the pre-retina MacBook Pro (2008 - 2013), v5.
I've tested your patchset on a MacBookPro8,1, with an integrated Intel and discrete AMD/ATI GPUs.
Hm, it must be either an 8,2 or 8,3. The 8,1 was a 13" machine and only had an integrated GPU.
Sorry, it's an "8,2". That's what I get for having not having my mail on that machine.
I've used the COPR repository here to cut down on my compilation time: https://copr.fedorainfracloud.org/coprs/firstyear/kernel-mbp/
I'm not certain how to test out your changes, or what the consequences should be on a stock Fedora 23/GNOME 3.18 installation. After booting (note that I did not change any command-line options in grub), a gnome-shell/gdm X11 session comes up (I disabled Wayland, to rule out behavioural changes), I'd log in to GNOME and gnome-shell (which starts another X11 session on another VT).
Switching and power control currently requires manual intervention by echoing commands to /sys/kernel/debug/vgaswitcheroo/switch as documented here: https://01.org/linuxgraphics/gfx-docs/drm/modes_of_use.html
Right, though I would intend on automating this.
As you've correctly observed, the machine is initially switched to the discrete GPU and both GPUs are turned on. By echoing "IGD" to the sysfs file, you'll switch to the integrated GPU and turn off the discrete GPU.
It's possible to let the EFI firmware switch to the integrated GPU on boot by using this tool: https://github.com/0xbb/gpu-switch However still both GPUs will be powered up, so you have to issue the "OFF" command to sysfs to power the discrete GPU down. Also, once you boot into OS X, the setting made by the gpu-switch tool will be overwritten and the machine will be switched to the discrete GPU again the next time you boot Linux.
We could, on boot, force using the integrated GPU, turning off the discrete GPU that we're not interested in. So I'd push DIGD to the switch sysfs entry on boot. But I'm guessing that won't turn off the other output we're not interested in.
Note that switching is only possible from the text console, with X11/Wayland shut down. Obviously this is not great in terms of UX. A few years ago there was a GSoC proposal to get hot GPU switching to work on Linux (akin to what OS X does) but nothing ever came of it: http://www.phoronix.com/scan.php?page=news_item&px=OTIyMQ https://lists.x.org/archives/xorg/2011-March/052522.html
Unfortunately this seems to be a low priority item for kernel graphics developers since nowadays most dual GPU notebooks no longer have a mux and cannot switch. The MacBook Pro seems to be the last one supporting this but I've witnessed a bit of an anti-Apple sentiment among kernel graphics developers since everything is non-standard there. Which is unfortunate because these machines have a large market share and Apple software quality is deteriorating rapidly so a lot of Mac users are ripe for converting to Linux.
DIGD and DDIS should help (you need to log out/log in again), and if the current GPU is the integrated one, you could force running, say, a game on the discrete GPU using DRI_PRIME=1, correct?
FWIW, using OFF at runtime made my machine hang, and without any ways for me to get debug output.
Anyway, one short-term improvement will be to add runtime pm support (called "Driver power control" in the vga_switcheroo documentation linked above). That way it'll no longer be necessary to power the discrete GPU up and down manually, this will happen automatically as needed (when switching or using render offloading with DRI PRIME).
Ok, so using GIGD or DDIS would just switch the output, but not turn off the unused GPU without runtime PM management.
I have patches to enable this for radeon but they're completely untested:
http://wunner.de/mbp_switcheroo_v5-4.5.tar.gz%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%... gpu switching for 4.5
That's the same patch that's already applied to the kernel above (the ones from this patchset thread), right?
http://wunner.de/mbp_switcheroo_v5-4.5-runpm.tar.gz => runtime pm for radeon
OK, will test those out.
I have an Nvidia based machine and runtime pm doesn't work there yet because of bugs in nouveau that I haven't had the time to look into.
I did not use any external screens to test this.
Since your machine has Thunderbolt, the external port is no longer switchable between GPUs, it can only be driven by the discrete GPU. So you need to power it up manually for this to work. You don't need to switch to it, but it's probably recommendable to save energy. (Otherwise both GPUs are on with the integrated GPU driving the panel and the discrete GPU driving the DP port.)
The runpm tarball linked above contains a patch to automatically wake the discrete GPU on hotplug.
I've heard that the AMD GPU is picky about external monitors and doesn't recognize them unless they're plugged in at exactly the right moment, so you may need to retry a couple of times until it works.
To sum up, and if we take the above patches into consideration: - on boot, one or more GPUs are powered on, an init script would queue a GPU switch to the integrated one. The other GPU would be switched off (automatically?) - when X/wayland is running, queue the requests using DIGD or DDIS. If the integrated GPU is used, allow offloading to the discrete GPU with DRI_PRIME (which will again power up automatically thanks to the runtime PM patches above).
My concerns here would be: - Do we know how to turn off the integrated GPU automatically, if the user only used the internal screen and wanted to use the discrete GPU? - If only the discrete GPU is powered on, do we know how to power on the integrated GPU so it can drive the external screen when plugged in? - While plymouth is short-lived at boot, Wayland and X11 GNOME sessions use the GPU. The first user session will run on a VT that's separate from the greeter to implement fast-user switching. So, if we wanted to force using the other GPU for future sessions, we'd need to tell gdm to kill its graphical session and to respawn it so it doesn't hog the GPU and avoid the switch happening. Correct?
FWIW, this is what I had written down a couple of months ago, about supporting dual-GPU computers with GNOME: https://wiki.gnome.org/Design/OS/DualGPU
Those use-cases are a lot simpler than what could be achieved with the vga_switcheroo sub-system, but they probably cover the "90%" use cases we're interested in.
Once I've managed to get the MacBook Pro into a good state, I also have a Lenovo machine around with dual GPU, though I couldn't tell you what the discrete one is.
Cheers
To sum up, and if we take the above patches into consideration:
- on boot, one or more GPUs are powered on, an init script would queue
a GPU switch to the integrated one. The other GPU would be switched off (automatically?)
- when X/wayland is running, queue the requests using DIGD or DDIS. If
the integrated GPU is used, allow offloading to the discrete GPU with DRI_PRIME (which will again power up automatically thanks to the runtime PM patches above).
My concerns here would be:
- Do we know how to turn off the integrated GPU automatically, if the
user only used the internal screen and wanted to use the discrete GPU?
- If only the discrete GPU is powered on, do we know how to power on
the integrated GPU so it can drive the external screen when plugged in?
- While plymouth is short-lived at boot, Wayland and X11 GNOME sessions
use the GPU. The first user session will run on a VT that's separate from the greeter to implement fast-user switching. So, if we wanted to force using the other GPU for future sessions, we'd need to tell gdm to kill its graphical session and to respawn it so it doesn't hog the GPU and avoid the switch happening. Correct?
FWIW, this is what I had written down a couple of months ago, about supporting dual-GPU computers with GNOME: https://wiki.gnome.org/Design/OS/DualGPU
Those use-cases are a lot simpler than what could be achieved with the vga_switcheroo sub-system, but they probably cover the "90%" use cases we're interested in.
Once I've managed to get the MacBook Pro into a good state, I also have a Lenovo machine around with dual GPU, though I couldn't tell you what the discrete one is.
Okay so I'm not sure you are heading in the best direction here.
My first suggestion is to stop using the MBP, start using the Lenovo. At least from a Fedora perspective, that is the hw we have more installs of and care more about.
Apple HW is not the same as PC hw in this case and we aren't going to achieve the same level of integration that OSX has, not without some serious rewrites of mutter and the whole X stack.
You shouldn't be caring about the MUX. MUXed hw apart from the MBP is pretty much gone since Windows 7 timeframes. So I don't think we should be putting too much effort into the MUX yet. With the current way we keep gdm running, we can't do MUX switch on logout anymore. It was only ever a hack. So I'd just not send commands to vga switcheroo at all.
So I'm missing what the overall goal here is. To provide better support for dual-gpu laptops and hotpluggable USB devices in the DE?
Under X, Fedora carries a server patch to autoconfigure providers, we'd need to drop that and have something in the DE notice when a new provider shows up and configures it, perhaps something to allow removal of providers that are already bound (so we could detach a secondary GPU for boxes to passthrough).
Then we need something in the DE to allow us to launch or have some app info that would decide to launch certain 3D using apps on the more powerful processor. However since nouveau doesn't quite reclock most of the secondary GPUs that can often end up not being that much more powerful.
We also want reverse prime to work properly, so if you plug in an external monitor to a port connected to the secondary GPU that we can pick it up and configure it just like all the other monitors.
As for the MBP, if we want to spend time chasing the rainbow of OS X, then we've a lot of work to do. OSX can smoothly switch the compositor from rendering on the intel gpu to the nvidia gpu in a vblank. It's truly seamless. To do that we'd need to a) move to wayland, b) get mutter to be a lot smarter than mutter currently is.
Dave.
Hey Dave,
On Thu, 2016-03-10 at 09:30 +1000, Dave Airlie wrote: <snip>
Okay so I'm not sure you are heading in the best direction here.
My first suggestion is to stop using the MBP, start using the Lenovo. At least from a Fedora perspective, that is the hw we have more installs of and care more about.
The Lenovo has an NVidia GPU, and there's no runtime PM support for nouveau.
Apple HW is not the same as PC hw in this case and we aren't going to achieve the same level of integration that OSX has, not without some serious rewrites of mutter and the whole X stack.
That's not the target goals. Did you read the wiki page I pointed to listing the goals?
https://wiki.gnome.org/Design/OS/DualGPU
You shouldn't be caring about the MUX.
I never talked about the MUX, didn't plan on using it either.
<snip>
So I'm missing what the overall goal here is. To provide better support for dual-gpu laptops and hotpluggable USB devices in the DE?
Just dual-GPU devices for now.
I'd be interested in supporting USB displays, but I only have proprietary drivers for my USB3 DisplayLink dock, and possibly networked display devices, but the AirTame I have is also still using an undocumented protocol.
Under X, Fedora carries a server patch to autoconfigure providers, we'd need to drop that and have something in the DE notice when a new provider shows up and configures it, perhaps something to allow removal of providers that are already bound (so we could detach a secondary GPU for boxes to passthrough).
I'd rather have that be automated so that Boxes can tell you what is using the 2nd GPU, not requiring any manual intervention.
Then we need something in the DE to allow us to launch or have some app info that would decide to launch certain 3D using apps on the more powerful processor.
That's what I started working on, exporting the fact that 2 GPUs are available through a D-Bus service, which also ensures that we only
However since nouveau doesn't quite reclock most of the secondary GPUs that can often end up not being that much more powerful.
There are supported laptops with Radeon GPUs as well, not sure whether that's more powerful.
We also want reverse prime to work properly, so if you plug in an external monitor to a port connected to the secondary GPU that we can pick it up and configure it just like all the other monitors.
I don't think I have any hardware that works this way.
As for the MBP, if we want to spend time chasing the rainbow of OS X, then we've a lot of work to do. OSX can smoothly switch the compositor from rendering on the intel gpu to the nvidia gpu in a vblank. It's truly seamless. To do that we'd need to a) move to wayland, b) get mutter to be a lot smarter than mutter currently is.
That's not what I'm aiming for right now.
Cheers
On Thu, 2016-03-10 at 16:29 +0100, Bastien Nocera wrote:
Then we need something in the DE to allow us to launch or have some app info that would decide to launch certain 3D using apps on the more powerful processor.
That's what I started working on, exporting the fact that 2 GPUs are available through a D-Bus service, which also ensures that we only
And I missed a bit of text there:
which also ensures that we only enable and use the internal GPU by default: https://github.com/hadess/switcheroo-control/blob/master/src/switcheroo-cont...
The full D-Bus service is at: https://github.com/hadess/switcheroo-control
I guess I should start with disabling the Fedora specific X patch to autoconfigure outputs and start adding code to mutter to set those up.
Hi Bastien,
sorry for the delay.
On Sat, Mar 05, 2016 at 05:31:55PM +0100, Bastien Nocera wrote:
We could, on boot, force using the integrated GPU, turning off the discrete GPU that we're not interested in.
Yes, many people "solve" this by having grub write the requisite commands to gmux' I/O ports, however this approach won't work with gummiboot. Also, the commands that need to be sent to gmux differ between retinas and pre-retinas.
So I'd push DIGD to the switch sysfs entry on boot. But I'm guessing that won't turn off the other output we're not interested in.
IGD and DIGD switch to the integrated GPU and also turn off the discrete GPU. However if the machine is already switched to the integrated GPU, the commands are no-ops and the discrete GPU is not turned off.
In other words you need to check (by reading the switch file) which GPU the machine is switched to. If it's the integrated GPU, write OFF to the switch file. If it's the discrete GPU, write DIGD to the switch file.
Once runtime pm works, writing DIGD is sufficient.
DIGD and DDIS should help (you need to log out/log in again), and if the current GPU is the integrated one, you could force running, say, a game on the discrete GPU using DRI_PRIME=1, correct?
If runtime pm works, then yes. Otherwise you'd have to manually power up the GPU before using DRI_PRIME=1 and power it down afterwards.
FWIW, using OFF at runtime made my machine hang, and without any ways for me to get debug output.
Which GPU were you switched to? Did you issue the command while X11 was running? Since the machine is switched to the AMD on boot, I guess you were powering down the Intel GPU. This works on my machine, I get a log entry "i915: switched off". If this doesn't work on your machine it might be an i915 bug on your Sandy Bridge GPU or it might be because X11 is running.
Try booting with "drm.debug=0xf log_buf_len=10M" and see if this gets you log output. If it doesn't, netconsole might help if the hang occurs while the console is locked.
Ok, so using GIGD or DDIS would just switch the output, but not turn off the unused GPU without runtime PM management.
No. They do switch off the other GPU if runtime pm is disabled.
http://wunner.de/mbp_switcheroo_v5-4.5.tar.gz%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%... gpu switching for 4.5
That's the same patch that's already applied to the kernel above (the ones from this patchset thread), right?
I'm not sure, the patches in the copr repository might be an earlier test version.
My concerns here would be:
- Do we know how to turn off the integrated GPU automatically, if the
user only used the internal screen and wanted to use the discrete GPU?
Runtime pm is currently disabled by default for i915. The Intel folks are on it. Until that works, the integrated GPU will be powered down when the user manually switches to the discrete GPU with DIS / DDIS and powered up when switching back with IGD / DIGD.
- If only the discrete GPU is powered on, do we know how to power on
the integrated GPU so it can drive the external screen when plugged in?
On the MBP5 2008/09 and MBP6 2010, the external DP port can be switched between GPUs and we switch it together with the panel. So if you switch to the discrete GPU, you can also drive the external DP port on these models and the integrated GPU can be turned off.
On the MBP8 2011 and newer, external ports are combined DP/Thunderbolt ports which can only be driven by the discrete GPU. For some reason the HPD/AUX pins of the ports can still be switched but not the other pins. We should nail these ports to the discrete GPU and not switch those pins together with the panel as we currently do. There's a patch in mbp_switcheroo_v5-4.5-runpm.tar.gz which fixes that. The patch also wakes up the discrete GPU on hotplug, which obviously needs runtime pm.
- While plymouth is short-lived at boot, Wayland and X11 GNOME sessions
use the GPU. The first user session will run on a VT that's separate from the greeter to implement fast-user switching. So, if we wanted to force using the other GPU for future sessions, we'd need to tell gdm to kill its graphical session and to respawn it so it doesn't hog the GPU and avoid the switch happening. Correct?
I think so, yes.
On the 8,2, still, and with the kernel from the COPR repo[1].
I tried running: echo DIGD > switch
to (later) switch to the integrated GPU. Log out, get to gdm, log back in to a black screen with the cursor visible and nothing else.
I'm wondering if it's the gdm X11 session running in the background that makes this fail.
That's possible, I have no idea. I have zero issues with GPU switching on my Nvidia-based MacBook Pro, though I only switch on the console with no X11 running. On the AMD-based MBP8 that you're using, the patches have only been tested by William Brown, a colleague of yours at Red Hat Australia.
If you send me dmesg output with "drm.debug=0xf log_buf_len=10M" I can see if I spot any issues.
I'd like to try and get this working properly before bringing the runtime PM support into it.
Absolutely.
Best regards,
Lukas
On Mon, 2016-03-14 at 13:41 +0100, Lukas Wunner wrote:
Hi Bastien,
sorry for the delay.
On Sat, Mar 05, 2016 at 05:31:55PM +0100, Bastien Nocera wrote:
We could, on boot, force using the integrated GPU, turning off the discrete GPU that we're not interested in.
Yes, many people "solve" this by having grub write the requisite commands to gmux' I/O ports, however this approach won't work with gummiboot. Also, the commands that need to be sent to gmux differ between retinas and pre-retinas.
Which is why I'd be fine with having user-space doing it later on. I don't think users should have to poke at the boot parameters under normal circumstances.
So I'd push DIGD to the switch sysfs entry on boot. But I'm guessing that won't turn off the other output we're not interested in.
IGD and DIGD switch to the integrated GPU and also turn off the discrete GPU. However if the machine is already switched to the integrated GPU, the commands are no-ops and the discrete GPU is not turned off.
In other words you need to check (by reading the switch file) which GPU the machine is switched to. If it's the integrated GPU, write OFF to the switch file. If it's the discrete GPU, write DIGD to the switch file.
Once runtime pm works, writing DIGD is sufficient.
This isn't the greatest API, but let me make a note of that: https://github.com/hadess/switcheroo-control/issues/1
I guess that's only useful until we get runtime PM support.
DIGD and DDIS should help (you need to log out/log in again), and if the current GPU is the integrated one, you could force running, say, a game on the discrete GPU using DRI_PRIME=1, correct?
If runtime pm works, then yes. Otherwise you'd have to manually power up the GPU before using DRI_PRIME=1 and power it down afterwards.
Another need for run-time PM.
FWIW, using OFF at runtime made my machine hang, and without any ways for me to get debug output.
Which GPU were you switched to? Did you issue the command while X11 was running? Since the machine is switched to the AMD on boot, I guess you were powering down the Intel GPU. This works on my machine, I get a log entry "i915: switched off". If this doesn't work on your machine it might be an i915 bug on your Sandy Bridge GPU or it might be because X11 is running.
Try booting with "drm.debug=0xf log_buf_len=10M" and see if this gets you log output. If it doesn't, netconsole might help if the hang occurs while the console is locked.
I'll try it out with your runtime PM patches on top of the latest upstream one.
Ok, so using GIGD or DDIS would just switch the output, but not turn off the unused GPU without runtime PM management.
No. They do switch off the other GPU if runtime pm is disabled.
http://wunner.de/mbp_switcheroo_v5-4.5.tar.gz%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%... gpu switching for 4.5
That's the same patch that's already applied to the kernel above (the ones from this patchset thread), right?
I'm not sure, the patches in the copr repository might be an earlier test version.
Might explain the problems I had.
My concerns here would be:
- Do we know how to turn off the integrated GPU automatically, if
the user only used the internal screen and wanted to use the discrete GPU?
Runtime pm is currently disabled by default for i915. The Intel folks are on it. Until that works, the integrated GPU will be powered down when the user manually switches to the discrete GPU with DIS / DDIS and powered up when switching back with IGD / DIGD.
Do we have a way to know whether runtime PM is available for one/both GPUs in the machine? Otherwise this really explodes the testing grid, and it really makes everything harder.
- If only the discrete GPU is powered on, do we know how to power
on the integrated GPU so it can drive the external screen when plugged in?
On the MBP5 2008/09 and MBP6 2010, the external DP port can be switched between GPUs and we switch it together with the panel. So if you switch to the discrete GPU, you can also drive the external DP port on these models and the integrated GPU can be turned off.
On the MBP8 2011 and newer, external ports are combined DP/Thunderbolt ports which can only be driven by the discrete GPU. For some reason the HPD/AUX pins of the ports can still be switched but not the other pins. We should nail these ports to the discrete GPU and not switch those pins together with the panel as we currently do. There's a patch in mbp_switcheroo_v5-4.5-runpm.tar.gz which fixes that. The patch also wakes up the discrete GPU on hotplug, which obviously needs runtime pm.
So that's something else that we can't handle properly without runtime PM support.
- While plymouth is short-lived at boot, Wayland and X11 GNOME
sessions use the GPU. The first user session will run on a VT that's separate from the greeter to implement fast-user switching. So, if we wanted to force using the other GPU for future sessions, we'd need to tell gdm to kill its graphical session and to respawn it so it doesn't hog the GPU and avoid the switch happening. Correct?
I think so, yes.
After looking at our use cases in the GNOME wiki, I think that might not be necessary as we'll want to always run the desktop on the integrated GPU. That'll something to keep in mind if we ever want to support running the desktop on the discrete GPU.
On the 8,2, still, and with the kernel from the COPR repo[1].
I tried running: echo DIGD > switch
to (later) switch to the integrated GPU. Log out, get to gdm, log back in to a black screen with the cursor visible and nothing else.
I'm wondering if it's the gdm X11 session running in the background that makes this fail.
That's possible, I have no idea. I have zero issues with GPU switching on my Nvidia-based MacBook Pro, though I only switch on the console with no X11 running. On the AMD-based MBP8 that you're using, the patches have only been tested by William Brown, a colleague of yours at Red Hat Australia.
If you send me dmesg output with "drm.debug=0xf log_buf_len=10M" I can see if I spot any issues.
OK. Something else to test on my MBP then.
I'd like to try and get this working properly before bringing the runtime PM support into it.
Absolutely.
Thanks for your help.
Reading through the whole mail it seems to me that it's close to impossible to implement a decent integration without runtime PM support: - DRI_PRIME wouldn't work - no external display detection on some machines
Do you have references for the i915 runtime PM support, a bugzilla or mailing-list thread?
Cheers
On Mon, Mar 14, 2016 at 02:37:44PM +0100, Bastien Nocera wrote:
Do you have references for the i915 runtime PM support, a bugzilla or mailing-list thread?
i915.ko has runtime PM support, it's just not yet enabled by default due to some funky corner cases. If you enable it you might hit a bunch of sanity check warnings in dmesg. But besides those it should mostly work. I didn't read the full context, just figured I'll throw this in. -Daniel
I guess that's only useful until we get runtime PM support.
For the discrete GPUs on regular laptops we have runtime PM support for powerdown already. Some newer laptops need a bit of work in the PCIE layer but for most things we have it covered. The known broken ones are Apple laptops. If the apple-gmux code is working well enough to power off GPUs, then it should be possible to hook up runtime-pm on those machines pretty simply.
So there shouldn't really be a case we care about.
runtime PM for the Intel GPU isn't as important. We don't even want to turn the i915 fully off anymore.
After looking at our use cases in the GNOME wiki, I think that might not be necessary as we'll want to always run the desktop on the integrated GPU. That'll something to keep in mind if we ever want to
Reading through the whole mail it seems to me that it's close to impossible to implement a decent integration without runtime PM support:
- DRI_PRIME wouldn't work
- no external display detection on some machines
Do you have references for the i915 runtime PM support, a bugzilla or mailing-list thread?
the i915 runtime PM doesn't matter for this. Only nouveau/radeon runtime PM matters for this, and that should work on most Windows compatible hw right now. For Windows 10 machines there are some patches going around to make things work. For Apple I'm pretty much in the it'll catch up or it won't, but don't block on it.
Dave.
On Tue, 2016-03-15 at 21:10 +1000, Dave Airlie wrote:
I guess that's only useful until we get runtime PM support.
For the discrete GPUs on regular laptops we have runtime PM support for powerdown already. Some newer laptops need a bit of work in the PCIE layer but for most things we have it covered. The known broken ones are Apple laptops. If the apple-gmux code is working well enough to power off GPUs, then it should be possible to hook up runtime-pm on those machines pretty simply.
So there shouldn't really be a case we care about.
runtime PM for the Intel GPU isn't as important. We don't even want to turn the i915 fully off anymore.
Fair enough. And it's not that big a problem if we want to try and run the compositor on the integrated card by default either.
On Mon, 2016-03-14 at 13:41 +0100, Lukas Wunner wrote:
<snip>
So I'd push DIGD to the switch sysfs entry on boot. But I'm guessing that won't turn off the other output we're not interested in.
IGD and DIGD switch to the integrated GPU and also turn off the discrete GPU. However if the machine is already switched to the integrated GPU, the commands are no-ops and the discrete GPU is not turned off.
In other words you need to check (by reading the switch file) which GPU the machine is switched to. If it's the integrated GPU, write OFF to the switch file. If it's the discrete GPU, write DIGD to the switch file.
Once runtime pm works, writing DIGD is sufficient.
I tested the runtime patches for Radeon on top of 4.6.0-rc2, and writing DIGD failed. I also saw a number of messages with the vga_switcheroo core in the kernel trying to switch GPUs but failed because "client 1" was keeping it busy.
Is there any way to see what this "client 1" actually is? I'm guessing plymouth. In any case, once I'm logged in, the "DIS" is powered and used, and the IGD is powered as well.
Cheers
Hi Bastien,
On Tue, Apr 05, 2016 at 06:59:40PM +0200, Bastien Nocera wrote:
I tested the runtime patches for Radeon on top of 4.6.0-rc2, and writing DIGD failed. I also saw a number of messages with the vga_switcheroo core in the kernel trying to switch GPUs but failed because "client 1" was keeping it busy.
Is there any way to see what this "client 1" actually is? I'm guessing plymouth. In any case, once I'm logged in, the "DIS" is powered and used, and the IGD is powered as well.
Client 1 is the discrete GPU, see enum vga_switcheroo_client_id in include/linux/vga_switcheroo.h: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/include...
The vga_switcheroo documentation explains how to find out which user space process is blocking the switch: https://01.org/linuxgraphics/gfx-docs/drm/modes_of_use.html
"Prerequisite is that no user space processes (e.g. Xorg, alsactl) have opened device files of the GPUs or the audio client. If the switch fails, the user may invoke lsof(8) or fuser(1) on /dev/dri/ and /dev/snd/controlC1 to identify processes blocking the switch."
HTH,
Lukas
On Sat, 2016-03-05 at 15:16 +0100, Lukas Wunner wrote:
Hi Bastien,
On Fri, Mar 04, 2016 at 04:12:27PM +0000, Bastien Nocera wrote:
Lukas Wunner <lukas <at> wunner.de> writes:
Enable GPU switching on the pre-retina MacBook Pro (2008 - 2013), v5.
I've tested your patchset on a MacBookPro8,1, with an integrated Intel and discrete AMD/ATI GPUs.
Hm, it must be either an 8,2 or 8,3. The 8,1 was a 13" machine and only had an integrated GPU.
On the 8,2, still, and with the kernel from the COPR repo[1].
I tried running: echo DIGD > switch
to (later) switch to the integrated GPU. Log out, get to gdm, log back in to a black screen with the cursor visible and nothing else.
I'm wondering if it's the gdm X11 session running in the background that makes this fail.
I'd like to try and get this working properly before bringing the runtime PM support into it.
[1]: That's the list of patches in the kernel: http://copr-dist-git.fedorainfracloud.org/cgit/firstyear/kernel-mbp/kernel.g...
On Sat, Mar 5, 2016 at 9:16 AM, Lukas Wunner lukas@wunner.de wrote:
Hi Bastien,
On Fri, Mar 04, 2016 at 04:12:27PM +0000, Bastien Nocera wrote:
Lukas Wunner <lukas <at> wunner.de> writes:
Enable GPU switching on the pre-retina MacBook Pro (2008 - 2013), v5.
I've tested your patchset on a MacBookPro8,1, with an integrated Intel and discrete AMD/ATI GPUs.
Hm, it must be either an 8,2 or 8,3. The 8,1 was a 13" machine and only had an integrated GPU.
I've used the COPR repository here to cut down on my compilation time: https://copr.fedorainfracloud.org/coprs/firstyear/kernel-mbp/
I'm not certain how to test out your changes, or what the consequences should be on a stock Fedora 23/GNOME 3.18 installation. After booting (note that I did not change any command-line options in grub), a gnome-shell/gdm X11 session comes up (I disabled Wayland, to rule out behavioural changes), I'd log in to GNOME and gnome-shell (which starts another X11 session on another VT).
Switching and power control currently requires manual intervention by echoing commands to /sys/kernel/debug/vgaswitcheroo/switch as documented here: https://01.org/linuxgraphics/gfx-docs/drm/modes_of_use.html
As you've correctly observed, the machine is initially switched to the discrete GPU and both GPUs are turned on. By echoing "IGD" to the sysfs file, you'll switch to the integrated GPU and turn off the discrete GPU.
It's possible to let the EFI firmware switch to the integrated GPU on boot by using this tool: https://github.com/0xbb/gpu-switch However still both GPUs will be powered up, so you have to issue the "OFF" command to sysfs to power the discrete GPU down. Also, once you boot into OS X, the setting made by the gpu-switch tool will be overwritten and the machine will be switched to the discrete GPU again the next time you boot Linux.
Note that switching is only possible from the text console, with X11/Wayland shut down. Obviously this is not great in terms of UX. A few years ago there was a GSoC proposal to get hot GPU switching to work on Linux (akin to what OS X does) but nothing ever came of it: http://www.phoronix.com/scan.php?page=news_item&px=OTIyMQ https://lists.x.org/archives/xorg/2011-March/052522.html
Unfortunately this seems to be a low priority item for kernel graphics developers since nowadays most dual GPU notebooks no longer have a mux and cannot switch. The MacBook Pro seems to be the last one supporting this but I've witnessed a bit of an anti-Apple sentiment among kernel graphics developers since everything is non-standard there. Which is unfortunate because these machines have a large market share and Apple software quality is deteriorating rapidly so a lot of Mac users are ripe for converting to Linux.
Is there any reason to make use of the mux? The usage model and amount of stack work for non-mux systems is a lot easier to deal with and covers a lot more systems overall. runtime pm generally works pretty seemlessly for mux-less systems. Properly handling the mux is a lot of work for relatively little gain as there are very few systems that use them.
Anyway, one short-term improvement will be to add runtime pm support (called "Driver power control" in the vga_switcheroo documentation linked above). That way it'll no longer be necessary to power the discrete GPU up and down manually, this will happen automatically as needed (when switching or using render offloading with DRI PRIME). I have patches to enable this for radeon but they're completely untested:
http://wunner.de/mbp_switcheroo_v5-4.5.tar.gz => gpu switching for 4.5 http://wunner.de/mbp_switcheroo_v5-4.5-runpm.tar.gz => runtime pm for radeon
I have an Nvidia based machine and runtime pm doesn't work there yet because of bugs in nouveau that I haven't had the time to look into.
I did not use any external screens to test this.
Since your machine has Thunderbolt, the external port is no longer switchable between GPUs, it can only be driven by the discrete GPU. So you need to power it up manually for this to work. You don't need to switch to it, but it's probably recommendable to save energy. (Otherwise both GPUs are on with the integrated GPU driving the panel and the discrete GPU driving the DP port.)
The runpm tarball linked above contains a patch to automatically wake the discrete GPU on hotplug.
I've heard that the AMD GPU is picky about external monitors and doesn't recognize them unless they're plugged in at exactly the right moment, so you may need to retry a couple of times until it works.
Are talking about some issue specific to these muxed apple systems or in general? If you are having issues, please file a bug.
Alex
Hi Alex,
On Sat, Mar 05, 2016 at 01:10:56PM -0500, Alex Deucher wrote:
Is there any reason to make use of the mux?
Performance (lower latency => no need for framebuffer writes over PCIe), improved battery life (no need to use 2 GPUs simultaneously).
Technically you can't just ignore that the mux is there on the MBP because the kernel has no control over the GPU used on boot. (It's determined by EFI).
I've heard that the AMD GPU is picky about external monitors and doesn't recognize them unless they're plugged in at exactly the right moment, so you may need to retry a couple of times until it works.
Are talking about some issue specific to these muxed apple systems or in general?
Feedback I got from William Brown of Red Hat who tested the GPU switching patches on an MBP8,2 and reported that (independently of the patches), a display connected with an original Apple DP-to-DVI adapter would only be recognized if plugged in at exactly the right moment and in the correct order (first adapter, then display). However it doesn't seem to work better on OS X.
If you are having issues, please file a bug.
I'm not having issues so can't file a bug. Besides, filing a bug is no guarantee that things get fixed. He had opened a bug for GPU switching 3 years ago (https://bugs.freedesktop.org/show_bug.cgi?id=61115) and nobody did a thing. Obviously whether something gets fixed is a function of the perceived importance by maintainers, unless a volunteer comes along and does the dirty work.
Best regards,
Lukas
On Tue, Mar 15, 2016 at 1:54 PM, Lukas Wunner lukas@wunner.de wrote:
Hi Alex,
On Sat, Mar 05, 2016 at 01:10:56PM -0500, Alex Deucher wrote:
Is there any reason to make use of the mux?
Performance (lower latency => no need for framebuffer writes over PCIe), improved battery life (no need to use 2 GPUs simultaneously).
Technically you can't just ignore that the mux is there on the MBP because the kernel has no control over the GPU used on boot. (It's determined by EFI).
Is GPU power switching also handled by the mux? Is it independent of the display mux?
I've heard that the AMD GPU is picky about external monitors and doesn't recognize them unless they're plugged in at exactly the right moment, so you may need to retry a couple of times until it works.
Are talking about some issue specific to these muxed apple systems or in general?
Feedback I got from William Brown of Red Hat who tested the GPU switching patches on an MBP8,2 and reported that (independently of the patches), a display connected with an original Apple DP-to-DVI adapter would only be recognized if plugged in at exactly the right moment and in the correct order (first adapter, then display). However it doesn't seem to work better on OS X.
Sounds like a issue with their adapter.
If you are having issues, please file a bug.
I'm not having issues so can't file a bug. Besides, filing a bug is no guarantee that things get fixed. He had opened a bug for GPU switching 3 years ago (https://bugs.freedesktop.org/show_bug.cgi?id=61115) and nobody did a thing. Obviously whether something gets fixed is a function of the perceived importance by maintainers, unless a volunteer comes along and does the dirty work.
Well, of course everyone is busy and developers will prioritize issues. However, bugs that are not reported have substantially less chance of getting fixed.
Alex
Hi Alex,
On Tue, Mar 15, 2016 at 02:33:56PM -0400, Alex Deucher wrote:
On Tue, Mar 15, 2016 at 1:54 PM, Lukas Wunner lukas@wunner.de wrote:
On Sat, Mar 05, 2016 at 01:10:56PM -0500, Alex Deucher wrote:
Is there any reason to make use of the mux?
Performance (lower latency => no need for framebuffer writes over PCIe), improved battery life (no need to use 2 GPUs simultaneously).
Technically you can't just ignore that the mux is there on the MBP because the kernel has no control over the GPU used on boot. (It's determined by EFI).
Is GPU power switching also handled by the mux? Is it independent of the display mux?
Yes and yes.
Best regards,
Lukas
dri-devel@lists.freedesktop.org
-
Alex Deucher -
Bastien Nocera -
Daniel Vetter -
Dave Airlie -
Lukas Wunner