On Mon, 2012-07-30 at 10:46 +0100, James Bottomley wrote:
On Sun, 2012-07-29 at 21:25 +0200, Rafael J. Wysocki wrote:
On Sunday, July 29, 2012, Rafael J. Wysocki wrote:
On Sunday, July 29, 2012, James Bottomley wrote:
On Sat, 2012-07-28 at 22:29 +0200, Rafael J. Wysocki wrote:
On Saturday, July 28, 2012, Rafael J. Wysocki wrote:
On Saturday, July 28, 2012, James Bottomley wrote: > One of the great things about the 3.4 kernel was the massive increase in > power savings on my x220i laptop. With full PCI suspend, it could get > down to 6.5W in idle with a dim screen, provided I used powertop 2.0 to > activate all the tunings). I just upgraded to 3.5 (the openSUSE > tumbleweed kernel) and all the power savings are gone. Now it's back to > its previous behaviour of idle somewhere between 16-18W.
Please check dbe9a2e (ACPI / PM: Make acpi_pm_device_sleep_state() follow the specification) for starters.
If that is not the culprit, 38c92ff (ACPI / PM: Make __acpi_bus_get_power() cover D3cold correctly) is worth ckecking too.
If none of the above, c2fb8a3 (USB: add NO_D3_DURING_SLEEP flag and revert 151b61284776be2) is one more candidate.
I can't recall anything else that might be related to the symptom at the moment.
Actually, dbe9a2e and c2fb8a3 only affect system suspend, so for runtime PM 38c92ff seems to be the only relevant one from the above.
I verified the problem shows up on vanilla 3.5 (just in case it was an openSUSE problem). I also tried reverting 38c92ff with no success. The symptoms appear to be that something is preventing the PCI/device subsystem from going into auto suspend.
The number of core power management commits during the 3.5 cycle was rather limited and none of them should affect PCI runtime PM. I have no idea what the root cause of that may be, quite frankly.
I've just realized that you said "auto suspend", which makes me think that the problem may be related to USB.
PCI doesn't really use any kind of auto suspend for what I know, but USB does and it may cause PCI USB controllers to be suspended as a result.
I'm trying to bisect this, but I've got stuck around here:
git bisect bad 2f78d8e249973f1eeb88315e6444e616c60177ae git bisect good 28f3d717618156c0dcd2f497d791b578a7931d87
That's around the drm tree ... unfortunately that broke a lot of the basics of my i915 based system (compositing and resolution) as I step into it.
OK, I've run the bisect as far as I can. It looks to be in the drm tree. Unfortunately, this tree has several merge points, some of which go further back than v3.4. Unfortunately, once the bisect steps back before 3.4, we lose the changes that gave us the power savings, making further debugging impossible
Here's the log
git bisect start # bad: [28a33cbc24e4256c143dce96c7d93bf423229f92] Linux 3.5 git bisect bad 28a33cbc24e4256c143dce96c7d93bf423229f92 # bad: [28a33cbc24e4256c143dce96c7d93bf423229f92] Linux 3.5 git bisect bad 28a33cbc24e4256c143dce96c7d93bf423229f92 # good: [76e10d158efb6d4516018846f60c2ab5501900bc] Linux 3.4 git bisect good 76e10d158efb6d4516018846f60c2ab5501900bc # good: [59cd358a7a5b2f6b61faa01dae6cfda3830ac62a] Merge tag 'perf-core-for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/urgent git bisect good 59cd358a7a5b2f6b61faa01dae6cfda3830ac62a # bad: [7e5b2db77b05746613516599c916a8cc2e321077] Merge branch 'upstream' of git://git.linux-mips.org/pub/scm/ralf/upstream-linus git bisect bad 7e5b2db77b05746613516599c916a8cc2e321077 # good: [f9369910a6225b8d4892c3f20ae740a711cd5ace] Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/signal git bisect good f9369910a6225b8d4892c3f20ae740a711cd5ace # bad: [2f78d8e249973f1eeb88315e6444e616c60177ae] Merge tag 'firewire-updates' of git://git.kernel.org/pub/scm/linux/kernel/git/ieee1394/linux1394 git bisect bad 2f78d8e249973f1eeb88315e6444e616c60177ae # good: [28f3d717618156c0dcd2f497d791b578a7931d87] Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net git bisect good 28f3d717618156c0dcd2f497d791b578a7931d87 # bad: [b3daeaef559d87b974c13a096582c5c70dc11061] drm/i915: move rps/emon function declarations git bisect bad b3daeaef559d87b974c13a096582c5c70dc11061 # bad: [246bdbeb0f0fb14c4c085b6ed7edbab11afccd33] drm/i915: share forcewaking code between IVB and HSW git bisect bad 246bdbeb0f0fb14c4c085b6ed7edbab11afccd33
If you do a gitk on the last bad and good
gitk 28f3d717618156c0dcd2f497d791b578a7931d87..246bdbeb0f0fb14c4c085b6ed7edbab11afccd33
You see there are only drm commits in there.
James
James Bottomley James.Bottomley@HansenPartnership.com writes:
OK, I've run the bisect as far as I can. It looks to be in the drm tree. Unfortunately, this tree has several merge points, some of which go further back than v3.4. Unfortunately, once the bisect steps back before 3.4, we lose the changes that gave us the power savings, making further debugging impossible
What machine is this on? There are a few 'disable some power savings' patches in that list to work around issues on various machines; knowing what machine you're using can isolate which ones might have had some effect on power usage...
On Mon, 2012-07-30 at 09:33 -0700, Keith Packard wrote:
James Bottomley James.Bottomley@HansenPartnership.com writes:
OK, I've run the bisect as far as I can. It looks to be in the drm tree. Unfortunately, this tree has several merge points, some of which go further back than v3.4. Unfortunately, once the bisect steps back before 3.4, we lose the changes that gave us the power savings, making further debugging impossible
What machine is this on? There are a few 'disable some power savings' patches in that list to work around issues on various machines; knowing what machine you're using can isolate which ones might have had some effect on power usage...
Lenovo X220i
The display device is
00:02.0 VGA compatible controller: Intel Corporation 82845G/GL[Brookdale-G]/GE Chipset Integrated Graphics Device (rev 03) (prog-if 00 [VGA controller]) Subsystem: Giga-byte Technology Device 2562 Flags: bus master, fast devsel, latency 0, IRQ 16 Memory at e0000000 (32-bit, prefetchable) [size=128M] Memory at e8200000 (32-bit, non-prefetchable) [size=512K] Expansion ROM at <unassigned> [disabled] Capabilities: <access denied> Kernel driver in use: i915
James
James Bottomley James.Bottomley@HansenPartnership.com writes:
On Mon, 2012-07-30 at 09:33 -0700, Keith Packard wrote:
James Bottomley James.Bottomley@HansenPartnership.com writes:
OK, I've run the bisect as far as I can. It looks to be in the drm tree. Unfortunately, this tree has several merge points, some of which go further back than v3.4. Unfortunately, once the bisect steps back before 3.4, we lose the changes that gave us the power savings, making further debugging impossible
What machine is this on? There are a few 'disable some power savings' patches in that list to work around issues on various machines; knowing what machine you're using can isolate which ones might have had some effect on power usage...
Lenovo X220i
I don't see a whole lot of context from the elided email bits you'd sent previously; can you summarize the issue in terms of how much power savings you're losing, how you're measuring it and what's going on in the system when the power savings is different?
Have you tried measuring power with X not running? How about with compositing and other desktop effects disabled?
On Mon, 2012-07-30 at 11:23 -0700, Keith Packard wrote:
James Bottomley James.Bottomley@HansenPartnership.com writes:
On Mon, 2012-07-30 at 09:33 -0700, Keith Packard wrote:
James Bottomley James.Bottomley@HansenPartnership.com writes:
OK, I've run the bisect as far as I can. It looks to be in the drm tree. Unfortunately, this tree has several merge points, some of which go further back than v3.4. Unfortunately, once the bisect steps back before 3.4, we lose the changes that gave us the power savings, making further debugging impossible
What machine is this on? There are a few 'disable some power savings' patches in that list to work around issues on various machines; knowing what machine you're using can isolate which ones might have had some effect on power usage...
Lenovo X220i
I don't see a whole lot of context from the elided email bits you'd sent previously; can you summarize the issue in terms of how much power savings you're losing, how you're measuring it and what's going on in the system when the power savings is different?
Sure. Going from 3.3->3.4 we saw massive increase in power savings due to various autosuspend updates. The idle power consumption of the X220i went from about 13W to 6.5W. In 3.5 this seems to be reversed, with the idle power consumption back up to around 14W. I can't quite believe the graphics chip is responsible for around 7W, so it looks like it's some interaction between graphics and other subsystems.
Have you tried measuring power with X not running? How about with compositing and other desktop effects disabled?
Sure, I can try doing that ... but remember this is a system where the drm is used for the console as well.
James
On Tue, 2012-07-31 at 08:31 +0100, James Bottomley wrote:
On Mon, 2012-07-30 at 11:23 -0700, Keith Packard wrote:
James Bottomley James.Bottomley@HansenPartnership.com writes:
On Mon, 2012-07-30 at 09:33 -0700, Keith Packard wrote:
James Bottomley James.Bottomley@HansenPartnership.com writes:
OK, I've run the bisect as far as I can. It looks to be in the drm tree. Unfortunately, this tree has several merge points, some of which go further back than v3.4. Unfortunately, once the bisect steps back before 3.4, we lose the changes that gave us the power savings, making further debugging impossible
What machine is this on? There are a few 'disable some power savings' patches in that list to work around issues on various machines; knowing what machine you're using can isolate which ones might have had some effect on power usage...
Lenovo X220i
I don't see a whole lot of context from the elided email bits you'd sent previously; can you summarize the issue in terms of how much power savings you're losing, how you're measuring it and what's going on in the system when the power savings is different?
Sure. Going from 3.3->3.4 we saw massive increase in power savings due to various autosuspend updates. The idle power consumption of the X220i went from about 13W to 6.5W. In 3.5 this seems to be reversed, with the idle power consumption back up to around 14W. I can't quite believe the graphics chip is responsible for around 7W, so it looks like it's some interaction between graphics and other subsystems.
Have you tried measuring power with X not running? How about with compositing and other desktop effects disabled?
Sure, I can try doing that ... but remember this is a system where the drm is used for the console as well.
Actually, bad news: it looks like the problem is drm:
on 3.5 killing X causes idle power to go 14W -> 5.9W on 3.4.6 killing X causes idle power to go 6.8W -> 5.7W
James
On Tue, 31 Jul 2012 09:06:42 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
Actually, bad news: it looks like the problem is drm:
on 3.5 killing X causes idle power to go 14W -> 5.9W on 3.4.6 killing X causes idle power to go 6.8W -> 5.7W
The files that will be the most interesting to compare at first are:
/sys/kernel/debug/dri/0/i915_drpc_info /sys/kernel/debug/dri/0/i915_cur_delayinfo /sys/kernel/debug/dri/0/i915_fbc_status
However if it was simple regression in drm, then the bisect would have continued to work despite the merge point jumping between 3.4 and 3.5, right?
Thanks, -Chris
On Tue, 2012-07-31 at 09:28 +0100, Chris Wilson wrote:
On Tue, 31 Jul 2012 09:06:42 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
Actually, bad news: it looks like the problem is drm:
on 3.5 killing X causes idle power to go 14W -> 5.9W on 3.4.6 killing X causes idle power to go 6.8W -> 5.7W
The files that will be the most interesting to compare at first are:
/sys/kernel/debug/dri/0/i915_drpc_info /sys/kernel/debug/dri/0/i915_cur_delayinfo /sys/kernel/debug/dri/0/i915_fbc_status
This is for the good kernel 3.4.6
jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_drpc_info RC information accurate: yes Video Turbo Mode: yes HW control enabled: yes SW control enabled: no RC1e Enabled: no RC6 Enabled: yes Deep RC6 Enabled: no Deepest RC6 Enabled: no Current RC state: RC6 Core Power Down: no jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_cur_delayinfo GT_PERF_STATUS: 0x00000d29 RPSTAT1: 0x00040d00 Render p-state ratio: 13 Render p-state VID: 41 Render p-state limit: 255 CAGF: 650MHz RP CUR UP EI: 20459us RP CUR UP: 172us RP PREV UP: 0us RP CUR DOWN EI: 0us RP CUR DOWN: 0us RP PREV DOWN: 0us Lowest (RPN) frequency: 650MHz Nominal (RP1) frequency: 650MHz Max non-overclocked (RP0) frequency: 1100MHz jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_fbc_status FBC disabled: disabled per module param (default off)
And the bad kernel 3.5
jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_drpc_info RC information accurate: yes Video Turbo Mode: yes HW control enabled: yes SW control enabled: no RC1e Enabled: no RC6 Enabled: yes Deep RC6 Enabled: no Deepest RC6 Enabled: no Current RC state: RC6 Core Power Down: no RC6 "Locked to RPn" residency since boot: 0 RC6 residency since boot: 97671911 RC6+ residency since boot: 0 RC6++ residency since boot: 0 jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_cur_delayinfo GT_PERF_STATUS: 0x00000d29 RPSTAT1: 0x00048d00 Render p-state ratio: 13 Render p-state VID: 41 Render p-state limit: 255 CAGF: 650MHz RP CUR UP EI: 63719us RP CUR UP: 26us RP PREV UP: 0us RP CUR DOWN EI: 0us RP CUR DOWN: 0us RP PREV DOWN: 0us Lowest (RPN) frequency: 650MHz Nominal (RP1) frequency: 650MHz Max non-overclocked (RP0) frequency: 1100MHz jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_fbc_status FBC disabled: disabled per module param (default off)
However if it was simple regression in drm, then the bisect would have continued to work despite the merge point jumping between 3.4 and 3.5, right?
No ... the bisect stepped back into 3.3 which mean I lost the ability to detect the regression. I think it might be fixable given I have a more precise identifier for the tree because it looks like none of the roots of the drm tree is before 3.4-rc.
James
On Tue, 31 Jul 2012 10:37:35 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
On Tue, 2012-07-31 at 09:28 +0100, Chris Wilson wrote:
On Tue, 31 Jul 2012 09:06:42 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
Actually, bad news: it looks like the problem is drm:
on 3.5 killing X causes idle power to go 14W -> 5.9W on 3.4.6 killing X causes idle power to go 6.8W -> 5.7W
The files that will be the most interesting to compare at first are:
/sys/kernel/debug/dri/0/i915_drpc_info /sys/kernel/debug/dri/0/i915_cur_delayinfo /sys/kernel/debug/dri/0/i915_fbc_status
This is for the good kernel 3.4.6
jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_drpc_info RC information accurate: yes Video Turbo Mode: yes HW control enabled: yes SW control enabled: no RC1e Enabled: no RC6 Enabled: yes Deep RC6 Enabled: no Deepest RC6 Enabled: no Current RC state: RC6 Core Power Down: no jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_cur_delayinfo GT_PERF_STATUS: 0x00000d29 RPSTAT1: 0x00040d00 Render p-state ratio: 13 Render p-state VID: 41 Render p-state limit: 255 CAGF: 650MHz RP CUR UP EI: 20459us RP CUR UP: 172us RP PREV UP: 0us RP CUR DOWN EI: 0us RP CUR DOWN: 0us RP PREV DOWN: 0us Lowest (RPN) frequency: 650MHz Nominal (RP1) frequency: 650MHz Max non-overclocked (RP0) frequency: 1100MHz jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_fbc_status FBC disabled: disabled per module param (default off)
And the bad kernel 3.5
jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_drpc_info RC information accurate: yes Video Turbo Mode: yes HW control enabled: yes SW control enabled: no RC1e Enabled: no RC6 Enabled: yes Deep RC6 Enabled: no Deepest RC6 Enabled: no Current RC state: RC6 Core Power Down: no RC6 "Locked to RPn" residency since boot: 0 RC6 residency since boot: 97671911 RC6+ residency since boot: 0 RC6++ residency since boot: 0 jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_cur_delayinfo GT_PERF_STATUS: 0x00000d29 RPSTAT1: 0x00048d00 Render p-state ratio: 13 Render p-state VID: 41 Render p-state limit: 255 CAGF: 650MHz RP CUR UP EI: 63719us RP CUR UP: 26us RP PREV UP: 0us RP CUR DOWN EI: 0us RP CUR DOWN: 0us RP PREV DOWN: 0us Lowest (RPN) frequency: 650MHz Nominal (RP1) frequency: 650MHz Max non-overclocked (RP0) frequency: 1100MHz jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_fbc_status FBC disabled: disabled per module param (default off)
Ok, that rules out the the easy case of rc6 being disabled or not functioning at all, which could easily account for 6W.
When did you inspect the debug files? One effect I can imagine is that if your system was previously stuck at RPn and never upclocking the GPU when X starts. The question would then be what is preventing the GPU from reaching its lowest power state again. -Chris
On Tue, 2012-07-31 at 10:54 +0100, Chris Wilson wrote:
On Tue, 31 Jul 2012 10:37:35 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
On Tue, 2012-07-31 at 09:28 +0100, Chris Wilson wrote:
On Tue, 31 Jul 2012 09:06:42 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
Actually, bad news: it looks like the problem is drm:
on 3.5 killing X causes idle power to go 14W -> 5.9W on 3.4.6 killing X causes idle power to go 6.8W -> 5.7W
The files that will be the most interesting to compare at first are:
/sys/kernel/debug/dri/0/i915_drpc_info /sys/kernel/debug/dri/0/i915_cur_delayinfo /sys/kernel/debug/dri/0/i915_fbc_status
This is for the good kernel 3.4.6
jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_drpc_info RC information accurate: yes Video Turbo Mode: yes HW control enabled: yes SW control enabled: no RC1e Enabled: no RC6 Enabled: yes Deep RC6 Enabled: no Deepest RC6 Enabled: no Current RC state: RC6 Core Power Down: no jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_cur_delayinfo GT_PERF_STATUS: 0x00000d29 RPSTAT1: 0x00040d00 Render p-state ratio: 13 Render p-state VID: 41 Render p-state limit: 255 CAGF: 650MHz RP CUR UP EI: 20459us RP CUR UP: 172us RP PREV UP: 0us RP CUR DOWN EI: 0us RP CUR DOWN: 0us RP PREV DOWN: 0us Lowest (RPN) frequency: 650MHz Nominal (RP1) frequency: 650MHz Max non-overclocked (RP0) frequency: 1100MHz jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_fbc_status FBC disabled: disabled per module param (default off)
And the bad kernel 3.5
jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_drpc_info RC information accurate: yes Video Turbo Mode: yes HW control enabled: yes SW control enabled: no RC1e Enabled: no RC6 Enabled: yes Deep RC6 Enabled: no Deepest RC6 Enabled: no Current RC state: RC6 Core Power Down: no RC6 "Locked to RPn" residency since boot: 0 RC6 residency since boot: 97671911 RC6+ residency since boot: 0 RC6++ residency since boot: 0 jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_cur_delayinfo GT_PERF_STATUS: 0x00000d29 RPSTAT1: 0x00048d00 Render p-state ratio: 13 Render p-state VID: 41 Render p-state limit: 255 CAGF: 650MHz RP CUR UP EI: 63719us RP CUR UP: 26us RP PREV UP: 0us RP CUR DOWN EI: 0us RP CUR DOWN: 0us RP PREV DOWN: 0us Lowest (RPN) frequency: 650MHz Nominal (RP1) frequency: 650MHz Max non-overclocked (RP0) frequency: 1100MHz jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_fbc_status FBC disabled: disabled per module param (default off)
Ok, that rules out the the easy case of rc6 being disabled or not functioning at all, which could easily account for 6W.
When did you inspect the debug files? One effect I can imagine is that if your system was previously stuck at RPn and never upclocking the GPU when X starts. The question would then be what is preventing the GPU from reaching its lowest power state again.
After I logged into an xfce4 session and powertop showed idle had been reached.
James
On Tue, 31 Jul 2012 10:57:10 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
When did you inspect the debug files? One effect I can imagine is that if your system was previously stuck at RPn and never upclocking the GPU when X starts. The question would then be what is preventing the GPU from reaching its lowest power state again.
After I logged into an xfce4 session and powertop showed idle had been reached.
So it looks like rc6 is functioning as the GPU is downclocked whilst idle and the rc6 residency counter is increasing. I guess we need a new tree to bark at. -Chris
On Tue, 31 Jul 2012 11:14:17 +0100, Chris Wilson chris@chris-wilson.co.uk wrote:
On Tue, 31 Jul 2012 10:57:10 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
When did you inspect the debug files? One effect I can imagine is that if your system was previously stuck at RPn and never upclocking the GPU when X starts. The question would then be what is preventing the GPU from reaching its lowest power state again.
After I logged into an xfce4 session and powertop showed idle had been reached.
That you are using xfce4 makes the use of semaphores for pageflips as being the root cause even more suspect. Pageflips are only used for a fullscreen DRI client caalling SwapBuffers, to my knowledge xfce4 does not use DRI at all - its compositing manager is XRender based if you happen to be using it.
Please can you try the small patch to disable the use of semaphores for pageflips and see if the regression remains (which I judge it will...):
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index 5c4657a..f301f2f 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -3067,7 +3067,7 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_o return ret;
if (pipelined != obj->ring) { - ret = i915_gem_object_sync(obj, pipelined); + ret = i915_gem_object_wait_rendering(obj); if (ret) return ret; }
On Tue, 2012-07-31 at 20:24 +0100, Chris Wilson wrote:
On Tue, 31 Jul 2012 11:14:17 +0100, Chris Wilson chris@chris-wilson.co.uk wrote:
On Tue, 31 Jul 2012 10:57:10 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
When did you inspect the debug files? One effect I can imagine is that if your system was previously stuck at RPn and never upclocking the GPU when X starts. The question would then be what is preventing the GPU from reaching its lowest power state again.
After I logged into an xfce4 session and powertop showed idle had been reached.
That you are using xfce4 makes the use of semaphores for pageflips as being the root cause even more suspect. Pageflips are only used for a fullscreen DRI client caalling SwapBuffers, to my knowledge xfce4 does not use DRI at all - its compositing manager is XRender based if you happen to be using it.
Please can you try the small patch to disable the use of semaphores for pageflips and see if the regression remains (which I judge it will...):
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index 5c4657a..f301f2f 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -3067,7 +3067,7 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_o return ret;
if (pipelined != obj->ring) {
ret = i915_gem_object_sync(obj, pipelined);
ret = i915_gem_object_wait_rendering(obj); if (ret) return ret; }
Your patch doesn't apply ... I think because in v3.5 this line is displaced by about 200 lines in the file.
patching file drivers/gpu/drm/i915/i915_gem.c Hunk #1 FAILED at 3067. 1 out of 1 hunk FAILED -- saving rejects to file drivers/gpu/drm/i915/i915_gem.c.rej
I got the attached to apply and it doesn't really improve the idle power much (12.5W).
James
---
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index 288d7b8..2f3f279 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -2869,7 +2869,7 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj, return ret;
if (pipelined != obj->ring) { - ret = i915_gem_object_sync(obj, pipelined); + ret = i915_gem_object_wait_rendering(obj); if (ret) return ret; }
On Wed, 01 Aug 2012 09:06:12 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
I got the attached to apply and it doesn't really improve the idle power much (12.5W).
That's good to know. Next step is to try overriding i915.semaphores. Can you please test with i915.semaphores=0 and i915.semaphores=1? -Chris
On Wed, 2012-08-01 at 09:16 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:06:12 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
I got the attached to apply and it doesn't really improve the idle power much (12.5W).
That's good to know. Next step is to try overriding i915.semaphores. Can you please test with i915.semaphores=0 and i915.semaphores=1?
There's not much point doing i915_semaphores=1 since that's the default on gen 6 hardware, but i915_semaphores=0 recovers and idle power of ~6.5W
James
On Wed, 01 Aug 2012 09:45:04 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
On Wed, 2012-08-01 at 09:16 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:06:12 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
I got the attached to apply and it doesn't really improve the idle power much (12.5W).
That's good to know. Next step is to try overriding i915.semaphores. Can you please test with i915.semaphores=0 and i915.semaphores=1?
There's not much point doing i915_semaphores=1 since that's the default on gen 6 hardware, but i915_semaphores=0 recovers and idle power of ~6.5W
It is only the default if iommu is off, and changing the default was one of the side-effects of the patch you bisected.
Can you please login to the desktop, let it idle, record /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info. Then trace-cmd record -e i915 sleep 10s, and follow up with a new pair of /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info.
This will let us see whether the pm counters are truly advancing and what activity the driver is performing whilst idle. -Chris
On Wed, 2012-08-01 at 09:58 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:45:04 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
On Wed, 2012-08-01 at 09:16 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:06:12 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
I got the attached to apply and it doesn't really improve the idle power much (12.5W).
That's good to know. Next step is to try overriding i915.semaphores. Can you please test with i915.semaphores=0 and i915.semaphores=1?
There's not much point doing i915_semaphores=1 since that's the default on gen 6 hardware, but i915_semaphores=0 recovers and idle power of ~6.5W
It is only the default if iommu is off, and changing the default was one of the side-effects of the patch you bisected.
Sandybridge mobile doesn't have an iommu (or at least, if it does, the kernel doesn't detect it).
Can you please login to the desktop, let it idle, record /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info. Then trace-cmd record -e i915 sleep 10s, and follow up with a new pair of /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info.
This will let us see whether the pm counters are truly advancing and what activity the driver is performing whilst idle.
With or without i915_semaphore=0?
James
On Wed, 01 Aug 2012 10:07:23 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
Can you please login to the desktop, let it idle, record /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info. Then trace-cmd record -e i915 sleep 10s, and follow up with a new pair of /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info.
This will let us see whether the pm counters are truly advancing and what activity the driver is performing whilst idle.
With or without i915_semaphore=0?
With semaphores enabled so that we can see if they are active during the idle period or if merely having used them at some point is enough to trigger the issue. And to see if the pm counters bear any relation to reality. -Chris
On Wed, 2012-08-01 at 09:58 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:45:04 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
On Wed, 2012-08-01 at 09:16 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:06:12 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
I got the attached to apply and it doesn't really improve the idle power much (12.5W).
That's good to know. Next step is to try overriding i915.semaphores. Can you please test with i915.semaphores=0 and i915.semaphores=1?
There's not much point doing i915_semaphores=1 since that's the default on gen 6 hardware, but i915_semaphores=0 recovers and idle power of ~6.5W
It is only the default if iommu is off, and changing the default was one of the side-effects of the patch you bisected.
Can you please login to the desktop, let it idle, record /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info. Then trace-cmd record -e i915 sleep 10s,
OK, what is trace-cmd? It looks similar to perf tools ... is that it?
James
and follow up with a new pair of /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info.
This will let us see whether the pm counters are truly advancing and what activity the driver is performing whilst idle. -Chris
On Wed, 01 Aug 2012 10:38:36 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
On Wed, 2012-08-01 at 09:58 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:45:04 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
On Wed, 2012-08-01 at 09:16 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:06:12 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
I got the attached to apply and it doesn't really improve the idle power much (12.5W).
That's good to know. Next step is to try overriding i915.semaphores. Can you please test with i915.semaphores=0 and i915.semaphores=1?
There's not much point doing i915_semaphores=1 since that's the default on gen 6 hardware, but i915_semaphores=0 recovers and idle power of ~6.5W
It is only the default if iommu is off, and changing the default was one of the side-effects of the patch you bisected.
Can you please login to the desktop, let it idle, record /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info. Then trace-cmd record -e i915 sleep 10s,
OK, what is trace-cmd? It looks similar to perf tools ... is that it?
Yes, it is roughly equivalent and you should be able to achieve the same with perf trace - except I haven't done it before so I don't have quick advice on how to drive it. :) -Chris
On 2012-08-01 03:06, Chris Wilson wrote:
On Wed, 01 Aug 2012 10:38:36 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
On Wed, 2012-08-01 at 09:58 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:45:04 +0100, James Bottomley
James.Bottomley@HansenPartnership.com wrote:
On Wed, 2012-08-01 at 09:16 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:06:12 +0100, James Bottomley
James.Bottomley@HansenPartnership.com wrote:
I got the attached to apply and it doesn't really improve
the idle power
much (12.5W).
That's good to know. Next step is to try overriding
i915.semaphores.
Can you please test with i915.semaphores=0 and
i915.semaphores=1?
There's not much point doing i915_semaphores=1 since that's the
default
on gen 6 hardware, but i915_semaphores=0 recovers and idle power
of
~6.5W
It is only the default if iommu is off, and changing the default was one of the side-effects of the patch you bisected.
Can you please login to the desktop, let it idle, record /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info. Then trace-cmd record -e i915 sleep 10s,
OK, what is trace-cmd? It looks similar to perf tools ... is that it?
Yes, it is roughly equivalent and you should be able to achieve the same with perf trace - except I haven't done it before so I don't have quick advice on how to drive it. :) -Chris
Should be something like: perf record -f -g -e i915:* -a
On Wed, 2012-08-01 at 22:08 -0700, bwidawsk wrote:
On 2012-08-01 03:06, Chris Wilson wrote:
On Wed, 01 Aug 2012 10:38:36 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
On Wed, 2012-08-01 at 09:58 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:45:04 +0100, James Bottomley
James.Bottomley@HansenPartnership.com wrote:
On Wed, 2012-08-01 at 09:16 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:06:12 +0100, James Bottomley
James.Bottomley@HansenPartnership.com wrote:
> I got the attached to apply and it doesn't really improve
the idle power
> much (12.5W).
That's good to know. Next step is to try overriding
i915.semaphores.
Can you please test with i915.semaphores=0 and
i915.semaphores=1?
There's not much point doing i915_semaphores=1 since that's the
default
on gen 6 hardware, but i915_semaphores=0 recovers and idle power
of
~6.5W
It is only the default if iommu is off, and changing the default was one of the side-effects of the patch you bisected.
Can you please login to the desktop, let it idle, record /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info. Then trace-cmd record -e i915 sleep 10s,
OK, what is trace-cmd? It looks similar to perf tools ... is that it?
Yes, it is roughly equivalent and you should be able to achieve the same with perf trace - except I haven't done it before so I don't have quick advice on how to drive it. :) -Chris
Should be something like: perf record -f -g -e i915:* -a
I already sent the output of trace-cmd ... is that enough?
James
On 2012-08-02 00:20, James Bottomley wrote:
On Wed, 2012-08-01 at 22:08 -0700, bwidawsk wrote:
On 2012-08-01 03:06, Chris Wilson wrote:
On Wed, 01 Aug 2012 10:38:36 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
On Wed, 2012-08-01 at 09:58 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:45:04 +0100, James Bottomley
James.Bottomley@HansenPartnership.com wrote:
On Wed, 2012-08-01 at 09:16 +0100, Chris Wilson wrote: > On Wed, 01 Aug 2012 09:06:12 +0100, James Bottomley
James.Bottomley@HansenPartnership.com wrote:
> > I got the attached to apply and it doesn't really improve
the idle power
> > much (12.5W). > > That's good to know. Next step is to try overriding
i915.semaphores.
> Can you please test with i915.semaphores=0 and
i915.semaphores=1?
There's not much point doing i915_semaphores=1 since that's
the
default
on gen 6 hardware, but i915_semaphores=0 recovers and idle
power
of
~6.5W
It is only the default if iommu is off, and changing the
default
was one of the side-effects of the patch you bisected.
Can you please login to the desktop, let it idle, record /sys/kernel/debug/dri/0/i915_cur_delayinfo and
.../i915_drpc_info.
Then trace-cmd record -e i915 sleep 10s,
OK, what is trace-cmd? It looks similar to perf tools ... is
that
it?
Yes, it is roughly equivalent and you should be able to achieve
the
same with perf trace - except I haven't done it before so I don't have quick advice on how to drive it. :) -Chris
Should be something like: perf record -f -g -e i915:* -a
I already sent the output of trace-cmd ... is that enough?
James
Yes, should do. Have we already eliminated the obvious? GPU semaphores will give time back to the GPU clients normally waiting on such things, X, XFCE, whatever else. It'd probably be handy to begin investigating what those guys are doing with their new extra time. Chris, this is what I was getting at on IRC the other day. What do you think?
On Wed, 2012-08-01 at 09:58 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:45:04 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
On Wed, 2012-08-01 at 09:16 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:06:12 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
I got the attached to apply and it doesn't really improve the idle power much (12.5W).
That's good to know. Next step is to try overriding i915.semaphores. Can you please test with i915.semaphores=0 and i915.semaphores=1?
There's not much point doing i915_semaphores=1 since that's the default on gen 6 hardware, but i915_semaphores=0 recovers and idle power of ~6.5W
It is only the default if iommu is off, and changing the default was one of the side-effects of the patch you bisected.
Can you please login to the desktop, let it idle, record /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info. Then trace-cmd record -e i915 sleep 10s, and follow up with a new pair of /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info.
This will let us see whether the pm counters are truly advancing and what activity the driver is performing whilst idle.
OK, so here it is
James
---
jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_cur_delayinfo GT_PERF_STATUS: 0x000016c7 RPSTAT1: 0x0004160d Render p-state ratio: 22 Render p-state VID: 199 Render p-state limit: 255 CAGF: 1100MHz RP CUR UP EI: 96491us RP CUR UP: 252us RP PREV UP: 0us RP CUR DOWN EI: 0us RP CUR DOWN: 513us RP PREV DOWN: 0us Lowest (RPN) frequency: 650MHz Nominal (RP1) frequency: 650MHz Max non-overclocked (RP0) frequency: 1100MHz jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_drpc_info RC information inaccurate because somebody holds a forcewake reference Video Turbo Mode: yes HW control enabled: yes SW control enabled: no RC1e Enabled: no RC6 Enabled: yes Deep RC6 Enabled: no Deepest RC6 Enabled: no Current RC state: on Core Power Down: no RC6 "Locked to RPn" residency since boot: 0 RC6 residency since boot: 360123443 RC6+ residency since boot: 0 RC6++ residency since boot: 0 jejb@dabdike> ./git/trace-cmd/trace-cmd record -e i915 sleep 10s trace-cmd: Permission denied opening '/sys/kernel/debug/tracing/tracing_on' jejb@dabdike> sudo ./git/trace-cmd/trace-cmd record -e i915 sleep 10s /sys/kernel/debug/tracing/events/i915/filter /sys/kernel/debug/tracing/events/*/i915/filter Kernel buffer statistics: Note: "entries" are the entries left in the kernel ring buffer and are not recorded in the trace data. They should all be zero.
CPU: 0 entries: 0 overrun: 0 commit overrun: 0 bytes: 1080 oldest event ts: 1076.352744 now ts: 1076.651396
CPU: 1 entries: 0 overrun: 0 commit overrun: 0 bytes: 932 oldest event ts: 1067.676405 now ts: 1076.651452
CPU: 2 entries: 0 overrun: 0 commit overrun: 0 bytes: 3784 oldest event ts: 1076.090225 now ts: 1076.651501
CPU: 3 entries: 0 overrun: 0 commit overrun: 0 bytes: 0 oldest event ts: 15281105439.050279 now ts: 1076.651550
CPU0 data recorded at offset=0x39a000 221184 bytes in size CPU1 data recorded at offset=0x3d0000 16384 bytes in size CPU2 data recorded at offset=0x3d4000 32768 bytes in size CPU3 data recorded at offset=0x3dc000 0 bytes in size jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_cur_delayinfo GT_PERF_STATUS: 0x000016c7 RPSTAT1: 0x0004160d Render p-state ratio: 22 Render p-state VID: 199 Render p-state limit: 255 CAGF: 1100MHz RP CUR UP EI: 49171us RP CUR UP: 122us RP PREV UP: 0us RP CUR DOWN EI: 0us RP CUR DOWN: 562us RP PREV DOWN: 0us Lowest (RPN) frequency: 650MHz Nominal (RP1) frequency: 650MHz Max non-overclocked (RP0) frequency: 1100MHz jejb@dabdike> cat /sys/kernel/debug/dri/0/i915_drpc_info RC information accurate: yes Video Turbo Mode: yes HW control enabled: yes SW control enabled: no RC1e Enabled: no RC6 Enabled: yes Deep RC6 Enabled: no Deepest RC6 Enabled: no Current RC state: RC6 Core Power Down: no RC6 "Locked to RPn" residency since boot: 0 RC6 residency since boot: 362653127 RC6+ residency since boot: 0 RC6++ residency since boot: 0
On Wed, Aug 01, 2012 at 11:08:19AM +0100, James Bottomley wrote:
On Wed, 2012-08-01 at 09:58 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:45:04 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
On Wed, 2012-08-01 at 09:16 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:06:12 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
I got the attached to apply and it doesn't really improve the idle power much (12.5W).
That's good to know. Next step is to try overriding i915.semaphores. Can you please test with i915.semaphores=0 and i915.semaphores=1?
There's not much point doing i915_semaphores=1 since that's the default on gen 6 hardware, but i915_semaphores=0 recovers and idle power of ~6.5W
It is only the default if iommu is off, and changing the default was one of the side-effects of the patch you bisected.
Can you please login to the desktop, let it idle, record /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info. Then trace-cmd record -e i915 sleep 10s, and follow up with a new pair of /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info.
This will let us see whether the pm counters are truly advancing and what activity the driver is performing whilst idle.
OK, so here it is
James
Hm, if I haven't botched the math, you have a rc6 residency of about 320 seconds between the two cats of drpc_info. Can you please script this so that we have exactly 10s in between? (Aside: 3.6 has a neat interface for rc6 residency in sysfs ...)
Also, you need to attach the output of trace-cmd report (like with perf), so that we see the tracepoints in detail.
Another quick thing to confirm: What is the power consumption on the old kernel when booting with i915.i915_semaphores=1?
Thanks, Daniel
On Sun, 2012-08-05 at 22:36 +0200, Daniel Vetter wrote:
On Wed, Aug 01, 2012 at 11:08:19AM +0100, James Bottomley wrote:
On Wed, 2012-08-01 at 09:58 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:45:04 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
On Wed, 2012-08-01 at 09:16 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:06:12 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
I got the attached to apply and it doesn't really improve the idle power much (12.5W).
That's good to know. Next step is to try overriding i915.semaphores. Can you please test with i915.semaphores=0 and i915.semaphores=1?
There's not much point doing i915_semaphores=1 since that's the default on gen 6 hardware, but i915_semaphores=0 recovers and idle power of ~6.5W
It is only the default if iommu is off, and changing the default was one of the side-effects of the patch you bisected.
Can you please login to the desktop, let it idle, record /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info. Then trace-cmd record -e i915 sleep 10s, and follow up with a new pair of /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info.
This will let us see whether the pm counters are truly advancing and what activity the driver is performing whilst idle.
OK, so here it is
James
Hm, if I haven't botched the math, you have a rc6 residency of about 320 seconds between the two cats of drpc_info. Can you please script this so that we have exactly 10s in between? (Aside: 3.6 has a neat interface for rc6 residency in sysfs ...)
You botched the maths, I think. The three cats after the sleep was three up arrows ... if it went over 11s I'd be surprised.
Also, you need to attach the output of trace-cmd report (like with perf), so that we see the tracepoints in detail.
You mean you want the full trace.dat file rather than what the output summary says? I can, but it's 800k compressed which is probably over the list limit ... I can upload it somewhere when I get back from holiday next Monday.
Another quick thing to confirm: What is the power consumption on the old kernel when booting with i915.i915_semaphores=1?
It idles at around 13W, which means the history of the problem must be this:
What looks to have happened seems to be because of a merge failure in drm:
In 3.2 Keith Packard disabled semaphores on sandybridge with
commit ebbd857e6b9a92c0aff4aacd1b1d2361d888633e Author: Keith Packard keithp@keithp.com Date: Mon Dec 26 17:02:10 2011 -0800
drm/i915: Disable semaphores by default on SNB
Because of an apparent bug causing a GPU hang.
I think this is what gave me the power savings in 3.4 when the PCI layer was ready for it.
It got re-enabled accidentally in 3.5 by a mismerge of
commit 2911a35b2e4eb87ec48d03aeb11f019e51ae3c0d Author: Ben Widawsky ben@bwidawsk.net Date: Thu Apr 5 14:47:36 2012 -0700
drm/i915: use semaphores for the display plane
Because that puts back the pre ebbd857e6b9a92c0aff4aacd1b1d2361d888633e semaphore enabling code, but in a different place, which is probably why it wasn't spotted, so semaphores got re-enabled on sandybridge.
Perhaps what we should be doing is verifying that semaphores aren't sucking the same 6W of power on ivybridge and if not, just re-disable them on sandybridge, since we'll have to do that anyway to re-apply the bug fix.
James
On 2012-08-07 13:43, James Bottomley wrote:
On Sun, 2012-08-05 at 22:36 +0200, Daniel Vetter wrote:
On Wed, Aug 01, 2012 at 11:08:19AM +0100, James Bottomley wrote:
On Wed, 2012-08-01 at 09:58 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:45:04 +0100, James Bottomley
James.Bottomley@HansenPartnership.com wrote:
On Wed, 2012-08-01 at 09:16 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:06:12 +0100, James Bottomley
James.Bottomley@HansenPartnership.com wrote:
> I got the attached to apply and it doesn't really improve
the idle power
> much (12.5W).
That's good to know. Next step is to try overriding
i915.semaphores.
Can you please test with i915.semaphores=0 and
i915.semaphores=1?
There's not much point doing i915_semaphores=1 since that's
the default
on gen 6 hardware, but i915_semaphores=0 recovers and idle
power of
~6.5W
It is only the default if iommu is off, and changing the default was one of the side-effects of the patch you bisected.
Can you please login to the desktop, let it idle, record /sys/kernel/debug/dri/0/i915_cur_delayinfo and
.../i915_drpc_info.
Then trace-cmd record -e i915 sleep 10s, and follow up with a
new pair
of /sys/kernel/debug/dri/0/i915_cur_delayinfo and
.../i915_drpc_info.
This will let us see whether the pm counters are truly advancing
and
what activity the driver is performing whilst idle.
OK, so here it is
James
Hm, if I haven't botched the math, you have a rc6 residency of about 320 seconds between the two cats of drpc_info. Can you please script this so that we have exactly 10s in between? (Aside: 3.6 has a neat interface for rc6 residency in sysfs ...)
You botched the maths, I think. The three cats after the sleep was three up arrows ... if it went over 11s I'd be surprised.
Also, you need to attach the output of trace-cmd report (like with perf), so that we see the tracepoints in detail.
You mean you want the full trace.dat file rather than what the output summary says? I can, but it's 800k compressed which is probably over the list limit ... I can upload it somewhere when I get back from holiday next Monday.
Another quick thing to confirm: What is the power consumption on the old kernel when booting with i915.i915_semaphores=1?
It idles at around 13W, which means the history of the problem must be this:
What looks to have happened seems to be because of a merge failure in drm:
In 3.2 Keith Packard disabled semaphores on sandybridge with
commit ebbd857e6b9a92c0aff4aacd1b1d2361d888633e Author: Keith Packard keithp@keithp.com Date: Mon Dec 26 17:02:10 2011 -0800
drm/i915: Disable semaphores by default on SNBBecause of an apparent bug causing a GPU hang.
I think this is what gave me the power savings in 3.4 when the PCI layer was ready for it.
It got re-enabled accidentally in 3.5 by a mismerge of
commit 2911a35b2e4eb87ec48d03aeb11f019e51ae3c0d Author: Ben Widawsky ben@bwidawsk.net Date: Thu Apr 5 14:47:36 2012 -0700
drm/i915: use semaphores for the display planeBecause that puts back the pre ebbd857e6b9a92c0aff4aacd1b1d2361d888633e semaphore enabling code, but in a different place, which is probably why it wasn't spotted, so semaphores got re-enabled on sandybridge.
Perhaps what we should be doing is verifying that semaphores aren't sucking the same 6W of power on ivybridge and if not, just re-disable them on sandybridge, since we'll have to do that anyway to re-apply the bug fix.
James
Hi James. Would you mind filing a bug on this? In trying to reproduce this issue, I ran into another similar, but different issue ie. not resolved with semaphores=0. The issue I see can be reproduced with intel-gpu-tools/tests/sysfs_rc6_residency. That test is basically the same thing as what Chris/Daniel was asking for earlier with the drpc debugfs file info. In any case, it would be good to centralize all the data we've collected somewhere other than a mailing list/attachments.
http://intellinuxgraphics.org/how_to_report_bug.html
P.S. sorry if you already filed a bug somewhere earlier in the thread. I've been having mail problems.
On Tue, Aug 07, 2012 at 09:43:48PM +0100, James Bottomley wrote:
On Sun, 2012-08-05 at 22:36 +0200, Daniel Vetter wrote:
On Wed, Aug 01, 2012 at 11:08:19AM +0100, James Bottomley wrote:
On Wed, 2012-08-01 at 09:58 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:45:04 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
On Wed, 2012-08-01 at 09:16 +0100, Chris Wilson wrote:
On Wed, 01 Aug 2012 09:06:12 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote: > I got the attached to apply and it doesn't really improve the idle power > much (12.5W).
That's good to know. Next step is to try overriding i915.semaphores. Can you please test with i915.semaphores=0 and i915.semaphores=1?
There's not much point doing i915_semaphores=1 since that's the default on gen 6 hardware, but i915_semaphores=0 recovers and idle power of ~6.5W
It is only the default if iommu is off, and changing the default was one of the side-effects of the patch you bisected.
Can you please login to the desktop, let it idle, record /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info. Then trace-cmd record -e i915 sleep 10s, and follow up with a new pair of /sys/kernel/debug/dri/0/i915_cur_delayinfo and .../i915_drpc_info.
This will let us see whether the pm counters are truly advancing and what activity the driver is performing whilst idle.
OK, so here it is
James
Hm, if I haven't botched the math, you have a rc6 residency of about 320 seconds between the two cats of drpc_info. Can you please script this so that we have exactly 10s in between? (Aside: 3.6 has a neat interface for rc6 residency in sysfs ...)
You botched the maths, I think. The three cats after the sleep was three up arrows ... if it went over 11s I'd be surprised.
Oops, indeed I've botched it. Redoing it you have an rc6 residency of 3.2s. Which is clearly not enough :(
Also, you need to attach the output of trace-cmd report (like with perf), so that we see the tracepoints in detail.
You mean you want the full trace.dat file rather than what the output summary says? I can, but it's 800k compressed which is probably over the list limit ... I can upload it somewhere when I get back from holiday next Monday.
Well, I need the decoded stuff from trace report, but yeah, the entire thing, please.
Another quick thing to confirm: What is the power consumption on the old kernel when booting with i915.i915_semaphores=1?
It idles at around 13W, which means the history of the problem must be this:
What looks to have happened seems to be because of a merge failure in drm:
In 3.2 Keith Packard disabled semaphores on sandybridge with
commit ebbd857e6b9a92c0aff4aacd1b1d2361d888633e Author: Keith Packard keithp@keithp.com Date: Mon Dec 26 17:02:10 2011 -0800
drm/i915: Disable semaphores by default on SNBBecause of an apparent bug causing a GPU hang.
I think this is what gave me the power savings in 3.4 when the PCI layer was ready for it.
It got re-enabled accidentally in 3.5 by a mismerge of
commit 2911a35b2e4eb87ec48d03aeb11f019e51ae3c0d Author: Ben Widawsky ben@bwidawsk.net Date: Thu Apr 5 14:47:36 2012 -0700
drm/i915: use semaphores for the display planeBecause that puts back the pre ebbd857e6b9a92c0aff4aacd1b1d2361d888633e semaphore enabling code, but in a different place, which is probably why it wasn't spotted, so semaphores got re-enabled on sandybridge.
Well, I wanted to enable semaphores by default for 3.5 anyway since we've tracked down the root-cause of these hangs. So it got enabled in that commit by accident due to a rebase goof-up, but it's not enabled by accident in 3.5.
Perhaps what we should be doing is verifying that semaphores aren't sucking the same 6W of power on ivybridge and if not, just re-disable them on sandybridge, since we'll have to do that anyway to re-apply the bug fix.
Ben has already been running around with a killawatt and we don't see any additional power consumption and we can't even reproduce your results on snb. The thing is we have two ways to synchronize the gpu engines: - blocking the thread that submits the rendering for a new engine until all dependencies have completed rendering on other engines. - not blocking the submitting process, but using hw semaphores.
In both cases (assuming userspace submits equal amounts of gpu rendering) we keep the gpu out of rc6 while doing so, but without semaphores we also wake up the cpu (we get tons of interrupts from the gpu when blocking, which we otherwise disable) and generally stall userspace. So there's something fishy going on your system.
To avoid botching up the math we have neat rc6 residency counters in sysfs at /sys/class/drm/card0/power/ (should be in 3.5). To check that the issue is really that the gpu doesn't go into rc6 often enough can you please check what percentage of rc6 residency at idle you get with - no X (iirc that worked powerconsumption-wise in both cases) - xfce running with semaphores=0 and semsphores=1 - maybe a different desktop (or just X running with xterm and no blinking cursor) to see whether it's some funny side-effect of using xfce.
Thanks, Daniel
While trying to track down the power regression, I noticed that on my SNB I had more severe problems, ie. forcewake seemed to never happen once i915 was loaded. After a bit of bisection, I tracked the bad commit to:
commit 7b0cfee1a24efdfe0235bac62e53f686fe8a8e24 Merge: 9756fe3 6b16351 Author: Daniel Vetter daniel.vetter@ffwll.ch Date: Mon Jun 25 19:06:12 2012 +0200
Merge tag 'v3.5-rc4' into drm-intel-next-queued
I changed the macro to the inline statement to debug a little better and noticed that the problem went away. It's a bit embarrassing, but I can't figure out what is wrong with the old macro.
In any case, I think this warrants further review.
Signed-off-by: Ben Widawsky ben@bwidawsk.net --- drivers/gpu/drm/i915/i915_drv.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c index ff569cc..a859169 100644 --- a/drivers/gpu/drm/i915/i915_drv.c +++ b/drivers/gpu/drm/i915/i915_drv.c @@ -1020,10 +1020,13 @@ MODULE_DESCRIPTION(DRIVER_DESC); MODULE_LICENSE("GPL and additional rights");
/* We give fast paths for the really cool registers */ -#define NEEDS_FORCE_WAKE(dev_priv, reg) \ - ((HAS_FORCE_WAKE((dev_priv)->dev)) && \ - ((reg) < 0x40000) && \ - ((reg) != FORCEWAKE)) +static inline bool NEEDS_FORCE_WAKE(struct drm_i915_private *dev_priv, + u32 reg) +{ + return (HAS_FORCE_WAKE(dev_priv->dev)) && + (reg < 0x40000) && + (reg != FORCEWAKE); +}
static bool IS_DISPLAYREG(u32 reg) {
On Mon, 6 Aug 2012 12:31:31 -0700 Ben Widawsky ben@bwidawsk.net wrote:
While trying to track down the power regression, I noticed that on my SNB I had more severe problems, ie. forcewake seemed to never happen once i915 was loaded. After a bit of bisection, I tracked the bad commit to:
commit 7b0cfee1a24efdfe0235bac62e53f686fe8a8e24 Merge: 9756fe3 6b16351 Author: Daniel Vetter daniel.vetter@ffwll.ch Date: Mon Jun 25 19:06:12 2012 +0200
Merge tag 'v3.5-rc4' into drm-intel-next-queuedI changed the macro to the inline statement to debug a little better and noticed that the problem went away. It's a bit embarrassing, but I can't figure out what is wrong with the old macro.
In any case, I think this warrants further review.
Signed-off-by: Ben Widawsky ben@bwidawsk.net
drivers/gpu/drm/i915/i915_drv.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c index ff569cc..a859169 100644 --- a/drivers/gpu/drm/i915/i915_drv.c +++ b/drivers/gpu/drm/i915/i915_drv.c @@ -1020,10 +1020,13 @@ MODULE_DESCRIPTION(DRIVER_DESC); MODULE_LICENSE("GPL and additional rights");
/* We give fast paths for the really cool registers */ -#define NEEDS_FORCE_WAKE(dev_priv, reg) \
- ((HAS_FORCE_WAKE((dev_priv)->dev)) && \
((reg) < 0x40000) && \((reg) != FORCEWAKE))+static inline bool NEEDS_FORCE_WAKE(struct drm_i915_private *dev_priv,
u32 reg)+{
- return (HAS_FORCE_WAKE(dev_priv->dev)) &&
(reg < 0x40000) &&(reg != FORCEWAKE);+}
static bool IS_DISPLAYREG(u32 reg) {
Quick followup for anyone paying attention. The sympon and fix are both reproducible. I've sent relevant info to Daniel to sift through the crap.
James Bottomley James.Bottomley@HansenPartnership.com writes:
on 3.5 killing X causes idle power to go 14W -> 5.9W on 3.4.6 killing X causes idle power to go 6.8W -> 5.7W
That's actually pretty good news -- you're just not getting to RC6 when X is running, but RC6 is otherwise working. And, yes, the GPU really can suck that much power. The debug info that Chris pointed you at should tell a more complete story. For comparison, on my sandybridge box this morning:
:::::::::::::: i915_drpc_info :::::::::::::: RC information accurate: yes Video Turbo Mode: yes HW control enabled: yes SW control enabled: no RC1e Enabled: no RC6 Enabled: yes Deep RC6 Enabled: no Deepest RC6 Enabled: no Current RC state: RC6 Core Power Down: no RC6 "Locked to RPn" residency since boot: 0 RC6 residency since boot: 898339862 RC6+ residency since boot: 0 RC6++ residency since boot: 0 :::::::::::::: i915_cur_delayinfo :::::::::::::: GT_PERF_STATUS: 0x00000729 RPSTAT1: 0x00048700 Render p-state ratio: 7 Render p-state VID: 41 Render p-state limit: 24 CAGF: 350MHz RP CUR UP EI: 54631us RP CUR UP: 136us RP PREV UP: 0us RP CUR DOWN EI: 0us RP CUR DOWN: 0us RP PREV DOWN: 0us Lowest (RPN) frequency: 350MHz Nominal (RP1) frequency: 350MHz Max non-overclocked (RP0) frequency: 1200MHz :::::::::::::: i915_fbc_status :::::::::::::: FBC disabled: multiple pipes are enabled
On Tue, 2012-07-31 at 07:27 -0700, Keith Packard wrote:
James Bottomley James.Bottomley@HansenPartnership.com writes:
on 3.5 killing X causes idle power to go 14W -> 5.9W on 3.4.6 killing X causes idle power to go 6.8W -> 5.7W
That's actually pretty good news -- you're just not getting to RC6 when X is running, but RC6 is otherwise working. And, yes, the GPU really can suck that much power. The debug info that Chris pointed you at should tell a more complete story. For comparison, on my sandybridge box this morning:
Well, I don't know what it means, but I have a culprit from bisect (I managed to manually verify the bisection heads which would step back into 3.3).
2911a35b2e4eb87ec48d03aeb11f019e51ae3c0d is the first bad commit commit 2911a35b2e4eb87ec48d03aeb11f019e51ae3c0d Author: Ben Widawsky ben@bwidawsk.net Date: Thu Apr 5 14:47:36 2012 -0700
drm/i915: use semaphores for the display plane
I'm going to try building 3.5 with this reverted (it doesn't revert cleanly).
James
On Tue, 31 Jul 2012 16:09:44 +0100, James Bottomley James.Bottomley@HansenPartnership.com wrote:
On Tue, 2012-07-31 at 07:27 -0700, Keith Packard wrote:
James Bottomley James.Bottomley@HansenPartnership.com writes:
on 3.5 killing X causes idle power to go 14W -> 5.9W on 3.4.6 killing X causes idle power to go 6.8W -> 5.7W
That's actually pretty good news -- you're just not getting to RC6 when X is running, but RC6 is otherwise working. And, yes, the GPU really can suck that much power. The debug info that Chris pointed you at should tell a more complete story. For comparison, on my sandybridge box this morning:
Well, I don't know what it means, but I have a culprit from bisect (I managed to manually verify the bisection heads which would step back into 3.3).
2911a35b2e4eb87ec48d03aeb11f019e51ae3c0d is the first bad commit commit 2911a35b2e4eb87ec48d03aeb11f019e51ae3c0d Author: Ben Widawsky ben@bwidawsk.net Date: Thu Apr 5 14:47:36 2012 -0700
drm/i915: use semaphores for the display planeI'm going to try building 3.5 with this reverted (it doesn't revert cleanly).
The quick test would be to simply revert the i915_gem_object_pin_to_display_plane() hunk. That would help narrow down whether it is a side-efffect of using the semaphore to synchronize the pageflip, or if the existing code was broken as it was moved.
Thanks, -Chris
On Tue, 2012-07-31 at 16:09 +0100, James Bottomley wrote:
On Tue, 2012-07-31 at 07:27 -0700, Keith Packard wrote:
James Bottomley James.Bottomley@HansenPartnership.com writes:
on 3.5 killing X causes idle power to go 14W -> 5.9W on 3.4.6 killing X causes idle power to go 6.8W -> 5.7W
That's actually pretty good news -- you're just not getting to RC6 when X is running, but RC6 is otherwise working. And, yes, the GPU really can suck that much power. The debug info that Chris pointed you at should tell a more complete story. For comparison, on my sandybridge box this morning:
Well, I don't know what it means, but I have a culprit from bisect (I managed to manually verify the bisection heads which would step back into 3.3).
2911a35b2e4eb87ec48d03aeb11f019e51ae3c0d is the first bad commit commit 2911a35b2e4eb87ec48d03aeb11f019e51ae3c0d Author: Ben Widawsky ben@bwidawsk.net Date: Thu Apr 5 14:47:36 2012 -0700
drm/i915: use semaphores for the display planeI'm going to try building 3.5 with this reverted (it doesn't revert cleanly).
OK, I confirm that reverting this patch in 3.5 recovers a ~6.5W idle power.
James
On 7/30/12 1:05 PM, James Bottomley wrote:
Lenovo X220i
The display device is
00:02.0 VGA compatible controller: Intel Corporation 82845G/GL[Brookdale-G]/GE Chipset Integrated Graphics Device (rev 03) (prog-if 00 [VGA controller]) Subsystem: Giga-byte Technology Device 2562 Flags: bus master, fast devsel, latency 0, IRQ 16 Memory at e0000000 (32-bit, prefetchable) [size=128M] Memory at e8200000 (32-bit, non-prefetchable) [size=512K] Expansion ROM at <unassigned> [disabled] Capabilities: <access denied> Kernel driver in use: i915
What in the world? That's an _ancient_ chip, I wouldn't expect to see it in a laptop that new. I would have assumed X220i to be a Sandybridge like X220.
For that matter when the 845 was current Lenovo wasn't the one making ThinkPads.
- ajax
dri-devel@lists.freedesktop.org
-
Adam Jackson -
Ben Widawsky -
bwidawsk -
Chris Wilson -
Daniel Vetter -
James Bottomley -
Keith Packard