On my test machine Xorg doesn't start anymore when I kexec into a 3.11.0-rc3 kernel. On cold boot everything is fine:
[drm] Initialized drm 1.1.0 20060810 [drm] radeon kernel modesetting enabled. [drm] initializing kernel modesetting (RS780 0x1002:0x9614 0x1043:0x834D). [drm] register mmio base: 0xFBEE0000 [drm] register mmio size: 65536 ATOM BIOS: 113 radeon 0000:01:05.0: VRAM: 128M 0x00000000C0000000 - 0x00000000C7FFFFFF (128M used) radeon 0000:01:05.0: GTT: 512M 0x00000000A0000000 - 0x00000000BFFFFFFF [drm] Detected VRAM RAM=128M, BAR=128M [drm] RAM width 32bits DDR [TTM] Zone kernel: Available graphics memory: 4082356 kiB [TTM] Zone dma32: Available graphics memory: 2097152 kiB [TTM] Initializing pool allocator [TTM] Initializing DMA pool allocator [drm] radeon: 128M of VRAM memory ready [drm] radeon: 512M of GTT memory ready. [drm] GART: num cpu pages 131072, num gpu pages 131072 [drm] Loading RS780 Microcode [drm] PCIE GART of 512M enabled (table at 0x00000000C0040000). radeon 0000:01:05.0: WB enabled radeon 0000:01:05.0: fence driver on ring 0 use gpu addr 0x00000000a0000c00 and cpu addr 0xffff880215c45c00 radeon 0000:01:05.0: fence driver on ring 3 use gpu addr 0x00000000a0000c0c and cpu addr 0xffff880215c45c0c [drm] Supports vblank timestamp caching Rev 1 (10.10.2010). [drm] Driver supports precise vblank timestamp query. [drm] radeon: irq initialized. radeon 0000:01:05.0: setting latency timer to 64 [drm] ring test on 0 succeeded in 0 usecs [drm] ring test on 3 succeeded in 1 usecs [drm] ib test on ring 0 succeeded in 0 usecs [drm] ib test on ring 3 succeeded in 0 usecs [drm] Radeon Display Connectors [drm] Connector 0: [drm] VGA-1 [drm] DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48 0x7e4c 0x7e4c [drm] Encoders: [drm] CRT1: INTERNAL_KLDSCP_DAC1 [drm] Connector 1: [drm] DVI-D-1 [drm] HPD3 [drm] DDC: 0x7e50 0x7e50 0x7e54 0x7e54 0x7e58 0x7e58 0x7e5c 0x7e5c [drm] Encoders: [drm] DFP3: INTERNAL_KLDSCP_LVTMA == power state 0 == ui class: none internal class: boot caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 2 power level 1 sclk: 50000 vddc_index: 2 status: c r b == power state 1 == ui class: performance internal class: none caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 70000 vddc_index: 2 status: == power state 2 == ui class: none internal class: uvd caps: video uvd vclk: 53300 dclk: 40000 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 50000 vddc_index: 1 status: switching from power state: ui class: none internal class: boot caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 2 power level 1 sclk: 50000 vddc_index: 2 status: c b switching to power state: ui class: performance internal class: none caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 70000 vddc_index: 2 status: r [drm] radeon: dpm initialized [drm] fb mappable at 0xF0142000 [drm] vram apper at 0xF0000000 [drm] size 7299072 [drm] fb depth is 24 [drm] pitch is 6912
But after I run kexec things go wrong:
[drm] Initialized drm 1.1.0 20060810 [drm] radeon kernel modesetting enabled. [drm] initializing kernel modesetting (RS780 0x1002:0x9614 0x1043:0x834D). [drm] register mmio base: 0xFBEE0000 [drm] register mmio size: 65536 ATOM BIOS: 113 radeon 0000:01:05.0: VRAM: 128M 0x00000000C0000000 - 0x00000000C7FFFFFF (128M used) radeon 0000:01:05.0: GTT: 512M 0x00000000A0000000 - 0x00000000BFFFFFFF [drm] Detected VRAM RAM=128M, BAR=128M [drm] RAM width 32bits DDR [TTM] Zone kernel: Available graphics memory: 4082356 kiB [TTM] Zone dma32: Available graphics memory: 2097152 kiB [TTM] Initializing pool allocator [TTM] Initializing DMA pool allocator [drm] radeon: 128M of VRAM memory ready [drm] radeon: 512M of GTT memory ready. [drm] GART: num cpu pages 131072, num gpu pages 131072 [drm] Loading RS780 Microcode [drm] PCIE GART of 512M enabled (table at 0x00000000C0040000). radeon 0000:01:05.0: WB enabled radeon 0000:01:05.0: fence driver on ring 0 use gpu addr 0x00000000a0000c00 and cpu addr 0xffff880215c45c00 radeon 0000:01:05.0: fence driver on ring 3 use gpu addr 0x00000000a0000c0c and cpu addr 0xffff880215c45c0c [drm] Supports vblank timestamp caching Rev 1 (10.10.2010). [drm] Driver supports precise vblank timestamp query. [drm] radeon: irq initialized. radeon 0000:01:05.0: setting latency timer to 64 [drm] ring test on 0 succeeded in 0 usecs [drm:r600_dma_ring_test] *ERROR* radeon: ring 3 test failed (0xCAFEDEAD) radeon 0000:01:05.0: disabling GPU acceleration radeon 0000:01:05.0: ffff8802161eb400 unpin not necessary [drm] Radeon Display Connectors [drm] Connector 0: [drm] VGA-1 [drm] DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48 0x7e4c 0x7e4c [drm] Encoders: [drm] CRT1: INTERNAL_KLDSCP_DAC1 [drm] Connector 1: [drm] DVI-D-1 [drm] HPD3 [drm] DDC: 0x7e50 0x7e50 0x7e54 0x7e54 0x7e58 0x7e58 0x7e5c 0x7e5c [drm] Encoders: [drm] DFP3: INTERNAL_KLDSCP_LVTMA == power state 0 == ui class: none internal class: boot caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 2 power level 1 sclk: 50000 vddc_index: 2 status: c r b == power state 1 == ui class: performance internal class: none caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 70000 vddc_index: 2 status: == power state 2 == ui class: none internal class: uvd caps: video uvd vclk: 53300 dclk: 40000 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 50000 vddc_index: 1 status: [drm:radeon_pm_init_dpm] *ERROR* radeon: dpm initialization failed [drm] fb mappable at 0xF0142000 [drm] vram apper at 0xF0000000 [drm] size 7299072 [drm] fb depth is 24 [drm] pitch is 6912
From Xorg.0.log:
... [ 5.217] (II) RADEON(0): VRAM usage limit set to 104864K [ 5.218] (==) RADEON(0): Backing store disabled [ 5.218] (WW) RADEON(0): Direct rendering disabled [ 5.218] (II) RADEON(0): Acceleration disabled [ 5.218] (==) RADEON(0): DPMS enabled [ 5.218] (==) RADEON(0): Silken mouse enabled [ 5.218] (II) RADEON(0): RandR 1.2 enabled, ignore the following RandR disabled message. [ 5.219] (--) RandR disabled [ 5.235] (II) AIGLX: Screen 0 is not DRI2 capable [ 5.235] (II) AIGLX: Screen 0 is not DRI capable [ 5.248] (II) AIGLX: Loaded and initialized swrast [ 5.248] (II) GLX: Initialized DRISWRAST GL provider for screen 0 [ 5.252] (EE) RADEON(0): Rotation requires acceleration! [ 5.252] (EE) Fatal server error: [ 5.252] (EE) failed to create screen resources(EE) [ 5.252] (EE) Please consult the The X.Org Foundation support at http://wiki.x.org for help. [ 5.252] (EE) Please also check the log file at "/var/log/Xorg.0.log" for additional information. [ 5.252] (EE) [ 5.436] (EE) Server terminated with error (1). Closing log file.
I've bisected the issue to: commit f5d9b7f0f93c6a7d40750b8b5528a1e0f0c678fb Author: Alex Deucher alexander.deucher@amd.com Date: Thu Jul 25 21:46:21 2013 -0400
drm/radeon/dpm: fix r600_enable_sclk_control()
Actually program the correct register to enable engine clock scaling control.
Signed-off-by: Alex Deucher alexander.deucher@amd.com
Reverting this commit on top of rc3 "fixes" the problem for me.
On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On my test machine Xorg doesn't start anymore when I kexec into a 3.11.0-rc3 kernel.
With kexec, dpm doesn't get torn down properly which can result in a bad hardware state when the driver loads again. Does the attached patch help? It attempts to disable dpm at startup in case it wasn't torn down properly previously.
Alex
On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On my test machine Xorg doesn't start anymore when I kexec into a 3.11.0-rc3 kernel.
With kexec, dpm doesn't get torn down properly which can result in a bad hardware state when the driver loads again. Does the attached patch help? It attempts to disable dpm at startup in case it wasn't torn down properly previously.
dpm initialization now works, but unfortunately GPU acceleration still gets disabled:
[drm] Initialized drm 1.1.0 20060810 [135/1104] [drm] radeon kernel modesetting enabled. [drm] initializing kernel modesetting (RS780 0x1002:0x9614 0x1043:0x834D). [drm] register mmio base: 0xFBEE0000 [drm] register mmio size: 65536 ATOM BIOS: 113 radeon 0000:01:05.0: VRAM: 128M 0x00000000C0000000 - 0x00000000C7FFFFFF (128M used) radeon 0000:01:05.0: GTT: 512M 0x00000000A0000000 - 0x00000000BFFFFFFF [drm] Detected VRAM RAM=128M, BAR=128M [drm] RAM width 32bits DDR [TTM] Zone kernel: Available graphics memory: 4082356 kiB [TTM] Zone dma32: Available graphics memory: 2097152 kiB [TTM] Initializing pool allocator [TTM] Initializing DMA pool allocator [drm] radeon: 128M of VRAM memory ready [drm] radeon: 512M of GTT memory ready. [drm] GART: num cpu pages 131072, num gpu pages 131072 [drm] Loading RS780 Microcode [drm] PCIE GART of 512M enabled (table at 0x00000000C0040000). radeon 0000:01:05.0: WB enabled radeon 0000:01:05.0: fence driver on ring 0 use gpu addr 0x00000000a0000c00 and cpu addr 0xffff880215c30c00 radeon 0000:01:05.0: fence driver on ring 3 use gpu addr 0x00000000a0000c0c and cpu addr 0xffff880215c30c0c [drm] Supports vblank timestamp caching Rev 1 (10.10.2010). [drm] Driver supports precise vblank timestamp query. [drm] radeon: irq initialized. radeon 0000:01:05.0: setting latency timer to 64 [drm] ring test on 0 succeeded in 1 usecs [drm:r600_dma_ring_test] *ERROR* radeon: ring 3 test failed (0xCAFEDEAD) radeon 0000:01:05.0: disabling GPU acceleration radeon 0000:01:05.0: ffff8802161cfc00 unpin not necessary [drm] Radeon Display Connectors [drm] Connector 0: [drm] VGA-1 [drm] DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48 0x7e4c 0x7e4c [drm] Encoders: [drm] CRT1: INTERNAL_KLDSCP_DAC1 [drm] Connector 1: [drm] DVI-D-1 [drm] HPD3 [drm] DDC: 0x7e50 0x7e50 0x7e54 0x7e54 0x7e58 0x7e58 0x7e5c 0x7e5c [drm] Encoders: [drm] DFP3: INTERNAL_KLDSCP_LVTMA == power state 0 == ui class: none internal class: boot caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 2 power level 1 sclk: 50000 vddc_index: 2 status: c r b == power state 1 == ui class: performance internal class: none caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 70000 vddc_index: 2 status: == power state 2 == ui class: none internal class: uvd caps: video uvd vclk: 53300 dclk: 40000 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 50000 vddc_index: 1 status: switching from power state: ui class: none internal class: boot caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 2 power level 1 sclk: 50000 vddc_index: 2 status: c b switching to power state: ui class: performance internal class: none caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 70000 vddc_index: 2 status: r [drm] radeon: dpm initialized [drm] fb mappable at 0xF0142000 [drm] vram apper at 0xF0000000 [drm] size 7299072 [drm] fb depth is 24 [drm] pitch is 6912 fbcon: radeondrmfb (fb0)
On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On my test machine Xorg doesn't start anymore when I kexec into a 3.11.0-rc3 kernel.
With kexec, dpm doesn't get torn down properly which can result in a bad hardware state when the driver loads again. Does the attached patch help? It attempts to disable dpm at startup in case it wasn't torn down properly previously.
dpm initialization now works, but unfortunately GPU acceleration still gets disabled:
Stupid kexec complicates things. We need to make sure dpm is torn down before we init the rest of the GPU, but dpm needs get initialized later in the init process since it depends on certain other state from the driver. I need to think about this for a bit. I'm not sure of a good way to handle this.
Alex
[drm] Initialized drm 1.1.0 20060810 [135/1104] [drm] radeon kernel modesetting enabled. [drm] initializing kernel modesetting (RS780 0x1002:0x9614 0x1043:0x834D). [drm] register mmio base: 0xFBEE0000 [drm] register mmio size: 65536 ATOM BIOS: 113 radeon 0000:01:05.0: VRAM: 128M 0x00000000C0000000 - 0x00000000C7FFFFFF (128M used) radeon 0000:01:05.0: GTT: 512M 0x00000000A0000000 - 0x00000000BFFFFFFF [drm] Detected VRAM RAM=128M, BAR=128M [drm] RAM width 32bits DDR [TTM] Zone kernel: Available graphics memory: 4082356 kiB [TTM] Zone dma32: Available graphics memory: 2097152 kiB [TTM] Initializing pool allocator [TTM] Initializing DMA pool allocator [drm] radeon: 128M of VRAM memory ready [drm] radeon: 512M of GTT memory ready. [drm] GART: num cpu pages 131072, num gpu pages 131072 [drm] Loading RS780 Microcode [drm] PCIE GART of 512M enabled (table at 0x00000000C0040000). radeon 0000:01:05.0: WB enabled radeon 0000:01:05.0: fence driver on ring 0 use gpu addr 0x00000000a0000c00 and cpu addr 0xffff880215c30c00 radeon 0000:01:05.0: fence driver on ring 3 use gpu addr 0x00000000a0000c0c and cpu addr 0xffff880215c30c0c [drm] Supports vblank timestamp caching Rev 1 (10.10.2010). [drm] Driver supports precise vblank timestamp query. [drm] radeon: irq initialized. radeon 0000:01:05.0: setting latency timer to 64 [drm] ring test on 0 succeeded in 1 usecs [drm:r600_dma_ring_test] *ERROR* radeon: ring 3 test failed (0xCAFEDEAD) radeon 0000:01:05.0: disabling GPU acceleration radeon 0000:01:05.0: ffff8802161cfc00 unpin not necessary [drm] Radeon Display Connectors [drm] Connector 0: [drm] VGA-1 [drm] DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48 0x7e4c 0x7e4c [drm] Encoders: [drm] CRT1: INTERNAL_KLDSCP_DAC1 [drm] Connector 1: [drm] DVI-D-1 [drm] HPD3 [drm] DDC: 0x7e50 0x7e50 0x7e54 0x7e54 0x7e58 0x7e58 0x7e5c 0x7e5c [drm] Encoders: [drm] DFP3: INTERNAL_KLDSCP_LVTMA == power state 0 == ui class: none internal class: boot caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 2 power level 1 sclk: 50000 vddc_index: 2 status: c r b == power state 1 == ui class: performance internal class: none caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 70000 vddc_index: 2 status: == power state 2 == ui class: none internal class: uvd caps: video uvd vclk: 53300 dclk: 40000 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 50000 vddc_index: 1 status: switching from power state: ui class: none internal class: boot caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 2 power level 1 sclk: 50000 vddc_index: 2 status: c b switching to power state: ui class: performance internal class: none caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 70000 vddc_index: 2 status: r [drm] radeon: dpm initialized [drm] fb mappable at 0xF0142000 [drm] vram apper at 0xF0000000 [drm] size 7299072 [drm] fb depth is 24 [drm] pitch is 6912 fbcon: radeondrmfb (fb0)
-- Markus
Alex Deucher alexdeucher@gmail.com wrote:
On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On my test machine Xorg doesn't start anymore when I kexec into a 3.11.0-rc3 kernel.
With kexec, dpm doesn't get torn down properly which can result in a bad hardware state when the driver loads again. Does the attached patch help? It attempts to disable dpm at startup in case it wasn't torn down properly previously.
dpm initialization now works, but unfortunately GPU acceleration
still gets
disabled:
Stupid kexec complicates things. We need to make sure dpm is torn down before we init the rest of the GPU, but dpm needs get initialized later in the init process since it depends on certain other state from the driver. I need to think about this for a bit. I'm not sure of a good way to handle this.
You might just want to implement a shutdown method that cleans things up properly. At least as a first pass until you worry about things like kexec on panic.
Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
Eric
Alex
[drm] Initialized drm 1.1.0 20060810
[135/1104][drm] radeon kernel modesetting enabled. [drm] initializing kernel modesetting (RS780 0x1002:0x9614
0x1043:0x834D).
[drm] register mmio base: 0xFBEE0000 [drm] register mmio size: 65536 ATOM BIOS: 113 radeon 0000:01:05.0: VRAM: 128M 0x00000000C0000000 -
0x00000000C7FFFFFF (128M used)
radeon 0000:01:05.0: GTT: 512M 0x00000000A0000000 -
0x00000000BFFFFFFF
[drm] Detected VRAM RAM=128M, BAR=128M [drm] RAM width 32bits DDR [TTM] Zone kernel: Available graphics memory: 4082356 kiB [TTM] Zone dma32: Available graphics memory: 2097152 kiB [TTM] Initializing pool allocator [TTM] Initializing DMA pool allocator [drm] radeon: 128M of VRAM memory ready [drm] radeon: 512M of GTT memory ready. [drm] GART: num cpu pages 131072, num gpu pages 131072 [drm] Loading RS780 Microcode [drm] PCIE GART of 512M enabled (table at 0x00000000C0040000). radeon 0000:01:05.0: WB enabled radeon 0000:01:05.0: fence driver on ring 0 use gpu addr
0x00000000a0000c00 and cpu addr 0xffff880215c30c00
radeon 0000:01:05.0: fence driver on ring 3 use gpu addr
0x00000000a0000c0c and cpu addr 0xffff880215c30c0c
[drm] Supports vblank timestamp caching Rev 1 (10.10.2010). [drm] Driver supports precise vblank timestamp query. [drm] radeon: irq initialized. radeon 0000:01:05.0: setting latency timer to 64 [drm] ring test on 0 succeeded in 1 usecs [drm:r600_dma_ring_test] *ERROR* radeon: ring 3 test failed
(0xCAFEDEAD)
radeon 0000:01:05.0: disabling GPU acceleration radeon 0000:01:05.0: ffff8802161cfc00 unpin not necessary [drm] Radeon Display Connectors [drm] Connector 0: [drm] VGA-1 [drm] DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48 0x7e4c 0x7e4c [drm] Encoders: [drm] CRT1: INTERNAL_KLDSCP_DAC1 [drm] Connector 1: [drm] DVI-D-1 [drm] HPD3 [drm] DDC: 0x7e50 0x7e50 0x7e54 0x7e54 0x7e58 0x7e58 0x7e5c 0x7e5c [drm] Encoders: [drm] DFP3: INTERNAL_KLDSCP_LVTMA == power state 0 == ui class: none internal class: boot caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 2 power level 1 sclk: 50000 vddc_index: 2 status: c r b == power state 1 == ui class: performance internal class: none caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 70000 vddc_index: 2 status: == power state 2 == ui class: none internal class: uvd caps: video uvd vclk: 53300 dclk: 40000 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 50000 vddc_index: 1 status: switching from power state: ui class: none internal class: boot caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 2 power level 1 sclk: 50000 vddc_index: 2 status: c b switching to power state: ui class: performance internal class: none caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 70000 vddc_index: 2 status: r [drm] radeon: dpm initialized [drm] fb mappable at 0xF0142000 [drm] vram apper at 0xF0000000 [drm] size 7299072 [drm] fb depth is 24 [drm] pitch is 6912 fbcon: radeondrmfb (fb0)
-- Markus
On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman ebiederm@xmission.com wrote:
Alex Deucher alexdeucher@gmail.com wrote:
On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On my test machine Xorg doesn't start anymore when I kexec into a 3.11.0-rc3 kernel.
With kexec, dpm doesn't get torn down properly which can result in a bad hardware state when the driver loads again. Does the attached patch help? It attempts to disable dpm at startup in case it wasn't torn down properly previously.
dpm initialization now works, but unfortunately GPU acceleration
still gets
disabled:
Stupid kexec complicates things. We need to make sure dpm is torn down before we init the rest of the GPU, but dpm needs get initialized later in the init process since it depends on certain other state from the driver. I need to think about this for a bit. I'm not sure of a good way to handle this.
You might just want to implement a shutdown method that cleans things up properly. At least as a first pass until you worry about things like kexec on panic.
Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
Does kexec actually call this shutdown method? The driver implements appropriate clean-up measures if it's shutdown properly.
Alex
Eric
Alex
[drm] Initialized drm 1.1.0 20060810
[135/1104][drm] radeon kernel modesetting enabled. [drm] initializing kernel modesetting (RS780 0x1002:0x9614
0x1043:0x834D).
[drm] register mmio base: 0xFBEE0000 [drm] register mmio size: 65536 ATOM BIOS: 113 radeon 0000:01:05.0: VRAM: 128M 0x00000000C0000000 -
0x00000000C7FFFFFF (128M used)
radeon 0000:01:05.0: GTT: 512M 0x00000000A0000000 -
0x00000000BFFFFFFF
[drm] Detected VRAM RAM=128M, BAR=128M [drm] RAM width 32bits DDR [TTM] Zone kernel: Available graphics memory: 4082356 kiB [TTM] Zone dma32: Available graphics memory: 2097152 kiB [TTM] Initializing pool allocator [TTM] Initializing DMA pool allocator [drm] radeon: 128M of VRAM memory ready [drm] radeon: 512M of GTT memory ready. [drm] GART: num cpu pages 131072, num gpu pages 131072 [drm] Loading RS780 Microcode [drm] PCIE GART of 512M enabled (table at 0x00000000C0040000). radeon 0000:01:05.0: WB enabled radeon 0000:01:05.0: fence driver on ring 0 use gpu addr
0x00000000a0000c00 and cpu addr 0xffff880215c30c00
radeon 0000:01:05.0: fence driver on ring 3 use gpu addr
0x00000000a0000c0c and cpu addr 0xffff880215c30c0c
[drm] Supports vblank timestamp caching Rev 1 (10.10.2010). [drm] Driver supports precise vblank timestamp query. [drm] radeon: irq initialized. radeon 0000:01:05.0: setting latency timer to 64 [drm] ring test on 0 succeeded in 1 usecs [drm:r600_dma_ring_test] *ERROR* radeon: ring 3 test failed
(0xCAFEDEAD)
radeon 0000:01:05.0: disabling GPU acceleration radeon 0000:01:05.0: ffff8802161cfc00 unpin not necessary [drm] Radeon Display Connectors [drm] Connector 0: [drm] VGA-1 [drm] DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48 0x7e4c 0x7e4c [drm] Encoders: [drm] CRT1: INTERNAL_KLDSCP_DAC1 [drm] Connector 1: [drm] DVI-D-1 [drm] HPD3 [drm] DDC: 0x7e50 0x7e50 0x7e54 0x7e54 0x7e58 0x7e58 0x7e5c 0x7e5c [drm] Encoders: [drm] DFP3: INTERNAL_KLDSCP_LVTMA == power state 0 == ui class: none internal class: boot caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 2 power level 1 sclk: 50000 vddc_index: 2 status: c r b == power state 1 == ui class: performance internal class: none caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 70000 vddc_index: 2 status: == power state 2 == ui class: none internal class: uvd caps: video uvd vclk: 53300 dclk: 40000 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 50000 vddc_index: 1 status: switching from power state: ui class: none internal class: boot caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 2 power level 1 sclk: 50000 vddc_index: 2 status: c b switching to power state: ui class: performance internal class: none caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 70000 vddc_index: 2 status: r [drm] radeon: dpm initialized [drm] fb mappable at 0xF0142000 [drm] vram apper at 0xF0000000 [drm] size 7299072 [drm] fb depth is 24 [drm] pitch is 6912 fbcon: radeondrmfb (fb0)
-- Markus
Alex Deucher alexdeucher@gmail.com writes:
On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman ebiederm@xmission.com wrote:
Alex Deucher alexdeucher@gmail.com wrote:
On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On my test machine Xorg doesn't start anymore when I kexec into a 3.11.0-rc3 kernel.
With kexec, dpm doesn't get torn down properly which can result in a bad hardware state when the driver loads again. Does the attached patch help? It attempts to disable dpm at startup in case it wasn't torn down properly previously.
dpm initialization now works, but unfortunately GPU acceleration
still gets
disabled:
Stupid kexec complicates things. We need to make sure dpm is torn down before we init the rest of the GPU, but dpm needs get initialized later in the init process since it depends on certain other state from the driver. I need to think about this for a bit. I'm not sure of a good way to handle this.
You might just want to implement a shutdown method that cleans things up properly. At least as a first pass until you worry about things like kexec on panic.
Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
Does kexec actually call this shutdown method? The driver implements appropriate clean-up measures if it's shutdown properly.
Absoltuely. All parts of the reboot path call ->shutdown. Including kexec.
You don't get a device remove/hotunplug but unless this is a kexec on panic ->shutdown is most definitely called. Now I am talking about the device layer/pci layer shutdown method I don't know how gpu drivers are wired up. GPU land was a little strange last I looked. Hopefully it isn't so strange that there is a method named shutdown that is not wired up.
Eric
On Mon, Jul 29, 2013 at 2:10 PM, Eric W. Biederman ebiederm@xmission.com wrote:
Alex Deucher alexdeucher@gmail.com writes:
On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman ebiederm@xmission.com wrote:
Alex Deucher alexdeucher@gmail.com wrote:
On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote: > On my test machine Xorg doesn't start anymore when I kexec into a > 3.11.0-rc3 kernel.
With kexec, dpm doesn't get torn down properly which can result in a bad hardware state when the driver loads again. Does the attached patch help? It attempts to disable dpm at startup in case it wasn't torn down properly previously.
dpm initialization now works, but unfortunately GPU acceleration
still gets
disabled:
Stupid kexec complicates things. We need to make sure dpm is torn down before we init the rest of the GPU, but dpm needs get initialized later in the init process since it depends on certain other state from the driver. I need to think about this for a bit. I'm not sure of a good way to handle this.
You might just want to implement a shutdown method that cleans things up properly. At least as a first pass until you worry about things like kexec on panic.
Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
Does kexec actually call this shutdown method? The driver implements appropriate clean-up measures if it's shutdown properly.
Absoltuely. All parts of the reboot path call ->shutdown. Including kexec.
You don't get a device remove/hotunplug but unless this is a kexec on panic ->shutdown is most definitely called. Now I am talking about the device layer/pci layer shutdown method I don't know how gpu drivers are wired up. GPU land was a little strange last I looked. Hopefully it isn't so strange that there is a method named shutdown that is not wired up.
It doesn't look like the drm infrastructure has a shutdown callback. The drm drivers register a drm_driver callback struct that includes an unload callback which takes care of all the device teardown (if you unload the module for example). I don't know that it actually gets called at kexec time however. I don't know enough about how kexec works.
Markus, does everything work ok after a reboot? Is it just kexec that is a problem?
Alex
On 2013.07.29 at 15:53 -0400, Alex Deucher wrote:
On Mon, Jul 29, 2013 at 2:10 PM, Eric W. Biederman ebiederm@xmission.com wrote:
Alex Deucher alexdeucher@gmail.com writes:
On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman ebiederm@xmission.com wrote:
Alex Deucher alexdeucher@gmail.com wrote:
On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On 2013.07.29 at 09:58 -0400, Alex Deucher wrote: > On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf > markus@trippelsdorf.de wrote: > > On my test machine Xorg doesn't start anymore when I kexec into a > > 3.11.0-rc3 kernel. > > With kexec, dpm doesn't get torn down properly which can result in a > bad hardware state when the driver loads again. Does the attached > patch help? It attempts to disable dpm at startup in case it wasn't > torn down properly previously.
dpm initialization now works, but unfortunately GPU acceleration
still gets
disabled:
Stupid kexec complicates things. We need to make sure dpm is torn down before we init the rest of the GPU, but dpm needs get initialized later in the init process since it depends on certain other state from the driver. I need to think about this for a bit. I'm not sure of a good way to handle this.
You might just want to implement a shutdown method that cleans things up properly. At least as a first pass until you worry about things like kexec on panic.
Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
Does kexec actually call this shutdown method? The driver implements appropriate clean-up measures if it's shutdown properly.
Absoltuely. All parts of the reboot path call ->shutdown. Including kexec.
You don't get a device remove/hotunplug but unless this is a kexec on panic ->shutdown is most definitely called. Now I am talking about the device layer/pci layer shutdown method I don't know how gpu drivers are wired up. GPU land was a little strange last I looked. Hopefully it isn't so strange that there is a method named shutdown that is not wired up.
It doesn't look like the drm infrastructure has a shutdown callback. The drm drivers register a drm_driver callback struct that includes an unload callback which takes care of all the device teardown (if you unload the module for example). I don't know that it actually gets called at kexec time however. I don't know enough about how kexec works.
Markus, does everything work ok after a reboot? Is it just kexec that is a problem?
Yes.
On 2013.07.29 at 15:53 -0400, Alex Deucher wrote:
On Mon, Jul 29, 2013 at 2:10 PM, Eric W. Biederman ebiederm@xmission.com wrote:
Alex Deucher alexdeucher@gmail.com writes:
On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman ebiederm@xmission.com wrote:
Alex Deucher alexdeucher@gmail.com wrote:
On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On 2013.07.29 at 09:58 -0400, Alex Deucher wrote: > On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf > markus@trippelsdorf.de wrote: > > On my test machine Xorg doesn't start anymore when I kexec into a > > 3.11.0-rc3 kernel. > > With kexec, dpm doesn't get torn down properly which can result in a > bad hardware state when the driver loads again. Does the attached > patch help? It attempts to disable dpm at startup in case it wasn't > torn down properly previously.
dpm initialization now works, but unfortunately GPU acceleration
still gets
disabled:
Stupid kexec complicates things. We need to make sure dpm is torn down before we init the rest of the GPU, but dpm needs get initialized later in the init process since it depends on certain other state from the driver. I need to think about this for a bit. I'm not sure of a good way to handle this.
You might just want to implement a shutdown method that cleans things up properly. At least as a first pass until you worry about things like kexec on panic.
Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
Does kexec actually call this shutdown method? The driver implements appropriate clean-up measures if it's shutdown properly.
Absoltuely. All parts of the reboot path call ->shutdown. Including kexec.
You don't get a device remove/hotunplug but unless this is a kexec on panic ->shutdown is most definitely called. Now I am talking about the device layer/pci layer shutdown method I don't know how gpu drivers are wired up. GPU land was a little strange last I looked. Hopefully it isn't so strange that there is a method named shutdown that is not wired up.
It doesn't look like the drm infrastructure has a shutdown callback. The drm drivers register a drm_driver callback struct that includes an unload callback which takes care of all the device teardown (if you unload the module for example). I don't know that it actually gets called at kexec time however. I don't know enough about how kexec works.
BTW there is r100_restore_sanity() in drm/radeon/r100.c that explicitly handles the kexec case during init. So maybe an r600_restore_sanity() function is needed?
(One of the advantages of using kexec (besides the much quicker boot time) is that the monitor is not switched off and then on during boot. I guess that advantage would be lost if the unload callback would be called.)
On 2013.07.30 at 13:27 +0200, Markus Trippelsdorf wrote:
On 2013.07.29 at 15:53 -0400, Alex Deucher wrote:
On Mon, Jul 29, 2013 at 2:10 PM, Eric W. Biederman ebiederm@xmission.com wrote:
Alex Deucher alexdeucher@gmail.com writes:
On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman ebiederm@xmission.com wrote:
Alex Deucher alexdeucher@gmail.com wrote:
On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote: > On 2013.07.29 at 09:58 -0400, Alex Deucher wrote: >> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf >> markus@trippelsdorf.de wrote: >> > On my test machine Xorg doesn't start anymore when I kexec into a >> > 3.11.0-rc3 kernel. >> >> With kexec, dpm doesn't get torn down properly which can result in a >> bad hardware state when the driver loads again. Does the attached >> patch help? It attempts to disable dpm at startup in case it wasn't >> torn down properly previously. > > dpm initialization now works, but unfortunately GPU acceleration still gets > disabled:
Stupid kexec complicates things. We need to make sure dpm is torn down before we init the rest of the GPU, but dpm needs get initialized later in the init process since it depends on certain other state from the driver. I need to think about this for a bit. I'm not sure of a good way to handle this.
You might just want to implement a shutdown method that cleans things up properly. At least as a first pass until you worry about things like kexec on panic.
Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
Does kexec actually call this shutdown method? The driver implements appropriate clean-up measures if it's shutdown properly.
Absoltuely. All parts of the reboot path call ->shutdown. Including kexec.
You don't get a device remove/hotunplug but unless this is a kexec on panic ->shutdown is most definitely called. Now I am talking about the device layer/pci layer shutdown method I don't know how gpu drivers are wired up. GPU land was a little strange last I looked. Hopefully it isn't so strange that there is a method named shutdown that is not wired up.
It doesn't look like the drm infrastructure has a shutdown callback. The drm drivers register a drm_driver callback struct that includes an unload callback which takes care of all the device teardown (if you unload the module for example). I don't know that it actually gets called at kexec time however. I don't know enough about how kexec works.
BTW there is r100_restore_sanity() in drm/radeon/r100.c that explicitly handles the kexec case during init. So maybe an r600_restore_sanity() function is needed?
Oh, I see r100_restore_sanity() gets also called for the other ASICs...
On Tue, Jul 30, 2013 at 7:27 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On 2013.07.29 at 15:53 -0400, Alex Deucher wrote:
On Mon, Jul 29, 2013 at 2:10 PM, Eric W. Biederman ebiederm@xmission.com wrote:
Alex Deucher alexdeucher@gmail.com writes:
On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman ebiederm@xmission.com wrote:
Alex Deucher alexdeucher@gmail.com wrote:
On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote: > On 2013.07.29 at 09:58 -0400, Alex Deucher wrote: >> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf >> markus@trippelsdorf.de wrote: >> > On my test machine Xorg doesn't start anymore when I kexec into a >> > 3.11.0-rc3 kernel. >> >> With kexec, dpm doesn't get torn down properly which can result in a >> bad hardware state when the driver loads again. Does the attached >> patch help? It attempts to disable dpm at startup in case it wasn't >> torn down properly previously. > > dpm initialization now works, but unfortunately GPU acceleration still gets > disabled:
Stupid kexec complicates things. We need to make sure dpm is torn down before we init the rest of the GPU, but dpm needs get initialized later in the init process since it depends on certain other state from the driver. I need to think about this for a bit. I'm not sure of a good way to handle this.
You might just want to implement a shutdown method that cleans things up properly. At least as a first pass until you worry about things like kexec on panic.
Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
Does kexec actually call this shutdown method? The driver implements appropriate clean-up measures if it's shutdown properly.
Absoltuely. All parts of the reboot path call ->shutdown. Including kexec.
You don't get a device remove/hotunplug but unless this is a kexec on panic ->shutdown is most definitely called. Now I am talking about the device layer/pci layer shutdown method I don't know how gpu drivers are wired up. GPU land was a little strange last I looked. Hopefully it isn't so strange that there is a method named shutdown that is not wired up.
It doesn't look like the drm infrastructure has a shutdown callback. The drm drivers register a drm_driver callback struct that includes an unload callback which takes care of all the device teardown (if you unload the module for example). I don't know that it actually gets called at kexec time however. I don't know enough about how kexec works.
BTW there is r100_restore_sanity() in drm/radeon/r100.c that explicitly handles the kexec case during init. So maybe an r600_restore_sanity() function is needed?
(One of the advantages of using kexec (besides the much quicker boot time) is that the monitor is not switched off and then on during boot. I guess that advantage would be lost if the unload callback would be called.)
r100_restore_sanity() is basically a set of hacks (that gets called at driver startup) to work around the fact that with kexec the drm driver is not torn down correctly. So we could add a bunch more asic specific tear down sequences to deal with dpm (and all the other engines on the GPU that may potentially cause problems if they are not torn down properly), but that will just turn into a mess. All of these hacks also add latency to the driver load. I think the best solution would probably be to figure how to hook up the drm unload callback to the shutdown method that kexec uses.
Alex
On 2013.07.30 at 10:53 -0400, Alex Deucher wrote:
On Tue, Jul 30, 2013 at 7:27 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On 2013.07.29 at 15:53 -0400, Alex Deucher wrote:
On Mon, Jul 29, 2013 at 2:10 PM, Eric W. Biederman ebiederm@xmission.com wrote:
Alex Deucher alexdeucher@gmail.com writes:
On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman ebiederm@xmission.com wrote:
Alex Deucher alexdeucher@gmail.com wrote: >On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf >markus@trippelsdorf.de wrote: >> On 2013.07.29 at 09:58 -0400, Alex Deucher wrote: >>> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf >>> markus@trippelsdorf.de wrote: >>> > On my test machine Xorg doesn't start anymore when I kexec into a >>> > 3.11.0-rc3 kernel. >>> >>> With kexec, dpm doesn't get torn down properly which can result in a >>> bad hardware state when the driver loads again. Does the attached >>> patch help? It attempts to disable dpm at startup in case it wasn't >>> torn down properly previously. >> >> dpm initialization now works, but unfortunately GPU acceleration >still gets >> disabled: > >Stupid kexec complicates things. We need to make sure dpm is torn >down before we init the rest of the GPU, but dpm needs get initialized >later in the init process since it depends on certain other state from >the driver. I need to think about this for a bit. I'm not sure of a >good way to handle this.
You might just want to implement a shutdown method that cleans things up properly. At least as a first pass until you worry about things like kexec on panic.
Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
Does kexec actually call this shutdown method? The driver implements appropriate clean-up measures if it's shutdown properly.
Absoltuely. All parts of the reboot path call ->shutdown. Including kexec.
You don't get a device remove/hotunplug but unless this is a kexec on panic ->shutdown is most definitely called. Now I am talking about the device layer/pci layer shutdown method I don't know how gpu drivers are wired up. GPU land was a little strange last I looked. Hopefully it isn't so strange that there is a method named shutdown that is not wired up.
It doesn't look like the drm infrastructure has a shutdown callback. The drm drivers register a drm_driver callback struct that includes an unload callback which takes care of all the device teardown (if you unload the module for example). I don't know that it actually gets called at kexec time however. I don't know enough about how kexec works.
BTW there is r100_restore_sanity() in drm/radeon/r100.c that explicitly handles the kexec case during init. So maybe an r600_restore_sanity() function is needed?
(One of the advantages of using kexec (besides the much quicker boot time) is that the monitor is not switched off and then on during boot. I guess that advantage would be lost if the unload callback would be called.)
r100_restore_sanity() is basically a set of hacks (that gets called at driver startup) to work around the fact that with kexec the drm driver is not torn down correctly. So we could add a bunch more asic specific tear down sequences to deal with dpm (and all the other engines on the GPU that may potentially cause problems if they are not torn down properly), but that will just turn into a mess. All of these hacks also add latency to the driver load. I think the best solution would probably be to figure how to hook up the drm unload callback to the shutdown method that kexec uses.
FYI the following (ugly) hack works for me. (If I don't comment out radeon_fbdev_fini(rdev) kexec will hang.)
diff --git a/drivers/gpu/drm/radeon/r100.c b/drivers/gpu/drm/radeon/r100.c index 75349cd..13e2988 100644 --- a/drivers/gpu/drm/radeon/r100.c +++ b/drivers/gpu/drm/radeon/r100.c @@ -3947,20 +3947,6 @@ void r100_fini(struct radeon_device *rdev) */ void r100_restore_sanity(struct radeon_device *rdev) { - u32 tmp; - - tmp = RREG32(RADEON_CP_CSQ_CNTL); - if (tmp) { - WREG32(RADEON_CP_CSQ_CNTL, 0); - } - tmp = RREG32(RADEON_CP_RB_CNTL); - if (tmp) { - WREG32(RADEON_CP_RB_CNTL, 0); - } - tmp = RREG32(RADEON_SCRATCH_UMSK); - if (tmp) { - WREG32(RADEON_SCRATCH_UMSK, 0); - } }
int r100_init(struct radeon_device *rdev) diff --git a/drivers/gpu/drm/radeon/radeon_display.c b/drivers/gpu/drm/radeon/radeon_display.c index c2b67b4..79b38e2 100644 --- a/drivers/gpu/drm/radeon/radeon_display.c +++ b/drivers/gpu/drm/radeon/radeon_display.c @@ -1405,7 +1405,7 @@ int radeon_modeset_init(struct radeon_device *rdev)
void radeon_modeset_fini(struct radeon_device *rdev) { - radeon_fbdev_fini(rdev); +// radeon_fbdev_fini(rdev); kfree(rdev->mode_info.bios_hardcoded_edid); radeon_pm_fini(rdev);
diff --git a/drivers/gpu/drm/radeon/radeon_drv.c b/drivers/gpu/drm/radeon/radeon_drv.c index 29876b1..e705e8c 100644 --- a/drivers/gpu/drm/radeon/radeon_drv.c +++ b/drivers/gpu/drm/radeon/radeon_drv.c @@ -388,6 +388,15 @@ static const struct file_operations radeon_driver_kms_fops = { #endif };
+ +static void +radeon_kexec_shutdown(struct pci_dev *pdev) +{ + struct drm_device *dev = pci_get_drvdata(pdev); + + radeon_driver_unload_kms(dev); +} + static struct drm_driver kms_driver = { .driver_features = DRIVER_USE_AGP | DRIVER_USE_MTRR | DRIVER_PCI_DMA | DRIVER_SG | @@ -463,6 +472,7 @@ static struct pci_driver radeon_kms_pci_driver = { .remove = radeon_pci_remove, .suspend = radeon_pci_suspend, .resume = radeon_pci_resume, + .shutdown = radeon_kexec_shutdown, };
static int __init radeon_init(void)
On 2013.07.30 at 20:46 +0200, Markus Trippelsdorf wrote:
On 2013.07.30 at 10:53 -0400, Alex Deucher wrote:
On Tue, Jul 30, 2013 at 7:27 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On 2013.07.29 at 15:53 -0400, Alex Deucher wrote:
On Mon, Jul 29, 2013 at 2:10 PM, Eric W. Biederman ebiederm@xmission.com wrote:
Alex Deucher alexdeucher@gmail.com writes:
On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman ebiederm@xmission.com wrote: > > > Alex Deucher alexdeucher@gmail.com wrote: >>On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf >>markus@trippelsdorf.de wrote: >>> On 2013.07.29 at 09:58 -0400, Alex Deucher wrote: >>>> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf >>>> markus@trippelsdorf.de wrote: >>>> > On my test machine Xorg doesn't start anymore when I kexec into a >>>> > 3.11.0-rc3 kernel. >>>> >>>> With kexec, dpm doesn't get torn down properly which can result in a >>>> bad hardware state when the driver loads again. Does the attached >>>> patch help? It attempts to disable dpm at startup in case it wasn't >>>> torn down properly previously. >>> >>> dpm initialization now works, but unfortunately GPU acceleration >>still gets >>> disabled: >> >>Stupid kexec complicates things. We need to make sure dpm is torn >>down before we init the rest of the GPU, but dpm needs get initialized >>later in the init process since it depends on certain other state from >>the driver. I need to think about this for a bit. I'm not sure of a >>good way to handle this. > > You might just want to implement a shutdown method that cleans things up properly. At least as a first pass until you worry about things like kexec on panic. > > Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
Does kexec actually call this shutdown method? The driver implements appropriate clean-up measures if it's shutdown properly.
Absoltuely. All parts of the reboot path call ->shutdown. Including kexec.
You don't get a device remove/hotunplug but unless this is a kexec on panic ->shutdown is most definitely called. Now I am talking about the device layer/pci layer shutdown method I don't know how gpu drivers are wired up. GPU land was a little strange last I looked. Hopefully it isn't so strange that there is a method named shutdown that is not wired up.
It doesn't look like the drm infrastructure has a shutdown callback. The drm drivers register a drm_driver callback struct that includes an unload callback which takes care of all the device teardown (if you unload the module for example). I don't know that it actually gets called at kexec time however. I don't know enough about how kexec works.
BTW there is r100_restore_sanity() in drm/radeon/r100.c that explicitly handles the kexec case during init. So maybe an r600_restore_sanity() function is needed?
(One of the advantages of using kexec (besides the much quicker boot time) is that the monitor is not switched off and then on during boot. I guess that advantage would be lost if the unload callback would be called.)
r100_restore_sanity() is basically a set of hacks (that gets called at driver startup) to work around the fact that with kexec the drm driver is not torn down correctly. So we could add a bunch more asic specific tear down sequences to deal with dpm (and all the other engines on the GPU that may potentially cause problems if they are not torn down properly), but that will just turn into a mess. All of these hacks also add latency to the driver load. I think the best solution would probably be to figure how to hook up the drm unload callback to the shutdown method that kexec uses.
FYI the following (ugly) hack works for me.
No. It still fails, although much more infrequently (~ on every 6-8 boot).
I begin to wonder if: [drm:r600_dma_ring_test] *ERROR* radeon: ring 3 test failed (0xCAFEDEAD) is an simple initialization bug that doesn't directly depend on kexec at all.
2013/7/30 Markus Trippelsdorf markus@trippelsdorf.de:
I begin to wonder if: [drm:r600_dma_ring_test] *ERROR* radeon: ring 3 test failed (0xCAFEDEAD) is an simple initialization bug that doesn't directly depend on kexec at all.
-- Markus
I bet on this (because I see the same error in another context)
2013/7/29 Alex Deucher alexdeucher@gmail.com:
On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On my test machine Xorg doesn't start anymore when I kexec into a 3.11.0-rc3 kernel.
With kexec, dpm doesn't get torn down properly which can result in a bad hardware state when the driver loads again. Does the attached patch help? It attempts to disable dpm at startup in case it wasn't torn down properly previously.
dpm initialization now works, but unfortunately GPU acceleration still gets disabled:
Stupid kexec complicates things. We need to make sure dpm is torn down before we init the rest of the GPU, but dpm needs get initialized later in the init process since it depends on certain other state from the driver. I need to think about this for a bit. I'm not sure of a good way to handle this.
Alex
[drm] Initialized drm 1.1.0 20060810 [135/1104] [drm] radeon kernel modesetting enabled. [drm] initializing kernel modesetting (RS780 0x1002:0x9614 0x1043:0x834D). [drm] register mmio base: 0xFBEE0000 [drm] register mmio size: 65536 ATOM BIOS: 113 radeon 0000:01:05.0: VRAM: 128M 0x00000000C0000000 - 0x00000000C7FFFFFF (128M used) radeon 0000:01:05.0: GTT: 512M 0x00000000A0000000 - 0x00000000BFFFFFFF [drm] Detected VRAM RAM=128M, BAR=128M [drm] RAM width 32bits DDR [TTM] Zone kernel: Available graphics memory: 4082356 kiB [TTM] Zone dma32: Available graphics memory: 2097152 kiB [TTM] Initializing pool allocator [TTM] Initializing DMA pool allocator [drm] radeon: 128M of VRAM memory ready [drm] radeon: 512M of GTT memory ready. [drm] GART: num cpu pages 131072, num gpu pages 131072 [drm] Loading RS780 Microcode [drm] PCIE GART of 512M enabled (table at 0x00000000C0040000). radeon 0000:01:05.0: WB enabled radeon 0000:01:05.0: fence driver on ring 0 use gpu addr 0x00000000a0000c00 and cpu addr 0xffff880215c30c00 radeon 0000:01:05.0: fence driver on ring 3 use gpu addr 0x00000000a0000c0c and cpu addr 0xffff880215c30c0c [drm] Supports vblank timestamp caching Rev 1 (10.10.2010). [drm] Driver supports precise vblank timestamp query. [drm] radeon: irq initialized. radeon 0000:01:05.0: setting latency timer to 64 [drm] ring test on 0 succeeded in 1 usecs [drm:r600_dma_ring_test] *ERROR* radeon: ring 3 test failed (0xCAFEDEAD) radeon 0000:01:05.0: disabling GPU acceleration radeon 0000:01:05.0: ffff8802161cfc00 unpin not necessary [drm] Radeon Display Connectors [drm] Connector 0: [drm] VGA-1 [drm] DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48 0x7e4c 0x7e4c [drm] Encoders: [drm] CRT1: INTERNAL_KLDSCP_DAC1 [drm] Connector 1: [drm] DVI-D-1 [drm] HPD3 [drm] DDC: 0x7e50 0x7e50 0x7e54 0x7e54 0x7e58 0x7e58 0x7e5c 0x7e5c [drm] Encoders: [drm] DFP3: INTERNAL_KLDSCP_LVTMA == power state 0 == ui class: none internal class: boot caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 2 power level 1 sclk: 50000 vddc_index: 2 status: c r b == power state 1 == ui class: performance internal class: none caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 70000 vddc_index: 2 status: == power state 2 == ui class: none internal class: uvd caps: video uvd vclk: 53300 dclk: 40000 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 50000 vddc_index: 1 status: switching from power state: ui class: none internal class: boot caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 2 power level 1 sclk: 50000 vddc_index: 2 status: c b switching to power state: ui class: performance internal class: none caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 70000 vddc_index: 2 status: r [drm] radeon: dpm initialized [drm] fb mappable at 0xF0142000 [drm] vram apper at 0xF0000000 [drm] size 7299072 [drm] fb depth is 24 [drm] pitch is 6912 fbcon: radeondrmfb (fb0)
-- Markus
dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel
This error message seems similar to mine "[drm:r600_uvd_ring_test] *ERROR* radeon: ring 5 test failed (0xCAFEDEAD)" Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=67276 In my case I blame another commit for this. Are these bugs related?
On 2013.07.29 at 18:14 +0200, Joshua C. wrote:
This error message seems similar to mine "[drm:r600_uvd_ring_test] *ERROR* radeon: ring 5 test failed (0xCAFEDEAD)" Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=67276 In my case I blame another commit for this. Are these bugs related?
I guess not, because reverting commit 9cc2e0e9f13 doesn't fix the issue for me. Can you check if reverting commit f5d9b7f0f9 does fixes the problem for you?
On Mon, Jul 29, 2013 at 12:14 PM, Joshua C. joshuacov@gmail.com wrote:
2013/7/29 Alex Deucher alexdeucher@gmail.com:
On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf markus@trippelsdorf.de wrote:
On my test machine Xorg doesn't start anymore when I kexec into a 3.11.0-rc3 kernel.
With kexec, dpm doesn't get torn down properly which can result in a bad hardware state when the driver loads again. Does the attached patch help? It attempts to disable dpm at startup in case it wasn't torn down properly previously.
dpm initialization now works, but unfortunately GPU acceleration still gets disabled:
Stupid kexec complicates things. We need to make sure dpm is torn down before we init the rest of the GPU, but dpm needs get initialized later in the init process since it depends on certain other state from the driver. I need to think about this for a bit. I'm not sure of a good way to handle this.
Alex
[drm] Initialized drm 1.1.0 20060810 [135/1104] [drm] radeon kernel modesetting enabled. [drm] initializing kernel modesetting (RS780 0x1002:0x9614 0x1043:0x834D). [drm] register mmio base: 0xFBEE0000 [drm] register mmio size: 65536 ATOM BIOS: 113 radeon 0000:01:05.0: VRAM: 128M 0x00000000C0000000 - 0x00000000C7FFFFFF (128M used) radeon 0000:01:05.0: GTT: 512M 0x00000000A0000000 - 0x00000000BFFFFFFF [drm] Detected VRAM RAM=128M, BAR=128M [drm] RAM width 32bits DDR [TTM] Zone kernel: Available graphics memory: 4082356 kiB [TTM] Zone dma32: Available graphics memory: 2097152 kiB [TTM] Initializing pool allocator [TTM] Initializing DMA pool allocator [drm] radeon: 128M of VRAM memory ready [drm] radeon: 512M of GTT memory ready. [drm] GART: num cpu pages 131072, num gpu pages 131072 [drm] Loading RS780 Microcode [drm] PCIE GART of 512M enabled (table at 0x00000000C0040000). radeon 0000:01:05.0: WB enabled radeon 0000:01:05.0: fence driver on ring 0 use gpu addr 0x00000000a0000c00 and cpu addr 0xffff880215c30c00 radeon 0000:01:05.0: fence driver on ring 3 use gpu addr 0x00000000a0000c0c and cpu addr 0xffff880215c30c0c [drm] Supports vblank timestamp caching Rev 1 (10.10.2010). [drm] Driver supports precise vblank timestamp query. [drm] radeon: irq initialized. radeon 0000:01:05.0: setting latency timer to 64 [drm] ring test on 0 succeeded in 1 usecs [drm:r600_dma_ring_test] *ERROR* radeon: ring 3 test failed (0xCAFEDEAD) radeon 0000:01:05.0: disabling GPU acceleration radeon 0000:01:05.0: ffff8802161cfc00 unpin not necessary [drm] Radeon Display Connectors [drm] Connector 0: [drm] VGA-1 [drm] DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48 0x7e4c 0x7e4c [drm] Encoders: [drm] CRT1: INTERNAL_KLDSCP_DAC1 [drm] Connector 1: [drm] DVI-D-1 [drm] HPD3 [drm] DDC: 0x7e50 0x7e50 0x7e54 0x7e54 0x7e58 0x7e58 0x7e5c 0x7e5c [drm] Encoders: [drm] DFP3: INTERNAL_KLDSCP_LVTMA == power state 0 == ui class: none internal class: boot caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 2 power level 1 sclk: 50000 vddc_index: 2 status: c r b == power state 1 == ui class: performance internal class: none caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 70000 vddc_index: 2 status: == power state 2 == ui class: none internal class: uvd caps: video uvd vclk: 53300 dclk: 40000 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 50000 vddc_index: 1 status: switching from power state: ui class: none internal class: boot caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 2 power level 1 sclk: 50000 vddc_index: 2 status: c b switching to power state: ui class: performance internal class: none caps: video uvd vclk: 0 dclk: 0 power level 0 sclk: 50000 vddc_index: 1 power level 1 sclk: 70000 vddc_index: 2 status: r [drm] radeon: dpm initialized [drm] fb mappable at 0xF0142000 [drm] vram apper at 0xF0000000 [drm] size 7299072 [drm] fb depth is 24 [drm] pitch is 6912 fbcon: radeondrmfb (fb0)
-- Markus
dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel
This error message seems similar to mine "[drm:r600_uvd_ring_test] *ERROR* radeon: ring 5 test failed (0xCAFEDEAD)" Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=67276 In my case I blame another commit for this. Are these bugs related?
I doubt it's related at all. You issue seems to be an initialization problem with UVD.
Alex
dri-devel@lists.freedesktop.org
-
Alex Deucher -
ebiederm＠xmission.com -
Eric W. Biederman
-
Joshua C. -
Markus Trippelsdorf