We unref the man->move fence in ttm_bo_clean_mm() and then call ttm_bo_force_list_clean() which waits on it, except the refcount is now zero so a warning is generated (or worse):
[149492.279301] refcount_t: increment on 0; use-after-free. [149492.279309] ------------[ cut here ]------------ [149492.279315] WARNING: CPU: 3 PID: 18726 at lib/refcount.c:150 refcount_inc+0x2b/0x30 [149492.279315] Modules linked in: vhost_net vhost tun x86_pkg_temp_thermal crc32_pclmul ghash_clmulni_intel efivarfs amdgpu( -) i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm [149492.279326] CPU: 3 PID: 18726 Comm: rmmod Not tainted 4.12.0-rc5-drm-next-4.13-ttmpatch+ #1 [149492.279326] Hardware name: Gigabyte Technology Co., Ltd. Z97X-UD3H-BK/Z97X-UD3H-BK-CF, BIOS F6 06/17/2014 [149492.279327] task: ffff8804ddfedcc0 task.stack: ffffc90008d20000 [149492.279329] RIP: 0010:refcount_inc+0x2b/0x30 [149492.279330] RSP: 0018:ffffc90008d23c30 EFLAGS: 00010286 [149492.279331] RAX: 000000000000002b RBX: 0000000000000170 RCX: 0000000000000000 [149492.279331] RDX: 0000000000000000 RSI: ffff88051ecccbe8 RDI: ffff88051ecccbe8 [149492.279332] RBP: ffffc90008d23c30 R08: 0000000000000001 R09: 00000000000003ee [149492.279333] R10: ffffc90008d23bb0 R11: 00000000000003ee R12: ffff88043aaac960 [149492.279333] R13: ffff8805005e28a8 R14: 0000000000000002 R15: ffff88050115e178 [149492.279334] FS: 00007fc540168700(0000) GS:ffff88051ecc0000(0000) knlGS:0000000000000000 [149492.279335] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [149492.279336] CR2: 00007fc3e8654140 CR3: 000000027ba77000 CR4: 00000000001426e0 [149492.279337] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [149492.279337] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [149492.279338] Call Trace: [149492.279345] ttm_bo_force_list_clean+0xb9/0x110 [ttm] [149492.279348] ttm_bo_clean_mm+0x7a/0xe0 [ttm] [149492.279375] amdgpu_ttm_fini+0xc9/0x1f0 [amdgpu] [149492.279392] amdgpu_bo_fini+0x12/0x40 [amdgpu] [149492.279415] gmc_v7_0_sw_fini+0x32/0x40 [amdgpu] [149492.279430] amdgpu_fini+0x2c9/0x490 [amdgpu] [149492.279445] amdgpu_device_fini+0x58/0x1b0 [amdgpu] [149492.279461] amdgpu_driver_unload_kms+0x4f/0xa0 [amdgpu] [149492.279470] drm_dev_unregister+0x3c/0xe0 [drm] [149492.279485] amdgpu_pci_remove+0x19/0x30 [amdgpu] [149492.279487] pci_device_remove+0x39/0xc0 [149492.279490] device_release_driver_internal+0x155/0x210 [149492.279491] driver_detach+0x38/0x70 [149492.279493] bus_remove_driver+0x4c/0xa0 [149492.279494] driver_unregister+0x2c/0x40 [149492.279496] pci_unregister_driver+0x21/0x90 [149492.279520] amdgpu_exit+0x15/0x406 [amdgpu] [149492.279523] SyS_delete_module+0x1a8/0x270 [149492.279525] ? exit_to_usermode_loop+0x92/0xa0 [149492.279528] entry_SYSCALL_64_fastpath+0x13/0x94 [149492.279529] RIP: 0033:0x7fc53fcb68e7 [149492.279529] RSP: 002b:00007ffcfbfaabb8 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0 [149492.279531] RAX: ffffffffffffffda RBX: 0000563117adb200 RCX: 00007fc53fcb68e7 [149492.279531] RDX: 000000000000000a RSI: 0000000000000800 RDI: 0000563117adb268 [149492.279532] RBP: 0000000000000003 R08: 0000000000000000 R09: 1999999999999999 [149492.279533] R10: 0000000000000883 R11: 0000000000000206 R12: 00007ffcfbfa9ba0 [149492.279533] R13: 0000000000000000 R14: 0000000000000000 R15: 0000563117adb200 [149492.279534] Code: 55 48 89 e5 e8 77 fe ff ff 84 c0 74 02 5d c3 80 3d 40 f2 a4 00 00 75 f5 48 c7 c7 20 3c ca 81 c6 05 30 f2 a4 00 01 e8 91 f0 d7 ff <0f> ff 5d c3 90 55 48 89 fe bf 01 00 00 00 48 89 e5 e8 9f fe ff [149492.279557] ---[ end trace 2d4e0ffcb66a1016 ]---
Unref the fence *after* waiting for it.
v2: Set man->move to NULL after dropping the last ref (Christian König)
Fixes: aff98ba1fdb8 (drm/ttm: wait for eviction in ttm_bo_force_list_clean) Signed-off-by: John Brooks john@fastquake.com Reviewed-by: Christian König christian.koenig@amd.com Reviewed-by: Alex Deucher alexander.deucher@amd.com --- drivers/gpu/drm/ttm/ttm_bo.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index a6d7fcb..22b5702 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -1353,7 +1353,6 @@ int ttm_bo_clean_mm(struct ttm_bo_device *bdev, unsigned mem_type) mem_type); return ret; } - dma_fence_put(man->move);
man->use_type = false; man->has_type = false; @@ -1369,6 +1368,9 @@ int ttm_bo_clean_mm(struct ttm_bo_device *bdev, unsigned mem_type) ret = (*man->func->takedown)(man); }
+ dma_fence_put(man->move); + man->move = NULL; + return ret; } EXPORT_SYMBOL(ttm_bo_clean_mm);
amd_powerplay_destroy() expects a handle pointing to a struct pp_instance. On chips without PowerPlay, pp_handle points to a struct amdgpu_device. The resulting attempt to kfree() fields of the wrong struct ends in fire:
[ 91.560405] BUG: unable to handle kernel paging request at ffffebe000000620 [ 91.560414] IP: kfree+0x57/0x160 [ 91.560416] PGD 0 [ 91.560416] P4D 0
[ 91.560420] Oops: 0000 [#1] SMP [ 91.560422] Modules linked in: tun x86_pkg_temp_thermal crc32_pclmul ghash_clmulni_intel efivarfs amdgpu(-) i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm [ 91.560438] CPU: 6 PID: 3598 Comm: rmmod Not tainted 4.12.0-rc5-drm-next-4.13-ttmpatch+ #1 [ 91.560443] Hardware name: Gigabyte Technology Co., Ltd. Z97X-UD3H-BK/Z97X-UD3H-BK-CF, BIOS F6 06/17/2014 [ 91.560448] task: ffff8805063d6a00 task.stack: ffffc90003400000 [ 91.560451] RIP: 0010:kfree+0x57/0x160 [ 91.560454] RSP: 0018:ffffc90003403cc0 EFLAGS: 00010286 [ 91.560457] RAX: 000077ff80000000 RBX: 00000000000186a0 RCX: 0000000180400035 [ 91.560460] RDX: 0000000180400036 RSI: ffffea001418e740 RDI: ffffea0000000000 [ 91.560463] RBP: ffffc90003403cd8 R08: 000000000639d201 R09: 0000000180400035 [ 91.560467] R10: ffffebe000000600 R11: 0000000000000300 R12: ffff880500530030 [ 91.560470] R13: ffffffffa01e70fc R14: 00000000ffffffff R15: ffff880500530000 [ 91.560473] FS: 00007f7e500c3700(0000) GS:ffff88051ed80000(0000) knlGS:0000000000000000 [ 91.560478] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 91.560480] CR2: ffffebe000000620 CR3: 0000000503103000 CR4: 00000000001406e0 [ 91.560483] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 91.560487] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 91.560489] Call Trace: [ 91.560530] amd_powerplay_destroy+0x1c/0x60 [amdgpu] [ 91.560558] amdgpu_pp_late_fini+0x44/0x60 [amdgpu] [ 91.560575] amdgpu_fini+0x254/0x490 [amdgpu] [ 91.560593] amdgpu_device_fini+0x58/0x1b0 [amdgpu] [ 91.560610] amdgpu_driver_unload_kms+0x4f/0xa0 [amdgpu] [ 91.560622] drm_dev_unregister+0x3c/0xe0 [drm] [ 91.560638] amdgpu_pci_remove+0x19/0x30 [amdgpu] [ 91.560643] pci_device_remove+0x39/0xc0 [ 91.560648] device_release_driver_internal+0x155/0x210 [ 91.560651] driver_detach+0x38/0x70 [ 91.560655] bus_remove_driver+0x4c/0xa0 [ 91.560658] driver_unregister+0x2c/0x40 [ 91.560662] pci_unregister_driver+0x21/0x90 [ 91.560689] amdgpu_exit+0x15/0x406 [amdgpu] [ 91.560694] SyS_delete_module+0x1a8/0x270 [ 91.560698] ? exit_to_usermode_loop+0x92/0xa0 [ 91.560702] entry_SYSCALL_64_fastpath+0x13/0x94 [ 91.560705] RIP: 0033:0x7f7e4fc118e7 [ 91.560708] RSP: 002b:00007fff978ca118 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0 [ 91.560713] RAX: ffffffffffffffda RBX: 000055afe21bc200 RCX: 00007f7e4fc118e7 [ 91.560716] RDX: 000000000000000a RSI: 0000000000000800 RDI: 000055afe21bc268 [ 91.560719] RBP: 0000000000000003 R08: 0000000000000000 R09: 1999999999999999 [ 91.560722] R10: 0000000000000883 R11: 0000000000000206 R12: 00007fff978c9100 [ 91.560725] R13: 0000000000000000 R14: 0000000000000000 R15: 000055afe21bc200 [ 91.560728] Code: 00 00 00 80 ff 77 00 00 48 bf 00 00 00 00 00 ea ff ff 49 01 da 48 0f 42 05 57 33 bd 00 49 01 c2 49 c1 ea 0c 49 c1 e2 06 49 01 fa <49> 8b 42 20 48 8d 78 ff a8 01 4c 0f 45 d7 49 8b 52 20 48 8d 42 [ 91.560759] RIP: kfree+0x57/0x160 RSP: ffffc90003403cc0 [ 91.560761] CR2: ffffebe000000620 [ 91.560765] ---[ end trace 08a9f3cd82223c1d ]---
Fixes: 1c8638024846 (drm/amd/powerplay: refine powerplay interface.) Signed-off-by: John Brooks john@fastquake.com Acked-by: Christian König christian.koenig@amd.com Reviewed-by: Alex Deucher alexander.deucher@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu_powerplay.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_powerplay.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_powerplay.c index 72c03c7..93ffb85 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_powerplay.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_powerplay.c @@ -209,7 +209,8 @@ static void amdgpu_pp_late_fini(void *handle) if (adev->pp_enabled && adev->pm.dpm_enabled) amdgpu_pm_sysfs_fini(adev);
- amd_powerplay_destroy(adev->powerplay.pp_handle); + if (adev->pp_enabled) + amd_powerplay_destroy(adev->powerplay.pp_handle); }
static int amdgpu_pp_suspend(void *handle)
I've gone ahead and pushed the two to our amd-staging-4.11 branch.
Alex will certainly pick that up for his next fixes pull request.
Thanks for the help, Christian.
Am 03.07.2017 um 20:05 schrieb John Brooks:
We unref the man->move fence in ttm_bo_clean_mm() and then call ttm_bo_force_list_clean() which waits on it, except the refcount is now zero so a warning is generated (or worse):
[149492.279301] refcount_t: increment on 0; use-after-free. [149492.279309] ------------[ cut here ]------------ [149492.279315] WARNING: CPU: 3 PID: 18726 at lib/refcount.c:150 refcount_inc+0x2b/0x30 [149492.279315] Modules linked in: vhost_net vhost tun x86_pkg_temp_thermal crc32_pclmul ghash_clmulni_intel efivarfs amdgpu( -) i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm [149492.279326] CPU: 3 PID: 18726 Comm: rmmod Not tainted 4.12.0-rc5-drm-next-4.13-ttmpatch+ #1 [149492.279326] Hardware name: Gigabyte Technology Co., Ltd. Z97X-UD3H-BK/Z97X-UD3H-BK-CF, BIOS F6 06/17/2014 [149492.279327] task: ffff8804ddfedcc0 task.stack: ffffc90008d20000 [149492.279329] RIP: 0010:refcount_inc+0x2b/0x30 [149492.279330] RSP: 0018:ffffc90008d23c30 EFLAGS: 00010286 [149492.279331] RAX: 000000000000002b RBX: 0000000000000170 RCX: 0000000000000000 [149492.279331] RDX: 0000000000000000 RSI: ffff88051ecccbe8 RDI: ffff88051ecccbe8 [149492.279332] RBP: ffffc90008d23c30 R08: 0000000000000001 R09: 00000000000003ee [149492.279333] R10: ffffc90008d23bb0 R11: 00000000000003ee R12: ffff88043aaac960 [149492.279333] R13: ffff8805005e28a8 R14: 0000000000000002 R15: ffff88050115e178 [149492.279334] FS: 00007fc540168700(0000) GS:ffff88051ecc0000(0000) knlGS:0000000000000000 [149492.279335] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [149492.279336] CR2: 00007fc3e8654140 CR3: 000000027ba77000 CR4: 00000000001426e0 [149492.279337] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [149492.279337] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [149492.279338] Call Trace: [149492.279345] ttm_bo_force_list_clean+0xb9/0x110 [ttm] [149492.279348] ttm_bo_clean_mm+0x7a/0xe0 [ttm] [149492.279375] amdgpu_ttm_fini+0xc9/0x1f0 [amdgpu] [149492.279392] amdgpu_bo_fini+0x12/0x40 [amdgpu] [149492.279415] gmc_v7_0_sw_fini+0x32/0x40 [amdgpu] [149492.279430] amdgpu_fini+0x2c9/0x490 [amdgpu] [149492.279445] amdgpu_device_fini+0x58/0x1b0 [amdgpu] [149492.279461] amdgpu_driver_unload_kms+0x4f/0xa0 [amdgpu] [149492.279470] drm_dev_unregister+0x3c/0xe0 [drm] [149492.279485] amdgpu_pci_remove+0x19/0x30 [amdgpu] [149492.279487] pci_device_remove+0x39/0xc0 [149492.279490] device_release_driver_internal+0x155/0x210 [149492.279491] driver_detach+0x38/0x70 [149492.279493] bus_remove_driver+0x4c/0xa0 [149492.279494] driver_unregister+0x2c/0x40 [149492.279496] pci_unregister_driver+0x21/0x90 [149492.279520] amdgpu_exit+0x15/0x406 [amdgpu] [149492.279523] SyS_delete_module+0x1a8/0x270 [149492.279525] ? exit_to_usermode_loop+0x92/0xa0 [149492.279528] entry_SYSCALL_64_fastpath+0x13/0x94 [149492.279529] RIP: 0033:0x7fc53fcb68e7 [149492.279529] RSP: 002b:00007ffcfbfaabb8 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0 [149492.279531] RAX: ffffffffffffffda RBX: 0000563117adb200 RCX: 00007fc53fcb68e7 [149492.279531] RDX: 000000000000000a RSI: 0000000000000800 RDI: 0000563117adb268 [149492.279532] RBP: 0000000000000003 R08: 0000000000000000 R09: 1999999999999999 [149492.279533] R10: 0000000000000883 R11: 0000000000000206 R12: 00007ffcfbfa9ba0 [149492.279533] R13: 0000000000000000 R14: 0000000000000000 R15: 0000563117adb200 [149492.279534] Code: 55 48 89 e5 e8 77 fe ff ff 84 c0 74 02 5d c3 80 3d 40 f2 a4 00 00 75 f5 48 c7 c7 20 3c ca 81 c6 05 30 f2 a4 00 01 e8 91 f0 d7 ff <0f> ff 5d c3 90 55 48 89 fe bf 01 00 00 00 48 89 e5 e8 9f fe ff [149492.279557] ---[ end trace 2d4e0ffcb66a1016 ]---
Unref the fence *after* waiting for it.
v2: Set man->move to NULL after dropping the last ref (Christian König)
Fixes: aff98ba1fdb8 (drm/ttm: wait for eviction in ttm_bo_force_list_clean) Signed-off-by: John Brooks john@fastquake.com Reviewed-by: Christian König christian.koenig@amd.com Reviewed-by: Alex Deucher alexander.deucher@amd.com
drivers/gpu/drm/ttm/ttm_bo.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index a6d7fcb..22b5702 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -1353,7 +1353,6 @@ int ttm_bo_clean_mm(struct ttm_bo_device *bdev, unsigned mem_type) mem_type); return ret; }
dma_fence_put(man->move);
man->use_type = false; man->has_type = false;
@@ -1369,6 +1368,9 @@ int ttm_bo_clean_mm(struct ttm_bo_device *bdev, unsigned mem_type) ret = (*man->func->takedown)(man); }
- dma_fence_put(man->move);
- man->move = NULL;
- return ret; } EXPORT_SYMBOL(ttm_bo_clean_mm);
dri-devel@lists.freedesktop.org
-
Christian König -
John Brooks