Until now extracting a card either by physical extraction (e.g. eGPU with thunderbolt connection or by emulation through syfs -> /sys/bus/pci/devices/device_id/remove) would cause random crashes in user apps. The random crashes in apps were mostly due to the app having mapped a device backed BO into its address space was still trying to access the BO while the backing device was gone. To answer this first problem Christian suggested to fix the handling of mapped memory in the clients when the device goes away by forcibly unmap all buffers the user processes has by clearing their respective VMAs mapping the device BOs. Then when the VMAs try to fill in the page tables again we check in the fault handlerif the device is removed and if so, return an error. This will generate a SIGBUS to the application which can then cleanly terminate.This indeed was done but this in turn created a problem of kernel OOPs were the OOPSes were due to the fact that while the app was terminating because of the SIGBUSit would trigger use after free in the driver by calling to accesses device structures that were already released from the pci remove sequence.This was handled by introducing a 'flush' sequence during device removal were we wait for drm file reference to drop to 0 meaning all user clients directly using this device terminated.
v2: Based on discussions in the mailing list with Daniel and Pekka [1] and based on the document produced by Pekka from those discussions [2] the whole approach with returning SIGBUS and waiting for all user clients having CPU mapping of device BOs to die was dropped. Instead as per the document suggestion the device structures are kept alive until the last reference to the device is dropped by user client and in the meanwhile all existing and new CPU mappings of the BOs belonging to the device directly or by dma-buf import are rerouted to per user process dummy rw page.Also, I skipped the 'Requirements for KMS UAPI' section of [2] since i am trying to get the minimal set of requirements that still give useful solution to work and this is the'Requirements for Render and Cross-Device UAPI' section and so my test case is removing a secondary device, which is render only and is not involved in KMS.
v3: More updates following comments from v2 such as removing loop to find DRM file when rerouting page faults to dummy page,getting rid of unnecessary sysfs handling refactoring and moving prevention of GPU recovery post device unplug from amdgpu to scheduler layer. On top of that added unplug support for the IOMMU enabled system.
v4: Drop last sysfs hack and use sysfs default attribute. Guard against write accesses after device removal to avoid modifying released memory. Update dummy pages handling to on demand allocation and release through drm managed framework. Add return value to scheduler job TO handler (by Luben Tuikov) and use this in amdgpu for prevention of GPU recovery post device unplug Also rebase on top of drm-misc-mext instead of amd-staging-drm-next
v5: The most significant in this series is the improved protection from kernel driver accessing MMIO ranges that were allocated for the device once the device is gone. To do this, first a patch 'drm/amdgpu: Unmap all MMIO mappings' is introduced. This patch unamps all MMIO mapped into the kernel address space in the form of BARs and kernel BOs with CPU visible VRAM mappings. This way it helped to discover multiple such access points because a page fault would be immediately generated on access. Most of them were solved by moving HW fini code into pci_remove stage (patch drm/amdgpu: Add early fini callback) and for some who were harder to unwind drm_dev_enter/exit scoping was used. In addition all the IOCTLs and all background work and timers are now protected with drm_dev_enter/exit at their root in an attempt that after drm_dev_unplug is finished none of them run anymore and the pci_remove thread is the only thread executing which might touch the HW. To prevent deadlocks in such case against threads stuck on various HW or SW fences patches 'drm/amdgpu: Finalise device fences on device remove' and drm/amdgpu: Add rw_sem to pushing job into sched queue' take care of force signaling all such existing fences and rejecting any newly added ones.
With these patches I am able to gracefully remove the secondary card using sysfs remove hook while glxgears is running off of secondary card (DRI_PRIME=1) without kernel oopses or hangs and keep working with the primary card or soft reset the device without hangs or oopses. Also as per Daniel's comment I added 3 tests to IGT [4] to core_hotunplug test suite - remove device while commands are submitted, exported BO and exported fence (not pushed yet). Also now it's possible to plug back the device after unplug Also some users now can successfully use those patches with eGPU boxes[3].
TODOs for followup work: Convert AMDGPU code to use devm (for hw stuff) and drmm (for sw stuff and allocations) (Daniel) Add support for 'Requirements for KMS UAPI' section of [2] - unplugging primary, display connected card.
[1] - Discussions during v4 of the patchset https://lists.freedesktop.org/archives/amd-gfx/2021-January/058595.html [2] - drm/doc: device hot-unplug for userspace https://www.spinics.net/lists/dri-devel/msg259755.html [3] - Related gitlab ticket https://gitlab.freedesktop.org/drm/amd/-/issues/1081 [4] - https://gitlab.freedesktop.org/agrodzov/igt-gpu-tools/-/commits/master
Andrey Grodzovsky (27): drm/ttm: Remap all page faults to per process dummy page. drm/ttm: Expose ttm_tt_unpopulate for driver use drm/amdgpu: Split amdgpu_device_fini into early and late drm/amdkfd: Split kfd suspend from devie exit drm/amdgpu: Add early fini callback drm/amdgpu: Handle IOMMU enabled case. drm/amdgpu: Remap all page faults to per process dummy page. PCI: add support for dev_groups to struct pci_device_driver dmr/amdgpu: Move some sysfs attrs creation to default_attr drm/amdgpu: Guard against write accesses after device removal drm/sched: Make timeout timer rearm conditional. drm/amdgpu: Prevent any job recoveries after device is unplugged. drm/amdgpu: When filizing the fence driver. stop scheduler first. drm/amdgpu: Fix hang on device removal. drm/scheduler: Fix hang when sched_entity released drm/amdgpu: Unmap all MMIO mappings drm/amdgpu: Add rw_sem to pushing job into sched queue drm/sched: Expose drm_sched_entity_kill_jobs drm/amdgpu: Finilise device fences on device remove. drm: Scope all DRM IOCTLs with drm_dev_enter/exit drm/amdgpu: Add support for hot-unplug feature at DRM level. drm/amd/display: Scope all DM queued work with drm_dev_enter/exit drm/amd/powerplay: Scope all PM queued work with drm_dev_enter/exit drm/amdkfd: Scope all KFD queued work with drm_dev_enter/exit drm/amdgpu: Scope all amdgpu queued work with drm_dev_enter/exit drm/amd/display: Remove superflous drm_mode_config_cleanup drm/amdgpu: Verify DMA opearations from device are done
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 18 +- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 13 +- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c | 17 +- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 13 +- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 353 ++++++++++++++---- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 34 +- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 34 +- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 9 + drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | 25 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c | 228 +++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 61 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 33 +- drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c | 28 +- drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 12 +- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 41 +- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 7 + drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c | 115 +++--- drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h | 3 + drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 56 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 70 ++++ drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 52 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 21 +- drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c | 74 ++-- drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c | 45 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c | 83 ++-- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 14 +- drivers/gpu/drm/amd/amdgpu/cik_ih.c | 3 +- drivers/gpu/drm/amd/amdgpu/cz_ih.c | 3 +- drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c | 10 +- drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 3 +- drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 5 +- drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 44 +-- drivers/gpu/drm/amd/amdgpu/psp_v12_0.c | 8 +- drivers/gpu/drm/amd/amdgpu/psp_v3_1.c | 8 +- drivers/gpu/drm/amd/amdgpu/si_ih.c | 3 +- drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 3 +- drivers/gpu/drm/amd/amdgpu/vce_v4_0.c | 26 +- drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c | 22 +- drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 5 +- drivers/gpu/drm/amd/amdgpu/vega20_ih.c | 2 +- drivers/gpu/drm/amd/amdkfd/kfd_device.c | 3 +- drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c | 14 +- .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 13 +- .../amd/display/amdgpu_dm/amdgpu_dm_hdcp.c | 124 +++--- .../drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c | 24 +- drivers/gpu/drm/amd/include/amd_shared.h | 2 + drivers/gpu/drm/amd/pm/amdgpu_dpm.c | 44 ++- .../drm/amd/pm/powerplay/smumgr/smu7_smumgr.c | 2 + drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c | 26 +- drivers/gpu/drm/drm_ioctl.c | 15 +- drivers/gpu/drm/scheduler/sched_entity.c | 6 +- drivers/gpu/drm/scheduler/sched_main.c | 35 +- drivers/gpu/drm/ttm/ttm_bo_vm.c | 79 +++- drivers/gpu/drm/ttm/ttm_tt.c | 1 + drivers/pci/pci-driver.c | 1 + include/drm/drm_drv.h | 6 + include/drm/gpu_scheduler.h | 1 + include/drm/ttm/ttm_bo_api.h | 2 + include/linux/pci.h | 3 + 64 files changed, 1388 insertions(+), 633 deletions(-)
On device removal reroute all CPU mappings to dummy page.
v3: Remove loop to find DRM file and instead access it by vma->vm_file->private_data. Move dummy page installation into a separate function.
v4: Map the entire BOs VA space into on demand allocated dummy page on the first fault for that BO.
v5: Remove duplicate return.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/ttm/ttm_bo_vm.c | 79 ++++++++++++++++++++++++++++++++- include/drm/ttm/ttm_bo_api.h | 2 + 2 files changed, 80 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c index b31b18058965..8b8300551a7f 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_vm.c +++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c @@ -34,6 +34,8 @@ #include <drm/ttm/ttm_bo_driver.h> #include <drm/ttm/ttm_placement.h> #include <drm/drm_vma_manager.h> +#include <drm/drm_drv.h> +#include <drm/drm_managed.h> #include <linux/mm.h> #include <linux/pfn_t.h> #include <linux/rbtree.h> @@ -380,19 +382,94 @@ vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf, } EXPORT_SYMBOL(ttm_bo_vm_fault_reserved);
+static void ttm_bo_release_dummy_page(struct drm_device *dev, void *res) +{ + struct page *dummy_page = (struct page *)res; + + __free_page(dummy_page); +} + +vm_fault_t ttm_bo_vm_dummy_page(struct vm_fault *vmf, pgprot_t prot) +{ + struct vm_area_struct *vma = vmf->vma; + struct ttm_buffer_object *bo = vma->vm_private_data; + struct drm_device *ddev = bo->base.dev; + vm_fault_t ret = VM_FAULT_NOPAGE; + unsigned long address = vma->vm_start; + unsigned long num_prefault = (vma->vm_end - vma->vm_start) >> PAGE_SHIFT; + unsigned long pfn; + struct page *page; + int i; + + /* + * Wait for buffer data in transit, due to a pipelined + * move. + */ + ret = ttm_bo_vm_fault_idle(bo, vmf); + if (unlikely(ret != 0)) + return ret; + + /* Allocate new dummy page to map all the VA range in this VMA to it*/ + page = alloc_page(GFP_KERNEL | __GFP_ZERO); + if (!page) + return VM_FAULT_OOM; + + pfn = page_to_pfn(page); + + /* + * Prefault the entire VMA range right away to avoid further faults + */ + for (i = 0; i < num_prefault; ++i) { + + if (unlikely(address >= vma->vm_end)) + break; + + if (vma->vm_flags & VM_MIXEDMAP) + ret = vmf_insert_mixed_prot(vma, address, + __pfn_to_pfn_t(pfn, PFN_DEV), + prot); + else + ret = vmf_insert_pfn_prot(vma, address, pfn, prot); + + /* Never error on prefaulted PTEs */ + if (unlikely((ret & VM_FAULT_ERROR))) { + if (i == 0) + return VM_FAULT_NOPAGE; + else + break; + } + + address += PAGE_SIZE; + } + + /* Set the page to be freed using drmm release action */ + if (drmm_add_action_or_reset(ddev, ttm_bo_release_dummy_page, page)) + return VM_FAULT_OOM; + + return ret; +} +EXPORT_SYMBOL(ttm_bo_vm_dummy_page); + vm_fault_t ttm_bo_vm_fault(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; pgprot_t prot; struct ttm_buffer_object *bo = vma->vm_private_data; + struct drm_device *ddev = bo->base.dev; vm_fault_t ret; + int idx;
ret = ttm_bo_vm_reserve(bo, vmf); if (ret) return ret;
prot = vma->vm_page_prot; - ret = ttm_bo_vm_fault_reserved(vmf, prot, TTM_BO_VM_NUM_PREFAULT, 1); + if (drm_dev_enter(ddev, &idx)) { + ret = ttm_bo_vm_fault_reserved(vmf, prot, TTM_BO_VM_NUM_PREFAULT, 1); + drm_dev_exit(idx); + } else { + ret = ttm_bo_vm_dummy_page(vmf, prot); + } if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) return ret;
diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h index 3587f660e8f4..dbb00e495cb4 100644 --- a/include/drm/ttm/ttm_bo_api.h +++ b/include/drm/ttm/ttm_bo_api.h @@ -635,4 +635,6 @@ int ttm_bo_vm_access(struct vm_area_struct *vma, unsigned long addr, void *buf, int len, int write); bool ttm_bo_delayed_delete(struct ttm_device *bdev, bool remove_all);
+vm_fault_t ttm_bo_vm_dummy_page(struct vm_fault *vmf, pgprot_t prot); + #endif
It's needed to drop iommu backed pages on device unplug before device's IOMMU group is released.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/ttm/ttm_tt.c | 1 + 1 file changed, 1 insertion(+)
diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c index 48c407cff112..f2ce1b372096 100644 --- a/drivers/gpu/drm/ttm/ttm_tt.c +++ b/drivers/gpu/drm/ttm/ttm_tt.c @@ -431,3 +431,4 @@ void ttm_tt_mgr_init(unsigned long num_pages, unsigned long num_dma32_pages) if (!ttm_dma32_pages_limit) ttm_dma32_pages_limit = num_dma32_pages; } +EXPORT_SYMBOL(ttm_tt_unpopulate);
Some of the stuff in amdgpu_device_fini such as HW interrupts disable and pending fences finilization must be done right away on pci_remove while most of the stuff which relates to finilizing and releasing driver data structures can be kept until drm_driver.release hook is called, i.e. when the last device reference is dropped.
v4: Change functions prefix early->hw and late->sw
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 6 ++++- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 +++++++++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 7 ++---- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 15 ++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 26 +++++++++++++--------- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h | 3 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 12 +++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 3 ++- drivers/gpu/drm/amd/amdgpu/cik_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/cz_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/si_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/vega20_ih.c | 2 +- 17 files changed, 79 insertions(+), 36 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 1af2fa1591fd..fddb82897e5d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1073,7 +1073,9 @@ static inline struct amdgpu_device *amdgpu_ttm_adev(struct ttm_device *bdev)
int amdgpu_device_init(struct amdgpu_device *adev, uint32_t flags); -void amdgpu_device_fini(struct amdgpu_device *adev); +void amdgpu_device_fini_hw(struct amdgpu_device *adev); +void amdgpu_device_fini_sw(struct amdgpu_device *adev); + int amdgpu_gpu_wait_for_idle(struct amdgpu_device *adev);
void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, @@ -1289,6 +1291,8 @@ void amdgpu_driver_lastclose_kms(struct drm_device *dev); int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv); void amdgpu_driver_postclose_kms(struct drm_device *dev, struct drm_file *file_priv); +void amdgpu_driver_release_kms(struct drm_device *dev); + int amdgpu_device_ip_suspend(struct amdgpu_device *adev); int amdgpu_device_suspend(struct drm_device *dev, bool fbcon); int amdgpu_device_resume(struct drm_device *dev, bool fbcon); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 6447cd6ca5a8..8d22b79fc1cd 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3590,14 +3590,12 @@ int amdgpu_device_init(struct amdgpu_device *adev, * Tear down the driver info (all asics). * Called at driver shutdown. */ -void amdgpu_device_fini(struct amdgpu_device *adev) +void amdgpu_device_fini_hw(struct amdgpu_device *adev) { dev_info(adev->dev, "amdgpu: finishing device.\n"); flush_delayed_work(&adev->delayed_init_work); adev->shutdown = true;
- kfree(adev->pci_state); - /* make sure IB test finished before entering exclusive mode * to avoid preemption on IB test * */ @@ -3614,11 +3612,24 @@ void amdgpu_device_fini(struct amdgpu_device *adev) else drm_atomic_helper_shutdown(adev_to_drm(adev)); } - amdgpu_fence_driver_fini(adev); + amdgpu_fence_driver_fini_hw(adev); + if (adev->pm_sysfs_en) amdgpu_pm_sysfs_fini(adev); + if (adev->ucode_sysfs_en) + amdgpu_ucode_sysfs_fini(adev); + sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); + + amdgpu_fbdev_fini(adev); + + amdgpu_irq_fini_hw(adev); +} + +void amdgpu_device_fini_sw(struct amdgpu_device *adev) +{ amdgpu_device_ip_fini(adev); + amdgpu_fence_driver_fini_sw(adev); release_firmware(adev->firmware.gpu_info_fw); adev->firmware.gpu_info_fw = NULL; adev->accel_working = false; @@ -3647,14 +3658,13 @@ void amdgpu_device_fini(struct amdgpu_device *adev) adev->rmmio = NULL; amdgpu_device_doorbell_fini(adev);
- if (adev->ucode_sysfs_en) - amdgpu_ucode_sysfs_fini(adev); - - sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); if (IS_ENABLED(CONFIG_PERF_EVENTS)) amdgpu_pmu_fini(adev); if (adev->mman.discovery_bin) amdgpu_discovery_fini(adev); + + kfree(adev->pci_state); + }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 671ec1002230..54cb5ee2f563 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1249,14 +1249,10 @@ amdgpu_pci_remove(struct pci_dev *pdev) { struct drm_device *dev = pci_get_drvdata(pdev);
-#ifdef MODULE - if (THIS_MODULE->state != MODULE_STATE_GOING) -#endif - DRM_ERROR("Hotplug removal is not supported\n"); drm_dev_unplug(dev); amdgpu_driver_unload_kms(dev); + pci_disable_device(pdev); - pci_set_drvdata(pdev, NULL); }
static void @@ -1587,6 +1583,7 @@ static const struct drm_driver amdgpu_kms_driver = { .dumb_create = amdgpu_mode_dumb_create, .dumb_map_offset = amdgpu_mode_dumb_mmap, .fops = &amdgpu_driver_kms_fops, + .release = &amdgpu_driver_release_kms,
.prime_handle_to_fd = drm_gem_prime_handle_to_fd, .prime_fd_to_handle = drm_gem_prime_fd_to_handle, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 8e0a5650d383..34d51e962799 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -523,7 +523,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev) * * Tear down the fence driver for all possible rings (all asics). */ -void amdgpu_fence_driver_fini(struct amdgpu_device *adev) +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) { unsigned i, j; int r; @@ -544,6 +544,19 @@ void amdgpu_fence_driver_fini(struct amdgpu_device *adev) if (!ring->no_scheduler) drm_sched_fini(&ring->sched); del_timer_sync(&ring->fence_drv.fallback_timer); + } +} + +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev) +{ + unsigned int i, j; + + for (i = 0; i < AMDGPU_MAX_RINGS; i++) { + struct amdgpu_ring *ring = adev->rings[i]; + + if (!ring || !ring->fence_drv.initialized) + continue; + for (j = 0; j <= ring->fence_drv.num_fences_mask; ++j) dma_fence_put(ring->fence_drv.fences[j]); kfree(ring->fence_drv.fences); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index afbbec82a289..63e815c27585 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -49,6 +49,7 @@ #include <drm/drm_irq.h> #include <drm/drm_vblank.h> #include <drm/amdgpu_drm.h> +#include <drm/drm_drv.h> #include "amdgpu.h" #include "amdgpu_ih.h" #include "atom.h" @@ -313,6 +314,20 @@ int amdgpu_irq_init(struct amdgpu_device *adev) return 0; }
+ +void amdgpu_irq_fini_hw(struct amdgpu_device *adev) +{ + if (adev->irq.installed) { + drm_irq_uninstall(&adev->ddev); + adev->irq.installed = false; + if (adev->irq.msi_enabled) + pci_free_irq_vectors(adev->pdev); + + if (!amdgpu_device_has_dc_support(adev)) + flush_work(&adev->hotplug_work); + } +} + /** * amdgpu_irq_fini - shut down interrupt handling * @@ -322,19 +337,10 @@ int amdgpu_irq_init(struct amdgpu_device *adev) * functionality, shuts down vblank, hotplug and reset interrupt handling, * turns off interrupts from all sources (all ASICs). */ -void amdgpu_irq_fini(struct amdgpu_device *adev) +void amdgpu_irq_fini_sw(struct amdgpu_device *adev) { unsigned i, j;
- if (adev->irq.installed) { - drm_irq_uninstall(adev_to_drm(adev)); - adev->irq.installed = false; - if (adev->irq.msi_enabled) - pci_free_irq_vectors(adev->pdev); - if (!amdgpu_device_has_dc_support(adev)) - flush_work(&adev->hotplug_work); - } - for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if (!adev->irq.client[i].sources) continue; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h index ac527e5deae6..392a7324e2b1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h @@ -104,7 +104,8 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev); irqreturn_t amdgpu_irq_handler(int irq, void *arg);
int amdgpu_irq_init(struct amdgpu_device *adev); -void amdgpu_irq_fini(struct amdgpu_device *adev); +void amdgpu_irq_fini_sw(struct amdgpu_device *adev); +void amdgpu_irq_fini_hw(struct amdgpu_device *adev); int amdgpu_irq_add_id(struct amdgpu_device *adev, unsigned client_id, unsigned src_id, struct amdgpu_irq_src *source); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c index 64beb3399604..1af3fba7bfd4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c @@ -29,6 +29,7 @@ #include "amdgpu.h" #include <drm/drm_debugfs.h> #include <drm/amdgpu_drm.h> +#include <drm/drm_drv.h> #include "amdgpu_uvd.h" #include "amdgpu_vce.h" #include "atom.h" @@ -93,7 +94,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev) }
amdgpu_acpi_fini(adev); - amdgpu_device_fini(adev); + amdgpu_device_fini_hw(adev); }
void amdgpu_register_gpu_instance(struct amdgpu_device *adev) @@ -1151,6 +1152,15 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev, pm_runtime_put_autosuspend(dev->dev); }
+ +void amdgpu_driver_release_kms(struct drm_device *dev) +{ + struct amdgpu_device *adev = drm_to_adev(dev); + + amdgpu_device_fini_sw(adev); + pci_set_drvdata(adev->pdev, NULL); +} + /* * VBlank related functions. */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c index 1fb2a91ad30a..c0a16eac4923 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c @@ -2142,6 +2142,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev) if (!con) return 0;
+ /* Need disable ras on all IPs here before ip [hw/sw]fini */ amdgpu_ras_disable_all_features(adev, 0); amdgpu_ras_recovery_fini(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h index 56acec1075ac..0f195f7bf797 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h @@ -107,7 +107,8 @@ struct amdgpu_fence_driver { };
int amdgpu_fence_driver_init(struct amdgpu_device *adev); -void amdgpu_fence_driver_fini(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev); void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring);
int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index d3745711d55f..183d44a6583c 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -309,7 +309,7 @@ static int cik_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index 307c01301c87..d32743949003 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -301,7 +301,7 @@ static int cz_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index cc957471f31e..da96c6013477 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -300,7 +300,7 @@ static int iceland_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index f4e4040bbd25..5eea4550b856 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -569,7 +569,7 @@ static int navi10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 51880f6ef634..751307f3252c 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -175,7 +175,7 @@ static int si_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih);
return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 249fcbee7871..973d80ec7f6c 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -312,7 +312,7 @@ static int tonga_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index 88626d83e07b..2d0094c276ca 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -523,7 +523,7 @@ static int vega10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c index 5a3c867d5881..9059b21b079f 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c @@ -558,7 +558,7 @@ static int vega20_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
Some of the stuff in amdgpu_device_fini such as HW interrupts disable and pending fences finilization must be done right away on pci_remove while most of the stuff which relates to finilizing and releasing driver data structures can be kept until drm_driver.release hook is called, i.e. when the last device reference is dropped.
v4: Change functions prefix early->hw and late->sw
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
Acked-by: Christian König christian.koenig@amd.com
But Alex should acknowledge this as well since it is general driver design.
Christian.
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 6 ++++- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 +++++++++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 7 ++---- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 15 ++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 26 +++++++++++++--------- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h | 3 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 12 +++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 3 ++- drivers/gpu/drm/amd/amdgpu/cik_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/cz_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/si_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/vega20_ih.c | 2 +- 17 files changed, 79 insertions(+), 36 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 1af2fa1591fd..fddb82897e5d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1073,7 +1073,9 @@ static inline struct amdgpu_device *amdgpu_ttm_adev(struct ttm_device *bdev)
int amdgpu_device_init(struct amdgpu_device *adev, uint32_t flags); -void amdgpu_device_fini(struct amdgpu_device *adev); +void amdgpu_device_fini_hw(struct amdgpu_device *adev); +void amdgpu_device_fini_sw(struct amdgpu_device *adev);
int amdgpu_gpu_wait_for_idle(struct amdgpu_device *adev);
void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos,
@@ -1289,6 +1291,8 @@ void amdgpu_driver_lastclose_kms(struct drm_device *dev); int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv); void amdgpu_driver_postclose_kms(struct drm_device *dev, struct drm_file *file_priv); +void amdgpu_driver_release_kms(struct drm_device *dev);
- int amdgpu_device_ip_suspend(struct amdgpu_device *adev); int amdgpu_device_suspend(struct drm_device *dev, bool fbcon); int amdgpu_device_resume(struct drm_device *dev, bool fbcon);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 6447cd6ca5a8..8d22b79fc1cd 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3590,14 +3590,12 @@ int amdgpu_device_init(struct amdgpu_device *adev,
- Tear down the driver info (all asics).
- Called at driver shutdown.
*/ -void amdgpu_device_fini(struct amdgpu_device *adev) +void amdgpu_device_fini_hw(struct amdgpu_device *adev) { dev_info(adev->dev, "amdgpu: finishing device.\n"); flush_delayed_work(&adev->delayed_init_work); adev->shutdown = true;
- kfree(adev->pci_state);
- /* make sure IB test finished before entering exclusive mode
- to avoid preemption on IB test
- */
@@ -3614,11 +3612,24 @@ void amdgpu_device_fini(struct amdgpu_device *adev) else drm_atomic_helper_shutdown(adev_to_drm(adev)); }
- amdgpu_fence_driver_fini(adev);
- amdgpu_fence_driver_fini_hw(adev);
- if (adev->pm_sysfs_en) amdgpu_pm_sysfs_fini(adev);
- if (adev->ucode_sysfs_en)
amdgpu_ucode_sysfs_fini(adev);- sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
- amdgpu_fbdev_fini(adev);
- amdgpu_irq_fini_hw(adev);
+}
+void amdgpu_device_fini_sw(struct amdgpu_device *adev) +{ amdgpu_device_ip_fini(adev);
- amdgpu_fence_driver_fini_sw(adev); release_firmware(adev->firmware.gpu_info_fw); adev->firmware.gpu_info_fw = NULL; adev->accel_working = false;
@@ -3647,14 +3658,13 @@ void amdgpu_device_fini(struct amdgpu_device *adev) adev->rmmio = NULL; amdgpu_device_doorbell_fini(adev);
- if (adev->ucode_sysfs_en)
amdgpu_ucode_sysfs_fini(adev);- sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); if (IS_ENABLED(CONFIG_PERF_EVENTS)) amdgpu_pmu_fini(adev); if (adev->mman.discovery_bin) amdgpu_discovery_fini(adev);
- kfree(adev->pci_state);
- }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 671ec1002230..54cb5ee2f563 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1249,14 +1249,10 @@ amdgpu_pci_remove(struct pci_dev *pdev) { struct drm_device *dev = pci_get_drvdata(pdev);
-#ifdef MODULE
- if (THIS_MODULE->state != MODULE_STATE_GOING)
-#endif
DRM_ERROR("Hotplug removal is not supported\n");
- pci_disable_device(pdev);
pci_set_drvdata(pdev, NULL); }
static void
@@ -1587,6 +1583,7 @@ static const struct drm_driver amdgpu_kms_driver = { .dumb_create = amdgpu_mode_dumb_create, .dumb_map_offset = amdgpu_mode_dumb_mmap, .fops = &amdgpu_driver_kms_fops,
.release = &amdgpu_driver_release_kms,
.prime_handle_to_fd = drm_gem_prime_handle_to_fd, .prime_fd_to_handle = drm_gem_prime_fd_to_handle,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 8e0a5650d383..34d51e962799 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -523,7 +523,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev)
- Tear down the fence driver for all possible rings (all asics).
*/ -void amdgpu_fence_driver_fini(struct amdgpu_device *adev) +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) { unsigned i, j; int r; @@ -544,6 +544,19 @@ void amdgpu_fence_driver_fini(struct amdgpu_device *adev) if (!ring->no_scheduler) drm_sched_fini(&ring->sched); del_timer_sync(&ring->fence_drv.fallback_timer);
- }
+}
+void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev) +{
- unsigned int i, j;
- for (i = 0; i < AMDGPU_MAX_RINGS; i++) {
struct amdgpu_ring *ring = adev->rings[i];if (!ring || !ring->fence_drv.initialized)continue;- for (j = 0; j <= ring->fence_drv.num_fences_mask; ++j) dma_fence_put(ring->fence_drv.fences[j]); kfree(ring->fence_drv.fences);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index afbbec82a289..63e815c27585 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -49,6 +49,7 @@ #include <drm/drm_irq.h> #include <drm/drm_vblank.h> #include <drm/amdgpu_drm.h> +#include <drm/drm_drv.h> #include "amdgpu.h" #include "amdgpu_ih.h" #include "atom.h" @@ -313,6 +314,20 @@ int amdgpu_irq_init(struct amdgpu_device *adev) return 0; }
+void amdgpu_irq_fini_hw(struct amdgpu_device *adev) +{
- if (adev->irq.installed) {
drm_irq_uninstall(&adev->ddev);adev->irq.installed = false;if (adev->irq.msi_enabled)pci_free_irq_vectors(adev->pdev);if (!amdgpu_device_has_dc_support(adev))flush_work(&adev->hotplug_work);- }
+}
- /**
- amdgpu_irq_fini - shut down interrupt handling
@@ -322,19 +337,10 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
- functionality, shuts down vblank, hotplug and reset interrupt handling,
- turns off interrupts from all sources (all ASICs).
*/ -void amdgpu_irq_fini(struct amdgpu_device *adev) +void amdgpu_irq_fini_sw(struct amdgpu_device *adev) { unsigned i, j;
- if (adev->irq.installed) {
drm_irq_uninstall(adev_to_drm(adev));adev->irq.installed = false;if (adev->irq.msi_enabled)pci_free_irq_vectors(adev->pdev);if (!amdgpu_device_has_dc_support(adev))flush_work(&adev->hotplug_work);- }
- for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if (!adev->irq.client[i].sources) continue;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h index ac527e5deae6..392a7324e2b1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h @@ -104,7 +104,8 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev); irqreturn_t amdgpu_irq_handler(int irq, void *arg);
int amdgpu_irq_init(struct amdgpu_device *adev); -void amdgpu_irq_fini(struct amdgpu_device *adev); +void amdgpu_irq_fini_sw(struct amdgpu_device *adev); +void amdgpu_irq_fini_hw(struct amdgpu_device *adev); int amdgpu_irq_add_id(struct amdgpu_device *adev, unsigned client_id, unsigned src_id, struct amdgpu_irq_src *source); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c index 64beb3399604..1af3fba7bfd4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c @@ -29,6 +29,7 @@ #include "amdgpu.h" #include <drm/drm_debugfs.h> #include <drm/amdgpu_drm.h> +#include <drm/drm_drv.h> #include "amdgpu_uvd.h" #include "amdgpu_vce.h" #include "atom.h" @@ -93,7 +94,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev) }
amdgpu_acpi_fini(adev);
- amdgpu_device_fini(adev);
amdgpu_device_fini_hw(adev); }
void amdgpu_register_gpu_instance(struct amdgpu_device *adev)
@@ -1151,6 +1152,15 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev, pm_runtime_put_autosuspend(dev->dev); }
+void amdgpu_driver_release_kms(struct drm_device *dev) +{
- struct amdgpu_device *adev = drm_to_adev(dev);
- amdgpu_device_fini_sw(adev);
- pci_set_drvdata(adev->pdev, NULL);
+}
- /*
*/
- VBlank related functions.
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c index 1fb2a91ad30a..c0a16eac4923 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c @@ -2142,6 +2142,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev) if (!con) return 0;
- /* Need disable ras on all IPs here before ip [hw/sw]fini */ amdgpu_ras_disable_all_features(adev, 0); amdgpu_ras_recovery_fini(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h index 56acec1075ac..0f195f7bf797 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h @@ -107,7 +107,8 @@ struct amdgpu_fence_driver { };
int amdgpu_fence_driver_init(struct amdgpu_device *adev); -void amdgpu_fence_driver_fini(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev); void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring);
int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index d3745711d55f..183d44a6583c 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -309,7 +309,7 @@ static int cik_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev);
- amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index 307c01301c87..d32743949003 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -301,7 +301,7 @@ static int cz_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev);
- amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index cc957471f31e..da96c6013477 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -300,7 +300,7 @@ static int iceland_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev);
- amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index f4e4040bbd25..5eea4550b856 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -569,7 +569,7 @@ static int navi10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev);
- amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 51880f6ef634..751307f3252c 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -175,7 +175,7 @@ static int si_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev);
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih);
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 249fcbee7871..973d80ec7f6c 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -312,7 +312,7 @@ static int tonga_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev);
- amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index 88626d83e07b..2d0094c276ca 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -523,7 +523,7 @@ static int vega10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev);
- amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c index 5a3c867d5881..9059b21b079f 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c @@ -558,7 +558,7 @@ static int vega20_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev);
- amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
On Thu, Apr 29, 2021 at 3:04 AM Christian König ckoenig.leichtzumerken@gmail.com wrote:
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
Some of the stuff in amdgpu_device_fini such as HW interrupts disable and pending fences finilization must be done right away on pci_remove while most of the stuff which relates to finilizing and releasing driver data structures can be kept until drm_driver.release hook is called, i.e. when the last device reference is dropped.
v4: Change functions prefix early->hw and late->sw
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
Acked-by: Christian König christian.koenig@amd.com
But Alex should acknowledge this as well since it is general driver design.
Looks good to me as well. Reviewed-by: Alex Deucher alexander.deucher@amd.com
Christian.
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 6 ++++- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 +++++++++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 7 ++---- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 15 ++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 26 +++++++++++++--------- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h | 3 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 12 +++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 3 ++- drivers/gpu/drm/amd/amdgpu/cik_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/cz_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/si_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/vega20_ih.c | 2 +- 17 files changed, 79 insertions(+), 36 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 1af2fa1591fd..fddb82897e5d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1073,7 +1073,9 @@ static inline struct amdgpu_device *amdgpu_ttm_adev(struct ttm_device *bdev)
int amdgpu_device_init(struct amdgpu_device *adev, uint32_t flags); -void amdgpu_device_fini(struct amdgpu_device *adev); +void amdgpu_device_fini_hw(struct amdgpu_device *adev); +void amdgpu_device_fini_sw(struct amdgpu_device *adev);
int amdgpu_gpu_wait_for_idle(struct amdgpu_device *adev);
void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos,
@@ -1289,6 +1291,8 @@ void amdgpu_driver_lastclose_kms(struct drm_device *dev); int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv); void amdgpu_driver_postclose_kms(struct drm_device *dev, struct drm_file *file_priv); +void amdgpu_driver_release_kms(struct drm_device *dev);
- int amdgpu_device_ip_suspend(struct amdgpu_device *adev); int amdgpu_device_suspend(struct drm_device *dev, bool fbcon); int amdgpu_device_resume(struct drm_device *dev, bool fbcon);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 6447cd6ca5a8..8d22b79fc1cd 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3590,14 +3590,12 @@ int amdgpu_device_init(struct amdgpu_device *adev,
- Tear down the driver info (all asics).
- Called at driver shutdown.
*/ -void amdgpu_device_fini(struct amdgpu_device *adev) +void amdgpu_device_fini_hw(struct amdgpu_device *adev) { dev_info(adev->dev, "amdgpu: finishing device.\n"); flush_delayed_work(&adev->delayed_init_work); adev->shutdown = true;
kfree(adev->pci_state);/* make sure IB test finished before entering exclusive mode * to avoid preemption on IB test * */@@ -3614,11 +3612,24 @@ void amdgpu_device_fini(struct amdgpu_device *adev) else drm_atomic_helper_shutdown(adev_to_drm(adev)); }
amdgpu_fence_driver_fini(adev);
amdgpu_fence_driver_fini_hw(adev);if (adev->pm_sysfs_en) amdgpu_pm_sysfs_fini(adev);if (adev->ucode_sysfs_en)amdgpu_ucode_sysfs_fini(adev);sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);amdgpu_fbdev_fini(adev);amdgpu_irq_fini_hw(adev);+}
+void amdgpu_device_fini_sw(struct amdgpu_device *adev) +{ amdgpu_device_ip_fini(adev);
amdgpu_fence_driver_fini_sw(adev); release_firmware(adev->firmware.gpu_info_fw); adev->firmware.gpu_info_fw = NULL; adev->accel_working = false;@@ -3647,14 +3658,13 @@ void amdgpu_device_fini(struct amdgpu_device *adev) adev->rmmio = NULL; amdgpu_device_doorbell_fini(adev);
if (adev->ucode_sysfs_en)amdgpu_ucode_sysfs_fini(adev);sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); if (IS_ENABLED(CONFIG_PERF_EVENTS)) amdgpu_pmu_fini(adev); if (adev->mman.discovery_bin) amdgpu_discovery_fini(adev);
kfree(adev->pci_state);- }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 671ec1002230..54cb5ee2f563 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1249,14 +1249,10 @@ amdgpu_pci_remove(struct pci_dev *pdev) { struct drm_device *dev = pci_get_drvdata(pdev);
-#ifdef MODULE
if (THIS_MODULE->state != MODULE_STATE_GOING)-#endif
DRM_ERROR("Hotplug removal is not supported\n"); drm_dev_unplug(dev); amdgpu_driver_unload_kms(dev);
pci_disable_device(pdev);
pci_set_drvdata(pdev, NULL);}
static void
@@ -1587,6 +1583,7 @@ static const struct drm_driver amdgpu_kms_driver = { .dumb_create = amdgpu_mode_dumb_create, .dumb_map_offset = amdgpu_mode_dumb_mmap, .fops = &amdgpu_driver_kms_fops,
.release = &amdgpu_driver_release_kms, .prime_handle_to_fd = drm_gem_prime_handle_to_fd, .prime_fd_to_handle = drm_gem_prime_fd_to_handle,diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 8e0a5650d383..34d51e962799 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -523,7 +523,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev)
- Tear down the fence driver for all possible rings (all asics).
*/ -void amdgpu_fence_driver_fini(struct amdgpu_device *adev) +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) { unsigned i, j; int r; @@ -544,6 +544,19 @@ void amdgpu_fence_driver_fini(struct amdgpu_device *adev) if (!ring->no_scheduler) drm_sched_fini(&ring->sched); del_timer_sync(&ring->fence_drv.fallback_timer);
}+}
+void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev) +{
unsigned int i, j;for (i = 0; i < AMDGPU_MAX_RINGS; i++) {struct amdgpu_ring *ring = adev->rings[i];if (!ring || !ring->fence_drv.initialized)continue;for (j = 0; j <= ring->fence_drv.num_fences_mask; ++j) dma_fence_put(ring->fence_drv.fences[j]); kfree(ring->fence_drv.fences);diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index afbbec82a289..63e815c27585 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -49,6 +49,7 @@ #include <drm/drm_irq.h> #include <drm/drm_vblank.h> #include <drm/amdgpu_drm.h> +#include <drm/drm_drv.h> #include "amdgpu.h" #include "amdgpu_ih.h" #include "atom.h" @@ -313,6 +314,20 @@ int amdgpu_irq_init(struct amdgpu_device *adev) return 0; }
+void amdgpu_irq_fini_hw(struct amdgpu_device *adev) +{
if (adev->irq.installed) {drm_irq_uninstall(&adev->ddev);adev->irq.installed = false;if (adev->irq.msi_enabled)pci_free_irq_vectors(adev->pdev);if (!amdgpu_device_has_dc_support(adev))flush_work(&adev->hotplug_work);}+}
- /**
- amdgpu_irq_fini - shut down interrupt handling
@@ -322,19 +337,10 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
- functionality, shuts down vblank, hotplug and reset interrupt handling,
- turns off interrupts from all sources (all ASICs).
*/ -void amdgpu_irq_fini(struct amdgpu_device *adev) +void amdgpu_irq_fini_sw(struct amdgpu_device *adev) { unsigned i, j;
if (adev->irq.installed) {drm_irq_uninstall(adev_to_drm(adev));adev->irq.installed = false;if (adev->irq.msi_enabled)pci_free_irq_vectors(adev->pdev);if (!amdgpu_device_has_dc_support(adev))flush_work(&adev->hotplug_work);}for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if (!adev->irq.client[i].sources) continue;diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h index ac527e5deae6..392a7324e2b1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h @@ -104,7 +104,8 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev); irqreturn_t amdgpu_irq_handler(int irq, void *arg);
int amdgpu_irq_init(struct amdgpu_device *adev); -void amdgpu_irq_fini(struct amdgpu_device *adev); +void amdgpu_irq_fini_sw(struct amdgpu_device *adev); +void amdgpu_irq_fini_hw(struct amdgpu_device *adev); int amdgpu_irq_add_id(struct amdgpu_device *adev, unsigned client_id, unsigned src_id, struct amdgpu_irq_src *source); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c index 64beb3399604..1af3fba7bfd4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c @@ -29,6 +29,7 @@ #include "amdgpu.h" #include <drm/drm_debugfs.h> #include <drm/amdgpu_drm.h> +#include <drm/drm_drv.h> #include "amdgpu_uvd.h" #include "amdgpu_vce.h" #include "atom.h" @@ -93,7 +94,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev) }
amdgpu_acpi_fini(adev);
amdgpu_device_fini(adev);
amdgpu_device_fini_hw(adev);}
void amdgpu_register_gpu_instance(struct amdgpu_device *adev)
@@ -1151,6 +1152,15 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev, pm_runtime_put_autosuspend(dev->dev); }
+void amdgpu_driver_release_kms(struct drm_device *dev) +{
struct amdgpu_device *adev = drm_to_adev(dev);amdgpu_device_fini_sw(adev);pci_set_drvdata(adev->pdev, NULL);+}
- /*
*/
- VBlank related functions.
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c index 1fb2a91ad30a..c0a16eac4923 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c @@ -2142,6 +2142,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev) if (!con) return 0;
/* Need disable ras on all IPs here before ip [hw/sw]fini */ amdgpu_ras_disable_all_features(adev, 0); amdgpu_ras_recovery_fini(adev);diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h index 56acec1075ac..0f195f7bf797 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h @@ -107,7 +107,8 @@ struct amdgpu_fence_driver { };
int amdgpu_fence_driver_init(struct amdgpu_device *adev); -void amdgpu_fence_driver_fini(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev); void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring);
int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index d3745711d55f..183d44a6583c 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -309,7 +309,7 @@ static int cik_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini(adev);
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index 307c01301c87..d32743949003 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -301,7 +301,7 @@ static int cz_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini(adev);
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index cc957471f31e..da96c6013477 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -300,7 +300,7 @@ static int iceland_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini(adev);
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index f4e4040bbd25..5eea4550b856 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -569,7 +569,7 @@ static int navi10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini(adev);
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1);diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 51880f6ef634..751307f3252c 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -175,7 +175,7 @@ static int si_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini(adev);
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0;diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 249fcbee7871..973d80ec7f6c 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -312,7 +312,7 @@ static int tonga_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini(adev);
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index 88626d83e07b..2d0094c276ca 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -523,7 +523,7 @@ static int vega10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini(adev);
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1);diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c index 5a3c867d5881..9059b21b079f 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c @@ -558,7 +558,7 @@ static int vega20_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini(adev);
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
[AMD Official Use Only - Internal Distribution Only]
sysfs cleanup is a sw cleanup to me but done inside hw fini. sw/hw separation is not strictly followed, or name it like stage1/stage2 fini.
Thanks, Lijo ________________________________ From: amd-gfx amd-gfx-bounces@lists.freedesktop.org on behalf of Andrey Grodzovsky andrey.grodzovsky@amd.com Sent: Wednesday, April 28, 2021 8:41:43 PM To: dri-devel@lists.freedesktop.org dri-devel@lists.freedesktop.org; amd-gfx@lists.freedesktop.org amd-gfx@lists.freedesktop.org; linux-pci@vger.kernel.org linux-pci@vger.kernel.org; ckoenig.leichtzumerken@gmail.com ckoenig.leichtzumerken@gmail.com; daniel.vetter@ffwll.ch daniel.vetter@ffwll.ch; Wentland, Harry Harry.Wentland@amd.com Cc: Grodzovsky, Andrey Andrey.Grodzovsky@amd.com; gregkh@linuxfoundation.org gregkh@linuxfoundation.org; Kuehling, Felix Felix.Kuehling@amd.com; ppaalanen@gmail.com ppaalanen@gmail.com; helgaas@kernel.org helgaas@kernel.org; Deucher, Alexander Alexander.Deucher@amd.com Subject: [PATCH v5 03/27] drm/amdgpu: Split amdgpu_device_fini into early and late
Some of the stuff in amdgpu_device_fini such as HW interrupts disable and pending fences finilization must be done right away on pci_remove while most of the stuff which relates to finilizing and releasing driver data structures can be kept until drm_driver.release hook is called, i.e. when the last device reference is dropped.
v4: Change functions prefix early->hw and late->sw
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 6 ++++- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 +++++++++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 7 ++---- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 15 ++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 26 +++++++++++++--------- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h | 3 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 12 +++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 3 ++- drivers/gpu/drm/amd/amdgpu/cik_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/cz_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/si_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/vega20_ih.c | 2 +- 17 files changed, 79 insertions(+), 36 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 1af2fa1591fd..fddb82897e5d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1073,7 +1073,9 @@ static inline struct amdgpu_device *amdgpu_ttm_adev(struct ttm_device *bdev)
int amdgpu_device_init(struct amdgpu_device *adev, uint32_t flags); -void amdgpu_device_fini(struct amdgpu_device *adev); +void amdgpu_device_fini_hw(struct amdgpu_device *adev); +void amdgpu_device_fini_sw(struct amdgpu_device *adev); + int amdgpu_gpu_wait_for_idle(struct amdgpu_device *adev);
void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, @@ -1289,6 +1291,8 @@ void amdgpu_driver_lastclose_kms(struct drm_device *dev); int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv); void amdgpu_driver_postclose_kms(struct drm_device *dev, struct drm_file *file_priv); +void amdgpu_driver_release_kms(struct drm_device *dev); + int amdgpu_device_ip_suspend(struct amdgpu_device *adev); int amdgpu_device_suspend(struct drm_device *dev, bool fbcon); int amdgpu_device_resume(struct drm_device *dev, bool fbcon); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 6447cd6ca5a8..8d22b79fc1cd 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3590,14 +3590,12 @@ int amdgpu_device_init(struct amdgpu_device *adev, * Tear down the driver info (all asics). * Called at driver shutdown. */ -void amdgpu_device_fini(struct amdgpu_device *adev) +void amdgpu_device_fini_hw(struct amdgpu_device *adev) { dev_info(adev->dev, "amdgpu: finishing device.\n"); flush_delayed_work(&adev->delayed_init_work); adev->shutdown = true;
- kfree(adev->pci_state); - /* make sure IB test finished before entering exclusive mode * to avoid preemption on IB test * */ @@ -3614,11 +3612,24 @@ void amdgpu_device_fini(struct amdgpu_device *adev) else drm_atomic_helper_shutdown(adev_to_drm(adev)); } - amdgpu_fence_driver_fini(adev); + amdgpu_fence_driver_fini_hw(adev); + if (adev->pm_sysfs_en) amdgpu_pm_sysfs_fini(adev); + if (adev->ucode_sysfs_en) + amdgpu_ucode_sysfs_fini(adev); + sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); + + amdgpu_fbdev_fini(adev); + + amdgpu_irq_fini_hw(adev); +} + +void amdgpu_device_fini_sw(struct amdgpu_device *adev) +{ amdgpu_device_ip_fini(adev); + amdgpu_fence_driver_fini_sw(adev); release_firmware(adev->firmware.gpu_info_fw); adev->firmware.gpu_info_fw = NULL; adev->accel_working = false; @@ -3647,14 +3658,13 @@ void amdgpu_device_fini(struct amdgpu_device *adev) adev->rmmio = NULL; amdgpu_device_doorbell_fini(adev);
- if (adev->ucode_sysfs_en) - amdgpu_ucode_sysfs_fini(adev); - - sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); if (IS_ENABLED(CONFIG_PERF_EVENTS)) amdgpu_pmu_fini(adev); if (adev->mman.discovery_bin) amdgpu_discovery_fini(adev); + + kfree(adev->pci_state); + }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 671ec1002230..54cb5ee2f563 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1249,14 +1249,10 @@ amdgpu_pci_remove(struct pci_dev *pdev) { struct drm_device *dev = pci_get_drvdata(pdev);
-#ifdef MODULE - if (THIS_MODULE->state != MODULE_STATE_GOING) -#endif - DRM_ERROR("Hotplug removal is not supported\n"); drm_dev_unplug(dev); amdgpu_driver_unload_kms(dev); + pci_disable_device(pdev); - pci_set_drvdata(pdev, NULL); }
static void @@ -1587,6 +1583,7 @@ static const struct drm_driver amdgpu_kms_driver = { .dumb_create = amdgpu_mode_dumb_create, .dumb_map_offset = amdgpu_mode_dumb_mmap, .fops = &amdgpu_driver_kms_fops, + .release = &amdgpu_driver_release_kms,
.prime_handle_to_fd = drm_gem_prime_handle_to_fd, .prime_fd_to_handle = drm_gem_prime_fd_to_handle, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 8e0a5650d383..34d51e962799 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -523,7 +523,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev) * * Tear down the fence driver for all possible rings (all asics). */ -void amdgpu_fence_driver_fini(struct amdgpu_device *adev) +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) { unsigned i, j; int r; @@ -544,6 +544,19 @@ void amdgpu_fence_driver_fini(struct amdgpu_device *adev) if (!ring->no_scheduler) drm_sched_fini(&ring->sched); del_timer_sync(&ring->fence_drv.fallback_timer); + } +} + +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev) +{ + unsigned int i, j; + + for (i = 0; i < AMDGPU_MAX_RINGS; i++) { + struct amdgpu_ring *ring = adev->rings[i]; + + if (!ring || !ring->fence_drv.initialized) + continue; + for (j = 0; j <= ring->fence_drv.num_fences_mask; ++j) dma_fence_put(ring->fence_drv.fences[j]); kfree(ring->fence_drv.fences); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index afbbec82a289..63e815c27585 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -49,6 +49,7 @@ #include <drm/drm_irq.h> #include <drm/drm_vblank.h> #include <drm/amdgpu_drm.h> +#include <drm/drm_drv.h> #include "amdgpu.h" #include "amdgpu_ih.h" #include "atom.h" @@ -313,6 +314,20 @@ int amdgpu_irq_init(struct amdgpu_device *adev) return 0; }
+ +void amdgpu_irq_fini_hw(struct amdgpu_device *adev) +{ + if (adev->irq.installed) { + drm_irq_uninstall(&adev->ddev); + adev->irq.installed = false; + if (adev->irq.msi_enabled) + pci_free_irq_vectors(adev->pdev); + + if (!amdgpu_device_has_dc_support(adev)) + flush_work(&adev->hotplug_work); + } +} + /** * amdgpu_irq_fini - shut down interrupt handling * @@ -322,19 +337,10 @@ int amdgpu_irq_init(struct amdgpu_device *adev) * functionality, shuts down vblank, hotplug and reset interrupt handling, * turns off interrupts from all sources (all ASICs). */ -void amdgpu_irq_fini(struct amdgpu_device *adev) +void amdgpu_irq_fini_sw(struct amdgpu_device *adev) { unsigned i, j;
- if (adev->irq.installed) { - drm_irq_uninstall(adev_to_drm(adev)); - adev->irq.installed = false; - if (adev->irq.msi_enabled) - pci_free_irq_vectors(adev->pdev); - if (!amdgpu_device_has_dc_support(adev)) - flush_work(&adev->hotplug_work); - } - for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if (!adev->irq.client[i].sources) continue; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h index ac527e5deae6..392a7324e2b1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h @@ -104,7 +104,8 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev); irqreturn_t amdgpu_irq_handler(int irq, void *arg);
int amdgpu_irq_init(struct amdgpu_device *adev); -void amdgpu_irq_fini(struct amdgpu_device *adev); +void amdgpu_irq_fini_sw(struct amdgpu_device *adev); +void amdgpu_irq_fini_hw(struct amdgpu_device *adev); int amdgpu_irq_add_id(struct amdgpu_device *adev, unsigned client_id, unsigned src_id, struct amdgpu_irq_src *source); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c index 64beb3399604..1af3fba7bfd4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c @@ -29,6 +29,7 @@ #include "amdgpu.h" #include <drm/drm_debugfs.h> #include <drm/amdgpu_drm.h> +#include <drm/drm_drv.h> #include "amdgpu_uvd.h" #include "amdgpu_vce.h" #include "atom.h" @@ -93,7 +94,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev) }
amdgpu_acpi_fini(adev); - amdgpu_device_fini(adev); + amdgpu_device_fini_hw(adev); }
void amdgpu_register_gpu_instance(struct amdgpu_device *adev) @@ -1151,6 +1152,15 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev, pm_runtime_put_autosuspend(dev->dev); }
+ +void amdgpu_driver_release_kms(struct drm_device *dev) +{ + struct amdgpu_device *adev = drm_to_adev(dev); + + amdgpu_device_fini_sw(adev); + pci_set_drvdata(adev->pdev, NULL); +} + /* * VBlank related functions. */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c index 1fb2a91ad30a..c0a16eac4923 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c @@ -2142,6 +2142,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev) if (!con) return 0;
+ /* Need disable ras on all IPs here before ip [hw/sw]fini */ amdgpu_ras_disable_all_features(adev, 0); amdgpu_ras_recovery_fini(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h index 56acec1075ac..0f195f7bf797 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h @@ -107,7 +107,8 @@ struct amdgpu_fence_driver { };
int amdgpu_fence_driver_init(struct amdgpu_device *adev); -void amdgpu_fence_driver_fini(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev); void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring);
int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index d3745711d55f..183d44a6583c 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -309,7 +309,7 @@ static int cik_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index 307c01301c87..d32743949003 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -301,7 +301,7 @@ static int cz_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index cc957471f31e..da96c6013477 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -300,7 +300,7 @@ static int iceland_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index f4e4040bbd25..5eea4550b856 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -569,7 +569,7 @@ static int navi10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 51880f6ef634..751307f3252c 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -175,7 +175,7 @@ static int si_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih);
return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 249fcbee7871..973d80ec7f6c 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -312,7 +312,7 @@ static int tonga_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index 88626d83e07b..2d0094c276ca 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -523,7 +523,7 @@ static int vega10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c index 5a3c867d5881..9059b21b079f 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c @@ -558,7 +558,7 @@ static int vega20_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); -- 2.25.1
_______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.free...
On 2021-04-30 1:19 a.m., Lazar, Lijo wrote:
[AMD Official Use Only - Internal Distribution Only]
sysfs cleanup is a sw cleanup to me but done inside hw fini. sw/hw separation is not strictly followed, or name it like stage1/stage2 fini.
The thing is that it was called early_fini and late_fini before and then according to Daniel's requirest it was changed to fini_hw and fini_sw
Andrey
Thanks, Lijo
*From:* amd-gfx amd-gfx-bounces@lists.freedesktop.org on behalf of Andrey Grodzovsky andrey.grodzovsky@amd.com *Sent:* Wednesday, April 28, 2021 8:41:43 PM *To:* dri-devel@lists.freedesktop.org dri-devel@lists.freedesktop.org; amd-gfx@lists.freedesktop.org amd-gfx@lists.freedesktop.org; linux-pci@vger.kernel.org linux-pci@vger.kernel.org; ckoenig.leichtzumerken@gmail.com ckoenig.leichtzumerken@gmail.com; daniel.vetter@ffwll.ch daniel.vetter@ffwll.ch; Wentland, Harry Harry.Wentland@amd.com *Cc:* Grodzovsky, Andrey Andrey.Grodzovsky@amd.com; gregkh@linuxfoundation.org gregkh@linuxfoundation.org; Kuehling, Felix Felix.Kuehling@amd.com; ppaalanen@gmail.com ppaalanen@gmail.com; helgaas@kernel.org helgaas@kernel.org; Deucher, Alexander Alexander.Deucher@amd.com *Subject:* [PATCH v5 03/27] drm/amdgpu: Split amdgpu_device_fini into early and late Some of the stuff in amdgpu_device_fini such as HW interrupts disable and pending fences finilization must be done right away on pci_remove while most of the stuff which relates to finilizing and releasing driver data structures can be kept until drm_driver.release hook is called, i.e. when the last device reference is dropped.
v4: Change functions prefix early->hw and late->sw
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 6 ++++- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 +++++++++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 7 ++---- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 15 ++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 26 +++++++++++++--------- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h | 3 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 12 +++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 3 ++- drivers/gpu/drm/amd/amdgpu/cik_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/cz_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/si_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/vega20_ih.c | 2 +- 17 files changed, 79 insertions(+), 36 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 1af2fa1591fd..fddb82897e5d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1073,7 +1073,9 @@ static inline struct amdgpu_device *amdgpu_ttm_adev(struct ttm_device *bdev)
int amdgpu_device_init(struct amdgpu_device *adev, uint32_t flags); -void amdgpu_device_fini(struct amdgpu_device *adev); +void amdgpu_device_fini_hw(struct amdgpu_device *adev); +void amdgpu_device_fini_sw(struct amdgpu_device *adev);
int amdgpu_gpu_wait_for_idle(struct amdgpu_device *adev);
void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, @@ -1289,6 +1291,8 @@ void amdgpu_driver_lastclose_kms(struct drm_device *dev); int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv); void amdgpu_driver_postclose_kms(struct drm_device *dev, struct drm_file *file_priv); +void amdgpu_driver_release_kms(struct drm_device *dev);
int amdgpu_device_ip_suspend(struct amdgpu_device *adev); int amdgpu_device_suspend(struct drm_device *dev, bool fbcon); int amdgpu_device_resume(struct drm_device *dev, bool fbcon); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 6447cd6ca5a8..8d22b79fc1cd 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3590,14 +3590,12 @@ int amdgpu_device_init(struct amdgpu_device *adev, * Tear down the driver info (all asics). * Called at driver shutdown. */ -void amdgpu_device_fini(struct amdgpu_device *adev) +void amdgpu_device_fini_hw(struct amdgpu_device *adev) { dev_info(adev->dev, "amdgpu: finishing device.\n"); flush_delayed_work(&adev->delayed_init_work); adev->shutdown = true;
- kfree(adev->pci_state);
/* make sure IB test finished before entering exclusive mode * to avoid preemption on IB test * */ @@ -3614,11 +3612,24 @@ void amdgpu_device_fini(struct amdgpu_device *adev) else drm_atomic_helper_shutdown(adev_to_drm(adev)); } - amdgpu_fence_driver_fini(adev); + amdgpu_fence_driver_fini_hw(adev);
if (adev->pm_sysfs_en) amdgpu_pm_sysfs_fini(adev); + if (adev->ucode_sysfs_en) + amdgpu_ucode_sysfs_fini(adev); + sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
amdgpu_fbdev_fini(adev);
+ amdgpu_irq_fini_hw(adev); +}
+void amdgpu_device_fini_sw(struct amdgpu_device *adev) +{ amdgpu_device_ip_fini(adev); + amdgpu_fence_driver_fini_sw(adev); release_firmware(adev->firmware.gpu_info_fw); adev->firmware.gpu_info_fw = NULL; adev->accel_working = false; @@ -3647,14 +3658,13 @@ void amdgpu_device_fini(struct amdgpu_device *adev) adev->rmmio = NULL; amdgpu_device_doorbell_fini(adev);
- if (adev->ucode_sysfs_en) - amdgpu_ucode_sysfs_fini(adev);
- sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); if (IS_ENABLED(CONFIG_PERF_EVENTS)) amdgpu_pmu_fini(adev); if (adev->mman.discovery_bin) amdgpu_discovery_fini(adev);
+ kfree(adev->pci_state);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 671ec1002230..54cb5ee2f563 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1249,14 +1249,10 @@ amdgpu_pci_remove(struct pci_dev *pdev) { struct drm_device *dev = pci_get_drvdata(pdev);
-#ifdef MODULE - if (THIS_MODULE->state != MODULE_STATE_GOING) -#endif - DRM_ERROR("Hotplug removal is not supported\n"); drm_dev_unplug(dev); amdgpu_driver_unload_kms(dev);
pci_disable_device(pdev); - pci_set_drvdata(pdev, NULL); }
static void @@ -1587,6 +1583,7 @@ static const struct drm_driver amdgpu_kms_driver = { .dumb_create = amdgpu_mode_dumb_create, .dumb_map_offset = amdgpu_mode_dumb_mmap, .fops = &amdgpu_driver_kms_fops, + .release = &amdgpu_driver_release_kms,
.prime_handle_to_fd = drm_gem_prime_handle_to_fd, .prime_fd_to_handle = drm_gem_prime_fd_to_handle, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 8e0a5650d383..34d51e962799 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -523,7 +523,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev) * * Tear down the fence driver for all possible rings (all asics). */ -void amdgpu_fence_driver_fini(struct amdgpu_device *adev) +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) { unsigned i, j; int r; @@ -544,6 +544,19 @@ void amdgpu_fence_driver_fini(struct amdgpu_device *adev) if (!ring->no_scheduler) drm_sched_fini(&ring->sched); del_timer_sync(&ring->fence_drv.fallback_timer); + } +}
+void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev) +{ + unsigned int i, j;
+ for (i = 0; i < AMDGPU_MAX_RINGS; i++) { + struct amdgpu_ring *ring = adev->rings[i];
+ if (!ring || !ring->fence_drv.initialized) + continue;
for (j = 0; j <= ring->fence_drv.num_fences_mask; ++j) dma_fence_put(ring->fence_drv.fences[j]); kfree(ring->fence_drv.fences); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index afbbec82a289..63e815c27585 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -49,6 +49,7 @@ #include <drm/drm_irq.h> #include <drm/drm_vblank.h> #include <drm/amdgpu_drm.h> +#include <drm/drm_drv.h> #include "amdgpu.h" #include "amdgpu_ih.h" #include "atom.h" @@ -313,6 +314,20 @@ int amdgpu_irq_init(struct amdgpu_device *adev) return 0; }
+void amdgpu_irq_fini_hw(struct amdgpu_device *adev) +{ + if (adev->irq.installed) { + drm_irq_uninstall(&adev->ddev); + adev->irq.installed = false; + if (adev->irq.msi_enabled) + pci_free_irq_vectors(adev->pdev);
+ if (!amdgpu_device_has_dc_support(adev)) + flush_work(&adev->hotplug_work); + } +}
/** * amdgpu_irq_fini - shut down interrupt handling * @@ -322,19 +337,10 @@ int amdgpu_irq_init(struct amdgpu_device *adev) * functionality, shuts down vblank, hotplug and reset interrupt handling, * turns off interrupts from all sources (all ASICs). */ -void amdgpu_irq_fini(struct amdgpu_device *adev) +void amdgpu_irq_fini_sw(struct amdgpu_device *adev) { unsigned i, j;
- if (adev->irq.installed) { - drm_irq_uninstall(adev_to_drm(adev)); - adev->irq.installed = false; - if (adev->irq.msi_enabled) - pci_free_irq_vectors(adev->pdev); - if (!amdgpu_device_has_dc_support(adev)) - flush_work(&adev->hotplug_work); - }
for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if (!adev->irq.client[i].sources) continue; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h index ac527e5deae6..392a7324e2b1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h @@ -104,7 +104,8 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev); irqreturn_t amdgpu_irq_handler(int irq, void *arg);
int amdgpu_irq_init(struct amdgpu_device *adev); -void amdgpu_irq_fini(struct amdgpu_device *adev); +void amdgpu_irq_fini_sw(struct amdgpu_device *adev); +void amdgpu_irq_fini_hw(struct amdgpu_device *adev); int amdgpu_irq_add_id(struct amdgpu_device *adev, unsigned client_id, unsigned src_id, struct amdgpu_irq_src *source); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c index 64beb3399604..1af3fba7bfd4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c @@ -29,6 +29,7 @@ #include "amdgpu.h" #include <drm/drm_debugfs.h> #include <drm/amdgpu_drm.h> +#include <drm/drm_drv.h> #include "amdgpu_uvd.h" #include "amdgpu_vce.h" #include "atom.h" @@ -93,7 +94,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev) }
amdgpu_acpi_fini(adev); - amdgpu_device_fini(adev); + amdgpu_device_fini_hw(adev); }
void amdgpu_register_gpu_instance(struct amdgpu_device *adev) @@ -1151,6 +1152,15 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev, pm_runtime_put_autosuspend(dev->dev); }
+void amdgpu_driver_release_kms(struct drm_device *dev) +{ + struct amdgpu_device *adev = drm_to_adev(dev);
+ amdgpu_device_fini_sw(adev); + pci_set_drvdata(adev->pdev, NULL); +}
/* * VBlank related functions. */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c index 1fb2a91ad30a..c0a16eac4923 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c @@ -2142,6 +2142,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev) if (!con) return 0;
/* Need disable ras on all IPs here before ip [hw/sw]fini */ amdgpu_ras_disable_all_features(adev, 0); amdgpu_ras_recovery_fini(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h index 56acec1075ac..0f195f7bf797 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h @@ -107,7 +107,8 @@ struct amdgpu_fence_driver { };
int amdgpu_fence_driver_init(struct amdgpu_device *adev); -void amdgpu_fence_driver_fini(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev); void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring);
int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index d3745711d55f..183d44a6583c 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -309,7 +309,7 @@ static int cik_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index 307c01301c87..d32743949003 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -301,7 +301,7 @@ static int cz_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index cc957471f31e..da96c6013477 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -300,7 +300,7 @@ static int iceland_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index f4e4040bbd25..5eea4550b856 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -569,7 +569,7 @@ static int navi10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 51880f6ef634..751307f3252c 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -175,7 +175,7 @@ static int si_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih);
return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 249fcbee7871..973d80ec7f6c 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -312,7 +312,7 @@ static int tonga_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index 88626d83e07b..2d0094c276ca 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -523,7 +523,7 @@ static int vega10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c index 5a3c867d5881..9059b21b079f 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c @@ -558,7 +558,7 @@ static int vega20_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); -- 2.25.1
amd-gfx mailing list amd-gfx@lists.freedesktop.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.free... https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&data=04%7C01%7Clijo.lazar%40amd.com%7C71f92dcef6d04dfea28308d90a5818c9%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637552195764790324%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=fzGS6ZXYffGH0DE%2BEM4jxyB8yDGpYO%2FmT%2FKAtgig1Tw%3D&reserved=0
[AMD Official Use Only - Internal Distribution Only]
That looks better to me :) As more things get added, I don't know how long you will be able to hold sw/hw cleanup separate and the name could confuse eventually.
Thanks, Lijo
Helps to expdite HW related stuff to amdgpu_pci_remove
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 2 +- drivers/gpu/drm/amd/amdkfd/kfd_device.c | 3 ++- 3 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c index c5343a5eecbe..9edb35ba181b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c @@ -169,7 +169,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) } }
-void amdgpu_amdkfd_device_fini(struct amdgpu_device *adev) +void amdgpu_amdkfd_device_fini_sw(struct amdgpu_device *adev) { if (adev->kfd.dev) { kgd2kfd_device_exit(adev->kfd.dev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index a81d9cacf9b8..c51001602a68 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -126,7 +126,7 @@ void amdgpu_amdkfd_interrupt(struct amdgpu_device *adev, const void *ih_ring_entry); void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev); void amdgpu_amdkfd_device_init(struct amdgpu_device *adev); -void amdgpu_amdkfd_device_fini(struct amdgpu_device *adev); +void amdgpu_amdkfd_device_fini_sw(struct amdgpu_device *adev); int amdgpu_amdkfd_submit_ib(struct kgd_dev *kgd, enum kgd_engine_type engine, uint32_t vmid, uint64_t gpu_addr, uint32_t *ib_cmd, uint32_t ib_len); diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c index 72c893fff61a..1bb8bc6d85f5 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c @@ -833,10 +833,11 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, return kfd->init_complete; }
+ + void kgd2kfd_device_exit(struct kfd_dev *kfd) { if (kfd->init_complete) { - kgd2kfd_suspend(kfd, false); device_queue_manager_uninit(kfd->dqm); kfd_interrupt_exit(kfd); kfd_topology_remove_device(kfd);
Use it to call disply code dependent on device->drv_data before it's set to NULL on device unplug
v5: Move HW finilization into this callback to prevent MMIO accesses post cpi remove.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 59 +++++++++++++------ .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 12 +++- drivers/gpu/drm/amd/include/amd_shared.h | 2 + 3 files changed, 52 insertions(+), 21 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 8d22b79fc1cd..46d646c40338 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -2536,34 +2536,26 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) return 0; }
-/** - * amdgpu_device_ip_fini - run fini for hardware IPs - * - * @adev: amdgpu_device pointer - * - * Main teardown pass for hardware IPs. The list of all the hardware - * IPs that make up the asic is walked and the hw_fini and sw_fini callbacks - * are run. hw_fini tears down the hardware associated with each IP - * and sw_fini tears down any software state associated with each IP. - * Returns 0 on success, negative error code on failure. - */ -static int amdgpu_device_ip_fini(struct amdgpu_device *adev) +static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) { int i, r;
- if (amdgpu_sriov_vf(adev) && adev->virt.ras_init_done) - amdgpu_virt_release_ras_err_handler_data(adev); + for (i = 0; i < adev->num_ip_blocks; i++) { + if (!adev->ip_blocks[i].version->funcs->early_fini) + continue;
- amdgpu_ras_pre_fini(adev); + r = adev->ip_blocks[i].version->funcs->early_fini((void *)adev); + if (r) { + DRM_DEBUG("early_fini of IP block <%s> failed %d\n", + adev->ip_blocks[i].version->funcs->name, r); + } + }
- if (adev->gmc.xgmi.num_physical_nodes > 1) - amdgpu_xgmi_remove_device(adev); + amdgpu_amdkfd_suspend(adev, false);
amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE); amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE);
- amdgpu_amdkfd_device_fini(adev); - /* need to disable SMC first */ for (i = 0; i < adev->num_ip_blocks; i++) { if (!adev->ip_blocks[i].status.hw) @@ -2594,6 +2586,33 @@ static int amdgpu_device_ip_fini(struct amdgpu_device *adev) adev->ip_blocks[i].status.hw = false; }
+ return 0; +} + +/** + * amdgpu_device_ip_fini - run fini for hardware IPs + * + * @adev: amdgpu_device pointer + * + * Main teardown pass for hardware IPs. The list of all the hardware + * IPs that make up the asic is walked and the hw_fini and sw_fini callbacks + * are run. hw_fini tears down the hardware associated with each IP + * and sw_fini tears down any software state associated with each IP. + * Returns 0 on success, negative error code on failure. + */ +static int amdgpu_device_ip_fini(struct amdgpu_device *adev) +{ + int i, r; + + if (amdgpu_sriov_vf(adev) && adev->virt.ras_init_done) + amdgpu_virt_release_ras_err_handler_data(adev); + + amdgpu_ras_pre_fini(adev); + + if (adev->gmc.xgmi.num_physical_nodes > 1) + amdgpu_xgmi_remove_device(adev); + + amdgpu_amdkfd_device_fini_sw(adev);
for (i = adev->num_ip_blocks - 1; i >= 0; i--) { if (!adev->ip_blocks[i].status.sw) @@ -3624,6 +3643,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_fbdev_fini(adev);
amdgpu_irq_fini_hw(adev); + + amdgpu_device_ip_fini_early(adev); }
void amdgpu_device_fini_sw(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index 55e39b462a5e..c0b9abb773a4 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -1170,6 +1170,15 @@ static int amdgpu_dm_init(struct amdgpu_device *adev) return -EINVAL; }
+static int amdgpu_dm_early_fini(void *handle) +{ + struct amdgpu_device *adev = (struct amdgpu_device *)handle; + + amdgpu_dm_audio_fini(adev); + + return 0; +} + static void amdgpu_dm_fini(struct amdgpu_device *adev) { int i; @@ -1178,8 +1187,6 @@ static void amdgpu_dm_fini(struct amdgpu_device *adev) drm_encoder_cleanup(&adev->dm.mst_encoders[i].base); }
- amdgpu_dm_audio_fini(adev); - amdgpu_dm_destroy_drm_device(&adev->dm);
#ifdef CONFIG_DRM_AMD_DC_HDCP @@ -2194,6 +2201,7 @@ static const struct amd_ip_funcs amdgpu_dm_funcs = { .late_init = dm_late_init, .sw_init = dm_sw_init, .sw_fini = dm_sw_fini, + .early_fini = amdgpu_dm_early_fini, .hw_init = dm_hw_init, .hw_fini = dm_hw_fini, .suspend = dm_suspend, diff --git a/drivers/gpu/drm/amd/include/amd_shared.h b/drivers/gpu/drm/amd/include/amd_shared.h index 43ed6291b2b8..1ad56da486e4 100644 --- a/drivers/gpu/drm/amd/include/amd_shared.h +++ b/drivers/gpu/drm/amd/include/amd_shared.h @@ -240,6 +240,7 @@ enum amd_dpm_forced_level; * @late_init: sets up late driver/hw state (post hw_init) - Optional * @sw_init: sets up driver state, does not configure hw * @sw_fini: tears down driver state, does not configure hw + * @early_fini: tears down stuff before dev detached from driver * @hw_init: sets up the hw state * @hw_fini: tears down the hw state * @late_fini: final cleanup @@ -268,6 +269,7 @@ struct amd_ip_funcs { int (*late_init)(void *handle); int (*sw_init)(void *handle); int (*sw_fini)(void *handle); + int (*early_fini)(void *handle); int (*hw_init)(void *handle); int (*hw_fini)(void *handle); void (*late_fini)(void *handle);
Handle all DMA IOMMU gropup related dependencies before the group is removed.
v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 3 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 9 +++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 2 ++ drivers/gpu/drm/amd/amdgpu/cik_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/cz_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 3 --- drivers/gpu/drm/amd/amdgpu/si_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 3 --- 14 files changed, 56 insertions(+), 16 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index fddb82897e5d..30a24db5f4d1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1054,6 +1054,8 @@ struct amdgpu_device {
bool in_pci_err_recovery; struct pci_saved_state *pci_state; + + struct list_head device_bo_list; };
static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 46d646c40338..91594ddc2459 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -70,6 +70,7 @@ #include <drm/task_barrier.h> #include <linux/pm_runtime.h>
+ MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin"); @@ -3211,7 +3212,6 @@ static const struct attribute *amdgpu_dev_attributes[] = { NULL };
- /** * amdgpu_device_init - initialize the driver * @@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func);
+ INIT_LIST_HEAD(&adev->device_bo_list); + adev->gfx.gfx_off_req_count = 1; adev->pm.ac_power = power_supply_is_system_supplied() > 0;
@@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev, return r; }
+static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev) +{ + struct amdgpu_bo *bo = NULL; + + /* + * Unmaps all DMA mappings before device will be removed from it's + * IOMMU group otherwise in case of IOMMU enabled system a crash + * will happen. + */ + + spin_lock(&adev->mman.bdev.lru_lock); + while (!list_empty(&adev->device_bo_list)) { + bo = list_first_entry(&adev->device_bo_list, struct amdgpu_bo, bo); + list_del_init(&bo->bo); + spin_unlock(&adev->mman.bdev.lru_lock); + if (bo->tbo.ttm) + ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm); + spin_lock(&adev->mman.bdev.lru_lock); + } + spin_unlock(&adev->mman.bdev.lru_lock); +} + /** * amdgpu_device_fini - tear down the driver * @@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_ucode_sysfs_fini(adev); sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
- amdgpu_fbdev_fini(adev);
amdgpu_irq_fini_hw(adev);
amdgpu_device_ip_fini_early(adev); + + amdgpu_clear_dma_mappings(adev); + + amdgpu_gart_dummy_page_fini(adev); }
void amdgpu_device_fini_sw(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c index fde2d899b2c4..49cdcaf8512d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev) * * Frees the dummy page used by the driver (all asics). */ -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) { if (!adev->dummy_page_addr) return; @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev) vfree(adev->gart.pages); adev->gart.pages = NULL; #endif - amdgpu_gart_dummy_page_fini(adev); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h index afa2e2877d87..5678d9c105ab 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev); void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev); int amdgpu_gart_init(struct amdgpu_device *adev); void amdgpu_gart_fini(struct amdgpu_device *adev); +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev); int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, int pages); int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 63e815c27585..a922154953a7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev) if (!amdgpu_device_has_dc_support(adev)) flush_work(&adev->hotplug_work); } + + if (adev->irq.ih_soft.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); + if (adev->irq.ih.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih); + if (adev->irq.ih1.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih1); + if (adev->irq.ih2.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih2); }
/** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 485f249d063a..62d829f5e62c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo) list_del_init(&bo->shadow_list); mutex_unlock(&adev->shadow_list_lock); } - amdgpu_bo_unref(&bo->parent);
+ + spin_lock(&adev->mman.bdev.lru_lock); + list_del(&bo->bo); + spin_unlock(&adev->mman.bdev.lru_lock); + + amdgpu_bo_unref(&bo->parent); kfree(bo->metadata); kfree(bo); } @@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, if (bp->type == ttm_bo_type_device) bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
+ INIT_LIST_HEAD(&bo->bo); + + spin_lock(&adev->mman.bdev.lru_lock); + list_add_tail(&bo->bo, &adev->device_bo_list); + spin_unlock(&adev->mman.bdev.lru_lock); + return 0;
fail_unreserve: diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index 9ac37569823f..5ae8555ef275 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -110,6 +110,8 @@ struct amdgpu_bo { struct list_head shadow_list;
struct kgd_mem *kfd_bo; + + struct list_head bo; };
static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo) diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index 183d44a6583c..df385ffc9768 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index d32743949003..b8c47e0cf37a 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index da96c6013477..ddfe4eaeea05 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index 5eea4550b856..e171a9e78544 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle)
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); - amdgpu_ih_ring_fini(adev, &adev->irq.ih2); - amdgpu_ih_ring_fini(adev, &adev->irq.ih1); - amdgpu_ih_ring_fini(adev, &adev->irq.ih);
return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 751307f3252c..9a24f17a5750 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih);
return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 973d80ec7f6c..b08905d1c00f 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index 2d0094c276ca..8c8abc00f710 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle)
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); - amdgpu_ih_ring_fini(adev, &adev->irq.ih2); - amdgpu_ih_ring_fini(adev, &adev->irq.ih1); - amdgpu_ih_ring_fini(adev, &adev->irq.ih);
return 0; }
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
Handle all DMA IOMMU gropup related dependencies before the group is removed.
v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate
Maybe split that up into more patches.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 3 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 9 +++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 2 ++ drivers/gpu/drm/amd/amdgpu/cik_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/cz_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 3 --- drivers/gpu/drm/amd/amdgpu/si_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 3 --- 14 files changed, 56 insertions(+), 16 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index fddb82897e5d..30a24db5f4d1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1054,6 +1054,8 @@ struct amdgpu_device {
bool in_pci_err_recovery; struct pci_saved_state *pci_state;
struct list_head device_bo_list; };
static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 46d646c40338..91594ddc2459 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -70,6 +70,7 @@ #include <drm/task_barrier.h> #include <linux/pm_runtime.h>
- MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin");
@@ -3211,7 +3212,6 @@ static const struct attribute *amdgpu_dev_attributes[] = { NULL };
- /**
- amdgpu_device_init - initialize the driver
@@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func);
- INIT_LIST_HEAD(&adev->device_bo_list);
- adev->gfx.gfx_off_req_count = 1; adev->pm.ac_power = power_supply_is_system_supplied() > 0;
@@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev, return r; }
+static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev) +{
- struct amdgpu_bo *bo = NULL;
- /*
* Unmaps all DMA mappings before device will be removed from it's* IOMMU group otherwise in case of IOMMU enabled system a crash* will happen.*/- spin_lock(&adev->mman.bdev.lru_lock);
- while (!list_empty(&adev->device_bo_list)) {
bo = list_first_entry(&adev->device_bo_list, struct amdgpu_bo, bo);list_del_init(&bo->bo);spin_unlock(&adev->mman.bdev.lru_lock);if (bo->tbo.ttm)ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm);spin_lock(&adev->mman.bdev.lru_lock);- }
- spin_unlock(&adev->mman.bdev.lru_lock);
Can you try to use the same approach as amdgpu_gtt_mgr_recover() instead of adding something to the BO?
Christian.
+}
- /**
- amdgpu_device_fini - tear down the driver
@@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_ucode_sysfs_fini(adev); sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
amdgpu_fbdev_fini(adev);
amdgpu_irq_fini_hw(adev);
amdgpu_device_ip_fini_early(adev);
amdgpu_clear_dma_mappings(adev);
amdgpu_gart_dummy_page_fini(adev); }
void amdgpu_device_fini_sw(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c index fde2d899b2c4..49cdcaf8512d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev)
- Frees the dummy page used by the driver (all asics).
*/ -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) { if (!adev->dummy_page_addr) return; @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev) vfree(adev->gart.pages); adev->gart.pages = NULL; #endif
- amdgpu_gart_dummy_page_fini(adev); }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h index afa2e2877d87..5678d9c105ab 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev); void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev); int amdgpu_gart_init(struct amdgpu_device *adev); void amdgpu_gart_fini(struct amdgpu_device *adev); +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev); int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, int pages); int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 63e815c27585..a922154953a7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev) if (!amdgpu_device_has_dc_support(adev)) flush_work(&adev->hotplug_work); }
if (adev->irq.ih_soft.ring)
amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);if (adev->irq.ih.ring)
amdgpu_ih_ring_fini(adev, &adev->irq.ih);if (adev->irq.ih1.ring)
amdgpu_ih_ring_fini(adev, &adev->irq.ih1);if (adev->irq.ih2.ring)
amdgpu_ih_ring_fini(adev, &adev->irq.ih2);}
/**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 485f249d063a..62d829f5e62c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo) list_del_init(&bo->shadow_list); mutex_unlock(&adev->shadow_list_lock); }
- amdgpu_bo_unref(&bo->parent);
- spin_lock(&adev->mman.bdev.lru_lock);
- list_del(&bo->bo);
- spin_unlock(&adev->mman.bdev.lru_lock);
- amdgpu_bo_unref(&bo->parent); kfree(bo->metadata); kfree(bo); }
@@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, if (bp->type == ttm_bo_type_device) bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
INIT_LIST_HEAD(&bo->bo);
spin_lock(&adev->mman.bdev.lru_lock);
list_add_tail(&bo->bo, &adev->device_bo_list);
spin_unlock(&adev->mman.bdev.lru_lock);
return 0;
fail_unreserve:
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index 9ac37569823f..5ae8555ef275 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -110,6 +110,8 @@ struct amdgpu_bo { struct list_head shadow_list;
struct kgd_mem *kfd_bo;
struct list_head bo; };
static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo)
diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index 183d44a6583c..df385ffc9768 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index d32743949003..b8c47e0cf37a 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index da96c6013477..ddfe4eaeea05 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index 5eea4550b856..e171a9e78544 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle)
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
amdgpu_ih_ring_fini(adev, &adev->irq.ih);
return 0; }
diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 751307f3252c..9a24f17a5750 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih);
return 0; }
diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 973d80ec7f6c..b08905d1c00f 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index 2d0094c276ca..8c8abc00f710 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle)
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
amdgpu_ih_ring_fini(adev, &adev->irq.ih);
return 0; }
On 2021-04-29 3:08 a.m., Christian König wrote:
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
Handle all DMA IOMMU gropup related dependencies before the group is removed.
v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate
Maybe split that up into more patches.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 3 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 9 +++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 2 ++ drivers/gpu/drm/amd/amdgpu/cik_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/cz_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 3 --- drivers/gpu/drm/amd/amdgpu/si_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 3 --- 14 files changed, 56 insertions(+), 16 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index fddb82897e5d..30a24db5f4d1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1054,6 +1054,8 @@ struct amdgpu_device { bool in_pci_err_recovery; struct pci_saved_state *pci_state;
+ struct list_head device_bo_list; }; static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 46d646c40338..91594ddc2459 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -70,6 +70,7 @@ #include <drm/task_barrier.h> #include <linux/pm_runtime.h>
MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin"); @@ -3211,7 +3212,6 @@ static const struct attribute *amdgpu_dev_attributes[] = { NULL };
/** * amdgpu_device_init - initialize the driver * @@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev, INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func); + INIT_LIST_HEAD(&adev->device_bo_list);
adev->gfx.gfx_off_req_count = 1; adev->pm.ac_power = power_supply_is_system_supplied() > 0; @@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev, return r; } +static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev) +{ + struct amdgpu_bo *bo = NULL;
+ /* + * Unmaps all DMA mappings before device will be removed from it's + * IOMMU group otherwise in case of IOMMU enabled system a crash + * will happen. + */
+ spin_lock(&adev->mman.bdev.lru_lock); + while (!list_empty(&adev->device_bo_list)) { + bo = list_first_entry(&adev->device_bo_list, struct amdgpu_bo, bo); + list_del_init(&bo->bo); + spin_unlock(&adev->mman.bdev.lru_lock); + if (bo->tbo.ttm) + ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm); + spin_lock(&adev->mman.bdev.lru_lock); + } + spin_unlock(&adev->mman.bdev.lru_lock);
Can you try to use the same approach as amdgpu_gtt_mgr_recover() instead of adding something to the BO?
Christian.
Are you sure that dma mappings limit themself only to GTT BOs which have allocated mm nodes ? Otherwsie we will crash and burn on missing IOMMU group when unampping post device remove. Problem for me to test this as in 5.12 kernel I don't crash even when removing this entire patch. Looks like iommu_dma_unmap_page was changed since 5.9 when I introdiced this patch.
Andrey
+}
/** * amdgpu_device_fini - tear down the driver * @@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_ucode_sysfs_fini(adev); sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
amdgpu_fbdev_fini(adev); amdgpu_irq_fini_hw(adev); amdgpu_device_ip_fini_early(adev);
+ amdgpu_clear_dma_mappings(adev);
+ amdgpu_gart_dummy_page_fini(adev); } void amdgpu_device_fini_sw(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c index fde2d899b2c4..49cdcaf8512d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev) * * Frees the dummy page used by the driver (all asics). */ -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) { if (!adev->dummy_page_addr) return; @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev) vfree(adev->gart.pages); adev->gart.pages = NULL; #endif - amdgpu_gart_dummy_page_fini(adev); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h index afa2e2877d87..5678d9c105ab 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev); void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev); int amdgpu_gart_init(struct amdgpu_device *adev); void amdgpu_gart_fini(struct amdgpu_device *adev); +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev); int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, int pages); int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 63e815c27585..a922154953a7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev) if (!amdgpu_device_has_dc_support(adev)) flush_work(&adev->hotplug_work); }
+ if (adev->irq.ih_soft.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); + if (adev->irq.ih.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih); + if (adev->irq.ih1.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih1); + if (adev->irq.ih2.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih2); } /** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 485f249d063a..62d829f5e62c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo) list_del_init(&bo->shadow_list); mutex_unlock(&adev->shadow_list_lock); } - amdgpu_bo_unref(&bo->parent);
+ spin_lock(&adev->mman.bdev.lru_lock); + list_del(&bo->bo); + spin_unlock(&adev->mman.bdev.lru_lock);
+ amdgpu_bo_unref(&bo->parent); kfree(bo->metadata); kfree(bo); } @@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, if (bp->type == ttm_bo_type_device) bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED; + INIT_LIST_HEAD(&bo->bo);
+ spin_lock(&adev->mman.bdev.lru_lock); + list_add_tail(&bo->bo, &adev->device_bo_list); + spin_unlock(&adev->mman.bdev.lru_lock);
return 0; fail_unreserve: diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index 9ac37569823f..5ae8555ef275 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -110,6 +110,8 @@ struct amdgpu_bo { struct list_head shadow_list; struct kgd_mem *kfd_bo;
+ struct list_head bo; }; static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo) diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index 183d44a6583c..df385ffc9768 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index d32743949003..b8c47e0cf37a 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index da96c6013477..ddfe4eaeea05 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index 5eea4550b856..e171a9e78544 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle) amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); - amdgpu_ih_ring_fini(adev, &adev->irq.ih2); - amdgpu_ih_ring_fini(adev, &adev->irq.ih1); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 751307f3252c..9a24f17a5750 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 973d80ec7f6c..b08905d1c00f 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index 2d0094c276ca..8c8abc00f710 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle) amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); - amdgpu_ih_ring_fini(adev, &adev->irq.ih2); - amdgpu_ih_ring_fini(adev, &adev->irq.ih1); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; }
Am 03.05.21 um 22:43 schrieb Andrey Grodzovsky:
On 2021-04-29 3:08 a.m., Christian König wrote:
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
Handle all DMA IOMMU gropup related dependencies before the group is removed.
v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate
Maybe split that up into more patches.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 3 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 9 +++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 2 ++ drivers/gpu/drm/amd/amdgpu/cik_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/cz_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 3 --- drivers/gpu/drm/amd/amdgpu/si_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 3 --- 14 files changed, 56 insertions(+), 16 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index fddb82897e5d..30a24db5f4d1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1054,6 +1054,8 @@ struct amdgpu_device { bool in_pci_err_recovery; struct pci_saved_state *pci_state;
+ struct list_head device_bo_list; }; static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 46d646c40338..91594ddc2459 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -70,6 +70,7 @@ #include <drm/task_barrier.h> #include <linux/pm_runtime.h>
MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin"); @@ -3211,7 +3212,6 @@ static const struct attribute *amdgpu_dev_attributes[] = { NULL };
/** * amdgpu_device_init - initialize the driver * @@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev, INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func); + INIT_LIST_HEAD(&adev->device_bo_list);
adev->gfx.gfx_off_req_count = 1; adev->pm.ac_power = power_supply_is_system_supplied() > 0; @@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev, return r; } +static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev) +{ + struct amdgpu_bo *bo = NULL;
+ /* + * Unmaps all DMA mappings before device will be removed from it's + * IOMMU group otherwise in case of IOMMU enabled system a crash + * will happen. + */
+ spin_lock(&adev->mman.bdev.lru_lock); + while (!list_empty(&adev->device_bo_list)) { + bo = list_first_entry(&adev->device_bo_list, struct amdgpu_bo, bo); + list_del_init(&bo->bo); + spin_unlock(&adev->mman.bdev.lru_lock); + if (bo->tbo.ttm) + ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm); + spin_lock(&adev->mman.bdev.lru_lock); + } + spin_unlock(&adev->mman.bdev.lru_lock);
Can you try to use the same approach as amdgpu_gtt_mgr_recover() instead of adding something to the BO?
Christian.
Are you sure that dma mappings limit themself only to GTT BOs which have allocated mm nodes ?
Yes, you would also need the system domain BOs. But those can be put on a similar list.
Otherwsie we will crash and burn on missing IOMMU group when unampping post device remove. Problem for me to test this as in 5.12 kernel I don't crash even when removing this entire patch. Looks like iommu_dma_unmap_page was changed since 5.9 when I introdiced this patch.
Do we really still need that stuff then? What exactly has changed?
Christian.
Andrey
+}
/** * amdgpu_device_fini - tear down the driver * @@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_ucode_sysfs_fini(adev); sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
amdgpu_fbdev_fini(adev); amdgpu_irq_fini_hw(adev); amdgpu_device_ip_fini_early(adev);
+ amdgpu_clear_dma_mappings(adev);
+ amdgpu_gart_dummy_page_fini(adev); } void amdgpu_device_fini_sw(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c index fde2d899b2c4..49cdcaf8512d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev) * * Frees the dummy page used by the driver (all asics). */ -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) { if (!adev->dummy_page_addr) return; @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev) vfree(adev->gart.pages); adev->gart.pages = NULL; #endif - amdgpu_gart_dummy_page_fini(adev); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h index afa2e2877d87..5678d9c105ab 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev); void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev); int amdgpu_gart_init(struct amdgpu_device *adev); void amdgpu_gart_fini(struct amdgpu_device *adev); +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev); int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, int pages); int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 63e815c27585..a922154953a7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev) if (!amdgpu_device_has_dc_support(adev)) flush_work(&adev->hotplug_work); }
+ if (adev->irq.ih_soft.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); + if (adev->irq.ih.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih); + if (adev->irq.ih1.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih1); + if (adev->irq.ih2.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih2); } /** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 485f249d063a..62d829f5e62c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo) list_del_init(&bo->shadow_list); mutex_unlock(&adev->shadow_list_lock); } - amdgpu_bo_unref(&bo->parent);
+ spin_lock(&adev->mman.bdev.lru_lock); + list_del(&bo->bo); + spin_unlock(&adev->mman.bdev.lru_lock);
+ amdgpu_bo_unref(&bo->parent); kfree(bo->metadata); kfree(bo); } @@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, if (bp->type == ttm_bo_type_device) bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED; + INIT_LIST_HEAD(&bo->bo);
+ spin_lock(&adev->mman.bdev.lru_lock); + list_add_tail(&bo->bo, &adev->device_bo_list); + spin_unlock(&adev->mman.bdev.lru_lock);
return 0; fail_unreserve: diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index 9ac37569823f..5ae8555ef275 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -110,6 +110,8 @@ struct amdgpu_bo { struct list_head shadow_list; struct kgd_mem *kfd_bo;
+ struct list_head bo; }; static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo) diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index 183d44a6583c..df385ffc9768 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index d32743949003..b8c47e0cf37a 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index da96c6013477..ddfe4eaeea05 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index 5eea4550b856..e171a9e78544 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle) amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); - amdgpu_ih_ring_fini(adev, &adev->irq.ih2); - amdgpu_ih_ring_fini(adev, &adev->irq.ih1); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 751307f3252c..9a24f17a5750 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 973d80ec7f6c..b08905d1c00f 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index 2d0094c276ca..8c8abc00f710 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle) amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); - amdgpu_ih_ring_fini(adev, &adev->irq.ih2); - amdgpu_ih_ring_fini(adev, &adev->irq.ih1); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; }
On 2021-05-04 3:03 a.m., Christian König wrote:
Am 03.05.21 um 22:43 schrieb Andrey Grodzovsky:
On 2021-04-29 3:08 a.m., Christian König wrote:
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
Handle all DMA IOMMU gropup related dependencies before the group is removed.
v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate
Maybe split that up into more patches.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 3 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 9 +++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 2 ++ drivers/gpu/drm/amd/amdgpu/cik_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/cz_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 3 --- drivers/gpu/drm/amd/amdgpu/si_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 3 --- 14 files changed, 56 insertions(+), 16 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index fddb82897e5d..30a24db5f4d1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1054,6 +1054,8 @@ struct amdgpu_device { bool in_pci_err_recovery; struct pci_saved_state *pci_state;
+ struct list_head device_bo_list; }; static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 46d646c40338..91594ddc2459 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -70,6 +70,7 @@ #include <drm/task_barrier.h> #include <linux/pm_runtime.h>
MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin"); @@ -3211,7 +3212,6 @@ static const struct attribute *amdgpu_dev_attributes[] = { NULL };
/** * amdgpu_device_init - initialize the driver * @@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev, INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func); + INIT_LIST_HEAD(&adev->device_bo_list);
adev->gfx.gfx_off_req_count = 1; adev->pm.ac_power = power_supply_is_system_supplied() > 0; @@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev, return r; } +static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev) +{ + struct amdgpu_bo *bo = NULL;
+ /* + * Unmaps all DMA mappings before device will be removed from it's + * IOMMU group otherwise in case of IOMMU enabled system a crash + * will happen. + */
+ spin_lock(&adev->mman.bdev.lru_lock); + while (!list_empty(&adev->device_bo_list)) { + bo = list_first_entry(&adev->device_bo_list, struct amdgpu_bo, bo); + list_del_init(&bo->bo); + spin_unlock(&adev->mman.bdev.lru_lock); + if (bo->tbo.ttm) + ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm); + spin_lock(&adev->mman.bdev.lru_lock); + } + spin_unlock(&adev->mman.bdev.lru_lock);
Can you try to use the same approach as amdgpu_gtt_mgr_recover() instead of adding something to the BO?
Christian.
Are you sure that dma mappings limit themself only to GTT BOs which have allocated mm nodes ?
Yes, you would also need the system domain BOs. But those can be put on a similar list.
What list ? Those BOs don't have ttm_resource_manager and so no drm_mm_node list they all bound to. Should I maintain a list for them spcifically for the unmap purpuse ?
Otherwsie we will crash and burn on missing IOMMU group when unampping post device remove. Problem for me to test this as in 5.12 kernel I don't crash even when removing this entire patch. Looks like iommu_dma_unmap_page was changed since 5.9 when I introdiced this patch.
Do we really still need that stuff then? What exactly has changed?
At first I assumed that because of this change 'iommu: Allow the dma-iommu api to use bounce buffers' Which changed iommu_dma_unmap_page to call __iommu_dma_unmap_swiotlb instead if __iommu_dma_unmap directly. But then i looked inside __iommu_dma_unmap_swiotlb and it still calls __iommu_dma_unmap evenetually. So maybe the fact that I moved the amd_ip_funcs.hw_fini call to inside amdgpu_pci_remove helps.
Andrey
Christian.
Andrey
+}
/** * amdgpu_device_fini - tear down the driver * @@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_ucode_sysfs_fini(adev); sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
amdgpu_fbdev_fini(adev); amdgpu_irq_fini_hw(adev); amdgpu_device_ip_fini_early(adev);
+ amdgpu_clear_dma_mappings(adev);
+ amdgpu_gart_dummy_page_fini(adev); } void amdgpu_device_fini_sw(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c index fde2d899b2c4..49cdcaf8512d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev) * * Frees the dummy page used by the driver (all asics). */ -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) { if (!adev->dummy_page_addr) return; @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev) vfree(adev->gart.pages); adev->gart.pages = NULL; #endif - amdgpu_gart_dummy_page_fini(adev); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h index afa2e2877d87..5678d9c105ab 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev); void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev); int amdgpu_gart_init(struct amdgpu_device *adev); void amdgpu_gart_fini(struct amdgpu_device *adev); +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev); int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, int pages); int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 63e815c27585..a922154953a7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev) if (!amdgpu_device_has_dc_support(adev)) flush_work(&adev->hotplug_work); }
+ if (adev->irq.ih_soft.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); + if (adev->irq.ih.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih); + if (adev->irq.ih1.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih1); + if (adev->irq.ih2.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih2); } /** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 485f249d063a..62d829f5e62c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo) list_del_init(&bo->shadow_list); mutex_unlock(&adev->shadow_list_lock); } - amdgpu_bo_unref(&bo->parent);
+ spin_lock(&adev->mman.bdev.lru_lock); + list_del(&bo->bo); + spin_unlock(&adev->mman.bdev.lru_lock);
+ amdgpu_bo_unref(&bo->parent); kfree(bo->metadata); kfree(bo); } @@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, if (bp->type == ttm_bo_type_device) bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED; + INIT_LIST_HEAD(&bo->bo);
+ spin_lock(&adev->mman.bdev.lru_lock); + list_add_tail(&bo->bo, &adev->device_bo_list); + spin_unlock(&adev->mman.bdev.lru_lock);
return 0; fail_unreserve: diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index 9ac37569823f..5ae8555ef275 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -110,6 +110,8 @@ struct amdgpu_bo { struct list_head shadow_list; struct kgd_mem *kfd_bo;
+ struct list_head bo; }; static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo) diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index 183d44a6583c..df385ffc9768 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index d32743949003..b8c47e0cf37a 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index da96c6013477..ddfe4eaeea05 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index 5eea4550b856..e171a9e78544 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle) amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); - amdgpu_ih_ring_fini(adev, &adev->irq.ih2); - amdgpu_ih_ring_fini(adev, &adev->irq.ih1); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 751307f3252c..9a24f17a5750 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 973d80ec7f6c..b08905d1c00f 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index 2d0094c276ca..8c8abc00f710 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle) amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); - amdgpu_ih_ring_fini(adev, &adev->irq.ih2); - amdgpu_ih_ring_fini(adev, &adev->irq.ih1); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; }
amd-gfx mailing list amd-gfx@lists.freedesktop.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.free...
Ping
Andrey
On 2021-05-04 11:43 a.m., Andrey Grodzovsky wrote:
On 2021-05-04 3:03 a.m., Christian König wrote:
Am 03.05.21 um 22:43 schrieb Andrey Grodzovsky:
On 2021-04-29 3:08 a.m., Christian König wrote:
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
Handle all DMA IOMMU gropup related dependencies before the group is removed.
v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate
Maybe split that up into more patches.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 3 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 9 +++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 2 ++ drivers/gpu/drm/amd/amdgpu/cik_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/cz_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 3 --- drivers/gpu/drm/amd/amdgpu/si_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 3 --- 14 files changed, 56 insertions(+), 16 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index fddb82897e5d..30a24db5f4d1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1054,6 +1054,8 @@ struct amdgpu_device { bool in_pci_err_recovery; struct pci_saved_state *pci_state;
+ struct list_head device_bo_list; }; static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 46d646c40338..91594ddc2459 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -70,6 +70,7 @@ #include <drm/task_barrier.h> #include <linux/pm_runtime.h>
MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin"); @@ -3211,7 +3212,6 @@ static const struct attribute *amdgpu_dev_attributes[] = { NULL };
/** * amdgpu_device_init - initialize the driver * @@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev, INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func); + INIT_LIST_HEAD(&adev->device_bo_list);
adev->gfx.gfx_off_req_count = 1; adev->pm.ac_power = power_supply_is_system_supplied() > 0; @@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev, return r; } +static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev) +{ + struct amdgpu_bo *bo = NULL;
+ /* + * Unmaps all DMA mappings before device will be removed from it's + * IOMMU group otherwise in case of IOMMU enabled system a crash + * will happen. + */
+ spin_lock(&adev->mman.bdev.lru_lock); + while (!list_empty(&adev->device_bo_list)) { + bo = list_first_entry(&adev->device_bo_list, struct amdgpu_bo, bo); + list_del_init(&bo->bo); + spin_unlock(&adev->mman.bdev.lru_lock); + if (bo->tbo.ttm) + ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm); + spin_lock(&adev->mman.bdev.lru_lock); + } + spin_unlock(&adev->mman.bdev.lru_lock);
Can you try to use the same approach as amdgpu_gtt_mgr_recover() instead of adding something to the BO?
Christian.
Are you sure that dma mappings limit themself only to GTT BOs which have allocated mm nodes ?
Yes, you would also need the system domain BOs. But those can be put on a similar list.
What list ? Those BOs don't have ttm_resource_manager and so no drm_mm_node list they all bound to. Should I maintain a list for them spcifically for the unmap purpuse ?
Otherwsie we will crash and burn on missing IOMMU group when unampping post device remove. Problem for me to test this as in 5.12 kernel I don't crash even when removing this entire patch. Looks like iommu_dma_unmap_page was changed since 5.9 when I introdiced this patch.
Do we really still need that stuff then? What exactly has changed?
At first I assumed that because of this change 'iommu: Allow the dma-iommu api to use bounce buffers' Which changed iommu_dma_unmap_page to call __iommu_dma_unmap_swiotlb instead if __iommu_dma_unmap directly. But then i looked inside __iommu_dma_unmap_swiotlb and it still calls __iommu_dma_unmap evenetually. So maybe the fact that I moved the amd_ip_funcs.hw_fini call to inside amdgpu_pci_remove helps.
Andrey
Christian.
Andrey
+}
/** * amdgpu_device_fini - tear down the driver * @@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_ucode_sysfs_fini(adev); sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
amdgpu_fbdev_fini(adev); amdgpu_irq_fini_hw(adev); amdgpu_device_ip_fini_early(adev);
+ amdgpu_clear_dma_mappings(adev);
+ amdgpu_gart_dummy_page_fini(adev); } void amdgpu_device_fini_sw(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c index fde2d899b2c4..49cdcaf8512d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev) * * Frees the dummy page used by the driver (all asics). */ -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) { if (!adev->dummy_page_addr) return; @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev) vfree(adev->gart.pages); adev->gart.pages = NULL; #endif - amdgpu_gart_dummy_page_fini(adev); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h index afa2e2877d87..5678d9c105ab 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev); void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev); int amdgpu_gart_init(struct amdgpu_device *adev); void amdgpu_gart_fini(struct amdgpu_device *adev); +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev); int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, int pages); int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 63e815c27585..a922154953a7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev) if (!amdgpu_device_has_dc_support(adev)) flush_work(&adev->hotplug_work); }
+ if (adev->irq.ih_soft.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); + if (adev->irq.ih.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih); + if (adev->irq.ih1.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih1); + if (adev->irq.ih2.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih2); } /** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 485f249d063a..62d829f5e62c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo) list_del_init(&bo->shadow_list); mutex_unlock(&adev->shadow_list_lock); } - amdgpu_bo_unref(&bo->parent);
+ spin_lock(&adev->mman.bdev.lru_lock); + list_del(&bo->bo); + spin_unlock(&adev->mman.bdev.lru_lock);
+ amdgpu_bo_unref(&bo->parent); kfree(bo->metadata); kfree(bo); } @@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, if (bp->type == ttm_bo_type_device) bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED; + INIT_LIST_HEAD(&bo->bo);
+ spin_lock(&adev->mman.bdev.lru_lock); + list_add_tail(&bo->bo, &adev->device_bo_list); + spin_unlock(&adev->mman.bdev.lru_lock);
return 0; fail_unreserve: diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index 9ac37569823f..5ae8555ef275 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -110,6 +110,8 @@ struct amdgpu_bo { struct list_head shadow_list; struct kgd_mem *kfd_bo;
+ struct list_head bo; }; static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo) diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index 183d44a6583c..df385ffc9768 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index d32743949003..b8c47e0cf37a 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index da96c6013477..ddfe4eaeea05 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index 5eea4550b856..e171a9e78544 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle) amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); - amdgpu_ih_ring_fini(adev, &adev->irq.ih2); - amdgpu_ih_ring_fini(adev, &adev->irq.ih1); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 751307f3252c..9a24f17a5750 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 973d80ec7f6c..b08905d1c00f 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index 2d0094c276ca..8c8abc00f710 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle) amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); - amdgpu_ih_ring_fini(adev, &adev->irq.ih2); - amdgpu_ih_ring_fini(adev, &adev->irq.ih1); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; }
amd-gfx mailing list amd-gfx@lists.freedesktop.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.free...
On Wed, Apr 28, 2021 at 11:13 AM Andrey Grodzovsky andrey.grodzovsky@amd.com wrote:
Handle all DMA IOMMU gropup related dependencies before the group is removed.
v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 3 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 9 +++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 2 ++ drivers/gpu/drm/amd/amdgpu/cik_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/cz_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 3 --- drivers/gpu/drm/amd/amdgpu/si_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 3 --- 14 files changed, 56 insertions(+), 16 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index fddb82897e5d..30a24db5f4d1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1054,6 +1054,8 @@ struct amdgpu_device {
bool in_pci_err_recovery; struct pci_saved_state *pci_state;
struct list_head device_bo_list;};
static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 46d646c40338..91594ddc2459 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -70,6 +70,7 @@ #include <drm/task_barrier.h> #include <linux/pm_runtime.h>
MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin"); @@ -3211,7 +3212,6 @@ static const struct attribute *amdgpu_dev_attributes[] = { NULL };
/**
- amdgpu_device_init - initialize the driver
@@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func);
INIT_LIST_HEAD(&adev->device_bo_list);adev->gfx.gfx_off_req_count = 1; adev->pm.ac_power = power_supply_is_system_supplied() > 0;@@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev, return r; }
+static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev)
Prefix this with amdgpu_device for consistency. E.g., amdgpu_device_clear_dma_mappings()
+{
struct amdgpu_bo *bo = NULL;/** Unmaps all DMA mappings before device will be removed from it's* IOMMU group otherwise in case of IOMMU enabled system a crash* will happen.*/spin_lock(&adev->mman.bdev.lru_lock);while (!list_empty(&adev->device_bo_list)) {bo = list_first_entry(&adev->device_bo_list, struct amdgpu_bo, bo);list_del_init(&bo->bo);spin_unlock(&adev->mman.bdev.lru_lock);if (bo->tbo.ttm)ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm);spin_lock(&adev->mman.bdev.lru_lock);}spin_unlock(&adev->mman.bdev.lru_lock);+}
/**
- amdgpu_device_fini - tear down the driver
@@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_ucode_sysfs_fini(adev); sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
amdgpu_fbdev_fini(adev); amdgpu_irq_fini_hw(adev); amdgpu_device_ip_fini_early(adev);
amdgpu_clear_dma_mappings(adev);amdgpu_gart_dummy_page_fini(adev);}
void amdgpu_device_fini_sw(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c index fde2d899b2c4..49cdcaf8512d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev)
- Frees the dummy page used by the driver (all asics).
*/ -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) { if (!adev->dummy_page_addr) return; @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev) vfree(adev->gart.pages); adev->gart.pages = NULL; #endif
amdgpu_gart_dummy_page_fini(adev);} diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h index afa2e2877d87..5678d9c105ab 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev); void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev); int amdgpu_gart_init(struct amdgpu_device *adev); void amdgpu_gart_fini(struct amdgpu_device *adev); +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev); int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, int pages); int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 63e815c27585..a922154953a7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev) if (!amdgpu_device_has_dc_support(adev)) flush_work(&adev->hotplug_work); }
if (adev->irq.ih_soft.ring)amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);if (adev->irq.ih.ring)amdgpu_ih_ring_fini(adev, &adev->irq.ih);if (adev->irq.ih1.ring)amdgpu_ih_ring_fini(adev, &adev->irq.ih1);if (adev->irq.ih2.ring)amdgpu_ih_ring_fini(adev, &adev->irq.ih2);}
/** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 485f249d063a..62d829f5e62c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo) list_del_init(&bo->shadow_list); mutex_unlock(&adev->shadow_list_lock); }
amdgpu_bo_unref(&bo->parent);
spin_lock(&adev->mman.bdev.lru_lock);list_del(&bo->bo);spin_unlock(&adev->mman.bdev.lru_lock);amdgpu_bo_unref(&bo->parent); kfree(bo->metadata); kfree(bo);} @@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, if (bp->type == ttm_bo_type_device) bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
INIT_LIST_HEAD(&bo->bo);spin_lock(&adev->mman.bdev.lru_lock);list_add_tail(&bo->bo, &adev->device_bo_list);spin_unlock(&adev->mman.bdev.lru_lock);return 0;fail_unreserve: diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index 9ac37569823f..5ae8555ef275 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -110,6 +110,8 @@ struct amdgpu_bo { struct list_head shadow_list;
struct kgd_mem *kfd_bo;
struct list_head bo;};
static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo) diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index 183d44a6583c..df385ffc9768 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0;diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index d32743949003..b8c47e0cf37a 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0;diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index da96c6013477..ddfe4eaeea05 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0;diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index 5eea4550b856..e171a9e78544 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle)
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
Shouldn't the soft ring be removed as well?
amdgpu_ih_ring_fini(adev, &adev->irq.ih2);amdgpu_ih_ring_fini(adev, &adev->irq.ih1);amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0;} diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 751307f3252c..9a24f17a5750 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0;} diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 973d80ec7f6c..b08905d1c00f 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0;diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index 2d0094c276ca..8c8abc00f710 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle)
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
Same here?
amdgpu_ih_ring_fini(adev, &adev->irq.ih2);amdgpu_ih_ring_fini(adev, &adev->irq.ih1);amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0;}
2.25.1
amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
On 2021-04-29 11:13 p.m., Alex Deucher wrote:
On Wed, Apr 28, 2021 at 11:13 AM Andrey Grodzovsky andrey.grodzovsky@amd.com wrote:
Handle all DMA IOMMU gropup related dependencies before the group is removed.
v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 3 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 9 +++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 2 ++ drivers/gpu/drm/amd/amdgpu/cik_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/cz_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 3 --- drivers/gpu/drm/amd/amdgpu/si_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 3 --- 14 files changed, 56 insertions(+), 16 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index fddb82897e5d..30a24db5f4d1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1054,6 +1054,8 @@ struct amdgpu_device {
bool in_pci_err_recovery; struct pci_saved_state *pci_state;
struct list_head device_bo_list;};
static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 46d646c40338..91594ddc2459 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -70,6 +70,7 @@ #include <drm/task_barrier.h> #include <linux/pm_runtime.h>
- MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin");
@@ -3211,7 +3212,6 @@ static const struct attribute *amdgpu_dev_attributes[] = { NULL };
- /**
- amdgpu_device_init - initialize the driver
@@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func);
INIT_LIST_HEAD(&adev->device_bo_list);adev->gfx.gfx_off_req_count = 1; adev->pm.ac_power = power_supply_is_system_supplied() > 0;@@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev, return r; }
+static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev)
Prefix this with amdgpu_device for consistency. E.g., amdgpu_device_clear_dma_mappings()
+{
struct amdgpu_bo *bo = NULL;/** Unmaps all DMA mappings before device will be removed from it's* IOMMU group otherwise in case of IOMMU enabled system a crash* will happen.*/spin_lock(&adev->mman.bdev.lru_lock);while (!list_empty(&adev->device_bo_list)) {bo = list_first_entry(&adev->device_bo_list, struct amdgpu_bo, bo);list_del_init(&bo->bo);spin_unlock(&adev->mman.bdev.lru_lock);if (bo->tbo.ttm)ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm);spin_lock(&adev->mman.bdev.lru_lock);}spin_unlock(&adev->mman.bdev.lru_lock);+}
- /**
- amdgpu_device_fini - tear down the driver
@@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_ucode_sysfs_fini(adev); sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
amdgpu_fbdev_fini(adev); amdgpu_irq_fini_hw(adev); amdgpu_device_ip_fini_early(adev);
amdgpu_clear_dma_mappings(adev);amdgpu_gart_dummy_page_fini(adev);}
void amdgpu_device_fini_sw(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c index fde2d899b2c4..49cdcaf8512d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev)
- Frees the dummy page used by the driver (all asics).
*/ -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) { if (!adev->dummy_page_addr) return; @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev) vfree(adev->gart.pages); adev->gart.pages = NULL; #endif
amdgpu_gart_dummy_page_fini(adev);diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h index afa2e2877d87..5678d9c105ab 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev); void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev); int amdgpu_gart_init(struct amdgpu_device *adev); void amdgpu_gart_fini(struct amdgpu_device *adev); +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev); int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, int pages); int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 63e815c27585..a922154953a7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev) if (!amdgpu_device_has_dc_support(adev)) flush_work(&adev->hotplug_work); }
if (adev->irq.ih_soft.ring)amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);if (adev->irq.ih.ring)amdgpu_ih_ring_fini(adev, &adev->irq.ih);if (adev->irq.ih1.ring)amdgpu_ih_ring_fini(adev, &adev->irq.ih1);if (adev->irq.ih2.ring)amdgpu_ih_ring_fini(adev, &adev->irq.ih2);}
/**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 485f249d063a..62d829f5e62c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo) list_del_init(&bo->shadow_list); mutex_unlock(&adev->shadow_list_lock); }
amdgpu_bo_unref(&bo->parent);
spin_lock(&adev->mman.bdev.lru_lock);list_del(&bo->bo);spin_unlock(&adev->mman.bdev.lru_lock);amdgpu_bo_unref(&bo->parent); kfree(bo->metadata); kfree(bo);@@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, if (bp->type == ttm_bo_type_device) bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
INIT_LIST_HEAD(&bo->bo);spin_lock(&adev->mman.bdev.lru_lock);list_add_tail(&bo->bo, &adev->device_bo_list);spin_unlock(&adev->mman.bdev.lru_lock);return 0;fail_unreserve:
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index 9ac37569823f..5ae8555ef275 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -110,6 +110,8 @@ struct amdgpu_bo { struct list_head shadow_list;
struct kgd_mem *kfd_bo;
struct list_head bo;};
static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo)
diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index 183d44a6583c..df385ffc9768 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0;diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index d32743949003..b8c47e0cf37a 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0;diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index da96c6013477..ddfe4eaeea05 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0;diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index 5eea4550b856..e171a9e78544 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle)
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);Shouldn't the soft ring be removed as well?
Right, thanks for noticing.
amdgpu_ih_ring_fini(adev, &adev->irq.ih2);amdgpu_ih_ring_fini(adev, &adev->irq.ih1);amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0;}
diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 751307f3252c..9a24f17a5750 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0;}
diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 973d80ec7f6c..b08905d1c00f 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0;diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index 2d0094c276ca..8c8abc00f710 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle)
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);Same here?
Yep.
Andrey
amdgpu_ih_ring_fini(adev, &adev->irq.ih2);amdgpu_ih_ring_fini(adev, &adev->irq.ih1);amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0;}
-- 2.25.1
amd-gfx mailing list amd-gfx@lists.freedesktop.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.free...
Am 2021-04-28 um 11:11 a.m. schrieb Andrey Grodzovsky:
Handle all DMA IOMMU gropup related dependencies before the group is removed.
v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 3 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 9 +++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 2 ++ drivers/gpu/drm/amd/amdgpu/cik_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/cz_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 3 --- drivers/gpu/drm/amd/amdgpu/si_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 3 --- 14 files changed, 56 insertions(+), 16 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index fddb82897e5d..30a24db5f4d1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1054,6 +1054,8 @@ struct amdgpu_device {
bool in_pci_err_recovery; struct pci_saved_state *pci_state;
- struct list_head device_bo_list;
};
static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 46d646c40338..91594ddc2459 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -70,6 +70,7 @@ #include <drm/task_barrier.h> #include <linux/pm_runtime.h>
MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin"); @@ -3211,7 +3212,6 @@ static const struct attribute *amdgpu_dev_attributes[] = { NULL };
/**
- amdgpu_device_init - initialize the driver
@@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func);
- INIT_LIST_HEAD(&adev->device_bo_list);
- adev->gfx.gfx_off_req_count = 1; adev->pm.ac_power = power_supply_is_system_supplied() > 0;
@@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev, return r; }
+static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev) +{
- struct amdgpu_bo *bo = NULL;
- /*
* Unmaps all DMA mappings before device will be removed from it's* IOMMU group otherwise in case of IOMMU enabled system a crash* will happen.*/- spin_lock(&adev->mman.bdev.lru_lock);
- while (!list_empty(&adev->device_bo_list)) {
bo = list_first_entry(&adev->device_bo_list, struct amdgpu_bo, bo);list_del_init(&bo->bo);spin_unlock(&adev->mman.bdev.lru_lock);if (bo->tbo.ttm)ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm);
I have a patch pending (reviewed by Christian) that moves the dma-unmapping to amdgpu_ttm_backend_unbind. With that patch, ttm_tt_unpopulate would no longer be the right way to remove the DMA mapping.
Maybe I'd need to add a check in ttm_tt_unpopulate to call backend_unbind first, if necessary. Or is there some other mechanism that moves the BO to the CPU domain before unpopulating it?
Regards, Felix
spin_lock(&adev->mman.bdev.lru_lock);- }
- spin_unlock(&adev->mman.bdev.lru_lock);
+}
/**
- amdgpu_device_fini - tear down the driver
@@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_ucode_sysfs_fini(adev); sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
amdgpu_fbdev_fini(adev);
amdgpu_irq_fini_hw(adev);
amdgpu_device_ip_fini_early(adev);
- amdgpu_clear_dma_mappings(adev);
- amdgpu_gart_dummy_page_fini(adev);
}
void amdgpu_device_fini_sw(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c index fde2d899b2c4..49cdcaf8512d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev)
- Frees the dummy page used by the driver (all asics).
*/ -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) { if (!adev->dummy_page_addr) return; @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev) vfree(adev->gart.pages); adev->gart.pages = NULL; #endif
- amdgpu_gart_dummy_page_fini(adev);
} diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h index afa2e2877d87..5678d9c105ab 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev); void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev); int amdgpu_gart_init(struct amdgpu_device *adev); void amdgpu_gart_fini(struct amdgpu_device *adev); +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev); int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, int pages); int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 63e815c27585..a922154953a7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev) if (!amdgpu_device_has_dc_support(adev)) flush_work(&adev->hotplug_work); }
- if (adev->irq.ih_soft.ring)
amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);- if (adev->irq.ih.ring)
amdgpu_ih_ring_fini(adev, &adev->irq.ih);- if (adev->irq.ih1.ring)
amdgpu_ih_ring_fini(adev, &adev->irq.ih1);- if (adev->irq.ih2.ring)
amdgpu_ih_ring_fini(adev, &adev->irq.ih2);}
/** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 485f249d063a..62d829f5e62c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo) list_del_init(&bo->shadow_list); mutex_unlock(&adev->shadow_list_lock); }
- amdgpu_bo_unref(&bo->parent);
- spin_lock(&adev->mman.bdev.lru_lock);
- list_del(&bo->bo);
- spin_unlock(&adev->mman.bdev.lru_lock);
- amdgpu_bo_unref(&bo->parent); kfree(bo->metadata); kfree(bo);
} @@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, if (bp->type == ttm_bo_type_device) bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
- INIT_LIST_HEAD(&bo->bo);
- spin_lock(&adev->mman.bdev.lru_lock);
- list_add_tail(&bo->bo, &adev->device_bo_list);
- spin_unlock(&adev->mman.bdev.lru_lock);
- return 0;
fail_unreserve: diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index 9ac37569823f..5ae8555ef275 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -110,6 +110,8 @@ struct amdgpu_bo { struct list_head shadow_list;
struct kgd_mem *kfd_bo;
- struct list_head bo;
};
static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo) diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index 183d44a6583c..df385ffc9768 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index d32743949003..b8c47e0cf37a 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index da96c6013477..ddfe4eaeea05 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index 5eea4550b856..e171a9e78544 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle)
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
amdgpu_ih_ring_fini(adev, &adev->irq.ih);
return 0;
} diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 751307f3252c..9a24f17a5750 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih);
return 0;
} diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 973d80ec7f6c..b08905d1c00f 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index 2d0094c276ca..8c8abc00f710 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle)
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
amdgpu_ih_ring_fini(adev, &adev->irq.ih);
return 0;
}
On 2021-05-04 1:05 p.m., Felix Kuehling wrote:
Am 2021-04-28 um 11:11 a.m. schrieb Andrey Grodzovsky:
Handle all DMA IOMMU gropup related dependencies before the group is removed.
v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 3 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 9 +++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 2 ++ drivers/gpu/drm/amd/amdgpu/cik_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/cz_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 3 --- drivers/gpu/drm/amd/amdgpu/si_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 3 --- 14 files changed, 56 insertions(+), 16 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index fddb82897e5d..30a24db5f4d1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1054,6 +1054,8 @@ struct amdgpu_device {
bool in_pci_err_recovery; struct pci_saved_state *pci_state;
struct list_head device_bo_list; };
static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 46d646c40338..91594ddc2459 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -70,6 +70,7 @@ #include <drm/task_barrier.h> #include <linux/pm_runtime.h>
- MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin");
@@ -3211,7 +3212,6 @@ static const struct attribute *amdgpu_dev_attributes[] = { NULL };
- /**
- amdgpu_device_init - initialize the driver
@@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func);
- INIT_LIST_HEAD(&adev->device_bo_list);
- adev->gfx.gfx_off_req_count = 1; adev->pm.ac_power = power_supply_is_system_supplied() > 0;
@@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev, return r; }
+static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev) +{
- struct amdgpu_bo *bo = NULL;
- /*
* Unmaps all DMA mappings before device will be removed from it's* IOMMU group otherwise in case of IOMMU enabled system a crash* will happen.*/- spin_lock(&adev->mman.bdev.lru_lock);
- while (!list_empty(&adev->device_bo_list)) {
bo = list_first_entry(&adev->device_bo_list, struct amdgpu_bo, bo);list_del_init(&bo->bo);spin_unlock(&adev->mman.bdev.lru_lock);if (bo->tbo.ttm)ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm);I have a patch pending (reviewed by Christian) that moves the dma-unmapping to amdgpu_ttm_backend_unbind. With that patch, ttm_tt_unpopulate would no longer be the right way to remove the DMA mapping.
Maybe I'd need to add a check in ttm_tt_unpopulate to call backend_unbind first, if necessary. Or is there some other mechanism that moves the BO to the CPU domain before unpopulating it?
Regards, Felix
At least in the context of this patch we don't move the BO to system domain but rather preemptively remove DMA mappings before IOMMU grpoup is gone post pci_remove. So yes, I would say yes, we need to check here for backend_unbind first.
Andrey
spin_lock(&adev->mman.bdev.lru_lock);- }
- spin_unlock(&adev->mman.bdev.lru_lock);
+}
- /**
- amdgpu_device_fini - tear down the driver
@@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_ucode_sysfs_fini(adev); sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
amdgpu_fbdev_fini(adev);
amdgpu_irq_fini_hw(adev);
amdgpu_device_ip_fini_early(adev);
amdgpu_clear_dma_mappings(adev);
amdgpu_gart_dummy_page_fini(adev); }
void amdgpu_device_fini_sw(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c index fde2d899b2c4..49cdcaf8512d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev)
- Frees the dummy page used by the driver (all asics).
*/ -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) { if (!adev->dummy_page_addr) return; @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev) vfree(adev->gart.pages); adev->gart.pages = NULL; #endif
- amdgpu_gart_dummy_page_fini(adev); }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h index afa2e2877d87..5678d9c105ab 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev); void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev); int amdgpu_gart_init(struct amdgpu_device *adev); void amdgpu_gart_fini(struct amdgpu_device *adev); +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev); int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, int pages); int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 63e815c27585..a922154953a7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev) if (!amdgpu_device_has_dc_support(adev)) flush_work(&adev->hotplug_work); }
if (adev->irq.ih_soft.ring)
amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);if (adev->irq.ih.ring)
amdgpu_ih_ring_fini(adev, &adev->irq.ih);if (adev->irq.ih1.ring)
amdgpu_ih_ring_fini(adev, &adev->irq.ih1);if (adev->irq.ih2.ring)
amdgpu_ih_ring_fini(adev, &adev->irq.ih2);}
/**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 485f249d063a..62d829f5e62c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo) list_del_init(&bo->shadow_list); mutex_unlock(&adev->shadow_list_lock); }
- amdgpu_bo_unref(&bo->parent);
- spin_lock(&adev->mman.bdev.lru_lock);
- list_del(&bo->bo);
- spin_unlock(&adev->mman.bdev.lru_lock);
- amdgpu_bo_unref(&bo->parent); kfree(bo->metadata); kfree(bo); }
@@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, if (bp->type == ttm_bo_type_device) bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
INIT_LIST_HEAD(&bo->bo);
spin_lock(&adev->mman.bdev.lru_lock);
list_add_tail(&bo->bo, &adev->device_bo_list);
spin_unlock(&adev->mman.bdev.lru_lock);
return 0;
fail_unreserve:
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index 9ac37569823f..5ae8555ef275 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -110,6 +110,8 @@ struct amdgpu_bo { struct list_head shadow_list;
struct kgd_mem *kfd_bo;
struct list_head bo; };
static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo)
diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index 183d44a6583c..df385ffc9768 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index d32743949003..b8c47e0cf37a 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index da96c6013477..ddfe4eaeea05 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index 5eea4550b856..e171a9e78544 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle)
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
amdgpu_ih_ring_fini(adev, &adev->irq.ih);
return 0; }
diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 751307f3252c..9a24f17a5750 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih);
return 0; }
diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 973d80ec7f6c..b08905d1c00f 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle;
amdgpu_irq_fini_sw(adev);
amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev);
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index 2d0094c276ca..8c8abc00f710 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle)
amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
amdgpu_ih_ring_fini(adev, &adev->irq.ih);
return 0; }
On device removal reroute all CPU mappings to dummy page per drm_file instance or imported GEM object.
v4: Update for modified ttm_bo_vm_dummy_page
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index a785acc09f20..93163b220e46 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -49,6 +49,7 @@
#include <drm/drm_debugfs.h> #include <drm/amdgpu_drm.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "amdgpu_object.h" @@ -1982,18 +1983,28 @@ void amdgpu_ttm_set_buffer_funcs_status(struct amdgpu_device *adev, bool enable) static vm_fault_t amdgpu_ttm_fault(struct vm_fault *vmf) { struct ttm_buffer_object *bo = vmf->vma->vm_private_data; + struct drm_device *ddev = bo->base.dev; vm_fault_t ret; + int idx;
ret = ttm_bo_vm_reserve(bo, vmf); if (ret) return ret;
- ret = amdgpu_bo_fault_reserve_notify(bo); - if (ret) - goto unlock; + if (drm_dev_enter(ddev, &idx)) { + ret = amdgpu_bo_fault_reserve_notify(bo); + if (ret) { + drm_dev_exit(idx); + goto unlock; + }
- ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot, - TTM_BO_VM_NUM_PREFAULT, 1); + ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot, + TTM_BO_VM_NUM_PREFAULT, 1); + + drm_dev_exit(idx); + } else { + ret = ttm_bo_vm_dummy_page(vmf, vmf->vma->vm_page_prot); + } if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) return ret;
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
On device removal reroute all CPU mappings to dummy page per drm_file instance or imported GEM object.
v4: Update for modified ttm_bo_vm_dummy_page
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
Reviewed-by: Christian König christian.koenig@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index a785acc09f20..93163b220e46 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -49,6 +49,7 @@
#include <drm/drm_debugfs.h> #include <drm/amdgpu_drm.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "amdgpu_object.h" @@ -1982,18 +1983,28 @@ void amdgpu_ttm_set_buffer_funcs_status(struct amdgpu_device *adev, bool enable) static vm_fault_t amdgpu_ttm_fault(struct vm_fault *vmf) { struct ttm_buffer_object *bo = vmf->vma->vm_private_data;
struct drm_device *ddev = bo->base.dev; vm_fault_t ret;
int idx;
ret = ttm_bo_vm_reserve(bo, vmf); if (ret) return ret;
- ret = amdgpu_bo_fault_reserve_notify(bo);
- if (ret)
goto unlock;
- if (drm_dev_enter(ddev, &idx)) {
ret = amdgpu_bo_fault_reserve_notify(bo);if (ret) {drm_dev_exit(idx);goto unlock;}
- ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot,
TTM_BO_VM_NUM_PREFAULT, 1);
ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot,TTM_BO_VM_NUM_PREFAULT, 1);drm_dev_exit(idx);- } else {
ret = ttm_bo_vm_dummy_page(vmf, vmf->vma->vm_page_prot);- } if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) return ret;
This is exact copy of 'USB: add support for dev_groups to struct usb_device_driver' patch by Greg but just for the PCI case.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com Suggested-by: Greg Kroah-Hartman gregkh@linuxfoundation.org --- drivers/pci/pci-driver.c | 1 + include/linux/pci.h | 3 +++ 2 files changed, 4 insertions(+)
diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index ec44a79e951a..3a72352aa5cf 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -1385,6 +1385,7 @@ int __pci_register_driver(struct pci_driver *drv, struct module *owner, drv->driver.owner = owner; drv->driver.mod_name = mod_name; drv->driver.groups = drv->groups; + drv->driver.dev_groups = drv->dev_groups;
spin_lock_init(&drv->dynids.lock); INIT_LIST_HEAD(&drv->dynids.list); diff --git a/include/linux/pci.h b/include/linux/pci.h index 86c799c97b77..b57755b03009 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -858,6 +858,8 @@ struct module; * number of VFs to enable via sysfs "sriov_numvfs" file. * @err_handler: See Documentation/PCI/pci-error-recovery.rst * @groups: Sysfs attribute groups. + * @dev_groups: Attributes attached to the device that will be + * created once it is bound to the driver. * @driver: Driver model structure. * @dynids: List of dynamically added device IDs. */ @@ -873,6 +875,7 @@ struct pci_driver { int (*sriov_configure)(struct pci_dev *dev, int num_vfs); /* On PF */ const struct pci_error_handlers *err_handler; const struct attribute_group **groups; + const struct attribute_group **dev_groups; struct device_driver driver; struct pci_dynids dynids; };
In subject:
s/PCI: add support/PCI: Add support/
to match convention ("git log --oneline drivers/pci/pci-driver.c" to learn this).
On Wed, Apr 28, 2021 at 11:11:48AM -0400, Andrey Grodzovsky wrote:
This is exact copy of 'USB: add support for dev_groups to struct usb_device_driver' patch by Greg but just for the PCI case.
Ideally this would be an imperative sentence telling us what the patch *does*, not just that it's a copy of something else.
I'd also like a brief comment about why this is useful, i.e., why you need this and what you're going to use it for.
The usual commit citation format is 7d9c1d2f7aca ("USB: add support for dev_groups to struct usb_device_driver") so it's easier to locate the commit.
I see there is also b71b283e3d6d ("USB: add support for dev_groups to struct usb_driver"). I don't know enough about USB to know whether 7d9c1d2f7aca or b71b283e3d6d is a closer analogue to what you're doing here, but I do see that struct usb_driver is far more common than struct usb_device_driver.
PCI has struct pci_driver, but doesn't have the concept of a struct pci_device_driver.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com Suggested-by: Greg Kroah-Hartman gregkh@linuxfoundation.org
drivers/pci/pci-driver.c | 1 + include/linux/pci.h | 3 +++ 2 files changed, 4 insertions(+)
diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index ec44a79e951a..3a72352aa5cf 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -1385,6 +1385,7 @@ int __pci_register_driver(struct pci_driver *drv, struct module *owner, drv->driver.owner = owner; drv->driver.mod_name = mod_name; drv->driver.groups = drv->groups;
drv->driver.dev_groups = drv->dev_groups;
spin_lock_init(&drv->dynids.lock); INIT_LIST_HEAD(&drv->dynids.list);
diff --git a/include/linux/pci.h b/include/linux/pci.h index 86c799c97b77..b57755b03009 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -858,6 +858,8 @@ struct module;
number of VFs to enable via sysfs "sriov_numvfs" file.- @err_handler: See Documentation/PCI/pci-error-recovery.rst
- @groups: Sysfs attribute groups.
- @dev_groups: Attributes attached to the device that will be
*/
created once it is bound to the driver.- @driver: Driver model structure.
- @dynids: List of dynamically added device IDs.
@@ -873,6 +875,7 @@ struct pci_driver { int (*sriov_configure)(struct pci_dev *dev, int num_vfs); /* On PF */ const struct pci_error_handlers *err_handler; const struct attribute_group **groups;
- const struct attribute_group **dev_groups; struct device_driver driver; struct pci_dynids dynids;
};
2.25.1
On 2021-04-28 12:53 p.m., Bjorn Helgaas wrote:
In subject:
s/PCI: add support/PCI: Add support/
to match convention ("git log --oneline drivers/pci/pci-driver.c" to learn this).
On Wed, Apr 28, 2021 at 11:11:48AM -0400, Andrey Grodzovsky wrote:
This is exact copy of 'USB: add support for dev_groups to struct usb_device_driver' patch by Greg but just for the PCI case.
Ideally this would be an imperative sentence telling us what the patch *does*, not just that it's a copy of something else.
I'd also like a brief comment about why this is useful, i.e., why you need this and what you're going to use it for.
The usual commit citation format is 7d9c1d2f7aca ("USB: add support for dev_groups to struct usb_device_driver") so it's easier to locate the commit.
I see there is also b71b283e3d6d ("USB: add support for dev_groups to struct usb_driver"). I don't know enough about USB to know whether 7d9c1d2f7aca or b71b283e3d6d is a closer analogue to what you're doing here, but I do see that struct usb_driver is far more common than struct usb_device_driver.
PCI has struct pci_driver, but doesn't have the concept of a struct pci_device_driver.
Since we don't have pci_device_driver then pci_driver is the best place for it then, no ?
Andrey
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com Suggested-by: Greg Kroah-Hartman gregkh@linuxfoundation.org
drivers/pci/pci-driver.c | 1 + include/linux/pci.h | 3 +++ 2 files changed, 4 insertions(+)
diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index ec44a79e951a..3a72352aa5cf 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -1385,6 +1385,7 @@ int __pci_register_driver(struct pci_driver *drv, struct module *owner, drv->driver.owner = owner; drv->driver.mod_name = mod_name; drv->driver.groups = drv->groups;
drv->driver.dev_groups = drv->dev_groups;
spin_lock_init(&drv->dynids.lock); INIT_LIST_HEAD(&drv->dynids.list);
diff --git a/include/linux/pci.h b/include/linux/pci.h index 86c799c97b77..b57755b03009 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -858,6 +858,8 @@ struct module;
number of VFs to enable via sysfs "sriov_numvfs" file.- @err_handler: See Documentation/PCI/pci-error-recovery.rst
- @groups: Sysfs attribute groups.
- @dev_groups: Attributes attached to the device that will be
*/
created once it is bound to the driver.- @driver: Driver model structure.
- @dynids: List of dynamically added device IDs.
@@ -873,6 +875,7 @@ struct pci_driver { int (*sriov_configure)(struct pci_dev *dev, int num_vfs); /* On PF */ const struct pci_error_handlers *err_handler; const struct attribute_group **groups;
- const struct attribute_group **dev_groups; struct device_driver driver; struct pci_dynids dynids; };
-- 2.25.1
On Thu, Apr 29, 2021 at 12:53:15PM -0400, Andrey Grodzovsky wrote:
On 2021-04-28 12:53 p.m., Bjorn Helgaas wrote:
On Wed, Apr 28, 2021 at 11:11:48AM -0400, Andrey Grodzovsky wrote:
This is exact copy of 'USB: add support for dev_groups to struct usb_device_driver' patch by Greg but just for the PCI case.
... The usual commit citation format is 7d9c1d2f7aca ("USB: add support for dev_groups to struct usb_device_driver") so it's easier to locate the commit.
I see there is also b71b283e3d6d ("USB: add support for dev_groups to struct usb_driver"). I don't know enough about USB to know whether 7d9c1d2f7aca or b71b283e3d6d is a closer analogue to what you're doing here, but I do see that struct usb_driver is far more common than struct usb_device_driver.
PCI has struct pci_driver, but doesn't have the concept of a struct pci_device_driver.
Since we don't have pci_device_driver then pci_driver is the best place for it then, no ?
Of course. My point was just that maybe you should say this is similar to b71b283e3d6d ("USB: add support for dev_groups to struct usb_driver"), not similar to 7d9c1d2f7aca ("USB: add support for dev_groups to struct usb_device_driver").
Bjorn
On 2021-04-29 3:23 p.m., Bjorn Helgaas wrote:
On Thu, Apr 29, 2021 at 12:53:15PM -0400, Andrey Grodzovsky wrote:
On 2021-04-28 12:53 p.m., Bjorn Helgaas wrote:
On Wed, Apr 28, 2021 at 11:11:48AM -0400, Andrey Grodzovsky wrote:
This is exact copy of 'USB: add support for dev_groups to struct usb_device_driver' patch by Greg but just for the PCI case.
... The usual commit citation format is 7d9c1d2f7aca ("USB: add support for dev_groups to struct usb_device_driver") so it's easier to locate the commit.
I see there is also b71b283e3d6d ("USB: add support for dev_groups to struct usb_driver"). I don't know enough about USB to know whether 7d9c1d2f7aca or b71b283e3d6d is a closer analogue to what you're doing here, but I do see that struct usb_driver is far more common than struct usb_device_driver.
PCI has struct pci_driver, but doesn't have the concept of a struct pci_device_driver.
Since we don't have pci_device_driver then pci_driver is the best place for it then, no ?
Of course. My point was just that maybe you should say this is similar to b71b283e3d6d ("USB: add support for dev_groups to struct usb_driver"), not similar to 7d9c1d2f7aca ("USB: add support for dev_groups to struct usb_device_driver").
Got it.
Andrey
Bjorn _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.free...
This allows to remove explicit creation and destruction of those attrs and by this avoids warnings on device finilizing post physical device extraction.
v5: Use newly added pci_driver.dev_groups directly
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c | 17 ++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 13 ++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | 25 ++++++++------------ drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 14 ++++------- 4 files changed, 37 insertions(+), 32 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c index 86add0f4ea4d..0346e124ab8c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c @@ -1953,6 +1953,15 @@ static ssize_t amdgpu_atombios_get_vbios_version(struct device *dev, static DEVICE_ATTR(vbios_version, 0444, amdgpu_atombios_get_vbios_version, NULL);
+static struct attribute *amdgpu_vbios_version_attrs[] = { + &dev_attr_vbios_version.attr, + NULL +}; + +const struct attribute_group amdgpu_vbios_version_attr_group = { + .attrs = amdgpu_vbios_version_attrs +}; + /** * amdgpu_atombios_fini - free the driver info and callbacks for atombios * @@ -1972,7 +1981,6 @@ void amdgpu_atombios_fini(struct amdgpu_device *adev) adev->mode_info.atom_context = NULL; kfree(adev->mode_info.atom_card_info); adev->mode_info.atom_card_info = NULL; - device_remove_file(adev->dev, &dev_attr_vbios_version); }
/** @@ -1989,7 +1997,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev) { struct card_info *atom_card_info = kzalloc(sizeof(struct card_info), GFP_KERNEL); - int ret;
if (!atom_card_info) return -ENOMEM; @@ -2027,12 +2034,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev) amdgpu_atombios_allocate_fb_scratch(adev); }
- ret = device_create_file(adev->dev, &dev_attr_vbios_version); - if (ret) { - DRM_ERROR("Failed to create device file for VBIOS version\n"); - return ret; - } - return 0; }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 54cb5ee2f563..f799c40d7e72 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1605,6 +1605,18 @@ static struct pci_error_handlers amdgpu_pci_err_handler = { .resume = amdgpu_pci_resume, };
+extern const struct attribute_group amdgpu_vram_mgr_attr_group; +extern const struct attribute_group amdgpu_gtt_mgr_attr_group; +extern const struct attribute_group amdgpu_vbios_version_attr_group; + +static const struct attribute_group *amdgpu_sysfs_groups[] = { + &amdgpu_vram_mgr_attr_group, + &amdgpu_gtt_mgr_attr_group, + &amdgpu_vbios_version_attr_group, + NULL, +}; + + static struct pci_driver amdgpu_kms_pci_driver = { .name = DRIVER_NAME, .id_table = pciidlist, @@ -1613,6 +1625,7 @@ static struct pci_driver amdgpu_kms_pci_driver = { .shutdown = amdgpu_pci_shutdown, .driver.pm = &amdgpu_pm_ops, .err_handler = &amdgpu_pci_err_handler, + .dev_groups = amdgpu_sysfs_groups, };
static int __init amdgpu_init(void) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c index 8980329cded0..3b7150e1c5ed 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c @@ -77,6 +77,16 @@ static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO, static DEVICE_ATTR(mem_info_gtt_used, S_IRUGO, amdgpu_mem_info_gtt_used_show, NULL);
+static struct attribute *amdgpu_gtt_mgr_attributes[] = { + &dev_attr_mem_info_gtt_total.attr, + &dev_attr_mem_info_gtt_used.attr, + NULL +}; + +const struct attribute_group amdgpu_gtt_mgr_attr_group = { + .attrs = amdgpu_gtt_mgr_attributes +}; + static const struct ttm_resource_manager_func amdgpu_gtt_mgr_func; /** * amdgpu_gtt_mgr_init - init GTT manager and DRM MM @@ -91,7 +101,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) struct amdgpu_gtt_mgr *mgr = &adev->mman.gtt_mgr; struct ttm_resource_manager *man = &mgr->manager; uint64_t start, size; - int ret;
man->use_tt = true; man->func = &amdgpu_gtt_mgr_func; @@ -104,17 +113,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) spin_lock_init(&mgr->lock); atomic64_set(&mgr->available, gtt_size >> PAGE_SHIFT);
- ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_total); - if (ret) { - DRM_ERROR("Failed to create device file mem_info_gtt_total\n"); - return ret; - } - ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_used); - if (ret) { - DRM_ERROR("Failed to create device file mem_info_gtt_used\n"); - return ret; - } - ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, &mgr->manager); ttm_resource_manager_set_used(man, true); return 0; @@ -144,9 +142,6 @@ void amdgpu_gtt_mgr_fini(struct amdgpu_device *adev) drm_mm_takedown(&mgr->mm); spin_unlock(&mgr->lock);
- device_remove_file(adev->dev, &dev_attr_mem_info_gtt_total); - device_remove_file(adev->dev, &dev_attr_mem_info_gtt_used); - ttm_resource_manager_cleanup(man); ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, NULL); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index c89b66bb70e2..68369b38aebb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -154,7 +154,7 @@ static DEVICE_ATTR(mem_info_vis_vram_used, S_IRUGO, static DEVICE_ATTR(mem_info_vram_vendor, S_IRUGO, amdgpu_mem_info_vram_vendor, NULL);
-static const struct attribute *amdgpu_vram_mgr_attributes[] = { +static struct attribute *amdgpu_vram_mgr_attributes[] = { &dev_attr_mem_info_vram_total.attr, &dev_attr_mem_info_vis_vram_total.attr, &dev_attr_mem_info_vram_used.attr, @@ -163,6 +163,10 @@ static const struct attribute *amdgpu_vram_mgr_attributes[] = { NULL };
+const struct attribute_group amdgpu_vram_mgr_attr_group = { + .attrs = amdgpu_vram_mgr_attributes +}; + static const struct ttm_resource_manager_func amdgpu_vram_mgr_func;
/** @@ -176,7 +180,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) { struct amdgpu_vram_mgr *mgr = &adev->mman.vram_mgr; struct ttm_resource_manager *man = &mgr->manager; - int ret;
ttm_resource_manager_init(man, adev->gmc.real_vram_size >> PAGE_SHIFT);
@@ -187,11 +190,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) INIT_LIST_HEAD(&mgr->reservations_pending); INIT_LIST_HEAD(&mgr->reserved_pages);
- /* Add the two VRAM-related sysfs files */ - ret = sysfs_create_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes); - if (ret) - DRM_ERROR("Failed to register sysfs\n"); - ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, &mgr->manager); ttm_resource_manager_set_used(man, true); return 0; @@ -229,8 +227,6 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev) drm_mm_takedown(&mgr->mm); spin_unlock(&mgr->lock);
- sysfs_remove_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes); - ttm_resource_manager_cleanup(man); ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, NULL); }
In subject,
s/dmr/drm/ s/Move some/Move/ ("some" consumes space without adding meaning)
Or maybe something like:
drm/amdgpu: Convert driver sysfs attributes to static attributes
On Wed, Apr 28, 2021 at 11:11:49AM -0400, Andrey Grodzovsky wrote:
This allows to remove explicit creation and destruction of those attrs and by this avoids warnings on device finilizing post physical device extraction.
s/finilizing/finalizing/
v5: Use newly added pci_driver.dev_groups directly
I don't know the DRM convention, but IMO, change notes like "v5: Use ..." can go after "---" so they don't go in the git log. To me, they're useful during review, but not after being merged.
I love the patch! Much cleaner than creating/removing all these attributes explicitly.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c | 17 ++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 13 ++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | 25 ++++++++------------ drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 14 ++++------- 4 files changed, 37 insertions(+), 32 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c index 86add0f4ea4d..0346e124ab8c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c @@ -1953,6 +1953,15 @@ static ssize_t amdgpu_atombios_get_vbios_version(struct device *dev, static DEVICE_ATTR(vbios_version, 0444, amdgpu_atombios_get_vbios_version, NULL);
+static struct attribute *amdgpu_vbios_version_attrs[] = {
- &dev_attr_vbios_version.attr,
- NULL
+};
+const struct attribute_group amdgpu_vbios_version_attr_group = {
- .attrs = amdgpu_vbios_version_attrs
+};
/**
- amdgpu_atombios_fini - free the driver info and callbacks for atombios
@@ -1972,7 +1981,6 @@ void amdgpu_atombios_fini(struct amdgpu_device *adev) adev->mode_info.atom_context = NULL; kfree(adev->mode_info.atom_card_info); adev->mode_info.atom_card_info = NULL;
- device_remove_file(adev->dev, &dev_attr_vbios_version);
}
/** @@ -1989,7 +1997,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev) { struct card_info *atom_card_info = kzalloc(sizeof(struct card_info), GFP_KERNEL);
int ret;
if (!atom_card_info) return -ENOMEM;
@@ -2027,12 +2034,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev) amdgpu_atombios_allocate_fb_scratch(adev); }
- ret = device_create_file(adev->dev, &dev_attr_vbios_version);
- if (ret) {
DRM_ERROR("Failed to create device file for VBIOS version\n");return ret;- }
- return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 54cb5ee2f563..f799c40d7e72 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1605,6 +1605,18 @@ static struct pci_error_handlers amdgpu_pci_err_handler = { .resume = amdgpu_pci_resume, };
+extern const struct attribute_group amdgpu_vram_mgr_attr_group; +extern const struct attribute_group amdgpu_gtt_mgr_attr_group; +extern const struct attribute_group amdgpu_vbios_version_attr_group;
+static const struct attribute_group *amdgpu_sysfs_groups[] = {
- &amdgpu_vram_mgr_attr_group,
- &amdgpu_gtt_mgr_attr_group,
- &amdgpu_vbios_version_attr_group,
- NULL,
+};
static struct pci_driver amdgpu_kms_pci_driver = { .name = DRIVER_NAME, .id_table = pciidlist, @@ -1613,6 +1625,7 @@ static struct pci_driver amdgpu_kms_pci_driver = { .shutdown = amdgpu_pci_shutdown, .driver.pm = &amdgpu_pm_ops, .err_handler = &amdgpu_pci_err_handler,
- .dev_groups = amdgpu_sysfs_groups,
};
static int __init amdgpu_init(void) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c index 8980329cded0..3b7150e1c5ed 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c @@ -77,6 +77,16 @@ static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO, static DEVICE_ATTR(mem_info_gtt_used, S_IRUGO, amdgpu_mem_info_gtt_used_show, NULL);
+static struct attribute *amdgpu_gtt_mgr_attributes[] = {
- &dev_attr_mem_info_gtt_total.attr,
- &dev_attr_mem_info_gtt_used.attr,
- NULL
+};
+const struct attribute_group amdgpu_gtt_mgr_attr_group = {
- .attrs = amdgpu_gtt_mgr_attributes
+};
static const struct ttm_resource_manager_func amdgpu_gtt_mgr_func; /**
- amdgpu_gtt_mgr_init - init GTT manager and DRM MM
@@ -91,7 +101,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) struct amdgpu_gtt_mgr *mgr = &adev->mman.gtt_mgr; struct ttm_resource_manager *man = &mgr->manager; uint64_t start, size;
int ret;
man->use_tt = true; man->func = &amdgpu_gtt_mgr_func;
@@ -104,17 +113,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) spin_lock_init(&mgr->lock); atomic64_set(&mgr->available, gtt_size >> PAGE_SHIFT);
- ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_total);
- if (ret) {
DRM_ERROR("Failed to create device file mem_info_gtt_total\n");return ret;- }
- ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_used);
- if (ret) {
DRM_ERROR("Failed to create device file mem_info_gtt_used\n");return ret;- }
- ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, &mgr->manager); ttm_resource_manager_set_used(man, true); return 0;
@@ -144,9 +142,6 @@ void amdgpu_gtt_mgr_fini(struct amdgpu_device *adev) drm_mm_takedown(&mgr->mm); spin_unlock(&mgr->lock);
- device_remove_file(adev->dev, &dev_attr_mem_info_gtt_total);
- device_remove_file(adev->dev, &dev_attr_mem_info_gtt_used);
- ttm_resource_manager_cleanup(man); ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, NULL);
} diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index c89b66bb70e2..68369b38aebb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -154,7 +154,7 @@ static DEVICE_ATTR(mem_info_vis_vram_used, S_IRUGO, static DEVICE_ATTR(mem_info_vram_vendor, S_IRUGO, amdgpu_mem_info_vram_vendor, NULL);
-static const struct attribute *amdgpu_vram_mgr_attributes[] = { +static struct attribute *amdgpu_vram_mgr_attributes[] = { &dev_attr_mem_info_vram_total.attr, &dev_attr_mem_info_vis_vram_total.attr, &dev_attr_mem_info_vram_used.attr, @@ -163,6 +163,10 @@ static const struct attribute *amdgpu_vram_mgr_attributes[] = { NULL };
+const struct attribute_group amdgpu_vram_mgr_attr_group = {
- .attrs = amdgpu_vram_mgr_attributes
+};
static const struct ttm_resource_manager_func amdgpu_vram_mgr_func;
/** @@ -176,7 +180,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) { struct amdgpu_vram_mgr *mgr = &adev->mman.vram_mgr; struct ttm_resource_manager *man = &mgr->manager;
int ret;
ttm_resource_manager_init(man, adev->gmc.real_vram_size >> PAGE_SHIFT);
@@ -187,11 +190,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) INIT_LIST_HEAD(&mgr->reservations_pending); INIT_LIST_HEAD(&mgr->reserved_pages);
- /* Add the two VRAM-related sysfs files */
- ret = sysfs_create_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes);
- if (ret)
DRM_ERROR("Failed to register sysfs\n");- ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, &mgr->manager); ttm_resource_manager_set_used(man, true); return 0;
@@ -229,8 +227,6 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev) drm_mm_takedown(&mgr->mm); spin_unlock(&mgr->lock);
- sysfs_remove_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes);
- ttm_resource_manager_cleanup(man); ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, NULL);
}
2.25.1
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
This allows to remove explicit creation and destruction of those attrs and by this avoids warnings on device finilizing post physical device extraction.
v5: Use newly added pci_driver.dev_groups directly
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
Looks like a nice cleanup to me, but this is beyond my understand of sysfs.
Acked-by: Christian König christian.koenig@amd.com
Christian.
drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c | 17 ++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 13 ++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | 25 ++++++++------------ drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 14 ++++------- 4 files changed, 37 insertions(+), 32 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c index 86add0f4ea4d..0346e124ab8c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c @@ -1953,6 +1953,15 @@ static ssize_t amdgpu_atombios_get_vbios_version(struct device *dev, static DEVICE_ATTR(vbios_version, 0444, amdgpu_atombios_get_vbios_version, NULL);
+static struct attribute *amdgpu_vbios_version_attrs[] = {
- &dev_attr_vbios_version.attr,
- NULL
+};
+const struct attribute_group amdgpu_vbios_version_attr_group = {
- .attrs = amdgpu_vbios_version_attrs
+};
- /**
- amdgpu_atombios_fini - free the driver info and callbacks for atombios
@@ -1972,7 +1981,6 @@ void amdgpu_atombios_fini(struct amdgpu_device *adev) adev->mode_info.atom_context = NULL; kfree(adev->mode_info.atom_card_info); adev->mode_info.atom_card_info = NULL;
device_remove_file(adev->dev, &dev_attr_vbios_version); }
/**
@@ -1989,7 +1997,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev) { struct card_info *atom_card_info = kzalloc(sizeof(struct card_info), GFP_KERNEL);
int ret;
if (!atom_card_info) return -ENOMEM;
@@ -2027,12 +2034,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev) amdgpu_atombios_allocate_fb_scratch(adev); }
- ret = device_create_file(adev->dev, &dev_attr_vbios_version);
- if (ret) {
DRM_ERROR("Failed to create device file for VBIOS version\n");return ret;- }
- return 0; }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 54cb5ee2f563..f799c40d7e72 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1605,6 +1605,18 @@ static struct pci_error_handlers amdgpu_pci_err_handler = { .resume = amdgpu_pci_resume, };
+extern const struct attribute_group amdgpu_vram_mgr_attr_group; +extern const struct attribute_group amdgpu_gtt_mgr_attr_group; +extern const struct attribute_group amdgpu_vbios_version_attr_group;
+static const struct attribute_group *amdgpu_sysfs_groups[] = {
- &amdgpu_vram_mgr_attr_group,
- &amdgpu_gtt_mgr_attr_group,
- &amdgpu_vbios_version_attr_group,
- NULL,
+};
- static struct pci_driver amdgpu_kms_pci_driver = { .name = DRIVER_NAME, .id_table = pciidlist,
@@ -1613,6 +1625,7 @@ static struct pci_driver amdgpu_kms_pci_driver = { .shutdown = amdgpu_pci_shutdown, .driver.pm = &amdgpu_pm_ops, .err_handler = &amdgpu_pci_err_handler,
.dev_groups = amdgpu_sysfs_groups, };
static int __init amdgpu_init(void)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c index 8980329cded0..3b7150e1c5ed 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c @@ -77,6 +77,16 @@ static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO, static DEVICE_ATTR(mem_info_gtt_used, S_IRUGO, amdgpu_mem_info_gtt_used_show, NULL);
+static struct attribute *amdgpu_gtt_mgr_attributes[] = {
- &dev_attr_mem_info_gtt_total.attr,
- &dev_attr_mem_info_gtt_used.attr,
- NULL
+};
+const struct attribute_group amdgpu_gtt_mgr_attr_group = {
- .attrs = amdgpu_gtt_mgr_attributes
+};
- static const struct ttm_resource_manager_func amdgpu_gtt_mgr_func; /**
- amdgpu_gtt_mgr_init - init GTT manager and DRM MM
@@ -91,7 +101,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) struct amdgpu_gtt_mgr *mgr = &adev->mman.gtt_mgr; struct ttm_resource_manager *man = &mgr->manager; uint64_t start, size;
int ret;
man->use_tt = true; man->func = &amdgpu_gtt_mgr_func;
@@ -104,17 +113,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) spin_lock_init(&mgr->lock); atomic64_set(&mgr->available, gtt_size >> PAGE_SHIFT);
- ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_total);
- if (ret) {
DRM_ERROR("Failed to create device file mem_info_gtt_total\n");return ret;- }
- ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_used);
- if (ret) {
DRM_ERROR("Failed to create device file mem_info_gtt_used\n");return ret;- }
- ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, &mgr->manager); ttm_resource_manager_set_used(man, true); return 0;
@@ -144,9 +142,6 @@ void amdgpu_gtt_mgr_fini(struct amdgpu_device *adev) drm_mm_takedown(&mgr->mm); spin_unlock(&mgr->lock);
- device_remove_file(adev->dev, &dev_attr_mem_info_gtt_total);
- device_remove_file(adev->dev, &dev_attr_mem_info_gtt_used);
- ttm_resource_manager_cleanup(man); ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, NULL); }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index c89b66bb70e2..68369b38aebb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -154,7 +154,7 @@ static DEVICE_ATTR(mem_info_vis_vram_used, S_IRUGO, static DEVICE_ATTR(mem_info_vram_vendor, S_IRUGO, amdgpu_mem_info_vram_vendor, NULL);
-static const struct attribute *amdgpu_vram_mgr_attributes[] = { +static struct attribute *amdgpu_vram_mgr_attributes[] = { &dev_attr_mem_info_vram_total.attr, &dev_attr_mem_info_vis_vram_total.attr, &dev_attr_mem_info_vram_used.attr, @@ -163,6 +163,10 @@ static const struct attribute *amdgpu_vram_mgr_attributes[] = { NULL };
+const struct attribute_group amdgpu_vram_mgr_attr_group = {
- .attrs = amdgpu_vram_mgr_attributes
+};
static const struct ttm_resource_manager_func amdgpu_vram_mgr_func;
/**
@@ -176,7 +180,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) { struct amdgpu_vram_mgr *mgr = &adev->mman.vram_mgr; struct ttm_resource_manager *man = &mgr->manager;
int ret;
ttm_resource_manager_init(man, adev->gmc.real_vram_size >> PAGE_SHIFT);
@@ -187,11 +190,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) INIT_LIST_HEAD(&mgr->reservations_pending); INIT_LIST_HEAD(&mgr->reserved_pages);
- /* Add the two VRAM-related sysfs files */
- ret = sysfs_create_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes);
- if (ret)
DRM_ERROR("Failed to register sysfs\n");- ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, &mgr->manager); ttm_resource_manager_set_used(man, true); return 0;
@@ -229,8 +227,6 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev) drm_mm_takedown(&mgr->mm); spin_unlock(&mgr->lock);
- sysfs_remove_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes);
- ttm_resource_manager_cleanup(man); ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, NULL); }
This should prevent writing to memory or IO ranges possibly already allocated for other uses after our device is removed.
v5: Protect more places wher memcopy_to/form_io takes place Protect IB submissions
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 75 +++--- drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 9 + drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c | 228 +++++++++--------- drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c | 115 +++++---- drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h | 3 + drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 70 ++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 49 +--- drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c | 31 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c | 11 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c | 22 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 +- drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 44 ++-- drivers/gpu/drm/amd/amdgpu/psp_v12_0.c | 8 +- drivers/gpu/drm/amd/amdgpu/psp_v3_1.c | 8 +- drivers/gpu/drm/amd/amdgpu/vce_v4_0.c | 26 +- drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c | 22 +- .../drm/amd/pm/powerplay/smumgr/smu7_smumgr.c | 2 + 17 files changed, 425 insertions(+), 305 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 91594ddc2459..22b09c4db255 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -71,6 +71,8 @@ #include <linux/pm_runtime.h>
+#include <drm/drm_drv.h> + MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin"); @@ -279,48 +281,55 @@ void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, unsigned long flags; uint32_t hi = ~0; uint64_t last; + int idx;
+ if (drm_dev_enter(&adev->ddev, &idx)) {
#ifdef CONFIG_64BIT - last = min(pos + size, adev->gmc.visible_vram_size); - if (last > pos) { - void __iomem *addr = adev->mman.aper_base_kaddr + pos; - size_t count = last - pos; - - if (write) { - memcpy_toio(addr, buf, count); - mb(); - amdgpu_asic_flush_hdp(adev, NULL); - } else { - amdgpu_asic_invalidate_hdp(adev, NULL); - mb(); - memcpy_fromio(buf, addr, count); - } + last = min(pos + size, adev->gmc.visible_vram_size); + if (last > pos) { + void __iomem *addr = adev->mman.aper_base_kaddr + pos; + size_t count = last - pos; + + if (write) { + memcpy_toio(addr, buf, count); + mb(); + amdgpu_asic_flush_hdp(adev, NULL); + } else { + amdgpu_asic_invalidate_hdp(adev, NULL); + mb(); + memcpy_fromio(buf, addr, count); + }
- if (count == size) - return; + if (count == size) { + drm_dev_exit(idx); + return; + }
- pos += count; - buf += count / 4; - size -= count; - } + pos += count; + buf += count / 4; + size -= count; + } #endif
- spin_lock_irqsave(&adev->mmio_idx_lock, flags); - for (last = pos + size; pos < last; pos += 4) { - uint32_t tmp = pos >> 31; + spin_lock_irqsave(&adev->mmio_idx_lock, flags); + for (last = pos + size; pos < last; pos += 4) { + uint32_t tmp = pos >> 31;
- WREG32_NO_KIQ(mmMM_INDEX, ((uint32_t)pos) | 0x80000000); - if (tmp != hi) { - WREG32_NO_KIQ(mmMM_INDEX_HI, tmp); - hi = tmp; + WREG32_NO_KIQ(mmMM_INDEX, ((uint32_t)pos) | 0x80000000); + if (tmp != hi) { + WREG32_NO_KIQ(mmMM_INDEX_HI, tmp); + hi = tmp; + } + if (write) + WREG32_NO_KIQ(mmMM_DATA, *buf++); + else + *buf++ = RREG32_NO_KIQ(mmMM_DATA); } - if (write) - WREG32_NO_KIQ(mmMM_DATA, *buf++); - else - *buf++ = RREG32_NO_KIQ(mmMM_DATA); + spin_unlock_irqrestore(&adev->mmio_idx_lock, flags); + + drm_dev_exit(idx); } - spin_unlock_irqrestore(&adev->mmio_idx_lock, flags); }
/* @@ -402,6 +411,7 @@ uint8_t amdgpu_mm_rreg8(struct amdgpu_device *adev, uint32_t offset) */ void amdgpu_mm_wreg8(struct amdgpu_device *adev, uint32_t offset, uint8_t value) { + if (adev->in_pci_err_recovery) return;
@@ -542,6 +552,7 @@ u32 amdgpu_mm_rdoorbell(struct amdgpu_device *adev, u32 index) */ void amdgpu_mm_wdoorbell(struct amdgpu_device *adev, u32 index, u32 v) { + if (adev->in_pci_err_recovery) return;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c index fe1a39ffda72..1beb4e64b884 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c @@ -31,6 +31,8 @@ #include "amdgpu_ras.h" #include "amdgpu_xgmi.h"
+#include <drm/drm_drv.h> + /** * amdgpu_gmc_get_pde_for_bo - get the PDE for a BO * @@ -98,6 +100,10 @@ int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr, { void __iomem *ptr = (void *)cpu_pt_addr; uint64_t value; + int idx; + + if (!drm_dev_enter(&adev->ddev, &idx)) + return 0;
/* * The following is for PTE only. GART does not have PDEs. @@ -105,6 +111,9 @@ int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr, value = addr & 0x0000FFFFFFFFF000ULL; value |= flags; writeq(value, ptr + (gpu_page_idx * 8)); + + drm_dev_exit(idx); + return 0; }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c index 7645223ea0ef..b3a1ff04c10f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c @@ -31,6 +31,7 @@
#include <drm/amdgpu_drm.h> #include <drm/drm_debugfs.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "atom.h" @@ -138,7 +139,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs, bool secure;
unsigned i; - int r = 0; + int idx, r = 0; bool need_pipe_sync = false;
if (num_ibs == 0) @@ -170,142 +171,151 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs, return -EINVAL; }
- alloc_size = ring->funcs->emit_frame_size + num_ibs * - ring->funcs->emit_ib_size; + if (drm_dev_enter(&adev->ddev, &idx)) {
- r = amdgpu_ring_alloc(ring, alloc_size); - if (r) { - dev_err(adev->dev, "scheduling IB failed (%d).\n", r); - return r; - } - - need_ctx_switch = ring->current_ctx != fence_ctx; - if (ring->funcs->emit_pipeline_sync && job && - ((tmp = amdgpu_sync_get_fence(&job->sched_sync)) || - (amdgpu_sriov_vf(adev) && need_ctx_switch) || - amdgpu_vm_need_pipeline_sync(ring, job))) { - need_pipe_sync = true; - - if (tmp) - trace_amdgpu_ib_pipe_sync(job, tmp); - - dma_fence_put(tmp); - } + alloc_size = ring->funcs->emit_frame_size + num_ibs * + ring->funcs->emit_ib_size;
- if ((ib->flags & AMDGPU_IB_FLAG_EMIT_MEM_SYNC) && ring->funcs->emit_mem_sync) - ring->funcs->emit_mem_sync(ring); + r = amdgpu_ring_alloc(ring, alloc_size); + if (r) { + dev_err(adev->dev, "scheduling IB failed (%d).\n", r); + goto exit; + }
- if (ring->funcs->emit_wave_limit && - ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH) - ring->funcs->emit_wave_limit(ring, true); + need_ctx_switch = ring->current_ctx != fence_ctx; + if (ring->funcs->emit_pipeline_sync && job && + ((tmp = amdgpu_sync_get_fence(&job->sched_sync)) || + (amdgpu_sriov_vf(adev) && need_ctx_switch) || + amdgpu_vm_need_pipeline_sync(ring, job))) { + need_pipe_sync = true;
- if (ring->funcs->insert_start) - ring->funcs->insert_start(ring); + if (tmp) + trace_amdgpu_ib_pipe_sync(job, tmp);
- if (job) { - r = amdgpu_vm_flush(ring, job, need_pipe_sync); - if (r) { - amdgpu_ring_undo(ring); - return r; + dma_fence_put(tmp); } - }
- if (job && ring->funcs->init_cond_exec) - patch_offset = amdgpu_ring_init_cond_exec(ring); + if ((ib->flags & AMDGPU_IB_FLAG_EMIT_MEM_SYNC) && ring->funcs->emit_mem_sync) + ring->funcs->emit_mem_sync(ring);
-#ifdef CONFIG_X86_64 - if (!(adev->flags & AMD_IS_APU)) -#endif - { - if (ring->funcs->emit_hdp_flush) - amdgpu_ring_emit_hdp_flush(ring); - else - amdgpu_asic_flush_hdp(adev, ring); - } + if (ring->funcs->emit_wave_limit && + ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH) + ring->funcs->emit_wave_limit(ring, true);
- if (need_ctx_switch) - status |= AMDGPU_HAVE_CTX_SWITCH; + if (ring->funcs->insert_start) + ring->funcs->insert_start(ring);
- skip_preamble = ring->current_ctx == fence_ctx; - if (job && ring->funcs->emit_cntxcntl) { - status |= job->preamble_status; - status |= job->preemption_status; - amdgpu_ring_emit_cntxcntl(ring, status); - } + if (job) { + r = amdgpu_vm_flush(ring, job, need_pipe_sync); + if (r) { + amdgpu_ring_undo(ring); + goto exit; + } + }
- /* Setup initial TMZiness and send it off. - */ - secure = false; - if (job && ring->funcs->emit_frame_cntl) { - secure = ib->flags & AMDGPU_IB_FLAGS_SECURE; - amdgpu_ring_emit_frame_cntl(ring, true, secure); - } + if (job && ring->funcs->init_cond_exec) + patch_offset = amdgpu_ring_init_cond_exec(ring); + + #ifdef CONFIG_X86_64 + if (!(adev->flags & AMD_IS_APU)) + #endif + { + if (ring->funcs->emit_hdp_flush) + amdgpu_ring_emit_hdp_flush(ring); + else + amdgpu_asic_flush_hdp(adev, ring); + }
- for (i = 0; i < num_ibs; ++i) { - ib = &ibs[i]; + if (need_ctx_switch) + status |= AMDGPU_HAVE_CTX_SWITCH;
- /* drop preamble IBs if we don't have a context switch */ - if ((ib->flags & AMDGPU_IB_FLAG_PREAMBLE) && - skip_preamble && - !(status & AMDGPU_PREAMBLE_IB_PRESENT_FIRST) && - !amdgpu_mcbp && - !amdgpu_sriov_vf(adev)) /* for SRIOV preemption, Preamble CE ib must be inserted anyway */ - continue; + skip_preamble = ring->current_ctx == fence_ctx; + if (job && ring->funcs->emit_cntxcntl) { + status |= job->preamble_status; + status |= job->preemption_status; + amdgpu_ring_emit_cntxcntl(ring, status); + }
+ /* Setup initial TMZiness and send it off. + */ + secure = false; if (job && ring->funcs->emit_frame_cntl) { - if (secure != !!(ib->flags & AMDGPU_IB_FLAGS_SECURE)) { - amdgpu_ring_emit_frame_cntl(ring, false, secure); - secure = !secure; - amdgpu_ring_emit_frame_cntl(ring, true, secure); + secure = ib->flags & AMDGPU_IB_FLAGS_SECURE; + amdgpu_ring_emit_frame_cntl(ring, true, secure); + } + + for (i = 0; i < num_ibs; ++i) { + ib = &ibs[i]; + + /* drop preamble IBs if we don't have a context switch */ + if ((ib->flags & AMDGPU_IB_FLAG_PREAMBLE) && + skip_preamble && + !(status & AMDGPU_PREAMBLE_IB_PRESENT_FIRST) && + !amdgpu_mcbp && + !amdgpu_sriov_vf(adev)) /* for SRIOV preemption, Preamble CE ib must be inserted anyway */ + continue; + + if (job && ring->funcs->emit_frame_cntl) { + if (secure != !!(ib->flags & AMDGPU_IB_FLAGS_SECURE)) { + amdgpu_ring_emit_frame_cntl(ring, false, secure); + secure = !secure; + amdgpu_ring_emit_frame_cntl(ring, true, secure); + } } + + amdgpu_ring_emit_ib(ring, job, ib, status); + status &= ~AMDGPU_HAVE_CTX_SWITCH; }
- amdgpu_ring_emit_ib(ring, job, ib, status); - status &= ~AMDGPU_HAVE_CTX_SWITCH; - } + if (job && ring->funcs->emit_frame_cntl) + amdgpu_ring_emit_frame_cntl(ring, false, secure);
- if (job && ring->funcs->emit_frame_cntl) - amdgpu_ring_emit_frame_cntl(ring, false, secure); + #ifdef CONFIG_X86_64 + if (!(adev->flags & AMD_IS_APU)) + #endif + amdgpu_asic_invalidate_hdp(adev, ring);
-#ifdef CONFIG_X86_64 - if (!(adev->flags & AMD_IS_APU)) -#endif - amdgpu_asic_invalidate_hdp(adev, ring); + if (ib->flags & AMDGPU_IB_FLAG_TC_WB_NOT_INVALIDATE) + fence_flags |= AMDGPU_FENCE_FLAG_TC_WB_ONLY;
- if (ib->flags & AMDGPU_IB_FLAG_TC_WB_NOT_INVALIDATE) - fence_flags |= AMDGPU_FENCE_FLAG_TC_WB_ONLY; + /* wrap the last IB with fence */ + if (job && job->uf_addr) { + amdgpu_ring_emit_fence(ring, job->uf_addr, job->uf_sequence, + fence_flags | AMDGPU_FENCE_FLAG_64BIT); + }
- /* wrap the last IB with fence */ - if (job && job->uf_addr) { - amdgpu_ring_emit_fence(ring, job->uf_addr, job->uf_sequence, - fence_flags | AMDGPU_FENCE_FLAG_64BIT); - } + r = amdgpu_fence_emit(ring, f, fence_flags); + if (r) { + dev_err(adev->dev, "failed to emit fence (%d)\n", r); + if (job && job->vmid) + amdgpu_vmid_reset(adev, ring->funcs->vmhub, job->vmid); + amdgpu_ring_undo(ring); + goto exit; + }
- r = amdgpu_fence_emit(ring, f, fence_flags); - if (r) { - dev_err(adev->dev, "failed to emit fence (%d)\n", r); - if (job && job->vmid) - amdgpu_vmid_reset(adev, ring->funcs->vmhub, job->vmid); - amdgpu_ring_undo(ring); - return r; - } + if (ring->funcs->insert_end) + ring->funcs->insert_end(ring);
- if (ring->funcs->insert_end) - ring->funcs->insert_end(ring); + if (patch_offset != ~0 && ring->funcs->patch_cond_exec) + amdgpu_ring_patch_cond_exec(ring, patch_offset);
- if (patch_offset != ~0 && ring->funcs->patch_cond_exec) - amdgpu_ring_patch_cond_exec(ring, patch_offset); + ring->current_ctx = fence_ctx; + if (vm && ring->funcs->emit_switch_buffer) + amdgpu_ring_emit_switch_buffer(ring);
- ring->current_ctx = fence_ctx; - if (vm && ring->funcs->emit_switch_buffer) - amdgpu_ring_emit_switch_buffer(ring); + if (ring->funcs->emit_wave_limit && + ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH) + ring->funcs->emit_wave_limit(ring, false);
- if (ring->funcs->emit_wave_limit && - ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH) - ring->funcs->emit_wave_limit(ring, false); + amdgpu_ring_commit(ring);
- amdgpu_ring_commit(ring); - return 0; + } else { + return -ENODEV; + } + +exit: + drm_dev_exit(idx); + return r; }
/** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c index 839917eb7bc3..638b7fd1857c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c @@ -25,6 +25,7 @@
#include <linux/firmware.h> #include <linux/dma-mapping.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "amdgpu_psp.h" @@ -38,6 +39,8 @@ #include "amdgpu_ras.h" #include "amdgpu_securedisplay.h"
+#include <drm/drm_drv.h> + static int psp_sysfs_init(struct amdgpu_device *adev); static void psp_sysfs_fini(struct amdgpu_device *adev);
@@ -249,7 +252,7 @@ psp_cmd_submit_buf(struct psp_context *psp, struct psp_gfx_cmd_resp *cmd, uint64_t fence_mc_addr) { int ret; - int index; + int index, idx; int timeout = 20000; bool ras_intr = false; bool skip_unsupport = false; @@ -257,6 +260,9 @@ psp_cmd_submit_buf(struct psp_context *psp, if (psp->adev->in_pci_err_recovery) return 0;
+ if (!drm_dev_enter(&psp->adev->ddev, &idx)) + return 0; + mutex_lock(&psp->mutex);
memset(psp->cmd_buf_mem, 0, PSP_CMD_BUFFER_SIZE); @@ -267,8 +273,7 @@ psp_cmd_submit_buf(struct psp_context *psp, ret = psp_ring_cmd_submit(psp, psp->cmd_buf_mc_addr, fence_mc_addr, index); if (ret) { atomic_dec(&psp->fence_value); - mutex_unlock(&psp->mutex); - return ret; + goto exit; }
amdgpu_asic_invalidate_hdp(psp->adev, NULL); @@ -308,8 +313,8 @@ psp_cmd_submit_buf(struct psp_context *psp, psp->cmd_buf_mem->cmd_id, psp->cmd_buf_mem->resp.status); if (!timeout) { - mutex_unlock(&psp->mutex); - return -EINVAL; + ret = -EINVAL; + goto exit; } }
@@ -317,8 +322,10 @@ psp_cmd_submit_buf(struct psp_context *psp, ucode->tmr_mc_addr_lo = psp->cmd_buf_mem->resp.fw_addr_lo; ucode->tmr_mc_addr_hi = psp->cmd_buf_mem->resp.fw_addr_hi; } - mutex_unlock(&psp->mutex);
+exit: + mutex_unlock(&psp->mutex); + drm_dev_exit(idx); return ret; }
@@ -355,8 +362,7 @@ static int psp_load_toc(struct psp_context *psp, if (!cmd) return -ENOMEM; /* Copy toc to psp firmware private buffer */ - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->toc_start_addr, psp->toc_bin_size); + psp_copy_fw(psp, psp->toc_start_addr, psp->toc_bin_size);
psp_prep_load_toc_cmd_buf(cmd, psp->fw_pri_mc_addr, psp->toc_bin_size);
@@ -571,8 +577,7 @@ static int psp_asd_load(struct psp_context *psp) if (!cmd) return -ENOMEM;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->asd_start_addr, psp->asd_ucode_size); + psp_copy_fw(psp, psp->asd_start_addr, psp->asd_ucode_size);
psp_prep_asd_load_cmd_buf(cmd, psp->fw_pri_mc_addr, psp->asd_ucode_size); @@ -727,8 +732,7 @@ static int psp_xgmi_load(struct psp_context *psp) if (!cmd) return -ENOMEM;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_xgmi_start_addr, psp->ta_xgmi_ucode_size); + psp_copy_fw(psp, psp->ta_xgmi_start_addr, psp->ta_xgmi_ucode_size);
psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -983,8 +987,7 @@ static int psp_ras_load(struct psp_context *psp) if (!cmd) return -ENOMEM;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_ras_start_addr, psp->ta_ras_ucode_size); + psp_copy_fw(psp, psp->ta_ras_start_addr, psp->ta_ras_ucode_size);
psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -1220,8 +1223,7 @@ static int psp_hdcp_load(struct psp_context *psp) if (!cmd) return -ENOMEM;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_hdcp_start_addr, + psp_copy_fw(psp, psp->ta_hdcp_start_addr, psp->ta_hdcp_ucode_size);
psp_prep_ta_load_cmd_buf(cmd, @@ -1372,8 +1374,7 @@ static int psp_dtm_load(struct psp_context *psp) if (!cmd) return -ENOMEM;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_dtm_start_addr, psp->ta_dtm_ucode_size); + psp_copy_fw(psp, psp->ta_dtm_start_addr, psp->ta_dtm_ucode_size);
psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -1518,8 +1519,7 @@ static int psp_rap_load(struct psp_context *psp) if (!cmd) return -ENOMEM;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_rap_start_addr, psp->ta_rap_ucode_size); + psp_copy_fw(psp, psp->ta_rap_start_addr, psp->ta_rap_ucode_size);
psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -2928,7 +2928,7 @@ static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev, struct amdgpu_device *adev = drm_to_adev(ddev); void *cpu_addr; dma_addr_t dma_addr; - int ret; + int ret, idx; char fw_name[100]; const struct firmware *usbc_pd_fw;
@@ -2937,47 +2937,66 @@ static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev, return -EBUSY; }
- snprintf(fw_name, sizeof(fw_name), "amdgpu/%s", buf); - ret = request_firmware(&usbc_pd_fw, fw_name, adev->dev); - if (ret) - goto fail; + if (drm_dev_enter(ddev, &idx)) {
- /* We need contiguous physical mem to place the FW for psp to access */ - cpu_addr = dma_alloc_coherent(adev->dev, usbc_pd_fw->size, &dma_addr, GFP_KERNEL); + snprintf(fw_name, sizeof(fw_name), "amdgpu/%s", buf); + ret = request_firmware(&usbc_pd_fw, fw_name, adev->dev); + if (ret) + goto fail;
- ret = dma_mapping_error(adev->dev, dma_addr); - if (ret) - goto rel_buf; + /* We need contiguous physical mem to place the FW for psp to access */ + cpu_addr = dma_alloc_coherent(adev->dev, usbc_pd_fw->size, &dma_addr, GFP_KERNEL);
- memcpy_toio(cpu_addr, usbc_pd_fw->data, usbc_pd_fw->size); + ret = dma_mapping_error(adev->dev, dma_addr); + if (ret) + goto rel_buf;
- /* - * x86 specific workaround. - * Without it the buffer is invisible in PSP. - * - * TODO Remove once PSP starts snooping CPU cache - */ + memcpy_toio(cpu_addr, usbc_pd_fw->data, usbc_pd_fw->size); + + /* + * x86 specific workaround. + * Without it the buffer is invisible in PSP. + * + * TODO Remove once PSP starts snooping CPU cache + */ #ifdef CONFIG_X86 - clflush_cache_range(cpu_addr, (usbc_pd_fw->size & ~(L1_CACHE_BYTES - 1))); + clflush_cache_range(cpu_addr, (usbc_pd_fw->size & ~(L1_CACHE_BYTES - 1))); #endif
- mutex_lock(&adev->psp.mutex); - ret = psp_load_usbc_pd_fw(&adev->psp, dma_addr); - mutex_unlock(&adev->psp.mutex); + mutex_lock(&adev->psp.mutex); + ret = psp_load_usbc_pd_fw(&adev->psp, dma_addr); + mutex_unlock(&adev->psp.mutex);
rel_buf: - dma_free_coherent(adev->dev, usbc_pd_fw->size, cpu_addr, dma_addr); - release_firmware(usbc_pd_fw); - + dma_free_coherent(adev->dev, usbc_pd_fw->size, cpu_addr, dma_addr); + release_firmware(usbc_pd_fw); fail: - if (ret) { - DRM_ERROR("Failed to load USBC PD FW, err = %d", ret); - return ret; + if (ret) { + DRM_ERROR("Failed to load USBC PD FW, err = %d", ret); + return ret; + } + + drm_dev_exit(idx); + return count; + } else { + return -ENODEV; } +} + +void psp_copy_fw(struct psp_context *psp, uint8_t *start_addr, uint32_t bin_size) +{ + int idx;
- return count; + if (!drm_dev_enter(&psp->adev->ddev, &idx)) + return; + + memset(psp->fw_pri_buf, 0, PSP_1_MEG); + memcpy(psp->fw_pri_buf, start_addr, bin_size); + + drm_dev_exit(idx); }
+ static DEVICE_ATTR(usbc_pd_fw, S_IRUGO | S_IWUSR, psp_usbc_pd_fw_sysfs_read, psp_usbc_pd_fw_sysfs_write); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h index cb50ba445f8c..0dbbedb84c84 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h @@ -417,4 +417,7 @@ int psp_init_ta_microcode(struct psp_context *psp, const char *chip_name); int psp_get_fw_attestation_records_addr(struct psp_context *psp, uint64_t *output_ptr); + +void psp_copy_fw(struct psp_context *psp, uint8_t *start_addr, uint32_t bin_size); + #endif diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c index b644c78475fd..f9f71008f454 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c @@ -35,6 +35,8 @@ #include "amdgpu.h" #include "atom.h"
+#include <drm/drm_drv.h> + /* * Rings * Most engines on the GPU are fed via ring buffers. Ring @@ -459,3 +461,71 @@ int amdgpu_ring_test_helper(struct amdgpu_ring *ring) ring->sched.ready = !r; return r; } + +void amdgpu_ring_clear_ring(struct amdgpu_ring *ring) +{ + int idx; + int i = 0; + + if (!drm_dev_enter(&ring->adev->ddev, &idx)) + return; + + while (i <= ring->buf_mask) + ring->ring[i++] = ring->funcs->nop; + + drm_dev_exit(idx); + +} + +void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v) +{ + int idx; + + if (!drm_dev_enter(&ring->adev->ddev, &idx)) + return; + + if (ring->count_dw <= 0) + DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n"); + ring->ring[ring->wptr++ & ring->buf_mask] = v; + ring->wptr &= ring->ptr_mask; + ring->count_dw--; + + drm_dev_exit(idx); +} + +void amdgpu_ring_write_multiple(struct amdgpu_ring *ring, + void *src, int count_dw) +{ + unsigned occupied, chunk1, chunk2; + void *dst; + int idx; + + if (!drm_dev_enter(&ring->adev->ddev, &idx)) + return; + + if (unlikely(ring->count_dw < count_dw)) + DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n"); + + occupied = ring->wptr & ring->buf_mask; + dst = (void *)&ring->ring[occupied]; + chunk1 = ring->buf_mask + 1 - occupied; + chunk1 = (chunk1 >= count_dw) ? count_dw: chunk1; + chunk2 = count_dw - chunk1; + chunk1 <<= 2; + chunk2 <<= 2; + + if (chunk1) + memcpy(dst, src, chunk1); + + if (chunk2) { + src += chunk1; + dst = (void *)ring->ring; + memcpy(dst, src, chunk2); + } + + ring->wptr += count_dw; + ring->wptr &= ring->ptr_mask; + ring->count_dw -= count_dw; + + drm_dev_exit(idx); +} diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h index 0f195f7bf797..c2b83f48e6d6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h @@ -297,53 +297,12 @@ static inline void amdgpu_ring_set_preempt_cond_exec(struct amdgpu_ring *ring, *ring->cond_exe_cpu_addr = cond_exec; }
-static inline void amdgpu_ring_clear_ring(struct amdgpu_ring *ring) -{ - int i = 0; - while (i <= ring->buf_mask) - ring->ring[i++] = ring->funcs->nop; - -} - -static inline void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v) -{ - if (ring->count_dw <= 0) - DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n"); - ring->ring[ring->wptr++ & ring->buf_mask] = v; - ring->wptr &= ring->ptr_mask; - ring->count_dw--; -} +void amdgpu_ring_clear_ring(struct amdgpu_ring *ring);
-static inline void amdgpu_ring_write_multiple(struct amdgpu_ring *ring, - void *src, int count_dw) -{ - unsigned occupied, chunk1, chunk2; - void *dst; - - if (unlikely(ring->count_dw < count_dw)) - DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n"); - - occupied = ring->wptr & ring->buf_mask; - dst = (void *)&ring->ring[occupied]; - chunk1 = ring->buf_mask + 1 - occupied; - chunk1 = (chunk1 >= count_dw) ? count_dw: chunk1; - chunk2 = count_dw - chunk1; - chunk1 <<= 2; - chunk2 <<= 2; +void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v);
- if (chunk1) - memcpy(dst, src, chunk1); - - if (chunk2) { - src += chunk1; - dst = (void *)ring->ring; - memcpy(dst, src, chunk2); - } - - ring->wptr += count_dw; - ring->wptr &= ring->ptr_mask; - ring->count_dw -= count_dw; -} +void amdgpu_ring_write_multiple(struct amdgpu_ring *ring, + void *src, int count_dw);
int amdgpu_ring_test_helper(struct amdgpu_ring *ring);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c index e2ed4689118a..df47f5ffa08f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c @@ -32,6 +32,7 @@ #include <linux/module.h>
#include <drm/drm.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "amdgpu_pm.h" @@ -375,7 +376,7 @@ int amdgpu_uvd_suspend(struct amdgpu_device *adev) { unsigned size; void *ptr; - int i, j; + int i, j, idx; bool in_ras_intr = amdgpu_ras_intr_triggered();
cancel_delayed_work_sync(&adev->uvd.idle_work); @@ -403,11 +404,15 @@ int amdgpu_uvd_suspend(struct amdgpu_device *adev) if (!adev->uvd.inst[j].saved_bo) return -ENOMEM;
- /* re-write 0 since err_event_athub will corrupt VCPU buffer */ - if (in_ras_intr) - memset(adev->uvd.inst[j].saved_bo, 0, size); - else - memcpy_fromio(adev->uvd.inst[j].saved_bo, ptr, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + /* re-write 0 since err_event_athub will corrupt VCPU buffer */ + if (in_ras_intr) + memset(adev->uvd.inst[j].saved_bo, 0, size); + else + memcpy_fromio(adev->uvd.inst[j].saved_bo, ptr, size); + + drm_dev_exit(idx); + } }
if (in_ras_intr) @@ -420,7 +425,7 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev) { unsigned size; void *ptr; - int i; + int i, idx;
for (i = 0; i < adev->uvd.num_uvd_inst; i++) { if (adev->uvd.harvest_config & (1 << i)) @@ -432,7 +437,10 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev) ptr = adev->uvd.inst[i].cpu_addr;
if (adev->uvd.inst[i].saved_bo != NULL) { - memcpy_toio(ptr, adev->uvd.inst[i].saved_bo, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(ptr, adev->uvd.inst[i].saved_bo, size); + drm_dev_exit(idx); + } kvfree(adev->uvd.inst[i].saved_bo); adev->uvd.inst[i].saved_bo = NULL; } else { @@ -442,8 +450,11 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev) hdr = (const struct common_firmware_header *)adev->uvd.fw->data; if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP) { offset = le32_to_cpu(hdr->ucode_array_offset_bytes); - memcpy_toio(adev->uvd.inst[i].cpu_addr, adev->uvd.fw->data + offset, - le32_to_cpu(hdr->ucode_size_bytes)); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(adev->uvd.inst[i].cpu_addr, adev->uvd.fw->data + offset, + le32_to_cpu(hdr->ucode_size_bytes)); + drm_dev_exit(idx); + } size -= le32_to_cpu(hdr->ucode_size_bytes); ptr += le32_to_cpu(hdr->ucode_size_bytes); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c index ea6a62f67e38..833203401ef4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c @@ -29,6 +29,7 @@ #include <linux/module.h>
#include <drm/drm.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "amdgpu_pm.h" @@ -293,7 +294,7 @@ int amdgpu_vce_resume(struct amdgpu_device *adev) void *cpu_addr; const struct common_firmware_header *hdr; unsigned offset; - int r; + int r, idx;
if (adev->vce.vcpu_bo == NULL) return -EINVAL; @@ -313,8 +314,12 @@ int amdgpu_vce_resume(struct amdgpu_device *adev)
hdr = (const struct common_firmware_header *)adev->vce.fw->data; offset = le32_to_cpu(hdr->ucode_array_offset_bytes); - memcpy_toio(cpu_addr, adev->vce.fw->data + offset, - adev->vce.fw->size - offset); + + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(cpu_addr, adev->vce.fw->data + offset, + adev->vce.fw->size - offset); + drm_dev_exit(idx); + }
amdgpu_bo_kunmap(adev->vce.vcpu_bo);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c index 99b82f3c2617..b42db22761b8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c @@ -27,6 +27,7 @@ #include <linux/firmware.h> #include <linux/module.h> #include <linux/pci.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "amdgpu_pm.h" @@ -267,7 +268,7 @@ int amdgpu_vcn_suspend(struct amdgpu_device *adev) { unsigned size; void *ptr; - int i; + int i, idx;
cancel_delayed_work_sync(&adev->vcn.idle_work);
@@ -284,7 +285,10 @@ int amdgpu_vcn_suspend(struct amdgpu_device *adev) if (!adev->vcn.inst[i].saved_bo) return -ENOMEM;
- memcpy_fromio(adev->vcn.inst[i].saved_bo, ptr, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_fromio(adev->vcn.inst[i].saved_bo, ptr, size); + drm_dev_exit(idx); + } } return 0; } @@ -293,7 +297,7 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev) { unsigned size; void *ptr; - int i; + int i, idx;
for (i = 0; i < adev->vcn.num_vcn_inst; ++i) { if (adev->vcn.harvest_config & (1 << i)) @@ -305,7 +309,10 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev) ptr = adev->vcn.inst[i].cpu_addr;
if (adev->vcn.inst[i].saved_bo != NULL) { - memcpy_toio(ptr, adev->vcn.inst[i].saved_bo, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(ptr, adev->vcn.inst[i].saved_bo, size); + drm_dev_exit(idx); + } kvfree(adev->vcn.inst[i].saved_bo); adev->vcn.inst[i].saved_bo = NULL; } else { @@ -315,8 +322,11 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev) hdr = (const struct common_firmware_header *)adev->vcn.fw->data; if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP) { offset = le32_to_cpu(hdr->ucode_array_offset_bytes); - memcpy_toio(adev->vcn.inst[i].cpu_addr, adev->vcn.fw->data + offset, - le32_to_cpu(hdr->ucode_size_bytes)); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(adev->vcn.inst[i].cpu_addr, adev->vcn.fw->data + offset, + le32_to_cpu(hdr->ucode_size_bytes)); + drm_dev_exit(idx); + } size -= le32_to_cpu(hdr->ucode_size_bytes); ptr += le32_to_cpu(hdr->ucode_size_bytes); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index ae18c0e32347..7b622056df58 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -31,6 +31,7 @@ #include <linux/dma-buf.h>
#include <drm/amdgpu_drm.h> +#include <drm/drm_drv.h> #include "amdgpu.h" #include "amdgpu_trace.h" #include "amdgpu_amdkfd.h" @@ -1604,7 +1605,10 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev, struct amdgpu_vm_update_params params; enum amdgpu_sync_mode sync_mode; uint64_t pfn; - int r; + int r, idx; + + if (!drm_dev_enter(&adev->ddev, &idx)) + return -ENODEV;
memset(¶ms, 0, sizeof(params)); params.adev = adev; @@ -1713,6 +1717,7 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev,
error_unlock: amdgpu_vm_eviction_unlock(vm); + drm_dev_exit(idx); return r; }
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c index c325d6f53a71..94cce172b98e 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c @@ -23,6 +23,7 @@ #include <linux/firmware.h> #include <linux/module.h> #include <linux/vmalloc.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "amdgpu_psp.h" @@ -269,10 +270,8 @@ static int psp_v11_0_bootloader_load_kdb(struct psp_context *psp) if (ret) return ret;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP KDB binary to memory */ - memcpy(psp->fw_pri_buf, psp->kdb_start_addr, psp->kdb_bin_size); + psp_copy_fw(psp, psp->kdb_start_addr, psp->kdb_bin_size);
/* Provide the PSP KDB to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -302,10 +301,8 @@ static int psp_v11_0_bootloader_load_spl(struct psp_context *psp) if (ret) return ret;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP SPL binary to memory */ - memcpy(psp->fw_pri_buf, psp->spl_start_addr, psp->spl_bin_size); + psp_copy_fw(psp, psp->spl_start_addr, psp->spl_bin_size);
/* Provide the PSP SPL to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -335,10 +332,8 @@ static int psp_v11_0_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP System Driver binary to memory */ - memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size); + psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size);
/* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -371,10 +366,8 @@ static int psp_v11_0_bootloader_load_sos(struct psp_context *psp) if (ret) return ret;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy Secure OS binary to PSP memory */ - memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size); + psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size);
/* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -608,7 +601,7 @@ static int psp_v11_0_memory_training(struct psp_context *psp, uint32_t ops) uint32_t p2c_header[4]; uint32_t sz; void *buf; - int ret; + int ret, idx;
if (ctx->init == PSP_MEM_TRAIN_NOT_SUPPORT) { DRM_DEBUG("Memory training is not supported.\n"); @@ -681,17 +674,24 @@ static int psp_v11_0_memory_training(struct psp_context *psp, uint32_t ops) return -ENOMEM; }
- memcpy_fromio(buf, adev->mman.aper_base_kaddr, sz); - ret = psp_v11_0_memory_training_send_msg(psp, PSP_BL__DRAM_LONG_TRAIN); - if (ret) { - DRM_ERROR("Send long training msg failed.\n"); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_fromio(buf, adev->mman.aper_base_kaddr, sz); + ret = psp_v11_0_memory_training_send_msg(psp, PSP_BL__DRAM_LONG_TRAIN); + if (ret) { + DRM_ERROR("Send long training msg failed.\n"); + vfree(buf); + drm_dev_exit(idx); + return ret; + } + + memcpy_toio(adev->mman.aper_base_kaddr, buf, sz); + adev->hdp.funcs->flush_hdp(adev, NULL); vfree(buf); - return ret; + drm_dev_exit(idx); + } else { + vfree(buf); + return -ENODEV; } - - memcpy_toio(adev->mman.aper_base_kaddr, buf, sz); - adev->hdp.funcs->flush_hdp(adev, NULL); - vfree(buf); }
if (ops & PSP_MEM_TRAIN_SAVE) { diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c index c4828bd3264b..618e5b6b85d9 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c @@ -138,10 +138,8 @@ static int psp_v12_0_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP System Driver binary to memory */ - memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size); + psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size);
/* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -179,10 +177,8 @@ static int psp_v12_0_bootloader_load_sos(struct psp_context *psp) if (ret) return ret;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy Secure OS binary to PSP memory */ - memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size); + psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size);
/* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c index f2e725f72d2f..d0a6cccd0897 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c @@ -102,10 +102,8 @@ static int psp_v3_1_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP System Driver binary to memory */ - memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size); + psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size);
/* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -143,10 +141,8 @@ static int psp_v3_1_bootloader_load_sos(struct psp_context *psp) if (ret) return ret;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy Secure OS binary to PSP memory */ - memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size); + psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size);
/* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c index 37fa163393fd..fd859c778df0 100644 --- a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c +++ b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c @@ -25,6 +25,7 @@ */
#include <linux/firmware.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "amdgpu_vce.h" @@ -555,16 +556,19 @@ static int vce_v4_0_hw_fini(void *handle) static int vce_v4_0_suspend(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - int r; + int r, idx;
if (adev->vce.vcpu_bo == NULL) return 0;
- if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) { - unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo); - void *ptr = adev->vce.cpu_addr; + if (drm_dev_enter(&adev->ddev, &idx)) { + if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) { + unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo); + void *ptr = adev->vce.cpu_addr;
- memcpy_fromio(adev->vce.saved_bo, ptr, size); + memcpy_fromio(adev->vce.saved_bo, ptr, size); + } + drm_dev_exit(idx); }
r = vce_v4_0_hw_fini(adev); @@ -577,16 +581,20 @@ static int vce_v4_0_suspend(void *handle) static int vce_v4_0_resume(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - int r; + int r, idx;
if (adev->vce.vcpu_bo == NULL) return -EINVAL;
if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) { - unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo); - void *ptr = adev->vce.cpu_addr;
- memcpy_toio(ptr, adev->vce.saved_bo, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo); + void *ptr = adev->vce.cpu_addr; + + memcpy_toio(ptr, adev->vce.saved_bo, size); + drm_dev_exit(idx); + } } else { r = amdgpu_vce_resume(adev); if (r) diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c index def583916294..5ac8ffe197aa 100644 --- a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c +++ b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c @@ -34,6 +34,8 @@ #include "vcn/vcn_3_0_0_sh_mask.h" #include "ivsrcid/vcn/irqsrcs_vcn_2_0.h"
+#include <drm/drm_drv.h> + #define mmUVD_CONTEXT_ID_INTERNAL_OFFSET 0x27 #define mmUVD_GPCOM_VCPU_CMD_INTERNAL_OFFSET 0x0f #define mmUVD_GPCOM_VCPU_DATA0_INTERNAL_OFFSET 0x10 @@ -263,16 +265,20 @@ static int vcn_v3_0_sw_init(void *handle) static int vcn_v3_0_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - int i, r; + int i, r, idx;
- for (i = 0; i < adev->vcn.num_vcn_inst; i++) { - volatile struct amdgpu_fw_shared *fw_shared; + if (drm_dev_enter(&adev->ddev, &idx)) { + for (i = 0; i < adev->vcn.num_vcn_inst; i++) { + volatile struct amdgpu_fw_shared *fw_shared;
- if (adev->vcn.harvest_config & (1 << i)) - continue; - fw_shared = adev->vcn.inst[i].fw_shared_cpu_addr; - fw_shared->present_flag_0 = 0; - fw_shared->sw_ring.is_enabled = false; + if (adev->vcn.harvest_config & (1 << i)) + continue; + fw_shared = adev->vcn.inst[i].fw_shared_cpu_addr; + fw_shared->present_flag_0 = 0; + fw_shared->sw_ring.is_enabled = false; + } + + drm_dev_exit(idx); }
if (amdgpu_sriov_vf(adev)) diff --git a/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c b/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c index aae25243eb10..d628b91846c9 100644 --- a/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c +++ b/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c @@ -405,6 +405,8 @@ int smu7_request_smu_load_fw(struct pp_hwmgr *hwmgr) UCODE_ID_MEC_STORAGE, &toc->entry[toc->num_entries++]), "Failed to Get Firmware Entry.", r = -EINVAL; goto failed); } + + /* AG TODO Can't call drm_dev_enter/exit because access adev->ddev here ... */ memcpy_toio(smu_data->header_buffer.kaddr, smu_data->toc, sizeof(struct SMU_DRAMData_TOC)); smum_send_msg_to_smc_with_parameter(hwmgr,
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
This should prevent writing to memory or IO ranges possibly already allocated for other uses after our device is removed.
v5: Protect more places wher memcopy_to/form_io takes place Protect IB submissions
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 75 +++--- drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 9 + drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c | 228 +++++++++--------- drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c | 115 +++++---- drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h | 3 + drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 70 ++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 49 +--- drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c | 31 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c | 11 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c | 22 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 +- drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 44 ++-- drivers/gpu/drm/amd/amdgpu/psp_v12_0.c | 8 +- drivers/gpu/drm/amd/amdgpu/psp_v3_1.c | 8 +- drivers/gpu/drm/amd/amdgpu/vce_v4_0.c | 26 +- drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c | 22 +- .../drm/amd/pm/powerplay/smumgr/smu7_smumgr.c | 2 + 17 files changed, 425 insertions(+), 305 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 91594ddc2459..22b09c4db255 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -71,6 +71,8 @@ #include <linux/pm_runtime.h>
+#include <drm/drm_drv.h>
- MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin");
@@ -279,48 +281,55 @@ void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, unsigned long flags; uint32_t hi = ~0; uint64_t last;
int idx;
if (drm_dev_enter(&adev->ddev, &idx)) {
In general I think it would be better to do stuff like this as
if (!drm_dev_enter(...)) return; /* or goto error_nodev; */
...
drm_dev_exit();
This is essentially error handling and we avoid modifying the whole function for adding the prerequisite.
Regards, Christian.
#ifdef CONFIG_64BIT
- last = min(pos + size, adev->gmc.visible_vram_size);
- if (last > pos) {
void __iomem *addr = adev->mman.aper_base_kaddr + pos;size_t count = last - pos;if (write) {memcpy_toio(addr, buf, count);mb();amdgpu_asic_flush_hdp(adev, NULL);} else {amdgpu_asic_invalidate_hdp(adev, NULL);mb();memcpy_fromio(buf, addr, count);}
last = min(pos + size, adev->gmc.visible_vram_size);if (last > pos) {void __iomem *addr = adev->mman.aper_base_kaddr + pos;size_t count = last - pos;if (write) {memcpy_toio(addr, buf, count);mb();amdgpu_asic_flush_hdp(adev, NULL);} else {amdgpu_asic_invalidate_hdp(adev, NULL);mb();memcpy_fromio(buf, addr, count);}
if (count == size)return;
if (count == size) {drm_dev_exit(idx);return;}
pos += count;buf += count / 4;size -= count;- }
pos += count;buf += count / 4;size -= count;}
- spin_lock_irqsave(&adev->mmio_idx_lock, flags);
- for (last = pos + size; pos < last; pos += 4) {
uint32_t tmp = pos >> 31;
spin_lock_irqsave(&adev->mmio_idx_lock, flags);for (last = pos + size; pos < last; pos += 4) {uint32_t tmp = pos >> 31;
WREG32_NO_KIQ(mmMM_INDEX, ((uint32_t)pos) | 0x80000000);if (tmp != hi) {WREG32_NO_KIQ(mmMM_INDEX_HI, tmp);hi = tmp;
WREG32_NO_KIQ(mmMM_INDEX, ((uint32_t)pos) | 0x80000000);if (tmp != hi) {WREG32_NO_KIQ(mmMM_INDEX_HI, tmp);hi = tmp;}if (write)WREG32_NO_KIQ(mmMM_DATA, *buf++);else*buf++ = RREG32_NO_KIQ(mmMM_DATA);
if (write)WREG32_NO_KIQ(mmMM_DATA, *buf++);else*buf++ = RREG32_NO_KIQ(mmMM_DATA);
spin_unlock_irqrestore(&adev->mmio_idx_lock, flags);drm_dev_exit(idx);
spin_unlock_irqrestore(&adev->mmio_idx_lock, flags); }
/*
@@ -402,6 +411,7 @@ uint8_t amdgpu_mm_rreg8(struct amdgpu_device *adev, uint32_t offset) */ void amdgpu_mm_wreg8(struct amdgpu_device *adev, uint32_t offset, uint8_t value) {
- if (adev->in_pci_err_recovery) return;
@@ -542,6 +552,7 @@ u32 amdgpu_mm_rdoorbell(struct amdgpu_device *adev, u32 index) */ void amdgpu_mm_wdoorbell(struct amdgpu_device *adev, u32 index, u32 v) {
- if (adev->in_pci_err_recovery) return;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c index fe1a39ffda72..1beb4e64b884 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c @@ -31,6 +31,8 @@ #include "amdgpu_ras.h" #include "amdgpu_xgmi.h"
+#include <drm/drm_drv.h>
- /**
- amdgpu_gmc_get_pde_for_bo - get the PDE for a BO
@@ -98,6 +100,10 @@ int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr, { void __iomem *ptr = (void *)cpu_pt_addr; uint64_t value;
int idx;
if (!drm_dev_enter(&adev->ddev, &idx))
return 0;/*
- The following is for PTE only. GART does not have PDEs.
@@ -105,6 +111,9 @@ int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr, value = addr & 0x0000FFFFFFFFF000ULL; value |= flags; writeq(value, ptr + (gpu_page_idx * 8));
- drm_dev_exit(idx);
- return 0; }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c index 7645223ea0ef..b3a1ff04c10f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c @@ -31,6 +31,7 @@
#include <drm/amdgpu_drm.h> #include <drm/drm_debugfs.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "atom.h" @@ -138,7 +139,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs, bool secure;
unsigned i;
- int r = 0;
int idx, r = 0; bool need_pipe_sync = false;
if (num_ibs == 0)
@@ -170,142 +171,151 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs, return -EINVAL; }
- alloc_size = ring->funcs->emit_frame_size + num_ibs *
ring->funcs->emit_ib_size;
- if (drm_dev_enter(&adev->ddev, &idx)) {
- r = amdgpu_ring_alloc(ring, alloc_size);
- if (r) {
dev_err(adev->dev, "scheduling IB failed (%d).\n", r);return r;- }
- need_ctx_switch = ring->current_ctx != fence_ctx;
- if (ring->funcs->emit_pipeline_sync && job &&
((tmp = amdgpu_sync_get_fence(&job->sched_sync)) ||(amdgpu_sriov_vf(adev) && need_ctx_switch) ||amdgpu_vm_need_pipeline_sync(ring, job))) {need_pipe_sync = true;if (tmp)trace_amdgpu_ib_pipe_sync(job, tmp);dma_fence_put(tmp);- }
alloc_size = ring->funcs->emit_frame_size + num_ibs *ring->funcs->emit_ib_size;
- if ((ib->flags & AMDGPU_IB_FLAG_EMIT_MEM_SYNC) && ring->funcs->emit_mem_sync)
ring->funcs->emit_mem_sync(ring);
r = amdgpu_ring_alloc(ring, alloc_size);if (r) {dev_err(adev->dev, "scheduling IB failed (%d).\n", r);goto exit;}
- if (ring->funcs->emit_wave_limit &&
ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH)ring->funcs->emit_wave_limit(ring, true);
need_ctx_switch = ring->current_ctx != fence_ctx;if (ring->funcs->emit_pipeline_sync && job &&((tmp = amdgpu_sync_get_fence(&job->sched_sync)) ||(amdgpu_sriov_vf(adev) && need_ctx_switch) ||amdgpu_vm_need_pipeline_sync(ring, job))) {need_pipe_sync = true;
- if (ring->funcs->insert_start)
ring->funcs->insert_start(ring);
if (tmp)trace_amdgpu_ib_pipe_sync(job, tmp);
- if (job) {
r = amdgpu_vm_flush(ring, job, need_pipe_sync);if (r) {amdgpu_ring_undo(ring);return r;
dma_fence_put(tmp);
}
if (job && ring->funcs->init_cond_exec)
patch_offset = amdgpu_ring_init_cond_exec(ring);
if ((ib->flags & AMDGPU_IB_FLAG_EMIT_MEM_SYNC) && ring->funcs->emit_mem_sync)ring->funcs->emit_mem_sync(ring);-#ifdef CONFIG_X86_64
- if (!(adev->flags & AMD_IS_APU))
-#endif
- {
if (ring->funcs->emit_hdp_flush)amdgpu_ring_emit_hdp_flush(ring);elseamdgpu_asic_flush_hdp(adev, ring);- }
if (ring->funcs->emit_wave_limit &&ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH)ring->funcs->emit_wave_limit(ring, true);
- if (need_ctx_switch)
status |= AMDGPU_HAVE_CTX_SWITCH;
if (ring->funcs->insert_start)ring->funcs->insert_start(ring);
- skip_preamble = ring->current_ctx == fence_ctx;
- if (job && ring->funcs->emit_cntxcntl) {
status |= job->preamble_status;status |= job->preemption_status;amdgpu_ring_emit_cntxcntl(ring, status);- }
if (job) {r = amdgpu_vm_flush(ring, job, need_pipe_sync);if (r) {amdgpu_ring_undo(ring);goto exit;}}
- /* Setup initial TMZiness and send it off.
*/- secure = false;
- if (job && ring->funcs->emit_frame_cntl) {
secure = ib->flags & AMDGPU_IB_FLAGS_SECURE;amdgpu_ring_emit_frame_cntl(ring, true, secure);- }
if (job && ring->funcs->init_cond_exec)patch_offset = amdgpu_ring_init_cond_exec(ring);- #ifdef CONFIG_X86_64
if (!(adev->flags & AMD_IS_APU))- #endif
{if (ring->funcs->emit_hdp_flush)amdgpu_ring_emit_hdp_flush(ring);elseamdgpu_asic_flush_hdp(adev, ring);}
- for (i = 0; i < num_ibs; ++i) {
ib = &ibs[i];
if (need_ctx_switch)status |= AMDGPU_HAVE_CTX_SWITCH;
/* drop preamble IBs if we don't have a context switch */if ((ib->flags & AMDGPU_IB_FLAG_PREAMBLE) &&skip_preamble &&!(status & AMDGPU_PREAMBLE_IB_PRESENT_FIRST) &&!amdgpu_mcbp &&!amdgpu_sriov_vf(adev)) /* for SRIOV preemption, Preamble CE ib must be inserted anyway */continue;
skip_preamble = ring->current_ctx == fence_ctx;if (job && ring->funcs->emit_cntxcntl) {status |= job->preamble_status;status |= job->preemption_status;amdgpu_ring_emit_cntxcntl(ring, status);}/* Setup initial TMZiness and send it off.*/secure = false;if (job && ring->funcs->emit_frame_cntl) {
if (secure != !!(ib->flags & AMDGPU_IB_FLAGS_SECURE)) {amdgpu_ring_emit_frame_cntl(ring, false, secure);secure = !secure;amdgpu_ring_emit_frame_cntl(ring, true, secure);
secure = ib->flags & AMDGPU_IB_FLAGS_SECURE;amdgpu_ring_emit_frame_cntl(ring, true, secure);}for (i = 0; i < num_ibs; ++i) {ib = &ibs[i];/* drop preamble IBs if we don't have a context switch */if ((ib->flags & AMDGPU_IB_FLAG_PREAMBLE) &&skip_preamble &&!(status & AMDGPU_PREAMBLE_IB_PRESENT_FIRST) &&!amdgpu_mcbp &&!amdgpu_sriov_vf(adev)) /* for SRIOV preemption, Preamble CE ib must be inserted anyway */continue;if (job && ring->funcs->emit_frame_cntl) {if (secure != !!(ib->flags & AMDGPU_IB_FLAGS_SECURE)) {amdgpu_ring_emit_frame_cntl(ring, false, secure);secure = !secure;amdgpu_ring_emit_frame_cntl(ring, true, secure);} }amdgpu_ring_emit_ib(ring, job, ib, status);status &= ~AMDGPU_HAVE_CTX_SWITCH;
amdgpu_ring_emit_ib(ring, job, ib, status);status &= ~AMDGPU_HAVE_CTX_SWITCH;- }
if (job && ring->funcs->emit_frame_cntl)amdgpu_ring_emit_frame_cntl(ring, false, secure);
- if (job && ring->funcs->emit_frame_cntl)
amdgpu_ring_emit_frame_cntl(ring, false, secure);
- #ifdef CONFIG_X86_64
if (!(adev->flags & AMD_IS_APU))- #endif
amdgpu_asic_invalidate_hdp(adev, ring);-#ifdef CONFIG_X86_64
- if (!(adev->flags & AMD_IS_APU))
-#endif
amdgpu_asic_invalidate_hdp(adev, ring);
if (ib->flags & AMDGPU_IB_FLAG_TC_WB_NOT_INVALIDATE)fence_flags |= AMDGPU_FENCE_FLAG_TC_WB_ONLY;
- if (ib->flags & AMDGPU_IB_FLAG_TC_WB_NOT_INVALIDATE)
fence_flags |= AMDGPU_FENCE_FLAG_TC_WB_ONLY;
/* wrap the last IB with fence */if (job && job->uf_addr) {amdgpu_ring_emit_fence(ring, job->uf_addr, job->uf_sequence,fence_flags | AMDGPU_FENCE_FLAG_64BIT);}
- /* wrap the last IB with fence */
- if (job && job->uf_addr) {
amdgpu_ring_emit_fence(ring, job->uf_addr, job->uf_sequence,fence_flags | AMDGPU_FENCE_FLAG_64BIT);- }
r = amdgpu_fence_emit(ring, f, fence_flags);if (r) {dev_err(adev->dev, "failed to emit fence (%d)\n", r);if (job && job->vmid)amdgpu_vmid_reset(adev, ring->funcs->vmhub, job->vmid);amdgpu_ring_undo(ring);goto exit;}
- r = amdgpu_fence_emit(ring, f, fence_flags);
- if (r) {
dev_err(adev->dev, "failed to emit fence (%d)\n", r);if (job && job->vmid)amdgpu_vmid_reset(adev, ring->funcs->vmhub, job->vmid);amdgpu_ring_undo(ring);return r;- }
if (ring->funcs->insert_end)ring->funcs->insert_end(ring);
- if (ring->funcs->insert_end)
ring->funcs->insert_end(ring);
if (patch_offset != ~0 && ring->funcs->patch_cond_exec)amdgpu_ring_patch_cond_exec(ring, patch_offset);
- if (patch_offset != ~0 && ring->funcs->patch_cond_exec)
amdgpu_ring_patch_cond_exec(ring, patch_offset);
ring->current_ctx = fence_ctx;if (vm && ring->funcs->emit_switch_buffer)amdgpu_ring_emit_switch_buffer(ring);
- ring->current_ctx = fence_ctx;
- if (vm && ring->funcs->emit_switch_buffer)
amdgpu_ring_emit_switch_buffer(ring);
if (ring->funcs->emit_wave_limit &&ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH)ring->funcs->emit_wave_limit(ring, false);
- if (ring->funcs->emit_wave_limit &&
ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH)ring->funcs->emit_wave_limit(ring, false);
amdgpu_ring_commit(ring);
- amdgpu_ring_commit(ring);
- return 0;
- } else {
return -ENODEV;- }
+exit:
drm_dev_exit(idx);
return r; }
/**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c index 839917eb7bc3..638b7fd1857c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c @@ -25,6 +25,7 @@
#include <linux/firmware.h> #include <linux/dma-mapping.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "amdgpu_psp.h" @@ -38,6 +39,8 @@ #include "amdgpu_ras.h" #include "amdgpu_securedisplay.h"
+#include <drm/drm_drv.h>
- static int psp_sysfs_init(struct amdgpu_device *adev); static void psp_sysfs_fini(struct amdgpu_device *adev);
@@ -249,7 +252,7 @@ psp_cmd_submit_buf(struct psp_context *psp, struct psp_gfx_cmd_resp *cmd, uint64_t fence_mc_addr) { int ret;
- int index;
- int index, idx; int timeout = 20000; bool ras_intr = false; bool skip_unsupport = false;
@@ -257,6 +260,9 @@ psp_cmd_submit_buf(struct psp_context *psp, if (psp->adev->in_pci_err_recovery) return 0;
if (!drm_dev_enter(&psp->adev->ddev, &idx))
return 0;mutex_lock(&psp->mutex);
memset(psp->cmd_buf_mem, 0, PSP_CMD_BUFFER_SIZE);
@@ -267,8 +273,7 @@ psp_cmd_submit_buf(struct psp_context *psp, ret = psp_ring_cmd_submit(psp, psp->cmd_buf_mc_addr, fence_mc_addr, index); if (ret) { atomic_dec(&psp->fence_value);
mutex_unlock(&psp->mutex);return ret;
goto exit;}
amdgpu_asic_invalidate_hdp(psp->adev, NULL);
@@ -308,8 +313,8 @@ psp_cmd_submit_buf(struct psp_context *psp, psp->cmd_buf_mem->cmd_id, psp->cmd_buf_mem->resp.status); if (!timeout) {
mutex_unlock(&psp->mutex);return -EINVAL;
ret = -EINVAL;goto exit;@@ -317,8 +322,10 @@ psp_cmd_submit_buf(struct psp_context *psp, ucode->tmr_mc_addr_lo = psp->cmd_buf_mem->resp.fw_addr_lo; ucode->tmr_mc_addr_hi = psp->cmd_buf_mem->resp.fw_addr_hi; }
- mutex_unlock(&psp->mutex);
+exit:
- mutex_unlock(&psp->mutex);
- drm_dev_exit(idx); return ret; }
@@ -355,8 +362,7 @@ static int psp_load_toc(struct psp_context *psp, if (!cmd) return -ENOMEM; /* Copy toc to psp firmware private buffer */
- memset(psp->fw_pri_buf, 0, PSP_1_MEG);
- memcpy(psp->fw_pri_buf, psp->toc_start_addr, psp->toc_bin_size);
psp_copy_fw(psp, psp->toc_start_addr, psp->toc_bin_size);
psp_prep_load_toc_cmd_buf(cmd, psp->fw_pri_mc_addr, psp->toc_bin_size);
@@ -571,8 +577,7 @@ static int psp_asd_load(struct psp_context *psp) if (!cmd) return -ENOMEM;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG);
- memcpy(psp->fw_pri_buf, psp->asd_start_addr, psp->asd_ucode_size);
psp_copy_fw(psp, psp->asd_start_addr, psp->asd_ucode_size);
psp_prep_asd_load_cmd_buf(cmd, psp->fw_pri_mc_addr, psp->asd_ucode_size);
@@ -727,8 +732,7 @@ static int psp_xgmi_load(struct psp_context *psp) if (!cmd) return -ENOMEM;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG);
- memcpy(psp->fw_pri_buf, psp->ta_xgmi_start_addr, psp->ta_xgmi_ucode_size);
psp_copy_fw(psp, psp->ta_xgmi_start_addr, psp->ta_xgmi_ucode_size);
psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr,
@@ -983,8 +987,7 @@ static int psp_ras_load(struct psp_context *psp) if (!cmd) return -ENOMEM;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG);
- memcpy(psp->fw_pri_buf, psp->ta_ras_start_addr, psp->ta_ras_ucode_size);
psp_copy_fw(psp, psp->ta_ras_start_addr, psp->ta_ras_ucode_size);
psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr,
@@ -1220,8 +1223,7 @@ static int psp_hdcp_load(struct psp_context *psp) if (!cmd) return -ENOMEM;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG);
- memcpy(psp->fw_pri_buf, psp->ta_hdcp_start_addr,
psp_copy_fw(psp, psp->ta_hdcp_start_addr, psp->ta_hdcp_ucode_size);
psp_prep_ta_load_cmd_buf(cmd,
@@ -1372,8 +1374,7 @@ static int psp_dtm_load(struct psp_context *psp) if (!cmd) return -ENOMEM;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG);
- memcpy(psp->fw_pri_buf, psp->ta_dtm_start_addr, psp->ta_dtm_ucode_size);
psp_copy_fw(psp, psp->ta_dtm_start_addr, psp->ta_dtm_ucode_size);
psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr,
@@ -1518,8 +1519,7 @@ static int psp_rap_load(struct psp_context *psp) if (!cmd) return -ENOMEM;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG);
- memcpy(psp->fw_pri_buf, psp->ta_rap_start_addr, psp->ta_rap_ucode_size);
psp_copy_fw(psp, psp->ta_rap_start_addr, psp->ta_rap_ucode_size);
psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr,
@@ -2928,7 +2928,7 @@ static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev, struct amdgpu_device *adev = drm_to_adev(ddev); void *cpu_addr; dma_addr_t dma_addr;
- int ret;
- int ret, idx; char fw_name[100]; const struct firmware *usbc_pd_fw;
@@ -2937,47 +2937,66 @@ static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev, return -EBUSY; }
- snprintf(fw_name, sizeof(fw_name), "amdgpu/%s", buf);
- ret = request_firmware(&usbc_pd_fw, fw_name, adev->dev);
- if (ret)
goto fail;
- if (drm_dev_enter(ddev, &idx)) {
- /* We need contiguous physical mem to place the FW for psp to access */
- cpu_addr = dma_alloc_coherent(adev->dev, usbc_pd_fw->size, &dma_addr, GFP_KERNEL);
snprintf(fw_name, sizeof(fw_name), "amdgpu/%s", buf);ret = request_firmware(&usbc_pd_fw, fw_name, adev->dev);if (ret)goto fail;
- ret = dma_mapping_error(adev->dev, dma_addr);
- if (ret)
goto rel_buf;
/* We need contiguous physical mem to place the FW for psp to access */cpu_addr = dma_alloc_coherent(adev->dev, usbc_pd_fw->size, &dma_addr, GFP_KERNEL);
- memcpy_toio(cpu_addr, usbc_pd_fw->data, usbc_pd_fw->size);
ret = dma_mapping_error(adev->dev, dma_addr);if (ret)goto rel_buf;
- /*
* x86 specific workaround.* Without it the buffer is invisible in PSP.** TODO Remove once PSP starts snooping CPU cache*/
memcpy_toio(cpu_addr, usbc_pd_fw->data, usbc_pd_fw->size);/** x86 specific workaround.* Without it the buffer is invisible in PSP.** TODO Remove once PSP starts snooping CPU cache*/
- clflush_cache_range(cpu_addr, (usbc_pd_fw->size & ~(L1_CACHE_BYTES - 1)));
clflush_cache_range(cpu_addr, (usbc_pd_fw->size & ~(L1_CACHE_BYTES - 1)));
- mutex_lock(&adev->psp.mutex);
- ret = psp_load_usbc_pd_fw(&adev->psp, dma_addr);
- mutex_unlock(&adev->psp.mutex);
mutex_lock(&adev->psp.mutex);ret = psp_load_usbc_pd_fw(&adev->psp, dma_addr);mutex_unlock(&adev->psp.mutex);rel_buf:
- dma_free_coherent(adev->dev, usbc_pd_fw->size, cpu_addr, dma_addr);
- release_firmware(usbc_pd_fw);
dma_free_coherent(adev->dev, usbc_pd_fw->size, cpu_addr, dma_addr);release_firmware(usbc_pd_fw);
- if (ret) {
DRM_ERROR("Failed to load USBC PD FW, err = %d", ret);return ret;
if (ret) {DRM_ERROR("Failed to load USBC PD FW, err = %d", ret);return ret;}drm_dev_exit(idx);return count;- } else {
return -ENODEV;+}
+void psp_copy_fw(struct psp_context *psp, uint8_t *start_addr, uint32_t bin_size) +{
- int idx;
- return count;
if (!drm_dev_enter(&psp->adev->ddev, &idx))
return;memset(psp->fw_pri_buf, 0, PSP_1_MEG);
memcpy(psp->fw_pri_buf, start_addr, bin_size);
drm_dev_exit(idx); }
static DEVICE_ATTR(usbc_pd_fw, S_IRUGO | S_IWUSR, psp_usbc_pd_fw_sysfs_read, psp_usbc_pd_fw_sysfs_write);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h index cb50ba445f8c..0dbbedb84c84 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h @@ -417,4 +417,7 @@ int psp_init_ta_microcode(struct psp_context *psp, const char *chip_name); int psp_get_fw_attestation_records_addr(struct psp_context *psp, uint64_t *output_ptr);
+void psp_copy_fw(struct psp_context *psp, uint8_t *start_addr, uint32_t bin_size);
- #endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c index b644c78475fd..f9f71008f454 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c @@ -35,6 +35,8 @@ #include "amdgpu.h" #include "atom.h"
+#include <drm/drm_drv.h>
- /*
- Rings
- Most engines on the GPU are fed via ring buffers. Ring
@@ -459,3 +461,71 @@ int amdgpu_ring_test_helper(struct amdgpu_ring *ring) ring->sched.ready = !r; return r; }
+void amdgpu_ring_clear_ring(struct amdgpu_ring *ring) +{
- int idx;
- int i = 0;
- if (!drm_dev_enter(&ring->adev->ddev, &idx))
return;- while (i <= ring->buf_mask)
ring->ring[i++] = ring->funcs->nop;- drm_dev_exit(idx);
+}
+void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v) +{
- int idx;
- if (!drm_dev_enter(&ring->adev->ddev, &idx))
return;- if (ring->count_dw <= 0)
DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n");- ring->ring[ring->wptr++ & ring->buf_mask] = v;
- ring->wptr &= ring->ptr_mask;
- ring->count_dw--;
- drm_dev_exit(idx);
+}
+void amdgpu_ring_write_multiple(struct amdgpu_ring *ring,
void *src, int count_dw)+{
- unsigned occupied, chunk1, chunk2;
- void *dst;
- int idx;
- if (!drm_dev_enter(&ring->adev->ddev, &idx))
return;- if (unlikely(ring->count_dw < count_dw))
DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n");- occupied = ring->wptr & ring->buf_mask;
- dst = (void *)&ring->ring[occupied];
- chunk1 = ring->buf_mask + 1 - occupied;
- chunk1 = (chunk1 >= count_dw) ? count_dw: chunk1;
- chunk2 = count_dw - chunk1;
- chunk1 <<= 2;
- chunk2 <<= 2;
- if (chunk1)
memcpy(dst, src, chunk1);- if (chunk2) {
src += chunk1;dst = (void *)ring->ring;memcpy(dst, src, chunk2);- }
- ring->wptr += count_dw;
- ring->wptr &= ring->ptr_mask;
- ring->count_dw -= count_dw;
- drm_dev_exit(idx);
+} diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h index 0f195f7bf797..c2b83f48e6d6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h @@ -297,53 +297,12 @@ static inline void amdgpu_ring_set_preempt_cond_exec(struct amdgpu_ring *ring, *ring->cond_exe_cpu_addr = cond_exec; }
-static inline void amdgpu_ring_clear_ring(struct amdgpu_ring *ring) -{
- int i = 0;
- while (i <= ring->buf_mask)
ring->ring[i++] = ring->funcs->nop;-}
-static inline void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v) -{
- if (ring->count_dw <= 0)
DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n");- ring->ring[ring->wptr++ & ring->buf_mask] = v;
- ring->wptr &= ring->ptr_mask;
- ring->count_dw--;
-} +void amdgpu_ring_clear_ring(struct amdgpu_ring *ring);
-static inline void amdgpu_ring_write_multiple(struct amdgpu_ring *ring,
void *src, int count_dw)-{
- unsigned occupied, chunk1, chunk2;
- void *dst;
- if (unlikely(ring->count_dw < count_dw))
DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n");- occupied = ring->wptr & ring->buf_mask;
- dst = (void *)&ring->ring[occupied];
- chunk1 = ring->buf_mask + 1 - occupied;
- chunk1 = (chunk1 >= count_dw) ? count_dw: chunk1;
- chunk2 = count_dw - chunk1;
- chunk1 <<= 2;
- chunk2 <<= 2;
+void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v);
- if (chunk1)
memcpy(dst, src, chunk1);- if (chunk2) {
src += chunk1;dst = (void *)ring->ring;memcpy(dst, src, chunk2);- }
- ring->wptr += count_dw;
- ring->wptr &= ring->ptr_mask;
- ring->count_dw -= count_dw;
-} +void amdgpu_ring_write_multiple(struct amdgpu_ring *ring,
void *src, int count_dw);int amdgpu_ring_test_helper(struct amdgpu_ring *ring);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c index e2ed4689118a..df47f5ffa08f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c @@ -32,6 +32,7 @@ #include <linux/module.h>
#include <drm/drm.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "amdgpu_pm.h" @@ -375,7 +376,7 @@ int amdgpu_uvd_suspend(struct amdgpu_device *adev) { unsigned size; void *ptr;
- int i, j;
int i, j, idx; bool in_ras_intr = amdgpu_ras_intr_triggered();
cancel_delayed_work_sync(&adev->uvd.idle_work);
@@ -403,11 +404,15 @@ int amdgpu_uvd_suspend(struct amdgpu_device *adev) if (!adev->uvd.inst[j].saved_bo) return -ENOMEM;
/* re-write 0 since err_event_athub will corrupt VCPU buffer */if (in_ras_intr)memset(adev->uvd.inst[j].saved_bo, 0, size);elsememcpy_fromio(adev->uvd.inst[j].saved_bo, ptr, size);
if (drm_dev_enter(&adev->ddev, &idx)) {/* re-write 0 since err_event_athub will corrupt VCPU buffer */if (in_ras_intr)memset(adev->uvd.inst[j].saved_bo, 0, size);elsememcpy_fromio(adev->uvd.inst[j].saved_bo, ptr, size);drm_dev_exit(idx);}}
if (in_ras_intr)
@@ -420,7 +425,7 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev) { unsigned size; void *ptr;
- int i;
int i, idx;
for (i = 0; i < adev->uvd.num_uvd_inst; i++) { if (adev->uvd.harvest_config & (1 << i))
@@ -432,7 +437,10 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev) ptr = adev->uvd.inst[i].cpu_addr;
if (adev->uvd.inst[i].saved_bo != NULL) {
memcpy_toio(ptr, adev->uvd.inst[i].saved_bo, size);
if (drm_dev_enter(&adev->ddev, &idx)) {memcpy_toio(ptr, adev->uvd.inst[i].saved_bo, size);drm_dev_exit(idx);} kvfree(adev->uvd.inst[i].saved_bo); adev->uvd.inst[i].saved_bo = NULL;@@ -442,8 +450,11 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev) hdr = (const struct common_firmware_header *)adev->uvd.fw->data; if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP) { offset = le32_to_cpu(hdr->ucode_array_offset_bytes);
memcpy_toio(adev->uvd.inst[i].cpu_addr, adev->uvd.fw->data + offset,le32_to_cpu(hdr->ucode_size_bytes));
if (drm_dev_enter(&adev->ddev, &idx)) {memcpy_toio(adev->uvd.inst[i].cpu_addr, adev->uvd.fw->data + offset,le32_to_cpu(hdr->ucode_size_bytes));drm_dev_exit(idx);} size -= le32_to_cpu(hdr->ucode_size_bytes); ptr += le32_to_cpu(hdr->ucode_size_bytes); }diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c index ea6a62f67e38..833203401ef4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c @@ -29,6 +29,7 @@ #include <linux/module.h>
#include <drm/drm.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "amdgpu_pm.h" @@ -293,7 +294,7 @@ int amdgpu_vce_resume(struct amdgpu_device *adev) void *cpu_addr; const struct common_firmware_header *hdr; unsigned offset;
- int r;
int r, idx;
if (adev->vce.vcpu_bo == NULL) return -EINVAL;
@@ -313,8 +314,12 @@ int amdgpu_vce_resume(struct amdgpu_device *adev)
hdr = (const struct common_firmware_header *)adev->vce.fw->data; offset = le32_to_cpu(hdr->ucode_array_offset_bytes);
- memcpy_toio(cpu_addr, adev->vce.fw->data + offset,
adev->vce.fw->size - offset);
if (drm_dev_enter(&adev->ddev, &idx)) {
memcpy_toio(cpu_addr, adev->vce.fw->data + offset,adev->vce.fw->size - offset);drm_dev_exit(idx);}
amdgpu_bo_kunmap(adev->vce.vcpu_bo);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c index 99b82f3c2617..b42db22761b8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c @@ -27,6 +27,7 @@ #include <linux/firmware.h> #include <linux/module.h> #include <linux/pci.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "amdgpu_pm.h" @@ -267,7 +268,7 @@ int amdgpu_vcn_suspend(struct amdgpu_device *adev) { unsigned size; void *ptr;
- int i;
int i, idx;
cancel_delayed_work_sync(&adev->vcn.idle_work);
@@ -284,7 +285,10 @@ int amdgpu_vcn_suspend(struct amdgpu_device *adev) if (!adev->vcn.inst[i].saved_bo) return -ENOMEM;
memcpy_fromio(adev->vcn.inst[i].saved_bo, ptr, size);
if (drm_dev_enter(&adev->ddev, &idx)) {memcpy_fromio(adev->vcn.inst[i].saved_bo, ptr, size);drm_dev_exit(idx);}@@ -293,7 +297,7 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev) { unsigned size; void *ptr;
- int i;
int i, idx;
for (i = 0; i < adev->vcn.num_vcn_inst; ++i) { if (adev->vcn.harvest_config & (1 << i))
@@ -305,7 +309,10 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev) ptr = adev->vcn.inst[i].cpu_addr;
if (adev->vcn.inst[i].saved_bo != NULL) {
memcpy_toio(ptr, adev->vcn.inst[i].saved_bo, size);
if (drm_dev_enter(&adev->ddev, &idx)) {memcpy_toio(ptr, adev->vcn.inst[i].saved_bo, size);drm_dev_exit(idx);} kvfree(adev->vcn.inst[i].saved_bo); adev->vcn.inst[i].saved_bo = NULL;@@ -315,8 +322,11 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev) hdr = (const struct common_firmware_header *)adev->vcn.fw->data; if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP) { offset = le32_to_cpu(hdr->ucode_array_offset_bytes);
memcpy_toio(adev->vcn.inst[i].cpu_addr, adev->vcn.fw->data + offset,le32_to_cpu(hdr->ucode_size_bytes));
if (drm_dev_enter(&adev->ddev, &idx)) {memcpy_toio(adev->vcn.inst[i].cpu_addr, adev->vcn.fw->data + offset,le32_to_cpu(hdr->ucode_size_bytes));drm_dev_exit(idx);} size -= le32_to_cpu(hdr->ucode_size_bytes); ptr += le32_to_cpu(hdr->ucode_size_bytes); }diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index ae18c0e32347..7b622056df58 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -31,6 +31,7 @@ #include <linux/dma-buf.h>
#include <drm/amdgpu_drm.h> +#include <drm/drm_drv.h> #include "amdgpu.h" #include "amdgpu_trace.h" #include "amdgpu_amdkfd.h" @@ -1604,7 +1605,10 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev, struct amdgpu_vm_update_params params; enum amdgpu_sync_mode sync_mode; uint64_t pfn;
- int r;
int r, idx;
if (!drm_dev_enter(&adev->ddev, &idx))
return -ENODEV;memset(¶ms, 0, sizeof(params)); params.adev = adev;
@@ -1713,6 +1717,7 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev,
error_unlock: amdgpu_vm_eviction_unlock(vm);
- drm_dev_exit(idx); return r; }
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c index c325d6f53a71..94cce172b98e 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c @@ -23,6 +23,7 @@ #include <linux/firmware.h> #include <linux/module.h> #include <linux/vmalloc.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "amdgpu_psp.h" @@ -269,10 +270,8 @@ static int psp_v11_0_bootloader_load_kdb(struct psp_context *psp) if (ret) return ret;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG);
- /* Copy PSP KDB binary to memory */
- memcpy(psp->fw_pri_buf, psp->kdb_start_addr, psp->kdb_bin_size);
psp_copy_fw(psp, psp->kdb_start_addr, psp->kdb_bin_size);
/* Provide the PSP KDB to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
@@ -302,10 +301,8 @@ static int psp_v11_0_bootloader_load_spl(struct psp_context *psp) if (ret) return ret;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG);
- /* Copy PSP SPL binary to memory */
- memcpy(psp->fw_pri_buf, psp->spl_start_addr, psp->spl_bin_size);
psp_copy_fw(psp, psp->spl_start_addr, psp->spl_bin_size);
/* Provide the PSP SPL to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
@@ -335,10 +332,8 @@ static int psp_v11_0_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG);
- /* Copy PSP System Driver binary to memory */
- memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size);
psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size);
/* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
@@ -371,10 +366,8 @@ static int psp_v11_0_bootloader_load_sos(struct psp_context *psp) if (ret) return ret;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG);
- /* Copy Secure OS binary to PSP memory */
- memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size);
psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size);
/* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
@@ -608,7 +601,7 @@ static int psp_v11_0_memory_training(struct psp_context *psp, uint32_t ops) uint32_t p2c_header[4]; uint32_t sz; void *buf;
- int ret;
int ret, idx;
if (ctx->init == PSP_MEM_TRAIN_NOT_SUPPORT) { DRM_DEBUG("Memory training is not supported.\n");
@@ -681,17 +674,24 @@ static int psp_v11_0_memory_training(struct psp_context *psp, uint32_t ops) return -ENOMEM; }
memcpy_fromio(buf, adev->mman.aper_base_kaddr, sz);ret = psp_v11_0_memory_training_send_msg(psp, PSP_BL__DRAM_LONG_TRAIN);if (ret) {DRM_ERROR("Send long training msg failed.\n");
if (drm_dev_enter(&adev->ddev, &idx)) {memcpy_fromio(buf, adev->mman.aper_base_kaddr, sz);ret = psp_v11_0_memory_training_send_msg(psp, PSP_BL__DRAM_LONG_TRAIN);if (ret) {DRM_ERROR("Send long training msg failed.\n");vfree(buf);drm_dev_exit(idx);return ret;}memcpy_toio(adev->mman.aper_base_kaddr, buf, sz);adev->hdp.funcs->flush_hdp(adev, NULL); vfree(buf);
return ret;
drm_dev_exit(idx);} else {vfree(buf);return -ENODEV;
memcpy_toio(adev->mman.aper_base_kaddr, buf, sz);adev->hdp.funcs->flush_hdp(adev, NULL);vfree(buf);}
if (ops & PSP_MEM_TRAIN_SAVE) {
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c index c4828bd3264b..618e5b6b85d9 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c @@ -138,10 +138,8 @@ static int psp_v12_0_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG);
- /* Copy PSP System Driver binary to memory */
- memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size);
psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size);
/* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
@@ -179,10 +177,8 @@ static int psp_v12_0_bootloader_load_sos(struct psp_context *psp) if (ret) return ret;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG);
- /* Copy Secure OS binary to PSP memory */
- memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size);
psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size);
/* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c index f2e725f72d2f..d0a6cccd0897 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c @@ -102,10 +102,8 @@ static int psp_v3_1_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG);
- /* Copy PSP System Driver binary to memory */
- memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size);
psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size);
/* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
@@ -143,10 +141,8 @@ static int psp_v3_1_bootloader_load_sos(struct psp_context *psp) if (ret) return ret;
- memset(psp->fw_pri_buf, 0, PSP_1_MEG);
- /* Copy Secure OS binary to PSP memory */
- memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size);
psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size);
/* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c index 37fa163393fd..fd859c778df0 100644 --- a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c +++ b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c @@ -25,6 +25,7 @@ */
#include <linux/firmware.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "amdgpu_vce.h" @@ -555,16 +556,19 @@ static int vce_v4_0_hw_fini(void *handle) static int vce_v4_0_suspend(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- int r;
int r, idx;
if (adev->vce.vcpu_bo == NULL) return 0;
- if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) {
unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo);void *ptr = adev->vce.cpu_addr;
- if (drm_dev_enter(&adev->ddev, &idx)) {
if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) {unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo);void *ptr = adev->vce.cpu_addr;
memcpy_fromio(adev->vce.saved_bo, ptr, size);
memcpy_fromio(adev->vce.saved_bo, ptr, size);}drm_dev_exit(idx);}
r = vce_v4_0_hw_fini(adev);
@@ -577,16 +581,20 @@ static int vce_v4_0_suspend(void *handle) static int vce_v4_0_resume(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- int r;
int r, idx;
if (adev->vce.vcpu_bo == NULL) return -EINVAL;
if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) {
unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo);void *ptr = adev->vce.cpu_addr;memcpy_toio(ptr, adev->vce.saved_bo, size);
if (drm_dev_enter(&adev->ddev, &idx)) {unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo);void *ptr = adev->vce.cpu_addr;memcpy_toio(ptr, adev->vce.saved_bo, size);drm_dev_exit(idx);}diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c index def583916294..5ac8ffe197aa 100644 --- a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c +++ b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c @@ -34,6 +34,8 @@ #include "vcn/vcn_3_0_0_sh_mask.h" #include "ivsrcid/vcn/irqsrcs_vcn_2_0.h"
+#include <drm/drm_drv.h>
- #define mmUVD_CONTEXT_ID_INTERNAL_OFFSET 0x27 #define mmUVD_GPCOM_VCPU_CMD_INTERNAL_OFFSET 0x0f #define mmUVD_GPCOM_VCPU_DATA0_INTERNAL_OFFSET 0x10
@@ -263,16 +265,20 @@ static int vcn_v3_0_sw_init(void *handle) static int vcn_v3_0_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- int i, r;
- int i, r, idx;
- for (i = 0; i < adev->vcn.num_vcn_inst; i++) {
volatile struct amdgpu_fw_shared *fw_shared;
- if (drm_dev_enter(&adev->ddev, &idx)) {
for (i = 0; i < adev->vcn.num_vcn_inst; i++) {volatile struct amdgpu_fw_shared *fw_shared;
if (adev->vcn.harvest_config & (1 << i))continue;fw_shared = adev->vcn.inst[i].fw_shared_cpu_addr;fw_shared->present_flag_0 = 0;fw_shared->sw_ring.is_enabled = false;
if (adev->vcn.harvest_config & (1 << i))continue;fw_shared = adev->vcn.inst[i].fw_shared_cpu_addr;fw_shared->present_flag_0 = 0;fw_shared->sw_ring.is_enabled = false;}drm_dev_exit(idx);}
if (amdgpu_sriov_vf(adev))
diff --git a/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c b/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c index aae25243eb10..d628b91846c9 100644 --- a/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c +++ b/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c @@ -405,6 +405,8 @@ int smu7_request_smu_load_fw(struct pp_hwmgr *hwmgr) UCODE_ID_MEC_STORAGE, &toc->entry[toc->num_entries++]), "Failed to Get Firmware Entry.", r = -EINVAL; goto failed); }
- /* AG TODO Can't call drm_dev_enter/exit because access adev->ddev here ... */ memcpy_toio(smu_data->header_buffer.kaddr, smu_data->toc, sizeof(struct SMU_DRAMData_TOC)); smum_send_msg_to_smc_with_parameter(hwmgr,
We don't want to rearm the timer if driver hook reports that the device is gone.
v5: Update drm_gpu_sched_stat values in code.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/scheduler/sched_main.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index d82a7ebf6099..908b0b56032d 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -314,6 +314,7 @@ static void drm_sched_job_timedout(struct work_struct *work) { struct drm_gpu_scheduler *sched; struct drm_sched_job *job; + enum drm_gpu_sched_stat status = DRM_GPU_SCHED_STAT_NOMINAL;
sched = container_of(work, struct drm_gpu_scheduler, work_tdr.work);
@@ -331,7 +332,7 @@ static void drm_sched_job_timedout(struct work_struct *work) list_del_init(&job->list); spin_unlock(&sched->job_list_lock);
- job->sched->ops->timedout_job(job); + status = job->sched->ops->timedout_job(job);
/* * Guilty job did complete and hence needs to be manually removed @@ -345,9 +346,11 @@ static void drm_sched_job_timedout(struct work_struct *work) spin_unlock(&sched->job_list_lock); }
- spin_lock(&sched->job_list_lock); - drm_sched_start_timeout(sched); - spin_unlock(&sched->job_list_lock); + if (status != DRM_GPU_SCHED_STAT_ENODEV) { + spin_lock(&sched->job_list_lock); + drm_sched_start_timeout(sched); + spin_unlock(&sched->job_list_lock); + } }
/**
Return DRM_TASK_STATUS_ENODEV back to the scheduler when device is not present so they timeout timer will not be rearmed.
v5: Update to match updated return values in enum drm_gpu_sched_stat
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c index 759b34799221..d33e6d97cc89 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c @@ -25,6 +25,8 @@ #include <linux/wait.h> #include <linux/sched.h>
+#include <drm/drm_drv.h> + #include "amdgpu.h" #include "amdgpu_trace.h"
@@ -34,6 +36,15 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job) struct amdgpu_job *job = to_amdgpu_job(s_job); struct amdgpu_task_info ti; struct amdgpu_device *adev = ring->adev; + int idx; + + if (!drm_dev_enter(&adev->ddev, &idx)) { + DRM_INFO("%s - device unplugged skipping recovery on scheduler:%s", + __func__, s_job->sched->name); + + /* Effectively the job is aborted as the device is gone */ + return DRM_GPU_SCHED_STAT_ENODEV; + }
memset(&ti, 0, sizeof(struct amdgpu_task_info));
@@ -41,7 +52,7 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job) amdgpu_ring_soft_recovery(ring, job->vmid, s_job->s_fence->parent)) { DRM_ERROR("ring %s timeout, but soft recovered\n", s_job->sched->name); - return DRM_GPU_SCHED_STAT_NOMINAL; + goto exit; }
amdgpu_vm_get_task_info(ring->adev, job->pasid, &ti); @@ -53,13 +64,15 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job)
if (amdgpu_device_should_recover_gpu(ring->adev)) { amdgpu_device_gpu_recover(ring->adev, job); - return DRM_GPU_SCHED_STAT_NOMINAL; } else { drm_sched_suspend_timeout(&ring->sched); if (amdgpu_sriov_vf(adev)) adev->virt.tdr_debug = true; - return DRM_GPU_SCHED_STAT_NOMINAL; } + +exit: + drm_dev_exit(idx); + return DRM_GPU_SCHED_STAT_NOMINAL; }
int amdgpu_job_alloc(struct amdgpu_device *adev, unsigned num_ibs,
No point calling amdgpu_fence_wait_empty before stopping the SW scheduler otherwise there is always a chance another job sneaked in after the wait.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 34d51e962799..fd9282637549 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -533,6 +533,12 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
if (!ring || !ring->fence_drv.initialized) continue; + + /* Stop any new job submissions from sched before flushing the ring */ + /* TODO Handle amdgpu_job_submit_direct and amdgpu_amdkfd_submit_ib */ + if (!ring->no_scheduler) + drm_sched_fini(&ring->sched); + r = amdgpu_fence_wait_empty(ring); if (r) { /* no need to trigger GPU reset as we are unloading */ @@ -541,8 +547,7 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) if (ring->fence_drv.irq_src) amdgpu_irq_put(adev, ring->fence_drv.irq_src, ring->fence_drv.irq_type); - if (!ring->no_scheduler) - drm_sched_fini(&ring->sched); + del_timer_sync(&ring->fence_drv.fallback_timer); } }
Filizing the fences? You mean finishing the fences, don't you? :)
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
No point calling amdgpu_fence_wait_empty before stopping the SW scheduler otherwise there is always a chance another job sneaked in after the wait.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 34d51e962799..fd9282637549 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -533,6 +533,12 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
if (!ring || !ring->fence_drv.initialized) continue;
/* Stop any new job submissions from sched before flushing the ring *//* TODO Handle amdgpu_job_submit_direct and amdgpu_amdkfd_submit_ib */if (!ring->no_scheduler)drm_sched_fini(&ring->sched);- r = amdgpu_fence_wait_empty(ring); if (r) { /* no need to trigger GPU reset as we are unloading */
@@ -541,8 +547,7 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) if (ring->fence_drv.irq_src) amdgpu_irq_put(adev, ring->fence_drv.irq_src, ring->fence_drv.irq_type);
if (!ring->no_scheduler)drm_sched_fini(&ring->sched);
- del_timer_sync(&ring->fence_drv.fallback_timer); } }
On 2021-04-29 3:15 a.m., Christian König wrote:
Filizing the fences? You mean finishing the fences, don't you? :)
Yes, my bad.
Andrey
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
No point calling amdgpu_fence_wait_empty before stopping the SW scheduler otherwise there is always a chance another job sneaked in after the wait.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 34d51e962799..fd9282637549 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -533,6 +533,12 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) if (!ring || !ring->fence_drv.initialized) continue;
+ /* Stop any new job submissions from sched before flushing the ring */ + /* TODO Handle amdgpu_job_submit_direct and amdgpu_amdkfd_submit_ib */ + if (!ring->no_scheduler) + drm_sched_fini(&ring->sched);
r = amdgpu_fence_wait_empty(ring); if (r) { /* no need to trigger GPU reset as we are unloading */ @@ -541,8 +547,7 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) if (ring->fence_drv.irq_src) amdgpu_irq_put(adev, ring->fence_drv.irq_src, ring->fence_drv.irq_type); - if (!ring->no_scheduler) - drm_sched_fini(&ring->sched);
del_timer_sync(&ring->fence_drv.fallback_timer); } }
If removing while commands in flight you cannot wait to flush the HW fences on a ring since the device is gone.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index fd9282637549..2670201e78d3 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -37,6 +37,7 @@ #include <linux/pm_runtime.h>
#include <drm/drm_debugfs.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "amdgpu_trace.h" @@ -525,8 +526,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev) */ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) { - unsigned i, j; - int r; + int i, r;
for (i = 0; i < AMDGPU_MAX_RINGS; i++) { struct amdgpu_ring *ring = adev->rings[i]; @@ -539,11 +539,15 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) if (!ring->no_scheduler) drm_sched_fini(&ring->sched);
- r = amdgpu_fence_wait_empty(ring); - if (r) { - /* no need to trigger GPU reset as we are unloading */ + /* You can't wait for HW to signal if it's gone */ + if (!drm_dev_is_unplugged(&adev->ddev)) + r = amdgpu_fence_wait_empty(ring); + else + r = -ENODEV; + /* no need to trigger GPU reset as we are unloading */ + if (r) amdgpu_fence_driver_force_completion(ring); - } + if (ring->fence_drv.irq_src) amdgpu_irq_put(adev, ring->fence_drv.irq_src, ring->fence_drv.irq_type);
Problem: If scheduler is already stopped by the time sched_entity is released and entity's job_queue not empty I encountred a hang in drm_sched_entity_flush. This is because drm_sched_entity_is_idle never becomes false.
Fix: In drm_sched_fini detach all sched_entities from the scheduler's run queues. This will satisfy drm_sched_entity_is_idle. Also wakeup all those processes stuck in sched_entity flushing as the scheduler main thread which wakes them up is stopped by now.
v2: Reverse order of drm_sched_rq_remove_entity and marking s_entity as stopped to prevent reinserion back to rq due to race.
v3: Drop drm_sched_rq_remove_entity, only modify entity->stopped and check for it in drm_sched_entity_is_idle
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com Reviewed-by: Christian König christian.koenig@amd.com --- drivers/gpu/drm/scheduler/sched_entity.c | 3 ++- drivers/gpu/drm/scheduler/sched_main.c | 24 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c index f0790e9471d1..cb58f692dad9 100644 --- a/drivers/gpu/drm/scheduler/sched_entity.c +++ b/drivers/gpu/drm/scheduler/sched_entity.c @@ -116,7 +116,8 @@ static bool drm_sched_entity_is_idle(struct drm_sched_entity *entity) rmb(); /* for list_empty to work without lock */
if (list_empty(&entity->list) || - spsc_queue_count(&entity->job_queue) == 0) + spsc_queue_count(&entity->job_queue) == 0 || + entity->stopped) return true;
return false; diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 908b0b56032d..ba087354d0a8 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -897,9 +897,33 @@ EXPORT_SYMBOL(drm_sched_init); */ void drm_sched_fini(struct drm_gpu_scheduler *sched) { + struct drm_sched_entity *s_entity; + int i; + if (sched->thread) kthread_stop(sched->thread);
+ for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) { + struct drm_sched_rq *rq = &sched->sched_rq[i]; + + if (!rq) + continue; + + spin_lock(&rq->lock); + list_for_each_entry(s_entity, &rq->entities, list) + /* + * Prevents reinsertion and marks job_queue as idle, + * it will removed from rq in drm_sched_entity_fini + * eventually + */ + s_entity->stopped = true; + spin_unlock(&rq->lock); + + } + + /* Wakeup everyone stuck in drm_sched_entity_flush for this scheduler */ + wake_up_all(&sched->job_scheduled); + /* Confirm no work left behind accessing device structures */ cancel_delayed_work_sync(&sched->work_tdr);
I need to take another look at this part when I don't have a massive headache any more.
Maybe split the patch set up into different parts, something like: 1. Adding general infrastructure. 2. Making sure all memory is unpolated. 3. Job and fence handling
Christian.
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
Problem: If scheduler is already stopped by the time sched_entity is released and entity's job_queue not empty I encountred a hang in drm_sched_entity_flush. This is because drm_sched_entity_is_idle never becomes false.
Fix: In drm_sched_fini detach all sched_entities from the scheduler's run queues. This will satisfy drm_sched_entity_is_idle. Also wakeup all those processes stuck in sched_entity flushing as the scheduler main thread which wakes them up is stopped by now.
v2: Reverse order of drm_sched_rq_remove_entity and marking s_entity as stopped to prevent reinserion back to rq due to race.
v3: Drop drm_sched_rq_remove_entity, only modify entity->stopped and check for it in drm_sched_entity_is_idle
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com Reviewed-by: Christian König christian.koenig@amd.com
drivers/gpu/drm/scheduler/sched_entity.c | 3 ++- drivers/gpu/drm/scheduler/sched_main.c | 24 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c index f0790e9471d1..cb58f692dad9 100644 --- a/drivers/gpu/drm/scheduler/sched_entity.c +++ b/drivers/gpu/drm/scheduler/sched_entity.c @@ -116,7 +116,8 @@ static bool drm_sched_entity_is_idle(struct drm_sched_entity *entity) rmb(); /* for list_empty to work without lock */
if (list_empty(&entity->list) ||
spsc_queue_count(&entity->job_queue) == 0)
spsc_queue_count(&entity->job_queue) == 0 ||entity->stopped)return true;
return false;
diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 908b0b56032d..ba087354d0a8 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -897,9 +897,33 @@ EXPORT_SYMBOL(drm_sched_init); */ void drm_sched_fini(struct drm_gpu_scheduler *sched) {
struct drm_sched_entity *s_entity;
int i;
if (sched->thread) kthread_stop(sched->thread);
for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) {
struct drm_sched_rq *rq = &sched->sched_rq[i];if (!rq)continue;spin_lock(&rq->lock);list_for_each_entry(s_entity, &rq->entities, list)/** Prevents reinsertion and marks job_queue as idle,* it will removed from rq in drm_sched_entity_fini* eventually*/s_entity->stopped = true;spin_unlock(&rq->lock);}
/* Wakeup everyone stuck in drm_sched_entity_flush for this scheduler */
wake_up_all(&sched->job_scheduled);
/* Confirm no work left behind accessing device structures */ cancel_delayed_work_sync(&sched->work_tdr);
On 2021-04-29 3:18 a.m., Christian König wrote:
I need to take another look at this part when I don't have a massive headache any more.
Maybe split the patch set up into different parts, something like:
- Adding general infrastructure.
- Making sure all memory is unpolated.
- Job and fence handling
I am not sure you mean this patch here, maybe another one ? Also note you already RBed it.
Andrey
Christian.
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
Problem: If scheduler is already stopped by the time sched_entity is released and entity's job_queue not empty I encountred a hang in drm_sched_entity_flush. This is because drm_sched_entity_is_idle never becomes false.
Fix: In drm_sched_fini detach all sched_entities from the scheduler's run queues. This will satisfy drm_sched_entity_is_idle. Also wakeup all those processes stuck in sched_entity flushing as the scheduler main thread which wakes them up is stopped by now.
v2: Reverse order of drm_sched_rq_remove_entity and marking s_entity as stopped to prevent reinserion back to rq due to race.
v3: Drop drm_sched_rq_remove_entity, only modify entity->stopped and check for it in drm_sched_entity_is_idle
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com Reviewed-by: Christian König christian.koenig@amd.com
drivers/gpu/drm/scheduler/sched_entity.c | 3 ++- drivers/gpu/drm/scheduler/sched_main.c | 24 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c index f0790e9471d1..cb58f692dad9 100644 --- a/drivers/gpu/drm/scheduler/sched_entity.c +++ b/drivers/gpu/drm/scheduler/sched_entity.c @@ -116,7 +116,8 @@ static bool drm_sched_entity_is_idle(struct drm_sched_entity *entity) rmb(); /* for list_empty to work without lock */ if (list_empty(&entity->list) || - spsc_queue_count(&entity->job_queue) == 0) + spsc_queue_count(&entity->job_queue) == 0 || + entity->stopped) return true; return false; diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 908b0b56032d..ba087354d0a8 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -897,9 +897,33 @@ EXPORT_SYMBOL(drm_sched_init); */ void drm_sched_fini(struct drm_gpu_scheduler *sched) { + struct drm_sched_entity *s_entity; + int i;
if (sched->thread) kthread_stop(sched->thread); + for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) { + struct drm_sched_rq *rq = &sched->sched_rq[i];
+ if (!rq) + continue;
+ spin_lock(&rq->lock); + list_for_each_entry(s_entity, &rq->entities, list) + /* + * Prevents reinsertion and marks job_queue as idle, + * it will removed from rq in drm_sched_entity_fini + * eventually + */ + s_entity->stopped = true; + spin_unlock(&rq->lock);
+ }
+ /* Wakeup everyone stuck in drm_sched_entity_flush for this scheduler */ + wake_up_all(&sched->job_scheduled);
/* Confirm no work left behind accessing device structures */ cancel_delayed_work_sync(&sched->work_tdr);
Am 29.04.21 um 19:06 schrieb Andrey Grodzovsky:
On 2021-04-29 3:18 a.m., Christian König wrote:
I need to take another look at this part when I don't have a massive headache any more.
Maybe split the patch set up into different parts, something like:
- Adding general infrastructure.
- Making sure all memory is unpolated.
- Job and fence handling
I am not sure you mean this patch here, maybe another one ? Also note you already RBed it.
No what I meant was to send out the patches before this one as #1 and #2.
That is the easier stuff which can easily go into the drm-misc-next or amd-staging-drm-next branch.
The scheduler stuff certainly need to go into drm-misc-next.
Christian.
Andrey
Christian.
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
Problem: If scheduler is already stopped by the time sched_entity is released and entity's job_queue not empty I encountred a hang in drm_sched_entity_flush. This is because drm_sched_entity_is_idle never becomes false.
Fix: In drm_sched_fini detach all sched_entities from the scheduler's run queues. This will satisfy drm_sched_entity_is_idle. Also wakeup all those processes stuck in sched_entity flushing as the scheduler main thread which wakes them up is stopped by now.
v2: Reverse order of drm_sched_rq_remove_entity and marking s_entity as stopped to prevent reinserion back to rq due to race.
v3: Drop drm_sched_rq_remove_entity, only modify entity->stopped and check for it in drm_sched_entity_is_idle
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com Reviewed-by: Christian König christian.koenig@amd.com
drivers/gpu/drm/scheduler/sched_entity.c | 3 ++- drivers/gpu/drm/scheduler/sched_main.c | 24 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c index f0790e9471d1..cb58f692dad9 100644 --- a/drivers/gpu/drm/scheduler/sched_entity.c +++ b/drivers/gpu/drm/scheduler/sched_entity.c @@ -116,7 +116,8 @@ static bool drm_sched_entity_is_idle(struct drm_sched_entity *entity) rmb(); /* for list_empty to work without lock */ if (list_empty(&entity->list) || - spsc_queue_count(&entity->job_queue) == 0) + spsc_queue_count(&entity->job_queue) == 0 || + entity->stopped) return true; return false; diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 908b0b56032d..ba087354d0a8 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -897,9 +897,33 @@ EXPORT_SYMBOL(drm_sched_init); */ void drm_sched_fini(struct drm_gpu_scheduler *sched) { + struct drm_sched_entity *s_entity; + int i;
if (sched->thread) kthread_stop(sched->thread); + for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) { + struct drm_sched_rq *rq = &sched->sched_rq[i];
+ if (!rq) + continue;
+ spin_lock(&rq->lock); + list_for_each_entry(s_entity, &rq->entities, list) + /* + * Prevents reinsertion and marks job_queue as idle, + * it will removed from rq in drm_sched_entity_fini + * eventually + */ + s_entity->stopped = true; + spin_unlock(&rq->lock);
+ }
+ /* Wakeup everyone stuck in drm_sched_entity_flush for this scheduler */ + wake_up_all(&sched->job_scheduled);
/* Confirm no work left behind accessing device structures */ cancel_delayed_work_sync(&sched->work_tdr);
On 2021-04-30 2:47 a.m., Christian König wrote:
Am 29.04.21 um 19:06 schrieb Andrey Grodzovsky:
On 2021-04-29 3:18 a.m., Christian König wrote:
I need to take another look at this part when I don't have a massive headache any more.
Maybe split the patch set up into different parts, something like:
- Adding general infrastructure.
- Making sure all memory is unpolated.
- Job and fence handling
I am not sure you mean this patch here, maybe another one ? Also note you already RBed it.
No what I meant was to send out the patches before this one as #1 and #2.
That is the easier stuff which can easily go into the drm-misc-next or amd-staging-drm-next branch.
The scheduler stuff certainly need to go into drm-misc-next.
Christian.
Got you. I am fine with it. What we have here is a working hot-unplug code but, one with potential use after free MMIO ranges frpom the zombie device. The followup patches after this patch are all about preventing this and so the patch-set up until this patch including, is functional on it's own. While it's necessary to solve the above issue, it's has complications as can be seen from the discussion with Daniel on later patch in this series. Still, in my opinion it's better to rollout some initial support to hot-unplug without use after free protection then having no support for hot-unplug at all. It will also make the merge work easier as I need to constantly rebase the patches on top latest kernel and solve new regressions.
Daniel - given the arguments above can you sound your opinion on this approach ?
Andrey
Andrey
Christian.
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
Problem: If scheduler is already stopped by the time sched_entity is released and entity's job_queue not empty I encountred a hang in drm_sched_entity_flush. This is because drm_sched_entity_is_idle never becomes false.
Fix: In drm_sched_fini detach all sched_entities from the scheduler's run queues. This will satisfy drm_sched_entity_is_idle. Also wakeup all those processes stuck in sched_entity flushing as the scheduler main thread which wakes them up is stopped by now.
v2: Reverse order of drm_sched_rq_remove_entity and marking s_entity as stopped to prevent reinserion back to rq due to race.
v3: Drop drm_sched_rq_remove_entity, only modify entity->stopped and check for it in drm_sched_entity_is_idle
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com Reviewed-by: Christian König christian.koenig@amd.com
drivers/gpu/drm/scheduler/sched_entity.c | 3 ++- drivers/gpu/drm/scheduler/sched_main.c | 24 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c index f0790e9471d1..cb58f692dad9 100644 --- a/drivers/gpu/drm/scheduler/sched_entity.c +++ b/drivers/gpu/drm/scheduler/sched_entity.c @@ -116,7 +116,8 @@ static bool drm_sched_entity_is_idle(struct drm_sched_entity *entity) rmb(); /* for list_empty to work without lock */ if (list_empty(&entity->list) || - spsc_queue_count(&entity->job_queue) == 0) + spsc_queue_count(&entity->job_queue) == 0 || + entity->stopped) return true; return false; diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 908b0b56032d..ba087354d0a8 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -897,9 +897,33 @@ EXPORT_SYMBOL(drm_sched_init); */ void drm_sched_fini(struct drm_gpu_scheduler *sched) { + struct drm_sched_entity *s_entity; + int i;
if (sched->thread) kthread_stop(sched->thread); + for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) { + struct drm_sched_rq *rq = &sched->sched_rq[i];
+ if (!rq) + continue;
+ spin_lock(&rq->lock); + list_for_each_entry(s_entity, &rq->entities, list) + /* + * Prevents reinsertion and marks job_queue as idle, + * it will removed from rq in drm_sched_entity_fini + * eventually + */ + s_entity->stopped = true; + spin_unlock(&rq->lock);
+ }
+ /* Wakeup everyone stuck in drm_sched_entity_flush for this scheduler */ + wake_up_all(&sched->job_scheduled);
/* Confirm no work left behind accessing device structures */ cancel_delayed_work_sync(&sched->work_tdr);
Ping
Andrey
On 2021-04-30 12:10 p.m., Andrey Grodzovsky wrote:
On 2021-04-30 2:47 a.m., Christian König wrote:
Am 29.04.21 um 19:06 schrieb Andrey Grodzovsky:
On 2021-04-29 3:18 a.m., Christian König wrote:
I need to take another look at this part when I don't have a massive headache any more.
Maybe split the patch set up into different parts, something like:
- Adding general infrastructure.
- Making sure all memory is unpolated.
- Job and fence handling
I am not sure you mean this patch here, maybe another one ? Also note you already RBed it.
No what I meant was to send out the patches before this one as #1 and #2.
That is the easier stuff which can easily go into the drm-misc-next or amd-staging-drm-next branch.
The scheduler stuff certainly need to go into drm-misc-next.
Christian.
Got you. I am fine with it. What we have here is a working hot-unplug code but, one with potential use after free MMIO ranges frpom the zombie device. The followup patches after this patch are all about preventing this and so the patch-set up until this patch including, is functional on it's own. While it's necessary to solve the above issue, it's has complications as can be seen from the discussion with Daniel on later patch in this series. Still, in my opinion it's better to rollout some initial support to hot-unplug without use after free protection then having no support for hot-unplug at all. It will also make the merge work easier as I need to constantly rebase the patches on top latest kernel and solve new regressions.
Daniel - given the arguments above can you sound your opinion on this approach ?
Andrey
Andrey
Christian.
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
Problem: If scheduler is already stopped by the time sched_entity is released and entity's job_queue not empty I encountred a hang in drm_sched_entity_flush. This is because drm_sched_entity_is_idle never becomes false.
Fix: In drm_sched_fini detach all sched_entities from the scheduler's run queues. This will satisfy drm_sched_entity_is_idle. Also wakeup all those processes stuck in sched_entity flushing as the scheduler main thread which wakes them up is stopped by now.
v2: Reverse order of drm_sched_rq_remove_entity and marking s_entity as stopped to prevent reinserion back to rq due to race.
v3: Drop drm_sched_rq_remove_entity, only modify entity->stopped and check for it in drm_sched_entity_is_idle
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com Reviewed-by: Christian König christian.koenig@amd.com
drivers/gpu/drm/scheduler/sched_entity.c | 3 ++- drivers/gpu/drm/scheduler/sched_main.c | 24 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c index f0790e9471d1..cb58f692dad9 100644 --- a/drivers/gpu/drm/scheduler/sched_entity.c +++ b/drivers/gpu/drm/scheduler/sched_entity.c @@ -116,7 +116,8 @@ static bool drm_sched_entity_is_idle(struct drm_sched_entity *entity) rmb(); /* for list_empty to work without lock */ if (list_empty(&entity->list) || - spsc_queue_count(&entity->job_queue) == 0) + spsc_queue_count(&entity->job_queue) == 0 || + entity->stopped) return true; return false; diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 908b0b56032d..ba087354d0a8 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -897,9 +897,33 @@ EXPORT_SYMBOL(drm_sched_init); */ void drm_sched_fini(struct drm_gpu_scheduler *sched) { + struct drm_sched_entity *s_entity; + int i;
if (sched->thread) kthread_stop(sched->thread); + for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) { + struct drm_sched_rq *rq = &sched->sched_rq[i];
+ if (!rq) + continue;
+ spin_lock(&rq->lock); + list_for_each_entry(s_entity, &rq->entities, list) + /* + * Prevents reinsertion and marks job_queue as idle, + * it will removed from rq in drm_sched_entity_fini + * eventually + */ + s_entity->stopped = true; + spin_unlock(&rq->lock);
+ }
+ /* Wakeup everyone stuck in drm_sched_entity_flush for this scheduler */ + wake_up_all(&sched->job_scheduled);
/* Confirm no work left behind accessing device structures */ cancel_delayed_work_sync(&sched->work_tdr);
On Fri, Apr 30, 2021 at 12:10:57PM -0400, Andrey Grodzovsky wrote:
On 2021-04-30 2:47 a.m., Christian König wrote:
Am 29.04.21 um 19:06 schrieb Andrey Grodzovsky:
On 2021-04-29 3:18 a.m., Christian König wrote:
I need to take another look at this part when I don't have a massive headache any more.
Maybe split the patch set up into different parts, something like:
- Adding general infrastructure.
- Making sure all memory is unpolated.
- Job and fence handling
I am not sure you mean this patch here, maybe another one ? Also note you already RBed it.
No what I meant was to send out the patches before this one as #1 and #2.
That is the easier stuff which can easily go into the drm-misc-next or amd-staging-drm-next branch.
The scheduler stuff certainly need to go into drm-misc-next.
Christian.
Got you. I am fine with it. What we have here is a working hot-unplug code but, one with potential use after free MMIO ranges frpom the zombie device. The followup patches after this patch are all about preventing this and so the patch-set up until this patch including, is functional on it's own. While it's necessary to solve the above issue, it's has complications as can be seen from the discussion with Daniel on later patch in this series. Still, in my opinion it's better to rollout some initial support to hot-unplug without use after free protection then having no support for hot-unplug at all. It will also make the merge work easier as I need to constantly rebase the patches on top latest kernel and solve new regressions.
Daniel - given the arguments above can you sound your opinion on this approach ?
I'm all for incrementally landing this, because it's really hard and tricky. We might need to go back to some of the decisions, or clarify things more, or more headaches and pondering how to fix all the parts that works best to make sure there's no nasty races right across hotunplug if you're unlucky enough.
But yeah better aim for something and then readjust than bikeshed forever out of tree.
Cheers, Daniel
Andrey
Andrey
Christian.
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
Problem: If scheduler is already stopped by the time sched_entity is released and entity's job_queue not empty I encountred a hang in drm_sched_entity_flush. This is because drm_sched_entity_is_idle never becomes false.
Fix: In drm_sched_fini detach all sched_entities from the scheduler's run queues. This will satisfy drm_sched_entity_is_idle. Also wakeup all those processes stuck in sched_entity flushing as the scheduler main thread which wakes them up is stopped by now.
v2: Reverse order of drm_sched_rq_remove_entity and marking s_entity as stopped to prevent reinserion back to rq due to race.
v3: Drop drm_sched_rq_remove_entity, only modify entity->stopped and check for it in drm_sched_entity_is_idle
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com Reviewed-by: Christian König christian.koenig@amd.com
drivers/gpu/drm/scheduler/sched_entity.c | 3 ++- drivers/gpu/drm/scheduler/sched_main.c | 24 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c index f0790e9471d1..cb58f692dad9 100644 --- a/drivers/gpu/drm/scheduler/sched_entity.c +++ b/drivers/gpu/drm/scheduler/sched_entity.c @@ -116,7 +116,8 @@ static bool drm_sched_entity_is_idle(struct drm_sched_entity *entity) rmb(); /* for list_empty to work without lock */ if (list_empty(&entity->list) || - spsc_queue_count(&entity->job_queue) == 0) + spsc_queue_count(&entity->job_queue) == 0 || + entity->stopped) return true; return false; diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 908b0b56032d..ba087354d0a8 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -897,9 +897,33 @@ EXPORT_SYMBOL(drm_sched_init); */ void drm_sched_fini(struct drm_gpu_scheduler *sched) { + struct drm_sched_entity *s_entity; + int i;
if (sched->thread) kthread_stop(sched->thread); + for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) { + struct drm_sched_rq *rq = &sched->sched_rq[i];
+ if (!rq) + continue;
+ spin_lock(&rq->lock); + list_for_each_entry(s_entity, &rq->entities, list) + /* + * Prevents reinsertion and marks job_queue as idle, + * it will removed from rq in drm_sched_entity_fini + * eventually + */ + s_entity->stopped = true; + spin_unlock(&rq->lock);
+ }
+ /* Wakeup everyone stuck in drm_sched_entity_flush for this scheduler */ + wake_up_all(&sched->job_scheduled);
/* Confirm no work left behind accessing device structures */ cancel_delayed_work_sync(&sched->work_tdr);
On 2021-05-07 12:29 p.m., Daniel Vetter wrote:
On Fri, Apr 30, 2021 at 12:10:57PM -0400, Andrey Grodzovsky wrote:
On 2021-04-30 2:47 a.m., Christian König wrote:
Am 29.04.21 um 19:06 schrieb Andrey Grodzovsky:
On 2021-04-29 3:18 a.m., Christian König wrote:
I need to take another look at this part when I don't have a massive headache any more.
Maybe split the patch set up into different parts, something like:
- Adding general infrastructure.
- Making sure all memory is unpolated.
- Job and fence handling
I am not sure you mean this patch here, maybe another one ? Also note you already RBed it.
No what I meant was to send out the patches before this one as #1 and #2.
That is the easier stuff which can easily go into the drm-misc-next or amd-staging-drm-next branch.
The scheduler stuff certainly need to go into drm-misc-next.
Christian.
Got you. I am fine with it. What we have here is a working hot-unplug code but, one with potential use after free MMIO ranges frpom the zombie device. The followup patches after this patch are all about preventing this and so the patch-set up until this patch including, is functional on it's own. While it's necessary to solve the above issue, it's has complications as can be seen from the discussion with Daniel on later patch in this series. Still, in my opinion it's better to rollout some initial support to hot-unplug without use after free protection then having no support for hot-unplug at all. It will also make the merge work easier as I need to constantly rebase the patches on top latest kernel and solve new regressions.
Daniel - given the arguments above can you sound your opinion on this approach ?
I'm all for incrementally landing this, because it's really hard and tricky. We might need to go back to some of the decisions, or clarify things more, or more headaches and pondering how to fix all the parts that works best to make sure there's no nasty races right across hotunplug if you're unlucky enough.
But yeah better aim for something and then readjust than bikeshed forever out of tree.
Cheers, Daniel
Thanks, I will send out V6 limited in scope up to here and fixing any relevant comments.
Andrey
Andrey
Andrey
Christian.
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
Problem: If scheduler is already stopped by the time sched_entity is released and entity's job_queue not empty I encountred a hang in drm_sched_entity_flush. This is because drm_sched_entity_is_idle never becomes false.
Fix: In drm_sched_fini detach all sched_entities from the scheduler's run queues. This will satisfy drm_sched_entity_is_idle. Also wakeup all those processes stuck in sched_entity flushing as the scheduler main thread which wakes them up is stopped by now.
v2: Reverse order of drm_sched_rq_remove_entity and marking s_entity as stopped to prevent reinserion back to rq due to race.
v3: Drop drm_sched_rq_remove_entity, only modify entity->stopped and check for it in drm_sched_entity_is_idle
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com Reviewed-by: Christian König christian.koenig@amd.com
drivers/gpu/drm/scheduler/sched_entity.c | 3 ++- drivers/gpu/drm/scheduler/sched_main.c | 24 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c index f0790e9471d1..cb58f692dad9 100644 --- a/drivers/gpu/drm/scheduler/sched_entity.c +++ b/drivers/gpu/drm/scheduler/sched_entity.c @@ -116,7 +116,8 @@ static bool drm_sched_entity_is_idle(struct drm_sched_entity *entity) rmb(); /* for list_empty to work without lock */ if (list_empty(&entity->list) || - spsc_queue_count(&entity->job_queue) == 0) + spsc_queue_count(&entity->job_queue) == 0 || + entity->stopped) return true; return false; diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 908b0b56032d..ba087354d0a8 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -897,9 +897,33 @@ EXPORT_SYMBOL(drm_sched_init); */ void drm_sched_fini(struct drm_gpu_scheduler *sched) { + struct drm_sched_entity *s_entity; + int i;
if (sched->thread) kthread_stop(sched->thread); + for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) { + struct drm_sched_rq *rq = &sched->sched_rq[i];
+ if (!rq) + continue;
+ spin_lock(&rq->lock); + list_for_each_entry(s_entity, &rq->entities, list) + /* + * Prevents reinsertion and marks job_queue as idle, + * it will removed from rq in drm_sched_entity_fini + * eventually + */ + s_entity->stopped = true; + spin_unlock(&rq->lock);
+ }
+ /* Wakeup everyone stuck in drm_sched_entity_flush for this scheduler */ + wake_up_all(&sched->job_scheduled);
/* Confirm no work left behind accessing device structures */ cancel_delayed_work_sync(&sched->work_tdr);
Access to those must be prevented post pci_remove
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 5 +++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 38 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 28 ++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 5 +++ 4 files changed, 71 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 30a24db5f4d1..3e4755fc10c8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1056,6 +1056,11 @@ struct amdgpu_device { struct pci_saved_state *pci_state;
struct list_head device_bo_list; + + /* List of all MMIO BOs */ + struct list_head mmio_list; + struct mutex mmio_list_lock; + };
static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 22b09c4db255..3ddad6cba62d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3320,6 +3320,9 @@ int amdgpu_device_init(struct amdgpu_device *adev, INIT_LIST_HEAD(&adev->shadow_list); mutex_init(&adev->shadow_list_lock);
+ INIT_LIST_HEAD(&adev->mmio_list); + mutex_init(&adev->mmio_list_lock); + INIT_DELAYED_WORK(&adev->delayed_init_work, amdgpu_device_delayed_init_work_handler); INIT_DELAYED_WORK(&adev->gfx.gfx_off_delay_work, @@ -3636,6 +3639,36 @@ static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev) spin_unlock(&adev->mman.bdev.lru_lock); }
+static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev) +{ + struct amdgpu_bo *bo; + + /* Clear all CPU mappings pointing to this device */ + unmap_mapping_range(adev->ddev.anon_inode->i_mapping, 0, 0, 1); + + /* Unmap all MMIO mapped kernel BOs */ + mutex_lock(&adev->mmio_list_lock); + list_for_each_entry(bo, &adev->mmio_list, mmio_list) { + amdgpu_bo_kunmap(bo); + if (*bo->kmap_ptr) + *bo->kmap_ptr = NULL; + } + mutex_unlock(&adev->mmio_list_lock); + + /* Unmap all mapped bars - Doorbell, registers and VRAM */ + amdgpu_device_doorbell_fini(adev); + + iounmap(adev->rmmio); + adev->rmmio = NULL; + if (adev->mman.aper_base_kaddr) + iounmap(adev->mman.aper_base_kaddr); + adev->mman.aper_base_kaddr = NULL; + + /* Memory manager related */ + arch_phys_wc_del(adev->gmc.vram_mtrr); + arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper_size); +} + /** * amdgpu_device_fini - tear down the driver * @@ -3683,6 +3716,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_clear_dma_mappings(adev);
amdgpu_gart_dummy_page_fini(adev); + + amdgpu_device_unmap_mmio(adev); }
void amdgpu_device_fini_sw(struct amdgpu_device *adev) @@ -3713,9 +3748,6 @@ void amdgpu_device_fini_sw(struct amdgpu_device *adev) if (adev->rio_mem) pci_iounmap(adev->pdev, adev->rio_mem); adev->rio_mem = NULL; - iounmap(adev->rmmio); - adev->rmmio = NULL; - amdgpu_device_doorbell_fini(adev);
if (IS_ENABLED(CONFIG_PERF_EVENTS)) amdgpu_pmu_fini(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 62d829f5e62c..9b05e3b96fa0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -531,6 +531,9 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, return -ENOMEM; drm_gem_private_object_init(adev_to_drm(adev), &bo->tbo.base, size); INIT_LIST_HEAD(&bo->shadow_list); + + INIT_LIST_HEAD(&bo->mmio_list); + bo->vm_bo = NULL; bo->preferred_domains = bp->preferred_domain ? bp->preferred_domain : bp->domain; @@ -774,9 +777,21 @@ int amdgpu_bo_kmap(struct amdgpu_bo *bo, void **ptr) if (r) return r;
- if (ptr) + if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap) { + struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); + + mutex_lock(&adev->mmio_list_lock); + list_add_tail(&bo->mmio_list, &adev->mmio_list); + mutex_unlock(&adev->mmio_list_lock); + } + + if (ptr) { *ptr = amdgpu_bo_kptr(bo);
+ if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap) + bo->kmap_ptr = ptr; + } + return 0; }
@@ -804,8 +819,17 @@ void *amdgpu_bo_kptr(struct amdgpu_bo *bo) */ void amdgpu_bo_kunmap(struct amdgpu_bo *bo) { - if (bo->kmap.bo) + struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); + + if (bo->kmap.bo) { + if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap) { + mutex_lock(&adev->mmio_list_lock); + list_del_init(&bo->mmio_list); + mutex_unlock(&adev->mmio_list_lock); + } + ttm_bo_kunmap(&bo->kmap); + } }
/** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index 5ae8555ef275..3129d9bbfa22 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -112,6 +112,11 @@ struct amdgpu_bo { struct kgd_mem *kfd_bo;
struct list_head bo; + + struct list_head mmio_list; + /* Address of kernel VA pointer to MMIO so they can be updated post remap */ + void **kmap_ptr; + };
static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo)
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
Access to those must be prevented post pci_remove
That is certainly a no-go. We want to get rid of the kernel pointers in BOs, not add another one.
Christian.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 5 +++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 38 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 28 ++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 5 +++ 4 files changed, 71 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 30a24db5f4d1..3e4755fc10c8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1056,6 +1056,11 @@ struct amdgpu_device { struct pci_saved_state *pci_state;
struct list_head device_bo_list;
/* List of all MMIO BOs */
struct list_head mmio_list;
struct mutex mmio_list_lock;
};
static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 22b09c4db255..3ddad6cba62d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3320,6 +3320,9 @@ int amdgpu_device_init(struct amdgpu_device *adev, INIT_LIST_HEAD(&adev->shadow_list); mutex_init(&adev->shadow_list_lock);
- INIT_LIST_HEAD(&adev->mmio_list);
- mutex_init(&adev->mmio_list_lock);
- INIT_DELAYED_WORK(&adev->delayed_init_work, amdgpu_device_delayed_init_work_handler); INIT_DELAYED_WORK(&adev->gfx.gfx_off_delay_work,
@@ -3636,6 +3639,36 @@ static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev) spin_unlock(&adev->mman.bdev.lru_lock); }
+static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev) +{
- struct amdgpu_bo *bo;
- /* Clear all CPU mappings pointing to this device */
- unmap_mapping_range(adev->ddev.anon_inode->i_mapping, 0, 0, 1);
- /* Unmap all MMIO mapped kernel BOs */
- mutex_lock(&adev->mmio_list_lock);
- list_for_each_entry(bo, &adev->mmio_list, mmio_list) {
amdgpu_bo_kunmap(bo);if (*bo->kmap_ptr)*bo->kmap_ptr = NULL;- }
- mutex_unlock(&adev->mmio_list_lock);
- /* Unmap all mapped bars - Doorbell, registers and VRAM */
- amdgpu_device_doorbell_fini(adev);
- iounmap(adev->rmmio);
- adev->rmmio = NULL;
- if (adev->mman.aper_base_kaddr)
iounmap(adev->mman.aper_base_kaddr);- adev->mman.aper_base_kaddr = NULL;
- /* Memory manager related */
- arch_phys_wc_del(adev->gmc.vram_mtrr);
- arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper_size);
+}
- /**
- amdgpu_device_fini - tear down the driver
@@ -3683,6 +3716,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_clear_dma_mappings(adev);
amdgpu_gart_dummy_page_fini(adev);
amdgpu_device_unmap_mmio(adev); }
void amdgpu_device_fini_sw(struct amdgpu_device *adev)
@@ -3713,9 +3748,6 @@ void amdgpu_device_fini_sw(struct amdgpu_device *adev) if (adev->rio_mem) pci_iounmap(adev->pdev, adev->rio_mem); adev->rio_mem = NULL;
iounmap(adev->rmmio);
adev->rmmio = NULL;
amdgpu_device_doorbell_fini(adev);
if (IS_ENABLED(CONFIG_PERF_EVENTS)) amdgpu_pmu_fini(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 62d829f5e62c..9b05e3b96fa0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -531,6 +531,9 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, return -ENOMEM; drm_gem_private_object_init(adev_to_drm(adev), &bo->tbo.base, size); INIT_LIST_HEAD(&bo->shadow_list);
- INIT_LIST_HEAD(&bo->mmio_list);
- bo->vm_bo = NULL; bo->preferred_domains = bp->preferred_domain ? bp->preferred_domain : bp->domain;
@@ -774,9 +777,21 @@ int amdgpu_bo_kmap(struct amdgpu_bo *bo, void **ptr) if (r) return r;
- if (ptr)
if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap) {
struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);mutex_lock(&adev->mmio_list_lock);list_add_tail(&bo->mmio_list, &adev->mmio_list);mutex_unlock(&adev->mmio_list_lock);}
if (ptr) { *ptr = amdgpu_bo_kptr(bo);
if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap)bo->kmap_ptr = ptr;}
return 0; }
@@ -804,8 +819,17 @@ void *amdgpu_bo_kptr(struct amdgpu_bo *bo) */ void amdgpu_bo_kunmap(struct amdgpu_bo *bo) {
- if (bo->kmap.bo)
struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
if (bo->kmap.bo) {
if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap) {mutex_lock(&adev->mmio_list_lock);list_del_init(&bo->mmio_list);mutex_unlock(&adev->mmio_list_lock);}ttm_bo_kunmap(&bo->kmap);
} }
/**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index 5ae8555ef275..3129d9bbfa22 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -112,6 +112,11 @@ struct amdgpu_bo { struct kgd_mem *kfd_bo;
struct list_head bo;
struct list_head mmio_list;
/* Address of kernel VA pointer to MMIO so they can be updated post remap */
void **kmap_ptr;
};
static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo)
On 2021-04-29 3:19 a.m., Christian König wrote:
Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
Access to those must be prevented post pci_remove
That is certainly a no-go. We want to get rid of the kernel pointers in BOs, not add another one.
Christian.
As we discussed internally, will drop the entire explicit BOs unmapping approach as unmapping the VRAM bar alone will give the same results.
Andrey
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 5 +++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 38 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 28 ++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 5 +++ 4 files changed, 71 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 30a24db5f4d1..3e4755fc10c8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1056,6 +1056,11 @@ struct amdgpu_device { struct pci_saved_state *pci_state; struct list_head device_bo_list;
+ /* List of all MMIO BOs */ + struct list_head mmio_list; + struct mutex mmio_list_lock;
}; static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 22b09c4db255..3ddad6cba62d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3320,6 +3320,9 @@ int amdgpu_device_init(struct amdgpu_device *adev, INIT_LIST_HEAD(&adev->shadow_list); mutex_init(&adev->shadow_list_lock); + INIT_LIST_HEAD(&adev->mmio_list); + mutex_init(&adev->mmio_list_lock);
INIT_DELAYED_WORK(&adev->delayed_init_work, amdgpu_device_delayed_init_work_handler); INIT_DELAYED_WORK(&adev->gfx.gfx_off_delay_work, @@ -3636,6 +3639,36 @@ static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev) spin_unlock(&adev->mman.bdev.lru_lock); } +static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev) +{ + struct amdgpu_bo *bo;
+ /* Clear all CPU mappings pointing to this device */ + unmap_mapping_range(adev->ddev.anon_inode->i_mapping, 0, 0, 1);
+ /* Unmap all MMIO mapped kernel BOs */ + mutex_lock(&adev->mmio_list_lock); + list_for_each_entry(bo, &adev->mmio_list, mmio_list) { + amdgpu_bo_kunmap(bo); + if (*bo->kmap_ptr) + *bo->kmap_ptr = NULL; + } + mutex_unlock(&adev->mmio_list_lock);
+ /* Unmap all mapped bars - Doorbell, registers and VRAM */ + amdgpu_device_doorbell_fini(adev);
+ iounmap(adev->rmmio); + adev->rmmio = NULL; + if (adev->mman.aper_base_kaddr) + iounmap(adev->mman.aper_base_kaddr); + adev->mman.aper_base_kaddr = NULL;
+ /* Memory manager related */ + arch_phys_wc_del(adev->gmc.vram_mtrr); + arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper_size); +}
/** * amdgpu_device_fini - tear down the driver * @@ -3683,6 +3716,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_clear_dma_mappings(adev); amdgpu_gart_dummy_page_fini(adev);
+ amdgpu_device_unmap_mmio(adev); } void amdgpu_device_fini_sw(struct amdgpu_device *adev) @@ -3713,9 +3748,6 @@ void amdgpu_device_fini_sw(struct amdgpu_device *adev) if (adev->rio_mem) pci_iounmap(adev->pdev, adev->rio_mem); adev->rio_mem = NULL; - iounmap(adev->rmmio); - adev->rmmio = NULL; - amdgpu_device_doorbell_fini(adev); if (IS_ENABLED(CONFIG_PERF_EVENTS)) amdgpu_pmu_fini(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 62d829f5e62c..9b05e3b96fa0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -531,6 +531,9 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, return -ENOMEM; drm_gem_private_object_init(adev_to_drm(adev), &bo->tbo.base, size); INIT_LIST_HEAD(&bo->shadow_list);
+ INIT_LIST_HEAD(&bo->mmio_list);
bo->vm_bo = NULL; bo->preferred_domains = bp->preferred_domain ? bp->preferred_domain : bp->domain; @@ -774,9 +777,21 @@ int amdgpu_bo_kmap(struct amdgpu_bo *bo, void **ptr) if (r) return r; - if (ptr) + if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap) { + struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
+ mutex_lock(&adev->mmio_list_lock); + list_add_tail(&bo->mmio_list, &adev->mmio_list); + mutex_unlock(&adev->mmio_list_lock); + }
+ if (ptr) { *ptr = amdgpu_bo_kptr(bo); + if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap) + bo->kmap_ptr = ptr; + }
return 0; } @@ -804,8 +819,17 @@ void *amdgpu_bo_kptr(struct amdgpu_bo *bo) */ void amdgpu_bo_kunmap(struct amdgpu_bo *bo) { - if (bo->kmap.bo) + struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
+ if (bo->kmap.bo) { + if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap) { + mutex_lock(&adev->mmio_list_lock); + list_del_init(&bo->mmio_list); + mutex_unlock(&adev->mmio_list_lock); + }
ttm_bo_kunmap(&bo->kmap); + } } /** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index 5ae8555ef275..3129d9bbfa22 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -112,6 +112,11 @@ struct amdgpu_bo { struct kgd_mem *kfd_bo; struct list_head bo;
+ struct list_head mmio_list; + /* Address of kernel VA pointer to MMIO so they can be updated post remap */ + void **kmap_ptr;
}; static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo)
Will be later used block further submissions once device is removed. Also complete schedule fence if scheduling failed due to submission blocking.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 3 +++ drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 13 ++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 14 +++++++++++++- 4 files changed, 29 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 3e4755fc10c8..0db0ba4fba89 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1057,6 +1057,9 @@ struct amdgpu_device {
struct list_head device_bo_list;
+ bool stop_job_submissions; + struct rw_semaphore sched_fence_completion_sem; + /* List of all MMIO BOs */ struct list_head mmio_list; struct mutex mmio_list_lock; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c index 3e240b952e79..ac092a5eb4e7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c @@ -1256,7 +1256,18 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
trace_amdgpu_cs_ioctl(job); amdgpu_vm_bo_trace_cs(&fpriv->vm, &p->ticket); - drm_sched_entity_push_job(&job->base, entity); + + down_read(&p->adev->sched_fence_completion_sem); + if (!p->adev->stop_job_submissions) { + drm_sched_entity_push_job(&job->base, entity); + } else { + dma_fence_set_error(&job->base.s_fence->scheduled, -ENODEV); + dma_fence_set_error(&job->base.s_fence->finished, -ENODEV); + dma_fence_signal(&job->base.s_fence->scheduled); + dma_fence_signal(&job->base.s_fence->finished); + } + + up_read(&p->adev->sched_fence_completion_sem);
amdgpu_vm_move_to_lru_tail(p->adev, &fpriv->vm);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 3ddad6cba62d..33e8e9e1d1fe 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3302,6 +3302,7 @@ int amdgpu_device_init(struct amdgpu_device *adev, init_rwsem(&adev->reset_sem); mutex_init(&adev->psp.mutex); mutex_init(&adev->notifier_lock); + init_rwsem(&adev->sched_fence_completion_sem);
r = amdgpu_device_check_arguments(adev); if (r) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c index d33e6d97cc89..26d8b79ea165 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c @@ -162,6 +162,7 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity, void *owner, struct dma_fence **f) { int r; + struct amdgpu_ring *ring = to_amdgpu_ring(job->base.sched);
if (!f) return -EINVAL; @@ -172,7 +173,18 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
*f = dma_fence_get(&job->base.s_fence->finished); amdgpu_job_free_resources(job); - drm_sched_entity_push_job(&job->base, entity); + + down_read(&ring->adev->sched_fence_completion_sem); + if (!ring->adev->stop_job_submissions) { + drm_sched_entity_push_job(&job->base, entity); + } else { + dma_fence_set_error(&job->base.s_fence->scheduled, -ENODEV); + dma_fence_set_error(&job->base.s_fence->finished, -ENODEV); + dma_fence_signal(&job->base.s_fence->scheduled); + dma_fence_signal(&job->base.s_fence->finished); + + } + up_read(&ring->adev->sched_fence_completion_sem);
return 0; }
Will be used to complete all schedulte fences on device remove
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/scheduler/sched_entity.c | 3 ++- include/drm/gpu_scheduler.h | 1 + 2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c index cb58f692dad9..9ff4bfd8f548 100644 --- a/drivers/gpu/drm/scheduler/sched_entity.c +++ b/drivers/gpu/drm/scheduler/sched_entity.c @@ -219,7 +219,7 @@ static void drm_sched_entity_kill_jobs_cb(struct dma_fence *f, * Makes sure that all remaining jobs in an entity are killed before it is * destroyed. */ -static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity) +void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity) { struct drm_sched_job *job; int r; @@ -249,6 +249,7 @@ static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity) DRM_ERROR("fence add callback failed (%d)\n", r); } } +EXPORT_SYMBOL(drm_sched_entity_kill_jobs);
/** * drm_sched_entity_cleanup - Destroy a context entity diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h index f888b5e9583a..9601d5b966ba 100644 --- a/include/drm/gpu_scheduler.h +++ b/include/drm/gpu_scheduler.h @@ -339,6 +339,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, unsigned int num_sched_list, atomic_t *guilty); long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout); +void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity); void drm_sched_entity_fini(struct drm_sched_entity *entity); void drm_sched_entity_destroy(struct drm_sched_entity *entity); void drm_sched_entity_select_rq(struct drm_sched_entity *entity);
Make sure all fecens dependent on HW present are force signaled when handling device removal. This helpes later to scope all HW accesing code such as IOCTLs in drm_dev_enter/exit and use drm_dev_unplug as synchronization point past which we know HW will not be accessed anymore outside of pci remove driver callback.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 98 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 6 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 12 +-- 4 files changed, 103 insertions(+), 15 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 0db0ba4fba89..df6c5ed676b1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1374,6 +1374,8 @@ void amdgpu_pci_resume(struct pci_dev *pdev); bool amdgpu_device_cache_pci_state(struct pci_dev *pdev); bool amdgpu_device_load_pci_state(struct pci_dev *pdev);
+void amdgpu_finilize_device_fences(struct drm_device *dev); + #include "amdgpu_object.h"
static inline bool amdgpu_is_tmz(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 33e8e9e1d1fe..55afc11c17e6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3692,15 +3692,12 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_virt_fini_data_exchange(adev); }
- /* disable all interrupts */ - amdgpu_irq_disable_all(adev); if (adev->mode_info.mode_config_initialized){ if (!amdgpu_device_has_dc_support(adev)) drm_helper_force_disable_all(adev_to_drm(adev)); else drm_atomic_helper_shutdown(adev_to_drm(adev)); } - amdgpu_fence_driver_fini_hw(adev);
if (adev->pm_sysfs_en) amdgpu_pm_sysfs_fini(adev); @@ -4567,14 +4564,19 @@ static bool amdgpu_device_lock_adev(struct amdgpu_device *adev, return true; }
-static void amdgpu_device_unlock_adev(struct amdgpu_device *adev) +static void amdgpu_device_unlock_adev_imp(struct amdgpu_device *adev, bool skip_in_gpu_reset) { amdgpu_vf_error_trans_all(adev); adev->mp1_state = PP_MP1_STATE_NONE; - atomic_set(&adev->in_gpu_reset, 0); + !skip_in_gpu_reset ? atomic_set(&adev->in_gpu_reset, 0) : 0; up_write(&adev->reset_sem); }
+static void amdgpu_device_unlock_adev(struct amdgpu_device *adev) +{ + amdgpu_device_unlock_adev_imp(adev, false); +} + /* * to lockup a list of amdgpu devices in a hive safely, if not a hive * with multiple nodes, it will be similar as amdgpu_device_lock_adev. @@ -5321,3 +5323,89 @@ bool amdgpu_device_load_pci_state(struct pci_dev *pdev) }
+static void amdgpu_finilize_schedulded_fences(struct amdgpu_ctx_mgr *mgr) +{ + struct amdgpu_ctx *ctx; + struct idr *idp; + uint32_t id, i, j; + + idp = &mgr->ctx_handles; + + idr_for_each_entry(idp, ctx, id) { + for (i = 0; i < AMDGPU_HW_IP_NUM; ++i) { + for (j = 0; j < amdgpu_ctx_num_entities[i]; ++j) { + struct drm_sched_entity *entity; + + if (!ctx->entities[i][j]) + continue; + + entity = &ctx->entities[i][j]->entity; + drm_sched_entity_kill_jobs(entity); + } + } + } +} + +/** + * amdgpu_finilize_device_fences() - Finilize all device fences + * @pdev: pointer to PCI device + * + * Will disable and finilise ISRs and will signal all fences + * that might hang if HW is gone + */ +void amdgpu_finilize_device_fences(struct drm_device *dev) +{ + struct amdgpu_device *adev = drm_to_adev(dev); + struct drm_file *file; + + /* + * Block TDRs from further execution by setting adev->in_gpu_reset + * instead of holding full reset lock in order to not deadlock + * further ahead against any thread locking the reset lock when we + * wait for it's completion + */ + while (!amdgpu_device_lock_adev(adev, NULL)) + amdgpu_cancel_all_tdr(adev); + amdgpu_device_unlock_adev_imp(adev, true); + + + /* disable all HW interrupts */ + amdgpu_irq_disable_all(adev); + + /* stop and flush all in flight HW interrupts handlers */ + disable_irq(pci_irq_vector(adev->pdev, 0)); + + /* + * Stop SW GPU schedulers and force completion on all HW fences. Since + * in the prev. step all ISRs were disabled and completed the + * HW fence array is idle (no insertions or extractions) and so it's + * safe to iterate it bellow. + * After this step all HW fences in the system are signaled. As a result + * also all the scheduler 'finished' fences are also signaled. + */ + amdgpu_fence_driver_fini_hw(adev); + + /* + * Reject any further jobs to any scheduler entity queue. After this + * step no new insertions and because schedulers are stopped also no + * new extractions. + */ + down_read(&adev->sched_fence_completion_sem); + adev->stop_job_submissions = true; + up_read(&adev->sched_fence_completion_sem); + + /* + * Complete all scheduler 'scheduled' fences currently pending. + * It's OK if new contexts and sched entities are concurrently + * still created as they will fail in pushing jobs to SW queues + * and their schedule fences will be signaled with error + */ + mutex_lock(&adev->ddev.filelist_mutex); + list_for_each_entry(file, &adev->ddev.filelist, lhead) { + struct amdgpu_fpriv *fpriv = file->driver_priv; + amdgpu_finilize_schedulded_fences(&fpriv->ctx_mgr); + } + mutex_unlock(&adev->ddev.filelist_mutex); +} + + diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index f799c40d7e72..8a19b8dd02ee 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1249,6 +1249,12 @@ amdgpu_pci_remove(struct pci_dev *pdev) { struct drm_device *dev = pci_get_drvdata(pdev);
+ /* + * Force completion of all device related fences that might hang us when + * synchronizing SRCU in the following step. + */ + amdgpu_finilize_device_fences(dev); + drm_dev_unplug(dev); amdgpu_driver_unload_kms(dev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 2670201e78d3..af592b28cd35 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -526,7 +526,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev) */ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) { - int i, r; + int i;
for (i = 0; i < AMDGPU_MAX_RINGS; i++) { struct amdgpu_ring *ring = adev->rings[i]; @@ -535,18 +535,10 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) continue;
/* Stop any new job submissions from sched before flushing the ring */ - /* TODO Handle amdgpu_job_submit_direct and amdgpu_amdkfd_submit_ib */ if (!ring->no_scheduler) drm_sched_fini(&ring->sched);
- /* You can't wait for HW to signal if it's gone */ - if (!drm_dev_is_unplugged(&adev->ddev)) - r = amdgpu_fence_wait_empty(ring); - else - r = -ENODEV; - /* no need to trigger GPU reset as we are unloading */ - if (r) - amdgpu_fence_driver_force_completion(ring); + amdgpu_fence_driver_force_completion(ring);
if (ring->fence_drv.irq_src) amdgpu_irq_put(adev, ring->fence_drv.irq_src,
On 2021-04-28 11:11 a.m., Andrey Grodzovsky wrote:
Make sure all fecens dependent on HW present are force signaled when handling device removal. This helpes later to scope all HW accesing code such as IOCTLs in drm_dev_enter/exit and use drm_dev_unplug as synchronization point past which we know HW will not be accessed anymore outside of pci remove driver callback.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 98 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 6 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 12 +-- 4 files changed, 103 insertions(+), 15 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 0db0ba4fba89..df6c5ed676b1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1374,6 +1374,8 @@ void amdgpu_pci_resume(struct pci_dev *pdev); bool amdgpu_device_cache_pci_state(struct pci_dev *pdev); bool amdgpu_device_load_pci_state(struct pci_dev *pdev);
+void amdgpu_finilize_device_fences(struct drm_device *dev);
#include "amdgpu_object.h"
static inline bool amdgpu_is_tmz(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 33e8e9e1d1fe..55afc11c17e6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3692,15 +3692,12 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_virt_fini_data_exchange(adev); }
/* disable all interrupts */
amdgpu_irq_disable_all(adev); if (adev->mode_info.mode_config_initialized){ if (!amdgpu_device_has_dc_support(adev)) drm_helper_force_disable_all(adev_to_drm(adev)); else drm_atomic_helper_shutdown(adev_to_drm(adev)); }
amdgpu_fence_driver_fini_hw(adev);
if (adev->pm_sysfs_en) amdgpu_pm_sysfs_fini(adev);
@@ -4567,14 +4564,19 @@ static bool amdgpu_device_lock_adev(struct amdgpu_device *adev, return true; }
-static void amdgpu_device_unlock_adev(struct amdgpu_device *adev) +static void amdgpu_device_unlock_adev_imp(struct amdgpu_device *adev, bool skip_in_gpu_reset) { amdgpu_vf_error_trans_all(adev); adev->mp1_state = PP_MP1_STATE_NONE;
- atomic_set(&adev->in_gpu_reset, 0);
- !skip_in_gpu_reset ? atomic_set(&adev->in_gpu_reset, 0) : 0; up_write(&adev->reset_sem); }
+static void amdgpu_device_unlock_adev(struct amdgpu_device *adev) +{
- amdgpu_device_unlock_adev_imp(adev, false);
+}
- /*
- to lockup a list of amdgpu devices in a hive safely, if not a hive
- with multiple nodes, it will be similar as amdgpu_device_lock_adev.
@@ -5321,3 +5323,89 @@ bool amdgpu_device_load_pci_state(struct pci_dev *pdev) }
+static void amdgpu_finilize_schedulded_fences(struct amdgpu_ctx_mgr *mgr) +{
- struct amdgpu_ctx *ctx;
- struct idr *idp;
- uint32_t id, i, j;
- idp = &mgr->ctx_handles;
- idr_for_each_entry(idp, ctx, id) {
for (i = 0; i < AMDGPU_HW_IP_NUM; ++i) {for (j = 0; j < amdgpu_ctx_num_entities[i]; ++j) {struct drm_sched_entity *entity;if (!ctx->entities[i][j])continue;entity = &ctx->entities[i][j]->entity;drm_sched_entity_kill_jobs(entity);}}- }
+}
+/**
- amdgpu_finilize_device_fences() - Finilize all device fences
- @pdev: pointer to PCI device
- Will disable and finilise ISRs and will signal all fences
- that might hang if HW is gone
- */
+void amdgpu_finilize_device_fences(struct drm_device *dev) +{
- struct amdgpu_device *adev = drm_to_adev(dev);
- struct drm_file *file;
- /*
* Block TDRs from further execution by setting adev->in_gpu_reset* instead of holding full reset lock in order to not deadlock* further ahead against any thread locking the reset lock when we* wait for it's completion*/- while (!amdgpu_device_lock_adev(adev, NULL))
amdgpu_cancel_all_tdr(adev);- amdgpu_device_unlock_adev_imp(adev, true);
- /* disable all HW interrupts */
- amdgpu_irq_disable_all(adev);
- /* stop and flush all in flight HW interrupts handlers */
- disable_irq(pci_irq_vector(adev->pdev, 0));
- /*
* Stop SW GPU schedulers and force completion on all HW fences. Since* in the prev. step all ISRs were disabled and completed the* HW fence array is idle (no insertions or extractions) and so it's* safe to iterate it bellow.* After this step all HW fences in the system are signaled. As a result* also all the scheduler 'finished' fences are also signaled.*/- amdgpu_fence_driver_fini_hw(adev);
- /*
* Reject any further jobs to any scheduler entity queue. After this* step no new insertions and because schedulers are stopped also no* new extractions.*/- down_read(&adev->sched_fence_completion_sem);
- adev->stop_job_submissions = true;
- up_read(&adev->sched_fence_completion_sem);
FYI: Typo here - down_write and up_write obviously.
Andrey
- /*
* Complete all scheduler 'scheduled' fences currently pending.* It's OK if new contexts and sched entities are concurrently* still created as they will fail in pushing jobs to SW queues* and their schedule fences will be signaled with error*/- mutex_lock(&adev->ddev.filelist_mutex);
- list_for_each_entry(file, &adev->ddev.filelist, lhead) {
struct amdgpu_fpriv *fpriv = file->driver_priv;amdgpu_finilize_schedulded_fences(&fpriv->ctx_mgr);- }
- mutex_unlock(&adev->ddev.filelist_mutex);
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index f799c40d7e72..8a19b8dd02ee 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1249,6 +1249,12 @@ amdgpu_pci_remove(struct pci_dev *pdev) { struct drm_device *dev = pci_get_drvdata(pdev);
- /*
* Force completion of all device related fences that might hang us when* synchronizing SRCU in the following step.*/- amdgpu_finilize_device_fences(dev);
- drm_dev_unplug(dev); amdgpu_driver_unload_kms(dev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 2670201e78d3..af592b28cd35 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -526,7 +526,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev) */ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) {
- int i, r;
int i;
for (i = 0; i < AMDGPU_MAX_RINGS; i++) { struct amdgpu_ring *ring = adev->rings[i];
@@ -535,18 +535,10 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) continue;
/* Stop any new job submissions from sched before flushing the ring */
/* TODO Handle amdgpu_job_submit_direct and amdgpu_amdkfd_submit_ib */if (!ring->no_scheduler) drm_sched_fini(&ring->sched);
/* You can't wait for HW to signal if it's gone */if (!drm_dev_is_unplugged(&adev->ddev))r = amdgpu_fence_wait_empty(ring);elser = -ENODEV;/* no need to trigger GPU reset as we are unloading */if (r)amdgpu_fence_driver_force_completion(ring);
amdgpu_fence_driver_force_completion(ring);if (ring->fence_drv.irq_src) amdgpu_irq_put(adev, ring->fence_drv.irq_src,
With this calling drm_dev_unplug will flush and block all in flight IOCTLs
Also, add feature such that if device supports graceful unplug we enclose entire IOCTL in SRCU critical section.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/drm_ioctl.c | 15 +++++++++++++-- include/drm/drm_drv.h | 6 ++++++ 2 files changed, 19 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/drm_ioctl.c b/drivers/gpu/drm/drm_ioctl.c index d273d1a8603a..5882ef2183bb 100644 --- a/drivers/gpu/drm/drm_ioctl.c +++ b/drivers/gpu/drm/drm_ioctl.c @@ -815,7 +815,7 @@ long drm_ioctl(struct file *filp, const struct drm_ioctl_desc *ioctl = NULL; drm_ioctl_t *func; unsigned int nr = DRM_IOCTL_NR(cmd); - int retcode = -EINVAL; + int idx, retcode = -EINVAL; char stack_kdata[128]; char *kdata = NULL; unsigned int in_size, out_size, drv_size, ksize; @@ -884,7 +884,18 @@ long drm_ioctl(struct file *filp, if (ksize > in_size) memset(kdata + in_size, 0, ksize - in_size);
- retcode = drm_ioctl_kernel(filp, func, kdata, ioctl->flags); + if (drm_core_check_feature(dev, DRIVER_HOTUNPLUG_SUPPORT)) { + if (drm_dev_enter(dev, &idx)) { + retcode = drm_ioctl_kernel(filp, func, kdata, ioctl->flags); + drm_dev_exit(idx); + } else { + retcode = -ENODEV; + goto err_i1; + } + } else { + retcode = drm_ioctl_kernel(filp, func, kdata, ioctl->flags); + } + if (copy_to_user((void __user *)arg, kdata, out_size) != 0) retcode = -EFAULT;
diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h index b439ae1921b8..63e05cec46c1 100644 --- a/include/drm/drm_drv.h +++ b/include/drm/drm_drv.h @@ -94,6 +94,12 @@ enum drm_driver_feature { * synchronization of command submission. */ DRIVER_SYNCOBJ_TIMELINE = BIT(6), + /** + * @DRIVER_NO_HOTUNPLUG_SUPPORT: + * + * Driver support gracefull remove. + */ + DRIVER_HOTUNPLUG_SUPPORT = BIT(7),
/* IMPORTANT: Below are all the legacy flags, add new ones above. */
On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
With this calling drm_dev_unplug will flush and block all in flight IOCTLs
Also, add feature such that if device supports graceful unplug we enclose entire IOCTL in SRCU critical section.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
Nope.
The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
Especially not with an opt-in flag so that it could be shrugged of as a driver hack. Most of these ioctls should have absolutely no problem working after hotunplug.
Also, doing this defeats the point since it pretty much guarantees userspace will die in assert()s and stuff. E.g. on i915 the rough contract is that only execbuf (and even that only when userspace has indicated support for non-recoverable hw ctx) is allowed to fail. Anything else might crash userspace.
You probably need similar (and very precisely defined) rules for amdgpu. And those must definitely exclude any shard ioctls from randomly failing with EIO, because that just kills the box and defeats the point of trying to gracefully handling hotunplug and making sure userspace has a chance of survival. E.g. for atomic everything should continue, including flip completion, but we set all outputs to "disconnected" and send out the uevent. Maybe crtc enabling can fail too, but that can also be handled through the async status we're using to signal DP link failures to userspace.
I guess we should clarify this in the hotunplug doc?
Cheers, Daniel
drivers/gpu/drm/drm_ioctl.c | 15 +++++++++++++-- include/drm/drm_drv.h | 6 ++++++ 2 files changed, 19 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/drm_ioctl.c b/drivers/gpu/drm/drm_ioctl.c index d273d1a8603a..5882ef2183bb 100644 --- a/drivers/gpu/drm/drm_ioctl.c +++ b/drivers/gpu/drm/drm_ioctl.c @@ -815,7 +815,7 @@ long drm_ioctl(struct file *filp, const struct drm_ioctl_desc *ioctl = NULL; drm_ioctl_t *func; unsigned int nr = DRM_IOCTL_NR(cmd);
- int retcode = -EINVAL;
- int idx, retcode = -EINVAL; char stack_kdata[128]; char *kdata = NULL; unsigned int in_size, out_size, drv_size, ksize;
@@ -884,7 +884,18 @@ long drm_ioctl(struct file *filp, if (ksize > in_size) memset(kdata + in_size, 0, ksize - in_size);
- retcode = drm_ioctl_kernel(filp, func, kdata, ioctl->flags);
- if (drm_core_check_feature(dev, DRIVER_HOTUNPLUG_SUPPORT)) {
if (drm_dev_enter(dev, &idx)) {retcode = drm_ioctl_kernel(filp, func, kdata, ioctl->flags);drm_dev_exit(idx);} else {retcode = -ENODEV;goto err_i1;}- } else {
retcode = drm_ioctl_kernel(filp, func, kdata, ioctl->flags);- }
- if (copy_to_user((void __user *)arg, kdata, out_size) != 0) retcode = -EFAULT;
diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h index b439ae1921b8..63e05cec46c1 100644 --- a/include/drm/drm_drv.h +++ b/include/drm/drm_drv.h @@ -94,6 +94,12 @@ enum drm_driver_feature { * synchronization of command submission. */ DRIVER_SYNCOBJ_TIMELINE = BIT(6),
/**
* @DRIVER_NO_HOTUNPLUG_SUPPORT:** Driver support gracefull remove.*/DRIVER_HOTUNPLUG_SUPPORT = BIT(7),
/* IMPORTANT: Below are all the legacy flags, add new ones above. */
-- 2.25.1
On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
With this calling drm_dev_unplug will flush and block all in flight IOCTLs
Also, add feature such that if device supports graceful unplug we enclose entire IOCTL in SRCU critical section.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
Nope.
The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
Especially not with an opt-in flag so that it could be shrugged of as a driver hack. Most of these ioctls should have absolutely no problem working after hotunplug.
Also, doing this defeats the point since it pretty much guarantees userspace will die in assert()s and stuff. E.g. on i915 the rough contract is that only execbuf (and even that only when userspace has indicated support for non-recoverable hw ctx) is allowed to fail. Anything else might crash userspace.
You probably need similar (and very precisely defined) rules for amdgpu. And those must definitely exclude any shard ioctls from randomly failing with EIO, because that just kills the box and defeats the point of trying to gracefully handling hotunplug and making sure userspace has a chance of survival. E.g. for atomic everything should continue, including flip completion, but we set all outputs to "disconnected" and send out the uevent. Maybe crtc enabling can fail too, but that can also be handled through the async status we're using to signal DP link failures to userspace.
I guess we should clarify this in the hotunplug doc?
To clarify: I'm not against throwing an ENODEV at userspace for ioctl that really make no sense, and where we're rather confident that all properly implemented userspace will gracefully handle failures. Like a modeset, or opening a device, or trying to import a dma-buf or stuff like that which can already fail in normal operation for any kind of reason.
But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail after hotunplug.
And then there's the middle ground, like doing a pageflip or buffer flush, which I guess some userspace might handle, but risky to inflict those consequences on them. atomic modeset is especially fun since depending what you're doing it can be both "failures expected" and "failures not really expected in normal operation".
Also, this really should be consistent across drivers, not solved with a driver flag for every possible combination.
If you look at the current hotunplug kms drivers, they have drm_dev_enter/exit sprinkled in specific hw callback functions because of the above problems. But maybe it makes sense to change things in a few cases. But then we should do it across the board.
Cheers, Daniel
On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
With this calling drm_dev_unplug will flush and block all in flight IOCTLs
Also, add feature such that if device supports graceful unplug we enclose entire IOCTL in SRCU critical section.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
Nope.
The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
Then I am confused why we have https://elixir.bootlin.com/linux/v5.12/source/drivers/gpu/drm/drm_ioctl.c#L8... currently in code ?
Especially not with an opt-in flag so that it could be shrugged of as a driver hack. Most of these ioctls should have absolutely no problem working after hotunplug.
Also, doing this defeats the point since it pretty much guarantees userspace will die in assert()s and stuff. E.g. on i915 the rough contract is that only execbuf (and even that only when userspace has indicated support for non-recoverable hw ctx) is allowed to fail. Anything else might crash userspace.
Given that as I pointed above we already fail any IOCTls with -ENODEV when device is unplugged, it seems those crashes don't happen that often ? Also, in all my testing I don't think I saw a user space crash I could attribute to this.
You probably need similar (and very precisely defined) rules for amdgpu. And those must definitely exclude any shard ioctls from randomly failing with EIO, because that just kills the box and defeats the point of trying to gracefully handling hotunplug and making sure userspace has a chance of survival. E.g. for atomic everything should continue, including flip completion, but we set all outputs to "disconnected" and send out the uevent. Maybe crtc enabling can fail too, but that can also be handled through the async status we're using to signal DP link failures to userspace.
As I pointed before, because of the complexity of the topic I prefer to take it step by step and solve first for secondary device use case, not for primary, display attached device.
I guess we should clarify this in the hotunplug doc?
Agree
To clarify: I'm not against throwing an ENODEV at userspace for ioctl that really make no sense, and where we're rather confident that all properly implemented userspace will gracefully handle failures. Like a modeset, or opening a device, or trying to import a dma-buf or stuff like that which can already fail in normal operation for any kind of reason.
But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail after hotunplug.
As I pointed above, this a bit confuses me given that we already do blanker rejection of IOCTLs if device is unplugged.
And then there's the middle ground, like doing a pageflip or buffer flush, which I guess some userspace might handle, but risky to inflict those consequences on them. atomic modeset is especially fun since depending what you're doing it can be both "failures expected" and "failures not really expected in normal operation".
Also, this really should be consistent across drivers, not solved with a driver flag for every possible combination.
If you look at the current hotunplug kms drivers, they have drm_dev_enter/exit sprinkled in specific hw callback functions because of the above problems. But maybe it makes sense to change things in a few cases. But then we should do it across the board.
So as I understand your preferred approach is that I scope any back_end, HW specific function with drm_dev_enter/exit because that where MMIO access takes place. But besides explicit MMIO access thorough register accessors in the HW back-end there is also indirect MMIO access taking place throughout the code in the driver because of various VRAM BOs which provide CPU access to VRAM through the VRAM BAR. This kind of access is spread all over in the driver and even in mid-layers such as TTM and not limited to HW back-end functions. It means it's much harder to spot such places to surgically scope them with drm_dev_enter/exit and also that any new such code introduced will immediately break hot unplug because the developers can't be expected to remember making their code robust to this specific use case. That why when we discussed internally what approach to take to protecting code with drm_dev_enter/exit we opted for using the widest available scope.
Andrey
Cheers, Daniel
Am 2021-04-29 um 12:04 p.m. schrieb Andrey Grodzovsky:
So as I understand your preferred approach is that I scope any back_end, HW specific function with drm_dev_enter/exit because that where MMIO access takes place. But besides explicit MMIO access thorough register accessors in the HW back-end there is also indirect MMIO access taking place throughout the code in the driver because of various VRAM BOs which provide CPU access to VRAM through the VRAM BAR. This kind of access is spread all over in the driver and even in mid-layers such as TTM and not limited to HW back-end functions. It means it's much harder to spot such places to surgically scope them with drm_dev_enter/exit and also that any new such code introduced will immediately break hot unplug because the developers can't be expected to remember making their code robust to this specific use case. That why when we discussed internally what approach to take to protecting code with drm_dev_enter/exit we opted for using the widest available scope.
VRAM can also be mapped in user mode. Is there anything preventing user mode from accessing the memory after unplug? I guess the best you could do is unmap it from the CPU page table and let the application segfault on the next access. Or replace the mapping with a dummy page in system memory?
Regards, Felix
Andrey
On 2021-04-29 12:15 p.m., Felix Kuehling wrote:
Am 2021-04-29 um 12:04 p.m. schrieb Andrey Grodzovsky:
So as I understand your preferred approach is that I scope any back_end, HW specific function with drm_dev_enter/exit because that where MMIO access takes place. But besides explicit MMIO access thorough register accessors in the HW back-end there is also indirect MMIO access taking place throughout the code in the driver because of various VRAM BOs which provide CPU access to VRAM through the VRAM BAR. This kind of access is spread all over in the driver and even in mid-layers such as TTM and not limited to HW back-end functions. It means it's much harder to spot such places to surgically scope them with drm_dev_enter/exit and also that any new such code introduced will immediately break hot unplug because the developers can't be expected to remember making their code robust to this specific use case. That why when we discussed internally what approach to take to protecting code with drm_dev_enter/exit we opted for using the widest available scope.
VRAM can also be mapped in user mode. Is there anything preventing user mode from accessing the memory after unplug? I guess the best you could do is unmap it from the CPU page table and let the application segfault on the next access. Or replace the mapping with a dummy page in system memory?
We indeed unmap but instead of letting it segfault insert dummy page on the next page fault. See here https://cgit.freedesktop.org/~agrodzov/linux/commit/?h=drm-misc-next&id=... And I am aware that this doesn't take care of KFD user mapping. As you know, we had some discussions with you on this topic and it's on my TODO list to follow up on this to solve it for KFD too.
Andrey
Regards, Felix
Andrey
Am 2021-04-29 um 12:21 p.m. schrieb Andrey Grodzovsky:
On 2021-04-29 12:15 p.m., Felix Kuehling wrote:
Am 2021-04-29 um 12:04 p.m. schrieb Andrey Grodzovsky:
So as I understand your preferred approach is that I scope any back_end, HW specific function with drm_dev_enter/exit because that where MMIO access takes place. But besides explicit MMIO access thorough register accessors in the HW back-end there is also indirect MMIO access taking place throughout the code in the driver because of various VRAM BOs which provide CPU access to VRAM through the VRAM BAR. This kind of access is spread all over in the driver and even in mid-layers such as TTM and not limited to HW back-end functions. It means it's much harder to spot such places to surgically scope them with drm_dev_enter/exit and also that any new such code introduced will immediately break hot unplug because the developers can't be expected to remember making their code robust to this specific use case. That why when we discussed internally what approach to take to protecting code with drm_dev_enter/exit we opted for using the widest available scope.
VRAM can also be mapped in user mode. Is there anything preventing user mode from accessing the memory after unplug? I guess the best you could do is unmap it from the CPU page table and let the application segfault on the next access. Or replace the mapping with a dummy page in system memory?
We indeed unmap but instead of letting it segfault insert dummy page on the next page fault. See here https://cgit.freedesktop.org/~agrodzov/linux/commit/?h=drm-misc-next&id=... And I am aware that this doesn't take care of KFD user mapping. As you know, we had some discussions with you on this topic and it's on my TODO list to follow up on this to solve it for KFD too.
ROCm user mode maps VRAM BOs using render nodes. So I'd expect ttm_bo_vm_dummy_page to work for KFD as well.
I guess we'd need something special for KFD's doorbell and MMIO (HDP flush) mappings. Was that the discussion about the file address space?
Regards, Felix
Andrey
Regards, Felix
Andrey
On 2021-04-29 12:29 p.m., Felix Kuehling wrote:
Am 2021-04-29 um 12:21 p.m. schrieb Andrey Grodzovsky:
On 2021-04-29 12:15 p.m., Felix Kuehling wrote:
Am 2021-04-29 um 12:04 p.m. schrieb Andrey Grodzovsky:
So as I understand your preferred approach is that I scope any back_end, HW specific function with drm_dev_enter/exit because that where MMIO access takes place. But besides explicit MMIO access thorough register accessors in the HW back-end there is also indirect MMIO access taking place throughout the code in the driver because of various VRAM BOs which provide CPU access to VRAM through the VRAM BAR. This kind of access is spread all over in the driver and even in mid-layers such as TTM and not limited to HW back-end functions. It means it's much harder to spot such places to surgically scope them with drm_dev_enter/exit and also that any new such code introduced will immediately break hot unplug because the developers can't be expected to remember making their code robust to this specific use case. That why when we discussed internally what approach to take to protecting code with drm_dev_enter/exit we opted for using the widest available scope.
VRAM can also be mapped in user mode. Is there anything preventing user mode from accessing the memory after unplug? I guess the best you could do is unmap it from the CPU page table and let the application segfault on the next access. Or replace the mapping with a dummy page in system memory?
We indeed unmap but instead of letting it segfault insert dummy page on the next page fault. See here https://cgit.freedesktop.org/~agrodzov/linux/commit/?h=drm-misc-next&id=... And I am aware that this doesn't take care of KFD user mapping. As you know, we had some discussions with you on this topic and it's on my TODO list to follow up on this to solve it for KFD too.
ROCm user mode maps VRAM BOs using render nodes. So I'd expect ttm_bo_vm_dummy_page to work for KFD as well.
I guess we'd need something special for KFD's doorbell and MMIO (HDP flush) mappings. Was that the discussion about the file address space?
Yes
Andrey
Regards, Felix
Andrey
Regards, Felix
Andrey
On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
With this calling drm_dev_unplug will flush and block all in flight IOCTLs
Also, add feature such that if device supports graceful unplug we enclose entire IOCTL in SRCU critical section.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
Nope.
The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
Then I am confused why we have https://elixir.bootlin.com/linux/v5.12/source/drivers/gpu/drm/drm_ioctl.c#L8... currently in code ?
I forgot about this one, again. Thanks for reminding.
Especially not with an opt-in flag so that it could be shrugged of as a driver hack. Most of these ioctls should have absolutely no problem working after hotunplug.
Also, doing this defeats the point since it pretty much guarantees userspace will die in assert()s and stuff. E.g. on i915 the rough contract is that only execbuf (and even that only when userspace has indicated support for non-recoverable hw ctx) is allowed to fail. Anything else might crash userspace.
Given that as I pointed above we already fail any IOCTls with -ENODEV when device is unplugged, it seems those crashes don't happen that often ? Also, in all my testing I don't think I saw a user space crash I could attribute to this.
I guess it should be ok.
My reasons for making this work is both less trouble for userspace (did you test with various wayland compositors out there, not just amdgpu x86 driver?), but also testing.
We still need a bunch of these checks in various places or you'll wait a very long time for a pending modeset or similar to complete. Being able to run that code easily after hotunplug has completed should help a lot with testing.
Plus various drivers already acquired drm_dev_enter/exit and now I wonder whether that was properly tested or not ...
I guess maybe we need a drm module option to disable this check, so that we can exercise the code as if the ioctl has raced with hotunplug at the worst possible moment.
Also atomic is really tricky here: I assume your testing has just done normal synchronous commits, but anything that goes through atomic can be done nonblocking in a separate thread. Which the ioctl catch-all here wont capture.
You probably need similar (and very precisely defined) rules for amdgpu. And those must definitely exclude any shard ioctls from randomly failing with EIO, because that just kills the box and defeats the point of trying to gracefully handling hotunplug and making sure userspace has a chance of survival. E.g. for atomic everything should continue, including flip completion, but we set all outputs to "disconnected" and send out the uevent. Maybe crtc enabling can fail too, but that can also be handled through the async status we're using to signal DP link failures to userspace.
As I pointed before, because of the complexity of the topic I prefer to take it step by step and solve first for secondary device use case, not for primary, display attached device.
Yeah makes sense. But then I think the right patch is to roll this out for all drivers, properly justified with existing code. Not behind a driver flag, because with all these different compositors the last thing we want is a proliferation of driver-specific behaviour. That's imo the worst option of all of them and needs to be avoided.
Cheers, Daniel
I guess we should clarify this in the hotunplug doc?
Agree
To clarify: I'm not against throwing an ENODEV at userspace for ioctl that really make no sense, and where we're rather confident that all properly implemented userspace will gracefully handle failures. Like a modeset, or opening a device, or trying to import a dma-buf or stuff like that which can already fail in normal operation for any kind of reason.
But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail after hotunplug.
As I pointed above, this a bit confuses me given that we already do blanker rejection of IOCTLs if device is unplugged.
Well I'm confused about this too :-/
And then there's the middle ground, like doing a pageflip or buffer flush, which I guess some userspace might handle, but risky to inflict those consequences on them. atomic modeset is especially fun since depending what you're doing it can be both "failures expected" and "failures not really expected in normal operation".
Also, this really should be consistent across drivers, not solved with a driver flag for every possible combination.
If you look at the current hotunplug kms drivers, they have drm_dev_enter/exit sprinkled in specific hw callback functions because of the above problems. But maybe it makes sense to change things in a few cases. But then we should do it across the board.
So as I understand your preferred approach is that I scope any back_end, HW specific function with drm_dev_enter/exit because that where MMIO access takes place. But besides explicit MMIO access thorough register accessors in the HW back-end there is also indirect MMIO access taking place throughout the code in the driver because of various VRAM BOs which provide CPU access to VRAM through the VRAM BAR. This kind of access is spread all over in the driver and even in mid-layers such as TTM and not limited to HW back-end functions. It means it's much harder to spot such places to surgically scope them with drm_dev_enter/exit and also that any new such code introduced will immediately break hot unplug because the developers can't be expected to remember making their code robust to this specific use case. That why when we discussed internally what approach to take to protecting code with drm_dev_enter/exit we opted for using the widest available scope.
The thing is, you kinda have to anyway. There's enormous amounts of asynchronous processing going on. E.g. nonblocking atomic commits also do ttm unpinning and fun stuff like that, which if you sync things wrong can happen way late. So the door for bad fallout is wide open :-(
I'm not sure where the right tradeoff is to make sure we catch them all, and can make sure with testing that we've indeed caught them all. -Daniel
On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
With this calling drm_dev_unplug will flush and block all in flight IOCTLs
Also, add feature such that if device supports graceful unplug we enclose entire IOCTL in SRCU critical section.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
Nope.
The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.boo... currently in code ?
I forgot about this one, again. Thanks for reminding.
Especially not with an opt-in flag so that it could be shrugged of as a driver hack. Most of these ioctls should have absolutely no problem working after hotunplug.
Also, doing this defeats the point since it pretty much guarantees userspace will die in assert()s and stuff. E.g. on i915 the rough contract is that only execbuf (and even that only when userspace has indicated support for non-recoverable hw ctx) is allowed to fail. Anything else might crash userspace.
Given that as I pointed above we already fail any IOCTls with -ENODEV when device is unplugged, it seems those crashes don't happen that often ? Also, in all my testing I don't think I saw a user space crash I could attribute to this.
I guess it should be ok.
What should be ok ?
My reasons for making this work is both less trouble for userspace (did you test with various wayland compositors out there, not just amdgpu x86
I didn't - will give it a try.
driver?), but also testing.
We still need a bunch of these checks in various places or you'll wait a very long time for a pending modeset or similar to complete. Being able to run that code easily after hotunplug has completed should help a lot with testing.
Plus various drivers already acquired drm_dev_enter/exit and now I wonder whether that was properly tested or not ...
I guess maybe we need a drm module option to disable this check, so that we can exercise the code as if the ioctl has raced with hotunplug at the worst possible moment.
Also atomic is really tricky here: I assume your testing has just done normal synchronous commits, but anything that goes through atomic can be done nonblocking in a separate thread. Which the ioctl catch-all here wont capture.
Yes, async commit was on my mind and thanks for reminding me. Indeed I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in drm_dev_enter/exit. Note that i have a bunch of patches, all name's starting with 'Scope....' that just methodically put all the background work items and timers the drivers schedules in drm_dev_enter/exit scope. This was supposed to be part of the 'Scope Display code' patch.
You probably need similar (and very precisely defined) rules for amdgpu. And those must definitely exclude any shard ioctls from randomly failing with EIO, because that just kills the box and defeats the point of trying to gracefully handling hotunplug and making sure userspace has a chance of survival. E.g. for atomic everything should continue, including flip completion, but we set all outputs to "disconnected" and send out the uevent. Maybe crtc enabling can fail too, but that can also be handled through the async status we're using to signal DP link failures to userspace.
As I pointed before, because of the complexity of the topic I prefer to take it step by step and solve first for secondary device use case, not for primary, display attached device.
Yeah makes sense. But then I think the right patch is to roll this out for all drivers, properly justified with existing code. Not behind a driver flag, because with all these different compositors the last thing we want is a proliferation of driver-specific behaviour. That's imo the worst option of all of them and needs to be avoided.
So this kind of patch would be acceptable to you if I unconditionally scope the drm_ioctl with drm_dev_enter/exit without the driver flag ? I am worried to break other drivers with this, see patch https://cgit.freedesktop.org/~agrodzov/linux/commit/?h=drm-misc-next&id=... Before setting drm_dev_unplug I go through a whole process of signalling all possible fences in the system which some one some where might be waiting on. My concern is that in the absence of HW those fences won't signal and so unless I signal them myself srcu_synchrionize in drm_dev_unplug will hang waiting for any such code scoped by drm_dev_enter/exit.
Andrey
Cheers, Daniel
I guess we should clarify this in the hotunplug doc?
Agree
To clarify: I'm not against throwing an ENODEV at userspace for ioctl that really make no sense, and where we're rather confident that all properly implemented userspace will gracefully handle failures. Like a modeset, or opening a device, or trying to import a dma-buf or stuff like that which can already fail in normal operation for any kind of reason.
But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail after hotunplug.
As I pointed above, this a bit confuses me given that we already do blanker rejection of IOCTLs if device is unplugged.
Well I'm confused about this too :-/
And then there's the middle ground, like doing a pageflip or buffer flush, which I guess some userspace might handle, but risky to inflict those consequences on them. atomic modeset is especially fun since depending what you're doing it can be both "failures expected" and "failures not really expected in normal operation".
Also, this really should be consistent across drivers, not solved with a driver flag for every possible combination.
If you look at the current hotunplug kms drivers, they have drm_dev_enter/exit sprinkled in specific hw callback functions because of the above problems. But maybe it makes sense to change things in a few cases. But then we should do it across the board.
So as I understand your preferred approach is that I scope any back_end, HW specific function with drm_dev_enter/exit because that where MMIO access takes place. But besides explicit MMIO access thorough register accessors in the HW back-end there is also indirect MMIO access taking place throughout the code in the driver because of various VRAM BOs which provide CPU access to VRAM through the VRAM BAR. This kind of access is spread all over in the driver and even in mid-layers such as TTM and not limited to HW back-end functions. It means it's much harder to spot such places to surgically scope them with drm_dev_enter/exit and also that any new such code introduced will immediately break hot unplug because the developers can't be expected to remember making their code robust to this specific use case. That why when we discussed internally what approach to take to protecting code with drm_dev_enter/exit we opted for using the widest available scope.
The thing is, you kinda have to anyway. There's enormous amounts of asynchronous processing going on. E.g. nonblocking atomic commits also do ttm unpinning and fun stuff like that, which if you sync things wrong can happen way late. So the door for bad fallout is wide open :-(
I'm not sure where the right tradeoff is to make sure we catch them all, and can make sure with testing that we've indeed caught them all. -Daniel
On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote:
On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
With this calling drm_dev_unplug will flush and block all in flight IOCTLs
Also, add feature such that if device supports graceful unplug we enclose entire IOCTL in SRCU critical section.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
Nope.
The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.boo... currently in code ?
I forgot about this one, again. Thanks for reminding.
Especially not with an opt-in flag so that it could be shrugged of as a driver hack. Most of these ioctls should have absolutely no problem working after hotunplug.
Also, doing this defeats the point since it pretty much guarantees userspace will die in assert()s and stuff. E.g. on i915 the rough contract is that only execbuf (and even that only when userspace has indicated support for non-recoverable hw ctx) is allowed to fail. Anything else might crash userspace.
Given that as I pointed above we already fail any IOCTls with -ENODEV when device is unplugged, it seems those crashes don't happen that often ? Also, in all my testing I don't think I saw a user space crash I could attribute to this.
I guess it should be ok.
What should be ok ?
Your approach, but not your patch. If we go with this let's just lift it to drm_ioctl() as the default behavior. No driver opt-in flag, because that's definitely worse than any other approach because we really need to get rid of driver specific behaviour for generic ioctls, especially anything a compositor will use directly.
My reasons for making this work is both less trouble for userspace (did you test with various wayland compositors out there, not just amdgpu x86
I didn't - will give it a try.
driver?), but also testing.
We still need a bunch of these checks in various places or you'll wait a very long time for a pending modeset or similar to complete. Being able to run that code easily after hotunplug has completed should help a lot with testing.
Plus various drivers already acquired drm_dev_enter/exit and now I wonder whether that was properly tested or not ...
I guess maybe we need a drm module option to disable this check, so that we can exercise the code as if the ioctl has raced with hotunplug at the worst possible moment.
Also atomic is really tricky here: I assume your testing has just done normal synchronous commits, but anything that goes through atomic can be done nonblocking in a separate thread. Which the ioctl catch-all here wont capture.
Yes, async commit was on my mind and thanks for reminding me. Indeed I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in drm_dev_enter/exit. Note that i have a bunch of patches, all name's starting with 'Scope....' that just methodically put all the background work items and timers the drivers schedules in drm_dev_enter/exit scope. This was supposed to be part of the 'Scope Display code' patch.
That's too much. You still have to arrange that the flip completion event gets sent out. So it's a bit tricky.
In other places the same problem applies, e.g. probe functions need to make sure they report "disconnected".
You probably need similar (and very precisely defined) rules for amdgpu. And those must definitely exclude any shard ioctls from randomly failing with EIO, because that just kills the box and defeats the point of trying to gracefully handling hotunplug and making sure userspace has a chance of survival. E.g. for atomic everything should continue, including flip completion, but we set all outputs to "disconnected" and send out the uevent. Maybe crtc enabling can fail too, but that can also be handled through the async status we're using to signal DP link failures to userspace.
As I pointed before, because of the complexity of the topic I prefer to take it step by step and solve first for secondary device use case, not for primary, display attached device.
Yeah makes sense. But then I think the right patch is to roll this out for all drivers, properly justified with existing code. Not behind a driver flag, because with all these different compositors the last thing we want is a proliferation of driver-specific behaviour. That's imo the worst option of all of them and needs to be avoided.
So this kind of patch would be acceptable to you if I unconditionally scope the drm_ioctl with drm_dev_enter/exit without the driver flag ? I am worried to break other drivers with this, see patch https://cgit.freedesktop.org/~agrodzov/linux/commit/?h=drm-misc-next&id=... Before setting drm_dev_unplug I go through a whole process of signalling all possible fences in the system which some one some where might be waiting on. My concern is that in the absence of HW those fences won't signal and so unless I signal them myself srcu_synchrionize in drm_dev_unplug will hang waiting for any such code scoped by drm_dev_enter/exit.
Uh right. I forgot about this.
Which would kinda mean the top level scope is maybe not the best idea, and perhaps we should indeed drill it down. But then the testing issue definitely gets a lot worse.
So what if we'd push that drm_dev_is_unplugged check down into ioctls? Then we can make a case-by case decision whether it should be converted to drm_dev_enter/exit, needs to be pushed down further into drivers (due to fence wait issues) or other concerns?
Also I guess we need to have a subsystem wide rule on whether you need to force complete all fences before you call drm_dev_unplug, or afterwards. If we have mixed behaviour on this there will be disappointment. And since hotunplug and dma_fence completion are both userspace visible that inconsistency might have bigger impact.
This is all very tricky indeed :-/
btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go with that: We could do the same trick we've done for DRM_UNLOCKED: - drm_dev_enter/exit is called for any ioctl that has not set the DRM_HOTUNPLUG_SAFE flag - for drm core ioctls we push them into all ioctls and decide how to handle/where (with the aim to have the least amount of code flow different during hotunplug vs after hotunplug has finished, to reduce testing scope) - then we make DRM_HOTUNPLUG_SAFE the implied default
This would have us left with render ioctls, and I think the defensive assumption there is that they're all hotunplug safe. We might hang on a fence wait, but that's fixable, and it's better than blowing up on a use-after-free security bug.
Thoughts?
It is unfortunately even more work until we've reached the goal, but I think it's safest and most flexible approach overall.
Cheers, Daniel
Andrey
Cheers, Daniel
I guess we should clarify this in the hotunplug doc?
Agree
To clarify: I'm not against throwing an ENODEV at userspace for ioctl that really make no sense, and where we're rather confident that all properly implemented userspace will gracefully handle failures. Like a modeset, or opening a device, or trying to import a dma-buf or stuff like that which can already fail in normal operation for any kind of reason.
But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail after hotunplug.
As I pointed above, this a bit confuses me given that we already do blanker rejection of IOCTLs if device is unplugged.
Well I'm confused about this too :-/
And then there's the middle ground, like doing a pageflip or buffer flush, which I guess some userspace might handle, but risky to inflict those consequences on them. atomic modeset is especially fun since depending what you're doing it can be both "failures expected" and "failures not really expected in normal operation".
Also, this really should be consistent across drivers, not solved with a driver flag for every possible combination.
If you look at the current hotunplug kms drivers, they have drm_dev_enter/exit sprinkled in specific hw callback functions because of the above problems. But maybe it makes sense to change things in a few cases. But then we should do it across the board.
So as I understand your preferred approach is that I scope any back_end, HW specific function with drm_dev_enter/exit because that where MMIO access takes place. But besides explicit MMIO access thorough register accessors in the HW back-end there is also indirect MMIO access taking place throughout the code in the driver because of various VRAM BOs which provide CPU access to VRAM through the VRAM BAR. This kind of access is spread all over in the driver and even in mid-layers such as TTM and not limited to HW back-end functions. It means it's much harder to spot such places to surgically scope them with drm_dev_enter/exit and also that any new such code introduced will immediately break hot unplug because the developers can't be expected to remember making their code robust to this specific use case. That why when we discussed internally what approach to take to protecting code with drm_dev_enter/exit we opted for using the widest available scope.
The thing is, you kinda have to anyway. There's enormous amounts of asynchronous processing going on. E.g. nonblocking atomic commits also do ttm unpinning and fun stuff like that, which if you sync things wrong can happen way late. So the door for bad fallout is wide open :-(
I'm not sure where the right tradeoff is to make sure we catch them all, and can make sure with testing that we've indeed caught them all. -Daniel
On 2021-04-30 6:25 a.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote:
On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote: > With this calling drm_dev_unplug will flush and block > all in flight IOCTLs > > Also, add feature such that if device supports graceful unplug > we enclose entire IOCTL in SRCU critical section. > > Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com
Nope.
The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.boo... currently in code ?
I forgot about this one, again. Thanks for reminding.
Especially not with an opt-in flag so that it could be shrugged of as a driver hack. Most of these ioctls should have absolutely no problem working after hotunplug.
Also, doing this defeats the point since it pretty much guarantees userspace will die in assert()s and stuff. E.g. on i915 the rough contract is that only execbuf (and even that only when userspace has indicated support for non-recoverable hw ctx) is allowed to fail. Anything else might crash userspace.
Given that as I pointed above we already fail any IOCTls with -ENODEV when device is unplugged, it seems those crashes don't happen that often ? Also, in all my testing I don't think I saw a user space crash I could attribute to this.
I guess it should be ok.
What should be ok ?
Your approach, but not your patch. If we go with this let's just lift it to drm_ioctl() as the default behavior. No driver opt-in flag, because that's definitely worse than any other approach because we really need to get rid of driver specific behaviour for generic ioctls, especially anything a compositor will use directly.
My reasons for making this work is both less trouble for userspace (did you test with various wayland compositors out there, not just amdgpu x86
I didn't - will give it a try.
Weston worked without crashes, run the egl tester cube there.
driver?), but also testing.
We still need a bunch of these checks in various places or you'll wait a very long time for a pending modeset or similar to complete. Being able to run that code easily after hotunplug has completed should help a lot with testing.
Plus various drivers already acquired drm_dev_enter/exit and now I wonder whether that was properly tested or not ...
I guess maybe we need a drm module option to disable this check, so that we can exercise the code as if the ioctl has raced with hotunplug at the worst possible moment.
Also atomic is really tricky here: I assume your testing has just done normal synchronous commits, but anything that goes through atomic can be done nonblocking in a separate thread. Which the ioctl catch-all here wont capture.
Yes, async commit was on my mind and thanks for reminding me. Indeed I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in drm_dev_enter/exit. Note that i have a bunch of patches, all name's starting with 'Scope....' that just methodically put all the background work items and timers the drivers schedules in drm_dev_enter/exit scope. This was supposed to be part of the 'Scope Display code' patch.
That's too much. You still have to arrange that the flip completion event gets sent out. So it's a bit tricky.
In other places the same problem applies, e.g. probe functions need to make sure they report "disconnected".
I see, well, this is all part of KMS support which I defer for now anyway. Will tackle it then.
You probably need similar (and very precisely defined) rules for amdgpu. And those must definitely exclude any shard ioctls from randomly failing with EIO, because that just kills the box and defeats the point of trying to gracefully handling hotunplug and making sure userspace has a chance of survival. E.g. for atomic everything should continue, including flip completion, but we set all outputs to "disconnected" and send out the uevent. Maybe crtc enabling can fail too, but that can also be handled through the async status we're using to signal DP link failures to userspace.
As I pointed before, because of the complexity of the topic I prefer to take it step by step and solve first for secondary device use case, not for primary, display attached device.
Yeah makes sense. But then I think the right patch is to roll this out for all drivers, properly justified with existing code. Not behind a driver flag, because with all these different compositors the last thing we want is a proliferation of driver-specific behaviour. That's imo the worst option of all of them and needs to be avoided.
So this kind of patch would be acceptable to you if I unconditionally scope the drm_ioctl with drm_dev_enter/exit without the driver flag ? I am worried to break other drivers with this, see patch https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedes... Before setting drm_dev_unplug I go through a whole process of signalling all possible fences in the system which some one some where might be waiting on. My concern is that in the absence of HW those fences won't signal and so unless I signal them myself srcu_synchrionize in drm_dev_unplug will hang waiting for any such code scoped by drm_dev_enter/exit.
Uh right. I forgot about this.
Which would kinda mean the top level scope is maybe not the best idea, and perhaps we should indeed drill it down. But then the testing issue definitely gets a lot worse.
So what if we'd push that drm_dev_is_unplugged check down into ioctls? Then we can make a case-by case decision whether it should be converted to drm_dev_enter/exit, needs to be pushed down further into drivers (due to fence wait issues) or other concerns?
Also I guess we need to have a subsystem wide rule on whether you need to force complete all fences before you call drm_dev_unplug, or afterwards.
I don't see how you can handle it afterwards. If a thread is stuck in dma_fence_wait in non interruptible wait (any kernel thread) and with no timeout there is nothing you can do to stop the wait. Any such code scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug. The only way then is to preemptively force signal all such fences before calling drm_dev_unplug - as I do in the above mentioned patch.
If we have mixed behaviour on this there will be disappointment. And since hotunplug and dma_fence completion are both userspace visible that inconsistency might have bigger impact.
This is all very tricky indeed :-/
btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go with that: We could do the same trick we've done for DRM_UNLOCKED:
- drm_dev_enter/exit is called for any ioctl that has not set the DRM_HOTUNPLUG_SAFE flag
- for drm core ioctls we push them into all ioctls and decide how to handle/where (with the aim to have the least amount of code flow different during hotunplug vs after hotunplug has finished, to reduce testing scope)
- then we make DRM_HOTUNPLUG_SAFE the implied default
This would have us left with render ioctls, and I think the defensive assumption there is that they're all hotunplug safe. We might hang on a fence wait, but that's fixable, and it's better than blowing up on a use-after-free security bug.
Thoughts?
I don't fully see a difference between the approach described above and the full drill down to each driver and even within the driver, to the HW back-ends - what criteria I would use to decide if for a given IOCTL i scope with drm_dev_enter/exit at the highest level while for another i go all the way down ? If we would agree that signaling the fences preemptively before engaging drm_dev_unplug is generically the right approach maybe we can then scope drm_ioctl unconditionally with drm_dev_enter/exit and then for each driver go through the same process I do for amdgpu - writing driver specific function which takes care of all the fences. We could then just create a drm callback which would be called from drm_ioctl before drm_dev_unplug is called.
Andrey
It is unfortunately even more work until we've reached the goal, but I think it's safest and most flexible approach overall.
Cheers, Daniel
Andrey
Cheers, Daniel
I guess we should clarify this in the hotunplug doc?
Agree
To clarify: I'm not against throwing an ENODEV at userspace for ioctl that really make no sense, and where we're rather confident that all properly implemented userspace will gracefully handle failures. Like a modeset, or opening a device, or trying to import a dma-buf or stuff like that which can already fail in normal operation for any kind of reason.
But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail after hotunplug.
As I pointed above, this a bit confuses me given that we already do blanker rejection of IOCTLs if device is unplugged.
Well I'm confused about this too :-/
And then there's the middle ground, like doing a pageflip or buffer flush, which I guess some userspace might handle, but risky to inflict those consequences on them. atomic modeset is especially fun since depending what you're doing it can be both "failures expected" and "failures not really expected in normal operation".
Also, this really should be consistent across drivers, not solved with a driver flag for every possible combination.
If you look at the current hotunplug kms drivers, they have drm_dev_enter/exit sprinkled in specific hw callback functions because of the above problems. But maybe it makes sense to change things in a few cases. But then we should do it across the board.
So as I understand your preferred approach is that I scope any back_end, HW specific function with drm_dev_enter/exit because that where MMIO access takes place. But besides explicit MMIO access thorough register accessors in the HW back-end there is also indirect MMIO access taking place throughout the code in the driver because of various VRAM BOs which provide CPU access to VRAM through the VRAM BAR. This kind of access is spread all over in the driver and even in mid-layers such as TTM and not limited to HW back-end functions. It means it's much harder to spot such places to surgically scope them with drm_dev_enter/exit and also that any new such code introduced will immediately break hot unplug because the developers can't be expected to remember making their code robust to this specific use case. That why when we discussed internally what approach to take to protecting code with drm_dev_enter/exit we opted for using the widest available scope.
The thing is, you kinda have to anyway. There's enormous amounts of asynchronous processing going on. E.g. nonblocking atomic commits also do ttm unpinning and fun stuff like that, which if you sync things wrong can happen way late. So the door for bad fallout is wide open :-(
I'm not sure where the right tradeoff is to make sure we catch them all, and can make sure with testing that we've indeed caught them all. -Daniel
Ping
Andrey
On 2021-04-30 1:27 p.m., Andrey Grodzovsky wrote:
On 2021-04-30 6:25 a.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote:
On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote: > On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote: >> With this calling drm_dev_unplug will flush and block >> all in flight IOCTLs >> >> Also, add feature such that if device supports graceful unplug >> we enclose entire IOCTL in SRCU critical section. >> >> Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com > > Nope. > > The idea of drm_dev_enter/exit is to mark up hw access. Not > entire ioctl.
Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.boo...
currently in code ?
I forgot about this one, again. Thanks for reminding.
> Especially not with an opt-in flag so that it could be shrugged > of as a > driver hack. Most of these ioctls should have absolutely no problem > working after hotunplug. > > Also, doing this defeats the point since it pretty much guarantees > userspace will die in assert()s and stuff. E.g. on i915 the rough > contract > is that only execbuf (and even that only when userspace has > indicated > support for non-recoverable hw ctx) is allowed to fail. Anything > else > might crash userspace.
Given that as I pointed above we already fail any IOCTls with -ENODEV when device is unplugged, it seems those crashes don't happen that often ? Also, in all my testing I don't think I saw a user space crash I could attribute to this.
I guess it should be ok.
What should be ok ?
Your approach, but not your patch. If we go with this let's just lift it to drm_ioctl() as the default behavior. No driver opt-in flag, because that's definitely worse than any other approach because we really need to get rid of driver specific behaviour for generic ioctls, especially anything a compositor will use directly.
My reasons for making this work is both less trouble for userspace (did you test with various wayland compositors out there, not just amdgpu x86
I didn't - will give it a try.
Weston worked without crashes, run the egl tester cube there.
driver?), but also testing.
We still need a bunch of these checks in various places or you'll wait a very long time for a pending modeset or similar to complete. Being able to run that code easily after hotunplug has completed should help a lot with testing.
Plus various drivers already acquired drm_dev_enter/exit and now I wonder whether that was properly tested or not ...
I guess maybe we need a drm module option to disable this check, so that we can exercise the code as if the ioctl has raced with hotunplug at the worst possible moment.
Also atomic is really tricky here: I assume your testing has just done normal synchronous commits, but anything that goes through atomic can be done nonblocking in a separate thread. Which the ioctl catch-all here wont capture.
Yes, async commit was on my mind and thanks for reminding me. Indeed I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in drm_dev_enter/exit. Note that i have a bunch of patches, all name's starting with 'Scope....' that just methodically put all the background work items and timers the drivers schedules in drm_dev_enter/exit scope. This was supposed to be part of the 'Scope Display code' patch.
That's too much. You still have to arrange that the flip completion event gets sent out. So it's a bit tricky.
In other places the same problem applies, e.g. probe functions need to make sure they report "disconnected".
I see, well, this is all part of KMS support which I defer for now anyway. Will tackle it then.
> You probably need similar (and very precisely defined) rules for > amdgpu. > And those must definitely exclude any shard ioctls from randomly > failing > with EIO, because that just kills the box and defeats the point > of trying > to gracefully handling hotunplug and making sure userspace has a > chance of > survival. E.g. for atomic everything should continue, including flip > completion, but we set all outputs to "disconnected" and send out > the > uevent. Maybe crtc enabling can fail too, but that can also be > handled > through the async status we're using to signal DP link failures to > userspace.
As I pointed before, because of the complexity of the topic I prefer to take it step by step and solve first for secondary device use case, not for primary, display attached device.
Yeah makes sense. But then I think the right patch is to roll this out for all drivers, properly justified with existing code. Not behind a driver flag, because with all these different compositors the last thing we want is a proliferation of driver-specific behaviour. That's imo the worst option of all of them and needs to be avoided.
So this kind of patch would be acceptable to you if I unconditionally scope the drm_ioctl with drm_dev_enter/exit without the driver flag ? I am worried to break other drivers with this, see patch https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedes...
Before setting drm_dev_unplug I go through a whole process of signalling all possible fences in the system which some one some where might be waiting on. My concern is that in the absence of HW those fences won't signal and so unless I signal them myself srcu_synchrionize in drm_dev_unplug will hang waiting for any such code scoped by drm_dev_enter/exit.
Uh right. I forgot about this.
Which would kinda mean the top level scope is maybe not the best idea, and perhaps we should indeed drill it down. But then the testing issue definitely gets a lot worse.
So what if we'd push that drm_dev_is_unplugged check down into ioctls? Then we can make a case-by case decision whether it should be converted to drm_dev_enter/exit, needs to be pushed down further into drivers (due to fence wait issues) or other concerns?
Also I guess we need to have a subsystem wide rule on whether you need to force complete all fences before you call drm_dev_unplug, or afterwards.
I don't see how you can handle it afterwards. If a thread is stuck in dma_fence_wait in non interruptible wait (any kernel thread) and with no timeout there is nothing you can do to stop the wait. Any such code scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug. The only way then is to preemptively force signal all such fences before calling drm_dev_unplug - as I do in the above mentioned patch.
If we have mixed behaviour on this there will be disappointment. And since hotunplug and dma_fence completion are both userspace visible that inconsistency might have bigger impact.
This is all very tricky indeed :-/
btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go with that: We could do the same trick we've done for DRM_UNLOCKED:
- drm_dev_enter/exit is called for any ioctl that has not set the
DRM_HOTUNPLUG_SAFE flag
- for drm core ioctls we push them into all ioctls and decide how to
handle/where (with the aim to have the least amount of code flow different during hotunplug vs after hotunplug has finished, to reduce testing scope)
- then we make DRM_HOTUNPLUG_SAFE the implied default
This would have us left with render ioctls, and I think the defensive assumption there is that they're all hotunplug safe. We might hang on a fence wait, but that's fixable, and it's better than blowing up on a use-after-free security bug.
Thoughts?
I don't fully see a difference between the approach described above and the full drill down to each driver and even within the driver, to the HW back-ends - what criteria I would use to decide if for a given IOCTL i scope with drm_dev_enter/exit at the highest level while for another i go all the way down ? If we would agree that signaling the fences preemptively before engaging drm_dev_unplug is generically the right approach maybe we can then scope drm_ioctl unconditionally with drm_dev_enter/exit and then for each driver go through the same process I do for amdgpu - writing driver specific function which takes care of all the fences. We could then just create a drm callback which would be called from drm_ioctl before drm_dev_unplug is called.
Andrey
It is unfortunately even more work until we've reached the goal, but I think it's safest and most flexible approach overall.
Cheers, Daniel
Andrey
Cheers, Daniel
> > I guess we should clarify this in the hotunplug doc?
Agree
To clarify: I'm not against throwing an ENODEV at userspace for ioctl that really make no sense, and where we're rather confident that all properly implemented userspace will gracefully handle failures. Like a modeset, or opening a device, or trying to import a dma-buf or stuff like that which can already fail in normal operation for any kind of reason.
But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail after hotunplug.
As I pointed above, this a bit confuses me given that we already do blanker rejection of IOCTLs if device is unplugged.
Well I'm confused about this too :-/
And then there's the middle ground, like doing a pageflip or buffer flush, which I guess some userspace might handle, but risky to inflict those consequences on them. atomic modeset is especially fun since depending what you're doing it can be both "failures expected" and "failures not really expected in normal operation".
Also, this really should be consistent across drivers, not solved with a driver flag for every possible combination.
If you look at the current hotunplug kms drivers, they have drm_dev_enter/exit sprinkled in specific hw callback functions because of the above problems. But maybe it makes sense to change things in a few cases. But then we should do it across the board.
So as I understand your preferred approach is that I scope any back_end, HW specific function with drm_dev_enter/exit because that where MMIO access takes place. But besides explicit MMIO access thorough register accessors in the HW back-end there is also indirect MMIO access taking place throughout the code in the driver because of various VRAM BOs which provide CPU access to VRAM through the VRAM BAR. This kind of access is spread all over in the driver and even in mid-layers such as TTM and not limited to HW back-end functions. It means it's much harder to spot such places to surgically scope them with drm_dev_enter/exit and also that any new such code introduced will immediately break hot unplug because the developers can't be expected to remember making their code robust to this specific use case. That why when we discussed internally what approach to take to protecting code with drm_dev_enter/exit we opted for using the widest available scope.
The thing is, you kinda have to anyway. There's enormous amounts of asynchronous processing going on. E.g. nonblocking atomic commits also do ttm unpinning and fun stuff like that, which if you sync things wrong can happen way late. So the door for bad fallout is wide open :-(
I'm not sure where the right tradeoff is to make sure we catch them all, and can make sure with testing that we've indeed caught them all. -Daniel
On Fri, Apr 30, 2021 at 01:27:37PM -0400, Andrey Grodzovsky wrote:
On 2021-04-30 6:25 a.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote:
On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote: > On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote: > > With this calling drm_dev_unplug will flush and block > > all in flight IOCTLs > > > > Also, add feature such that if device supports graceful unplug > > we enclose entire IOCTL in SRCU critical section. > > > > Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com > > Nope. > > The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.boo... currently in code ?
I forgot about this one, again. Thanks for reminding.
> Especially not with an opt-in flag so that it could be shrugged of as a > driver hack. Most of these ioctls should have absolutely no problem > working after hotunplug. > > Also, doing this defeats the point since it pretty much guarantees > userspace will die in assert()s and stuff. E.g. on i915 the rough contract > is that only execbuf (and even that only when userspace has indicated > support for non-recoverable hw ctx) is allowed to fail. Anything else > might crash userspace.
Given that as I pointed above we already fail any IOCTls with -ENODEV when device is unplugged, it seems those crashes don't happen that often ? Also, in all my testing I don't think I saw a user space crash I could attribute to this.
I guess it should be ok.
What should be ok ?
Your approach, but not your patch. If we go with this let's just lift it to drm_ioctl() as the default behavior. No driver opt-in flag, because that's definitely worse than any other approach because we really need to get rid of driver specific behaviour for generic ioctls, especially anything a compositor will use directly.
My reasons for making this work is both less trouble for userspace (did you test with various wayland compositors out there, not just amdgpu x86
I didn't - will give it a try.
Weston worked without crashes, run the egl tester cube there.
driver?), but also testing.
We still need a bunch of these checks in various places or you'll wait a very long time for a pending modeset or similar to complete. Being able to run that code easily after hotunplug has completed should help a lot with testing.
Plus various drivers already acquired drm_dev_enter/exit and now I wonder whether that was properly tested or not ...
I guess maybe we need a drm module option to disable this check, so that we can exercise the code as if the ioctl has raced with hotunplug at the worst possible moment.
Also atomic is really tricky here: I assume your testing has just done normal synchronous commits, but anything that goes through atomic can be done nonblocking in a separate thread. Which the ioctl catch-all here wont capture.
Yes, async commit was on my mind and thanks for reminding me. Indeed I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in drm_dev_enter/exit. Note that i have a bunch of patches, all name's starting with 'Scope....' that just methodically put all the background work items and timers the drivers schedules in drm_dev_enter/exit scope. This was supposed to be part of the 'Scope Display code' patch.
That's too much. You still have to arrange that the flip completion event gets sent out. So it's a bit tricky.
In other places the same problem applies, e.g. probe functions need to make sure they report "disconnected".
I see, well, this is all part of KMS support which I defer for now anyway. Will tackle it then.
> You probably need similar (and very precisely defined) rules for amdgpu. > And those must definitely exclude any shard ioctls from randomly failing > with EIO, because that just kills the box and defeats the point of trying > to gracefully handling hotunplug and making sure userspace has a chance of > survival. E.g. for atomic everything should continue, including flip > completion, but we set all outputs to "disconnected" and send out the > uevent. Maybe crtc enabling can fail too, but that can also be handled > through the async status we're using to signal DP link failures to > userspace.
As I pointed before, because of the complexity of the topic I prefer to take it step by step and solve first for secondary device use case, not for primary, display attached device.
Yeah makes sense. But then I think the right patch is to roll this out for all drivers, properly justified with existing code. Not behind a driver flag, because with all these different compositors the last thing we want is a proliferation of driver-specific behaviour. That's imo the worst option of all of them and needs to be avoided.
So this kind of patch would be acceptable to you if I unconditionally scope the drm_ioctl with drm_dev_enter/exit without the driver flag ? I am worried to break other drivers with this, see patch https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedes... Before setting drm_dev_unplug I go through a whole process of signalling all possible fences in the system which some one some where might be waiting on. My concern is that in the absence of HW those fences won't signal and so unless I signal them myself srcu_synchrionize in drm_dev_unplug will hang waiting for any such code scoped by drm_dev_enter/exit.
Uh right. I forgot about this.
Which would kinda mean the top level scope is maybe not the best idea, and perhaps we should indeed drill it down. But then the testing issue definitely gets a lot worse.
So what if we'd push that drm_dev_is_unplugged check down into ioctls? Then we can make a case-by case decision whether it should be converted to drm_dev_enter/exit, needs to be pushed down further into drivers (due to fence wait issues) or other concerns?
Also I guess we need to have a subsystem wide rule on whether you need to force complete all fences before you call drm_dev_unplug, or afterwards.
I don't see how you can handle it afterwards. If a thread is stuck in dma_fence_wait in non interruptible wait (any kernel thread) and with no timeout there is nothing you can do to stop the wait. Any such code scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug. The only way then is to preemptively force signal all such fences before calling drm_dev_unplug - as I do in the above mentioned patch.
Yeah, which is why I don't think top-level drm_dev_enter/exit is a good idea.
If we have mixed behaviour on this there will be disappointment. And since hotunplug and dma_fence completion are both userspace visible that inconsistency might have bigger impact.
This is all very tricky indeed :-/
btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go with that: We could do the same trick we've done for DRM_UNLOCKED:
- drm_dev_enter/exit is called for any ioctl that has not set the DRM_HOTUNPLUG_SAFE flag
- for drm core ioctls we push them into all ioctls and decide how to handle/where (with the aim to have the least amount of code flow different during hotunplug vs after hotunplug has finished, to reduce testing scope)
- then we make DRM_HOTUNPLUG_SAFE the implied default
This would have us left with render ioctls, and I think the defensive assumption there is that they're all hotunplug safe. We might hang on a fence wait, but that's fixable, and it's better than blowing up on a use-after-free security bug.
Thoughts?
I don't fully see a difference between the approach described above and the full drill down to each driver and even within the driver, to the HW back-ends - what criteria I would use to decide if for a given IOCTL i scope with drm_dev_enter/exit at the highest level while for another i go all the way down ? If we would agree that signaling the fences preemptively before engaging drm_dev_unplug is generically the right approach maybe we can then scope drm_ioctl unconditionally with drm_dev_enter/exit and then for each driver go through the same process I do for amdgpu - writing driver specific function which takes care of all the fences. We could then just create a drm callback which would be called from drm_ioctl before drm_dev_unplug is called.
So I see the appeal of just nuking all the fences, but I'm not sure that's a good plan. We've done this in the old i915 gpu reset code too, and the issue is it's defacto inverting the locking. But also the hw is truly gone, so it also makes sense.
The problem is a bit roll-out, if we state that dma_fence_wait is allowed with a drm_dev_enter/exit, then all drivers need to force-retire their fences.
The other option would be that we require that dma_fence_wait is _not_ allowed in drm_dev_enter/exit, and that therefore these areas must be marked up more fine-grained to avoid deadlocks. I like this more from the testing aspect (it makes it easier to be reasonable sure your code handles concurrent hotunplug), but also it's pretty easy to validate with the dma_fence lockdep annotations we have I think.
A third reasons for not requiring force-retiring of dma_fence before drm_dev_unplug is the races: Before drm_dev_unplug you haven't stopped new fences from happening, but until you've stopped new fences it's hard to guarantee they're all retired. How do you solve this currently.
Finally there's still hangcheck and all that, so if we go with forbidding dma_fence_wait from within drm_dev_enter/exit sections, then drivers don't need to have additional tricky code to force-retire fences. TDR will take care already (albeit with maybe a slightly annoying long timeout, which we can shorten to "time out everything immediately" after drm_dev_unplug).
What we definitely can't have is half the drivers doing it one way, and the other half the other way. So your driver flag to wrap the ioctl optionally in a drm_dev_enter/exit path is a no-go still I think.
I guess my tldr; is: I definitely see how your current approach gives quicker results for amdgpu right now, but long term I'm seeing more positives on the other one. At least I expect less special cases due to hotunplug with that.
Cheers, Daniel
Andrey
It is unfortunately even more work until we've reached the goal, but I think it's safest and most flexible approach overall.
Cheers, Daniel
Andrey
Cheers, Daniel
> > I guess we should clarify this in the hotunplug doc?
Agree
To clarify: I'm not against throwing an ENODEV at userspace for ioctl that really make no sense, and where we're rather confident that all properly implemented userspace will gracefully handle failures. Like a modeset, or opening a device, or trying to import a dma-buf or stuff like that which can already fail in normal operation for any kind of reason.
But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail after hotunplug.
As I pointed above, this a bit confuses me given that we already do blanker rejection of IOCTLs if device is unplugged.
Well I'm confused about this too :-/
And then there's the middle ground, like doing a pageflip or buffer flush, which I guess some userspace might handle, but risky to inflict those consequences on them. atomic modeset is especially fun since depending what you're doing it can be both "failures expected" and "failures not really expected in normal operation".
Also, this really should be consistent across drivers, not solved with a driver flag for every possible combination.
If you look at the current hotunplug kms drivers, they have drm_dev_enter/exit sprinkled in specific hw callback functions because of the above problems. But maybe it makes sense to change things in a few cases. But then we should do it across the board.
So as I understand your preferred approach is that I scope any back_end, HW specific function with drm_dev_enter/exit because that where MMIO access takes place. But besides explicit MMIO access thorough register accessors in the HW back-end there is also indirect MMIO access taking place throughout the code in the driver because of various VRAM BOs which provide CPU access to VRAM through the VRAM BAR. This kind of access is spread all over in the driver and even in mid-layers such as TTM and not limited to HW back-end functions. It means it's much harder to spot such places to surgically scope them with drm_dev_enter/exit and also that any new such code introduced will immediately break hot unplug because the developers can't be expected to remember making their code robust to this specific use case. That why when we discussed internally what approach to take to protecting code with drm_dev_enter/exit we opted for using the widest available scope.
The thing is, you kinda have to anyway. There's enormous amounts of asynchronous processing going on. E.g. nonblocking atomic commits also do ttm unpinning and fun stuff like that, which if you sync things wrong can happen way late. So the door for bad fallout is wide open :-(
I'm not sure where the right tradeoff is to make sure we catch them all, and can make sure with testing that we've indeed caught them all. -Daniel
On 2021-05-06 5:40 a.m., Daniel Vetter wrote:
On Fri, Apr 30, 2021 at 01:27:37PM -0400, Andrey Grodzovsky wrote:
On 2021-04-30 6:25 a.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote:
On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
On 2021-04-29 7:32 a.m., Daniel Vetter wrote: > On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote: >> On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote: >>> With this calling drm_dev_unplug will flush and block >>> all in flight IOCTLs >>> >>> Also, add feature such that if device supports graceful unplug >>> we enclose entire IOCTL in SRCU critical section. >>> >>> Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com >> >> Nope. >> >> The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.boo... currently in code ?
I forgot about this one, again. Thanks for reminding.
>> Especially not with an opt-in flag so that it could be shrugged of as a >> driver hack. Most of these ioctls should have absolutely no problem >> working after hotunplug. >> >> Also, doing this defeats the point since it pretty much guarantees >> userspace will die in assert()s and stuff. E.g. on i915 the rough contract >> is that only execbuf (and even that only when userspace has indicated >> support for non-recoverable hw ctx) is allowed to fail. Anything else >> might crash userspace.
Given that as I pointed above we already fail any IOCTls with -ENODEV when device is unplugged, it seems those crashes don't happen that often ? Also, in all my testing I don't think I saw a user space crash I could attribute to this.
I guess it should be ok.
What should be ok ?
Your approach, but not your patch. If we go with this let's just lift it to drm_ioctl() as the default behavior. No driver opt-in flag, because that's definitely worse than any other approach because we really need to get rid of driver specific behaviour for generic ioctls, especially anything a compositor will use directly.
My reasons for making this work is both less trouble for userspace (did you test with various wayland compositors out there, not just amdgpu x86
I didn't - will give it a try.
Weston worked without crashes, run the egl tester cube there.
driver?), but also testing.
We still need a bunch of these checks in various places or you'll wait a very long time for a pending modeset or similar to complete. Being able to run that code easily after hotunplug has completed should help a lot with testing.
Plus various drivers already acquired drm_dev_enter/exit and now I wonder whether that was properly tested or not ...
I guess maybe we need a drm module option to disable this check, so that we can exercise the code as if the ioctl has raced with hotunplug at the worst possible moment.
Also atomic is really tricky here: I assume your testing has just done normal synchronous commits, but anything that goes through atomic can be done nonblocking in a separate thread. Which the ioctl catch-all here wont capture.
Yes, async commit was on my mind and thanks for reminding me. Indeed I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in drm_dev_enter/exit. Note that i have a bunch of patches, all name's starting with 'Scope....' that just methodically put all the background work items and timers the drivers schedules in drm_dev_enter/exit scope. This was supposed to be part of the 'Scope Display code' patch.
That's too much. You still have to arrange that the flip completion event gets sent out. So it's a bit tricky.
In other places the same problem applies, e.g. probe functions need to make sure they report "disconnected".
I see, well, this is all part of KMS support which I defer for now anyway. Will tackle it then.
>> You probably need similar (and very precisely defined) rules for amdgpu. >> And those must definitely exclude any shard ioctls from randomly failing >> with EIO, because that just kills the box and defeats the point of trying >> to gracefully handling hotunplug and making sure userspace has a chance of >> survival. E.g. for atomic everything should continue, including flip >> completion, but we set all outputs to "disconnected" and send out the >> uevent. Maybe crtc enabling can fail too, but that can also be handled >> through the async status we're using to signal DP link failures to >> userspace.
As I pointed before, because of the complexity of the topic I prefer to take it step by step and solve first for secondary device use case, not for primary, display attached device.
Yeah makes sense. But then I think the right patch is to roll this out for all drivers, properly justified with existing code. Not behind a driver flag, because with all these different compositors the last thing we want is a proliferation of driver-specific behaviour. That's imo the worst option of all of them and needs to be avoided.
So this kind of patch would be acceptable to you if I unconditionally scope the drm_ioctl with drm_dev_enter/exit without the driver flag ? I am worried to break other drivers with this, see patch https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedes... Before setting drm_dev_unplug I go through a whole process of signalling all possible fences in the system which some one some where might be waiting on. My concern is that in the absence of HW those fences won't signal and so unless I signal them myself srcu_synchrionize in drm_dev_unplug will hang waiting for any such code scoped by drm_dev_enter/exit.
Uh right. I forgot about this.
Which would kinda mean the top level scope is maybe not the best idea, and perhaps we should indeed drill it down. But then the testing issue definitely gets a lot worse.
So what if we'd push that drm_dev_is_unplugged check down into ioctls? Then we can make a case-by case decision whether it should be converted to drm_dev_enter/exit, needs to be pushed down further into drivers (due to fence wait issues) or other concerns?
Also I guess we need to have a subsystem wide rule on whether you need to force complete all fences before you call drm_dev_unplug, or afterwards.
I don't see how you can handle it afterwards. If a thread is stuck in dma_fence_wait in non interruptible wait (any kernel thread) and with no timeout there is nothing you can do to stop the wait. Any such code scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug. The only way then is to preemptively force signal all such fences before calling drm_dev_unplug - as I do in the above mentioned patch.
Yeah, which is why I don't think top-level drm_dev_enter/exit is a good idea.
If we have mixed behaviour on this there will be disappointment. And since hotunplug and dma_fence completion are both userspace visible that inconsistency might have bigger impact.
This is all very tricky indeed :-/
btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go with that: We could do the same trick we've done for DRM_UNLOCKED:
- drm_dev_enter/exit is called for any ioctl that has not set the DRM_HOTUNPLUG_SAFE flag
- for drm core ioctls we push them into all ioctls and decide how to handle/where (with the aim to have the least amount of code flow different during hotunplug vs after hotunplug has finished, to reduce testing scope)
- then we make DRM_HOTUNPLUG_SAFE the implied default
This would have us left with render ioctls, and I think the defensive assumption there is that they're all hotunplug safe. We might hang on a fence wait, but that's fixable, and it's better than blowing up on a use-after-free security bug.
Thoughts?
I don't fully see a difference between the approach described above and the full drill down to each driver and even within the driver, to the HW back-ends - what criteria I would use to decide if for a given IOCTL i scope with drm_dev_enter/exit at the highest level while for another i go all the way down ? If we would agree that signaling the fences preemptively before engaging drm_dev_unplug is generically the right approach maybe we can then scope drm_ioctl unconditionally with drm_dev_enter/exit and then for each driver go through the same process I do for amdgpu - writing driver specific function which takes care of all the fences. We could then just create a drm callback which would be called from drm_ioctl before drm_dev_unplug is called.
So I see the appeal of just nuking all the fences, but I'm not sure that's a good plan. We've done this in the old i915 gpu reset code too, and the issue is it's defacto inverting the locking. But also the hw is truly gone, so it also makes sense.
The problem is a bit roll-out, if we state that dma_fence_wait is allowed with a drm_dev_enter/exit, then all drivers need to force-retire their fences.
The other option would be that we require that dma_fence_wait is _not_ allowed in drm_dev_enter/exit, and that therefore these areas must be marked up more fine-grained to avoid deadlocks. I like this more from the testing aspect (it makes it easier to be reasonable sure your code handles concurrent hotunplug), but also it's pretty easy to validate with the dma_fence lockdep annotations we have I think.
They key question as I see it - is it ok for a device to be unplugged while it's driver has anywhere in it's code a dma_fence_wait waiting for work completion from this device. The answers seems to me is no, the HW is gone, this fence will never signal and so you will be left with indefinitely hanged code thread with all it's unreleased resources. If i am correct in the above statement then avoiding scoping code with drm_dev_enter/exit because a dma_fence_wait might be there in the middle just hides the problem. Also, then the only solution for each driver wanting to support hot-unplug is to force retire all it's HW fences once it's notified of device removal.
A third reasons for not requiring force-retiring of dma_fence before drm_dev_unplug is the races: Before drm_dev_unplug you haven't stopped new fences from happening, but until you've stopped new fences it's hard to guarantee they're all retired. How do you solve this currently.
See amdgpu_finilize_device_fences in https://patchwork.ozlabs.org/project/linux-pci/patch/20210428151207.1212258-... I think the steps described there answer your concern here.
Finally there's still hangcheck and all that, so if we go with forbidding dma_fence_wait from within drm_dev_enter/exit sections, then drivers don't need to have additional tricky code to force-retire fences. TDR will take care already (albeit with maybe a slightly annoying long timeout, which we can shorten to "time out everything immediately" after drm_dev_unplug).
I am not aware of TDR handlers that do it today, at least we don't, we don't check that if device is gone let's instead of resetting the device and resubmit jobs just force retire all the HW fences. In any case, this can and i think should be done in pci remove callback because this is the place that supposed to handle device extraction. I for example in amdgpu_finilize_device_fences just block all TDRs from taking place as first step in the process. If other drivers want to force retire fences in their TDR handlers they still need to block and wait for all such TDRs in their pci_remove handler.
What we definitely can't have is half the drivers doing it one way, and the other half the other way. So your driver flag to wrap the ioctl optionally in a drm_dev_enter/exit path is a no-go still I think.
I guess my tldr; is: I definitely see how your current approach gives quicker results for amdgpu right now, but long term I'm seeing more positives on the other one. At least I expect less special cases due to hotunplug with that.
As i expressed my viewpoint above - seems to me any driver in need to support hot-unplug must force retire it's fences because of need to unblock all dma_fence waits and so it will not be a special case.
Andrey
Cheers, Daniel
Andrey
It is unfortunately even more work until we've reached the goal, but I think it's safest and most flexible approach overall.
Cheers, Daniel
Andrey
Cheers, Daniel
>> >> I guess we should clarify this in the hotunplug doc?
Agree
> > To clarify: I'm not against throwing an ENODEV at userspace for ioctl that > really make no sense, and where we're rather confident that all properly > implemented userspace will gracefully handle failures. Like a modeset, or > opening a device, or trying to import a dma-buf or stuff like that which > can already fail in normal operation for any kind of reason. > > But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail > after hotunplug.
As I pointed above, this a bit confuses me given that we already do blanker rejection of IOCTLs if device is unplugged.
Well I'm confused about this too :-/
> And then there's the middle ground, like doing a pageflip or buffer flush, > which I guess some userspace might handle, but risky to inflict those > consequences on them. atomic modeset is especially fun since depending > what you're doing it can be both "failures expected" and "failures not > really expected in normal operation". > > Also, this really should be consistent across drivers, not solved with a > driver flag for every possible combination. > > If you look at the current hotunplug kms drivers, they have > drm_dev_enter/exit sprinkled in specific hw callback functions because of > the above problems. But maybe it makes sense to change things in a few > cases. But then we should do it across the board.
So as I understand your preferred approach is that I scope any back_end, HW specific function with drm_dev_enter/exit because that where MMIO access takes place. But besides explicit MMIO access thorough register accessors in the HW back-end there is also indirect MMIO access taking place throughout the code in the driver because of various VRAM BOs which provide CPU access to VRAM through the VRAM BAR. This kind of access is spread all over in the driver and even in mid-layers such as TTM and not limited to HW back-end functions. It means it's much harder to spot such places to surgically scope them with drm_dev_enter/exit and also that any new such code introduced will immediately break hot unplug because the developers can't be expected to remember making their code robust to this specific use case. That why when we discussed internally what approach to take to protecting code with drm_dev_enter/exit we opted for using the widest available scope.
The thing is, you kinda have to anyway. There's enormous amounts of asynchronous processing going on. E.g. nonblocking atomic commits also do ttm unpinning and fun stuff like that, which if you sync things wrong can happen way late. So the door for bad fallout is wide open :-(
I'm not sure where the right tradeoff is to make sure we catch them all, and can make sure with testing that we've indeed caught them all. -Daniel
On Thu, May 06, 2021 at 12:25:06PM -0400, Andrey Grodzovsky wrote:
On 2021-05-06 5:40 a.m., Daniel Vetter wrote:
On Fri, Apr 30, 2021 at 01:27:37PM -0400, Andrey Grodzovsky wrote:
On 2021-04-30 6:25 a.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote:
On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote: > > > On 2021-04-29 7:32 a.m., Daniel Vetter wrote: > > On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote: > > > On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote: > > > > With this calling drm_dev_unplug will flush and block > > > > all in flight IOCTLs > > > > > > > > Also, add feature such that if device supports graceful unplug > > > > we enclose entire IOCTL in SRCU critical section. > > > > > > > > Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com > > > > > > Nope. > > > > > > The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl. > > Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.boo... > currently in code ?
I forgot about this one, again. Thanks for reminding.
> > > Especially not with an opt-in flag so that it could be shrugged of as a > > > driver hack. Most of these ioctls should have absolutely no problem > > > working after hotunplug. > > > > > > Also, doing this defeats the point since it pretty much guarantees > > > userspace will die in assert()s and stuff. E.g. on i915 the rough contract > > > is that only execbuf (and even that only when userspace has indicated > > > support for non-recoverable hw ctx) is allowed to fail. Anything else > > > might crash userspace. > > Given that as I pointed above we already fail any IOCTls with -ENODEV > when device is unplugged, it seems those crashes don't happen that > often ? Also, in all my testing I don't think I saw a user space crash > I could attribute to this.
I guess it should be ok.
What should be ok ?
Your approach, but not your patch. If we go with this let's just lift it to drm_ioctl() as the default behavior. No driver opt-in flag, because that's definitely worse than any other approach because we really need to get rid of driver specific behaviour for generic ioctls, especially anything a compositor will use directly.
My reasons for making this work is both less trouble for userspace (did you test with various wayland compositors out there, not just amdgpu x86
I didn't - will give it a try.
Weston worked without crashes, run the egl tester cube there.
driver?), but also testing.
We still need a bunch of these checks in various places or you'll wait a very long time for a pending modeset or similar to complete. Being able to run that code easily after hotunplug has completed should help a lot with testing.
Plus various drivers already acquired drm_dev_enter/exit and now I wonder whether that was properly tested or not ...
I guess maybe we need a drm module option to disable this check, so that we can exercise the code as if the ioctl has raced with hotunplug at the worst possible moment.
Also atomic is really tricky here: I assume your testing has just done normal synchronous commits, but anything that goes through atomic can be done nonblocking in a separate thread. Which the ioctl catch-all here wont capture.
Yes, async commit was on my mind and thanks for reminding me. Indeed I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in drm_dev_enter/exit. Note that i have a bunch of patches, all name's starting with 'Scope....' that just methodically put all the background work items and timers the drivers schedules in drm_dev_enter/exit scope. This was supposed to be part of the 'Scope Display code' patch.
That's too much. You still have to arrange that the flip completion event gets sent out. So it's a bit tricky.
In other places the same problem applies, e.g. probe functions need to make sure they report "disconnected".
I see, well, this is all part of KMS support which I defer for now anyway. Will tackle it then.
> > > You probably need similar (and very precisely defined) rules for amdgpu. > > > And those must definitely exclude any shard ioctls from randomly failing > > > with EIO, because that just kills the box and defeats the point of trying > > > to gracefully handling hotunplug and making sure userspace has a chance of > > > survival. E.g. for atomic everything should continue, including flip > > > completion, but we set all outputs to "disconnected" and send out the > > > uevent. Maybe crtc enabling can fail too, but that can also be handled > > > through the async status we're using to signal DP link failures to > > > userspace. > > As I pointed before, because of the complexity of the topic I prefer to > take it step by step and solve first for secondary device use case, not > for primary, display attached device.
Yeah makes sense. But then I think the right patch is to roll this out for all drivers, properly justified with existing code. Not behind a driver flag, because with all these different compositors the last thing we want is a proliferation of driver-specific behaviour. That's imo the worst option of all of them and needs to be avoided.
So this kind of patch would be acceptable to you if I unconditionally scope the drm_ioctl with drm_dev_enter/exit without the driver flag ? I am worried to break other drivers with this, see patch https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedes... Before setting drm_dev_unplug I go through a whole process of signalling all possible fences in the system which some one some where might be waiting on. My concern is that in the absence of HW those fences won't signal and so unless I signal them myself srcu_synchrionize in drm_dev_unplug will hang waiting for any such code scoped by drm_dev_enter/exit.
Uh right. I forgot about this.
Which would kinda mean the top level scope is maybe not the best idea, and perhaps we should indeed drill it down. But then the testing issue definitely gets a lot worse.
So what if we'd push that drm_dev_is_unplugged check down into ioctls? Then we can make a case-by case decision whether it should be converted to drm_dev_enter/exit, needs to be pushed down further into drivers (due to fence wait issues) or other concerns?
Also I guess we need to have a subsystem wide rule on whether you need to force complete all fences before you call drm_dev_unplug, or afterwards.
I don't see how you can handle it afterwards. If a thread is stuck in dma_fence_wait in non interruptible wait (any kernel thread) and with no timeout there is nothing you can do to stop the wait. Any such code scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug. The only way then is to preemptively force signal all such fences before calling drm_dev_unplug - as I do in the above mentioned patch.
Yeah, which is why I don't think top-level drm_dev_enter/exit is a good idea.
If we have mixed behaviour on this there will be disappointment. And since hotunplug and dma_fence completion are both userspace visible that inconsistency might have bigger impact.
This is all very tricky indeed :-/
btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go with that: We could do the same trick we've done for DRM_UNLOCKED:
- drm_dev_enter/exit is called for any ioctl that has not set the DRM_HOTUNPLUG_SAFE flag
- for drm core ioctls we push them into all ioctls and decide how to handle/where (with the aim to have the least amount of code flow different during hotunplug vs after hotunplug has finished, to reduce testing scope)
- then we make DRM_HOTUNPLUG_SAFE the implied default
This would have us left with render ioctls, and I think the defensive assumption there is that they're all hotunplug safe. We might hang on a fence wait, but that's fixable, and it's better than blowing up on a use-after-free security bug.
Thoughts?
I don't fully see a difference between the approach described above and the full drill down to each driver and even within the driver, to the HW back-ends - what criteria I would use to decide if for a given IOCTL i scope with drm_dev_enter/exit at the highest level while for another i go all the way down ? If we would agree that signaling the fences preemptively before engaging drm_dev_unplug is generically the right approach maybe we can then scope drm_ioctl unconditionally with drm_dev_enter/exit and then for each driver go through the same process I do for amdgpu - writing driver specific function which takes care of all the fences. We could then just create a drm callback which would be called from drm_ioctl before drm_dev_unplug is called.
So I see the appeal of just nuking all the fences, but I'm not sure that's a good plan. We've done this in the old i915 gpu reset code too, and the issue is it's defacto inverting the locking. But also the hw is truly gone, so it also makes sense.
The problem is a bit roll-out, if we state that dma_fence_wait is allowed with a drm_dev_enter/exit, then all drivers need to force-retire their fences.
The other option would be that we require that dma_fence_wait is _not_ allowed in drm_dev_enter/exit, and that therefore these areas must be marked up more fine-grained to avoid deadlocks. I like this more from the testing aspect (it makes it easier to be reasonable sure your code handles concurrent hotunplug), but also it's pretty easy to validate with the dma_fence lockdep annotations we have I think.
They key question as I see it - is it ok for a device to be unplugged while it's driver has anywhere in it's code a dma_fence_wait waiting for work completion from this device. The answers seems to me is no, the HW is gone, this fence will never signal and so you will be left with indefinitely hanged code thread with all it's unreleased resources. If i am correct in the above statement then avoiding scoping code with drm_dev_enter/exit because a dma_fence_wait might be there in the middle just hides the problem. Also, then the only solution for each driver wanting to support hot-unplug is to force retire all it's HW fences once it's notified of device removal.
At a high level, yes dma_fence must always complete. I don't think we have a disagreement here on that.
What we're discussing here is the precise sequencing and barriers, where things get tricky. Requiring that you force-complete all dma_fence that might be affected before you hotunplug is one solution, the other is tuning the critical sections that drm_dev_enter/exit annotates.
This isn't about avoiding anything or hiding problems, this is about locking/synchronization design. And for that we must agree on what is allowed inside/outside of a critical section for all possible combinations.
E.g. we're also "hiding" problems with calling dma_fence_wait from shrinkers/mmu notifiers by forbidding allocations in dma_fence_begin/end_signalling critical paths.
A third reasons for not requiring force-retiring of dma_fence before drm_dev_unplug is the races: Before drm_dev_unplug you haven't stopped new fences from happening, but until you've stopped new fences it's hard to guarantee they're all retired. How do you solve this currently.
See amdgpu_finilize_device_fences in https://patchwork.ozlabs.org/project/linux-pci/patch/20210428151207.1212258-... I think the steps described there answer your concern here.
The hard problem is stopping further command submission. Not seeing how you solve that.
But I'm definitely scared about all the scheduler/tdr interactions you already have there, and that looks quite a bit like fallout from doing things the wrong way round.
Also given that drm/scheduler is shared, why can't this be a drm/scheduler helper function?
Finally there's still hangcheck and all that, so if we go with forbidding dma_fence_wait from within drm_dev_enter/exit sections, then drivers don't need to have additional tricky code to force-retire fences. TDR will take care already (albeit with maybe a slightly annoying long timeout, which we can shorten to "time out everything immediately" after drm_dev_unplug).
I am not aware of TDR handlers that do it today, at least we don't, we don't check that if device is gone let's instead of resetting the device and resubmit jobs just force retire all the HW fences. In any case, this can and i think should be done in pci remove callback because this is the place that supposed to handle device extraction. I for example in amdgpu_finilize_device_fences just block all TDRs from taking place as first step in the process. If other drivers want to force retire fences in their TDR handlers they still need to block and wait for all such TDRs in their pci_remove handler.
TDR definitely force-completes the fence that did hang. Of course it'll take a while until they've all completed this way, but we do have guaranteed forward progress since we've stopped all further fences from showing up because drm_dev_unplug is called already.
And yes after drm_dev_unplug you can then force-retire the tdr stuff.
What we definitely can't have is half the drivers doing it one way, and the other half the other way. So your driver flag to wrap the ioctl optionally in a drm_dev_enter/exit path is a no-go still I think.
I guess my tldr; is: I definitely see how your current approach gives quicker results for amdgpu right now, but long term I'm seeing more positives on the other one. At least I expect less special cases due to hotunplug with that.
As i expressed my viewpoint above - seems to me any driver in need to support hot-unplug must force retire it's fences because of need to unblock all dma_fence waits and so it will not be a special case.
This isn't the special case I meant. It's the very tricky force-retire-before-you-unplugged-officially which is large scale nasty.
Also if your driver doesn't force-retire already, it's buggy. The additional need of hotunplug is just that we're trying to force-retire a bit faster, because we know it's all hopeless. But e.g. i915 already has a fallback that does this automatically: - first we reset only the engine/context, keeping everyone else running - if that doesn't pan out, we reset the entire chip and give up an anything that's in-flight, which (iirc, it did so at least in the past) force retires everything outstanding.
I think amdgpu only has full chip reset, so your first step tries to reissue all other tasks. But that's not necessarily how it needs to happen.
Either way drivers must force retire everything (albeit maybe a bit at a slow pace) if the hw ceased to work properly already. Hotunplug really isn't anything new here. -Daniel
Andrey
Cheers, Daniel
Andrey
It is unfortunately even more work until we've reached the goal, but I think it's safest and most flexible approach overall.
Cheers, Daniel
Andrey
Cheers, Daniel
> > > > > > > I guess we should clarify this in the hotunplug doc? > > Agree > > > > > To clarify: I'm not against throwing an ENODEV at userspace for ioctl that > > really make no sense, and where we're rather confident that all properly > > implemented userspace will gracefully handle failures. Like a modeset, or > > opening a device, or trying to import a dma-buf or stuff like that which > > can already fail in normal operation for any kind of reason. > > > > But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail > > after hotunplug. > > As I pointed above, this a bit confuses me given that we already do > blanker rejection of IOCTLs if device is unplugged.
Well I'm confused about this too :-/
> > And then there's the middle ground, like doing a pageflip or buffer flush, > > which I guess some userspace might handle, but risky to inflict those > > consequences on them. atomic modeset is especially fun since depending > > what you're doing it can be both "failures expected" and "failures not > > really expected in normal operation". > > > > Also, this really should be consistent across drivers, not solved with a > > driver flag for every possible combination. > > > > If you look at the current hotunplug kms drivers, they have > > drm_dev_enter/exit sprinkled in specific hw callback functions because of > > the above problems. But maybe it makes sense to change things in a few > > cases. But then we should do it across the board. > > So as I understand your preferred approach is that I scope any back_end, HW > specific function with drm_dev_enter/exit because that where MMIO > access takes place. But besides explicit MMIO access thorough > register accessors in the HW back-end there is also indirect MMIO access > taking place throughout the code in the driver because of various VRAM > BOs which provide CPU access to VRAM through the VRAM BAR. This kind of > access is spread all over in the driver and even in mid-layers such as > TTM and not limited to HW back-end functions. It means it's much harder > to spot such places to surgically scope them with drm_dev_enter/exit and > also that any new such code introduced will immediately break hot unplug > because the developers can't be expected to remember making their code > robust to this specific use case. That why when we discussed internally > what approach to take to protecting code with drm_dev_enter/exit we > opted for using the widest available scope.
The thing is, you kinda have to anyway. There's enormous amounts of asynchronous processing going on. E.g. nonblocking atomic commits also do ttm unpinning and fun stuff like that, which if you sync things wrong can happen way late. So the door for bad fallout is wide open :-(
I'm not sure where the right tradeoff is to make sure we catch them all, and can make sure with testing that we've indeed caught them all. -Daniel
On 2021-05-07 5:11 a.m., Daniel Vetter wrote:
On Thu, May 06, 2021 at 12:25:06PM -0400, Andrey Grodzovsky wrote:
On 2021-05-06 5:40 a.m., Daniel Vetter wrote:
On Fri, Apr 30, 2021 at 01:27:37PM -0400, Andrey Grodzovsky wrote:
On 2021-04-30 6:25 a.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote:
On 2021-04-29 3:05 p.m., Daniel Vetter wrote: > On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote: >> >> >> On 2021-04-29 7:32 a.m., Daniel Vetter wrote: >>> On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote: >>>> On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote: >>>>> With this calling drm_dev_unplug will flush and block >>>>> all in flight IOCTLs >>>>> >>>>> Also, add feature such that if device supports graceful unplug >>>>> we enclose entire IOCTL in SRCU critical section. >>>>> >>>>> Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com >>>> >>>> Nope. >>>> >>>> The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl. >> >> Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.boo... >> currently in code ? > > I forgot about this one, again. Thanks for reminding. > >>>> Especially not with an opt-in flag so that it could be shrugged of as a >>>> driver hack. Most of these ioctls should have absolutely no problem >>>> working after hotunplug. >>>> >>>> Also, doing this defeats the point since it pretty much guarantees >>>> userspace will die in assert()s and stuff. E.g. on i915 the rough contract >>>> is that only execbuf (and even that only when userspace has indicated >>>> support for non-recoverable hw ctx) is allowed to fail. Anything else >>>> might crash userspace. >> >> Given that as I pointed above we already fail any IOCTls with -ENODEV >> when device is unplugged, it seems those crashes don't happen that >> often ? Also, in all my testing I don't think I saw a user space crash >> I could attribute to this. > > I guess it should be ok.
What should be ok ?
Your approach, but not your patch. If we go with this let's just lift it to drm_ioctl() as the default behavior. No driver opt-in flag, because that's definitely worse than any other approach because we really need to get rid of driver specific behaviour for generic ioctls, especially anything a compositor will use directly.
> My reasons for making this work is both less trouble for userspace (did > you test with various wayland compositors out there, not just amdgpu x86
I didn't - will give it a try.
Weston worked without crashes, run the egl tester cube there.
> driver?), but also testing. > > We still need a bunch of these checks in various places or you'll wait a > very long time for a pending modeset or similar to complete. Being able to > run that code easily after hotunplug has completed should help a lot with > testing. > > Plus various drivers already acquired drm_dev_enter/exit and now I wonder > whether that was properly tested or not ... > > I guess maybe we need a drm module option to disable this check, so that > we can exercise the code as if the ioctl has raced with hotunplug at the > worst possible moment. > > Also atomic is really tricky here: I assume your testing has just done > normal synchronous commits, but anything that goes through atomic can be > done nonblocking in a separate thread. Which the ioctl catch-all here wont > capture.
Yes, async commit was on my mind and thanks for reminding me. Indeed I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in drm_dev_enter/exit. Note that i have a bunch of patches, all name's starting with 'Scope....' that just methodically put all the background work items and timers the drivers schedules in drm_dev_enter/exit scope. This was supposed to be part of the 'Scope Display code' patch.
That's too much. You still have to arrange that the flip completion event gets sent out. So it's a bit tricky.
In other places the same problem applies, e.g. probe functions need to make sure they report "disconnected".
I see, well, this is all part of KMS support which I defer for now anyway. Will tackle it then.
>>>> You probably need similar (and very precisely defined) rules for amdgpu. >>>> And those must definitely exclude any shard ioctls from randomly failing >>>> with EIO, because that just kills the box and defeats the point of trying >>>> to gracefully handling hotunplug and making sure userspace has a chance of >>>> survival. E.g. for atomic everything should continue, including flip >>>> completion, but we set all outputs to "disconnected" and send out the >>>> uevent. Maybe crtc enabling can fail too, but that can also be handled >>>> through the async status we're using to signal DP link failures to >>>> userspace. >> >> As I pointed before, because of the complexity of the topic I prefer to >> take it step by step and solve first for secondary device use case, not >> for primary, display attached device. > > Yeah makes sense. But then I think the right patch is to roll this out for > all drivers, properly justified with existing code. Not behind a driver > flag, because with all these different compositors the last thing we want > is a proliferation of driver-specific behaviour. That's imo the worst > option of all of them and needs to be avoided.
So this kind of patch would be acceptable to you if I unconditionally scope the drm_ioctl with drm_dev_enter/exit without the driver flag ? I am worried to break other drivers with this, see patch https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedes... Before setting drm_dev_unplug I go through a whole process of signalling all possible fences in the system which some one some where might be waiting on. My concern is that in the absence of HW those fences won't signal and so unless I signal them myself srcu_synchrionize in drm_dev_unplug will hang waiting for any such code scoped by drm_dev_enter/exit.
Uh right. I forgot about this.
Which would kinda mean the top level scope is maybe not the best idea, and perhaps we should indeed drill it down. But then the testing issue definitely gets a lot worse.
So what if we'd push that drm_dev_is_unplugged check down into ioctls? Then we can make a case-by case decision whether it should be converted to drm_dev_enter/exit, needs to be pushed down further into drivers (due to fence wait issues) or other concerns?
Also I guess we need to have a subsystem wide rule on whether you need to force complete all fences before you call drm_dev_unplug, or afterwards.
I don't see how you can handle it afterwards. If a thread is stuck in dma_fence_wait in non interruptible wait (any kernel thread) and with no timeout there is nothing you can do to stop the wait. Any such code scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug. The only way then is to preemptively force signal all such fences before calling drm_dev_unplug - as I do in the above mentioned patch.
Yeah, which is why I don't think top-level drm_dev_enter/exit is a good idea.
If we have mixed behaviour on this there will be disappointment. And since hotunplug and dma_fence completion are both userspace visible that inconsistency might have bigger impact.
This is all very tricky indeed :-/
btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go with that: We could do the same trick we've done for DRM_UNLOCKED:
- drm_dev_enter/exit is called for any ioctl that has not set the DRM_HOTUNPLUG_SAFE flag
- for drm core ioctls we push them into all ioctls and decide how to handle/where (with the aim to have the least amount of code flow different during hotunplug vs after hotunplug has finished, to reduce testing scope)
- then we make DRM_HOTUNPLUG_SAFE the implied default
This would have us left with render ioctls, and I think the defensive assumption there is that they're all hotunplug safe. We might hang on a fence wait, but that's fixable, and it's better than blowing up on a use-after-free security bug.
Thoughts?
I don't fully see a difference between the approach described above and the full drill down to each driver and even within the driver, to the HW back-ends - what criteria I would use to decide if for a given IOCTL i scope with drm_dev_enter/exit at the highest level while for another i go all the way down ? If we would agree that signaling the fences preemptively before engaging drm_dev_unplug is generically the right approach maybe we can then scope drm_ioctl unconditionally with drm_dev_enter/exit and then for each driver go through the same process I do for amdgpu - writing driver specific function which takes care of all the fences. We could then just create a drm callback which would be called from drm_ioctl before drm_dev_unplug is called.
So I see the appeal of just nuking all the fences, but I'm not sure that's a good plan. We've done this in the old i915 gpu reset code too, and the issue is it's defacto inverting the locking. But also the hw is truly gone, so it also makes sense.
The problem is a bit roll-out, if we state that dma_fence_wait is allowed with a drm_dev_enter/exit, then all drivers need to force-retire their fences.
The other option would be that we require that dma_fence_wait is _not_ allowed in drm_dev_enter/exit, and that therefore these areas must be marked up more fine-grained to avoid deadlocks. I like this more from the testing aspect (it makes it easier to be reasonable sure your code handles concurrent hotunplug), but also it's pretty easy to validate with the dma_fence lockdep annotations we have I think.
They key question as I see it - is it ok for a device to be unplugged while it's driver has anywhere in it's code a dma_fence_wait waiting for work completion from this device. The answers seems to me is no, the HW is gone, this fence will never signal and so you will be left with indefinitely hanged code thread with all it's unreleased resources. If i am correct in the above statement then avoiding scoping code with drm_dev_enter/exit because a dma_fence_wait might be there in the middle just hides the problem. Also, then the only solution for each driver wanting to support hot-unplug is to force retire all it's HW fences once it's notified of device removal.
At a high level, yes dma_fence must always complete. I don't think we have a disagreement here on that.
What we're discussing here is the precise sequencing and barriers, where things get tricky. Requiring that you force-complete all dma_fence that might be affected before you hotunplug is one solution, the other is tuning the critical sections that drm_dev_enter/exit annotates.
This isn't about avoiding anything or hiding problems, this is about locking/synchronization design. And for that we must agree on what is allowed inside/outside of a critical section for all possible combinations.
E.g. we're also "hiding" problems with calling dma_fence_wait from shrinkers/mmu notifiers by forbidding allocations in dma_fence_begin/end_signalling critical paths.
A third reasons for not requiring force-retiring of dma_fence before drm_dev_unplug is the races: Before drm_dev_unplug you haven't stopped new fences from happening, but until you've stopped new fences it's hard to guarantee they're all retired. How do you solve this currently.
See amdgpu_finilize_device_fences in https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.... I think the steps described there answer your concern here.
The hard problem is stopping further command submission. Not seeing how you solve that.
By stopping GPU SW scheduler before force completion of HW fences, see amdgpu_finilize_device_fences->amdgpu_fence_driver_fini_hw and the comment above it.
But I'm definitely scared about all the scheduler/tdr interactions you already have there, and that looks quite a bit like fallout from doing things the wrong way round.
Also given that drm/scheduler is shared, why can't this be a drm/scheduler helper function?
I was thinking about it, what stopped me is that HW fences signaling is done from driver specific HW fence array. But we could do it generic by instead iterating sched->pending_list and signaling s_job->s_fence->parent instead. You also need to retire scheduler's scheduled fences once you stopped the schedulers as they are waited on as dependencies for other jobs submissions (i do take care of it).
Finally there's still hangcheck and all that, so if we go with forbidding dma_fence_wait from within drm_dev_enter/exit sections, then drivers don't need to have additional tricky code to force-retire fences. TDR will take care already (albeit with maybe a slightly annoying long timeout, which we can shorten to "time out everything immediately" after drm_dev_unplug).
I am not aware of TDR handlers that do it today, at least we don't, we don't check that if device is gone let's instead of resetting the device and resubmit jobs just force retire all the HW fences. In any case, this can and i think should be done in pci remove callback because this is the place that supposed to handle device extraction. I for example in amdgpu_finilize_device_fences just block all TDRs from taking place as first step in the process. If other drivers want to force retire fences in their TDR handlers they still need to block and wait for all such TDRs in their pci_remove handler.
TDR definitely force-completes the fence that did hang. Of course it'll take a while until they've all completed this way, but we do have guaranteed forward progress since we've stopped all further fences from showing up because drm_dev_unplug is called already.
And yes after drm_dev_unplug you can then force-retire the tdr stuff.
What we definitely can't have is half the drivers doing it one way, and the other half the other way. So your driver flag to wrap the ioctl optionally in a drm_dev_enter/exit path is a no-go still I think.
I guess my tldr; is: I definitely see how your current approach gives quicker results for amdgpu right now, but long term I'm seeing more positives on the other one. At least I expect less special cases due to hotunplug with that.
As i expressed my viewpoint above - seems to me any driver in need to support hot-unplug must force retire it's fences because of need to unblock all dma_fence waits and so it will not be a special case.
This isn't the special case I meant. It's the very tricky force-retire-before-you-unplugged-officially which is large scale nasty.
Also if your driver doesn't force-retire already, it's buggy. The additional need of hotunplug is just that we're trying to force-retire a bit faster, because we know it's all hopeless. But e.g. i915 already has a fallback that does this automatically:
- first we reset only the engine/context, keeping everyone else running
- if that doesn't pan out, we reset the entire chip and give up an anything that's in-flight, which (iirc, it did so at least in the past) force retires everything outstanding.
I think amdgpu only has full chip reset, so your first step tries to reissue all other tasks. But that's not necessarily how it needs to happen.
Either way drivers must force retire everything (albeit maybe a bit at a slow pace) if the hw ceased to work properly already. Hotunplug really isn't anything new here. -Daniel
Let's then agree on the way forward -
You raised before the following suggestion -
" btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go with that: We could do the same trick we've done for DRM_UNLOCKED: - drm_dev_enter/exit is called for any ioctl that has not set the DRM_HOTUNPLUG_SAFE flag - for drm core ioctls we push them into all ioctls and decide how to handle/where (with the aim to have the least amount of code flow different during hotunplug vs after hotunplug has finished, to reduce testing scope) - then we make DRM_HOTUNPLUG_SAFE the implied default " My problem here is that I have no good understating, criteria for how to decide per each ioctl on the right scope of drm_dev_enter/ exit. It depends on whether each next function call can lead somewhere down the call stack to dma_fence_wait and/or whether it can lead to registers access. Seems to me very hard to cover and error prone.
Another options which we discussed internally before and is basically same as current drivers i guess is simply to scope with drm_dev_enter/ exit all the back-end HW specific callbacks. Those are most of the places MMIO access takes place and by definition no dma_fence_wait can be there as it's HW specific code. This leaves MMIO access through pointers (memcpy, and various pointer de-references) which will need to be protected on case by case, but given that I unmap all MMIO anyway as last step of PCI remove callback, all of them will be found by try and error eventually. I feel more comfortable with this approach as I have a clear understating of how to deal with it.
P.S Please respond on the question for you on the other thread at 'PATCH v5 15/27] drm/scheduler: Fix hang when sched_entity released' about suggestion by Christian of partial up-streaming of this code up to and before the patches dealing with scoping of drm_dev_enter/exit scoping.
Andrey
Andrey
Cheers, Daniel
Andrey
It is unfortunately even more work until we've reached the goal, but I think it's safest and most flexible approach overall.
Cheers, Daniel
Andrey
> > Cheers, Daniel > > >> >>>> >>>> I guess we should clarify this in the hotunplug doc? >> >> Agree >> >>> >>> To clarify: I'm not against throwing an ENODEV at userspace for ioctl that >>> really make no sense, and where we're rather confident that all properly >>> implemented userspace will gracefully handle failures. Like a modeset, or >>> opening a device, or trying to import a dma-buf or stuff like that which >>> can already fail in normal operation for any kind of reason. >>> >>> But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail >>> after hotunplug. >> >> As I pointed above, this a bit confuses me given that we already do >> blanker rejection of IOCTLs if device is unplugged. > > Well I'm confused about this too :-/ > >>> And then there's the middle ground, like doing a pageflip or buffer flush, >>> which I guess some userspace might handle, but risky to inflict those >>> consequences on them. atomic modeset is especially fun since depending >>> what you're doing it can be both "failures expected" and "failures not >>> really expected in normal operation". >>> >>> Also, this really should be consistent across drivers, not solved with a >>> driver flag for every possible combination. >>> >>> If you look at the current hotunplug kms drivers, they have >>> drm_dev_enter/exit sprinkled in specific hw callback functions because of >>> the above problems. But maybe it makes sense to change things in a few >>> cases. But then we should do it across the board. >> >> So as I understand your preferred approach is that I scope any back_end, HW >> specific function with drm_dev_enter/exit because that where MMIO >> access takes place. But besides explicit MMIO access thorough >> register accessors in the HW back-end there is also indirect MMIO access >> taking place throughout the code in the driver because of various VRAM >> BOs which provide CPU access to VRAM through the VRAM BAR. This kind of >> access is spread all over in the driver and even in mid-layers such as >> TTM and not limited to HW back-end functions. It means it's much harder >> to spot such places to surgically scope them with drm_dev_enter/exit and >> also that any new such code introduced will immediately break hot unplug >> because the developers can't be expected to remember making their code >> robust to this specific use case. That why when we discussed internally >> what approach to take to protecting code with drm_dev_enter/exit we >> opted for using the widest available scope. > > The thing is, you kinda have to anyway. There's enormous amounts of > asynchronous processing going on. E.g. nonblocking atomic commits also do > ttm unpinning and fun stuff like that, which if you sync things wrong can > happen way late. So the door for bad fallout is wide open :-( > > I'm not sure where the right tradeoff is to make sure we catch them all, > and can make sure with testing that we've indeed caught them all. > -Daniel >
On Fri, May 07, 2021 at 11:39:49AM -0400, Andrey Grodzovsky wrote:
On 2021-05-07 5:11 a.m., Daniel Vetter wrote:
On Thu, May 06, 2021 at 12:25:06PM -0400, Andrey Grodzovsky wrote:
On 2021-05-06 5:40 a.m., Daniel Vetter wrote:
On Fri, Apr 30, 2021 at 01:27:37PM -0400, Andrey Grodzovsky wrote:
On 2021-04-30 6:25 a.m., Daniel Vetter wrote:
On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote: > > > On 2021-04-29 3:05 p.m., Daniel Vetter wrote: > > On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote: > > > > > > > > > On 2021-04-29 7:32 a.m., Daniel Vetter wrote: > > > > On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote: > > > > > On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote: > > > > > > With this calling drm_dev_unplug will flush and block > > > > > > all in flight IOCTLs > > > > > > > > > > > > Also, add feature such that if device supports graceful unplug > > > > > > we enclose entire IOCTL in SRCU critical section. > > > > > > > > > > > > Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com > > > > > > > > > > Nope. > > > > > > > > > > The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl. > > > > > > Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.boo... > > > currently in code ? > > > > I forgot about this one, again. Thanks for reminding. > > > > > > > Especially not with an opt-in flag so that it could be shrugged of as a > > > > > driver hack. Most of these ioctls should have absolutely no problem > > > > > working after hotunplug. > > > > > > > > > > Also, doing this defeats the point since it pretty much guarantees > > > > > userspace will die in assert()s and stuff. E.g. on i915 the rough contract > > > > > is that only execbuf (and even that only when userspace has indicated > > > > > support for non-recoverable hw ctx) is allowed to fail. Anything else > > > > > might crash userspace. > > > > > > Given that as I pointed above we already fail any IOCTls with -ENODEV > > > when device is unplugged, it seems those crashes don't happen that > > > often ? Also, in all my testing I don't think I saw a user space crash > > > I could attribute to this. > > > > I guess it should be ok. > > What should be ok ?
Your approach, but not your patch. If we go with this let's just lift it to drm_ioctl() as the default behavior. No driver opt-in flag, because that's definitely worse than any other approach because we really need to get rid of driver specific behaviour for generic ioctls, especially anything a compositor will use directly.
> > My reasons for making this work is both less trouble for userspace (did > > you test with various wayland compositors out there, not just amdgpu x86 > > I didn't - will give it a try.
Weston worked without crashes, run the egl tester cube there.
> > > driver?), but also testing. > > > > We still need a bunch of these checks in various places or you'll wait a > > very long time for a pending modeset or similar to complete. Being able to > > run that code easily after hotunplug has completed should help a lot with > > testing. > > > > Plus various drivers already acquired drm_dev_enter/exit and now I wonder > > whether that was properly tested or not ... > > > > I guess maybe we need a drm module option to disable this check, so that > > we can exercise the code as if the ioctl has raced with hotunplug at the > > worst possible moment. > > > > Also atomic is really tricky here: I assume your testing has just done > > normal synchronous commits, but anything that goes through atomic can be > > done nonblocking in a separate thread. Which the ioctl catch-all here wont > > capture. > > Yes, async commit was on my mind and thanks for reminding me. Indeed > I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in > drm_dev_enter/exit. Note that i have a bunch of patches, all name's > starting with 'Scope....' that just methodically put all the background > work items and timers the drivers schedules in drm_dev_enter/exit scope. > This was supposed to be part of the 'Scope Display code' patch.
That's too much. You still have to arrange that the flip completion event gets sent out. So it's a bit tricky.
In other places the same problem applies, e.g. probe functions need to make sure they report "disconnected".
I see, well, this is all part of KMS support which I defer for now anyway. Will tackle it then.
> > > > > You probably need similar (and very precisely defined) rules for amdgpu. > > > > > And those must definitely exclude any shard ioctls from randomly failing > > > > > with EIO, because that just kills the box and defeats the point of trying > > > > > to gracefully handling hotunplug and making sure userspace has a chance of > > > > > survival. E.g. for atomic everything should continue, including flip > > > > > completion, but we set all outputs to "disconnected" and send out the > > > > > uevent. Maybe crtc enabling can fail too, but that can also be handled > > > > > through the async status we're using to signal DP link failures to > > > > > userspace. > > > > > > As I pointed before, because of the complexity of the topic I prefer to > > > take it step by step and solve first for secondary device use case, not > > > for primary, display attached device. > > > > Yeah makes sense. But then I think the right patch is to roll this out for > > all drivers, properly justified with existing code. Not behind a driver > > flag, because with all these different compositors the last thing we want > > is a proliferation of driver-specific behaviour. That's imo the worst > > option of all of them and needs to be avoided. > > So this kind of patch would be acceptable to you if I unconditionally > scope the drm_ioctl with drm_dev_enter/exit without the driver flag ? > I am worried to break other drivers with this, see patch https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedes... > Before setting drm_dev_unplug I go through a whole process of signalling > all possible fences in the system which some one some where might be > waiting on. My concern is that in the absence of HW those fences won't > signal and so unless I signal them myself srcu_synchrionize in > drm_dev_unplug will hang waiting for any such code scoped by > drm_dev_enter/exit.
Uh right. I forgot about this.
Which would kinda mean the top level scope is maybe not the best idea, and perhaps we should indeed drill it down. But then the testing issue definitely gets a lot worse.
So what if we'd push that drm_dev_is_unplugged check down into ioctls? Then we can make a case-by case decision whether it should be converted to drm_dev_enter/exit, needs to be pushed down further into drivers (due to fence wait issues) or other concerns?
Also I guess we need to have a subsystem wide rule on whether you need to force complete all fences before you call drm_dev_unplug, or afterwards.
I don't see how you can handle it afterwards. If a thread is stuck in dma_fence_wait in non interruptible wait (any kernel thread) and with no timeout there is nothing you can do to stop the wait. Any such code scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug. The only way then is to preemptively force signal all such fences before calling drm_dev_unplug - as I do in the above mentioned patch.
Yeah, which is why I don't think top-level drm_dev_enter/exit is a good idea.
If we have mixed behaviour on this there will be disappointment. And since hotunplug and dma_fence completion are both userspace visible that inconsistency might have bigger impact.
This is all very tricky indeed :-/
btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go with that: We could do the same trick we've done for DRM_UNLOCKED:
- drm_dev_enter/exit is called for any ioctl that has not set the DRM_HOTUNPLUG_SAFE flag
- for drm core ioctls we push them into all ioctls and decide how to handle/where (with the aim to have the least amount of code flow different during hotunplug vs after hotunplug has finished, to reduce testing scope)
- then we make DRM_HOTUNPLUG_SAFE the implied default
This would have us left with render ioctls, and I think the defensive assumption there is that they're all hotunplug safe. We might hang on a fence wait, but that's fixable, and it's better than blowing up on a use-after-free security bug.
Thoughts?
I don't fully see a difference between the approach described above and the full drill down to each driver and even within the driver, to the HW back-ends - what criteria I would use to decide if for a given IOCTL i scope with drm_dev_enter/exit at the highest level while for another i go all the way down ? If we would agree that signaling the fences preemptively before engaging drm_dev_unplug is generically the right approach maybe we can then scope drm_ioctl unconditionally with drm_dev_enter/exit and then for each driver go through the same process I do for amdgpu - writing driver specific function which takes care of all the fences. We could then just create a drm callback which would be called from drm_ioctl before drm_dev_unplug is called.
So I see the appeal of just nuking all the fences, but I'm not sure that's a good plan. We've done this in the old i915 gpu reset code too, and the issue is it's defacto inverting the locking. But also the hw is truly gone, so it also makes sense.
The problem is a bit roll-out, if we state that dma_fence_wait is allowed with a drm_dev_enter/exit, then all drivers need to force-retire their fences.
The other option would be that we require that dma_fence_wait is _not_ allowed in drm_dev_enter/exit, and that therefore these areas must be marked up more fine-grained to avoid deadlocks. I like this more from the testing aspect (it makes it easier to be reasonable sure your code handles concurrent hotunplug), but also it's pretty easy to validate with the dma_fence lockdep annotations we have I think.
They key question as I see it - is it ok for a device to be unplugged while it's driver has anywhere in it's code a dma_fence_wait waiting for work completion from this device. The answers seems to me is no, the HW is gone, this fence will never signal and so you will be left with indefinitely hanged code thread with all it's unreleased resources. If i am correct in the above statement then avoiding scoping code with drm_dev_enter/exit because a dma_fence_wait might be there in the middle just hides the problem. Also, then the only solution for each driver wanting to support hot-unplug is to force retire all it's HW fences once it's notified of device removal.
At a high level, yes dma_fence must always complete. I don't think we have a disagreement here on that.
What we're discussing here is the precise sequencing and barriers, where things get tricky. Requiring that you force-complete all dma_fence that might be affected before you hotunplug is one solution, the other is tuning the critical sections that drm_dev_enter/exit annotates.
This isn't about avoiding anything or hiding problems, this is about locking/synchronization design. And for that we must agree on what is allowed inside/outside of a critical section for all possible combinations.
E.g. we're also "hiding" problems with calling dma_fence_wait from shrinkers/mmu notifiers by forbidding allocations in dma_fence_begin/end_signalling critical paths.
A third reasons for not requiring force-retiring of dma_fence before drm_dev_unplug is the races: Before drm_dev_unplug you haven't stopped new fences from happening, but until you've stopped new fences it's hard to guarantee they're all retired. How do you solve this currently.
See amdgpu_finilize_device_fences in https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.... I think the steps described there answer your concern here.
The hard problem is stopping further command submission. Not seeing how you solve that.
By stopping GPU SW scheduler before force completion of HW fences, see amdgpu_finilize_device_fences->amdgpu_fence_driver_fini_hw and the comment above it.
But I'm definitely scared about all the scheduler/tdr interactions you already have there, and that looks quite a bit like fallout from doing things the wrong way round.
Also given that drm/scheduler is shared, why can't this be a drm/scheduler helper function?
I was thinking about it, what stopped me is that HW fences signaling is done from driver specific HW fence array. But we could do it generic by instead iterating sched->pending_list and signaling s_job->s_fence->parent instead. You also need to retire scheduler's scheduled fences once you stopped the schedulers as they are waited on as dependencies for other jobs submissions (i do take care of it).
Finally there's still hangcheck and all that, so if we go with forbidding dma_fence_wait from within drm_dev_enter/exit sections, then drivers don't need to have additional tricky code to force-retire fences. TDR will take care already (albeit with maybe a slightly annoying long timeout, which we can shorten to "time out everything immediately" after drm_dev_unplug).
I am not aware of TDR handlers that do it today, at least we don't, we don't check that if device is gone let's instead of resetting the device and resubmit jobs just force retire all the HW fences. In any case, this can and i think should be done in pci remove callback because this is the place that supposed to handle device extraction. I for example in amdgpu_finilize_device_fences just block all TDRs from taking place as first step in the process. If other drivers want to force retire fences in their TDR handlers they still need to block and wait for all such TDRs in their pci_remove handler.
TDR definitely force-completes the fence that did hang. Of course it'll take a while until they've all completed this way, but we do have guaranteed forward progress since we've stopped all further fences from showing up because drm_dev_unplug is called already.
And yes after drm_dev_unplug you can then force-retire the tdr stuff.
What we definitely can't have is half the drivers doing it one way, and the other half the other way. So your driver flag to wrap the ioctl optionally in a drm_dev_enter/exit path is a no-go still I think.
I guess my tldr; is: I definitely see how your current approach gives quicker results for amdgpu right now, but long term I'm seeing more positives on the other one. At least I expect less special cases due to hotunplug with that.
As i expressed my viewpoint above - seems to me any driver in need to support hot-unplug must force retire it's fences because of need to unblock all dma_fence waits and so it will not be a special case.
This isn't the special case I meant. It's the very tricky force-retire-before-you-unplugged-officially which is large scale nasty.
Also if your driver doesn't force-retire already, it's buggy. The additional need of hotunplug is just that we're trying to force-retire a bit faster, because we know it's all hopeless. But e.g. i915 already has a fallback that does this automatically:
- first we reset only the engine/context, keeping everyone else running
- if that doesn't pan out, we reset the entire chip and give up an anything that's in-flight, which (iirc, it did so at least in the past) force retires everything outstanding.
I think amdgpu only has full chip reset, so your first step tries to reissue all other tasks. But that's not necessarily how it needs to happen.
Either way drivers must force retire everything (albeit maybe a bit at a slow pace) if the hw ceased to work properly already. Hotunplug really isn't anything new here. -Daniel
Let's then agree on the way forward -
You raised before the following suggestion -
" btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go with that: We could do the same trick we've done for DRM_UNLOCKED:
- drm_dev_enter/exit is called for any ioctl that has not set the DRM_HOTUNPLUG_SAFE flag
- for drm core ioctls we push them into all ioctls and decide how to handle/where (with the aim to have the least amount of code flow different during hotunplug vs after hotunplug has finished, to reduce testing scope)
- then we make DRM_HOTUNPLUG_SAFE the implied default
" My problem here is that I have no good understating, criteria for how to decide per each ioctl on the right scope of drm_dev_enter/ exit. It depends on whether each next function call can lead somewhere down the call stack to dma_fence_wait and/or whether it can lead to registers access. Seems to me very hard to cover and error prone.
Tbh, neither do I. This requires a lot of work to analyze.
Another options which we discussed internally before and is basically same as current drivers i guess is simply to scope with drm_dev_enter/ exit all the back-end HW specific callbacks. Those are most of the places MMIO access takes place and by definition no dma_fence_wait can be there as it's HW specific code. This leaves MMIO access through pointers (memcpy, and various pointer de-references) which will need to be protected on case by case, but given that I unmap all MMIO anyway as last step of PCI remove callback, all of them will be found by try and error eventually. I feel more comfortable with this approach as I have a clear understating of how to deal with it.
Hm ... I'm maybe failing to see the difference, but at least on the kms side "put drm_dev_enter/exit into driver callbacks" feels like the right place for them.
Render (and I guess kfd for amd as the special case) ioctl are different, especially all the driver specific ones.
So one thing that cross my mind maybe as step 0 is to annotate the rules for drm_dev_enter/exit using lockdep. With lockdep we can both check whether a lock is held, but also whether it's not held (but the latter is only possible with CONFIG_PROVE_LOCKING enabled). I think it would be good to annotate all the major locks in the kernel against drm_dev_enter/exit:
- dma_fence_wait could check that the drm_dev_enter/exit srcu is _not_ held.
- because srcu nest _very_ freely there's kinda no real restrictions for putting drm_dev_enter/exit within a lock critical section. Might still be good to explicitly call out in docs where it's all ok:
- interrupt handlers (I hope that's the case, otherwise I screwed up) - shrinkers/mmu_notifier callbacks
- anything else that's not allowed in within drm_dev_enter/exit. I'm e.g. wondering whether we should disallow drm_modeset_lock() or maybe dma_resv_lock(), or whether that's too restrictive. It could help quite a bit in finding places where the drm_dev_enter/exit section is too wide.
- another one is the inverse, but I guess you have that already with putting a drm_dev_is_held() into mmio helpers and all that to make sure we really have them all caught.
Above is just examples, I think the more we nail down these rules one way or the other, the better for consistency across drivers. And without consistency everyone will be forced to write their own mmap helpers instead of one in ttm, or scheduler cleanup helpers instead of one in drm/scheduler.
P.S Please respond on the question for you on the other thread at 'PATCH v5 15/27] drm/scheduler: Fix hang when sched_entity released' about suggestion by Christian of partial up-streaming of this code up to and before the patches dealing with scoping of drm_dev_enter/exit scoping.
Ok will try, I'm a bit burried unfortunately so thanks for reminder when I miss something. -Daniel
Andrey
Andrey
Cheers, Daniel
Andrey
It is unfortunately even more work until we've reached the goal, but I think it's safest and most flexible approach overall.
Cheers, Daniel
> > Andrey > > > > > Cheers, Daniel > > > > > > > > > > > > > > > > > I guess we should clarify this in the hotunplug doc? > > > > > > Agree > > > > > > > > > > > To clarify: I'm not against throwing an ENODEV at userspace for ioctl that > > > > really make no sense, and where we're rather confident that all properly > > > > implemented userspace will gracefully handle failures. Like a modeset, or > > > > opening a device, or trying to import a dma-buf or stuff like that which > > > > can already fail in normal operation for any kind of reason. > > > > > > > > But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail > > > > after hotunplug. > > > > > > As I pointed above, this a bit confuses me given that we already do > > > blanker rejection of IOCTLs if device is unplugged. > > > > Well I'm confused about this too :-/ > > > > > > And then there's the middle ground, like doing a pageflip or buffer flush, > > > > which I guess some userspace might handle, but risky to inflict those > > > > consequences on them. atomic modeset is especially fun since depending > > > > what you're doing it can be both "failures expected" and "failures not > > > > really expected in normal operation". > > > > > > > > Also, this really should be consistent across drivers, not solved with a > > > > driver flag for every possible combination. > > > > > > > > If you look at the current hotunplug kms drivers, they have > > > > drm_dev_enter/exit sprinkled in specific hw callback functions because of > > > > the above problems. But maybe it makes sense to change things in a few > > > > cases. But then we should do it across the board. > > > > > > So as I understand your preferred approach is that I scope any back_end, HW > > > specific function with drm_dev_enter/exit because that where MMIO > > > access takes place. But besides explicit MMIO access thorough > > > register accessors in the HW back-end there is also indirect MMIO access > > > taking place throughout the code in the driver because of various VRAM > > > BOs which provide CPU access to VRAM through the VRAM BAR. This kind of > > > access is spread all over in the driver and even in mid-layers such as > > > TTM and not limited to HW back-end functions. It means it's much harder > > > to spot such places to surgically scope them with drm_dev_enter/exit and > > > also that any new such code introduced will immediately break hot unplug > > > because the developers can't be expected to remember making their code > > > robust to this specific use case. That why when we discussed internally > > > what approach to take to protecting code with drm_dev_enter/exit we > > > opted for using the widest available scope. > > > > The thing is, you kinda have to anyway. There's enormous amounts of > > asynchronous processing going on. E.g. nonblocking atomic commits also do > > ttm unpinning and fun stuff like that, which if you sync things wrong can > > happen way late. So the door for bad fallout is wide open :-( > > > > I'm not sure where the right tradeoff is to make sure we catch them all, > > and can make sure with testing that we've indeed caught them all. > > -Daniel > >
On 2021-05-07 12:24 p.m., Daniel Vetter wrote:
On Fri, May 07, 2021 at 11:39:49AM -0400, Andrey Grodzovsky wrote:
On 2021-05-07 5:11 a.m., Daniel Vetter wrote:
On Thu, May 06, 2021 at 12:25:06PM -0400, Andrey Grodzovsky wrote:
On 2021-05-06 5:40 a.m., Daniel Vetter wrote:
On Fri, Apr 30, 2021 at 01:27:37PM -0400, Andrey Grodzovsky wrote:
On 2021-04-30 6:25 a.m., Daniel Vetter wrote: > On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote: >> >> >> On 2021-04-29 3:05 p.m., Daniel Vetter wrote: >>> On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote: >>>> >>>> >>>> On 2021-04-29 7:32 a.m., Daniel Vetter wrote: >>>>> On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote: >>>>>> On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote: >>>>>>> With this calling drm_dev_unplug will flush and block >>>>>>> all in flight IOCTLs >>>>>>> >>>>>>> Also, add feature such that if device supports graceful unplug >>>>>>> we enclose entire IOCTL in SRCU critical section. >>>>>>> >>>>>>> Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com >>>>>> >>>>>> Nope. >>>>>> >>>>>> The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl. >>>> >>>> Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.boo... >>>> currently in code ? >>> >>> I forgot about this one, again. Thanks for reminding. >>> >>>>>> Especially not with an opt-in flag so that it could be shrugged of as a >>>>>> driver hack. Most of these ioctls should have absolutely no problem >>>>>> working after hotunplug. >>>>>> >>>>>> Also, doing this defeats the point since it pretty much guarantees >>>>>> userspace will die in assert()s and stuff. E.g. on i915 the rough contract >>>>>> is that only execbuf (and even that only when userspace has indicated >>>>>> support for non-recoverable hw ctx) is allowed to fail. Anything else >>>>>> might crash userspace. >>>> >>>> Given that as I pointed above we already fail any IOCTls with -ENODEV >>>> when device is unplugged, it seems those crashes don't happen that >>>> often ? Also, in all my testing I don't think I saw a user space crash >>>> I could attribute to this. >>> >>> I guess it should be ok. >> >> What should be ok ? > > Your approach, but not your patch. If we go with this let's just lift it > to drm_ioctl() as the default behavior. No driver opt-in flag, because > that's definitely worse than any other approach because we really need to > get rid of driver specific behaviour for generic ioctls, especially > anything a compositor will use directly. > >>> My reasons for making this work is both less trouble for userspace (did >>> you test with various wayland compositors out there, not just amdgpu x86 >> >> I didn't - will give it a try.
Weston worked without crashes, run the egl tester cube there.
>> >>> driver?), but also testing. >>> >>> We still need a bunch of these checks in various places or you'll wait a >>> very long time for a pending modeset or similar to complete. Being able to >>> run that code easily after hotunplug has completed should help a lot with >>> testing. >>> >>> Plus various drivers already acquired drm_dev_enter/exit and now I wonder >>> whether that was properly tested or not ... >>> >>> I guess maybe we need a drm module option to disable this check, so that >>> we can exercise the code as if the ioctl has raced with hotunplug at the >>> worst possible moment. >>> >>> Also atomic is really tricky here: I assume your testing has just done >>> normal synchronous commits, but anything that goes through atomic can be >>> done nonblocking in a separate thread. Which the ioctl catch-all here wont >>> capture. >> >> Yes, async commit was on my mind and thanks for reminding me. Indeed >> I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in >> drm_dev_enter/exit. Note that i have a bunch of patches, all name's >> starting with 'Scope....' that just methodically put all the background >> work items and timers the drivers schedules in drm_dev_enter/exit scope. >> This was supposed to be part of the 'Scope Display code' patch. > > That's too much. You still have to arrange that the flip completion event > gets sent out. So it's a bit tricky. > > In other places the same problem applies, e.g. probe functions need to > make sure they report "disconnected".
I see, well, this is all part of KMS support which I defer for now anyway. Will tackle it then.
> >>>>>> You probably need similar (and very precisely defined) rules for amdgpu. >>>>>> And those must definitely exclude any shard ioctls from randomly failing >>>>>> with EIO, because that just kills the box and defeats the point of trying >>>>>> to gracefully handling hotunplug and making sure userspace has a chance of >>>>>> survival. E.g. for atomic everything should continue, including flip >>>>>> completion, but we set all outputs to "disconnected" and send out the >>>>>> uevent. Maybe crtc enabling can fail too, but that can also be handled >>>>>> through the async status we're using to signal DP link failures to >>>>>> userspace. >>>> >>>> As I pointed before, because of the complexity of the topic I prefer to >>>> take it step by step and solve first for secondary device use case, not >>>> for primary, display attached device. >>> >>> Yeah makes sense. But then I think the right patch is to roll this out for >>> all drivers, properly justified with existing code. Not behind a driver >>> flag, because with all these different compositors the last thing we want >>> is a proliferation of driver-specific behaviour. That's imo the worst >>> option of all of them and needs to be avoided. >> >> So this kind of patch would be acceptable to you if I unconditionally >> scope the drm_ioctl with drm_dev_enter/exit without the driver flag ? >> I am worried to break other drivers with this, see patch https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedes... >> Before setting drm_dev_unplug I go through a whole process of signalling >> all possible fences in the system which some one some where might be >> waiting on. My concern is that in the absence of HW those fences won't >> signal and so unless I signal them myself srcu_synchrionize in >> drm_dev_unplug will hang waiting for any such code scoped by >> drm_dev_enter/exit. > > Uh right. I forgot about this. > > Which would kinda mean the top level scope is maybe not the best idea, and > perhaps we should indeed drill it down. But then the testing issue > definitely gets a lot worse. > > So what if we'd push that drm_dev_is_unplugged check down into ioctls? > Then we can make a case-by case decision whether it should be converted to > drm_dev_enter/exit, needs to be pushed down further into drivers (due to > fence wait issues) or other concerns? > > Also I guess we need to have a subsystem wide rule on whether you need to > force complete all fences before you call drm_dev_unplug, or afterwards.
I don't see how you can handle it afterwards. If a thread is stuck in dma_fence_wait in non interruptible wait (any kernel thread) and with no timeout there is nothing you can do to stop the wait. Any such code scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug. The only way then is to preemptively force signal all such fences before calling drm_dev_unplug - as I do in the above mentioned patch.
Yeah, which is why I don't think top-level drm_dev_enter/exit is a good idea.
> If we have mixed behaviour on this there will be disappointment. And since > hotunplug and dma_fence completion are both userspace visible that > inconsistency might have bigger impact. > > This is all very tricky indeed :-/ > > btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go > with that: We could do the same trick we've done for DRM_UNLOCKED: > - drm_dev_enter/exit is called for any ioctl that has not set the > DRM_HOTUNPLUG_SAFE flag > - for drm core ioctls we push them into all ioctls and decide how to > handle/where (with the aim to have the least amount of code flow > different during hotunplug vs after hotunplug has finished, to reduce > testing scope) > - then we make DRM_HOTUNPLUG_SAFE the implied default > > This would have us left with render ioctls, and I think the defensive > assumption there is that they're all hotunplug safe. We might hang on a > fence wait, but that's fixable, and it's better than blowing up on a > use-after-free security bug. > > Thoughts?
I don't fully see a difference between the approach described above and the full drill down to each driver and even within the driver, to the HW back-ends - what criteria I would use to decide if for a given IOCTL i scope with drm_dev_enter/exit at the highest level while for another i go all the way down ? If we would agree that signaling the fences preemptively before engaging drm_dev_unplug is generically the right approach maybe we can then scope drm_ioctl unconditionally with drm_dev_enter/exit and then for each driver go through the same process I do for amdgpu - writing driver specific function which takes care of all the fences. We could then just create a drm callback which would be called from drm_ioctl before drm_dev_unplug is called.
So I see the appeal of just nuking all the fences, but I'm not sure that's a good plan. We've done this in the old i915 gpu reset code too, and the issue is it's defacto inverting the locking. But also the hw is truly gone, so it also makes sense.
The problem is a bit roll-out, if we state that dma_fence_wait is allowed with a drm_dev_enter/exit, then all drivers need to force-retire their fences.
The other option would be that we require that dma_fence_wait is _not_ allowed in drm_dev_enter/exit, and that therefore these areas must be marked up more fine-grained to avoid deadlocks. I like this more from the testing aspect (it makes it easier to be reasonable sure your code handles concurrent hotunplug), but also it's pretty easy to validate with the dma_fence lockdep annotations we have I think.
They key question as I see it - is it ok for a device to be unplugged while it's driver has anywhere in it's code a dma_fence_wait waiting for work completion from this device. The answers seems to me is no, the HW is gone, this fence will never signal and so you will be left with indefinitely hanged code thread with all it's unreleased resources. If i am correct in the above statement then avoiding scoping code with drm_dev_enter/exit because a dma_fence_wait might be there in the middle just hides the problem. Also, then the only solution for each driver wanting to support hot-unplug is to force retire all it's HW fences once it's notified of device removal.
At a high level, yes dma_fence must always complete. I don't think we have a disagreement here on that.
What we're discussing here is the precise sequencing and barriers, where things get tricky. Requiring that you force-complete all dma_fence that might be affected before you hotunplug is one solution, the other is tuning the critical sections that drm_dev_enter/exit annotates.
This isn't about avoiding anything or hiding problems, this is about locking/synchronization design. And for that we must agree on what is allowed inside/outside of a critical section for all possible combinations.
E.g. we're also "hiding" problems with calling dma_fence_wait from shrinkers/mmu notifiers by forbidding allocations in dma_fence_begin/end_signalling critical paths.
A third reasons for not requiring force-retiring of dma_fence before drm_dev_unplug is the races: Before drm_dev_unplug you haven't stopped new fences from happening, but until you've stopped new fences it's hard to guarantee they're all retired. How do you solve this currently.
See amdgpu_finilize_device_fences in https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.... I think the steps described there answer your concern here.
The hard problem is stopping further command submission. Not seeing how you solve that.
By stopping GPU SW scheduler before force completion of HW fences, see amdgpu_finilize_device_fences->amdgpu_fence_driver_fini_hw and the comment above it.
But I'm definitely scared about all the scheduler/tdr interactions you already have there, and that looks quite a bit like fallout from doing things the wrong way round.
Also given that drm/scheduler is shared, why can't this be a drm/scheduler helper function?
I was thinking about it, what stopped me is that HW fences signaling is done from driver specific HW fence array. But we could do it generic by instead iterating sched->pending_list and signaling s_job->s_fence->parent instead. You also need to retire scheduler's scheduled fences once you stopped the schedulers as they are waited on as dependencies for other jobs submissions (i do take care of it).
Finally there's still hangcheck and all that, so if we go with forbidding dma_fence_wait from within drm_dev_enter/exit sections, then drivers don't need to have additional tricky code to force-retire fences. TDR will take care already (albeit with maybe a slightly annoying long timeout, which we can shorten to "time out everything immediately" after drm_dev_unplug).
I am not aware of TDR handlers that do it today, at least we don't, we don't check that if device is gone let's instead of resetting the device and resubmit jobs just force retire all the HW fences. In any case, this can and i think should be done in pci remove callback because this is the place that supposed to handle device extraction. I for example in amdgpu_finilize_device_fences just block all TDRs from taking place as first step in the process. If other drivers want to force retire fences in their TDR handlers they still need to block and wait for all such TDRs in their pci_remove handler.
TDR definitely force-completes the fence that did hang. Of course it'll take a while until they've all completed this way, but we do have guaranteed forward progress since we've stopped all further fences from showing up because drm_dev_unplug is called already.
And yes after drm_dev_unplug you can then force-retire the tdr stuff.
What we definitely can't have is half the drivers doing it one way, and the other half the other way. So your driver flag to wrap the ioctl optionally in a drm_dev_enter/exit path is a no-go still I think.
I guess my tldr; is: I definitely see how your current approach gives quicker results for amdgpu right now, but long term I'm seeing more positives on the other one. At least I expect less special cases due to hotunplug with that.
As i expressed my viewpoint above - seems to me any driver in need to support hot-unplug must force retire it's fences because of need to unblock all dma_fence waits and so it will not be a special case.
This isn't the special case I meant. It's the very tricky force-retire-before-you-unplugged-officially which is large scale nasty.
Also if your driver doesn't force-retire already, it's buggy. The additional need of hotunplug is just that we're trying to force-retire a bit faster, because we know it's all hopeless. But e.g. i915 already has a fallback that does this automatically:
- first we reset only the engine/context, keeping everyone else running
- if that doesn't pan out, we reset the entire chip and give up an anything that's in-flight, which (iirc, it did so at least in the past) force retires everything outstanding.
I think amdgpu only has full chip reset, so your first step tries to reissue all other tasks. But that's not necessarily how it needs to happen.
Either way drivers must force retire everything (albeit maybe a bit at a slow pace) if the hw ceased to work properly already. Hotunplug really isn't anything new here. -Daniel
Let's then agree on the way forward -
You raised before the following suggestion -
" btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go with that: We could do the same trick we've done for DRM_UNLOCKED:
- drm_dev_enter/exit is called for any ioctl that has not set the DRM_HOTUNPLUG_SAFE flag
- for drm core ioctls we push them into all ioctls and decide how to handle/where (with the aim to have the least amount of code flow different during hotunplug vs after hotunplug has finished, to reduce testing scope)
- then we make DRM_HOTUNPLUG_SAFE the implied default
" My problem here is that I have no good understating, criteria for how to decide per each ioctl on the right scope of drm_dev_enter/ exit. It depends on whether each next function call can lead somewhere down the call stack to dma_fence_wait and/or whether it can lead to registers access. Seems to me very hard to cover and error prone.
Tbh, neither do I. This requires a lot of work to analyze.
Another options which we discussed internally before and is basically same as current drivers i guess is simply to scope with drm_dev_enter/ exit all the back-end HW specific callbacks. Those are most of the places MMIO access takes place and by definition no dma_fence_wait can be there as it's HW specific code. This leaves MMIO access through pointers (memcpy, and various pointer de-references) which will need to be protected on case by case, but given that I unmap all MMIO anyway as last step of PCI remove callback, all of them will be found by try and error eventually. I feel more comfortable with this approach as I have a clear understating of how to deal with it.
Hm ... I'm maybe failing to see the difference, but at least on the kms side "put drm_dev_enter/exit into driver callbacks" feels like the right place for them.
Render (and I guess kfd for amd as the special case) ioctl are different, especially all the driver specific ones.
So one thing that cross my mind maybe as step 0 is to annotate the rules for drm_dev_enter/exit using lockdep. With lockdep we can both check whether a lock is held, but also whether it's not held (but the latter is only possible with CONFIG_PROVE_LOCKING enabled). I think it would be good to annotate all the major locks in the kernel against drm_dev_enter/exit:
- dma_fence_wait could check that the drm_dev_enter/exit srcu is _not_ held.
We can't insert this directly inside dma_fence_wait as a DRM agnostic layer, are you porpoising to wrap all dma_fence_waits in DRM and drivers code with thin wrapper calling lockdep_assert_held(drm_unplug_srcu) ? What about dma_resv objs and dma_resv_wait_timeout_rcu style of waits ?
because srcu nest _very_ freely there's kinda no real restrictions for putting drm_dev_enter/exit within a lock critical section. Might still be good to explicitly call out in docs where it's all ok:
- interrupt handlers (I hope that's the case, otherwise I screwed up)
- shrinkers/mmu_notifier callbacks
Is there a problem scooping these with drm_dev_enter/exit ? Might they hang indefinitely once a device is gone ?
anything else that's not allowed in within drm_dev_enter/exit. I'm e.g. wondering whether we should disallow drm_modeset_lock() or maybe dma_resv_lock(), or whether that's too restrictive. It could help quite a bit in finding places where the drm_dev_enter/exit section is too wide.
another one is the inverse, but I guess you have that already with putting a drm_dev_is_held() into mmio helpers and all that to make sure we really have them all caught.
Not sure what drm_dev_is_held means here ? Also what do you mean by MMIO helpers ? Maybe you meant the page fault helpers like ttm_bo_vm_fault ?
Andrey
Above is just examples, I think the more we nail down these rules one way or the other, the better for consistency across drivers. And without consistency everyone will be forced to write their own mmap helpers instead of one in ttm, or scheduler cleanup helpers instead of one in drm/scheduler.
P.S Please respond on the question for you on the other thread at 'PATCH v5 15/27] drm/scheduler: Fix hang when sched_entity released' about suggestion by Christian of partial up-streaming of this code up to and before the patches dealing with scoping of drm_dev_enter/exit scoping.
Ok will try, I'm a bit burried unfortunately so thanks for reminder when I miss something. -Daniel
Andrey
Andrey
Cheers, Daniel
Andrey
> > It is unfortunately even more work until we've reached the goal, but I > think it's safest and most flexible approach overall. > > Cheers, Daniel > >> >> Andrey >> >>> >>> Cheers, Daniel >>> >>> >>>> >>>>>> >>>>>> I guess we should clarify this in the hotunplug doc? >>>> >>>> Agree >>>> >>>>> >>>>> To clarify: I'm not against throwing an ENODEV at userspace for ioctl that >>>>> really make no sense, and where we're rather confident that all properly >>>>> implemented userspace will gracefully handle failures. Like a modeset, or >>>>> opening a device, or trying to import a dma-buf or stuff like that which >>>>> can already fail in normal operation for any kind of reason. >>>>> >>>>> But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail >>>>> after hotunplug. >>>> >>>> As I pointed above, this a bit confuses me given that we already do >>>> blanker rejection of IOCTLs if device is unplugged. >>> >>> Well I'm confused about this too :-/ >>> >>>>> And then there's the middle ground, like doing a pageflip or buffer flush, >>>>> which I guess some userspace might handle, but risky to inflict those >>>>> consequences on them. atomic modeset is especially fun since depending >>>>> what you're doing it can be both "failures expected" and "failures not >>>>> really expected in normal operation". >>>>> >>>>> Also, this really should be consistent across drivers, not solved with a >>>>> driver flag for every possible combination. >>>>> >>>>> If you look at the current hotunplug kms drivers, they have >>>>> drm_dev_enter/exit sprinkled in specific hw callback functions because of >>>>> the above problems. But maybe it makes sense to change things in a few >>>>> cases. But then we should do it across the board. >>>> >>>> So as I understand your preferred approach is that I scope any back_end, HW >>>> specific function with drm_dev_enter/exit because that where MMIO >>>> access takes place. But besides explicit MMIO access thorough >>>> register accessors in the HW back-end there is also indirect MMIO access >>>> taking place throughout the code in the driver because of various VRAM >>>> BOs which provide CPU access to VRAM through the VRAM BAR. This kind of >>>> access is spread all over in the driver and even in mid-layers such as >>>> TTM and not limited to HW back-end functions. It means it's much harder >>>> to spot such places to surgically scope them with drm_dev_enter/exit and >>>> also that any new such code introduced will immediately break hot unplug >>>> because the developers can't be expected to remember making their code >>>> robust to this specific use case. That why when we discussed internally >>>> what approach to take to protecting code with drm_dev_enter/exit we >>>> opted for using the widest available scope. >>> >>> The thing is, you kinda have to anyway. There's enormous amounts of >>> asynchronous processing going on. E.g. nonblocking atomic commits also do >>> ttm unpinning and fun stuff like that, which if you sync things wrong can >>> happen way late. So the door for bad fallout is wide open :-( >>> >>> I'm not sure where the right tradeoff is to make sure we catch them all, >>> and can make sure with testing that we've indeed caught them all. >>> -Daniel >>> >
On Fri, May 07, 2021 at 02:00:14PM -0400, Andrey Grodzovsky wrote:
On 2021-05-07 12:24 p.m., Daniel Vetter wrote:
On Fri, May 07, 2021 at 11:39:49AM -0400, Andrey Grodzovsky wrote:
On 2021-05-07 5:11 a.m., Daniel Vetter wrote:
On Thu, May 06, 2021 at 12:25:06PM -0400, Andrey Grodzovsky wrote:
On 2021-05-06 5:40 a.m., Daniel Vetter wrote:
On Fri, Apr 30, 2021 at 01:27:37PM -0400, Andrey Grodzovsky wrote: > > > On 2021-04-30 6:25 a.m., Daniel Vetter wrote: > > On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote: > > > > > > > > > On 2021-04-29 3:05 p.m., Daniel Vetter wrote: > > > > On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote: > > > > > > > > > > > > > > > On 2021-04-29 7:32 a.m., Daniel Vetter wrote: > > > > > > On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote: > > > > > > > On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote: > > > > > > > > With this calling drm_dev_unplug will flush and block > > > > > > > > all in flight IOCTLs > > > > > > > > > > > > > > > > Also, add feature such that if device supports graceful unplug > > > > > > > > we enclose entire IOCTL in SRCU critical section. > > > > > > > > > > > > > > > > Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com > > > > > > > > > > > > > > Nope. > > > > > > > > > > > > > > The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl. > > > > > > > > > > Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.boo... > > > > > currently in code ? > > > > > > > > I forgot about this one, again. Thanks for reminding. > > > > > > > > > > > Especially not with an opt-in flag so that it could be shrugged of as a > > > > > > > driver hack. Most of these ioctls should have absolutely no problem > > > > > > > working after hotunplug. > > > > > > > > > > > > > > Also, doing this defeats the point since it pretty much guarantees > > > > > > > userspace will die in assert()s and stuff. E.g. on i915 the rough contract > > > > > > > is that only execbuf (and even that only when userspace has indicated > > > > > > > support for non-recoverable hw ctx) is allowed to fail. Anything else > > > > > > > might crash userspace. > > > > > > > > > > Given that as I pointed above we already fail any IOCTls with -ENODEV > > > > > when device is unplugged, it seems those crashes don't happen that > > > > > often ? Also, in all my testing I don't think I saw a user space crash > > > > > I could attribute to this. > > > > > > > > I guess it should be ok. > > > > > > What should be ok ? > > > > Your approach, but not your patch. If we go with this let's just lift it > > to drm_ioctl() as the default behavior. No driver opt-in flag, because > > that's definitely worse than any other approach because we really need to > > get rid of driver specific behaviour for generic ioctls, especially > > anything a compositor will use directly. > > > > > > My reasons for making this work is both less trouble for userspace (did > > > > you test with various wayland compositors out there, not just amdgpu x86 > > > > > > I didn't - will give it a try. > > Weston worked without crashes, run the egl tester cube there. > > > > > > > > driver?), but also testing. > > > > > > > > We still need a bunch of these checks in various places or you'll wait a > > > > very long time for a pending modeset or similar to complete. Being able to > > > > run that code easily after hotunplug has completed should help a lot with > > > > testing. > > > > > > > > Plus various drivers already acquired drm_dev_enter/exit and now I wonder > > > > whether that was properly tested or not ... > > > > > > > > I guess maybe we need a drm module option to disable this check, so that > > > > we can exercise the code as if the ioctl has raced with hotunplug at the > > > > worst possible moment. > > > > > > > > Also atomic is really tricky here: I assume your testing has just done > > > > normal synchronous commits, but anything that goes through atomic can be > > > > done nonblocking in a separate thread. Which the ioctl catch-all here wont > > > > capture. > > > > > > Yes, async commit was on my mind and thanks for reminding me. Indeed > > > I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in > > > drm_dev_enter/exit. Note that i have a bunch of patches, all name's > > > starting with 'Scope....' that just methodically put all the background > > > work items and timers the drivers schedules in drm_dev_enter/exit scope. > > > This was supposed to be part of the 'Scope Display code' patch. > > > > That's too much. You still have to arrange that the flip completion event > > gets sent out. So it's a bit tricky. > > > > In other places the same problem applies, e.g. probe functions need to > > make sure they report "disconnected". > > I see, well, this is all part of KMS support which I defer for now > anyway. Will tackle it then. > > > > > > > > > > You probably need similar (and very precisely defined) rules for amdgpu. > > > > > > > And those must definitely exclude any shard ioctls from randomly failing > > > > > > > with EIO, because that just kills the box and defeats the point of trying > > > > > > > to gracefully handling hotunplug and making sure userspace has a chance of > > > > > > > survival. E.g. for atomic everything should continue, including flip > > > > > > > completion, but we set all outputs to "disconnected" and send out the > > > > > > > uevent. Maybe crtc enabling can fail too, but that can also be handled > > > > > > > through the async status we're using to signal DP link failures to > > > > > > > userspace. > > > > > > > > > > As I pointed before, because of the complexity of the topic I prefer to > > > > > take it step by step and solve first for secondary device use case, not > > > > > for primary, display attached device. > > > > > > > > Yeah makes sense. But then I think the right patch is to roll this out for > > > > all drivers, properly justified with existing code. Not behind a driver > > > > flag, because with all these different compositors the last thing we want > > > > is a proliferation of driver-specific behaviour. That's imo the worst > > > > option of all of them and needs to be avoided. > > > > > > So this kind of patch would be acceptable to you if I unconditionally > > > scope the drm_ioctl with drm_dev_enter/exit without the driver flag ? > > > I am worried to break other drivers with this, see patch https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedes... > > > Before setting drm_dev_unplug I go through a whole process of signalling > > > all possible fences in the system which some one some where might be > > > waiting on. My concern is that in the absence of HW those fences won't > > > signal and so unless I signal them myself srcu_synchrionize in > > > drm_dev_unplug will hang waiting for any such code scoped by > > > drm_dev_enter/exit. > > > > Uh right. I forgot about this. > > > > Which would kinda mean the top level scope is maybe not the best idea, and > > perhaps we should indeed drill it down. But then the testing issue > > definitely gets a lot worse. > > > > So what if we'd push that drm_dev_is_unplugged check down into ioctls? > > Then we can make a case-by case decision whether it should be converted to > > drm_dev_enter/exit, needs to be pushed down further into drivers (due to > > fence wait issues) or other concerns? > > > > Also I guess we need to have a subsystem wide rule on whether you need to > > force complete all fences before you call drm_dev_unplug, or afterwards. > > I don't see how you can handle it afterwards. If a thread is stuck in > dma_fence_wait in non interruptible wait (any kernel thread) and with no > timeout there is nothing you can do to stop the wait. Any such code > scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug. > The only way then is to preemptively force signal all such fences before > calling drm_dev_unplug - as I do in the above mentioned patch.
Yeah, which is why I don't think top-level drm_dev_enter/exit is a good idea.
> > If we have mixed behaviour on this there will be disappointment. And since > > hotunplug and dma_fence completion are both userspace visible that > > inconsistency might have bigger impact. > > > > This is all very tricky indeed :-/ > > > > btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go > > with that: We could do the same trick we've done for DRM_UNLOCKED: > > - drm_dev_enter/exit is called for any ioctl that has not set the > > DRM_HOTUNPLUG_SAFE flag > > - for drm core ioctls we push them into all ioctls and decide how to > > handle/where (with the aim to have the least amount of code flow > > different during hotunplug vs after hotunplug has finished, to reduce > > testing scope) > > - then we make DRM_HOTUNPLUG_SAFE the implied default > > > > This would have us left with render ioctls, and I think the defensive > > assumption there is that they're all hotunplug safe. We might hang on a > > fence wait, but that's fixable, and it's better than blowing up on a > > use-after-free security bug. > > > > Thoughts? > > I don't fully see a difference between the approach described above and > the full drill down to each driver and even within the driver, to the HW > back-ends - what criteria I would use to decide if for a given IOCTL i > scope with drm_dev_enter/exit at the highest level while for another > i go all the way down ? If we would agree that signaling the fences > preemptively before engaging drm_dev_unplug is generically the right > approach maybe we can then scope drm_ioctl unconditionally with > drm_dev_enter/exit and then for each driver go through the same process > I do for amdgpu - writing driver specific function which takes care of > all the fences. We could then just create a drm callback which would > be called from drm_ioctl before drm_dev_unplug is called.
So I see the appeal of just nuking all the fences, but I'm not sure that's a good plan. We've done this in the old i915 gpu reset code too, and the issue is it's defacto inverting the locking. But also the hw is truly gone, so it also makes sense.
The problem is a bit roll-out, if we state that dma_fence_wait is allowed with a drm_dev_enter/exit, then all drivers need to force-retire their fences.
The other option would be that we require that dma_fence_wait is _not_ allowed in drm_dev_enter/exit, and that therefore these areas must be marked up more fine-grained to avoid deadlocks. I like this more from the testing aspect (it makes it easier to be reasonable sure your code handles concurrent hotunplug), but also it's pretty easy to validate with the dma_fence lockdep annotations we have I think.
They key question as I see it - is it ok for a device to be unplugged while it's driver has anywhere in it's code a dma_fence_wait waiting for work completion from this device. The answers seems to me is no, the HW is gone, this fence will never signal and so you will be left with indefinitely hanged code thread with all it's unreleased resources. If i am correct in the above statement then avoiding scoping code with drm_dev_enter/exit because a dma_fence_wait might be there in the middle just hides the problem. Also, then the only solution for each driver wanting to support hot-unplug is to force retire all it's HW fences once it's notified of device removal.
At a high level, yes dma_fence must always complete. I don't think we have a disagreement here on that.
What we're discussing here is the precise sequencing and barriers, where things get tricky. Requiring that you force-complete all dma_fence that might be affected before you hotunplug is one solution, the other is tuning the critical sections that drm_dev_enter/exit annotates.
This isn't about avoiding anything or hiding problems, this is about locking/synchronization design. And for that we must agree on what is allowed inside/outside of a critical section for all possible combinations.
E.g. we're also "hiding" problems with calling dma_fence_wait from shrinkers/mmu notifiers by forbidding allocations in dma_fence_begin/end_signalling critical paths.
A third reasons for not requiring force-retiring of dma_fence before drm_dev_unplug is the races: Before drm_dev_unplug you haven't stopped new fences from happening, but until you've stopped new fences it's hard to guarantee they're all retired. How do you solve this currently.
See amdgpu_finilize_device_fences in https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.... I think the steps described there answer your concern here.
The hard problem is stopping further command submission. Not seeing how you solve that.
By stopping GPU SW scheduler before force completion of HW fences, see amdgpu_finilize_device_fences->amdgpu_fence_driver_fini_hw and the comment above it.
But I'm definitely scared about all the scheduler/tdr interactions you already have there, and that looks quite a bit like fallout from doing things the wrong way round.
Also given that drm/scheduler is shared, why can't this be a drm/scheduler helper function?
I was thinking about it, what stopped me is that HW fences signaling is done from driver specific HW fence array. But we could do it generic by instead iterating sched->pending_list and signaling s_job->s_fence->parent instead. You also need to retire scheduler's scheduled fences once you stopped the schedulers as they are waited on as dependencies for other jobs submissions (i do take care of it).
Finally there's still hangcheck and all that, so if we go with forbidding dma_fence_wait from within drm_dev_enter/exit sections, then drivers don't need to have additional tricky code to force-retire fences. TDR will take care already (albeit with maybe a slightly annoying long timeout, which we can shorten to "time out everything immediately" after drm_dev_unplug).
I am not aware of TDR handlers that do it today, at least we don't, we don't check that if device is gone let's instead of resetting the device and resubmit jobs just force retire all the HW fences. In any case, this can and i think should be done in pci remove callback because this is the place that supposed to handle device extraction. I for example in amdgpu_finilize_device_fences just block all TDRs from taking place as first step in the process. If other drivers want to force retire fences in their TDR handlers they still need to block and wait for all such TDRs in their pci_remove handler.
TDR definitely force-completes the fence that did hang. Of course it'll take a while until they've all completed this way, but we do have guaranteed forward progress since we've stopped all further fences from showing up because drm_dev_unplug is called already.
And yes after drm_dev_unplug you can then force-retire the tdr stuff.
What we definitely can't have is half the drivers doing it one way, and the other half the other way. So your driver flag to wrap the ioctl optionally in a drm_dev_enter/exit path is a no-go still I think.
I guess my tldr; is: I definitely see how your current approach gives quicker results for amdgpu right now, but long term I'm seeing more positives on the other one. At least I expect less special cases due to hotunplug with that.
As i expressed my viewpoint above - seems to me any driver in need to support hot-unplug must force retire it's fences because of need to unblock all dma_fence waits and so it will not be a special case.
This isn't the special case I meant. It's the very tricky force-retire-before-you-unplugged-officially which is large scale nasty.
Also if your driver doesn't force-retire already, it's buggy. The additional need of hotunplug is just that we're trying to force-retire a bit faster, because we know it's all hopeless. But e.g. i915 already has a fallback that does this automatically:
- first we reset only the engine/context, keeping everyone else running
- if that doesn't pan out, we reset the entire chip and give up an anything that's in-flight, which (iirc, it did so at least in the past) force retires everything outstanding.
I think amdgpu only has full chip reset, so your first step tries to reissue all other tasks. But that's not necessarily how it needs to happen.
Either way drivers must force retire everything (albeit maybe a bit at a slow pace) if the hw ceased to work properly already. Hotunplug really isn't anything new here. -Daniel
Let's then agree on the way forward -
You raised before the following suggestion -
" btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go with that: We could do the same trick we've done for DRM_UNLOCKED:
- drm_dev_enter/exit is called for any ioctl that has not set the DRM_HOTUNPLUG_SAFE flag
- for drm core ioctls we push them into all ioctls and decide how to handle/where (with the aim to have the least amount of code flow different during hotunplug vs after hotunplug has finished, to reduce testing scope)
- then we make DRM_HOTUNPLUG_SAFE the implied default
" My problem here is that I have no good understating, criteria for how to decide per each ioctl on the right scope of drm_dev_enter/ exit. It depends on whether each next function call can lead somewhere down the call stack to dma_fence_wait and/or whether it can lead to registers access. Seems to me very hard to cover and error prone.
Tbh, neither do I. This requires a lot of work to analyze.
Another options which we discussed internally before and is basically same as current drivers i guess is simply to scope with drm_dev_enter/ exit all the back-end HW specific callbacks. Those are most of the places MMIO access takes place and by definition no dma_fence_wait can be there as it's HW specific code. This leaves MMIO access through pointers (memcpy, and various pointer de-references) which will need to be protected on case by case, but given that I unmap all MMIO anyway as last step of PCI remove callback, all of them will be found by try and error eventually. I feel more comfortable with this approach as I have a clear understating of how to deal with it.
Hm ... I'm maybe failing to see the difference, but at least on the kms side "put drm_dev_enter/exit into driver callbacks" feels like the right place for them.
Render (and I guess kfd for amd as the special case) ioctl are different, especially all the driver specific ones.
So one thing that cross my mind maybe as step 0 is to annotate the rules for drm_dev_enter/exit using lockdep. With lockdep we can both check whether a lock is held, but also whether it's not held (but the latter is only possible with CONFIG_PROVE_LOCKING enabled). I think it would be good to annotate all the major locks in the kernel against drm_dev_enter/exit:
- dma_fence_wait could check that the drm_dev_enter/exit srcu is _not_ held.
We can't insert this directly inside dma_fence_wait as a DRM agnostic layer, are you porpoising to wrap all dma_fence_waits in DRM and drivers code with thin wrapper calling lockdep_assert_held(drm_unplug_srcu) ? What about dma_resv objs and dma_resv_wait_timeout_rcu style of waits ?
Hm ... yeah this doesn't work. Least because we also don't have a generic way to get from dma_fence to anything resembling a drm_device or any other device. Which is really annoying.
The dma_resv waits boil down to dma_fence waits underneath, which is why I wanted to annotate dma_fence_wait ...
The problem is also that because srcu sections nest completely freely I don't think we can teach lockdep that it's not allowed to hold the srcu read side lock when we call dma_fence_wait. The latter has it's own lockdep_key, so maybe annotating things that way is possible. Not sure. Lockdep has some big limitations for read side locks.
Maybe you can figure out something, my current bandwidth for screaming at clever lockdep annotations is a bit thin :-(
because srcu nest _very_ freely there's kinda no real restrictions for putting drm_dev_enter/exit within a lock critical section. Might still be good to explicitly call out in docs where it's all ok:
- interrupt handlers (I hope that's the case, otherwise I screwed up)
- shrinkers/mmu_notifier callbacks
Is there a problem scooping these with drm_dev_enter/exit ? Might they hang indefinitely once a device is gone ?
Well your irq handler must be robust, since drm_dev_enter/exit only helps so much.
The shrinker still needs to work somewhat so it can help free memory, while the hotunplug code
anything else that's not allowed in within drm_dev_enter/exit. I'm e.g. wondering whether we should disallow drm_modeset_lock() or maybe dma_resv_lock(), or whether that's too restrictive. It could help quite a bit in finding places where the drm_dev_enter/exit section is too wide.
another one is the inverse, but I guess you have that already with putting a drm_dev_is_held() into mmio helpers and all that to make sure we really have them all caught.
Not sure what drm_dev_is_held means here ? Also what do you mean by MMIO helpers ? Maybe you meant the page fault helpers like ttm_bo_vm_fault ?
Checking that we're in the srcu read side critical section is what I meant with drm_dev_is_held. Maybe drm_dev_is_entered is clearer, but also sounds a bit silly. -Daniel
Andrey
Above is just examples, I think the more we nail down these rules one way or the other, the better for consistency across drivers. And without consistency everyone will be forced to write their own mmap helpers instead of one in ttm, or scheduler cleanup helpers instead of one in drm/scheduler.
P.S Please respond on the question for you on the other thread at 'PATCH v5 15/27] drm/scheduler: Fix hang when sched_entity released' about suggestion by Christian of partial up-streaming of this code up to and before the patches dealing with scoping of drm_dev_enter/exit scoping.
Ok will try, I'm a bit burried unfortunately so thanks for reminder when I miss something. -Daniel
Andrey
Andrey
Cheers, Daniel
> > Andrey > > > > > It is unfortunately even more work until we've reached the goal, but I > > think it's safest and most flexible approach overall. > > > > Cheers, Daniel > > > > > > > > Andrey > > > > > > > > > > > Cheers, Daniel > > > > > > > > > > > > > > > > > > > > > > > > > > > I guess we should clarify this in the hotunplug doc? > > > > > > > > > > Agree > > > > > > > > > > > > > > > > > To clarify: I'm not against throwing an ENODEV at userspace for ioctl that > > > > > > really make no sense, and where we're rather confident that all properly > > > > > > implemented userspace will gracefully handle failures. Like a modeset, or > > > > > > opening a device, or trying to import a dma-buf or stuff like that which > > > > > > can already fail in normal operation for any kind of reason. > > > > > > > > > > > > But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail > > > > > > after hotunplug. > > > > > > > > > > As I pointed above, this a bit confuses me given that we already do > > > > > blanker rejection of IOCTLs if device is unplugged. > > > > > > > > Well I'm confused about this too :-/ > > > > > > > > > > And then there's the middle ground, like doing a pageflip or buffer flush, > > > > > > which I guess some userspace might handle, but risky to inflict those > > > > > > consequences on them. atomic modeset is especially fun since depending > > > > > > what you're doing it can be both "failures expected" and "failures not > > > > > > really expected in normal operation". > > > > > > > > > > > > Also, this really should be consistent across drivers, not solved with a > > > > > > driver flag for every possible combination. > > > > > > > > > > > > If you look at the current hotunplug kms drivers, they have > > > > > > drm_dev_enter/exit sprinkled in specific hw callback functions because of > > > > > > the above problems. But maybe it makes sense to change things in a few > > > > > > cases. But then we should do it across the board. > > > > > > > > > > So as I understand your preferred approach is that I scope any back_end, HW > > > > > specific function with drm_dev_enter/exit because that where MMIO > > > > > access takes place. But besides explicit MMIO access thorough > > > > > register accessors in the HW back-end there is also indirect MMIO access > > > > > taking place throughout the code in the driver because of various VRAM > > > > > BOs which provide CPU access to VRAM through the VRAM BAR. This kind of > > > > > access is spread all over in the driver and even in mid-layers such as > > > > > TTM and not limited to HW back-end functions. It means it's much harder > > > > > to spot such places to surgically scope them with drm_dev_enter/exit and > > > > > also that any new such code introduced will immediately break hot unplug > > > > > because the developers can't be expected to remember making their code > > > > > robust to this specific use case. That why when we discussed internally > > > > > what approach to take to protecting code with drm_dev_enter/exit we > > > > > opted for using the widest available scope. > > > > > > > > The thing is, you kinda have to anyway. There's enormous amounts of > > > > asynchronous processing going on. E.g. nonblocking atomic commits also do > > > > ttm unpinning and fun stuff like that, which if you sync things wrong can > > > > happen way late. So the door for bad fallout is wide open :-( > > > > > > > > I'm not sure where the right tradeoff is to make sure we catch them all, > > > > and can make sure with testing that we've indeed caught them all. > > > > -Daniel > > > > > >
To allow scoping DRM IOCTLs with drm_dev_enter/exit.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 8a19b8dd02ee..d0f34f230ef3 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1579,7 +1579,7 @@ static const struct drm_driver amdgpu_kms_driver = { DRIVER_ATOMIC | DRIVER_GEM | DRIVER_RENDER | DRIVER_MODESET | DRIVER_SYNCOBJ | - DRIVER_SYNCOBJ_TIMELINE, + DRIVER_SYNCOBJ_TIMELINE | DRIVER_HOTUNPLUG_SUPPORT, .open = amdgpu_driver_open_kms, .postclose = amdgpu_driver_postclose_kms, .lastclose = amdgpu_driver_lastclose_kms,
To allow completion and further block of HW accesses post device PCI remove.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- .../amd/display/amdgpu_dm/amdgpu_dm_hdcp.c | 124 +++++++++++------- .../drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c | 24 +++- 2 files changed, 98 insertions(+), 50 deletions(-)
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c index 0cdbfcd475ec..81ea5a1ea46b 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c @@ -28,6 +28,7 @@ #include "amdgpu_dm.h" #include "dm_helpers.h" #include <drm/drm_hdcp.h> +#include <drm/drm_drv.h> #include "hdcp_psp.h"
/* @@ -260,20 +261,27 @@ void hdcp_handle_cpirq(struct hdcp_workqueue *hdcp_work, unsigned int link_index static void event_callback(struct work_struct *work) { struct hdcp_workqueue *hdcp_work; + int idx;
hdcp_work = container_of(to_delayed_work(work), struct hdcp_workqueue, callback_dwork);
- mutex_lock(&hdcp_work->mutex); + if (drm_dev_enter(hdcp_work->aconnector->base.dev, &idx)) {
- cancel_delayed_work(&hdcp_work->callback_dwork); + mutex_lock(&hdcp_work->mutex);
- mod_hdcp_process_event(&hdcp_work->hdcp, MOD_HDCP_EVENT_CALLBACK, - &hdcp_work->output); + cancel_delayed_work(&hdcp_work->callback_dwork); + + mod_hdcp_process_event(&hdcp_work->hdcp, MOD_HDCP_EVENT_CALLBACK, + &hdcp_work->output);
- process_output(hdcp_work); + process_output(hdcp_work);
- mutex_unlock(&hdcp_work->mutex); + mutex_unlock(&hdcp_work->mutex); + + drm_dev_exit(idx); + + }
} @@ -284,34 +292,41 @@ static void event_property_update(struct work_struct *work) struct amdgpu_dm_connector *aconnector = hdcp_work->aconnector; struct drm_device *dev = hdcp_work->aconnector->base.dev; long ret; + int idx; + + if (drm_dev_enter(dev, &idx)) { + + drm_modeset_lock(&dev->mode_config.connection_mutex, NULL); + mutex_lock(&hdcp_work->mutex);
- drm_modeset_lock(&dev->mode_config.connection_mutex, NULL); - mutex_lock(&hdcp_work->mutex);
+ if (aconnector->base.state->commit) { + ret = wait_for_completion_interruptible_timeout(&aconnector->base.state->commit->hw_done, 10 * HZ);
- if (aconnector->base.state->commit) { - ret = wait_for_completion_interruptible_timeout(&aconnector->base.state->commit->hw_done, 10 * HZ); + if (ret == 0) { + DRM_ERROR("HDCP state unknown! Setting it to DESIRED"); + hdcp_work->encryption_status = MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF; + } + }
- if (ret == 0) { - DRM_ERROR("HDCP state unknown! Setting it to DESIRED"); - hdcp_work->encryption_status = MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF; + if (hdcp_work->encryption_status != MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF) { + if (aconnector->base.state->hdcp_content_type == DRM_MODE_HDCP_CONTENT_TYPE0 && + hdcp_work->encryption_status <= MOD_HDCP_ENCRYPTION_STATUS_HDCP2_TYPE0_ON) + drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_ENABLED); + else if (aconnector->base.state->hdcp_content_type == DRM_MODE_HDCP_CONTENT_TYPE1 && + hdcp_work->encryption_status == MOD_HDCP_ENCRYPTION_STATUS_HDCP2_TYPE1_ON) + drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_ENABLED); + } else { + drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_DESIRED); } - }
- if (hdcp_work->encryption_status != MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF) { - if (aconnector->base.state->hdcp_content_type == DRM_MODE_HDCP_CONTENT_TYPE0 && - hdcp_work->encryption_status <= MOD_HDCP_ENCRYPTION_STATUS_HDCP2_TYPE0_ON) - drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_ENABLED); - else if (aconnector->base.state->hdcp_content_type == DRM_MODE_HDCP_CONTENT_TYPE1 && - hdcp_work->encryption_status == MOD_HDCP_ENCRYPTION_STATUS_HDCP2_TYPE1_ON) - drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_ENABLED); - } else { - drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_DESIRED); - }
+ mutex_unlock(&hdcp_work->mutex); + drm_modeset_unlock(&dev->mode_config.connection_mutex); + + drm_dev_exit(idx);
- mutex_unlock(&hdcp_work->mutex); - drm_modeset_unlock(&dev->mode_config.connection_mutex); + } }
static void event_property_validate(struct work_struct *work) @@ -320,58 +335,77 @@ static void event_property_validate(struct work_struct *work) container_of(to_delayed_work(work), struct hdcp_workqueue, property_validate_dwork); struct mod_hdcp_display_query query; struct amdgpu_dm_connector *aconnector = hdcp_work->aconnector; + int idx;
if (!aconnector) return;
- mutex_lock(&hdcp_work->mutex); + if (drm_dev_enter(hdcp_work->aconnector->base.dev, &idx)) {
- query.encryption_status = MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF; - mod_hdcp_query_display(&hdcp_work->hdcp, aconnector->base.index, &query); + mutex_lock(&hdcp_work->mutex);
- if (query.encryption_status != hdcp_work->encryption_status) { - hdcp_work->encryption_status = query.encryption_status; - schedule_work(&hdcp_work->property_update_work); - } + query.encryption_status = MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF; + mod_hdcp_query_display(&hdcp_work->hdcp, aconnector->base.index, &query); + + if (query.encryption_status != hdcp_work->encryption_status) { + hdcp_work->encryption_status = query.encryption_status; + schedule_work(&hdcp_work->property_update_work); + } + + mutex_unlock(&hdcp_work->mutex);
- mutex_unlock(&hdcp_work->mutex); + drm_dev_exit(idx); + + } }
static void event_watchdog_timer(struct work_struct *work) { struct hdcp_workqueue *hdcp_work; + int idx;
hdcp_work = container_of(to_delayed_work(work), struct hdcp_workqueue, watchdog_timer_dwork);
- mutex_lock(&hdcp_work->mutex); + if (drm_dev_enter(hdcp_work->aconnector->base.dev, &idx)) {
- cancel_delayed_work(&hdcp_work->watchdog_timer_dwork); + mutex_lock(&hdcp_work->mutex);
- mod_hdcp_process_event(&hdcp_work->hdcp, - MOD_HDCP_EVENT_WATCHDOG_TIMEOUT, - &hdcp_work->output); + cancel_delayed_work(&hdcp_work->watchdog_timer_dwork);
- process_output(hdcp_work); + mod_hdcp_process_event(&hdcp_work->hdcp, + MOD_HDCP_EVENT_WATCHDOG_TIMEOUT, + &hdcp_work->output);
- mutex_unlock(&hdcp_work->mutex); + process_output(hdcp_work); + + mutex_unlock(&hdcp_work->mutex); + + drm_dev_exit(idx); + }
}
static void event_cpirq(struct work_struct *work) { struct hdcp_workqueue *hdcp_work; + int idx;
hdcp_work = container_of(work, struct hdcp_workqueue, cpirq_work);
- mutex_lock(&hdcp_work->mutex); + if (drm_dev_enter(hdcp_work->aconnector->base.dev, &idx)) {
- mod_hdcp_process_event(&hdcp_work->hdcp, MOD_HDCP_EVENT_CPIRQ, &hdcp_work->output); + mutex_lock(&hdcp_work->mutex);
- process_output(hdcp_work); + mod_hdcp_process_event(&hdcp_work->hdcp, MOD_HDCP_EVENT_CPIRQ, &hdcp_work->output);
- mutex_unlock(&hdcp_work->mutex); + process_output(hdcp_work); + + mutex_unlock(&hdcp_work->mutex); + + drm_dev_exit(idx); + }
}
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c index e0000c180ed1..d8ee552d373e 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c @@ -30,6 +30,8 @@ #include "amdgpu_dm.h" #include "amdgpu_dm_irq.h"
+#include <drm/drm_drv.h> + /** * DOC: overview * @@ -115,15 +117,27 @@ static void dm_irq_work_func(struct work_struct *work) container_of(work, struct irq_list_head, work); struct list_head *handler_list = &irq_list_head->head; struct amdgpu_dm_irq_handler_data *handler_data; + int idx; + + + handler_data = list_first_entry_or_null(handler_list, struct amdgpu_dm_irq_handler_data, list); + + if (!handler_data) + return;
- list_for_each_entry(handler_data, handler_list, list) { - DRM_DEBUG_KMS("DM_IRQ: work_func: for dal_src=%d\n", + if (drm_dev_enter(handler_data->dm->ddev, &idx)) { + + list_for_each_entry(handler_data, handler_list, list) { + DRM_DEBUG_KMS("DM_IRQ: work_func: for dal_src=%d\n", + handler_data->irq_source); + + DRM_DEBUG_KMS("DM_IRQ: schedule_work: for dal_src=%d\n", handler_data->irq_source);
- DRM_DEBUG_KMS("DM_IRQ: schedule_work: for dal_src=%d\n", - handler_data->irq_source); + handler_data->handler(handler_data->handler_arg); + }
- handler_data->handler(handler_data->handler_arg); + drm_dev_exit(idx); }
/* Call a DAL subcomponent which registered for interrupt notification
To allow completion and further block of HW accesses post device PCI remove.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/pm/amdgpu_dpm.c | 44 +++++++++++++---------- drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c | 26 +++++++++++--- 2 files changed, 47 insertions(+), 23 deletions(-)
diff --git a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c index 8fb12afe3c96..649e10d52d17 100644 --- a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c +++ b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c @@ -31,6 +31,7 @@ #include "amdgpu_display.h" #include "hwmgr.h" #include <linux/power_supply.h> +#include <drm/drm_drv.h>
#define WIDTH_4K 3840
@@ -1316,29 +1317,36 @@ void amdgpu_dpm_thermal_work_handler(struct work_struct *work) /* switch to the thermal state */ enum amd_pm_state_type dpm_state = POWER_STATE_TYPE_INTERNAL_THERMAL; int temp, size = sizeof(temp); + int idx;
if (!adev->pm.dpm_enabled) return;
- if (!amdgpu_dpm_read_sensor(adev, AMDGPU_PP_SENSOR_GPU_TEMP, - (void *)&temp, &size)) { - if (temp < adev->pm.dpm.thermal.min_temp) - /* switch back the user state */ - dpm_state = adev->pm.dpm.user_state; - } else { - if (adev->pm.dpm.thermal.high_to_low) - /* switch back the user state */ - dpm_state = adev->pm.dpm.user_state; - } - mutex_lock(&adev->pm.mutex); - if (dpm_state == POWER_STATE_TYPE_INTERNAL_THERMAL) - adev->pm.dpm.thermal_active = true; - else - adev->pm.dpm.thermal_active = false; - adev->pm.dpm.state = dpm_state; - mutex_unlock(&adev->pm.mutex); + if (drm_dev_enter(&adev->ddev, &idx)) {
- amdgpu_pm_compute_clocks(adev); + if (!amdgpu_dpm_read_sensor(adev, AMDGPU_PP_SENSOR_GPU_TEMP, + (void *)&temp, &size)) { + if (temp < adev->pm.dpm.thermal.min_temp) + /* switch back the user state */ + dpm_state = adev->pm.dpm.user_state; + } else { + if (adev->pm.dpm.thermal.high_to_low) + /* switch back the user state */ + dpm_state = adev->pm.dpm.user_state; + } + mutex_lock(&adev->pm.mutex); + if (dpm_state == POWER_STATE_TYPE_INTERNAL_THERMAL) + adev->pm.dpm.thermal_active = true; + else + adev->pm.dpm.thermal_active = false; + adev->pm.dpm.state = dpm_state; + mutex_unlock(&adev->pm.mutex); + + amdgpu_pm_compute_clocks(adev); + + drm_dev_exit(idx); + + } }
static struct amdgpu_ps *amdgpu_dpm_pick_power_state(struct amdgpu_device *adev, diff --git a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c index d143ef1b460b..f034c8a5eb44 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c +++ b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c @@ -25,6 +25,8 @@ #include <linux/firmware.h> #include <linux/pci.h>
+#include <drm/drm_drv.h> + #include "amdgpu.h" #include "amdgpu_smu.h" #include "smu_internal.h" @@ -904,21 +906,35 @@ static void smu_throttling_logging_work_fn(struct work_struct *work) { struct smu_context *smu = container_of(work, struct smu_context, throttling_logging_work); + int idx; + + + if (drm_dev_enter(&smu->adev->ddev, &idx)) { + + smu_log_thermal_throttling(smu);
- smu_log_thermal_throttling(smu); + drm_dev_exit(idx); + } }
static void smu_interrupt_work_fn(struct work_struct *work) { struct smu_context *smu = container_of(work, struct smu_context, interrupt_work); + int idx;
- mutex_lock(&smu->mutex); + if (drm_dev_enter(&smu->adev->ddev, &idx)) {
- if (smu->ppt_funcs && smu->ppt_funcs->interrupt_work) - smu->ppt_funcs->interrupt_work(smu); + mutex_lock(&smu->mutex);
- mutex_unlock(&smu->mutex); + if (smu->ppt_funcs && smu->ppt_funcs->interrupt_work) + smu->ppt_funcs->interrupt_work(smu); + + mutex_unlock(&smu->mutex); + + drm_dev_exit(idx); + + } }
static int smu_sw_init(void *handle)
To allow completion and further block of HW accesses post device PCI remove.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c b/drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c index bc47f6a44456..563f02ab5b95 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c @@ -43,8 +43,10 @@ #include <linux/slab.h> #include <linux/device.h> #include <linux/kfifo.h> +#include <drm/drm_drv.h> #include "kfd_priv.h"
+ #define KFD_IH_NUM_ENTRIES 8192
static void interrupt_wq(struct work_struct *); @@ -145,15 +147,21 @@ static void interrupt_wq(struct work_struct *work) struct kfd_dev *dev = container_of(work, struct kfd_dev, interrupt_work); uint32_t ih_ring_entry[KFD_MAX_RING_ENTRY_SIZE]; + int idx;
if (dev->device_info->ih_ring_entry_size > sizeof(ih_ring_entry)) { dev_err_once(kfd_chardev(), "Ring entry too small\n"); return; }
- while (dequeue_ih_ring_entry(dev, ih_ring_entry)) - dev->device_info->event_interrupt_class->interrupt_wq(dev, - ih_ring_entry); + if (drm_dev_enter(dev->ddev, &idx)) { + + while (dequeue_ih_ring_entry(dev, ih_ring_entry)) + dev->device_info->event_interrupt_class->interrupt_wq(dev, + ih_ring_entry); + + drm_dev_exit(idx); + } }
bool interrupt_is_wanted(struct kfd_dev *dev,
To allow completion and further block of HW accesses post device PCI remove.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 11 +++- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 ++++++---- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 26 +++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c | 28 ++++++---- drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 55 ++++++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c | 43 ++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c | 30 ++++++----- drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c | 61 ++++++++++++---------- drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c | 10 +++- 9 files changed, 189 insertions(+), 104 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c index 9edb35ba181b..f942496c2b35 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c @@ -30,6 +30,7 @@ #include <linux/dma-buf.h> #include "amdgpu_xgmi.h" #include <uapi/linux/kfd_ioctl.h> +#include <drm/drm_drv.h>
/* Total memory size in system memory and all GPU VRAM. Used to * estimate worst case amount of memory to reserve for page tables @@ -223,9 +224,15 @@ int amdgpu_amdkfd_post_reset(struct amdgpu_device *adev) void amdgpu_amdkfd_gpu_reset(struct kgd_dev *kgd) { struct amdgpu_device *adev = (struct amdgpu_device *)kgd; + int idx;
- if (amdgpu_device_should_recover_gpu(adev)) - amdgpu_device_gpu_recover(adev, NULL); + if (drm_dev_enter(&adev->ddev, &idx)) { + + if (amdgpu_device_should_recover_gpu(adev)) + amdgpu_device_gpu_recover(adev, NULL); + + drm_dev_exit(idx); + } }
int amdgpu_amdkfd_alloc_gtt_mem(struct kgd_dev *kgd, size_t size, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 55afc11c17e6..c30e0b0596a5 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -2674,24 +2674,35 @@ static void amdgpu_device_delayed_init_work_handler(struct work_struct *work) { struct amdgpu_device *adev = container_of(work, struct amdgpu_device, delayed_init_work.work); - int r; + int r, idx;
- r = amdgpu_ib_ring_tests(adev); - if (r) - DRM_ERROR("ib ring test failed (%d).\n", r); + if (drm_dev_enter(&adev->ddev, &idx)) { + r = amdgpu_ib_ring_tests(adev); + if (r) + DRM_ERROR("ib ring test failed (%d).\n", r); + + drm_dev_exit(idx); + } }
static void amdgpu_device_delay_enable_gfx_off(struct work_struct *work) { struct amdgpu_device *adev = container_of(work, struct amdgpu_device, gfx.gfx_off_delay_work.work); + int idx; + + if (drm_dev_enter(&adev->ddev, &idx)) { + + mutex_lock(&adev->gfx.gfx_off_mutex); + if (!adev->gfx.gfx_off_state && !adev->gfx.gfx_off_req_count) { + if (!amdgpu_dpm_set_powergating_by_smu(adev, AMD_IP_BLOCK_TYPE_GFX, true)) + adev->gfx.gfx_off_state = true; + } + mutex_unlock(&adev->gfx.gfx_off_mutex); + + drm_dev_exit(idx);
- mutex_lock(&adev->gfx.gfx_off_mutex); - if (!adev->gfx.gfx_off_state && !adev->gfx.gfx_off_req_count) { - if (!amdgpu_dpm_set_powergating_by_smu(adev, AMD_IP_BLOCK_TYPE_GFX, true)) - adev->gfx.gfx_off_state = true; } - mutex_unlock(&adev->gfx.gfx_off_mutex); }
/** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index a922154953a7..5eda0d0fc974 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -188,8 +188,15 @@ static void amdgpu_irq_handle_ih1(struct work_struct *work) { struct amdgpu_device *adev = container_of(work, struct amdgpu_device, irq.ih1_work); + int idx;
- amdgpu_ih_process(adev, &adev->irq.ih1); + if (drm_dev_enter(&adev->ddev, &idx)) { + + amdgpu_ih_process(adev, &adev->irq.ih1); + + drm_dev_exit(idx); + + } }
/** @@ -203,8 +210,14 @@ static void amdgpu_irq_handle_ih2(struct work_struct *work) { struct amdgpu_device *adev = container_of(work, struct amdgpu_device, irq.ih2_work); + int idx; + + if (drm_dev_enter(&adev->ddev, &idx)) { + + amdgpu_ih_process(adev, &adev->irq.ih2);
- amdgpu_ih_process(adev, &adev->irq.ih2); + drm_dev_exit(idx); + } }
/** @@ -218,8 +231,15 @@ static void amdgpu_irq_handle_ih_soft(struct work_struct *work) { struct amdgpu_device *adev = container_of(work, struct amdgpu_device, irq.ih_soft_work); + int idx; + + if (drm_dev_enter(&adev->ddev, &idx)) { + + amdgpu_ih_process(adev, &adev->irq.ih_soft);
- amdgpu_ih_process(adev, &adev->irq.ih_soft); + drm_dev_exit(idx); + + } }
/** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c index 8996cb4ed57a..1e8fd66c1e43 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c @@ -29,6 +29,7 @@ #include "amdgpu_pm.h" #include "soc15d.h" #include "soc15_common.h" +#include <drm/drm_drv.h>
#define JPEG_IDLE_TIMEOUT msecs_to_jiffies(1000)
@@ -78,20 +79,25 @@ static void amdgpu_jpeg_idle_work_handler(struct work_struct *work) struct amdgpu_device *adev = container_of(work, struct amdgpu_device, jpeg.idle_work.work); unsigned int fences = 0; - unsigned int i; + unsigned int i, idx;
- for (i = 0; i < adev->jpeg.num_jpeg_inst; ++i) { - if (adev->jpeg.harvest_config & (1 << i)) - continue; + if (drm_dev_enter(&adev->ddev, &idx)) {
- fences += amdgpu_fence_count_emitted(&adev->jpeg.inst[i].ring_dec); - } + for (i = 0; i < adev->jpeg.num_jpeg_inst; ++i) { + if (adev->jpeg.harvest_config & (1 << i)) + continue;
- if (!fences && !atomic_read(&adev->jpeg.total_submission_cnt)) - amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_JPEG, - AMD_PG_STATE_GATE); - else - schedule_delayed_work(&adev->jpeg.idle_work, JPEG_IDLE_TIMEOUT); + fences += amdgpu_fence_count_emitted(&adev->jpeg.inst[i].ring_dec); + } + + if (!fences && !atomic_read(&adev->jpeg.total_submission_cnt)) + amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_JPEG, + AMD_PG_STATE_GATE); + else + schedule_delayed_work(&adev->jpeg.idle_work, JPEG_IDLE_TIMEOUT); + + drm_dev_exit(idx); + } }
void amdgpu_jpeg_ring_begin_use(struct amdgpu_ring *ring) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c index c0a16eac4923..97a6c028ac74 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c @@ -27,6 +27,7 @@ #include <linux/uaccess.h> #include <linux/reboot.h> #include <linux/syscalls.h> +#include <drm/drm_drv.h>
#include "amdgpu.h" #include "amdgpu_ras.h" @@ -1334,7 +1335,15 @@ static void amdgpu_ras_interrupt_process_handler(struct work_struct *work) struct ras_manager *obj = container_of(data, struct ras_manager, ih_data);
- amdgpu_ras_interrupt_handler(obj); + int idx; + + if (drm_dev_enter(&obj->adev->ddev, &idx)) { + + amdgpu_ras_interrupt_handler(obj); + + drm_dev_exit(idx); + + } }
int amdgpu_ras_interrupt_dispatch(struct amdgpu_device *adev, @@ -1565,31 +1574,37 @@ static void amdgpu_ras_do_recovery(struct work_struct *work) struct amdgpu_device *remote_adev = NULL; struct amdgpu_device *adev = ras->adev; struct list_head device_list, *device_list_handle = NULL; + int idx;
- if (!ras->disable_ras_err_cnt_harvest) { - struct amdgpu_hive_info *hive = amdgpu_get_xgmi_hive(adev); + if (drm_dev_enter(&adev->ddev, &idx)) {
- /* Build list of devices to query RAS related errors */ - if (hive && adev->gmc.xgmi.num_physical_nodes > 1) { - device_list_handle = &hive->device_list; - } else { - INIT_LIST_HEAD(&device_list); - list_add_tail(&adev->gmc.xgmi.head, &device_list); - device_list_handle = &device_list; - } + if (!ras->disable_ras_err_cnt_harvest) { + struct amdgpu_hive_info *hive = amdgpu_get_xgmi_hive(adev);
- list_for_each_entry(remote_adev, - device_list_handle, gmc.xgmi.head) { - amdgpu_ras_query_err_status(remote_adev); - amdgpu_ras_log_on_err_counter(remote_adev); + /* Build list of devices to query RAS related errors */ + if (hive && adev->gmc.xgmi.num_physical_nodes > 1) { + device_list_handle = &hive->device_list; + } else { + INIT_LIST_HEAD(&device_list); + list_add_tail(&adev->gmc.xgmi.head, &device_list); + device_list_handle = &device_list; + } + + list_for_each_entry(remote_adev, + device_list_handle, gmc.xgmi.head) { + amdgpu_ras_query_err_status(remote_adev); + amdgpu_ras_log_on_err_counter(remote_adev); + } + + amdgpu_put_xgmi_hive(hive); }
- amdgpu_put_xgmi_hive(hive); - } + if (amdgpu_device_should_recover_gpu(ras->adev)) + amdgpu_device_gpu_recover(ras->adev, NULL); + atomic_set(&ras->in_recovery, 0);
- if (amdgpu_device_should_recover_gpu(ras->adev)) - amdgpu_device_gpu_recover(ras->adev, NULL); - atomic_set(&ras->in_recovery, 0); + drm_dev_exit(idx); + } }
/* alloc/realloc bps array */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c index df47f5ffa08f..19790afd2893 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c @@ -1234,29 +1234,36 @@ static void amdgpu_uvd_idle_work_handler(struct work_struct *work) struct amdgpu_device *adev = container_of(work, struct amdgpu_device, uvd.idle_work.work); unsigned fences = 0, i, j; + int idx;
- for (i = 0; i < adev->uvd.num_uvd_inst; ++i) { - if (adev->uvd.harvest_config & (1 << i)) - continue; - fences += amdgpu_fence_count_emitted(&adev->uvd.inst[i].ring); - for (j = 0; j < adev->uvd.num_enc_rings; ++j) { - fences += amdgpu_fence_count_emitted(&adev->uvd.inst[i].ring_enc[j]); + if (drm_dev_enter(&adev->ddev, &idx)) { + + for (i = 0; i < adev->uvd.num_uvd_inst; ++i) { + if (adev->uvd.harvest_config & (1 << i)) + continue; + fences += amdgpu_fence_count_emitted(&adev->uvd.inst[i].ring); + for (j = 0; j < adev->uvd.num_enc_rings; ++j) { + fences += amdgpu_fence_count_emitted(&adev->uvd.inst[i].ring_enc[j]); + } } - }
- if (fences == 0) { - if (adev->pm.dpm_enabled) { - amdgpu_dpm_enable_uvd(adev, false); + if (fences == 0) { + if (adev->pm.dpm_enabled) { + amdgpu_dpm_enable_uvd(adev, false); + } else { + amdgpu_asic_set_uvd_clocks(adev, 0, 0); + /* shutdown the UVD block */ + amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_UVD, + AMD_PG_STATE_GATE); + amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_UVD, + AMD_CG_STATE_GATE); + } } else { - amdgpu_asic_set_uvd_clocks(adev, 0, 0); - /* shutdown the UVD block */ - amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_UVD, - AMD_PG_STATE_GATE); - amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_UVD, - AMD_CG_STATE_GATE); + schedule_delayed_work(&adev->uvd.idle_work, UVD_IDLE_TIMEOUT); } - } else { - schedule_delayed_work(&adev->uvd.idle_work, UVD_IDLE_TIMEOUT); + + drm_dev_exit(idx); + } }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c index 833203401ef4..81ad937936bb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c @@ -340,22 +340,28 @@ static void amdgpu_vce_idle_work_handler(struct work_struct *work) struct amdgpu_device *adev = container_of(work, struct amdgpu_device, vce.idle_work.work); unsigned i, count = 0; + int idx;
- for (i = 0; i < adev->vce.num_rings; i++) - count += amdgpu_fence_count_emitted(&adev->vce.ring[i]); + if (drm_dev_enter(&adev->ddev, &idx)) {
- if (count == 0) { - if (adev->pm.dpm_enabled) { - amdgpu_dpm_enable_vce(adev, false); + for (i = 0; i < adev->vce.num_rings; i++) + count += amdgpu_fence_count_emitted(&adev->vce.ring[i]); + + if (count == 0) { + if (adev->pm.dpm_enabled) { + amdgpu_dpm_enable_vce(adev, false); + } else { + amdgpu_asic_set_vce_clocks(adev, 0, 0); + amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_VCE, + AMD_PG_STATE_GATE); + amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_VCE, + AMD_CG_STATE_GATE); + } } else { - amdgpu_asic_set_vce_clocks(adev, 0, 0); - amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_VCE, - AMD_PG_STATE_GATE); - amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_VCE, - AMD_CG_STATE_GATE); + schedule_delayed_work(&adev->vce.idle_work, VCE_IDLE_TIMEOUT); } - } else { - schedule_delayed_work(&adev->vce.idle_work, VCE_IDLE_TIMEOUT); + + drm_dev_exit(idx); } }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c index b42db22761b8..0e7404653ac5 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c @@ -342,42 +342,47 @@ static void amdgpu_vcn_idle_work_handler(struct work_struct *work) container_of(work, struct amdgpu_device, vcn.idle_work.work); unsigned int fences = 0, fence[AMDGPU_MAX_VCN_INSTANCES] = {0}; unsigned int i, j; - int r = 0; + int idx, r = 0;
- for (j = 0; j < adev->vcn.num_vcn_inst; ++j) { - if (adev->vcn.harvest_config & (1 << j)) - continue; + if (drm_dev_enter(&adev->ddev, &idx)) {
- for (i = 0; i < adev->vcn.num_enc_rings; ++i) { - fence[j] += amdgpu_fence_count_emitted(&adev->vcn.inst[j].ring_enc[i]); - } + for (j = 0; j < adev->vcn.num_vcn_inst; ++j) { + if (adev->vcn.harvest_config & (1 << j)) + continue;
- if (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG) { - struct dpg_pause_state new_state; + for (i = 0; i < adev->vcn.num_enc_rings; ++i) + fence[j] += amdgpu_fence_count_emitted(&adev->vcn.inst[j].ring_enc[i]);
- if (fence[j] || - unlikely(atomic_read(&adev->vcn.inst[j].dpg_enc_submission_cnt))) - new_state.fw_based = VCN_DPG_STATE__PAUSE; - else - new_state.fw_based = VCN_DPG_STATE__UNPAUSE; + if (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG) { + struct dpg_pause_state new_state; + + if (fence[j] || + unlikely(atomic_read(&adev->vcn.inst[j].dpg_enc_submission_cnt))) + new_state.fw_based = VCN_DPG_STATE__PAUSE; + else + new_state.fw_based = VCN_DPG_STATE__UNPAUSE;
- adev->vcn.pause_dpg_mode(adev, j, &new_state); + adev->vcn.pause_dpg_mode(adev, j, &new_state); + } + + fence[j] += amdgpu_fence_count_emitted(&adev->vcn.inst[j].ring_dec); + fences += fence[j]; }
- fence[j] += amdgpu_fence_count_emitted(&adev->vcn.inst[j].ring_dec); - fences += fence[j]; - } + if (!fences && !atomic_read(&adev->vcn.total_submission_cnt)) { + amdgpu_gfx_off_ctrl(adev, true); + amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_VCN, + AMD_PG_STATE_GATE); + r = amdgpu_dpm_switch_power_profile(adev, PP_SMC_POWER_PROFILE_VIDEO, + false); + if (r) + dev_warn(adev->dev, "(%d) failed to disable video power profile mode\n", r); + } else { + schedule_delayed_work(&adev->vcn.idle_work, VCN_IDLE_TIMEOUT); + } + + drm_dev_exit(idx);
- if (!fences && !atomic_read(&adev->vcn.total_submission_cnt)) { - amdgpu_gfx_off_ctrl(adev, true); - amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_VCN, - AMD_PG_STATE_GATE); - r = amdgpu_dpm_switch_power_profile(adev, PP_SMC_POWER_PROFILE_VIDEO, - false); - if (r) - dev_warn(adev->dev, "(%d) failed to disable video power profile mode\n", r); - } else { - schedule_delayed_work(&adev->vcn.idle_work, VCN_IDLE_TIMEOUT); } }
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c index 84d2eaa38101..4799290e5625 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c @@ -57,6 +57,8 @@
#include "ivsrcid/ivsrcid_vislands30.h"
+#include <drm/drm_drv.h> + #define GFX8_NUM_GFX_RINGS 1 #define GFX8_MEC_HPD_SIZE 4096
@@ -6793,8 +6795,14 @@ static void gfx_v8_0_sq_irq_work_func(struct work_struct *work)
struct amdgpu_device *adev = container_of(work, struct amdgpu_device, gfx.sq_work.work); struct sq_work *sq_work = container_of(work, struct sq_work, work); + int idx; + + if (drm_dev_enter(&adev->ddev, &idx)) {
- gfx_v8_0_parse_sq_irq(adev, sq_work->ih_data); + gfx_v8_0_parse_sq_irq(adev, sq_work->ih_data); + + drm_dev_exit(idx); + } }
static int gfx_v8_0_sq_irq(struct amdgpu_device *adev,
It's already being released by DRM core through devm
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 1 - 1 file changed, 1 deletion(-)
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index c0b9abb773a4..b9aa15f22cfc 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -3590,7 +3590,6 @@ static int amdgpu_dm_initialize_drm_device(struct amdgpu_device *adev)
static void amdgpu_dm_destroy_drm_device(struct amdgpu_display_manager *dm) { - drm_mode_config_cleanup(dm->ddev); drm_atomic_private_obj_fini(&dm->atomic_obj); return; }
In case device remove is just simualted by sysfs then verify device doesn't keep doing DMA to the released memory after pci_remove is done.
Signed-off-by: Andrey Grodzovsky andrey.grodzovsky@amd.com --- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 6 ++++++ 1 file changed, 6 insertions(+)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index d0f34f230ef3..f3e8fbde62a0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1258,7 +1258,13 @@ amdgpu_pci_remove(struct pci_dev *pdev) drm_dev_unplug(dev); amdgpu_driver_unload_kms(dev);
+ /* + * Flush any in flight DMA operations from device. + * Clear the Bus Master Enable bit and then wait on the PCIe Device + * StatusTransactions Pending bit. + */ pci_disable_device(pdev); + pci_wait_for_pending_transaction(pdev); }
static void
On Wed, Apr 28, 2021 at 11:11:40AM -0400, Andrey Grodzovsky wrote:
Until now extracting a card either by physical extraction (e.g. eGPU with thunderbolt connection or by emulation through syfs -> /sys/bus/pci/devices/device_id/remove) would cause random crashes in user apps. The random crashes in apps were mostly due to the app having mapped a device backed BO into its address space was still trying to access the BO while the backing device was gone. To answer this first problem Christian suggested to fix the handling of mapped memory in the clients when the device goes away by forcibly unmap all buffers the user processes has by clearing their respective VMAs mapping the device BOs. Then when the VMAs try to fill in the page tables again we check in the fault handlerif the device is removed and if so, return an error. This will generate a SIGBUS to the application which can then cleanly terminate.This indeed was done but this in turn created a problem of kernel OOPs were the OOPSes were due to the fact that while the app was terminating because of the SIGBUSit would trigger use after free in the driver by calling to accesses device structures that were already released from the pci remove sequence.This was handled by introducing a 'flush' sequence during device removal were we wait for drm file reference to drop to 0 meaning all user clients directly using this device terminated.
If DRM includes cover letters in merges, maybe fix the below. If they also include the v2, v3, etc below, also consider picking a line width and sticking to it. It seems to be creeping wider every rev.
BO? s/syfs/sysfs/ s/forcibly unmap/forcibly unmapping/ s/handlerif/handler if/ s/processes has/processes have/ s/terminate.This/terminate. This/ s/were the/where the/ s/SIGBUSit/SIGBUS it/ s/to accesses/to access/ s/sequence.This/sequence. This/ s/were we/where we/
v2: Based on discussions in the mailing list with Daniel and Pekka [1] and based on the document produced by Pekka from those discussions [2] the whole approach with returning SIGBUS and waiting for all user clients having CPU mapping of device BOs to die was dropped. Instead as per the document suggestion the device structures are kept alive until the last reference to the device is dropped by user client and in the meanwhile all existing and new CPU mappings of the BOs belonging to the device directly or by dma-buf import are rerouted to per user process dummy rw page.Also, I skipped the 'Requirements for KMS UAPI' section of [2] since i am trying to get the minimal set of requirements that still give useful solution to work and this is the'Requirements for Render and Cross-Device UAPI' section and so my test case is removing a secondary device, which is render only and is not involved in KMS.
v3: More updates following comments from v2 such as removing loop to find DRM file when rerouting page faults to dummy page,getting rid of unnecessary sysfs handling refactoring and moving prevention of GPU recovery post device unplug from amdgpu to scheduler layer. On top of that added unplug support for the IOMMU enabled system.
v4: Drop last sysfs hack and use sysfs default attribute. Guard against write accesses after device removal to avoid modifying released memory. Update dummy pages handling to on demand allocation and release through drm managed framework. Add return value to scheduler job TO handler (by Luben Tuikov) and use this in amdgpu for prevention of GPU recovery post device unplug Also rebase on top of drm-misc-mext instead of amd-staging-drm-next
v5: The most significant in this series is the improved protection from kernel driver accessing MMIO ranges that were allocated for the device once the device is gone. To do this, first a patch 'drm/amdgpu: Unmap all MMIO mappings' is introduced. This patch unamps all MMIO mapped into the kernel address space in the form of BARs and kernel BOs with CPU visible VRAM mappings. This way it helped to discover multiple such access points because a page fault would be immediately generated on access. Most of them were solved by moving HW fini code into pci_remove stage (patch drm/amdgpu: Add early fini callback) and for some who were harder to unwind drm_dev_enter/exit scoping was used. In addition all the IOCTLs and all background work and timers are now protected with drm_dev_enter/exit at their root in an attempt that after drm_dev_unplug is finished none of them run anymore and the pci_remove thread is the only thread executing which might touch the HW. To prevent deadlocks in such case against threads stuck on various HW or SW fences patches 'drm/amdgpu: Finalise device fences on device remove' and drm/amdgpu: Add rw_sem to pushing job into sched queue' take care of force signaling all such existing fences and rejecting any newly added ones.
With these patches I am able to gracefully remove the secondary card using sysfs remove hook while glxgears is running off of secondary card (DRI_PRIME=1) without kernel oopses or hangs and keep working with the primary card or soft reset the device without hangs or oopses. Also as per Daniel's comment I added 3 tests to IGT [4] to core_hotunplug test suite - remove device while commands are submitted, exported BO and exported fence (not pushed yet). Also now it's possible to plug back the device after unplug Also some users now can successfully use those patches with eGPU boxes[3].
TODOs for followup work: Convert AMDGPU code to use devm (for hw stuff) and drmm (for sw stuff and allocations) (Daniel) Add support for 'Requirements for KMS UAPI' section of [2] - unplugging primary, display connected card.
[1] - Discussions during v4 of the patchset https://lists.freedesktop.org/archives/amd-gfx/2021-January/058595.html [2] - drm/doc: device hot-unplug for userspace https://www.spinics.net/lists/dri-devel/msg259755.html [3] - Related gitlab ticket https://gitlab.freedesktop.org/drm/amd/-/issues/1081 [4] - https://gitlab.freedesktop.org/agrodzov/igt-gpu-tools/-/commits/master
Andrey Grodzovsky (27): drm/ttm: Remap all page faults to per process dummy page. drm/ttm: Expose ttm_tt_unpopulate for driver use drm/amdgpu: Split amdgpu_device_fini into early and late drm/amdkfd: Split kfd suspend from devie exit drm/amdgpu: Add early fini callback drm/amdgpu: Handle IOMMU enabled case. drm/amdgpu: Remap all page faults to per process dummy page. PCI: add support for dev_groups to struct pci_device_driver dmr/amdgpu: Move some sysfs attrs creation to default_attr drm/amdgpu: Guard against write accesses after device removal drm/sched: Make timeout timer rearm conditional. drm/amdgpu: Prevent any job recoveries after device is unplugged. drm/amdgpu: When filizing the fence driver. stop scheduler first. drm/amdgpu: Fix hang on device removal. drm/scheduler: Fix hang when sched_entity released drm/amdgpu: Unmap all MMIO mappings drm/amdgpu: Add rw_sem to pushing job into sched queue drm/sched: Expose drm_sched_entity_kill_jobs drm/amdgpu: Finilise device fences on device remove. drm: Scope all DRM IOCTLs with drm_dev_enter/exit drm/amdgpu: Add support for hot-unplug feature at DRM level. drm/amd/display: Scope all DM queued work with drm_dev_enter/exit drm/amd/powerplay: Scope all PM queued work with drm_dev_enter/exit drm/amdkfd: Scope all KFD queued work with drm_dev_enter/exit drm/amdgpu: Scope all amdgpu queued work with drm_dev_enter/exit drm/amd/display: Remove superflous drm_mode_config_cleanup drm/amdgpu: Verify DMA opearations from device are done
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 18 +- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 13 +- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c | 17 +- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 13 +- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 353 ++++++++++++++---- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 34 +- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 34 +- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 9 + drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | 25 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c | 228 +++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 61 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 33 +- drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c | 28 +- drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 12 +- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 41 +- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 7 + drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c | 115 +++--- drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h | 3 + drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 56 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 70 ++++ drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 52 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 21 +- drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c | 74 ++-- drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c | 45 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c | 83 ++-- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 14 +- drivers/gpu/drm/amd/amdgpu/cik_ih.c | 3 +- drivers/gpu/drm/amd/amdgpu/cz_ih.c | 3 +- drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c | 10 +- drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 3 +- drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 5 +- drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 44 +-- drivers/gpu/drm/amd/amdgpu/psp_v12_0.c | 8 +- drivers/gpu/drm/amd/amdgpu/psp_v3_1.c | 8 +- drivers/gpu/drm/amd/amdgpu/si_ih.c | 3 +- drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 3 +- drivers/gpu/drm/amd/amdgpu/vce_v4_0.c | 26 +- drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c | 22 +- drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 5 +- drivers/gpu/drm/amd/amdgpu/vega20_ih.c | 2 +- drivers/gpu/drm/amd/amdkfd/kfd_device.c | 3 +- drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c | 14 +- .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 13 +- .../amd/display/amdgpu_dm/amdgpu_dm_hdcp.c | 124 +++--- .../drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c | 24 +- drivers/gpu/drm/amd/include/amd_shared.h | 2 + drivers/gpu/drm/amd/pm/amdgpu_dpm.c | 44 ++- .../drm/amd/pm/powerplay/smumgr/smu7_smumgr.c | 2 + drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c | 26 +- drivers/gpu/drm/drm_ioctl.c | 15 +- drivers/gpu/drm/scheduler/sched_entity.c | 6 +- drivers/gpu/drm/scheduler/sched_main.c | 35 +- drivers/gpu/drm/ttm/ttm_bo_vm.c | 79 +++- drivers/gpu/drm/ttm/ttm_tt.c | 1 + drivers/pci/pci-driver.c | 1 + include/drm/drm_drv.h | 6 + include/drm/gpu_scheduler.h | 1 + include/drm/ttm/ttm_bo_api.h | 2 + include/linux/pci.h | 3 + 64 files changed, 1388 insertions(+), 633 deletions(-)
-- 2.25.1
dri-devel@lists.freedesktop.org
-
Alex Deucher -
Andrey Grodzovsky -
Bjorn Helgaas -
Christian König -
Christian König -
Daniel Vetter -
Felix Kuehling -
Lazar, Lijo