This patch series introduces failsafe migration blits. The reason for this seemingly strange concept is that if the initial clearing or readback of LMEM fails for some reason, and we then set up either GPU- or CPU ptes to the allocated LMEM, we can expose old contents from other clients.
So after each migration blit we attach a struct dma-fence-work that checks the error value and if it's an error, perform a memcpy blit, instead.
This comes with some needed infrastructure updates:
Patch 1, updates dma_fence_work to do the work even if there is an error. The work callback needs to check for error and take action accordingly. Patch 2, Introduces refcounted sg-tables. The sg-tables are needed async for the memcpy. Patch 3, Introduces the failsafe migration blits and selftests. Patch 4, Adds the possibility to attach the struct dma_fence_work to a timeline. Patch 5, Attached the migration fence to a timeline since TTM requires that for upcoming async eviction. Patch 6 Adds an optimization for coalescing-only struct dma_fence_work.
Worth to consider during review: Patch 4-6 are probably better done in the context of struct dma_fence_array. Both since we perhaps shouldn't add irq work to yet another fence data structure and also because the i915 command submission can individualize struct dma_fence_arrays.
Also the memcpy solution here isn't a final one as it only works if the aperture covers all of lmem. We probably need to work on a solution where we intercept move_fence errors and refuse GPU- and CPU mappings.
Thomas Hellström (6): drm/i915: Update dma_fence_work drm/i915: Introduce refcounted sg-tables drm/i915/ttm: Failsafe migration blits drm/i915: Add a struct dma_fence_work timeline drm/i915/ttm: Attach the migration fence to a region timeline on eviction drm/i915: Use irq work for coalescing-only dma-fence-work
drivers/gpu/drm/i915/gem/i915_gem_clflush.c | 5 + .../gpu/drm/i915/gem/i915_gem_object_types.h | 3 +- drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 467 ++++++++++++++---- drivers/gpu/drm/i915/gem/i915_gem_ttm.h | 4 + .../drm/i915/gem/selftests/i915_gem_migrate.c | 24 +- drivers/gpu/drm/i915/i915_scatterlist.c | 62 ++- drivers/gpu/drm/i915/i915_scatterlist.h | 76 ++- drivers/gpu/drm/i915/i915_sw_fence_work.c | 145 +++++- drivers/gpu/drm/i915/i915_sw_fence_work.h | 61 +++ drivers/gpu/drm/i915/i915_vma.c | 12 +- drivers/gpu/drm/i915/intel_memory_region.c | 43 ++ drivers/gpu/drm/i915/intel_memory_region.h | 7 + drivers/gpu/drm/i915/intel_region_ttm.c | 15 +- drivers/gpu/drm/i915/intel_region_ttm.h | 5 +- drivers/gpu/drm/i915/selftests/mock_region.c | 12 +- 15 files changed, 776 insertions(+), 165 deletions(-)