On Wed, 23 Jan 2019 at 17:44, Christoph Hellwig hch@infradead.org wrote:
I think we just want a driver-local check for those combinations where we know this hack actually works, which really just seems to be x86-64 with PAT. Something like the patch below, but maybe with even more strong warnings to not do something like this elsewhere:
I agree that your patch seems like the right way to ensure that the WC optimization hack is only used where we know it works.
But my concern is that it seems likely that non-cache coherent implementations are relying on this hack as well. There must be a reason that this hack is only disabled for PowerPC platforms if they are cache coherent, for instance, and I suspect that that reason is that the hack is the only thing ensuring that the CPU mapping attributes match the device ones used for these buffers (the vmap()ed ones), whereas the rings and other consistent data structures are using the DMA API as intended, and thus getting uncached attributes in the correct way.
Given that the same concern applies to ARM (and arm64 to some extent), I'd be cautious to disable this optimization entirely, and this is why I ended up disabling it only for cache coherent devices.
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 728e15e5d68a..5fe657f20232 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -456,33 +456,16 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
bo->flags = bp->flags;-#ifdef CONFIG_X86_32
/* XXX: Write-combined CPU mappings of GTT seem broken on 32-bit* See https://bugs.freedesktop.org/show_bug.cgi?id=84627*/bo->flags &= ~AMDGPU_GEM_CREATE_CPU_GTT_USWC;-#elif defined(CONFIG_X86) && !defined(CONFIG_X86_PAT)
/* Don't try to enable write-combining when it can't work, or things* may be slow* See https://bugs.freedesktop.org/show_bug.cgi?id=88758*/-#ifndef CONFIG_COMPILE_TEST -#warning Please enable CONFIG_MTRR and CONFIG_X86_PAT for better performance \
thanks to write-combining-#endif
if (bo->flags & AMDGPU_GEM_CREATE_CPU_GTT_USWC)DRM_INFO_ONCE("Please enable CONFIG_MTRR and CONFIG_X86_PAT for ""better performance thanks to write-combining\n");bo->flags &= ~AMDGPU_GEM_CREATE_CPU_GTT_USWC;-#else
/* For architectures that don't support WC memory,* mask out the WC flag from the BO
/** Don't try to enable write-combined CPU mappings unless we 100%* positively know it works, otherwise there may be dragons.** See:* - https://bugs.freedesktop.org/show_bug.cgi?id=88758* - https://bugs.freedesktop.org/show_bug.cgi?id=84627 */
if (!drm_arch_can_wc_memory())
if (!(IS_ENABLED(CONFIG_X86_64) && IS_ENABLED(CONFIG_X86_PAT))) bo->flags &= ~AMDGPU_GEM_CREATE_CPU_GTT_USWC;-#endif
bo->tbo.bdev = &adev->mman.bdev; amdgpu_bo_placement_from_domain(bo, bp->domain);diff --git a/drivers/gpu/drm/radeon/radeon_object.c b/drivers/gpu/drm/radeon/radeon_object.c index 833e909706a9..c1fb5ad4ab9a 100644 --- a/drivers/gpu/drm/radeon/radeon_object.c +++ b/drivers/gpu/drm/radeon/radeon_object.c @@ -226,32 +226,17 @@ int radeon_bo_create(struct radeon_device *rdev, if (rdev->family >= CHIP_RV610 && rdev->family <= CHIP_RV635) bo->flags &= ~(RADEON_GEM_GTT_WC | RADEON_GEM_GTT_UC);
-#ifdef CONFIG_X86_32
/* XXX: Write-combined CPU mappings of GTT seem broken on 32-bit* See https://bugs.freedesktop.org/show_bug.cgi?id=84627*/bo->flags &= ~(RADEON_GEM_GTT_WC | RADEON_GEM_GTT_UC);-#elif defined(CONFIG_X86) && !defined(CONFIG_X86_PAT)
/* Don't try to enable write-combining when it can't work, or things* may be slow* See https://bugs.freedesktop.org/show_bug.cgi?id=88758*/-#ifndef CONFIG_COMPILE_TEST -#warning Please enable CONFIG_MTRR and CONFIG_X86_PAT for better performance \
thanks to write-combining-#endif
if (bo->flags & RADEON_GEM_GTT_WC)DRM_INFO_ONCE("Please enable CONFIG_MTRR and CONFIG_X86_PAT for ""better performance thanks to write-combining\n");bo->flags &= ~(RADEON_GEM_GTT_WC | RADEON_GEM_GTT_UC);-#else
/* For architectures that don't support WC memory,* mask out the WC flag from the BO
/** Don't try to enable write-combined CPU mappings unless we 100%* positively know it works, otherwise there may be dragons.** See:* - https://bugs.freedesktop.org/show_bug.cgi?id=88758* - https://bugs.freedesktop.org/show_bug.cgi?id=84627 */
if (!drm_arch_can_wc_memory())bo->flags &= ~RADEON_GEM_GTT_WC;-#endif
if (!(IS_ENABLED(CONFIG_X86_64) && IS_ENABLED(CONFIG_X86_PAT)))bo->flags &= ~(RADEON_GEM_GTT_WC | RADEON_GEM_GTT_UC); radeon_ttm_placement_from_domain(bo, domain); /* Kernel allocation are uninterruptible */diff --git a/include/drm/drm_cache.h b/include/drm/drm_cache.h index bfe1639df02d..6c3960f4c477 100644 --- a/include/drm/drm_cache.h +++ b/include/drm/drm_cache.h @@ -40,16 +40,4 @@ void drm_clflush_sg(struct sg_table *st); void drm_clflush_virt_range(void *addr, unsigned long length); u64 drm_get_max_iomem(void);
-static inline bool drm_arch_can_wc_memory(void) -{ -#if defined(CONFIG_PPC) && !defined(CONFIG_NOT_COHERENT_CACHE)
return false;-#elif defined(CONFIG_MIPS) && defined(CONFIG_CPU_LOONGSON3)
return false;-#else
return true;-#endif -}
#endif
On 2019-01-23 5:52 p.m., Ard Biesheuvel wrote:
On Wed, 23 Jan 2019 at 17:44, Christoph Hellwig hch@infradead.org wrote:
I think we just want a driver-local check for those combinations where we know this hack actually works, which really just seems to be x86-64 with PAT. Something like the patch below, but maybe with even more strong warnings to not do something like this elsewhere:
I agree that your patch seems like the right way to ensure that the WC optimization hack is only used where we know it works.
But my concern is that it seems likely that non-cache coherent implementations are relying on this hack as well.
I've been trying to tell you they can't rely on that, because the amdgpu driver doesn't use this functionality for fundamentals such as ring buffers used for feeding the hardware with commands. Instead, for those it relies on snooped PCIe transfers being coherent with the CPU caches.
-#elif defined(CONFIG_X86) && !defined(CONFIG_X86_PAT)
/* Don't try to enable write-combining when it can't work, or things* may be slow* See https://bugs.freedesktop.org/show_bug.cgi?id=88758*/-#ifndef CONFIG_COMPILE_TEST -#warning Please enable CONFIG_MTRR and CONFIG_X86_PAT for better performance \
thanks to write-combining-#endif
if (bo->flags & AMDGPU_GEM_CREATE_CPU_GTT_USWC)DRM_INFO_ONCE("Please enable CONFIG_MTRR and CONFIG_X86_PAT for ""better performance thanks to write-combining\n");
FWIW, please don't drop these compile and build time warnings where we continue to take advantage of PAT.
On Thu, 24 Jan 2019 at 10:31, Michel Dänzer michel@daenzer.net wrote:
On 2019-01-23 5:52 p.m., Ard Biesheuvel wrote:
On Wed, 23 Jan 2019 at 17:44, Christoph Hellwig hch@infradead.org wrote:
I think we just want a driver-local check for those combinations where we know this hack actually works, which really just seems to be x86-64 with PAT. Something like the patch below, but maybe with even more strong warnings to not do something like this elsewhere:
I agree that your patch seems like the right way to ensure that the WC optimization hack is only used where we know it works.
But my concern is that it seems likely that non-cache coherent implementations are relying on this hack as well.
I've been trying to tell you they can't rely on that, because the amdgpu driver doesn't use this functionality for fundamentals such as ring buffers used for feeding the hardware with commands. Instead, for those it relies on snooped PCIe transfers being coherent with the CPU caches.
I understand it does not use this functionality for the ring. Instead, it uses the DMA API, no?
On non-cache coherent systems, that DMA API will allocate memory and map it uncached for the CPU so that it is coherent with the non-cache coherent device.
In any case, if non-cache coherent systems are unlikely to work, and unsupported in case they do, I am fine with disabling this optimization unconditionally for non-X86 architectures.
-#elif defined(CONFIG_X86) && !defined(CONFIG_X86_PAT)
/* Don't try to enable write-combining when it can't work, or things* may be slow* See https://bugs.freedesktop.org/show_bug.cgi?id=88758*/-#ifndef CONFIG_COMPILE_TEST -#warning Please enable CONFIG_MTRR and CONFIG_X86_PAT for better performance \
thanks to write-combining-#endif
if (bo->flags & AMDGPU_GEM_CREATE_CPU_GTT_USWC)DRM_INFO_ONCE("Please enable CONFIG_MTRR and CONFIG_X86_PAT for ""better performance thanks to write-combining\n");FWIW, please don't drop these compile and build time warnings where we continue to take advantage of PAT.
-- Earthling Michel Dänzer | http://www.amd.com Libre software enthusiast | Mesa and X developer
dri-devel@lists.freedesktop.org
-
Ard Biesheuvel -
Michel Dänzer