This series modifies the NV5x+ nouveau display backends to advertise appropriate format modifiers on their display planes in atomic mode setting blobs.
Corresponding modifications to Mesa/userspace are available here:
https://gitlab.freedesktop.org/cubanismo/mesa/tree/nouveau_work
But those need a bit of cleanup before they're ready to submit.
I've tested this on Tesla, Kepler, Pascal, and Turing-class hardware using various formats and all the exposed format modifiers, plus some negative testing with invalid ones.
NOTE: this series depends on the "[PATCH v3] drm: Generalized NV Block Linear DRM format mod" patch submitted to dri-devel.
v2: Used Tesla family instead of NV50 chipset compare to avoid treating oddly numbered NV4x-class chipsets as NV50+ GPUs. Other instances of compares with chipset number in the series were audited, deemed safe, and left as-is for consistency with existing code.
James Jones (3): drm/nouveau: Add format mod prop to base/ovly/nvdisp drm/nouveau: Check framebuffer size against bo drm/nouveau: Support NVIDIA format modifiers
drivers/gpu/drm/nouveau/dispnv50/base507c.c | 7 +- drivers/gpu/drm/nouveau/dispnv50/disp.c | 59 ++++++++ drivers/gpu/drm/nouveau/dispnv50/disp.h | 4 + drivers/gpu/drm/nouveau/dispnv50/wndw.c | 35 ++++- drivers/gpu/drm/nouveau/dispnv50/wndwc57e.c | 17 +++ drivers/gpu/drm/nouveau/nouveau_display.c | 154 ++++++++++++++++++++ drivers/gpu/drm/nouveau/nouveau_display.h | 4 + 7 files changed, 272 insertions(+), 8 deletions(-)