This series reworks i915's internal handling of slice/subslice/EU (SSEU) data to represent platforms like Xe_HP in a more natural manner and to prepare for future platforms where the masks will need to grow in size. One key idea of this series is that although we have a fixed ABI to convey SSEU data to userspace (i.e., multiple u8[] arrays with data stored at different strides), we don't need to use this cumbersome format for the driver's own internal storage. As long as we can convert into the uapi form properly when responding to the I915_QUERY ioctl, it's preferable to use an internal storage format that's easier for the driver to work with.
Another key point here is that we're reaching the point where subslice (DSS) masks will soon not fit within simple u32/u64 integer values. Xe_HP SDV and DG2 platforms today have subslice (DSS) masks that are 32 bits, which maxes out the current storage of a u32. With PVC the masks are represented by a pair of 32-bit registers, requiring a bump up to at least 64-bits of storage internally. We could switch to u64 for that in the short term, but since we already know that upcoming architectures intend to provide DSS fuse bits via three or more registers it's best to switch to a representation that's more future-proof but still easy to work with in the driver code. To accomodate this, we start storing our subslice mask for Xe_HP and beyond in a new typedef that can be processed by the linux/bitmap.h operations.
Finally, since no userspace for Xe_HP or beyond is using the legacy I915_GETPARAM ioctl lookups for I915_PARAM_SLICE_MASK and I915_PARAM_SUBSLICE_MASK (since they've migrated to the more flexible I915_QUERY ioctl that can return more than a simple u32 value), we take the opportunity to officially drop support for those GETPARAM lookups on modern platforms. Maintaining support for these GETPARAM lookups don't make sense for a number of reasons:
* Traditional slices no longer exist, and newer ideas like gslices, cslices, mslices, etc. aren't something userspace needs to query since it can be inferred from other information. * The GETPARAM ioctl doesn't have a way to distinguish between geometry subslice masks and compute subslice masks, which are distinct on Xe_HP and beyond. * The I915_GETPARAM ioctl is limited to returning a 32-bit value, so when subslice masks begin to exceed 32-bits (on PVC), it simply can't return the entire mask. * The GETPARAM ioctl doesn't have a way to give sensible information for multi-tile devices.
v2: - Switch to union of hsw/xehp formats to keep the representation in a natural format for different types of hardware. - Avoid accessing internals of intel_sseu_ss_mask_t directly outside of intel_sseu.[ch]. - Include PVC SSEU which needs the larger SS mask storage enabled by this series.
v3: - Correct a BIT(s) typo that should have been BIT(ss), causing incorrect handling on gen9 platforms.
v4: - Eliminate sseu->{ss,eu}_stride fields and just calculate the proper value in the UAPI code that needs them. - Handle unwanted ~u8 sign extension at the callsite instead of inside sseu_set_eus. - Use BITMAP_BITS() macro rather than passing I915_MAX_SS_FUSE_BITS around directly to bitmap operations. - Improved debugfs / dmesg reporting for Xe_HP dumps - Various assertion check improvements.
Cc: Tvrtko Ursulin tvrtko.ursulin@linux.intel.com
Matt Roper (6): drm/i915/xehp: Use separate sseu init function drm/i915/xehp: Drop GETPARAM lookups of I915_PARAM_[SUB]SLICE_MASK drm/i915/sseu: Simplify gen11+ SSEU handling drm/i915/sseu: Don't try to store EU mask internally in UAPI format drm/i915/sseu: Disassociate internal subslice mask representation from uapi drm/i915/pvc: Add SSEU changes
drivers/gpu/drm/i915/gem/i915_gem_context.c | 5 +- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 4 +- drivers/gpu/drm/i915/gt/intel_gt.c | 12 +- drivers/gpu/drm/i915/gt/intel_gt_regs.h | 1 + drivers/gpu/drm/i915/gt/intel_sseu.c | 450 ++++++++++++------- drivers/gpu/drm/i915/gt/intel_sseu.h | 94 ++-- drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c | 30 +- drivers/gpu/drm/i915/gt/intel_workarounds.c | 24 +- drivers/gpu/drm/i915/i915_drv.h | 2 + drivers/gpu/drm/i915/i915_getparam.c | 11 +- drivers/gpu/drm/i915/i915_pci.c | 3 +- drivers/gpu/drm/i915/i915_query.c | 26 +- drivers/gpu/drm/i915/intel_device_info.h | 1 + 13 files changed, 398 insertions(+), 265 deletions(-)
Xe_HP has enough fundamental differences from previous platforms that it makes sense to use a separate SSEU init function to keep things straightforward and easy to understand. We'll also add a has_xehp_dss flag to the SSEU structure that will be used by other upcoming changes.
v2: - Add has_xehp_dss flag
Signed-off-by: Matt Roper matthew.d.roper@intel.com --- drivers/gpu/drm/i915/gt/intel_sseu.c | 86 ++++++++++++++++------------ drivers/gpu/drm/i915/gt/intel_sseu.h | 5 ++ 2 files changed, 54 insertions(+), 37 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.c b/drivers/gpu/drm/i915/gt/intel_sseu.c index fdd25691beda..b5fd479a7b85 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu.c +++ b/drivers/gpu/drm/i915/gt/intel_sseu.c @@ -169,13 +169,43 @@ static void gen11_compute_sseu_info(struct sseu_dev_info *sseu, u8 s_en, sseu->eu_total = compute_eu_total(sseu); }
-static void gen12_sseu_info_init(struct intel_gt *gt) +static void xehp_sseu_info_init(struct intel_gt *gt) { struct sseu_dev_info *sseu = >->info.sseu; struct intel_uncore *uncore = gt->uncore; u32 g_dss_en, c_dss_en = 0; u16 eu_en = 0; u8 eu_en_fuse; + int eu; + + /* + * The concept of slice has been removed in Xe_HP. To be compatible + * with prior generations, assume a single slice across the entire + * device. Then calculate out the DSS for each workload type within + * that software slice. + */ + intel_sseu_set_info(sseu, 1, 32, 16); + sseu->has_xehp_dss = 1; + + g_dss_en = intel_uncore_read(uncore, GEN12_GT_GEOMETRY_DSS_ENABLE); + c_dss_en = intel_uncore_read(uncore, GEN12_GT_COMPUTE_DSS_ENABLE); + + eu_en_fuse = intel_uncore_read(uncore, XEHP_EU_ENABLE) & XEHP_EU_ENA_MASK; + + for (eu = 0; eu < sseu->max_eus_per_subslice / 2; eu++) + if (eu_en_fuse & BIT(eu)) + eu_en |= BIT(eu * 2) | BIT(eu * 2 + 1); + + gen11_compute_sseu_info(sseu, 0x1, g_dss_en, c_dss_en, eu_en); +} + +static void gen12_sseu_info_init(struct intel_gt *gt) +{ + struct sseu_dev_info *sseu = >->info.sseu; + struct intel_uncore *uncore = gt->uncore; + u32 g_dss_en; + u16 eu_en = 0; + u8 eu_en_fuse; u8 s_en; int eu;
@@ -183,43 +213,23 @@ static void gen12_sseu_info_init(struct intel_gt *gt) * Gen12 has Dual-Subslices, which behave similarly to 2 gen11 SS. * Instead of splitting these, provide userspace with an array * of DSS to more closely represent the hardware resource. - * - * In addition, the concept of slice has been removed in Xe_HP. - * To be compatible with prior generations, assume a single slice - * across the entire device. Then calculate out the DSS for each - * workload type within that software slice. */ - if (IS_DG2(gt->i915) || IS_XEHPSDV(gt->i915)) - intel_sseu_set_info(sseu, 1, 32, 16); - else - intel_sseu_set_info(sseu, 1, 6, 16); + intel_sseu_set_info(sseu, 1, 6, 16);
- /* - * As mentioned above, Xe_HP does not have the concept of a slice. - * Enable one for software backwards compatibility. - */ - if (GRAPHICS_VER_FULL(gt->i915) >= IP_VER(12, 50)) - s_en = 0x1; - else - s_en = intel_uncore_read(uncore, GEN11_GT_SLICE_ENABLE) & - GEN11_GT_S_ENA_MASK; + s_en = intel_uncore_read(uncore, GEN11_GT_SLICE_ENABLE) & + GEN11_GT_S_ENA_MASK;
g_dss_en = intel_uncore_read(uncore, GEN12_GT_GEOMETRY_DSS_ENABLE); - if (GRAPHICS_VER_FULL(gt->i915) >= IP_VER(12, 50)) - c_dss_en = intel_uncore_read(uncore, GEN12_GT_COMPUTE_DSS_ENABLE);
/* one bit per pair of EUs */ - if (GRAPHICS_VER_FULL(gt->i915) >= IP_VER(12, 50)) - eu_en_fuse = intel_uncore_read(uncore, XEHP_EU_ENABLE) & XEHP_EU_ENA_MASK; - else - eu_en_fuse = ~(intel_uncore_read(uncore, GEN11_EU_DISABLE) & - GEN11_EU_DIS_MASK); + eu_en_fuse = ~(intel_uncore_read(uncore, GEN11_EU_DISABLE) & + GEN11_EU_DIS_MASK);
for (eu = 0; eu < sseu->max_eus_per_subslice / 2; eu++) if (eu_en_fuse & BIT(eu)) eu_en |= BIT(eu * 2) | BIT(eu * 2 + 1);
- gen11_compute_sseu_info(sseu, s_en, g_dss_en, c_dss_en, eu_en); + gen11_compute_sseu_info(sseu, s_en, g_dss_en, 0, eu_en);
/* TGL only supports slice-level power gating */ sseu->has_slice_pg = 1; @@ -574,18 +584,20 @@ void intel_sseu_info_init(struct intel_gt *gt) { struct drm_i915_private *i915 = gt->i915;
- if (IS_HASWELL(i915)) - hsw_sseu_info_init(gt); - else if (IS_CHERRYVIEW(i915)) - cherryview_sseu_info_init(gt); - else if (IS_BROADWELL(i915)) - bdw_sseu_info_init(gt); - else if (GRAPHICS_VER(i915) == 9) - gen9_sseu_info_init(gt); - else if (GRAPHICS_VER(i915) == 11) - gen11_sseu_info_init(gt); + if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 50)) + xehp_sseu_info_init(gt); else if (GRAPHICS_VER(i915) >= 12) gen12_sseu_info_init(gt); + else if (GRAPHICS_VER(i915) >= 11) + gen11_sseu_info_init(gt); + else if (GRAPHICS_VER(i915) >= 9) + gen9_sseu_info_init(gt); + else if (IS_BROADWELL(i915)) + bdw_sseu_info_init(gt); + else if (IS_CHERRYVIEW(i915)) + cherryview_sseu_info_init(gt); + else if (IS_HASWELL(i915)) + hsw_sseu_info_init(gt); }
u32 intel_sseu_make_rpcs(struct intel_gt *gt, diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.h b/drivers/gpu/drm/i915/gt/intel_sseu.h index 5c078df4729c..4a041f9dc490 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu.h +++ b/drivers/gpu/drm/i915/gt/intel_sseu.h @@ -66,6 +66,11 @@ struct sseu_dev_info { u8 has_slice_pg:1; u8 has_subslice_pg:1; u8 has_eu_pg:1; + /* + * For Xe_HP and beyond, the hardware no longer has traditional slices + * so we just report the entire DSS pool under a fake "slice 0." + */ + u8 has_xehp_dss:1;
/* Topology fields */ u8 max_slices;
Slice/subslice/EU information should be obtained via the topology queries provided by the I915_QUERY interface; let's turn off support for the old GETPARAM lookups on Xe_HP and beyond where we can't return meaningful values.
The slice mask lookup is meaningless since Xe_HP doesn't support traditional slices (and we make no attempt to return the various new units like gslices, cslices, mslices, etc.) here.
The subslice mask lookup is even more problematic; given the distinct masks for geometry vs compute purposes, the combined mask returned here is likely not what userspace would want to act upon anyway. The value is also limited to 32-bits by the nature of the GETPARAM ioctl which is sufficient for the initial Xe_HP platforms, but is unable to convey the larger masks that will be needed on other upcoming platforms. Finally, the value returned here becomes even less meaningful when used on multi-tile platforms where each tile will have its own masks.
Signed-off-by: Matt Roper matthew.d.roper@intel.com --- drivers/gpu/drm/i915/i915_getparam.c | 8 ++++++++ 1 file changed, 8 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_getparam.c b/drivers/gpu/drm/i915/i915_getparam.c index c12a0adefda5..ac9767c56619 100644 --- a/drivers/gpu/drm/i915/i915_getparam.c +++ b/drivers/gpu/drm/i915/i915_getparam.c @@ -148,11 +148,19 @@ int i915_getparam_ioctl(struct drm_device *dev, void *data, value = intel_engines_has_context_isolation(i915); break; case I915_PARAM_SLICE_MASK: + /* Not supported from Xe_HP onward; use topology queries */ + if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 50)) + return -EINVAL; + value = sseu->slice_mask; if (!value) return -ENODEV; break; case I915_PARAM_SUBSLICE_MASK: + /* Not supported from Xe_HP onward; use topology queries */ + if (GRAPHICS_VER_FULL(i915) >= IP_VER(12, 50)) + return -EINVAL; + /* Only copy bits from the first slice */ memcpy(&value, sseu->subslice_mask, min(sseu->ss_stride, (u8)sizeof(value)));
Although gen11 and gen12 architectures supported the concept of multiple slices, in practice all the platforms that were actually designed only had a single slice (i.e., note the parameters to 'intel_sseu_set_info' that we pass for each platform). We can simplify the code slightly by dropping the multi-slice logic from gen11+ platforms.
v2: - Promote drm_dbg to drm_WARN_ON if the slice fuse register reports unexpected fusing. (Tvrtko)
Cc: Tvrtko Ursulin tvrtko.ursulin@linux.intel.com Signed-off-by: Matt Roper matthew.d.roper@intel.com --- drivers/gpu/drm/i915/gt/intel_sseu.c | 76 +++++++++++++--------------- 1 file changed, 36 insertions(+), 40 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.c b/drivers/gpu/drm/i915/gt/intel_sseu.c index b5fd479a7b85..7e5222ad2f98 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu.c +++ b/drivers/gpu/drm/i915/gt/intel_sseu.c @@ -119,52 +119,37 @@ static u16 compute_eu_total(const struct sseu_dev_info *sseu) return total; }
-static u32 get_ss_stride_mask(struct sseu_dev_info *sseu, u8 s, u32 ss_en) -{ - u32 ss_mask; - - ss_mask = ss_en >> (s * sseu->max_subslices); - ss_mask &= GENMASK(sseu->max_subslices - 1, 0); - - return ss_mask; -} - -static void gen11_compute_sseu_info(struct sseu_dev_info *sseu, u8 s_en, +static void gen11_compute_sseu_info(struct sseu_dev_info *sseu, u32 g_ss_en, u32 c_ss_en, u16 eu_en) { - int s, ss; + u32 valid_ss_mask = GENMASK(sseu->max_subslices - 1, 0); + int ss;
/* g_ss_en/c_ss_en represent entire subslice mask across all slices */ GEM_BUG_ON(sseu->max_slices * sseu->max_subslices > sizeof(g_ss_en) * BITS_PER_BYTE);
- for (s = 0; s < sseu->max_slices; s++) { - if ((s_en & BIT(s)) == 0) - continue; + sseu->slice_mask |= BIT(0);
- sseu->slice_mask |= BIT(s); - - /* - * XeHP introduces the concept of compute vs geometry DSS. To - * reduce variation between GENs around subslice usage, store a - * mask for both the geometry and compute enabled masks since - * userspace will need to be able to query these masks - * independently. Also compute a total enabled subslice count - * for the purposes of selecting subslices to use in a - * particular GEM context. - */ - intel_sseu_set_subslices(sseu, s, sseu->compute_subslice_mask, - get_ss_stride_mask(sseu, s, c_ss_en)); - intel_sseu_set_subslices(sseu, s, sseu->geometry_subslice_mask, - get_ss_stride_mask(sseu, s, g_ss_en)); - intel_sseu_set_subslices(sseu, s, sseu->subslice_mask, - get_ss_stride_mask(sseu, s, - g_ss_en | c_ss_en)); + /* + * XeHP introduces the concept of compute vs geometry DSS. To reduce + * variation between GENs around subslice usage, store a mask for both + * the geometry and compute enabled masks since userspace will need to + * be able to query these masks independently. Also compute a total + * enabled subslice count for the purposes of selecting subslices to + * use in a particular GEM context. + */ + intel_sseu_set_subslices(sseu, 0, sseu->compute_subslice_mask, + c_ss_en & valid_ss_mask); + intel_sseu_set_subslices(sseu, 0, sseu->geometry_subslice_mask, + g_ss_en & valid_ss_mask); + intel_sseu_set_subslices(sseu, 0, sseu->subslice_mask, + (g_ss_en | c_ss_en) & valid_ss_mask); + + for (ss = 0; ss < sseu->max_subslices; ss++) + if (intel_sseu_has_subslice(sseu, 0, ss)) + sseu_set_eus(sseu, 0, ss, eu_en);
- for (ss = 0; ss < sseu->max_subslices; ss++) - if (intel_sseu_has_subslice(sseu, s, ss)) - sseu_set_eus(sseu, s, ss, eu_en); - } sseu->eu_per_subslice = hweight16(eu_en); sseu->eu_total = compute_eu_total(sseu); } @@ -196,7 +181,7 @@ static void xehp_sseu_info_init(struct intel_gt *gt) if (eu_en_fuse & BIT(eu)) eu_en |= BIT(eu * 2) | BIT(eu * 2 + 1);
- gen11_compute_sseu_info(sseu, 0x1, g_dss_en, c_dss_en, eu_en); + gen11_compute_sseu_info(sseu, g_dss_en, c_dss_en, eu_en); }
static void gen12_sseu_info_init(struct intel_gt *gt) @@ -216,8 +201,13 @@ static void gen12_sseu_info_init(struct intel_gt *gt) */ intel_sseu_set_info(sseu, 1, 6, 16);
+ /* + * Although gen12 architecture supported multiple slices, TGL, RKL, + * DG1, and ADL only had a single slice. + */ s_en = intel_uncore_read(uncore, GEN11_GT_SLICE_ENABLE) & GEN11_GT_S_ENA_MASK; + drm_WARN_ON(>->i915->drm, s_en != 0x1);
g_dss_en = intel_uncore_read(uncore, GEN12_GT_GEOMETRY_DSS_ENABLE);
@@ -229,7 +219,7 @@ static void gen12_sseu_info_init(struct intel_gt *gt) if (eu_en_fuse & BIT(eu)) eu_en |= BIT(eu * 2) | BIT(eu * 2 + 1);
- gen11_compute_sseu_info(sseu, s_en, g_dss_en, 0, eu_en); + gen11_compute_sseu_info(sseu, g_dss_en, 0, eu_en);
/* TGL only supports slice-level power gating */ sseu->has_slice_pg = 1; @@ -248,14 +238,20 @@ static void gen11_sseu_info_init(struct intel_gt *gt) else intel_sseu_set_info(sseu, 1, 8, 8);
+ /* + * Although gen11 architecture supported multiple slices, ICL and + * EHL/JSL only had a single slice in practice. + */ s_en = intel_uncore_read(uncore, GEN11_GT_SLICE_ENABLE) & GEN11_GT_S_ENA_MASK; + drm_WARN_ON(>->i915->drm, s_en != 0x1); + ss_en = ~intel_uncore_read(uncore, GEN11_GT_SUBSLICE_DISABLE);
eu_en = ~(intel_uncore_read(uncore, GEN11_EU_DISABLE) & GEN11_EU_DIS_MASK);
- gen11_compute_sseu_info(sseu, s_en, ss_en, 0, eu_en); + gen11_compute_sseu_info(sseu, ss_en, 0, eu_en);
/* ICL has no power gating restrictions. */ sseu->has_slice_pg = 1;
Storing the EU mask internally in the same format the I915_QUERY topology queries use makes the final copy_to_user() a bit simpler, but makes the rest of the driver's SSEU more complicated and harder to follow. Let's switch to an internal representation that's more natural: Xe_HP platforms will be a simple array of u16 masks, whereas pre-Xe_HP platforms will be a two-dimensional array, indexed by [slice][subslice]. We'll convert to the uapi format only when the query uapi is called.
v2: - Drop has_common_ss_eumask. We waste some space repeating identical EU masks for every single DSS, but the code is simpler without it. (Tvrtko)
v3: - Mask down EUs passed to sseu_set_eus at the callsite rather than inside the function. (Tvrtko) - Eliminate sseu->eu_stride and calculate it when needed. (Tvrtko)
Cc: Tvrtko Ursulin tvrtko.ursulin@linux.intel.com Signed-off-by: Matt Roper matthew.d.roper@intel.com --- drivers/gpu/drm/i915/gt/intel_sseu.c | 88 ++++++++++++++++++---------- drivers/gpu/drm/i915/gt/intel_sseu.h | 10 +++- drivers/gpu/drm/i915/i915_query.c | 13 ++-- 3 files changed, 73 insertions(+), 38 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.c b/drivers/gpu/drm/i915/gt/intel_sseu.c index 7e5222ad2f98..285cfd758bdc 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu.c +++ b/drivers/gpu/drm/i915/gt/intel_sseu.c @@ -19,8 +19,6 @@ void intel_sseu_set_info(struct sseu_dev_info *sseu, u8 max_slices,
sseu->ss_stride = GEN_SSEU_STRIDE(sseu->max_subslices); GEM_BUG_ON(sseu->ss_stride > GEN_MAX_SUBSLICE_STRIDE); - sseu->eu_stride = GEN_SSEU_STRIDE(sseu->max_eus_per_subslice); - GEM_BUG_ON(sseu->eu_stride > GEN_MAX_EU_STRIDE); }
unsigned int @@ -78,47 +76,77 @@ intel_sseu_subslices_per_slice(const struct sseu_dev_info *sseu, u8 slice) return hweight32(intel_sseu_get_subslices(sseu, slice)); }
-static int sseu_eu_idx(const struct sseu_dev_info *sseu, int slice, - int subslice) -{ - int slice_stride = sseu->max_subslices * sseu->eu_stride; - - return slice * slice_stride + subslice * sseu->eu_stride; -} - static u16 sseu_get_eus(const struct sseu_dev_info *sseu, int slice, int subslice) { - int i, offset = sseu_eu_idx(sseu, slice, subslice); - u16 eu_mask = 0; - - for (i = 0; i < sseu->eu_stride; i++) - eu_mask |= - ((u16)sseu->eu_mask[offset + i]) << (i * BITS_PER_BYTE); - - return eu_mask; + if (sseu->has_xehp_dss) { + WARN_ON(slice > 0); + return sseu->eu_mask.xehp[subslice]; + } else { + return sseu->eu_mask.hsw[slice][subslice]; + } }
static void sseu_set_eus(struct sseu_dev_info *sseu, int slice, int subslice, u16 eu_mask) { - int i, offset = sseu_eu_idx(sseu, slice, subslice); - - for (i = 0; i < sseu->eu_stride; i++) - sseu->eu_mask[offset + i] = - (eu_mask >> (BITS_PER_BYTE * i)) & 0xff; + GEM_WARN_ON(eu_mask && __fls(eu_mask) >= sseu->max_eus_per_subslice); + if (sseu->has_xehp_dss) { + GEM_WARN_ON(slice > 0); + sseu->eu_mask.xehp[subslice] = eu_mask; + } else { + sseu->eu_mask.hsw[slice][subslice] = eu_mask; + } }
static u16 compute_eu_total(const struct sseu_dev_info *sseu) { - u16 i, total = 0; + int s, ss, total = 0;
- for (i = 0; i < ARRAY_SIZE(sseu->eu_mask); i++) - total += hweight8(sseu->eu_mask[i]); + for (s = 0; s < sseu->max_slices; s++) + for (ss = 0; ss < sseu->max_subslices; ss++) + if (sseu->has_xehp_dss) + total += hweight16(sseu->eu_mask.xehp[ss]); + else + total += hweight16(sseu->eu_mask.hsw[s][ss]);
return total; }
+/** + * intel_sseu_copy_eumask_to_user - Copy EU mask into a userspace buffer + * @to: Pointer to userspace buffer to copy to + * @sseu: SSEU structure containing EU mask to copy + * + * Copies the EU mask to a userspace buffer in the format expected by + * the query ioctl's topology queries. + * + * Returns the result of the copy_to_user() operation. + */ +int intel_sseu_copy_eumask_to_user(void __user *to, + const struct sseu_dev_info *sseu) +{ + u8 eu_mask[GEN_SS_MASK_SIZE * GEN_MAX_EU_STRIDE] = {}; + int eu_stride = GEN_SSEU_STRIDE(sseu->max_eus_per_subslice); + int len = sseu->max_slices * sseu->max_subslices * eu_stride; + int s, ss, i; + + for (s = 0; s < sseu->max_slices; s++) { + for (ss = 0; ss < sseu->max_subslices; ss++) { + int uapi_offset = + s * sseu->max_subslices * eu_stride + + ss * eu_stride; + u16 mask = sseu_get_eus(sseu, s, ss); + + for (i = 0; i < eu_stride; i++) + eu_mask[uapi_offset + i] = + (mask >> (BITS_PER_BYTE * i)) & 0xff; + } + } + + return copy_to_user(to, eu_mask, len); +} + static void gen11_compute_sseu_info(struct sseu_dev_info *sseu, u32 g_ss_en, u32 c_ss_en, u16 eu_en) { @@ -278,7 +306,7 @@ static void cherryview_sseu_info_init(struct intel_gt *gt) CHV_FGT_EU_DIS_SS0_R1_SHIFT) << 4);
subslice_mask |= BIT(0); - sseu_set_eus(sseu, 0, 0, ~disabled_mask); + sseu_set_eus(sseu, 0, 0, ~disabled_mask & 0xFF); }
if (!(fuse & CHV_FGT_DISABLE_SS1)) { @@ -289,7 +317,7 @@ static void cherryview_sseu_info_init(struct intel_gt *gt) CHV_FGT_EU_DIS_SS1_R1_SHIFT) << 4);
subslice_mask |= BIT(1); - sseu_set_eus(sseu, 0, 1, ~disabled_mask); + sseu_set_eus(sseu, 0, 1, ~disabled_mask & 0xFF); }
intel_sseu_set_subslices(sseu, 0, sseu->subslice_mask, subslice_mask); @@ -362,7 +390,7 @@ static void gen9_sseu_info_init(struct intel_gt *gt)
eu_disabled_mask = (eu_disable >> (ss * 8)) & eu_mask;
- sseu_set_eus(sseu, s, ss, ~eu_disabled_mask); + sseu_set_eus(sseu, s, ss, ~eu_disabled_mask & eu_mask);
eu_per_ss = sseu->max_eus_per_subslice - hweight8(eu_disabled_mask); @@ -475,7 +503,7 @@ static void bdw_sseu_info_init(struct intel_gt *gt) eu_disabled_mask = eu_disable[s] >> (ss * sseu->max_eus_per_subslice);
- sseu_set_eus(sseu, s, ss, ~eu_disabled_mask); + sseu_set_eus(sseu, s, ss, ~eu_disabled_mask & 0xFF);
n_disabled = hweight8(eu_disabled_mask);
diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.h b/drivers/gpu/drm/i915/gt/intel_sseu.h index 4a041f9dc490..ffa375e68959 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu.h +++ b/drivers/gpu/drm/i915/gt/intel_sseu.h @@ -57,7 +57,11 @@ struct sseu_dev_info { u8 subslice_mask[GEN_SS_MASK_SIZE]; u8 geometry_subslice_mask[GEN_SS_MASK_SIZE]; u8 compute_subslice_mask[GEN_SS_MASK_SIZE]; - u8 eu_mask[GEN_SS_MASK_SIZE * GEN_MAX_EU_STRIDE]; + union { + u16 hsw[GEN_MAX_HSW_SLICES][GEN_MAX_SS_PER_HSW_SLICE]; + u16 xehp[GEN_MAX_DSS]; + } eu_mask; + u16 eu_total; u8 eu_per_subslice; u8 min_eu_in_pool; @@ -78,7 +82,6 @@ struct sseu_dev_info { u8 max_eus_per_subslice;
u8 ss_stride; - u8 eu_stride; };
/* @@ -150,4 +153,7 @@ void intel_sseu_print_topology(struct drm_i915_private *i915,
u16 intel_slicemask_from_dssmask(u64 dss_mask, int dss_per_slice);
+int intel_sseu_copy_eumask_to_user(void __user *to, + const struct sseu_dev_info *sseu); + #endif /* __INTEL_SSEU_H__ */ diff --git a/drivers/gpu/drm/i915/i915_query.c b/drivers/gpu/drm/i915/i915_query.c index 7584cec53d5d..89c475d525b8 100644 --- a/drivers/gpu/drm/i915/i915_query.c +++ b/drivers/gpu/drm/i915/i915_query.c @@ -35,6 +35,7 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, { struct drm_i915_query_topology_info topo; u32 slice_length, subslice_length, eu_length, total_length; + int eu_stride = GEN_SSEU_STRIDE(sseu->max_eus_per_subslice); int ret;
BUILD_BUG_ON(sizeof(u8) != sizeof(sseu->slice_mask)); @@ -44,7 +45,7 @@ static int fill_topology_info(const struct sseu_dev_info *sseu,
slice_length = sizeof(sseu->slice_mask); subslice_length = sseu->max_slices * sseu->ss_stride; - eu_length = sseu->max_slices * sseu->max_subslices * sseu->eu_stride; + eu_length = sseu->max_slices * sseu->max_subslices * eu_stride; total_length = sizeof(topo) + slice_length + subslice_length + eu_length;
@@ -61,7 +62,7 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, topo.subslice_offset = slice_length; topo.subslice_stride = sseu->ss_stride; topo.eu_offset = slice_length + subslice_length; - topo.eu_stride = sseu->eu_stride; + topo.eu_stride = eu_stride;
if (copy_to_user(u64_to_user_ptr(query_item->data_ptr), &topo, sizeof(topo))) @@ -76,10 +77,10 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, subslice_mask, subslice_length)) return -EFAULT;
- if (copy_to_user(u64_to_user_ptr(query_item->data_ptr + - sizeof(topo) + - slice_length + subslice_length), - sseu->eu_mask, eu_length)) + if (intel_sseu_copy_eumask_to_user(u64_to_user_ptr(query_item->data_ptr + + sizeof(topo) + + slice_length + subslice_length), + sseu)) return -EFAULT;
return total_length;
As with EU masks, it's easier to store subslice/DSS masks internally in a format that's more natural for the driver to work with, and then only covert into the u8[] uapi form when the query ioctl is invoked. Since the hardware design changed significantly with Xe_HP, we'll use a union to choose between the old "hsw-style" subslice masks or the newer xehp mask. HSW-style masks will be stored in an array of u8's, indexed by slice (there's never more than 6 subslices per slice on older platforms). For Xe_HP and beyond where slices no longer exist, we only need a single bitmask. However we already know that this mask is eventually going to grow too large for a simple u64 to hold, so we'll represent it in a manner that can be operated on by the utilities in linux/bitmap.h.
v2: - Fix typo: BIT(s) -> BIT(ss) in gen9_sseu_device_status()
v3: - Eliminate sseu->ss_stride and just calculate the stride while specifically handling uapi. (Tvrtko) - Use BITMAP_BITS() macro to refer to size of masks rather than passing I915_MAX_SS_FUSE_BITS directly. (Tvrtko) - Report compute/geometry DSS masks separately when dumping Xe_HP SSEU info. (Tvrtko) - Restore dropped range checks to intel_sseu_has_subslice(). (Tvrtko)
Cc: Tvrtko Ursulin tvrtko.ursulin@linux.intel.com Signed-off-by: Matt Roper matthew.d.roper@intel.com --- drivers/gpu/drm/i915/gem/i915_gem_context.c | 5 +- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 4 +- drivers/gpu/drm/i915/gt/intel_gt.c | 12 +- drivers/gpu/drm/i915/gt/intel_sseu.c | 261 +++++++++++-------- drivers/gpu/drm/i915/gt/intel_sseu.h | 81 +++--- drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c | 30 +-- drivers/gpu/drm/i915/gt/intel_workarounds.c | 24 +- drivers/gpu/drm/i915/i915_getparam.c | 3 +- drivers/gpu/drm/i915/i915_query.c | 13 +- 9 files changed, 244 insertions(+), 189 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index ab4c5ab28e4d..a3bb73f5d53b 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -1875,6 +1875,7 @@ i915_gem_user_to_context_sseu(struct intel_gt *gt, { const struct sseu_dev_info *device = >->info.sseu; struct drm_i915_private *i915 = gt->i915; + unsigned int dev_subslice_mask = intel_sseu_get_hsw_subslices(device, 0);
/* No zeros in any field. */ if (!user->slice_mask || !user->subslice_mask || @@ -1901,7 +1902,7 @@ i915_gem_user_to_context_sseu(struct intel_gt *gt, if (user->slice_mask & ~device->slice_mask) return -EINVAL;
- if (user->subslice_mask & ~device->subslice_mask[0]) + if (user->subslice_mask & ~dev_subslice_mask) return -EINVAL;
if (user->max_eus_per_subslice > device->max_eus_per_subslice) @@ -1915,7 +1916,7 @@ i915_gem_user_to_context_sseu(struct intel_gt *gt, /* Part specific restrictions. */ if (GRAPHICS_VER(i915) == 11) { unsigned int hw_s = hweight8(device->slice_mask); - unsigned int hw_ss_per_s = hweight8(device->subslice_mask[0]); + unsigned int hw_ss_per_s = hweight8(dev_subslice_mask); unsigned int req_s = hweight8(context->slice_mask); unsigned int req_ss = hweight8(context->subslice_mask);
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c index 1adbf34c3632..f0acf8518a51 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c @@ -674,8 +674,8 @@ static void engine_mask_apply_compute_fuses(struct intel_gt *gt) if (GRAPHICS_VER_FULL(i915) < IP_VER(12, 50)) return;
- ccs_mask = intel_slicemask_from_dssmask(intel_sseu_get_compute_subslices(&info->sseu), - ss_per_ccs); + ccs_mask = intel_slicemask_from_xehp_dssmask(info->sseu.compute_subslice_mask, + ss_per_ccs); /* * If all DSS in a quadrant are fused off, the corresponding CCS * engine is not available for use. diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c index 034182f85501..2921f510642f 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -133,13 +133,6 @@ static const struct intel_mmio_range dg2_lncf_steering_table[] = { {}, };
-static u16 slicemask(struct intel_gt *gt, int count) -{ - u64 dss_mask = intel_sseu_get_subslices(>->info.sseu, 0); - - return intel_slicemask_from_dssmask(dss_mask, count); -} - int intel_gt_init_mmio(struct intel_gt *gt) { struct drm_i915_private *i915 = gt->i915; @@ -155,9 +148,12 @@ int intel_gt_init_mmio(struct intel_gt *gt) */ if (HAS_MSLICES(i915)) { gt->info.mslice_mask = - slicemask(gt, GEN_DSS_PER_MSLICE) | + intel_slicemask_from_xehp_dssmask(gt->info.sseu.subslice_mask, + GEN_DSS_PER_MSLICE); + gt->info.mslice_mask |= (intel_uncore_read(gt->uncore, GEN10_MIRROR_FUSE3) & GEN12_MEML3_EN_MASK); + if (!gt->info.mslice_mask) /* should be impossible! */ drm_warn(&i915->drm, "mslice mask all zero!\n"); } diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.c b/drivers/gpu/drm/i915/gt/intel_sseu.c index 285cfd758bdc..6fbc2ac507b7 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu.c +++ b/drivers/gpu/drm/i915/gt/intel_sseu.c @@ -10,15 +10,14 @@ #include "intel_gt_regs.h" #include "intel_sseu.h"
+#define BITMAP_BITS(mask) ((int)BITS_PER_TYPE(typeof(mask))) + void intel_sseu_set_info(struct sseu_dev_info *sseu, u8 max_slices, u8 max_subslices, u8 max_eus_per_subslice) { sseu->max_slices = max_slices; sseu->max_subslices = max_subslices; sseu->max_eus_per_subslice = max_eus_per_subslice; - - sseu->ss_stride = GEN_SSEU_STRIDE(sseu->max_subslices); - GEM_BUG_ON(sseu->ss_stride > GEN_MAX_SUBSLICE_STRIDE); }
unsigned int @@ -26,54 +25,24 @@ intel_sseu_subslice_total(const struct sseu_dev_info *sseu) { unsigned int i, total = 0;
- for (i = 0; i < ARRAY_SIZE(sseu->subslice_mask); i++) - total += hweight8(sseu->subslice_mask[i]); - - return total; -} - -static u32 -sseu_get_subslices(const struct sseu_dev_info *sseu, - const u8 *subslice_mask, u8 slice) -{ - int i, offset = slice * sseu->ss_stride; - u32 mask = 0; - - GEM_BUG_ON(slice >= sseu->max_slices); - - for (i = 0; i < sseu->ss_stride; i++) - mask |= (u32)subslice_mask[offset + i] << i * BITS_PER_BYTE; - - return mask; -} - -u32 intel_sseu_get_subslices(const struct sseu_dev_info *sseu, u8 slice) -{ - return sseu_get_subslices(sseu, sseu->subslice_mask, slice); -} + if (sseu->has_xehp_dss) + return bitmap_weight(sseu->subslice_mask.xehp, + BITMAP_BITS(sseu->subslice_mask));
-static u32 sseu_get_geometry_subslices(const struct sseu_dev_info *sseu) -{ - return sseu_get_subslices(sseu, sseu->geometry_subslice_mask, 0); -} + for (i = 0; i < ARRAY_SIZE(sseu->subslice_mask.hsw); i++) + total += hweight8(sseu->subslice_mask.hsw[i]);
-u32 intel_sseu_get_compute_subslices(const struct sseu_dev_info *sseu) -{ - return sseu_get_subslices(sseu, sseu->compute_subslice_mask, 0); -} - -void intel_sseu_set_subslices(struct sseu_dev_info *sseu, int slice, - u8 *subslice_mask, u32 ss_mask) -{ - int offset = slice * sseu->ss_stride; - - memcpy(&subslice_mask[offset], &ss_mask, sseu->ss_stride); + return total; }
unsigned int -intel_sseu_subslices_per_slice(const struct sseu_dev_info *sseu, u8 slice) +intel_sseu_get_hsw_subslices(const struct sseu_dev_info *sseu, u8 slice) { - return hweight32(intel_sseu_get_subslices(sseu, slice)); + WARN_ON(sseu->has_xehp_dss); + if (WARN_ON(slice >= sseu->max_slices)) + return 0; + + return sseu->subslice_mask.hsw[slice]; }
static u16 sseu_get_eus(const struct sseu_dev_info *sseu, int slice, @@ -147,32 +116,66 @@ int intel_sseu_copy_eumask_to_user(void __user *to, return copy_to_user(to, eu_mask, len); }
+/** + * intel_sseu_copy_ssmask_to_user - Copy subslice mask into a userspace buffer + * @to: Pointer to userspace buffer to copy to + * @sseu: SSEU structure containing subslice mask to copy + * + * Copies the subslice mask to a userspace buffer in the format expected by + * the query ioctl's topology queries. + * + * Returns the result of the copy_to_user() operation. + */ +int intel_sseu_copy_ssmask_to_user(void __user *to, + const struct sseu_dev_info *sseu) +{ + u8 ss_mask[GEN_SS_MASK_SIZE] = {}; + int ss_stride = GEN_SSEU_STRIDE(sseu->max_subslices); + int len = sseu->max_slices * ss_stride; + int s, ss, i; + + for (s = 0; s < sseu->max_slices; s++) { + for (ss = 0; ss < sseu->max_subslices; ss++) { + i = s * ss_stride * BITS_PER_BYTE + ss; + + if (!intel_sseu_has_subslice(sseu, s, ss)) + continue; + + ss_mask[i / BITS_PER_BYTE] |= BIT(i % BITS_PER_BYTE); + } + } + + return copy_to_user(to, ss_mask, len); +} + static void gen11_compute_sseu_info(struct sseu_dev_info *sseu, - u32 g_ss_en, u32 c_ss_en, u16 eu_en) + u32 ss_en, u16 eu_en) { u32 valid_ss_mask = GENMASK(sseu->max_subslices - 1, 0); int ss;
- /* g_ss_en/c_ss_en represent entire subslice mask across all slices */ - GEM_BUG_ON(sseu->max_slices * sseu->max_subslices > - sizeof(g_ss_en) * BITS_PER_BYTE); + sseu->slice_mask |= BIT(0); + sseu->subslice_mask.hsw[0] = ss_en & valid_ss_mask; + + for (ss = 0; ss < sseu->max_subslices; ss++) + if (intel_sseu_has_subslice(sseu, 0, ss)) + sseu_set_eus(sseu, 0, ss, eu_en); + + sseu->eu_per_subslice = hweight16(eu_en); + sseu->eu_total = compute_eu_total(sseu); +} + +static void xehp_compute_sseu_info(struct sseu_dev_info *sseu, + u16 eu_en) +{ + int ss;
sseu->slice_mask |= BIT(0);
- /* - * XeHP introduces the concept of compute vs geometry DSS. To reduce - * variation between GENs around subslice usage, store a mask for both - * the geometry and compute enabled masks since userspace will need to - * be able to query these masks independently. Also compute a total - * enabled subslice count for the purposes of selecting subslices to - * use in a particular GEM context. - */ - intel_sseu_set_subslices(sseu, 0, sseu->compute_subslice_mask, - c_ss_en & valid_ss_mask); - intel_sseu_set_subslices(sseu, 0, sseu->geometry_subslice_mask, - g_ss_en & valid_ss_mask); - intel_sseu_set_subslices(sseu, 0, sseu->subslice_mask, - (g_ss_en | c_ss_en) & valid_ss_mask); + bitmap_or(sseu->subslice_mask.xehp, + sseu->compute_subslice_mask.xehp, + sseu->geometry_subslice_mask.xehp, + BITMAP_BITS(sseu->subslice_mask));
for (ss = 0; ss < sseu->max_subslices; ss++) if (intel_sseu_has_subslice(sseu, 0, ss)) @@ -182,11 +185,31 @@ static void gen11_compute_sseu_info(struct sseu_dev_info *sseu, sseu->eu_total = compute_eu_total(sseu); }
+static void +xehp_load_dss_mask(struct intel_uncore *uncore, + intel_sseu_ss_mask_t *ssmask, + int numregs, + ...) +{ + va_list argp; + u32 fuse_val[I915_MAX_SS_FUSE_REGS] = {}; + int i; + + if (WARN_ON(numregs > I915_MAX_SS_FUSE_REGS)) + numregs = I915_MAX_SS_FUSE_REGS; + + va_start(argp, numregs); + for (i = 0; i < numregs; i++) + fuse_val[i] = intel_uncore_read(uncore, va_arg(argp, i915_reg_t)); + va_end(argp); + + bitmap_from_arr32(ssmask->xehp, fuse_val, numregs * 32); +} + static void xehp_sseu_info_init(struct intel_gt *gt) { struct sseu_dev_info *sseu = >->info.sseu; struct intel_uncore *uncore = gt->uncore; - u32 g_dss_en, c_dss_en = 0; u16 eu_en = 0; u8 eu_en_fuse; int eu; @@ -200,8 +223,10 @@ static void xehp_sseu_info_init(struct intel_gt *gt) intel_sseu_set_info(sseu, 1, 32, 16); sseu->has_xehp_dss = 1;
- g_dss_en = intel_uncore_read(uncore, GEN12_GT_GEOMETRY_DSS_ENABLE); - c_dss_en = intel_uncore_read(uncore, GEN12_GT_COMPUTE_DSS_ENABLE); + xehp_load_dss_mask(uncore, &sseu->geometry_subslice_mask, 1, + GEN12_GT_GEOMETRY_DSS_ENABLE); + xehp_load_dss_mask(uncore, &sseu->compute_subslice_mask, 1, + GEN12_GT_COMPUTE_DSS_ENABLE);
eu_en_fuse = intel_uncore_read(uncore, XEHP_EU_ENABLE) & XEHP_EU_ENA_MASK;
@@ -209,7 +234,7 @@ static void xehp_sseu_info_init(struct intel_gt *gt) if (eu_en_fuse & BIT(eu)) eu_en |= BIT(eu * 2) | BIT(eu * 2 + 1);
- gen11_compute_sseu_info(sseu, g_dss_en, c_dss_en, eu_en); + xehp_compute_sseu_info(sseu, eu_en); }
static void gen12_sseu_info_init(struct intel_gt *gt) @@ -247,7 +272,7 @@ static void gen12_sseu_info_init(struct intel_gt *gt) if (eu_en_fuse & BIT(eu)) eu_en |= BIT(eu * 2) | BIT(eu * 2 + 1);
- gen11_compute_sseu_info(sseu, g_dss_en, 0, eu_en); + gen11_compute_sseu_info(sseu, g_dss_en, eu_en);
/* TGL only supports slice-level power gating */ sseu->has_slice_pg = 1; @@ -279,7 +304,7 @@ static void gen11_sseu_info_init(struct intel_gt *gt) eu_en = ~(intel_uncore_read(uncore, GEN11_EU_DISABLE) & GEN11_EU_DIS_MASK);
- gen11_compute_sseu_info(sseu, ss_en, 0, eu_en); + gen11_compute_sseu_info(sseu, ss_en, eu_en);
/* ICL has no power gating restrictions. */ sseu->has_slice_pg = 1; @@ -291,7 +316,6 @@ static void cherryview_sseu_info_init(struct intel_gt *gt) { struct sseu_dev_info *sseu = >->info.sseu; u32 fuse; - u8 subslice_mask = 0;
fuse = intel_uncore_read(gt->uncore, CHV_FUSE_GT);
@@ -305,7 +329,7 @@ static void cherryview_sseu_info_init(struct intel_gt *gt) (((fuse & CHV_FGT_EU_DIS_SS0_R1_MASK) >> CHV_FGT_EU_DIS_SS0_R1_SHIFT) << 4);
- subslice_mask |= BIT(0); + sseu->subslice_mask.hsw[0] |= BIT(0); sseu_set_eus(sseu, 0, 0, ~disabled_mask & 0xFF); }
@@ -316,12 +340,10 @@ static void cherryview_sseu_info_init(struct intel_gt *gt) (((fuse & CHV_FGT_EU_DIS_SS1_R1_MASK) >> CHV_FGT_EU_DIS_SS1_R1_SHIFT) << 4);
- subslice_mask |= BIT(1); + sseu->subslice_mask.hsw[0] |= BIT(1); sseu_set_eus(sseu, 0, 1, ~disabled_mask & 0xFF); }
- intel_sseu_set_subslices(sseu, 0, sseu->subslice_mask, subslice_mask); - sseu->eu_total = compute_eu_total(sseu);
/* @@ -376,8 +398,7 @@ static void gen9_sseu_info_init(struct intel_gt *gt) /* skip disabled slice */ continue;
- intel_sseu_set_subslices(sseu, s, sseu->subslice_mask, - subslice_mask); + sseu->subslice_mask.hsw[s] = subslice_mask;
eu_disable = intel_uncore_read(uncore, GEN9_EU_DISABLE(s)); for (ss = 0; ss < sseu->max_subslices; ss++) { @@ -434,8 +455,8 @@ static void gen9_sseu_info_init(struct intel_gt *gt) sseu->has_eu_pg = sseu->eu_per_subslice > 2;
if (IS_GEN9_LP(i915)) { -#define IS_SS_DISABLED(ss) (!(sseu->subslice_mask[0] & BIT(ss))) - info->has_pooled_eu = hweight8(sseu->subslice_mask[0]) == 3; +#define IS_SS_DISABLED(ss) (!(sseu->subslice_mask.hsw[0] & BIT(ss))) + info->has_pooled_eu = hweight8(sseu->subslice_mask.hsw[0]) == 3;
sseu->min_eu_in_pool = 0; if (info->has_pooled_eu) { @@ -489,8 +510,7 @@ static void bdw_sseu_info_init(struct intel_gt *gt) /* skip disabled slice */ continue;
- intel_sseu_set_subslices(sseu, s, sseu->subslice_mask, - subslice_mask); + sseu->subslice_mask.hsw[s] = subslice_mask;
for (ss = 0; ss < sseu->max_subslices; ss++) { u8 eu_disabled_mask; @@ -587,8 +607,7 @@ static void hsw_sseu_info_init(struct intel_gt *gt) sseu->eu_per_subslice);
for (s = 0; s < sseu->max_slices; s++) { - intel_sseu_set_subslices(sseu, s, sseu->subslice_mask, - subslice_mask); + sseu->subslice_mask.hsw[s] = subslice_mask;
for (ss = 0; ss < sseu->max_subslices; ss++) { sseu_set_eus(sseu, s, ss, @@ -677,7 +696,7 @@ u32 intel_sseu_make_rpcs(struct intel_gt *gt, */ if (GRAPHICS_VER(i915) == 11 && slices == 1 && - subslices > min_t(u8, 4, hweight8(sseu->subslice_mask[0]) / 2)) { + subslices > min_t(u8, 4, hweight8(sseu->subslice_mask.hsw[0]) / 2)) { GEM_BUG_ON(subslices & 1);
subslice_pg = false; @@ -743,14 +762,29 @@ void intel_sseu_dump(const struct sseu_dev_info *sseu, struct drm_printer *p) { int s;
- drm_printf(p, "slice total: %u, mask=%04x\n", - hweight8(sseu->slice_mask), sseu->slice_mask); - drm_printf(p, "subslice total: %u\n", intel_sseu_subslice_total(sseu)); - for (s = 0; s < sseu->max_slices; s++) { - drm_printf(p, "slice%d: %u subslices, mask=%08x\n", - s, intel_sseu_subslices_per_slice(sseu, s), - intel_sseu_get_subslices(sseu, s)); + if (sseu->has_xehp_dss) { + drm_printf(p, "subslice total: %u\n", + intel_sseu_subslice_total(sseu)); + drm_printf(p, "geometry dss mask=%*pb\n", + BITMAP_BITS(sseu->geometry_subslice_mask), + sseu->geometry_subslice_mask.xehp); + drm_printf(p, "compute dss mask=%*pb\n", + BITMAP_BITS(sseu->compute_subslice_mask), + sseu->compute_subslice_mask.xehp); + } else { + drm_printf(p, "slice total: %u, mask=%04x\n", + hweight8(sseu->slice_mask), sseu->slice_mask); + drm_printf(p, "subslice total: %u\n", + intel_sseu_subslice_total(sseu)); + + for (s = 0; s < sseu->max_slices; s++) { + u8 ss_mask = sseu->subslice_mask.hsw[s]; + + drm_printf(p, "slice%d: %u subslices, mask=%08x\n", + s, hweight8(ss_mask), ss_mask); + } } + drm_printf(p, "EU total: %u\n", sseu->eu_total); drm_printf(p, "EU per subslice: %u\n", sseu->eu_per_subslice); drm_printf(p, "has slice power gating: %s\n", @@ -767,9 +801,10 @@ static void sseu_print_hsw_topology(const struct sseu_dev_info *sseu, int s, ss;
for (s = 0; s < sseu->max_slices; s++) { + u8 ss_mask = sseu->subslice_mask.hsw[s]; + drm_printf(p, "slice%d: %u subslice(s) (0x%08x):\n", - s, intel_sseu_subslices_per_slice(sseu, s), - intel_sseu_get_subslices(sseu, s)); + s, hweight8(ss_mask), ss_mask);
for (ss = 0; ss < sseu->max_subslices; ss++) { u16 enabled_eus = sseu_get_eus(sseu, s, ss); @@ -783,16 +818,14 @@ static void sseu_print_hsw_topology(const struct sseu_dev_info *sseu, static void sseu_print_xehp_topology(const struct sseu_dev_info *sseu, struct drm_printer *p) { - u32 g_dss_mask = sseu_get_geometry_subslices(sseu); - u32 c_dss_mask = intel_sseu_get_compute_subslices(sseu); int dss;
for (dss = 0; dss < sseu->max_subslices; dss++) { u16 enabled_eus = sseu_get_eus(sseu, 0, dss);
drm_printf(p, "DSS_%02d: G:%3s C:%3s, %2u EUs (0x%04hx)\n", dss, - str_yes_no(g_dss_mask & BIT(dss)), - str_yes_no(c_dss_mask & BIT(dss)), + str_yes_no(test_bit(dss, sseu->geometry_subslice_mask.xehp)), + str_yes_no(test_bit(dss, sseu->compute_subslice_mask.xehp)), hweight16(enabled_eus), enabled_eus); } } @@ -810,20 +843,44 @@ void intel_sseu_print_topology(struct drm_i915_private *i915, } }
-u16 intel_slicemask_from_dssmask(u64 dss_mask, int dss_per_slice) +void intel_sseu_print_ss_info(const char* type, + const struct sseu_dev_info *sseu, + struct seq_file *m) +{ + int s; + + if (sseu->has_xehp_dss) { + seq_printf(m, " %s Geometry DSS: %u\n", type, + bitmap_weight(sseu->geometry_subslice_mask.xehp, + BITMAP_BITS(sseu->geometry_subslice_mask))); + seq_printf(m, " %s Compute DSS: %u\n", type, + bitmap_weight(sseu->compute_subslice_mask.xehp, + BITMAP_BITS(sseu->compute_subslice_mask))); + } else { + for (s = 0; s < fls(sseu->slice_mask); s++) + seq_printf(m, " %s Slice%i subslices: %u\n", type, + s, hweight8(sseu->subslice_mask.hsw[s])); + } +} + +u16 intel_slicemask_from_xehp_dssmask(intel_sseu_ss_mask_t dss_mask, + int dss_per_slice) { + intel_sseu_ss_mask_t per_slice_mask = {}; u16 slice_mask = 0; int i;
- WARN_ON(sizeof(dss_mask) * 8 / dss_per_slice > 8 * sizeof(slice_mask)); + WARN_ON(DIV_ROUND_UP(BITMAP_BITS(dss_mask), dss_per_slice) > + 8 * sizeof(slice_mask));
- for (i = 0; dss_mask; i++) { - if (dss_mask & GENMASK(dss_per_slice - 1, 0)) + bitmap_fill(per_slice_mask.xehp, dss_per_slice); + for (i = 0; !bitmap_empty(dss_mask.xehp, BITMAP_BITS(dss_mask)); i++) { + if (bitmap_intersects(dss_mask.xehp, per_slice_mask.xehp, dss_per_slice)) slice_mask |= BIT(i);
- dss_mask >>= dss_per_slice; + bitmap_shift_right(dss_mask.xehp, dss_mask.xehp, dss_per_slice, + BITMAP_BITS(dss_mask)); }
return slice_mask; } - diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.h b/drivers/gpu/drm/i915/gt/intel_sseu.h index ffa375e68959..0d3def55e770 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu.h +++ b/drivers/gpu/drm/i915/gt/intel_sseu.h @@ -25,12 +25,16 @@ struct drm_printer; /* * Maximum number of subslices that can exist within a HSW-style slice. This * is only relevant to pre-Xe_HP platforms (Xe_HP and beyond use the - * GEN_MAX_DSS value below). + * I915_MAX_SS_FUSE_BITS value below). */ #define GEN_MAX_SS_PER_HSW_SLICE 6
-/* Maximum number of DSS on newer platforms (Xe_HP and beyond). */ -#define GEN_MAX_DSS 32 +/* + * Maximum number of 32-bit registers used by hardware to express the + * enabled/disabled subslices. + */ +#define I915_MAX_SS_FUSE_REGS 1 +#define I915_MAX_SS_FUSE_BITS (I915_MAX_SS_FUSE_REGS * 32)
/* Maximum number of EUs that can exist within a subslice or DSS. */ #define GEN_MAX_EUS_PER_SS 16 @@ -38,7 +42,7 @@ struct drm_printer; #define SSEU_MAX(a, b) ((a) > (b) ? (a) : (b))
/* The maximum number of bits needed to express each subslice/DSS independently */ -#define GEN_SS_MASK_SIZE SSEU_MAX(GEN_MAX_DSS, \ +#define GEN_SS_MASK_SIZE SSEU_MAX(I915_MAX_SS_FUSE_BITS, \ GEN_MAX_HSW_SLICES * GEN_MAX_SS_PER_HSW_SLICE)
#define GEN_SSEU_STRIDE(max_entries) DIV_ROUND_UP(max_entries, BITS_PER_BYTE) @@ -49,17 +53,24 @@ struct drm_printer; #define GEN_DSS_PER_CSLICE 8 #define GEN_DSS_PER_MSLICE 8
-#define GEN_MAX_GSLICES (GEN_MAX_DSS / GEN_DSS_PER_GSLICE) -#define GEN_MAX_CSLICES (GEN_MAX_DSS / GEN_DSS_PER_CSLICE) +#define GEN_MAX_GSLICES (I915_MAX_SS_FUSE_BITS / GEN_DSS_PER_GSLICE) +#define GEN_MAX_CSLICES (I915_MAX_SS_FUSE_BITS / GEN_DSS_PER_CSLICE) + +typedef union { + u8 hsw[GEN_MAX_HSW_SLICES]; + + /* Bitmap compatible with linux/bitmap.h; may exceed size of u64 */ + unsigned long xehp[BITS_TO_LONGS(I915_MAX_SS_FUSE_BITS)]; +} intel_sseu_ss_mask_t;
struct sseu_dev_info { u8 slice_mask; - u8 subslice_mask[GEN_SS_MASK_SIZE]; - u8 geometry_subslice_mask[GEN_SS_MASK_SIZE]; - u8 compute_subslice_mask[GEN_SS_MASK_SIZE]; + intel_sseu_ss_mask_t subslice_mask; + intel_sseu_ss_mask_t geometry_subslice_mask; + intel_sseu_ss_mask_t compute_subslice_mask; union { u16 hsw[GEN_MAX_HSW_SLICES][GEN_MAX_SS_PER_HSW_SLICE]; - u16 xehp[GEN_MAX_DSS]; + u16 xehp[I915_MAX_SS_FUSE_BITS]; } eu_mask;
u16 eu_total; @@ -80,8 +91,6 @@ struct sseu_dev_info { u8 max_slices; u8 max_subslices; u8 max_eus_per_subslice; - - u8 ss_stride; };
/* @@ -99,7 +108,7 @@ intel_sseu_from_device_info(const struct sseu_dev_info *sseu) { struct intel_sseu value = { .slice_mask = sseu->slice_mask, - .subslice_mask = sseu->subslice_mask[0], + .subslice_mask = sseu->subslice_mask.hsw[0], .min_eus_per_subslice = sseu->max_eus_per_subslice, .max_eus_per_subslice = sseu->max_eus_per_subslice, }; @@ -111,18 +120,28 @@ static inline bool intel_sseu_has_subslice(const struct sseu_dev_info *sseu, int slice, int subslice) { - u8 mask; - int ss_idx = subslice / BITS_PER_BYTE; - - if (slice >= sseu->max_slices || - subslice >= sseu->max_subslices) + if (GEM_WARN_ON(slice >= sseu->max_slices)) + return false; + if (GEM_WARN_ON(subslice >= sseu->max_subslices)) return false;
- GEM_BUG_ON(ss_idx >= sseu->ss_stride); - - mask = sseu->subslice_mask[slice * sseu->ss_stride + ss_idx]; + if (sseu->has_xehp_dss) + return test_bit(subslice, sseu->subslice_mask.xehp); + else + return sseu->subslice_mask.hsw[slice] & BIT(subslice); +}
- return mask & BIT(subslice % BITS_PER_BYTE); +/* + * Used to obtain the index of the first DSS. Can start searching from the + * beginning of a specific dss group (e.g., gslice, cslice, etc.) if + * groupsize and groupnum are non-zero. + */ +static inline unsigned int +intel_sseu_find_first_xehp_dss(const struct sseu_dev_info *sseu, int groupsize, + int groupnum) +{ + return find_next_bit(sseu->subslice_mask.xehp, I915_MAX_SS_FUSE_BITS, + groupnum * groupsize); }
void intel_sseu_set_info(struct sseu_dev_info *sseu, u8 max_slices, @@ -132,14 +151,10 @@ unsigned int intel_sseu_subslice_total(const struct sseu_dev_info *sseu);
unsigned int -intel_sseu_subslices_per_slice(const struct sseu_dev_info *sseu, u8 slice); +intel_sseu_get_hsw_subslices(const struct sseu_dev_info *sseu, u8 slice);
-u32 intel_sseu_get_subslices(const struct sseu_dev_info *sseu, u8 slice); - -u32 intel_sseu_get_compute_subslices(const struct sseu_dev_info *sseu); - -void intel_sseu_set_subslices(struct sseu_dev_info *sseu, int slice, - u8 *subslice_mask, u32 ss_mask); +intel_sseu_ss_mask_t +intel_sseu_get_compute_subslices(const struct sseu_dev_info *sseu);
void intel_sseu_info_init(struct intel_gt *gt);
@@ -151,9 +166,15 @@ void intel_sseu_print_topology(struct drm_i915_private *i915, const struct sseu_dev_info *sseu, struct drm_printer *p);
-u16 intel_slicemask_from_dssmask(u64 dss_mask, int dss_per_slice); +u16 intel_slicemask_from_xehp_dssmask(intel_sseu_ss_mask_t dss_mask, int dss_per_slice);
int intel_sseu_copy_eumask_to_user(void __user *to, const struct sseu_dev_info *sseu); +int intel_sseu_copy_ssmask_to_user(void __user *to, + const struct sseu_dev_info *sseu); + +void intel_sseu_print_ss_info(const char* type, + const struct sseu_dev_info *sseu, + struct seq_file *m);
#endif /* __INTEL_SSEU_H__ */ diff --git a/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c b/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c index 2d5d011e01db..c2ee5e1826b5 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c +++ b/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c @@ -4,6 +4,7 @@ * Copyright © 2020 Intel Corporation */
+#include <linux/bitmap.h> #include <linux/string_helpers.h>
#include "i915_drv.h" @@ -11,14 +12,6 @@ #include "intel_gt_regs.h" #include "intel_sseu_debugfs.h"
-static void sseu_copy_subslices(const struct sseu_dev_info *sseu, - int slice, u8 *to_mask) -{ - int offset = slice * sseu->ss_stride; - - memcpy(&to_mask[offset], &sseu->subslice_mask[offset], sseu->ss_stride); -} - static void cherryview_sseu_device_status(struct intel_gt *gt, struct sseu_dev_info *sseu) { @@ -41,7 +34,7 @@ static void cherryview_sseu_device_status(struct intel_gt *gt, continue;
sseu->slice_mask = BIT(0); - sseu->subslice_mask[0] |= BIT(ss); + sseu->subslice_mask.hsw[0] |= BIT(ss); eu_cnt = ((sig1[ss] & CHV_EU08_PG_ENABLE) ? 0 : 2) + ((sig1[ss] & CHV_EU19_PG_ENABLE) ? 0 : 2) + ((sig1[ss] & CHV_EU210_PG_ENABLE) ? 0 : 2) + @@ -92,7 +85,7 @@ static void gen11_sseu_device_status(struct intel_gt *gt, continue;
sseu->slice_mask |= BIT(s); - sseu_copy_subslices(&info->sseu, s, sseu->subslice_mask); + sseu->subslice_mask.hsw[s] = info->sseu.subslice_mask.hsw[s];
for (ss = 0; ss < info->sseu.max_subslices; ss++) { unsigned int eu_cnt; @@ -147,21 +140,17 @@ static void gen9_sseu_device_status(struct intel_gt *gt, sseu->slice_mask |= BIT(s);
if (IS_GEN9_BC(gt->i915)) - sseu_copy_subslices(&info->sseu, s, - sseu->subslice_mask); + sseu->subslice_mask.hsw[s] = info->sseu.subslice_mask.hsw[s];
for (ss = 0; ss < info->sseu.max_subslices; ss++) { unsigned int eu_cnt; - u8 ss_idx = s * info->sseu.ss_stride + - ss / BITS_PER_BYTE;
if (IS_GEN9_LP(gt->i915)) { if (!(s_reg[s] & (GEN9_PGCTL_SS_ACK(ss)))) /* skip disabled subslice */ continue;
- sseu->subslice_mask[ss_idx] |= - BIT(ss % BITS_PER_BYTE); + sseu->subslice_mask.hsw[s] |= BIT(ss); }
eu_cnt = eu_reg[2 * s + ss / 2] & eu_mask[ss % 2]; @@ -188,8 +177,7 @@ static void bdw_sseu_device_status(struct intel_gt *gt, if (sseu->slice_mask) { sseu->eu_per_subslice = info->sseu.eu_per_subslice; for (s = 0; s < fls(sseu->slice_mask); s++) - sseu_copy_subslices(&info->sseu, s, - sseu->subslice_mask); + sseu->subslice_mask.hsw[s] = info->sseu.subslice_mask.hsw[s]; sseu->eu_total = sseu->eu_per_subslice * intel_sseu_subslice_total(sseu);
@@ -208,7 +196,6 @@ static void i915_print_sseu_info(struct seq_file *m, const struct sseu_dev_info *sseu) { const char *type = is_available_info ? "Available" : "Enabled"; - int s;
seq_printf(m, " %s Slice Mask: %04x\n", type, sseu->slice_mask); @@ -216,10 +203,7 @@ static void i915_print_sseu_info(struct seq_file *m, hweight8(sseu->slice_mask)); seq_printf(m, " %s Subslice Total: %u\n", type, intel_sseu_subslice_total(sseu)); - for (s = 0; s < fls(sseu->slice_mask); s++) { - seq_printf(m, " %s Slice%i subslices: %u\n", type, - s, intel_sseu_subslices_per_slice(sseu, s)); - } + intel_sseu_print_ss_info(type, sseu, m); seq_printf(m, " %s EU Total: %u\n", type, sseu->eu_total); seq_printf(m, " %s EU Per Subslice: %u\n", type, diff --git a/drivers/gpu/drm/i915/gt/intel_workarounds.c b/drivers/gpu/drm/i915/gt/intel_workarounds.c index 73b59ea6fd3b..1bf62273e02d 100644 --- a/drivers/gpu/drm/i915/gt/intel_workarounds.c +++ b/drivers/gpu/drm/i915/gt/intel_workarounds.c @@ -948,8 +948,8 @@ gen9_wa_init_mcr(struct drm_i915_private *i915, struct i915_wa_list *wal) * on s/ss combo, the read should be done with read_subslice_reg. */ slice = ffs(sseu->slice_mask) - 1; - GEM_BUG_ON(slice >= ARRAY_SIZE(sseu->subslice_mask)); - subslice = ffs(intel_sseu_get_subslices(sseu, slice)); + GEM_BUG_ON(slice >= ARRAY_SIZE(sseu->subslice_mask.hsw)); + subslice = ffs(intel_sseu_get_hsw_subslices(sseu, slice)); GEM_BUG_ON(!subslice); subslice--;
@@ -1087,11 +1087,10 @@ static void icl_wa_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) { const struct sseu_dev_info *sseu = >->info.sseu; - unsigned int slice, subslice; + unsigned int subslice;
GEM_BUG_ON(GRAPHICS_VER(gt->i915) < 11); GEM_BUG_ON(hweight8(sseu->slice_mask) > 1); - slice = 0;
/* * Although a platform may have subslices, we need to always steer @@ -1102,7 +1101,7 @@ icl_wa_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) * one of the higher subslices, we run the risk of reading back 0's or * random garbage. */ - subslice = __ffs(intel_sseu_get_subslices(sseu, slice)); + subslice = __ffs(intel_sseu_get_hsw_subslices(sseu, 0));
/* * If the subslice we picked above also steers us to a valid L3 bank, @@ -1112,7 +1111,7 @@ icl_wa_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) if (gt->info.l3bank_mask & BIT(subslice)) gt->steering_table[L3BANK] = NULL;
- __add_mcr_wa(gt, wal, slice, subslice); + __add_mcr_wa(gt, wal, 0, subslice); }
static void @@ -1120,7 +1119,6 @@ xehp_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) { const struct sseu_dev_info *sseu = >->info.sseu; unsigned long slice, subslice = 0, slice_mask = 0; - u64 dss_mask = 0; u32 lncf_mask = 0; int i;
@@ -1151,8 +1149,8 @@ xehp_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) */
/* Find the potential gslice candidates */ - dss_mask = intel_sseu_get_subslices(sseu, 0); - slice_mask = intel_slicemask_from_dssmask(dss_mask, GEN_DSS_PER_GSLICE); + slice_mask = intel_slicemask_from_xehp_dssmask(sseu->subslice_mask, + GEN_DSS_PER_GSLICE);
/* * Find the potential LNCF candidates. Either LNCF within a valid @@ -1177,9 +1175,8 @@ xehp_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) }
slice = __ffs(slice_mask); - subslice = __ffs(dss_mask >> (slice * GEN_DSS_PER_GSLICE)); + subslice = intel_sseu_find_first_xehp_dss(sseu, GEN_DSS_PER_GSLICE, slice); WARN_ON(subslice > GEN_DSS_PER_GSLICE); - WARN_ON(dss_mask >> (slice * GEN_DSS_PER_GSLICE) == 0);
__add_mcr_wa(gt, wal, slice, subslice);
@@ -2030,9 +2027,8 @@ engine_fake_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
static bool needs_wa_1308578152(struct intel_engine_cs *engine) { - u64 dss_mask = intel_sseu_get_subslices(&engine->gt->info.sseu, 0); - - return (dss_mask & GENMASK(GEN_DSS_PER_GSLICE - 1, 0)) == 0; + return intel_sseu_find_first_xehp_dss(&engine->gt->info.sseu, 0, 0) > + GEN_DSS_PER_GSLICE; }
static void diff --git a/drivers/gpu/drm/i915/i915_getparam.c b/drivers/gpu/drm/i915/i915_getparam.c index ac9767c56619..6fd15b39570c 100644 --- a/drivers/gpu/drm/i915/i915_getparam.c +++ b/drivers/gpu/drm/i915/i915_getparam.c @@ -162,8 +162,7 @@ int i915_getparam_ioctl(struct drm_device *dev, void *data, return -EINVAL;
/* Only copy bits from the first slice */ - memcpy(&value, sseu->subslice_mask, - min(sseu->ss_stride, (u8)sizeof(value))); + value = intel_sseu_get_hsw_subslices(sseu, 0); if (!value) return -ENODEV; break; diff --git a/drivers/gpu/drm/i915/i915_query.c b/drivers/gpu/drm/i915/i915_query.c index 89c475d525b8..0094f67c63f2 100644 --- a/drivers/gpu/drm/i915/i915_query.c +++ b/drivers/gpu/drm/i915/i915_query.c @@ -31,10 +31,11 @@ static int copy_query_item(void *query_hdr, size_t query_sz,
static int fill_topology_info(const struct sseu_dev_info *sseu, struct drm_i915_query_item *query_item, - const u8 *subslice_mask) + intel_sseu_ss_mask_t subslice_mask) { struct drm_i915_query_topology_info topo; u32 slice_length, subslice_length, eu_length, total_length; + int ss_stride = GEN_SSEU_STRIDE(sseu->max_subslices); int eu_stride = GEN_SSEU_STRIDE(sseu->max_eus_per_subslice); int ret;
@@ -44,7 +45,7 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, return -ENODEV;
slice_length = sizeof(sseu->slice_mask); - subslice_length = sseu->max_slices * sseu->ss_stride; + subslice_length = sseu->max_slices * ss_stride; eu_length = sseu->max_slices * sseu->max_subslices * eu_stride; total_length = sizeof(topo) + slice_length + subslice_length + eu_length; @@ -60,7 +61,7 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, topo.max_eus_per_subslice = sseu->max_eus_per_subslice;
topo.subslice_offset = slice_length; - topo.subslice_stride = sseu->ss_stride; + topo.subslice_stride = ss_stride; topo.eu_offset = slice_length + subslice_length; topo.eu_stride = eu_stride;
@@ -72,9 +73,9 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, &sseu->slice_mask, slice_length)) return -EFAULT;
- if (copy_to_user(u64_to_user_ptr(query_item->data_ptr + - sizeof(topo) + slice_length), - subslice_mask, subslice_length)) + if (intel_sseu_copy_ssmask_to_user(u64_to_user_ptr(query_item->data_ptr + + sizeof(topo) + slice_length), + sseu)) return -EFAULT;
if (intel_sseu_copy_eumask_to_user(u64_to_user_ptr(query_item->data_ptr +
On 21/05/2022 00:04, Matt Roper wrote:
As with EU masks, it's easier to store subslice/DSS masks internally in a format that's more natural for the driver to work with, and then only covert into the u8[] uapi form when the query ioctl is invoked. Since the hardware design changed significantly with Xe_HP, we'll use a union to choose between the old "hsw-style" subslice masks or the newer xehp mask. HSW-style masks will be stored in an array of u8's, indexed by slice (there's never more than 6 subslices per slice on older platforms). For Xe_HP and beyond where slices no longer exist, we only need a single bitmask. However we already know that this mask is eventually going to grow too large for a simple u64 to hold, so we'll represent it in a manner that can be operated on by the utilities in linux/bitmap.h.
v2:
- Fix typo: BIT(s) -> BIT(ss) in gen9_sseu_device_status()
v3:
- Eliminate sseu->ss_stride and just calculate the stride while specifically handling uapi. (Tvrtko)
- Use BITMAP_BITS() macro to refer to size of masks rather than passing I915_MAX_SS_FUSE_BITS directly. (Tvrtko)
- Report compute/geometry DSS masks separately when dumping Xe_HP SSEU info. (Tvrtko)
- Restore dropped range checks to intel_sseu_has_subslice(). (Tvrtko)
Cc: Tvrtko Ursulin tvrtko.ursulin@linux.intel.com Signed-off-by: Matt Roper matthew.d.roper@intel.com
drivers/gpu/drm/i915/gem/i915_gem_context.c | 5 +- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 4 +- drivers/gpu/drm/i915/gt/intel_gt.c | 12 +- drivers/gpu/drm/i915/gt/intel_sseu.c | 261 +++++++++++-------- drivers/gpu/drm/i915/gt/intel_sseu.h | 81 +++--- drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c | 30 +-- drivers/gpu/drm/i915/gt/intel_workarounds.c | 24 +- drivers/gpu/drm/i915/i915_getparam.c | 3 +- drivers/gpu/drm/i915/i915_query.c | 13 +- 9 files changed, 244 insertions(+), 189 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index ab4c5ab28e4d..a3bb73f5d53b 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -1875,6 +1875,7 @@ i915_gem_user_to_context_sseu(struct intel_gt *gt, { const struct sseu_dev_info *device = >->info.sseu; struct drm_i915_private *i915 = gt->i915;
unsigned int dev_subslice_mask = intel_sseu_get_hsw_subslices(device, 0);
/* No zeros in any field. */ if (!user->slice_mask || !user->subslice_mask ||
@@ -1901,7 +1902,7 @@ i915_gem_user_to_context_sseu(struct intel_gt *gt, if (user->slice_mask & ~device->slice_mask) return -EINVAL;
- if (user->subslice_mask & ~device->subslice_mask[0])
if (user->subslice_mask & ~dev_subslice_mask) return -EINVAL;
if (user->max_eus_per_subslice > device->max_eus_per_subslice)
@@ -1915,7 +1916,7 @@ i915_gem_user_to_context_sseu(struct intel_gt *gt, /* Part specific restrictions. */ if (GRAPHICS_VER(i915) == 11) { unsigned int hw_s = hweight8(device->slice_mask);
unsigned int hw_ss_per_s = hweight8(device->subslice_mask[0]);
unsigned int hw_ss_per_s = hweight8(dev_subslice_mask);diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c index 1adbf34c3632..f0acf8518a51 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c @@ -674,8 +674,8 @@ static void engine_mask_apply_compute_fuses(struct intel_gt *gt) if (GRAPHICS_VER_FULL(i915) < IP_VER(12, 50)) return;
- ccs_mask = intel_slicemask_from_dssmask(intel_sseu_get_compute_subslices(&info->sseu),
ss_per_ccs);
- ccs_mask = intel_slicemask_from_xehp_dssmask(info->sseu.compute_subslice_mask,
ss_per_ccs);
- If all DSS in a quadrant are fused off, the corresponding CCS
- engine is not available for use.
diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c index 034182f85501..2921f510642f 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -133,13 +133,6 @@ static const struct intel_mmio_range dg2_lncf_steering_table[] = { {}, };
-static u16 slicemask(struct intel_gt *gt, int count) -{
- u64 dss_mask = intel_sseu_get_subslices(>->info.sseu, 0);
- return intel_slicemask_from_dssmask(dss_mask, count);
-}
- int intel_gt_init_mmio(struct intel_gt *gt) { struct drm_i915_private *i915 = gt->i915;
@@ -155,9 +148,12 @@ int intel_gt_init_mmio(struct intel_gt *gt) */ if (HAS_MSLICES(i915)) { gt->info.mslice_mask =
slicemask(gt, GEN_DSS_PER_MSLICE) |
intel_slicemask_from_xehp_dssmask(gt->info.sseu.subslice_mask,GEN_DSS_PER_MSLICE);gt->info.mslice_mask |= (intel_uncore_read(gt->uncore, GEN10_MIRROR_FUSE3) & GEN12_MEML3_EN_MASK);- if (!gt->info.mslice_mask) /* should be impossible! */ drm_warn(&i915->drm, "mslice mask all zero!\n"); }
diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.c b/drivers/gpu/drm/i915/gt/intel_sseu.c index 285cfd758bdc..6fbc2ac507b7 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu.c +++ b/drivers/gpu/drm/i915/gt/intel_sseu.c @@ -10,15 +10,14 @@ #include "intel_gt_regs.h" #include "intel_sseu.h"
+#define BITMAP_BITS(mask) ((int)BITS_PER_TYPE(typeof(mask)))
- void intel_sseu_set_info(struct sseu_dev_info *sseu, u8 max_slices, u8 max_subslices, u8 max_eus_per_subslice) { sseu->max_slices = max_slices; sseu->max_subslices = max_subslices; sseu->max_eus_per_subslice = max_eus_per_subslice;
sseu->ss_stride = GEN_SSEU_STRIDE(sseu->max_subslices);
GEM_BUG_ON(sseu->ss_stride > GEN_MAX_SUBSLICE_STRIDE); }
unsigned int
@@ -26,54 +25,24 @@ intel_sseu_subslice_total(const struct sseu_dev_info *sseu) { unsigned int i, total = 0;
- for (i = 0; i < ARRAY_SIZE(sseu->subslice_mask); i++)
total += hweight8(sseu->subslice_mask[i]);- return total;
-}
-static u32 -sseu_get_subslices(const struct sseu_dev_info *sseu,
const u8 *subslice_mask, u8 slice)-{
- int i, offset = slice * sseu->ss_stride;
- u32 mask = 0;
- GEM_BUG_ON(slice >= sseu->max_slices);
- for (i = 0; i < sseu->ss_stride; i++)
mask |= (u32)subslice_mask[offset + i] << i * BITS_PER_BYTE;- return mask;
-}
-u32 intel_sseu_get_subslices(const struct sseu_dev_info *sseu, u8 slice) -{
- return sseu_get_subslices(sseu, sseu->subslice_mask, slice);
-}
- if (sseu->has_xehp_dss)
return bitmap_weight(sseu->subslice_mask.xehp,BITMAP_BITS(sseu->subslice_mask));
Shouldn't argument to BITMAP_BITS strictly be ***.xehp? It works without it because union will be larger, same size as bitmap. (Or hm will it on 32-bit builds?) If so this tweak applies throughout the patch. Or append the .xehp union member name in the macro itself? If renamed to i915 specific name that would work I think.
Regards,
Tvrtko
-static u32 sseu_get_geometry_subslices(const struct sseu_dev_info *sseu) -{
- return sseu_get_subslices(sseu, sseu->geometry_subslice_mask, 0);
-}
- for (i = 0; i < ARRAY_SIZE(sseu->subslice_mask.hsw); i++)
total += hweight8(sseu->subslice_mask.hsw[i]);-u32 intel_sseu_get_compute_subslices(const struct sseu_dev_info *sseu) -{
- return sseu_get_subslices(sseu, sseu->compute_subslice_mask, 0);
-}
-void intel_sseu_set_subslices(struct sseu_dev_info *sseu, int slice,
u8 *subslice_mask, u32 ss_mask)-{
- int offset = slice * sseu->ss_stride;
- memcpy(&subslice_mask[offset], &ss_mask, sseu->ss_stride);
return total; }
unsigned int
-intel_sseu_subslices_per_slice(const struct sseu_dev_info *sseu, u8 slice) +intel_sseu_get_hsw_subslices(const struct sseu_dev_info *sseu, u8 slice) {
- return hweight32(intel_sseu_get_subslices(sseu, slice));
WARN_ON(sseu->has_xehp_dss);
if (WARN_ON(slice >= sseu->max_slices))
return 0;return sseu->subslice_mask.hsw[slice]; }
static u16 sseu_get_eus(const struct sseu_dev_info *sseu, int slice,
@@ -147,32 +116,66 @@ int intel_sseu_copy_eumask_to_user(void __user *to, return copy_to_user(to, eu_mask, len); }
+/**
- intel_sseu_copy_ssmask_to_user - Copy subslice mask into a userspace buffer
- @to: Pointer to userspace buffer to copy to
- @sseu: SSEU structure containing subslice mask to copy
- Copies the subslice mask to a userspace buffer in the format expected by
- the query ioctl's topology queries.
- Returns the result of the copy_to_user() operation.
- */
+int intel_sseu_copy_ssmask_to_user(void __user *to,
const struct sseu_dev_info *sseu)+{
- u8 ss_mask[GEN_SS_MASK_SIZE] = {};
- int ss_stride = GEN_SSEU_STRIDE(sseu->max_subslices);
- int len = sseu->max_slices * ss_stride;
- int s, ss, i;
- for (s = 0; s < sseu->max_slices; s++) {
for (ss = 0; ss < sseu->max_subslices; ss++) {i = s * ss_stride * BITS_PER_BYTE + ss;if (!intel_sseu_has_subslice(sseu, s, ss))continue;ss_mask[i / BITS_PER_BYTE] |= BIT(i % BITS_PER_BYTE);}- }
- return copy_to_user(to, ss_mask, len);
+}
- static void gen11_compute_sseu_info(struct sseu_dev_info *sseu,
u32 g_ss_en, u32 c_ss_en, u16 eu_en)
u32 ss_en, u16 eu_en)
- /* g_ss_en/c_ss_en represent entire subslice mask across all slices */
- GEM_BUG_ON(sseu->max_slices * sseu->max_subslices >
sizeof(g_ss_en) * BITS_PER_BYTE);
- sseu->slice_mask |= BIT(0);
- sseu->subslice_mask.hsw[0] = ss_en & valid_ss_mask;
- for (ss = 0; ss < sseu->max_subslices; ss++)
if (intel_sseu_has_subslice(sseu, 0, ss))sseu_set_eus(sseu, 0, ss, eu_en);- sseu->eu_per_subslice = hweight16(eu_en);
- sseu->eu_total = compute_eu_total(sseu);
+}
+static void xehp_compute_sseu_info(struct sseu_dev_info *sseu,
u16 eu_en)+{
int ss;
sseu->slice_mask |= BIT(0);
- /*
* XeHP introduces the concept of compute vs geometry DSS. To reduce* variation between GENs around subslice usage, store a mask for both* the geometry and compute enabled masks since userspace will need to* be able to query these masks independently. Also compute a total* enabled subslice count for the purposes of selecting subslices to* use in a particular GEM context.*/- intel_sseu_set_subslices(sseu, 0, sseu->compute_subslice_mask,
c_ss_en & valid_ss_mask);- intel_sseu_set_subslices(sseu, 0, sseu->geometry_subslice_mask,
g_ss_en & valid_ss_mask);- intel_sseu_set_subslices(sseu, 0, sseu->subslice_mask,
(g_ss_en | c_ss_en) & valid_ss_mask);
bitmap_or(sseu->subslice_mask.xehp,
sseu->compute_subslice_mask.xehp,sseu->geometry_subslice_mask.xehp,BITMAP_BITS(sseu->subslice_mask));for (ss = 0; ss < sseu->max_subslices; ss++) if (intel_sseu_has_subslice(sseu, 0, ss))
@@ -182,11 +185,31 @@ static void gen11_compute_sseu_info(struct sseu_dev_info *sseu, sseu->eu_total = compute_eu_total(sseu); }
+static void +xehp_load_dss_mask(struct intel_uncore *uncore,
intel_sseu_ss_mask_t *ssmask,int numregs,...)+{
- va_list argp;
- u32 fuse_val[I915_MAX_SS_FUSE_REGS] = {};
- int i;
- if (WARN_ON(numregs > I915_MAX_SS_FUSE_REGS))
numregs = I915_MAX_SS_FUSE_REGS;- va_start(argp, numregs);
- for (i = 0; i < numregs; i++)
fuse_val[i] = intel_uncore_read(uncore, va_arg(argp, i915_reg_t));- va_end(argp);
- bitmap_from_arr32(ssmask->xehp, fuse_val, numregs * 32);
+}
- static void xehp_sseu_info_init(struct intel_gt *gt) { struct sseu_dev_info *sseu = >->info.sseu; struct intel_uncore *uncore = gt->uncore;
- u32 g_dss_en, c_dss_en = 0; u16 eu_en = 0; u8 eu_en_fuse; int eu;
@@ -200,8 +223,10 @@ static void xehp_sseu_info_init(struct intel_gt *gt) intel_sseu_set_info(sseu, 1, 32, 16); sseu->has_xehp_dss = 1;
- g_dss_en = intel_uncore_read(uncore, GEN12_GT_GEOMETRY_DSS_ENABLE);
- c_dss_en = intel_uncore_read(uncore, GEN12_GT_COMPUTE_DSS_ENABLE);
xehp_load_dss_mask(uncore, &sseu->geometry_subslice_mask, 1,
GEN12_GT_GEOMETRY_DSS_ENABLE);xehp_load_dss_mask(uncore, &sseu->compute_subslice_mask, 1,
GEN12_GT_COMPUTE_DSS_ENABLE);eu_en_fuse = intel_uncore_read(uncore, XEHP_EU_ENABLE) & XEHP_EU_ENA_MASK;
@@ -209,7 +234,7 @@ static void xehp_sseu_info_init(struct intel_gt *gt) if (eu_en_fuse & BIT(eu)) eu_en |= BIT(eu * 2) | BIT(eu * 2 + 1);
- gen11_compute_sseu_info(sseu, g_dss_en, c_dss_en, eu_en);
xehp_compute_sseu_info(sseu, eu_en); }
static void gen12_sseu_info_init(struct intel_gt *gt)
@@ -247,7 +272,7 @@ static void gen12_sseu_info_init(struct intel_gt *gt) if (eu_en_fuse & BIT(eu)) eu_en |= BIT(eu * 2) | BIT(eu * 2 + 1);
- gen11_compute_sseu_info(sseu, g_dss_en, 0, eu_en);
gen11_compute_sseu_info(sseu, g_dss_en, eu_en);
/* TGL only supports slice-level power gating */ sseu->has_slice_pg = 1;
@@ -279,7 +304,7 @@ static void gen11_sseu_info_init(struct intel_gt *gt) eu_en = ~(intel_uncore_read(uncore, GEN11_EU_DISABLE) & GEN11_EU_DIS_MASK);
- gen11_compute_sseu_info(sseu, ss_en, 0, eu_en);
gen11_compute_sseu_info(sseu, ss_en, eu_en);
/* ICL has no power gating restrictions. */ sseu->has_slice_pg = 1;
@@ -291,7 +316,6 @@ static void cherryview_sseu_info_init(struct intel_gt *gt) { struct sseu_dev_info *sseu = >->info.sseu; u32 fuse;
u8 subslice_mask = 0;
fuse = intel_uncore_read(gt->uncore, CHV_FUSE_GT);
@@ -305,7 +329,7 @@ static void cherryview_sseu_info_init(struct intel_gt *gt) (((fuse & CHV_FGT_EU_DIS_SS0_R1_MASK) >> CHV_FGT_EU_DIS_SS0_R1_SHIFT) << 4);
subslice_mask |= BIT(0);
sseu->subslice_mask.hsw[0] |= BIT(0);@@ -316,12 +340,10 @@ static void cherryview_sseu_info_init(struct intel_gt *gt) (((fuse & CHV_FGT_EU_DIS_SS1_R1_MASK) >> CHV_FGT_EU_DIS_SS1_R1_SHIFT) << 4);
subslice_mask |= BIT(1);
sseu->subslice_mask.hsw[0] |= BIT(1);
intel_sseu_set_subslices(sseu, 0, sseu->subslice_mask, subslice_mask);
sseu->eu_total = compute_eu_total(sseu);
/*
@@ -376,8 +398,7 @@ static void gen9_sseu_info_init(struct intel_gt *gt) /* skip disabled slice */ continue;
intel_sseu_set_subslices(sseu, s, sseu->subslice_mask,subslice_mask);
sseu->subslice_mask.hsw[s] = subslice_mask;eu_disable = intel_uncore_read(uncore, GEN9_EU_DISABLE(s)); for (ss = 0; ss < sseu->max_subslices; ss++) {
@@ -434,8 +455,8 @@ static void gen9_sseu_info_init(struct intel_gt *gt) sseu->has_eu_pg = sseu->eu_per_subslice > 2;
if (IS_GEN9_LP(i915)) { -#define IS_SS_DISABLED(ss) (!(sseu->subslice_mask[0] & BIT(ss)))
info->has_pooled_eu = hweight8(sseu->subslice_mask[0]) == 3;+#define IS_SS_DISABLED(ss) (!(sseu->subslice_mask.hsw[0] & BIT(ss)))
info->has_pooled_eu = hweight8(sseu->subslice_mask.hsw[0]) == 3;sseu->min_eu_in_pool = 0; if (info->has_pooled_eu) {
@@ -489,8 +510,7 @@ static void bdw_sseu_info_init(struct intel_gt *gt) /* skip disabled slice */ continue;
intel_sseu_set_subslices(sseu, s, sseu->subslice_mask,subslice_mask);
sseu->subslice_mask.hsw[s] = subslice_mask;for (ss = 0; ss < sseu->max_subslices; ss++) { u8 eu_disabled_mask;
@@ -587,8 +607,7 @@ static void hsw_sseu_info_init(struct intel_gt *gt) sseu->eu_per_subslice);
for (s = 0; s < sseu->max_slices; s++) {
intel_sseu_set_subslices(sseu, s, sseu->subslice_mask,subslice_mask);
sseu->subslice_mask.hsw[s] = subslice_mask;for (ss = 0; ss < sseu->max_subslices; ss++) { sseu_set_eus(sseu, s, ss,
@@ -677,7 +696,7 @@ u32 intel_sseu_make_rpcs(struct intel_gt *gt, */ if (GRAPHICS_VER(i915) == 11 && slices == 1 &&
subslices > min_t(u8, 4, hweight8(sseu->subslice_mask[0]) / 2)) {
subslices > min_t(u8, 4, hweight8(sseu->subslice_mask.hsw[0]) / 2)) {GEM_BUG_ON(subslices & 1);
subslice_pg = false;
@@ -743,14 +762,29 @@ void intel_sseu_dump(const struct sseu_dev_info *sseu, struct drm_printer *p) { int s;
- drm_printf(p, "slice total: %u, mask=%04x\n",
hweight8(sseu->slice_mask), sseu->slice_mask);- drm_printf(p, "subslice total: %u\n", intel_sseu_subslice_total(sseu));
- for (s = 0; s < sseu->max_slices; s++) {
drm_printf(p, "slice%d: %u subslices, mask=%08x\n",s, intel_sseu_subslices_per_slice(sseu, s),intel_sseu_get_subslices(sseu, s));
- if (sseu->has_xehp_dss) {
drm_printf(p, "subslice total: %u\n",intel_sseu_subslice_total(sseu));drm_printf(p, "geometry dss mask=%*pb\n",BITMAP_BITS(sseu->geometry_subslice_mask),sseu->geometry_subslice_mask.xehp);drm_printf(p, "compute dss mask=%*pb\n",BITMAP_BITS(sseu->compute_subslice_mask),sseu->compute_subslice_mask.xehp);- } else {
drm_printf(p, "slice total: %u, mask=%04x\n",hweight8(sseu->slice_mask), sseu->slice_mask);drm_printf(p, "subslice total: %u\n",intel_sseu_subslice_total(sseu));for (s = 0; s < sseu->max_slices; s++) {u8 ss_mask = sseu->subslice_mask.hsw[s];drm_printf(p, "slice%d: %u subslices, mask=%08x\n",s, hweight8(ss_mask), ss_mask);}- drm_printf(p, "EU total: %u\n", sseu->eu_total); drm_printf(p, "EU per subslice: %u\n", sseu->eu_per_subslice); drm_printf(p, "has slice power gating: %s\n",
@@ -767,9 +801,10 @@ static void sseu_print_hsw_topology(const struct sseu_dev_info *sseu, int s, ss;
for (s = 0; s < sseu->max_slices; s++) {
u8 ss_mask = sseu->subslice_mask.hsw[s];- drm_printf(p, "slice%d: %u subslice(s) (0x%08x):\n",
s, intel_sseu_subslices_per_slice(sseu, s),intel_sseu_get_subslices(sseu, s));
s, hweight8(ss_mask), ss_mask);for (ss = 0; ss < sseu->max_subslices; ss++) { u16 enabled_eus = sseu_get_eus(sseu, s, ss);
@@ -783,16 +818,14 @@ static void sseu_print_hsw_topology(const struct sseu_dev_info *sseu, static void sseu_print_xehp_topology(const struct sseu_dev_info *sseu, struct drm_printer *p) {
u32 g_dss_mask = sseu_get_geometry_subslices(sseu);
u32 c_dss_mask = intel_sseu_get_compute_subslices(sseu); int dss;
for (dss = 0; dss < sseu->max_subslices; dss++) { u16 enabled_eus = sseu_get_eus(sseu, 0, dss);
drm_printf(p, "DSS_%02d: G:%3s C:%3s, %2u EUs (0x%04hx)\n", dss,
str_yes_no(g_dss_mask & BIT(dss)),str_yes_no(c_dss_mask & BIT(dss)),
str_yes_no(test_bit(dss, sseu->geometry_subslice_mask.xehp)),str_yes_no(test_bit(dss, sseu->compute_subslice_mask.xehp)), hweight16(enabled_eus), enabled_eus);@@ -810,20 +843,44 @@ void intel_sseu_print_topology(struct drm_i915_private *i915, } }
-u16 intel_slicemask_from_dssmask(u64 dss_mask, int dss_per_slice) +void intel_sseu_print_ss_info(const char* type,
const struct sseu_dev_info *sseu,struct seq_file *m)+{
- int s;
- if (sseu->has_xehp_dss) {
seq_printf(m, " %s Geometry DSS: %u\n", type,bitmap_weight(sseu->geometry_subslice_mask.xehp,BITMAP_BITS(sseu->geometry_subslice_mask)));seq_printf(m, " %s Compute DSS: %u\n", type,bitmap_weight(sseu->compute_subslice_mask.xehp,BITMAP_BITS(sseu->compute_subslice_mask)));- } else {
for (s = 0; s < fls(sseu->slice_mask); s++)seq_printf(m, " %s Slice%i subslices: %u\n", type,s, hweight8(sseu->subslice_mask.hsw[s]));- }
+}
+u16 intel_slicemask_from_xehp_dssmask(intel_sseu_ss_mask_t dss_mask,
int dss_per_slice)- intel_sseu_ss_mask_t per_slice_mask = {}; u16 slice_mask = 0; int i;
- WARN_ON(sizeof(dss_mask) * 8 / dss_per_slice > 8 * sizeof(slice_mask));
- WARN_ON(DIV_ROUND_UP(BITMAP_BITS(dss_mask), dss_per_slice) >
8 * sizeof(slice_mask));
- for (i = 0; dss_mask; i++) {
if (dss_mask & GENMASK(dss_per_slice - 1, 0))
- bitmap_fill(per_slice_mask.xehp, dss_per_slice);
- for (i = 0; !bitmap_empty(dss_mask.xehp, BITMAP_BITS(dss_mask)); i++) {
if (bitmap_intersects(dss_mask.xehp, per_slice_mask.xehp, dss_per_slice)) slice_mask |= BIT(i);
dss_mask >>= dss_per_slice;
bitmap_shift_right(dss_mask.xehp, dss_mask.xehp, dss_per_slice,BITMAP_BITS(dss_mask));}
return slice_mask; }
diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.h b/drivers/gpu/drm/i915/gt/intel_sseu.h index ffa375e68959..0d3def55e770 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu.h +++ b/drivers/gpu/drm/i915/gt/intel_sseu.h @@ -25,12 +25,16 @@ struct drm_printer; /*
- Maximum number of subslices that can exist within a HSW-style slice. This
- is only relevant to pre-Xe_HP platforms (Xe_HP and beyond use the
- GEN_MAX_DSS value below).
*/ #define GEN_MAX_SS_PER_HSW_SLICE 6
- I915_MAX_SS_FUSE_BITS value below).
-/* Maximum number of DSS on newer platforms (Xe_HP and beyond). */ -#define GEN_MAX_DSS 32 +/*
- Maximum number of 32-bit registers used by hardware to express the
- enabled/disabled subslices.
- */
+#define I915_MAX_SS_FUSE_REGS 1 +#define I915_MAX_SS_FUSE_BITS (I915_MAX_SS_FUSE_REGS * 32)
/* Maximum number of EUs that can exist within a subslice or DSS. */ #define GEN_MAX_EUS_PER_SS 16 @@ -38,7 +42,7 @@ struct drm_printer; #define SSEU_MAX(a, b) ((a) > (b) ? (a) : (b))
/* The maximum number of bits needed to express each subslice/DSS independently */ -#define GEN_SS_MASK_SIZE SSEU_MAX(GEN_MAX_DSS, \ +#define GEN_SS_MASK_SIZE SSEU_MAX(I915_MAX_SS_FUSE_BITS, \ GEN_MAX_HSW_SLICES * GEN_MAX_SS_PER_HSW_SLICE)
#define GEN_SSEU_STRIDE(max_entries) DIV_ROUND_UP(max_entries, BITS_PER_BYTE) @@ -49,17 +53,24 @@ struct drm_printer; #define GEN_DSS_PER_CSLICE 8 #define GEN_DSS_PER_MSLICE 8
-#define GEN_MAX_GSLICES (GEN_MAX_DSS / GEN_DSS_PER_GSLICE) -#define GEN_MAX_CSLICES (GEN_MAX_DSS / GEN_DSS_PER_CSLICE) +#define GEN_MAX_GSLICES (I915_MAX_SS_FUSE_BITS / GEN_DSS_PER_GSLICE) +#define GEN_MAX_CSLICES (I915_MAX_SS_FUSE_BITS / GEN_DSS_PER_CSLICE)
+typedef union {
- u8 hsw[GEN_MAX_HSW_SLICES];
- /* Bitmap compatible with linux/bitmap.h; may exceed size of u64 */
- unsigned long xehp[BITS_TO_LONGS(I915_MAX_SS_FUSE_BITS)];
+} intel_sseu_ss_mask_t;
struct sseu_dev_info { u8 slice_mask;
- u8 subslice_mask[GEN_SS_MASK_SIZE];
- u8 geometry_subslice_mask[GEN_SS_MASK_SIZE];
- u8 compute_subslice_mask[GEN_SS_MASK_SIZE];
- intel_sseu_ss_mask_t subslice_mask;
- intel_sseu_ss_mask_t geometry_subslice_mask;
- intel_sseu_ss_mask_t compute_subslice_mask; union { u16 hsw[GEN_MAX_HSW_SLICES][GEN_MAX_SS_PER_HSW_SLICE];
u16 xehp[GEN_MAX_DSS];
u16 xehp[I915_MAX_SS_FUSE_BITS];} eu_mask;
u16 eu_total;
@@ -80,8 +91,6 @@ struct sseu_dev_info { u8 max_slices; u8 max_subslices; u8 max_eus_per_subslice;
u8 ss_stride; };
/*
@@ -99,7 +108,7 @@ intel_sseu_from_device_info(const struct sseu_dev_info *sseu) { struct intel_sseu value = { .slice_mask = sseu->slice_mask,
.subslice_mask = sseu->subslice_mask[0],
.subslice_mask = sseu->subslice_mask.hsw[0],@@ -111,18 +120,28 @@ static inline bool intel_sseu_has_subslice(const struct sseu_dev_info *sseu, int slice, int subslice) {
- u8 mask;
- int ss_idx = subslice / BITS_PER_BYTE;
- if (slice >= sseu->max_slices ||
subslice >= sseu->max_subslices)
- if (GEM_WARN_ON(slice >= sseu->max_slices))
return false;- if (GEM_WARN_ON(subslice >= sseu->max_subslices)) return false;
- GEM_BUG_ON(ss_idx >= sseu->ss_stride);
- mask = sseu->subslice_mask[slice * sseu->ss_stride + ss_idx];
- if (sseu->has_xehp_dss)
return test_bit(subslice, sseu->subslice_mask.xehp);- else
return sseu->subslice_mask.hsw[slice] & BIT(subslice);+}
- return mask & BIT(subslice % BITS_PER_BYTE);
+/*
- Used to obtain the index of the first DSS. Can start searching from the
- beginning of a specific dss group (e.g., gslice, cslice, etc.) if
- groupsize and groupnum are non-zero.
- */
+static inline unsigned int +intel_sseu_find_first_xehp_dss(const struct sseu_dev_info *sseu, int groupsize,
int groupnum)+{
return find_next_bit(sseu->subslice_mask.xehp, I915_MAX_SS_FUSE_BITS,
groupnum * groupsize);}
void intel_sseu_set_info(struct sseu_dev_info *sseu, u8 max_slices,
@@ -132,14 +151,10 @@ unsigned int intel_sseu_subslice_total(const struct sseu_dev_info *sseu);
unsigned int -intel_sseu_subslices_per_slice(const struct sseu_dev_info *sseu, u8 slice); +intel_sseu_get_hsw_subslices(const struct sseu_dev_info *sseu, u8 slice);
-u32 intel_sseu_get_subslices(const struct sseu_dev_info *sseu, u8 slice);
-u32 intel_sseu_get_compute_subslices(const struct sseu_dev_info *sseu);
-void intel_sseu_set_subslices(struct sseu_dev_info *sseu, int slice,
u8 *subslice_mask, u32 ss_mask);+intel_sseu_ss_mask_t +intel_sseu_get_compute_subslices(const struct sseu_dev_info *sseu);
void intel_sseu_info_init(struct intel_gt *gt);
@@ -151,9 +166,15 @@ void intel_sseu_print_topology(struct drm_i915_private *i915, const struct sseu_dev_info *sseu, struct drm_printer *p);
-u16 intel_slicemask_from_dssmask(u64 dss_mask, int dss_per_slice); +u16 intel_slicemask_from_xehp_dssmask(intel_sseu_ss_mask_t dss_mask, int dss_per_slice);
int intel_sseu_copy_eumask_to_user(void __user *to, const struct sseu_dev_info *sseu); +int intel_sseu_copy_ssmask_to_user(void __user *to,
const struct sseu_dev_info *sseu);+void intel_sseu_print_ss_info(const char* type,
const struct sseu_dev_info *sseu,struct seq_file *m);#endif /* __INTEL_SSEU_H__ */
diff --git a/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c b/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c index 2d5d011e01db..c2ee5e1826b5 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c +++ b/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c @@ -4,6 +4,7 @@
- Copyright © 2020 Intel Corporation
*/
+#include <linux/bitmap.h> #include <linux/string_helpers.h>
#include "i915_drv.h" @@ -11,14 +12,6 @@ #include "intel_gt_regs.h" #include "intel_sseu_debugfs.h"
-static void sseu_copy_subslices(const struct sseu_dev_info *sseu,
int slice, u8 *to_mask)-{
- int offset = slice * sseu->ss_stride;
- memcpy(&to_mask[offset], &sseu->subslice_mask[offset], sseu->ss_stride);
-}
- static void cherryview_sseu_device_status(struct intel_gt *gt, struct sseu_dev_info *sseu) {
@@ -41,7 +34,7 @@ static void cherryview_sseu_device_status(struct intel_gt *gt, continue;
sseu->slice_mask = BIT(0);
sseu->subslice_mask[0] |= BIT(ss);
sseu->subslice_mask.hsw[0] |= BIT(ss);@@ -92,7 +85,7 @@ static void gen11_sseu_device_status(struct intel_gt *gt, continue;
sseu->slice_mask |= BIT(s);
sseu_copy_subslices(&info->sseu, s, sseu->subslice_mask);
sseu->subslice_mask.hsw[s] = info->sseu.subslice_mask.hsw[s];for (ss = 0; ss < info->sseu.max_subslices; ss++) { unsigned int eu_cnt;
@@ -147,21 +140,17 @@ static void gen9_sseu_device_status(struct intel_gt *gt, sseu->slice_mask |= BIT(s);
if (IS_GEN9_BC(gt->i915))
sseu_copy_subslices(&info->sseu, s,sseu->subslice_mask);
sseu->subslice_mask.hsw[s] = info->sseu.subslice_mask.hsw[s];for (ss = 0; ss < info->sseu.max_subslices; ss++) { unsigned int eu_cnt;
u8 ss_idx = s * info->sseu.ss_stride +ss / BITS_PER_BYTE; if (IS_GEN9_LP(gt->i915)) { if (!(s_reg[s] & (GEN9_PGCTL_SS_ACK(ss)))) /* skip disabled subslice */ continue;sseu->subslice_mask[ss_idx] |=BIT(ss % BITS_PER_BYTE);
sseu->subslice_mask.hsw[s] |= BIT(ss); } eu_cnt = eu_reg[2 * s + ss / 2] & eu_mask[ss % 2];@@ -188,8 +177,7 @@ static void bdw_sseu_device_status(struct intel_gt *gt, if (sseu->slice_mask) { sseu->eu_per_subslice = info->sseu.eu_per_subslice; for (s = 0; s < fls(sseu->slice_mask); s++)
sseu_copy_subslices(&info->sseu, s,sseu->subslice_mask);
sseu->subslice_mask.hsw[s] = info->sseu.subslice_mask.hsw[s];@@ -208,7 +196,6 @@ static void i915_print_sseu_info(struct seq_file *m, const struct sseu_dev_info *sseu) { const char *type = is_available_info ? "Available" : "Enabled";
int s;
seq_printf(m, " %s Slice Mask: %04x\n", type, sseu->slice_mask);
@@ -216,10 +203,7 @@ static void i915_print_sseu_info(struct seq_file *m, hweight8(sseu->slice_mask)); seq_printf(m, " %s Subslice Total: %u\n", type, intel_sseu_subslice_total(sseu));
- for (s = 0; s < fls(sseu->slice_mask); s++) {
seq_printf(m, " %s Slice%i subslices: %u\n", type,s, intel_sseu_subslices_per_slice(sseu, s));- }
- intel_sseu_print_ss_info(type, sseu, m); seq_printf(m, " %s EU Total: %u\n", type, sseu->eu_total); seq_printf(m, " %s EU Per Subslice: %u\n", type,
diff --git a/drivers/gpu/drm/i915/gt/intel_workarounds.c b/drivers/gpu/drm/i915/gt/intel_workarounds.c index 73b59ea6fd3b..1bf62273e02d 100644 --- a/drivers/gpu/drm/i915/gt/intel_workarounds.c +++ b/drivers/gpu/drm/i915/gt/intel_workarounds.c @@ -948,8 +948,8 @@ gen9_wa_init_mcr(struct drm_i915_private *i915, struct i915_wa_list *wal) * on s/ss combo, the read should be done with read_subslice_reg. */ slice = ffs(sseu->slice_mask) - 1;
- GEM_BUG_ON(slice >= ARRAY_SIZE(sseu->subslice_mask));
- subslice = ffs(intel_sseu_get_subslices(sseu, slice));
- GEM_BUG_ON(slice >= ARRAY_SIZE(sseu->subslice_mask.hsw));
- subslice = ffs(intel_sseu_get_hsw_subslices(sseu, slice)); GEM_BUG_ON(!subslice); subslice--;
@@ -1087,11 +1087,10 @@ static void icl_wa_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) { const struct sseu_dev_info *sseu = >->info.sseu;
- unsigned int slice, subslice;
unsigned int subslice;
GEM_BUG_ON(GRAPHICS_VER(gt->i915) < 11); GEM_BUG_ON(hweight8(sseu->slice_mask) > 1);
slice = 0;
/*
- Although a platform may have subslices, we need to always steer
@@ -1102,7 +1101,7 @@ icl_wa_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) * one of the higher subslices, we run the risk of reading back 0's or * random garbage. */
- subslice = __ffs(intel_sseu_get_subslices(sseu, slice));
subslice = __ffs(intel_sseu_get_hsw_subslices(sseu, 0));
/*
- If the subslice we picked above also steers us to a valid L3 bank,
@@ -1112,7 +1111,7 @@ icl_wa_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) if (gt->info.l3bank_mask & BIT(subslice)) gt->steering_table[L3BANK] = NULL;
- __add_mcr_wa(gt, wal, slice, subslice);
__add_mcr_wa(gt, wal, 0, subslice); }
static void
@@ -1120,7 +1119,6 @@ xehp_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) { const struct sseu_dev_info *sseu = >->info.sseu; unsigned long slice, subslice = 0, slice_mask = 0;
- u64 dss_mask = 0; u32 lncf_mask = 0; int i;
@@ -1151,8 +1149,8 @@ xehp_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) */
/* Find the potential gslice candidates */
- dss_mask = intel_sseu_get_subslices(sseu, 0);
- slice_mask = intel_slicemask_from_dssmask(dss_mask, GEN_DSS_PER_GSLICE);
slice_mask = intel_slicemask_from_xehp_dssmask(sseu->subslice_mask,
GEN_DSS_PER_GSLICE);/*
- Find the potential LNCF candidates. Either LNCF within a valid
@@ -1177,9 +1175,8 @@ xehp_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) }
slice = __ffs(slice_mask);
- subslice = __ffs(dss_mask >> (slice * GEN_DSS_PER_GSLICE));
- subslice = intel_sseu_find_first_xehp_dss(sseu, GEN_DSS_PER_GSLICE, slice); WARN_ON(subslice > GEN_DSS_PER_GSLICE);
WARN_ON(dss_mask >> (slice * GEN_DSS_PER_GSLICE) == 0);
__add_mcr_wa(gt, wal, slice, subslice);
@@ -2030,9 +2027,8 @@ engine_fake_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
static bool needs_wa_1308578152(struct intel_engine_cs *engine) {
- u64 dss_mask = intel_sseu_get_subslices(&engine->gt->info.sseu, 0);
- return (dss_mask & GENMASK(GEN_DSS_PER_GSLICE - 1, 0)) == 0;
return intel_sseu_find_first_xehp_dss(&engine->gt->info.sseu, 0, 0) >
GEN_DSS_PER_GSLICE;}
static void
diff --git a/drivers/gpu/drm/i915/i915_getparam.c b/drivers/gpu/drm/i915/i915_getparam.c index ac9767c56619..6fd15b39570c 100644 --- a/drivers/gpu/drm/i915/i915_getparam.c +++ b/drivers/gpu/drm/i915/i915_getparam.c @@ -162,8 +162,7 @@ int i915_getparam_ioctl(struct drm_device *dev, void *data, return -EINVAL;
/* Only copy bits from the first slice */
memcpy(&value, sseu->subslice_mask,min(sseu->ss_stride, (u8)sizeof(value)));
value = intel_sseu_get_hsw_subslices(sseu, 0);diff --git a/drivers/gpu/drm/i915/i915_query.c b/drivers/gpu/drm/i915/i915_query.c index 89c475d525b8..0094f67c63f2 100644 --- a/drivers/gpu/drm/i915/i915_query.c +++ b/drivers/gpu/drm/i915/i915_query.c @@ -31,10 +31,11 @@ static int copy_query_item(void *query_hdr, size_t query_sz,
static int fill_topology_info(const struct sseu_dev_info *sseu, struct drm_i915_query_item *query_item,
const u8 *subslice_mask)
intel_sseu_ss_mask_t subslice_mask)- int ss_stride = GEN_SSEU_STRIDE(sseu->max_subslices); int eu_stride = GEN_SSEU_STRIDE(sseu->max_eus_per_subslice); int ret;
@@ -44,7 +45,7 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, return -ENODEV;
slice_length = sizeof(sseu->slice_mask);
- subslice_length = sseu->max_slices * sseu->ss_stride;
- subslice_length = sseu->max_slices * ss_stride; eu_length = sseu->max_slices * sseu->max_subslices * eu_stride; total_length = sizeof(topo) + slice_length + subslice_length + eu_length;
@@ -60,7 +61,7 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, topo.max_eus_per_subslice = sseu->max_eus_per_subslice;
topo.subslice_offset = slice_length;
- topo.subslice_stride = sseu->ss_stride;
- topo.subslice_stride = ss_stride; topo.eu_offset = slice_length + subslice_length; topo.eu_stride = eu_stride;
@@ -72,9 +73,9 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, &sseu->slice_mask, slice_length)) return -EFAULT;
- if (copy_to_user(u64_to_user_ptr(query_item->data_ptr +
sizeof(topo) + slice_length),subslice_mask, subslice_length))
if (intel_sseu_copy_ssmask_to_user(u64_to_user_ptr(query_item->data_ptr +
sizeof(topo) + slice_length),sseu))return -EFAULT;
if (intel_sseu_copy_eumask_to_user(u64_to_user_ptr(query_item->data_ptr +
As with EU masks, it's easier to store subslice/DSS masks internally in a format that's more natural for the driver to work with, and then only covert into the u8[] uapi form when the query ioctl is invoked. Since the hardware design changed significantly with Xe_HP, we'll use a union to choose between the old "hsw-style" subslice masks or the newer xehp mask. HSW-style masks will be stored in an array of u8's, indexed by slice (there's never more than 6 subslices per slice on older platforms). For Xe_HP and beyond where slices no longer exist, we only need a single bitmask. However we already know that this mask is eventually going to grow too large for a simple u64 to hold, so we'll represent it in a manner that can be operated on by the utilities in linux/bitmap.h.
v2: - Fix typo: BIT(s) -> BIT(ss) in gen9_sseu_device_status()
v3: - Eliminate sseu->ss_stride and just calculate the stride while specifically handling uapi. (Tvrtko) - Use BITMAP_BITS() macro to refer to size of masks rather than passing I915_MAX_SS_FUSE_BITS directly. (Tvrtko) - Report compute/geometry DSS masks separately when dumping Xe_HP SSEU info. (Tvrtko) - Restore dropped range checks to intel_sseu_has_subslice(). (Tvrtko)
v4: - Make the bitmap size macro check the size of the .xehp field rather than the containing union. (Tvrtko) - Don't add GEM_BUG_ON() intel_sseu_has_subslice()'s check for whether slice or subslice ID exceed sseu->max_[sub]slices; various loops in the driver are expected to exceed these, so we should just silently return 'false.'
Cc: Tvrtko Ursulin tvrtko.ursulin@linux.intel.com Signed-off-by: Matt Roper matthew.d.roper@intel.com --- drivers/gpu/drm/i915/gem/i915_gem_context.c | 5 +- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 4 +- drivers/gpu/drm/i915/gt/intel_gt.c | 12 +- drivers/gpu/drm/i915/gt/intel_sseu.c | 261 +++++++++++-------- drivers/gpu/drm/i915/gt/intel_sseu.h | 76 ++++-- drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c | 30 +-- drivers/gpu/drm/i915/gt/intel_workarounds.c | 24 +- drivers/gpu/drm/i915/i915_getparam.c | 3 +- drivers/gpu/drm/i915/i915_query.c | 13 +- 9 files changed, 241 insertions(+), 187 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index ab4c5ab28e4d..a3bb73f5d53b 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -1875,6 +1875,7 @@ i915_gem_user_to_context_sseu(struct intel_gt *gt, { const struct sseu_dev_info *device = >->info.sseu; struct drm_i915_private *i915 = gt->i915; + unsigned int dev_subslice_mask = intel_sseu_get_hsw_subslices(device, 0);
/* No zeros in any field. */ if (!user->slice_mask || !user->subslice_mask || @@ -1901,7 +1902,7 @@ i915_gem_user_to_context_sseu(struct intel_gt *gt, if (user->slice_mask & ~device->slice_mask) return -EINVAL;
- if (user->subslice_mask & ~device->subslice_mask[0]) + if (user->subslice_mask & ~dev_subslice_mask) return -EINVAL;
if (user->max_eus_per_subslice > device->max_eus_per_subslice) @@ -1915,7 +1916,7 @@ i915_gem_user_to_context_sseu(struct intel_gt *gt, /* Part specific restrictions. */ if (GRAPHICS_VER(i915) == 11) { unsigned int hw_s = hweight8(device->slice_mask); - unsigned int hw_ss_per_s = hweight8(device->subslice_mask[0]); + unsigned int hw_ss_per_s = hweight8(dev_subslice_mask); unsigned int req_s = hweight8(context->slice_mask); unsigned int req_ss = hweight8(context->subslice_mask);
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c index 1adbf34c3632..f0acf8518a51 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c @@ -674,8 +674,8 @@ static void engine_mask_apply_compute_fuses(struct intel_gt *gt) if (GRAPHICS_VER_FULL(i915) < IP_VER(12, 50)) return;
- ccs_mask = intel_slicemask_from_dssmask(intel_sseu_get_compute_subslices(&info->sseu), - ss_per_ccs); + ccs_mask = intel_slicemask_from_xehp_dssmask(info->sseu.compute_subslice_mask, + ss_per_ccs); /* * If all DSS in a quadrant are fused off, the corresponding CCS * engine is not available for use. diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c index 034182f85501..2921f510642f 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -133,13 +133,6 @@ static const struct intel_mmio_range dg2_lncf_steering_table[] = { {}, };
-static u16 slicemask(struct intel_gt *gt, int count) -{ - u64 dss_mask = intel_sseu_get_subslices(>->info.sseu, 0); - - return intel_slicemask_from_dssmask(dss_mask, count); -} - int intel_gt_init_mmio(struct intel_gt *gt) { struct drm_i915_private *i915 = gt->i915; @@ -155,9 +148,12 @@ int intel_gt_init_mmio(struct intel_gt *gt) */ if (HAS_MSLICES(i915)) { gt->info.mslice_mask = - slicemask(gt, GEN_DSS_PER_MSLICE) | + intel_slicemask_from_xehp_dssmask(gt->info.sseu.subslice_mask, + GEN_DSS_PER_MSLICE); + gt->info.mslice_mask |= (intel_uncore_read(gt->uncore, GEN10_MIRROR_FUSE3) & GEN12_MEML3_EN_MASK); + if (!gt->info.mslice_mask) /* should be impossible! */ drm_warn(&i915->drm, "mslice mask all zero!\n"); } diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.c b/drivers/gpu/drm/i915/gt/intel_sseu.c index 285cfd758bdc..3a721db20290 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu.c +++ b/drivers/gpu/drm/i915/gt/intel_sseu.c @@ -10,15 +10,14 @@ #include "intel_gt_regs.h" #include "intel_sseu.h"
+#define XEHP_BITMAP_BITS(mask) ((int)BITS_PER_TYPE(typeof(mask.xehp))) + void intel_sseu_set_info(struct sseu_dev_info *sseu, u8 max_slices, u8 max_subslices, u8 max_eus_per_subslice) { sseu->max_slices = max_slices; sseu->max_subslices = max_subslices; sseu->max_eus_per_subslice = max_eus_per_subslice; - - sseu->ss_stride = GEN_SSEU_STRIDE(sseu->max_subslices); - GEM_BUG_ON(sseu->ss_stride > GEN_MAX_SUBSLICE_STRIDE); }
unsigned int @@ -26,54 +25,24 @@ intel_sseu_subslice_total(const struct sseu_dev_info *sseu) { unsigned int i, total = 0;
- for (i = 0; i < ARRAY_SIZE(sseu->subslice_mask); i++) - total += hweight8(sseu->subslice_mask[i]); - - return total; -} - -static u32 -sseu_get_subslices(const struct sseu_dev_info *sseu, - const u8 *subslice_mask, u8 slice) -{ - int i, offset = slice * sseu->ss_stride; - u32 mask = 0; - - GEM_BUG_ON(slice >= sseu->max_slices); - - for (i = 0; i < sseu->ss_stride; i++) - mask |= (u32)subslice_mask[offset + i] << i * BITS_PER_BYTE; - - return mask; -} - -u32 intel_sseu_get_subslices(const struct sseu_dev_info *sseu, u8 slice) -{ - return sseu_get_subslices(sseu, sseu->subslice_mask, slice); -} + if (sseu->has_xehp_dss) + return bitmap_weight(sseu->subslice_mask.xehp, + XEHP_BITMAP_BITS(sseu->subslice_mask));
-static u32 sseu_get_geometry_subslices(const struct sseu_dev_info *sseu) -{ - return sseu_get_subslices(sseu, sseu->geometry_subslice_mask, 0); -} + for (i = 0; i < ARRAY_SIZE(sseu->subslice_mask.hsw); i++) + total += hweight8(sseu->subslice_mask.hsw[i]);
-u32 intel_sseu_get_compute_subslices(const struct sseu_dev_info *sseu) -{ - return sseu_get_subslices(sseu, sseu->compute_subslice_mask, 0); -} - -void intel_sseu_set_subslices(struct sseu_dev_info *sseu, int slice, - u8 *subslice_mask, u32 ss_mask) -{ - int offset = slice * sseu->ss_stride; - - memcpy(&subslice_mask[offset], &ss_mask, sseu->ss_stride); + return total; }
unsigned int -intel_sseu_subslices_per_slice(const struct sseu_dev_info *sseu, u8 slice) +intel_sseu_get_hsw_subslices(const struct sseu_dev_info *sseu, u8 slice) { - return hweight32(intel_sseu_get_subslices(sseu, slice)); + WARN_ON(sseu->has_xehp_dss); + if (WARN_ON(slice >= sseu->max_slices)) + return 0; + + return sseu->subslice_mask.hsw[slice]; }
static u16 sseu_get_eus(const struct sseu_dev_info *sseu, int slice, @@ -147,32 +116,66 @@ int intel_sseu_copy_eumask_to_user(void __user *to, return copy_to_user(to, eu_mask, len); }
+/** + * intel_sseu_copy_ssmask_to_user - Copy subslice mask into a userspace buffer + * @to: Pointer to userspace buffer to copy to + * @sseu: SSEU structure containing subslice mask to copy + * + * Copies the subslice mask to a userspace buffer in the format expected by + * the query ioctl's topology queries. + * + * Returns the result of the copy_to_user() operation. + */ +int intel_sseu_copy_ssmask_to_user(void __user *to, + const struct sseu_dev_info *sseu) +{ + u8 ss_mask[GEN_SS_MASK_SIZE] = {}; + int ss_stride = GEN_SSEU_STRIDE(sseu->max_subslices); + int len = sseu->max_slices * ss_stride; + int s, ss, i; + + for (s = 0; s < sseu->max_slices; s++) { + for (ss = 0; ss < sseu->max_subslices; ss++) { + i = s * ss_stride * BITS_PER_BYTE + ss; + + if (!intel_sseu_has_subslice(sseu, s, ss)) + continue; + + ss_mask[i / BITS_PER_BYTE] |= BIT(i % BITS_PER_BYTE); + } + } + + return copy_to_user(to, ss_mask, len); +} + static void gen11_compute_sseu_info(struct sseu_dev_info *sseu, - u32 g_ss_en, u32 c_ss_en, u16 eu_en) + u32 ss_en, u16 eu_en) { u32 valid_ss_mask = GENMASK(sseu->max_subslices - 1, 0); int ss;
- /* g_ss_en/c_ss_en represent entire subslice mask across all slices */ - GEM_BUG_ON(sseu->max_slices * sseu->max_subslices > - sizeof(g_ss_en) * BITS_PER_BYTE); + sseu->slice_mask |= BIT(0); + sseu->subslice_mask.hsw[0] = ss_en & valid_ss_mask; + + for (ss = 0; ss < sseu->max_subslices; ss++) + if (intel_sseu_has_subslice(sseu, 0, ss)) + sseu_set_eus(sseu, 0, ss, eu_en); + + sseu->eu_per_subslice = hweight16(eu_en); + sseu->eu_total = compute_eu_total(sseu); +} + +static void xehp_compute_sseu_info(struct sseu_dev_info *sseu, + u16 eu_en) +{ + int ss;
sseu->slice_mask |= BIT(0);
- /* - * XeHP introduces the concept of compute vs geometry DSS. To reduce - * variation between GENs around subslice usage, store a mask for both - * the geometry and compute enabled masks since userspace will need to - * be able to query these masks independently. Also compute a total - * enabled subslice count for the purposes of selecting subslices to - * use in a particular GEM context. - */ - intel_sseu_set_subslices(sseu, 0, sseu->compute_subslice_mask, - c_ss_en & valid_ss_mask); - intel_sseu_set_subslices(sseu, 0, sseu->geometry_subslice_mask, - g_ss_en & valid_ss_mask); - intel_sseu_set_subslices(sseu, 0, sseu->subslice_mask, - (g_ss_en | c_ss_en) & valid_ss_mask); + bitmap_or(sseu->subslice_mask.xehp, + sseu->compute_subslice_mask.xehp, + sseu->geometry_subslice_mask.xehp, + XEHP_BITMAP_BITS(sseu->subslice_mask));
for (ss = 0; ss < sseu->max_subslices; ss++) if (intel_sseu_has_subslice(sseu, 0, ss)) @@ -182,11 +185,31 @@ static void gen11_compute_sseu_info(struct sseu_dev_info *sseu, sseu->eu_total = compute_eu_total(sseu); }
+static void +xehp_load_dss_mask(struct intel_uncore *uncore, + intel_sseu_ss_mask_t *ssmask, + int numregs, + ...) +{ + va_list argp; + u32 fuse_val[I915_MAX_SS_FUSE_REGS] = {}; + int i; + + if (WARN_ON(numregs > I915_MAX_SS_FUSE_REGS)) + numregs = I915_MAX_SS_FUSE_REGS; + + va_start(argp, numregs); + for (i = 0; i < numregs; i++) + fuse_val[i] = intel_uncore_read(uncore, va_arg(argp, i915_reg_t)); + va_end(argp); + + bitmap_from_arr32(ssmask->xehp, fuse_val, numregs * 32); +} + static void xehp_sseu_info_init(struct intel_gt *gt) { struct sseu_dev_info *sseu = >->info.sseu; struct intel_uncore *uncore = gt->uncore; - u32 g_dss_en, c_dss_en = 0; u16 eu_en = 0; u8 eu_en_fuse; int eu; @@ -200,8 +223,10 @@ static void xehp_sseu_info_init(struct intel_gt *gt) intel_sseu_set_info(sseu, 1, 32, 16); sseu->has_xehp_dss = 1;
- g_dss_en = intel_uncore_read(uncore, GEN12_GT_GEOMETRY_DSS_ENABLE); - c_dss_en = intel_uncore_read(uncore, GEN12_GT_COMPUTE_DSS_ENABLE); + xehp_load_dss_mask(uncore, &sseu->geometry_subslice_mask, 1, + GEN12_GT_GEOMETRY_DSS_ENABLE); + xehp_load_dss_mask(uncore, &sseu->compute_subslice_mask, 1, + GEN12_GT_COMPUTE_DSS_ENABLE);
eu_en_fuse = intel_uncore_read(uncore, XEHP_EU_ENABLE) & XEHP_EU_ENA_MASK;
@@ -209,7 +234,7 @@ static void xehp_sseu_info_init(struct intel_gt *gt) if (eu_en_fuse & BIT(eu)) eu_en |= BIT(eu * 2) | BIT(eu * 2 + 1);
- gen11_compute_sseu_info(sseu, g_dss_en, c_dss_en, eu_en); + xehp_compute_sseu_info(sseu, eu_en); }
static void gen12_sseu_info_init(struct intel_gt *gt) @@ -247,7 +272,7 @@ static void gen12_sseu_info_init(struct intel_gt *gt) if (eu_en_fuse & BIT(eu)) eu_en |= BIT(eu * 2) | BIT(eu * 2 + 1);
- gen11_compute_sseu_info(sseu, g_dss_en, 0, eu_en); + gen11_compute_sseu_info(sseu, g_dss_en, eu_en);
/* TGL only supports slice-level power gating */ sseu->has_slice_pg = 1; @@ -279,7 +304,7 @@ static void gen11_sseu_info_init(struct intel_gt *gt) eu_en = ~(intel_uncore_read(uncore, GEN11_EU_DISABLE) & GEN11_EU_DIS_MASK);
- gen11_compute_sseu_info(sseu, ss_en, 0, eu_en); + gen11_compute_sseu_info(sseu, ss_en, eu_en);
/* ICL has no power gating restrictions. */ sseu->has_slice_pg = 1; @@ -291,7 +316,6 @@ static void cherryview_sseu_info_init(struct intel_gt *gt) { struct sseu_dev_info *sseu = >->info.sseu; u32 fuse; - u8 subslice_mask = 0;
fuse = intel_uncore_read(gt->uncore, CHV_FUSE_GT);
@@ -305,7 +329,7 @@ static void cherryview_sseu_info_init(struct intel_gt *gt) (((fuse & CHV_FGT_EU_DIS_SS0_R1_MASK) >> CHV_FGT_EU_DIS_SS0_R1_SHIFT) << 4);
- subslice_mask |= BIT(0); + sseu->subslice_mask.hsw[0] |= BIT(0); sseu_set_eus(sseu, 0, 0, ~disabled_mask & 0xFF); }
@@ -316,12 +340,10 @@ static void cherryview_sseu_info_init(struct intel_gt *gt) (((fuse & CHV_FGT_EU_DIS_SS1_R1_MASK) >> CHV_FGT_EU_DIS_SS1_R1_SHIFT) << 4);
- subslice_mask |= BIT(1); + sseu->subslice_mask.hsw[0] |= BIT(1); sseu_set_eus(sseu, 0, 1, ~disabled_mask & 0xFF); }
- intel_sseu_set_subslices(sseu, 0, sseu->subslice_mask, subslice_mask); - sseu->eu_total = compute_eu_total(sseu);
/* @@ -376,8 +398,7 @@ static void gen9_sseu_info_init(struct intel_gt *gt) /* skip disabled slice */ continue;
- intel_sseu_set_subslices(sseu, s, sseu->subslice_mask, - subslice_mask); + sseu->subslice_mask.hsw[s] = subslice_mask;
eu_disable = intel_uncore_read(uncore, GEN9_EU_DISABLE(s)); for (ss = 0; ss < sseu->max_subslices; ss++) { @@ -434,8 +455,8 @@ static void gen9_sseu_info_init(struct intel_gt *gt) sseu->has_eu_pg = sseu->eu_per_subslice > 2;
if (IS_GEN9_LP(i915)) { -#define IS_SS_DISABLED(ss) (!(sseu->subslice_mask[0] & BIT(ss))) - info->has_pooled_eu = hweight8(sseu->subslice_mask[0]) == 3; +#define IS_SS_DISABLED(ss) (!(sseu->subslice_mask.hsw[0] & BIT(ss))) + info->has_pooled_eu = hweight8(sseu->subslice_mask.hsw[0]) == 3;
sseu->min_eu_in_pool = 0; if (info->has_pooled_eu) { @@ -489,8 +510,7 @@ static void bdw_sseu_info_init(struct intel_gt *gt) /* skip disabled slice */ continue;
- intel_sseu_set_subslices(sseu, s, sseu->subslice_mask, - subslice_mask); + sseu->subslice_mask.hsw[s] = subslice_mask;
for (ss = 0; ss < sseu->max_subslices; ss++) { u8 eu_disabled_mask; @@ -587,8 +607,7 @@ static void hsw_sseu_info_init(struct intel_gt *gt) sseu->eu_per_subslice);
for (s = 0; s < sseu->max_slices; s++) { - intel_sseu_set_subslices(sseu, s, sseu->subslice_mask, - subslice_mask); + sseu->subslice_mask.hsw[s] = subslice_mask;
for (ss = 0; ss < sseu->max_subslices; ss++) { sseu_set_eus(sseu, s, ss, @@ -677,7 +696,7 @@ u32 intel_sseu_make_rpcs(struct intel_gt *gt, */ if (GRAPHICS_VER(i915) == 11 && slices == 1 && - subslices > min_t(u8, 4, hweight8(sseu->subslice_mask[0]) / 2)) { + subslices > min_t(u8, 4, hweight8(sseu->subslice_mask.hsw[0]) / 2)) { GEM_BUG_ON(subslices & 1);
subslice_pg = false; @@ -743,14 +762,29 @@ void intel_sseu_dump(const struct sseu_dev_info *sseu, struct drm_printer *p) { int s;
- drm_printf(p, "slice total: %u, mask=%04x\n", - hweight8(sseu->slice_mask), sseu->slice_mask); - drm_printf(p, "subslice total: %u\n", intel_sseu_subslice_total(sseu)); - for (s = 0; s < sseu->max_slices; s++) { - drm_printf(p, "slice%d: %u subslices, mask=%08x\n", - s, intel_sseu_subslices_per_slice(sseu, s), - intel_sseu_get_subslices(sseu, s)); + if (sseu->has_xehp_dss) { + drm_printf(p, "subslice total: %u\n", + intel_sseu_subslice_total(sseu)); + drm_printf(p, "geometry dss mask=%*pb\n", + XEHP_BITMAP_BITS(sseu->geometry_subslice_mask), + sseu->geometry_subslice_mask.xehp); + drm_printf(p, "compute dss mask=%*pb\n", + XEHP_BITMAP_BITS(sseu->compute_subslice_mask), + sseu->compute_subslice_mask.xehp); + } else { + drm_printf(p, "slice total: %u, mask=%04x\n", + hweight8(sseu->slice_mask), sseu->slice_mask); + drm_printf(p, "subslice total: %u\n", + intel_sseu_subslice_total(sseu)); + + for (s = 0; s < sseu->max_slices; s++) { + u8 ss_mask = sseu->subslice_mask.hsw[s]; + + drm_printf(p, "slice%d: %u subslices, mask=%08x\n", + s, hweight8(ss_mask), ss_mask); + } } + drm_printf(p, "EU total: %u\n", sseu->eu_total); drm_printf(p, "EU per subslice: %u\n", sseu->eu_per_subslice); drm_printf(p, "has slice power gating: %s\n", @@ -767,9 +801,10 @@ static void sseu_print_hsw_topology(const struct sseu_dev_info *sseu, int s, ss;
for (s = 0; s < sseu->max_slices; s++) { + u8 ss_mask = sseu->subslice_mask.hsw[s]; + drm_printf(p, "slice%d: %u subslice(s) (0x%08x):\n", - s, intel_sseu_subslices_per_slice(sseu, s), - intel_sseu_get_subslices(sseu, s)); + s, hweight8(ss_mask), ss_mask);
for (ss = 0; ss < sseu->max_subslices; ss++) { u16 enabled_eus = sseu_get_eus(sseu, s, ss); @@ -783,16 +818,14 @@ static void sseu_print_hsw_topology(const struct sseu_dev_info *sseu, static void sseu_print_xehp_topology(const struct sseu_dev_info *sseu, struct drm_printer *p) { - u32 g_dss_mask = sseu_get_geometry_subslices(sseu); - u32 c_dss_mask = intel_sseu_get_compute_subslices(sseu); int dss;
for (dss = 0; dss < sseu->max_subslices; dss++) { u16 enabled_eus = sseu_get_eus(sseu, 0, dss);
drm_printf(p, "DSS_%02d: G:%3s C:%3s, %2u EUs (0x%04hx)\n", dss, - str_yes_no(g_dss_mask & BIT(dss)), - str_yes_no(c_dss_mask & BIT(dss)), + str_yes_no(test_bit(dss, sseu->geometry_subslice_mask.xehp)), + str_yes_no(test_bit(dss, sseu->compute_subslice_mask.xehp)), hweight16(enabled_eus), enabled_eus); } } @@ -810,20 +843,44 @@ void intel_sseu_print_topology(struct drm_i915_private *i915, } }
-u16 intel_slicemask_from_dssmask(u64 dss_mask, int dss_per_slice) +void intel_sseu_print_ss_info(const char* type, + const struct sseu_dev_info *sseu, + struct seq_file *m) +{ + int s; + + if (sseu->has_xehp_dss) { + seq_printf(m, " %s Geometry DSS: %u\n", type, + bitmap_weight(sseu->geometry_subslice_mask.xehp, + XEHP_BITMAP_BITS(sseu->geometry_subslice_mask))); + seq_printf(m, " %s Compute DSS: %u\n", type, + bitmap_weight(sseu->compute_subslice_mask.xehp, + XEHP_BITMAP_BITS(sseu->compute_subslice_mask))); + } else { + for (s = 0; s < fls(sseu->slice_mask); s++) + seq_printf(m, " %s Slice%i subslices: %u\n", type, + s, hweight8(sseu->subslice_mask.hsw[s])); + } +} + +u16 intel_slicemask_from_xehp_dssmask(intel_sseu_ss_mask_t dss_mask, + int dss_per_slice) { + intel_sseu_ss_mask_t per_slice_mask = {}; u16 slice_mask = 0; int i;
- WARN_ON(sizeof(dss_mask) * 8 / dss_per_slice > 8 * sizeof(slice_mask)); + WARN_ON(DIV_ROUND_UP(XEHP_BITMAP_BITS(dss_mask), dss_per_slice) > + 8 * sizeof(slice_mask));
- for (i = 0; dss_mask; i++) { - if (dss_mask & GENMASK(dss_per_slice - 1, 0)) + bitmap_fill(per_slice_mask.xehp, dss_per_slice); + for (i = 0; !bitmap_empty(dss_mask.xehp, XEHP_BITMAP_BITS(dss_mask)); i++) { + if (bitmap_intersects(dss_mask.xehp, per_slice_mask.xehp, dss_per_slice)) slice_mask |= BIT(i);
- dss_mask >>= dss_per_slice; + bitmap_shift_right(dss_mask.xehp, dss_mask.xehp, dss_per_slice, + XEHP_BITMAP_BITS(dss_mask)); }
return slice_mask; } - diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.h b/drivers/gpu/drm/i915/gt/intel_sseu.h index ffa375e68959..e7f336a7ffbb 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu.h +++ b/drivers/gpu/drm/i915/gt/intel_sseu.h @@ -25,12 +25,16 @@ struct drm_printer; /* * Maximum number of subslices that can exist within a HSW-style slice. This * is only relevant to pre-Xe_HP platforms (Xe_HP and beyond use the - * GEN_MAX_DSS value below). + * I915_MAX_SS_FUSE_BITS value below). */ #define GEN_MAX_SS_PER_HSW_SLICE 6
-/* Maximum number of DSS on newer platforms (Xe_HP and beyond). */ -#define GEN_MAX_DSS 32 +/* + * Maximum number of 32-bit registers used by hardware to express the + * enabled/disabled subslices. + */ +#define I915_MAX_SS_FUSE_REGS 1 +#define I915_MAX_SS_FUSE_BITS (I915_MAX_SS_FUSE_REGS * 32)
/* Maximum number of EUs that can exist within a subslice or DSS. */ #define GEN_MAX_EUS_PER_SS 16 @@ -38,7 +42,7 @@ struct drm_printer; #define SSEU_MAX(a, b) ((a) > (b) ? (a) : (b))
/* The maximum number of bits needed to express each subslice/DSS independently */ -#define GEN_SS_MASK_SIZE SSEU_MAX(GEN_MAX_DSS, \ +#define GEN_SS_MASK_SIZE SSEU_MAX(I915_MAX_SS_FUSE_BITS, \ GEN_MAX_HSW_SLICES * GEN_MAX_SS_PER_HSW_SLICE)
#define GEN_SSEU_STRIDE(max_entries) DIV_ROUND_UP(max_entries, BITS_PER_BYTE) @@ -49,17 +53,24 @@ struct drm_printer; #define GEN_DSS_PER_CSLICE 8 #define GEN_DSS_PER_MSLICE 8
-#define GEN_MAX_GSLICES (GEN_MAX_DSS / GEN_DSS_PER_GSLICE) -#define GEN_MAX_CSLICES (GEN_MAX_DSS / GEN_DSS_PER_CSLICE) +#define GEN_MAX_GSLICES (I915_MAX_SS_FUSE_BITS / GEN_DSS_PER_GSLICE) +#define GEN_MAX_CSLICES (I915_MAX_SS_FUSE_BITS / GEN_DSS_PER_CSLICE) + +typedef union { + u8 hsw[GEN_MAX_HSW_SLICES]; + + /* Bitmap compatible with linux/bitmap.h; may exceed size of u64 */ + unsigned long xehp[BITS_TO_LONGS(I915_MAX_SS_FUSE_BITS)]; +} intel_sseu_ss_mask_t;
struct sseu_dev_info { u8 slice_mask; - u8 subslice_mask[GEN_SS_MASK_SIZE]; - u8 geometry_subslice_mask[GEN_SS_MASK_SIZE]; - u8 compute_subslice_mask[GEN_SS_MASK_SIZE]; + intel_sseu_ss_mask_t subslice_mask; + intel_sseu_ss_mask_t geometry_subslice_mask; + intel_sseu_ss_mask_t compute_subslice_mask; union { u16 hsw[GEN_MAX_HSW_SLICES][GEN_MAX_SS_PER_HSW_SLICE]; - u16 xehp[GEN_MAX_DSS]; + u16 xehp[I915_MAX_SS_FUSE_BITS]; } eu_mask;
u16 eu_total; @@ -80,8 +91,6 @@ struct sseu_dev_info { u8 max_slices; u8 max_subslices; u8 max_eus_per_subslice; - - u8 ss_stride; };
/* @@ -99,7 +108,7 @@ intel_sseu_from_device_info(const struct sseu_dev_info *sseu) { struct intel_sseu value = { .slice_mask = sseu->slice_mask, - .subslice_mask = sseu->subslice_mask[0], + .subslice_mask = sseu->subslice_mask.hsw[0], .min_eus_per_subslice = sseu->max_eus_per_subslice, .max_eus_per_subslice = sseu->max_eus_per_subslice, }; @@ -111,18 +120,27 @@ static inline bool intel_sseu_has_subslice(const struct sseu_dev_info *sseu, int slice, int subslice) { - u8 mask; - int ss_idx = subslice / BITS_PER_BYTE; - if (slice >= sseu->max_slices || subslice >= sseu->max_subslices) return false;
- GEM_BUG_ON(ss_idx >= sseu->ss_stride); - - mask = sseu->subslice_mask[slice * sseu->ss_stride + ss_idx]; + if (sseu->has_xehp_dss) + return test_bit(subslice, sseu->subslice_mask.xehp); + else + return sseu->subslice_mask.hsw[slice] & BIT(subslice); +}
- return mask & BIT(subslice % BITS_PER_BYTE); +/* + * Used to obtain the index of the first DSS. Can start searching from the + * beginning of a specific dss group (e.g., gslice, cslice, etc.) if + * groupsize and groupnum are non-zero. + */ +static inline unsigned int +intel_sseu_find_first_xehp_dss(const struct sseu_dev_info *sseu, int groupsize, + int groupnum) +{ + return find_next_bit(sseu->subslice_mask.xehp, I915_MAX_SS_FUSE_BITS, + groupnum * groupsize); }
void intel_sseu_set_info(struct sseu_dev_info *sseu, u8 max_slices, @@ -132,14 +150,10 @@ unsigned int intel_sseu_subslice_total(const struct sseu_dev_info *sseu);
unsigned int -intel_sseu_subslices_per_slice(const struct sseu_dev_info *sseu, u8 slice); - -u32 intel_sseu_get_subslices(const struct sseu_dev_info *sseu, u8 slice); +intel_sseu_get_hsw_subslices(const struct sseu_dev_info *sseu, u8 slice);
-u32 intel_sseu_get_compute_subslices(const struct sseu_dev_info *sseu); - -void intel_sseu_set_subslices(struct sseu_dev_info *sseu, int slice, - u8 *subslice_mask, u32 ss_mask); +intel_sseu_ss_mask_t +intel_sseu_get_compute_subslices(const struct sseu_dev_info *sseu);
void intel_sseu_info_init(struct intel_gt *gt);
@@ -151,9 +165,15 @@ void intel_sseu_print_topology(struct drm_i915_private *i915, const struct sseu_dev_info *sseu, struct drm_printer *p);
-u16 intel_slicemask_from_dssmask(u64 dss_mask, int dss_per_slice); +u16 intel_slicemask_from_xehp_dssmask(intel_sseu_ss_mask_t dss_mask, int dss_per_slice);
int intel_sseu_copy_eumask_to_user(void __user *to, const struct sseu_dev_info *sseu); +int intel_sseu_copy_ssmask_to_user(void __user *to, + const struct sseu_dev_info *sseu); + +void intel_sseu_print_ss_info(const char* type, + const struct sseu_dev_info *sseu, + struct seq_file *m);
#endif /* __INTEL_SSEU_H__ */ diff --git a/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c b/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c index 2d5d011e01db..c2ee5e1826b5 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c +++ b/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c @@ -4,6 +4,7 @@ * Copyright © 2020 Intel Corporation */
+#include <linux/bitmap.h> #include <linux/string_helpers.h>
#include "i915_drv.h" @@ -11,14 +12,6 @@ #include "intel_gt_regs.h" #include "intel_sseu_debugfs.h"
-static void sseu_copy_subslices(const struct sseu_dev_info *sseu, - int slice, u8 *to_mask) -{ - int offset = slice * sseu->ss_stride; - - memcpy(&to_mask[offset], &sseu->subslice_mask[offset], sseu->ss_stride); -} - static void cherryview_sseu_device_status(struct intel_gt *gt, struct sseu_dev_info *sseu) { @@ -41,7 +34,7 @@ static void cherryview_sseu_device_status(struct intel_gt *gt, continue;
sseu->slice_mask = BIT(0); - sseu->subslice_mask[0] |= BIT(ss); + sseu->subslice_mask.hsw[0] |= BIT(ss); eu_cnt = ((sig1[ss] & CHV_EU08_PG_ENABLE) ? 0 : 2) + ((sig1[ss] & CHV_EU19_PG_ENABLE) ? 0 : 2) + ((sig1[ss] & CHV_EU210_PG_ENABLE) ? 0 : 2) + @@ -92,7 +85,7 @@ static void gen11_sseu_device_status(struct intel_gt *gt, continue;
sseu->slice_mask |= BIT(s); - sseu_copy_subslices(&info->sseu, s, sseu->subslice_mask); + sseu->subslice_mask.hsw[s] = info->sseu.subslice_mask.hsw[s];
for (ss = 0; ss < info->sseu.max_subslices; ss++) { unsigned int eu_cnt; @@ -147,21 +140,17 @@ static void gen9_sseu_device_status(struct intel_gt *gt, sseu->slice_mask |= BIT(s);
if (IS_GEN9_BC(gt->i915)) - sseu_copy_subslices(&info->sseu, s, - sseu->subslice_mask); + sseu->subslice_mask.hsw[s] = info->sseu.subslice_mask.hsw[s];
for (ss = 0; ss < info->sseu.max_subslices; ss++) { unsigned int eu_cnt; - u8 ss_idx = s * info->sseu.ss_stride + - ss / BITS_PER_BYTE;
if (IS_GEN9_LP(gt->i915)) { if (!(s_reg[s] & (GEN9_PGCTL_SS_ACK(ss)))) /* skip disabled subslice */ continue;
- sseu->subslice_mask[ss_idx] |= - BIT(ss % BITS_PER_BYTE); + sseu->subslice_mask.hsw[s] |= BIT(ss); }
eu_cnt = eu_reg[2 * s + ss / 2] & eu_mask[ss % 2]; @@ -188,8 +177,7 @@ static void bdw_sseu_device_status(struct intel_gt *gt, if (sseu->slice_mask) { sseu->eu_per_subslice = info->sseu.eu_per_subslice; for (s = 0; s < fls(sseu->slice_mask); s++) - sseu_copy_subslices(&info->sseu, s, - sseu->subslice_mask); + sseu->subslice_mask.hsw[s] = info->sseu.subslice_mask.hsw[s]; sseu->eu_total = sseu->eu_per_subslice * intel_sseu_subslice_total(sseu);
@@ -208,7 +196,6 @@ static void i915_print_sseu_info(struct seq_file *m, const struct sseu_dev_info *sseu) { const char *type = is_available_info ? "Available" : "Enabled"; - int s;
seq_printf(m, " %s Slice Mask: %04x\n", type, sseu->slice_mask); @@ -216,10 +203,7 @@ static void i915_print_sseu_info(struct seq_file *m, hweight8(sseu->slice_mask)); seq_printf(m, " %s Subslice Total: %u\n", type, intel_sseu_subslice_total(sseu)); - for (s = 0; s < fls(sseu->slice_mask); s++) { - seq_printf(m, " %s Slice%i subslices: %u\n", type, - s, intel_sseu_subslices_per_slice(sseu, s)); - } + intel_sseu_print_ss_info(type, sseu, m); seq_printf(m, " %s EU Total: %u\n", type, sseu->eu_total); seq_printf(m, " %s EU Per Subslice: %u\n", type, diff --git a/drivers/gpu/drm/i915/gt/intel_workarounds.c b/drivers/gpu/drm/i915/gt/intel_workarounds.c index 73b59ea6fd3b..1bf62273e02d 100644 --- a/drivers/gpu/drm/i915/gt/intel_workarounds.c +++ b/drivers/gpu/drm/i915/gt/intel_workarounds.c @@ -948,8 +948,8 @@ gen9_wa_init_mcr(struct drm_i915_private *i915, struct i915_wa_list *wal) * on s/ss combo, the read should be done with read_subslice_reg. */ slice = ffs(sseu->slice_mask) - 1; - GEM_BUG_ON(slice >= ARRAY_SIZE(sseu->subslice_mask)); - subslice = ffs(intel_sseu_get_subslices(sseu, slice)); + GEM_BUG_ON(slice >= ARRAY_SIZE(sseu->subslice_mask.hsw)); + subslice = ffs(intel_sseu_get_hsw_subslices(sseu, slice)); GEM_BUG_ON(!subslice); subslice--;
@@ -1087,11 +1087,10 @@ static void icl_wa_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) { const struct sseu_dev_info *sseu = >->info.sseu; - unsigned int slice, subslice; + unsigned int subslice;
GEM_BUG_ON(GRAPHICS_VER(gt->i915) < 11); GEM_BUG_ON(hweight8(sseu->slice_mask) > 1); - slice = 0;
/* * Although a platform may have subslices, we need to always steer @@ -1102,7 +1101,7 @@ icl_wa_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) * one of the higher subslices, we run the risk of reading back 0's or * random garbage. */ - subslice = __ffs(intel_sseu_get_subslices(sseu, slice)); + subslice = __ffs(intel_sseu_get_hsw_subslices(sseu, 0));
/* * If the subslice we picked above also steers us to a valid L3 bank, @@ -1112,7 +1111,7 @@ icl_wa_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) if (gt->info.l3bank_mask & BIT(subslice)) gt->steering_table[L3BANK] = NULL;
- __add_mcr_wa(gt, wal, slice, subslice); + __add_mcr_wa(gt, wal, 0, subslice); }
static void @@ -1120,7 +1119,6 @@ xehp_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) { const struct sseu_dev_info *sseu = >->info.sseu; unsigned long slice, subslice = 0, slice_mask = 0; - u64 dss_mask = 0; u32 lncf_mask = 0; int i;
@@ -1151,8 +1149,8 @@ xehp_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) */
/* Find the potential gslice candidates */ - dss_mask = intel_sseu_get_subslices(sseu, 0); - slice_mask = intel_slicemask_from_dssmask(dss_mask, GEN_DSS_PER_GSLICE); + slice_mask = intel_slicemask_from_xehp_dssmask(sseu->subslice_mask, + GEN_DSS_PER_GSLICE);
/* * Find the potential LNCF candidates. Either LNCF within a valid @@ -1177,9 +1175,8 @@ xehp_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) }
slice = __ffs(slice_mask); - subslice = __ffs(dss_mask >> (slice * GEN_DSS_PER_GSLICE)); + subslice = intel_sseu_find_first_xehp_dss(sseu, GEN_DSS_PER_GSLICE, slice); WARN_ON(subslice > GEN_DSS_PER_GSLICE); - WARN_ON(dss_mask >> (slice * GEN_DSS_PER_GSLICE) == 0);
__add_mcr_wa(gt, wal, slice, subslice);
@@ -2030,9 +2027,8 @@ engine_fake_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
static bool needs_wa_1308578152(struct intel_engine_cs *engine) { - u64 dss_mask = intel_sseu_get_subslices(&engine->gt->info.sseu, 0); - - return (dss_mask & GENMASK(GEN_DSS_PER_GSLICE - 1, 0)) == 0; + return intel_sseu_find_first_xehp_dss(&engine->gt->info.sseu, 0, 0) > + GEN_DSS_PER_GSLICE; }
static void diff --git a/drivers/gpu/drm/i915/i915_getparam.c b/drivers/gpu/drm/i915/i915_getparam.c index ac9767c56619..6fd15b39570c 100644 --- a/drivers/gpu/drm/i915/i915_getparam.c +++ b/drivers/gpu/drm/i915/i915_getparam.c @@ -162,8 +162,7 @@ int i915_getparam_ioctl(struct drm_device *dev, void *data, return -EINVAL;
/* Only copy bits from the first slice */ - memcpy(&value, sseu->subslice_mask, - min(sseu->ss_stride, (u8)sizeof(value))); + value = intel_sseu_get_hsw_subslices(sseu, 0); if (!value) return -ENODEV; break; diff --git a/drivers/gpu/drm/i915/i915_query.c b/drivers/gpu/drm/i915/i915_query.c index 89c475d525b8..0094f67c63f2 100644 --- a/drivers/gpu/drm/i915/i915_query.c +++ b/drivers/gpu/drm/i915/i915_query.c @@ -31,10 +31,11 @@ static int copy_query_item(void *query_hdr, size_t query_sz,
static int fill_topology_info(const struct sseu_dev_info *sseu, struct drm_i915_query_item *query_item, - const u8 *subslice_mask) + intel_sseu_ss_mask_t subslice_mask) { struct drm_i915_query_topology_info topo; u32 slice_length, subslice_length, eu_length, total_length; + int ss_stride = GEN_SSEU_STRIDE(sseu->max_subslices); int eu_stride = GEN_SSEU_STRIDE(sseu->max_eus_per_subslice); int ret;
@@ -44,7 +45,7 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, return -ENODEV;
slice_length = sizeof(sseu->slice_mask); - subslice_length = sseu->max_slices * sseu->ss_stride; + subslice_length = sseu->max_slices * ss_stride; eu_length = sseu->max_slices * sseu->max_subslices * eu_stride; total_length = sizeof(topo) + slice_length + subslice_length + eu_length; @@ -60,7 +61,7 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, topo.max_eus_per_subslice = sseu->max_eus_per_subslice;
topo.subslice_offset = slice_length; - topo.subslice_stride = sseu->ss_stride; + topo.subslice_stride = ss_stride; topo.eu_offset = slice_length + subslice_length; topo.eu_stride = eu_stride;
@@ -72,9 +73,9 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, &sseu->slice_mask, slice_length)) return -EFAULT;
- if (copy_to_user(u64_to_user_ptr(query_item->data_ptr + - sizeof(topo) + slice_length), - subslice_mask, subslice_length)) + if (intel_sseu_copy_ssmask_to_user(u64_to_user_ptr(query_item->data_ptr + + sizeof(topo) + slice_length), + sseu)) return -EFAULT;
if (intel_sseu_copy_eumask_to_user(u64_to_user_ptr(query_item->data_ptr +
On 23.05.2022 13:45, Matt Roper wrote:
As with EU masks, it's easier to store subslice/DSS masks internally in a format that's more natural for the driver to work with, and then only covert into the u8[] uapi form when the query ioctl is invoked. Since the hardware design changed significantly with Xe_HP, we'll use a union to choose between the old "hsw-style" subslice masks or the newer xehp mask. HSW-style masks will be stored in an array of u8's, indexed by slice (there's never more than 6 subslices per slice on older platforms). For Xe_HP and beyond where slices no longer exist, we only need a single bitmask. However we already know that this mask is eventually going to grow too large for a simple u64 to hold, so we'll represent it in a manner that can be operated on by the utilities in linux/bitmap.h.
v2:
- Fix typo: BIT(s) -> BIT(ss) in gen9_sseu_device_status()
v3:
- Eliminate sseu->ss_stride and just calculate the stride while specifically handling uapi. (Tvrtko)
- Use BITMAP_BITS() macro to refer to size of masks rather than passing I915_MAX_SS_FUSE_BITS directly. (Tvrtko)
- Report compute/geometry DSS masks separately when dumping Xe_HP SSEU info. (Tvrtko)
- Restore dropped range checks to intel_sseu_has_subslice(). (Tvrtko)
v4:
- Make the bitmap size macro check the size of the .xehp field rather than the containing union. (Tvrtko)
- Don't add GEM_BUG_ON() intel_sseu_has_subslice()'s check for whether slice or subslice ID exceed sseu->max_[sub]slices; various loops in the driver are expected to exceed these, so we should just silently return 'false.'
Cc: Tvrtko Ursulin tvrtko.ursulin@linux.intel.com Signed-off-by: Matt Roper matthew.d.roper@intel.com
drivers/gpu/drm/i915/gem/i915_gem_context.c | 5 +- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 4 +- drivers/gpu/drm/i915/gt/intel_gt.c | 12 +- drivers/gpu/drm/i915/gt/intel_sseu.c | 261 +++++++++++-------- drivers/gpu/drm/i915/gt/intel_sseu.h | 76 ++++-- drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c | 30 +-- drivers/gpu/drm/i915/gt/intel_workarounds.c | 24 +- drivers/gpu/drm/i915/i915_getparam.c | 3 +- drivers/gpu/drm/i915/i915_query.c | 13 +- 9 files changed, 241 insertions(+), 187 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index ab4c5ab28e4d..a3bb73f5d53b 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -1875,6 +1875,7 @@ i915_gem_user_to_context_sseu(struct intel_gt *gt, { const struct sseu_dev_info *device = >->info.sseu; struct drm_i915_private *i915 = gt->i915;
unsigned int dev_subslice_mask = intel_sseu_get_hsw_subslices(device, 0);
/* No zeros in any field. */ if (!user->slice_mask || !user->subslice_mask ||
@@ -1901,7 +1902,7 @@ i915_gem_user_to_context_sseu(struct intel_gt *gt, if (user->slice_mask & ~device->slice_mask) return -EINVAL;
- if (user->subslice_mask & ~device->subslice_mask[0])
if (user->subslice_mask & ~dev_subslice_mask) return -EINVAL;
if (user->max_eus_per_subslice > device->max_eus_per_subslice)
@@ -1915,7 +1916,7 @@ i915_gem_user_to_context_sseu(struct intel_gt *gt, /* Part specific restrictions. */ if (GRAPHICS_VER(i915) == 11) { unsigned int hw_s = hweight8(device->slice_mask);
unsigned int hw_ss_per_s = hweight8(device->subslice_mask[0]);
unsigned int hw_ss_per_s = hweight8(dev_subslice_mask);diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c index 1adbf34c3632..f0acf8518a51 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c @@ -674,8 +674,8 @@ static void engine_mask_apply_compute_fuses(struct intel_gt *gt) if (GRAPHICS_VER_FULL(i915) < IP_VER(12, 50)) return;
- ccs_mask = intel_slicemask_from_dssmask(intel_sseu_get_compute_subslices(&info->sseu),
ss_per_ccs);
- ccs_mask = intel_slicemask_from_xehp_dssmask(info->sseu.compute_subslice_mask,
ss_per_ccs);
- If all DSS in a quadrant are fused off, the corresponding CCS
- engine is not available for use.
diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c index 034182f85501..2921f510642f 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -133,13 +133,6 @@ static const struct intel_mmio_range dg2_lncf_steering_table[] = { {}, };
-static u16 slicemask(struct intel_gt *gt, int count) -{
- u64 dss_mask = intel_sseu_get_subslices(>->info.sseu, 0);
- return intel_slicemask_from_dssmask(dss_mask, count);
-}
int intel_gt_init_mmio(struct intel_gt *gt) { struct drm_i915_private *i915 = gt->i915; @@ -155,9 +148,12 @@ int intel_gt_init_mmio(struct intel_gt *gt) */ if (HAS_MSLICES(i915)) { gt->info.mslice_mask =
slicemask(gt, GEN_DSS_PER_MSLICE) |
intel_slicemask_from_xehp_dssmask(gt->info.sseu.subslice_mask,GEN_DSS_PER_MSLICE);gt->info.mslice_mask |= (intel_uncore_read(gt->uncore, GEN10_MIRROR_FUSE3) & GEN12_MEML3_EN_MASK);- if (!gt->info.mslice_mask) /* should be impossible! */ drm_warn(&i915->drm, "mslice mask all zero!\n"); }
diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.c b/drivers/gpu/drm/i915/gt/intel_sseu.c index 285cfd758bdc..3a721db20290 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu.c +++ b/drivers/gpu/drm/i915/gt/intel_sseu.c @@ -10,15 +10,14 @@ #include "intel_gt_regs.h" #include "intel_sseu.h"
+#define XEHP_BITMAP_BITS(mask) ((int)BITS_PER_TYPE(typeof(mask.xehp)))
void intel_sseu_set_info(struct sseu_dev_info *sseu, u8 max_slices, u8 max_subslices, u8 max_eus_per_subslice) { sseu->max_slices = max_slices; sseu->max_subslices = max_subslices; sseu->max_eus_per_subslice = max_eus_per_subslice;
- sseu->ss_stride = GEN_SSEU_STRIDE(sseu->max_subslices);
- GEM_BUG_ON(sseu->ss_stride > GEN_MAX_SUBSLICE_STRIDE);
}
unsigned int @@ -26,54 +25,24 @@ intel_sseu_subslice_total(const struct sseu_dev_info *sseu) { unsigned int i, total = 0;
- for (i = 0; i < ARRAY_SIZE(sseu->subslice_mask); i++)
total += hweight8(sseu->subslice_mask[i]);- return total;
-}
-static u32 -sseu_get_subslices(const struct sseu_dev_info *sseu,
const u8 *subslice_mask, u8 slice)-{
- int i, offset = slice * sseu->ss_stride;
- u32 mask = 0;
- GEM_BUG_ON(slice >= sseu->max_slices);
- for (i = 0; i < sseu->ss_stride; i++)
mask |= (u32)subslice_mask[offset + i] << i * BITS_PER_BYTE;- return mask;
-}
-u32 intel_sseu_get_subslices(const struct sseu_dev_info *sseu, u8 slice) -{
- return sseu_get_subslices(sseu, sseu->subslice_mask, slice);
-}
- if (sseu->has_xehp_dss)
return bitmap_weight(sseu->subslice_mask.xehp,XEHP_BITMAP_BITS(sseu->subslice_mask));-static u32 sseu_get_geometry_subslices(const struct sseu_dev_info *sseu) -{
- return sseu_get_subslices(sseu, sseu->geometry_subslice_mask, 0);
-}
- for (i = 0; i < ARRAY_SIZE(sseu->subslice_mask.hsw); i++)
total += hweight8(sseu->subslice_mask.hsw[i]);-u32 intel_sseu_get_compute_subslices(const struct sseu_dev_info *sseu) -{
- return sseu_get_subslices(sseu, sseu->compute_subslice_mask, 0);
-}
-void intel_sseu_set_subslices(struct sseu_dev_info *sseu, int slice,
u8 *subslice_mask, u32 ss_mask)-{
- int offset = slice * sseu->ss_stride;
- memcpy(&subslice_mask[offset], &ss_mask, sseu->ss_stride);
- return total;
}
unsigned int -intel_sseu_subslices_per_slice(const struct sseu_dev_info *sseu, u8 slice) +intel_sseu_get_hsw_subslices(const struct sseu_dev_info *sseu, u8 slice) {
- return hweight32(intel_sseu_get_subslices(sseu, slice));
- WARN_ON(sseu->has_xehp_dss);
- if (WARN_ON(slice >= sseu->max_slices))
return 0;- return sseu->subslice_mask.hsw[slice];
}
static u16 sseu_get_eus(const struct sseu_dev_info *sseu, int slice, @@ -147,32 +116,66 @@ int intel_sseu_copy_eumask_to_user(void __user *to, return copy_to_user(to, eu_mask, len); }
+/**
- intel_sseu_copy_ssmask_to_user - Copy subslice mask into a userspace buffer
- @to: Pointer to userspace buffer to copy to
- @sseu: SSEU structure containing subslice mask to copy
- Copies the subslice mask to a userspace buffer in the format expected by
- the query ioctl's topology queries.
- Returns the result of the copy_to_user() operation.
- */
+int intel_sseu_copy_ssmask_to_user(void __user *to,
const struct sseu_dev_info *sseu)+{
- u8 ss_mask[GEN_SS_MASK_SIZE] = {};
- int ss_stride = GEN_SSEU_STRIDE(sseu->max_subslices);
- int len = sseu->max_slices * ss_stride;
- int s, ss, i;
- for (s = 0; s < sseu->max_slices; s++) {
for (ss = 0; ss < sseu->max_subslices; ss++) {i = s * ss_stride * BITS_PER_BYTE + ss;if (!intel_sseu_has_subslice(sseu, s, ss))continue;ss_mask[i / BITS_PER_BYTE] |= BIT(i % BITS_PER_BYTE);}- }
- return copy_to_user(to, ss_mask, len);
+}
static void gen11_compute_sseu_info(struct sseu_dev_info *sseu,
u32 g_ss_en, u32 c_ss_en, u16 eu_en)
u32 ss_en, u16 eu_en){ u32 valid_ss_mask = GENMASK(sseu->max_subslices - 1, 0); int ss;
- /* g_ss_en/c_ss_en represent entire subslice mask across all slices */
- GEM_BUG_ON(sseu->max_slices * sseu->max_subslices >
sizeof(g_ss_en) * BITS_PER_BYTE);
- sseu->slice_mask |= BIT(0);
- sseu->subslice_mask.hsw[0] = ss_en & valid_ss_mask;
- for (ss = 0; ss < sseu->max_subslices; ss++)
if (intel_sseu_has_subslice(sseu, 0, ss))sseu_set_eus(sseu, 0, ss, eu_en);- sseu->eu_per_subslice = hweight16(eu_en);
- sseu->eu_total = compute_eu_total(sseu);
+}
+static void xehp_compute_sseu_info(struct sseu_dev_info *sseu,
u16 eu_en)+{
int ss;
sseu->slice_mask |= BIT(0);
- /*
* XeHP introduces the concept of compute vs geometry DSS. To reduce* variation between GENs around subslice usage, store a mask for both* the geometry and compute enabled masks since userspace will need to* be able to query these masks independently. Also compute a total* enabled subslice count for the purposes of selecting subslices to* use in a particular GEM context.*/- intel_sseu_set_subslices(sseu, 0, sseu->compute_subslice_mask,
c_ss_en & valid_ss_mask);- intel_sseu_set_subslices(sseu, 0, sseu->geometry_subslice_mask,
g_ss_en & valid_ss_mask);- intel_sseu_set_subslices(sseu, 0, sseu->subslice_mask,
(g_ss_en | c_ss_en) & valid_ss_mask);
bitmap_or(sseu->subslice_mask.xehp,
sseu->compute_subslice_mask.xehp,sseu->geometry_subslice_mask.xehp,XEHP_BITMAP_BITS(sseu->subslice_mask));for (ss = 0; ss < sseu->max_subslices; ss++) if (intel_sseu_has_subslice(sseu, 0, ss))
@@ -182,11 +185,31 @@ static void gen11_compute_sseu_info(struct sseu_dev_info *sseu, sseu->eu_total = compute_eu_total(sseu); }
+static void +xehp_load_dss_mask(struct intel_uncore *uncore,
intel_sseu_ss_mask_t *ssmask,int numregs,...)+{
- va_list argp;
- u32 fuse_val[I915_MAX_SS_FUSE_REGS] = {};
- int i;
- if (WARN_ON(numregs > I915_MAX_SS_FUSE_REGS))
numregs = I915_MAX_SS_FUSE_REGS;- va_start(argp, numregs);
- for (i = 0; i < numregs; i++)
fuse_val[i] = intel_uncore_read(uncore, va_arg(argp, i915_reg_t));- va_end(argp);
- bitmap_from_arr32(ssmask->xehp, fuse_val, numregs * 32);
+}
static void xehp_sseu_info_init(struct intel_gt *gt) { struct sseu_dev_info *sseu = >->info.sseu; struct intel_uncore *uncore = gt->uncore;
- u32 g_dss_en, c_dss_en = 0; u16 eu_en = 0; u8 eu_en_fuse; int eu;
@@ -200,8 +223,10 @@ static void xehp_sseu_info_init(struct intel_gt *gt) intel_sseu_set_info(sseu, 1, 32, 16); sseu->has_xehp_dss = 1;
- g_dss_en = intel_uncore_read(uncore, GEN12_GT_GEOMETRY_DSS_ENABLE);
- c_dss_en = intel_uncore_read(uncore, GEN12_GT_COMPUTE_DSS_ENABLE);
xehp_load_dss_mask(uncore, &sseu->geometry_subslice_mask, 1,
GEN12_GT_GEOMETRY_DSS_ENABLE);xehp_load_dss_mask(uncore, &sseu->compute_subslice_mask, 1,
GEN12_GT_COMPUTE_DSS_ENABLE);eu_en_fuse = intel_uncore_read(uncore, XEHP_EU_ENABLE) & XEHP_EU_ENA_MASK;
@@ -209,7 +234,7 @@ static void xehp_sseu_info_init(struct intel_gt *gt) if (eu_en_fuse & BIT(eu)) eu_en |= BIT(eu * 2) | BIT(eu * 2 + 1);
- gen11_compute_sseu_info(sseu, g_dss_en, c_dss_en, eu_en);
- xehp_compute_sseu_info(sseu, eu_en);
}
static void gen12_sseu_info_init(struct intel_gt *gt) @@ -247,7 +272,7 @@ static void gen12_sseu_info_init(struct intel_gt *gt) if (eu_en_fuse & BIT(eu)) eu_en |= BIT(eu * 2) | BIT(eu * 2 + 1);
- gen11_compute_sseu_info(sseu, g_dss_en, 0, eu_en);
gen11_compute_sseu_info(sseu, g_dss_en, eu_en);
/* TGL only supports slice-level power gating */ sseu->has_slice_pg = 1;
@@ -279,7 +304,7 @@ static void gen11_sseu_info_init(struct intel_gt *gt) eu_en = ~(intel_uncore_read(uncore, GEN11_EU_DISABLE) & GEN11_EU_DIS_MASK);
- gen11_compute_sseu_info(sseu, ss_en, 0, eu_en);
gen11_compute_sseu_info(sseu, ss_en, eu_en);
/* ICL has no power gating restrictions. */ sseu->has_slice_pg = 1;
@@ -291,7 +316,6 @@ static void cherryview_sseu_info_init(struct intel_gt *gt) { struct sseu_dev_info *sseu = >->info.sseu; u32 fuse;
u8 subslice_mask = 0;
fuse = intel_uncore_read(gt->uncore, CHV_FUSE_GT);
@@ -305,7 +329,7 @@ static void cherryview_sseu_info_init(struct intel_gt *gt) (((fuse & CHV_FGT_EU_DIS_SS0_R1_MASK) >> CHV_FGT_EU_DIS_SS0_R1_SHIFT) << 4);
subslice_mask |= BIT(0);
sseu->subslice_mask.hsw[0] |= BIT(0);@@ -316,12 +340,10 @@ static void cherryview_sseu_info_init(struct intel_gt *gt) (((fuse & CHV_FGT_EU_DIS_SS1_R1_MASK) >> CHV_FGT_EU_DIS_SS1_R1_SHIFT) << 4);
subslice_mask |= BIT(1);
sseu->subslice_mask.hsw[0] |= BIT(1);
intel_sseu_set_subslices(sseu, 0, sseu->subslice_mask, subslice_mask);
sseu->eu_total = compute_eu_total(sseu);
/*
@@ -376,8 +398,7 @@ static void gen9_sseu_info_init(struct intel_gt *gt) /* skip disabled slice */ continue;
intel_sseu_set_subslices(sseu, s, sseu->subslice_mask,subslice_mask);
sseu->subslice_mask.hsw[s] = subslice_mask;eu_disable = intel_uncore_read(uncore, GEN9_EU_DISABLE(s)); for (ss = 0; ss < sseu->max_subslices; ss++) {
@@ -434,8 +455,8 @@ static void gen9_sseu_info_init(struct intel_gt *gt) sseu->has_eu_pg = sseu->eu_per_subslice > 2;
if (IS_GEN9_LP(i915)) { -#define IS_SS_DISABLED(ss) (!(sseu->subslice_mask[0] & BIT(ss)))
info->has_pooled_eu = hweight8(sseu->subslice_mask[0]) == 3;+#define IS_SS_DISABLED(ss) (!(sseu->subslice_mask.hsw[0] & BIT(ss)))
info->has_pooled_eu = hweight8(sseu->subslice_mask.hsw[0]) == 3;sseu->min_eu_in_pool = 0; if (info->has_pooled_eu) {
@@ -489,8 +510,7 @@ static void bdw_sseu_info_init(struct intel_gt *gt) /* skip disabled slice */ continue;
intel_sseu_set_subslices(sseu, s, sseu->subslice_mask,subslice_mask);
sseu->subslice_mask.hsw[s] = subslice_mask;for (ss = 0; ss < sseu->max_subslices; ss++) { u8 eu_disabled_mask;
@@ -587,8 +607,7 @@ static void hsw_sseu_info_init(struct intel_gt *gt) sseu->eu_per_subslice);
for (s = 0; s < sseu->max_slices; s++) {
intel_sseu_set_subslices(sseu, s, sseu->subslice_mask,subslice_mask);
sseu->subslice_mask.hsw[s] = subslice_mask;for (ss = 0; ss < sseu->max_subslices; ss++) { sseu_set_eus(sseu, s, ss,
@@ -677,7 +696,7 @@ u32 intel_sseu_make_rpcs(struct intel_gt *gt, */ if (GRAPHICS_VER(i915) == 11 && slices == 1 &&
subslices > min_t(u8, 4, hweight8(sseu->subslice_mask[0]) / 2)) {
subslices > min_t(u8, 4, hweight8(sseu->subslice_mask.hsw[0]) / 2)) {GEM_BUG_ON(subslices & 1);
subslice_pg = false;
@@ -743,14 +762,29 @@ void intel_sseu_dump(const struct sseu_dev_info *sseu, struct drm_printer *p) { int s;
- drm_printf(p, "slice total: %u, mask=%04x\n",
hweight8(sseu->slice_mask), sseu->slice_mask);- drm_printf(p, "subslice total: %u\n", intel_sseu_subslice_total(sseu));
- for (s = 0; s < sseu->max_slices; s++) {
drm_printf(p, "slice%d: %u subslices, mask=%08x\n",s, intel_sseu_subslices_per_slice(sseu, s),intel_sseu_get_subslices(sseu, s));
- if (sseu->has_xehp_dss) {
drm_printf(p, "subslice total: %u\n",intel_sseu_subslice_total(sseu));drm_printf(p, "geometry dss mask=%*pb\n",XEHP_BITMAP_BITS(sseu->geometry_subslice_mask),sseu->geometry_subslice_mask.xehp);drm_printf(p, "compute dss mask=%*pb\n",XEHP_BITMAP_BITS(sseu->compute_subslice_mask),sseu->compute_subslice_mask.xehp);- } else {
drm_printf(p, "slice total: %u, mask=%04x\n",hweight8(sseu->slice_mask), sseu->slice_mask);drm_printf(p, "subslice total: %u\n",intel_sseu_subslice_total(sseu));for (s = 0; s < sseu->max_slices; s++) {u8 ss_mask = sseu->subslice_mask.hsw[s];drm_printf(p, "slice%d: %u subslices, mask=%08x\n",s, hweight8(ss_mask), ss_mask);}- drm_printf(p, "EU total: %u\n", sseu->eu_total); drm_printf(p, "EU per subslice: %u\n", sseu->eu_per_subslice); drm_printf(p, "has slice power gating: %s\n",
@@ -767,9 +801,10 @@ static void sseu_print_hsw_topology(const struct sseu_dev_info *sseu, int s, ss;
for (s = 0; s < sseu->max_slices; s++) {
u8 ss_mask = sseu->subslice_mask.hsw[s];- drm_printf(p, "slice%d: %u subslice(s) (0x%08x):\n",
s, intel_sseu_subslices_per_slice(sseu, s),intel_sseu_get_subslices(sseu, s));
s, hweight8(ss_mask), ss_mask);for (ss = 0; ss < sseu->max_subslices; ss++) { u16 enabled_eus = sseu_get_eus(sseu, s, ss);
@@ -783,16 +818,14 @@ static void sseu_print_hsw_topology(const struct sseu_dev_info *sseu, static void sseu_print_xehp_topology(const struct sseu_dev_info *sseu, struct drm_printer *p) {
u32 g_dss_mask = sseu_get_geometry_subslices(sseu);
u32 c_dss_mask = intel_sseu_get_compute_subslices(sseu); int dss;
for (dss = 0; dss < sseu->max_subslices; dss++) { u16 enabled_eus = sseu_get_eus(sseu, 0, dss);
drm_printf(p, "DSS_%02d: G:%3s C:%3s, %2u EUs (0x%04hx)\n", dss,
str_yes_no(g_dss_mask & BIT(dss)),str_yes_no(c_dss_mask & BIT(dss)),
str_yes_no(test_bit(dss, sseu->geometry_subslice_mask.xehp)),str_yes_no(test_bit(dss, sseu->compute_subslice_mask.xehp)), hweight16(enabled_eus), enabled_eus);} @@ -810,20 +843,44 @@ void intel_sseu_print_topology(struct drm_i915_private *i915, } }
-u16 intel_slicemask_from_dssmask(u64 dss_mask, int dss_per_slice) +void intel_sseu_print_ss_info(const char* type,
const struct sseu_dev_info *sseu,struct seq_file *m)+{
- int s;
- if (sseu->has_xehp_dss) {
seq_printf(m, " %s Geometry DSS: %u\n", type,bitmap_weight(sseu->geometry_subslice_mask.xehp,XEHP_BITMAP_BITS(sseu->geometry_subslice_mask)));seq_printf(m, " %s Compute DSS: %u\n", type,bitmap_weight(sseu->compute_subslice_mask.xehp,XEHP_BITMAP_BITS(sseu->compute_subslice_mask)));- } else {
for (s = 0; s < fls(sseu->slice_mask); s++)seq_printf(m, " %s Slice%i subslices: %u\n", type,s, hweight8(sseu->subslice_mask.hsw[s]));- }
+}
+u16 intel_slicemask_from_xehp_dssmask(intel_sseu_ss_mask_t dss_mask,
int dss_per_slice){
- intel_sseu_ss_mask_t per_slice_mask = {}; u16 slice_mask = 0; int i;
- WARN_ON(sizeof(dss_mask) * 8 / dss_per_slice > 8 * sizeof(slice_mask));
- WARN_ON(DIV_ROUND_UP(XEHP_BITMAP_BITS(dss_mask), dss_per_slice) >
8 * sizeof(slice_mask));
- for (i = 0; dss_mask; i++) {
if (dss_mask & GENMASK(dss_per_slice - 1, 0))
- bitmap_fill(per_slice_mask.xehp, dss_per_slice);
- for (i = 0; !bitmap_empty(dss_mask.xehp, XEHP_BITMAP_BITS(dss_mask)); i++) {
if (bitmap_intersects(dss_mask.xehp, per_slice_mask.xehp, dss_per_slice)) slice_mask |= BIT(i);
dss_mask >>= dss_per_slice;
bitmap_shift_right(dss_mask.xehp, dss_mask.xehp, dss_per_slice,XEHP_BITMAP_BITS(dss_mask));}
return slice_mask;
}
diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.h b/drivers/gpu/drm/i915/gt/intel_sseu.h index ffa375e68959..e7f336a7ffbb 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu.h +++ b/drivers/gpu/drm/i915/gt/intel_sseu.h @@ -25,12 +25,16 @@ struct drm_printer; /*
- Maximum number of subslices that can exist within a HSW-style slice. This
- is only relevant to pre-Xe_HP platforms (Xe_HP and beyond use the
- GEN_MAX_DSS value below).
*/
- I915_MAX_SS_FUSE_BITS value below).
#define GEN_MAX_SS_PER_HSW_SLICE 6
-/* Maximum number of DSS on newer platforms (Xe_HP and beyond). */ -#define GEN_MAX_DSS 32 +/*
- Maximum number of 32-bit registers used by hardware to express the
- enabled/disabled subslices.
- */
+#define I915_MAX_SS_FUSE_REGS 1 +#define I915_MAX_SS_FUSE_BITS (I915_MAX_SS_FUSE_REGS * 32)
/* Maximum number of EUs that can exist within a subslice or DSS. */ #define GEN_MAX_EUS_PER_SS 16 @@ -38,7 +42,7 @@ struct drm_printer; #define SSEU_MAX(a, b) ((a) > (b) ? (a) : (b))
/* The maximum number of bits needed to express each subslice/DSS independently */ -#define GEN_SS_MASK_SIZE SSEU_MAX(GEN_MAX_DSS, \ +#define GEN_SS_MASK_SIZE SSEU_MAX(I915_MAX_SS_FUSE_BITS, \ GEN_MAX_HSW_SLICES * GEN_MAX_SS_PER_HSW_SLICE)
#define GEN_SSEU_STRIDE(max_entries) DIV_ROUND_UP(max_entries, BITS_PER_BYTE) @@ -49,17 +53,24 @@ struct drm_printer; #define GEN_DSS_PER_CSLICE 8 #define GEN_DSS_PER_MSLICE 8
-#define GEN_MAX_GSLICES (GEN_MAX_DSS / GEN_DSS_PER_GSLICE) -#define GEN_MAX_CSLICES (GEN_MAX_DSS / GEN_DSS_PER_CSLICE) +#define GEN_MAX_GSLICES (I915_MAX_SS_FUSE_BITS / GEN_DSS_PER_GSLICE) +#define GEN_MAX_CSLICES (I915_MAX_SS_FUSE_BITS / GEN_DSS_PER_CSLICE)
+typedef union {
- u8 hsw[GEN_MAX_HSW_SLICES];
- /* Bitmap compatible with linux/bitmap.h; may exceed size of u64 */
- unsigned long xehp[BITS_TO_LONGS(I915_MAX_SS_FUSE_BITS)];
+} intel_sseu_ss_mask_t;
struct sseu_dev_info { u8 slice_mask;
- u8 subslice_mask[GEN_SS_MASK_SIZE];
- u8 geometry_subslice_mask[GEN_SS_MASK_SIZE];
- u8 compute_subslice_mask[GEN_SS_MASK_SIZE];
- intel_sseu_ss_mask_t subslice_mask;
- intel_sseu_ss_mask_t geometry_subslice_mask;
- intel_sseu_ss_mask_t compute_subslice_mask; union { u16 hsw[GEN_MAX_HSW_SLICES][GEN_MAX_SS_PER_HSW_SLICE];
u16 xehp[GEN_MAX_DSS];
u16 xehp[I915_MAX_SS_FUSE_BITS];} eu_mask;
u16 eu_total;
@@ -80,8 +91,6 @@ struct sseu_dev_info { u8 max_slices; u8 max_subslices; u8 max_eus_per_subslice;
- u8 ss_stride;
};
/* @@ -99,7 +108,7 @@ intel_sseu_from_device_info(const struct sseu_dev_info *sseu) { struct intel_sseu value = { .slice_mask = sseu->slice_mask,
.subslice_mask = sseu->subslice_mask[0],
.subslice_mask = sseu->subslice_mask.hsw[0],@@ -111,18 +120,27 @@ static inline bool intel_sseu_has_subslice(const struct sseu_dev_info *sseu, int slice, int subslice) {
u8 mask;
int ss_idx = subslice / BITS_PER_BYTE;
if (slice >= sseu->max_slices || subslice >= sseu->max_subslices) return false;
GEM_BUG_ON(ss_idx >= sseu->ss_stride);
mask = sseu->subslice_mask[slice * sseu->ss_stride + ss_idx];
- if (sseu->has_xehp_dss)
return test_bit(subslice, sseu->subslice_mask.xehp);- else
return sseu->subslice_mask.hsw[slice] & BIT(subslice);+}
- return mask & BIT(subslice % BITS_PER_BYTE);
+/*
- Used to obtain the index of the first DSS. Can start searching from the
- beginning of a specific dss group (e.g., gslice, cslice, etc.) if
- groupsize and groupnum are non-zero.
- */
+static inline unsigned int +intel_sseu_find_first_xehp_dss(const struct sseu_dev_info *sseu, int groupsize,
int groupnum)+{
- return find_next_bit(sseu->subslice_mask.xehp, I915_MAX_SS_FUSE_BITS,
groupnum * groupsize);
Any specific reason for not replacing I915_MAX_SS_FUSE_BITS with XEHP_BITMAP_BITS here?
Regards, Bala
}
void intel_sseu_set_info(struct sseu_dev_info *sseu, u8 max_slices, @@ -132,14 +150,10 @@ unsigned int intel_sseu_subslice_total(const struct sseu_dev_info *sseu);
unsigned int -intel_sseu_subslices_per_slice(const struct sseu_dev_info *sseu, u8 slice);
-u32 intel_sseu_get_subslices(const struct sseu_dev_info *sseu, u8 slice); +intel_sseu_get_hsw_subslices(const struct sseu_dev_info *sseu, u8 slice);
-u32 intel_sseu_get_compute_subslices(const struct sseu_dev_info *sseu);
-void intel_sseu_set_subslices(struct sseu_dev_info *sseu, int slice,
u8 *subslice_mask, u32 ss_mask);+intel_sseu_ss_mask_t +intel_sseu_get_compute_subslices(const struct sseu_dev_info *sseu);
void intel_sseu_info_init(struct intel_gt *gt);
@@ -151,9 +165,15 @@ void intel_sseu_print_topology(struct drm_i915_private *i915, const struct sseu_dev_info *sseu, struct drm_printer *p);
-u16 intel_slicemask_from_dssmask(u64 dss_mask, int dss_per_slice); +u16 intel_slicemask_from_xehp_dssmask(intel_sseu_ss_mask_t dss_mask, int dss_per_slice);
int intel_sseu_copy_eumask_to_user(void __user *to, const struct sseu_dev_info *sseu); +int intel_sseu_copy_ssmask_to_user(void __user *to,
const struct sseu_dev_info *sseu);+void intel_sseu_print_ss_info(const char* type,
const struct sseu_dev_info *sseu,struct seq_file *m);#endif /* __INTEL_SSEU_H__ */ diff --git a/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c b/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c index 2d5d011e01db..c2ee5e1826b5 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c +++ b/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c @@ -4,6 +4,7 @@
- Copyright © 2020 Intel Corporation
*/
+#include <linux/bitmap.h> #include <linux/string_helpers.h>
#include "i915_drv.h" @@ -11,14 +12,6 @@ #include "intel_gt_regs.h" #include "intel_sseu_debugfs.h"
-static void sseu_copy_subslices(const struct sseu_dev_info *sseu,
int slice, u8 *to_mask)-{
- int offset = slice * sseu->ss_stride;
- memcpy(&to_mask[offset], &sseu->subslice_mask[offset], sseu->ss_stride);
-}
static void cherryview_sseu_device_status(struct intel_gt *gt, struct sseu_dev_info *sseu) { @@ -41,7 +34,7 @@ static void cherryview_sseu_device_status(struct intel_gt *gt, continue;
sseu->slice_mask = BIT(0);
sseu->subslice_mask[0] |= BIT(ss);
sseu->subslice_mask.hsw[0] |= BIT(ss);@@ -92,7 +85,7 @@ static void gen11_sseu_device_status(struct intel_gt *gt, continue;
sseu->slice_mask |= BIT(s);
sseu_copy_subslices(&info->sseu, s, sseu->subslice_mask);
sseu->subslice_mask.hsw[s] = info->sseu.subslice_mask.hsw[s];for (ss = 0; ss < info->sseu.max_subslices; ss++) { unsigned int eu_cnt;
@@ -147,21 +140,17 @@ static void gen9_sseu_device_status(struct intel_gt *gt, sseu->slice_mask |= BIT(s);
if (IS_GEN9_BC(gt->i915))
sseu_copy_subslices(&info->sseu, s,sseu->subslice_mask);
sseu->subslice_mask.hsw[s] = info->sseu.subslice_mask.hsw[s];for (ss = 0; ss < info->sseu.max_subslices; ss++) { unsigned int eu_cnt;
u8 ss_idx = s * info->sseu.ss_stride +ss / BITS_PER_BYTE; if (IS_GEN9_LP(gt->i915)) { if (!(s_reg[s] & (GEN9_PGCTL_SS_ACK(ss)))) /* skip disabled subslice */ continue;sseu->subslice_mask[ss_idx] |=BIT(ss % BITS_PER_BYTE);
sseu->subslice_mask.hsw[s] |= BIT(ss); } eu_cnt = eu_reg[2 * s + ss / 2] & eu_mask[ss % 2];@@ -188,8 +177,7 @@ static void bdw_sseu_device_status(struct intel_gt *gt, if (sseu->slice_mask) { sseu->eu_per_subslice = info->sseu.eu_per_subslice; for (s = 0; s < fls(sseu->slice_mask); s++)
sseu_copy_subslices(&info->sseu, s,sseu->subslice_mask);
sseu->subslice_mask.hsw[s] = info->sseu.subslice_mask.hsw[s];@@ -208,7 +196,6 @@ static void i915_print_sseu_info(struct seq_file *m, const struct sseu_dev_info *sseu) { const char *type = is_available_info ? "Available" : "Enabled";
int s;
seq_printf(m, " %s Slice Mask: %04x\n", type, sseu->slice_mask);
@@ -216,10 +203,7 @@ static void i915_print_sseu_info(struct seq_file *m, hweight8(sseu->slice_mask)); seq_printf(m, " %s Subslice Total: %u\n", type, intel_sseu_subslice_total(sseu));
- for (s = 0; s < fls(sseu->slice_mask); s++) {
seq_printf(m, " %s Slice%i subslices: %u\n", type,s, intel_sseu_subslices_per_slice(sseu, s));- }
- intel_sseu_print_ss_info(type, sseu, m); seq_printf(m, " %s EU Total: %u\n", type, sseu->eu_total); seq_printf(m, " %s EU Per Subslice: %u\n", type,
diff --git a/drivers/gpu/drm/i915/gt/intel_workarounds.c b/drivers/gpu/drm/i915/gt/intel_workarounds.c index 73b59ea6fd3b..1bf62273e02d 100644 --- a/drivers/gpu/drm/i915/gt/intel_workarounds.c +++ b/drivers/gpu/drm/i915/gt/intel_workarounds.c @@ -948,8 +948,8 @@ gen9_wa_init_mcr(struct drm_i915_private *i915, struct i915_wa_list *wal) * on s/ss combo, the read should be done with read_subslice_reg. */ slice = ffs(sseu->slice_mask) - 1;
- GEM_BUG_ON(slice >= ARRAY_SIZE(sseu->subslice_mask));
- subslice = ffs(intel_sseu_get_subslices(sseu, slice));
- GEM_BUG_ON(slice >= ARRAY_SIZE(sseu->subslice_mask.hsw));
- subslice = ffs(intel_sseu_get_hsw_subslices(sseu, slice)); GEM_BUG_ON(!subslice); subslice--;
@@ -1087,11 +1087,10 @@ static void icl_wa_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) { const struct sseu_dev_info *sseu = >->info.sseu;
- unsigned int slice, subslice;
unsigned int subslice;
GEM_BUG_ON(GRAPHICS_VER(gt->i915) < 11); GEM_BUG_ON(hweight8(sseu->slice_mask) > 1);
slice = 0;
/*
- Although a platform may have subslices, we need to always steer
@@ -1102,7 +1101,7 @@ icl_wa_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) * one of the higher subslices, we run the risk of reading back 0's or * random garbage. */
- subslice = __ffs(intel_sseu_get_subslices(sseu, slice));
subslice = __ffs(intel_sseu_get_hsw_subslices(sseu, 0));
/*
- If the subslice we picked above also steers us to a valid L3 bank,
@@ -1112,7 +1111,7 @@ icl_wa_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) if (gt->info.l3bank_mask & BIT(subslice)) gt->steering_table[L3BANK] = NULL;
- __add_mcr_wa(gt, wal, slice, subslice);
- __add_mcr_wa(gt, wal, 0, subslice);
}
static void @@ -1120,7 +1119,6 @@ xehp_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) { const struct sseu_dev_info *sseu = >->info.sseu; unsigned long slice, subslice = 0, slice_mask = 0;
- u64 dss_mask = 0; u32 lncf_mask = 0; int i;
@@ -1151,8 +1149,8 @@ xehp_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) */
/* Find the potential gslice candidates */
- dss_mask = intel_sseu_get_subslices(sseu, 0);
- slice_mask = intel_slicemask_from_dssmask(dss_mask, GEN_DSS_PER_GSLICE);
slice_mask = intel_slicemask_from_xehp_dssmask(sseu->subslice_mask,
GEN_DSS_PER_GSLICE);/*
- Find the potential LNCF candidates. Either LNCF within a valid
@@ -1177,9 +1175,8 @@ xehp_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) }
slice = __ffs(slice_mask);
- subslice = __ffs(dss_mask >> (slice * GEN_DSS_PER_GSLICE));
- subslice = intel_sseu_find_first_xehp_dss(sseu, GEN_DSS_PER_GSLICE, slice); WARN_ON(subslice > GEN_DSS_PER_GSLICE);
WARN_ON(dss_mask >> (slice * GEN_DSS_PER_GSLICE) == 0);
__add_mcr_wa(gt, wal, slice, subslice);
@@ -2030,9 +2027,8 @@ engine_fake_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
static bool needs_wa_1308578152(struct intel_engine_cs *engine) {
- u64 dss_mask = intel_sseu_get_subslices(&engine->gt->info.sseu, 0);
- return (dss_mask & GENMASK(GEN_DSS_PER_GSLICE - 1, 0)) == 0;
- return intel_sseu_find_first_xehp_dss(&engine->gt->info.sseu, 0, 0) >
GEN_DSS_PER_GSLICE;}
static void diff --git a/drivers/gpu/drm/i915/i915_getparam.c b/drivers/gpu/drm/i915/i915_getparam.c index ac9767c56619..6fd15b39570c 100644 --- a/drivers/gpu/drm/i915/i915_getparam.c +++ b/drivers/gpu/drm/i915/i915_getparam.c @@ -162,8 +162,7 @@ int i915_getparam_ioctl(struct drm_device *dev, void *data, return -EINVAL;
/* Only copy bits from the first slice */
memcpy(&value, sseu->subslice_mask,min(sseu->ss_stride, (u8)sizeof(value)));
value = intel_sseu_get_hsw_subslices(sseu, 0);diff --git a/drivers/gpu/drm/i915/i915_query.c b/drivers/gpu/drm/i915/i915_query.c index 89c475d525b8..0094f67c63f2 100644 --- a/drivers/gpu/drm/i915/i915_query.c +++ b/drivers/gpu/drm/i915/i915_query.c @@ -31,10 +31,11 @@ static int copy_query_item(void *query_hdr, size_t query_sz,
static int fill_topology_info(const struct sseu_dev_info *sseu, struct drm_i915_query_item *query_item,
const u8 *subslice_mask)
intel_sseu_ss_mask_t subslice_mask){ struct drm_i915_query_topology_info topo; u32 slice_length, subslice_length, eu_length, total_length;
- int ss_stride = GEN_SSEU_STRIDE(sseu->max_subslices); int eu_stride = GEN_SSEU_STRIDE(sseu->max_eus_per_subslice); int ret;
@@ -44,7 +45,7 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, return -ENODEV;
slice_length = sizeof(sseu->slice_mask);
- subslice_length = sseu->max_slices * sseu->ss_stride;
- subslice_length = sseu->max_slices * ss_stride; eu_length = sseu->max_slices * sseu->max_subslices * eu_stride; total_length = sizeof(topo) + slice_length + subslice_length + eu_length;
@@ -60,7 +61,7 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, topo.max_eus_per_subslice = sseu->max_eus_per_subslice;
topo.subslice_offset = slice_length;
- topo.subslice_stride = sseu->ss_stride;
- topo.subslice_stride = ss_stride; topo.eu_offset = slice_length + subslice_length; topo.eu_stride = eu_stride;
@@ -72,9 +73,9 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, &sseu->slice_mask, slice_length)) return -EFAULT;
- if (copy_to_user(u64_to_user_ptr(query_item->data_ptr +
sizeof(topo) + slice_length),subslice_mask, subslice_length))
if (intel_sseu_copy_ssmask_to_user(u64_to_user_ptr(query_item->data_ptr +
sizeof(topo) + slice_length),sseu))return -EFAULT;
if (intel_sseu_copy_eumask_to_user(u64_to_user_ptr(query_item->data_ptr +
-- 2.35.3
On Wed, Jun 01, 2022 at 01:48:56PM +0530, Balasubramani Vivekanandan wrote:
On 23.05.2022 13:45, Matt Roper wrote:
As with EU masks, it's easier to store subslice/DSS masks internally in a format that's more natural for the driver to work with, and then only covert into the u8[] uapi form when the query ioctl is invoked. Since the hardware design changed significantly with Xe_HP, we'll use a union to choose between the old "hsw-style" subslice masks or the newer xehp mask. HSW-style masks will be stored in an array of u8's, indexed by slice (there's never more than 6 subslices per slice on older platforms). For Xe_HP and beyond where slices no longer exist, we only need a single bitmask. However we already know that this mask is eventually going to grow too large for a simple u64 to hold, so we'll represent it in a manner that can be operated on by the utilities in linux/bitmap.h.
v2:
- Fix typo: BIT(s) -> BIT(ss) in gen9_sseu_device_status()
v3:
- Eliminate sseu->ss_stride and just calculate the stride while specifically handling uapi. (Tvrtko)
- Use BITMAP_BITS() macro to refer to size of masks rather than passing I915_MAX_SS_FUSE_BITS directly. (Tvrtko)
- Report compute/geometry DSS masks separately when dumping Xe_HP SSEU info. (Tvrtko)
- Restore dropped range checks to intel_sseu_has_subslice(). (Tvrtko)
v4:
- Make the bitmap size macro check the size of the .xehp field rather than the containing union. (Tvrtko)
- Don't add GEM_BUG_ON() intel_sseu_has_subslice()'s check for whether slice or subslice ID exceed sseu->max_[sub]slices; various loops in the driver are expected to exceed these, so we should just silently return 'false.'
Cc: Tvrtko Ursulin tvrtko.ursulin@linux.intel.com Signed-off-by: Matt Roper matthew.d.roper@intel.com
drivers/gpu/drm/i915/gem/i915_gem_context.c | 5 +- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 4 +- drivers/gpu/drm/i915/gt/intel_gt.c | 12 +- drivers/gpu/drm/i915/gt/intel_sseu.c | 261 +++++++++++-------- drivers/gpu/drm/i915/gt/intel_sseu.h | 76 ++++-- drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c | 30 +-- drivers/gpu/drm/i915/gt/intel_workarounds.c | 24 +- drivers/gpu/drm/i915/i915_getparam.c | 3 +- drivers/gpu/drm/i915/i915_query.c | 13 +- 9 files changed, 241 insertions(+), 187 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index ab4c5ab28e4d..a3bb73f5d53b 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -1875,6 +1875,7 @@ i915_gem_user_to_context_sseu(struct intel_gt *gt, { const struct sseu_dev_info *device = >->info.sseu; struct drm_i915_private *i915 = gt->i915;
unsigned int dev_subslice_mask = intel_sseu_get_hsw_subslices(device, 0);
/* No zeros in any field. */ if (!user->slice_mask || !user->subslice_mask ||
@@ -1901,7 +1902,7 @@ i915_gem_user_to_context_sseu(struct intel_gt *gt, if (user->slice_mask & ~device->slice_mask) return -EINVAL;
- if (user->subslice_mask & ~device->subslice_mask[0])
if (user->subslice_mask & ~dev_subslice_mask) return -EINVAL;
if (user->max_eus_per_subslice > device->max_eus_per_subslice)
@@ -1915,7 +1916,7 @@ i915_gem_user_to_context_sseu(struct intel_gt *gt, /* Part specific restrictions. */ if (GRAPHICS_VER(i915) == 11) { unsigned int hw_s = hweight8(device->slice_mask);
unsigned int hw_ss_per_s = hweight8(device->subslice_mask[0]);
unsigned int hw_ss_per_s = hweight8(dev_subslice_mask);diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c index 1adbf34c3632..f0acf8518a51 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c @@ -674,8 +674,8 @@ static void engine_mask_apply_compute_fuses(struct intel_gt *gt) if (GRAPHICS_VER_FULL(i915) < IP_VER(12, 50)) return;
- ccs_mask = intel_slicemask_from_dssmask(intel_sseu_get_compute_subslices(&info->sseu),
ss_per_ccs);
- ccs_mask = intel_slicemask_from_xehp_dssmask(info->sseu.compute_subslice_mask,
ss_per_ccs);
- If all DSS in a quadrant are fused off, the corresponding CCS
- engine is not available for use.
diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c index 034182f85501..2921f510642f 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -133,13 +133,6 @@ static const struct intel_mmio_range dg2_lncf_steering_table[] = { {}, };
-static u16 slicemask(struct intel_gt *gt, int count) -{
- u64 dss_mask = intel_sseu_get_subslices(>->info.sseu, 0);
- return intel_slicemask_from_dssmask(dss_mask, count);
-}
int intel_gt_init_mmio(struct intel_gt *gt) { struct drm_i915_private *i915 = gt->i915; @@ -155,9 +148,12 @@ int intel_gt_init_mmio(struct intel_gt *gt) */ if (HAS_MSLICES(i915)) { gt->info.mslice_mask =
slicemask(gt, GEN_DSS_PER_MSLICE) |
intel_slicemask_from_xehp_dssmask(gt->info.sseu.subslice_mask,GEN_DSS_PER_MSLICE);gt->info.mslice_mask |= (intel_uncore_read(gt->uncore, GEN10_MIRROR_FUSE3) & GEN12_MEML3_EN_MASK);- if (!gt->info.mslice_mask) /* should be impossible! */ drm_warn(&i915->drm, "mslice mask all zero!\n"); }
diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.c b/drivers/gpu/drm/i915/gt/intel_sseu.c index 285cfd758bdc..3a721db20290 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu.c +++ b/drivers/gpu/drm/i915/gt/intel_sseu.c @@ -10,15 +10,14 @@ #include "intel_gt_regs.h" #include "intel_sseu.h"
+#define XEHP_BITMAP_BITS(mask) ((int)BITS_PER_TYPE(typeof(mask.xehp)))
void intel_sseu_set_info(struct sseu_dev_info *sseu, u8 max_slices, u8 max_subslices, u8 max_eus_per_subslice) { sseu->max_slices = max_slices; sseu->max_subslices = max_subslices; sseu->max_eus_per_subslice = max_eus_per_subslice;
- sseu->ss_stride = GEN_SSEU_STRIDE(sseu->max_subslices);
- GEM_BUG_ON(sseu->ss_stride > GEN_MAX_SUBSLICE_STRIDE);
}
unsigned int @@ -26,54 +25,24 @@ intel_sseu_subslice_total(const struct sseu_dev_info *sseu) { unsigned int i, total = 0;
- for (i = 0; i < ARRAY_SIZE(sseu->subslice_mask); i++)
total += hweight8(sseu->subslice_mask[i]);- return total;
-}
-static u32 -sseu_get_subslices(const struct sseu_dev_info *sseu,
const u8 *subslice_mask, u8 slice)-{
- int i, offset = slice * sseu->ss_stride;
- u32 mask = 0;
- GEM_BUG_ON(slice >= sseu->max_slices);
- for (i = 0; i < sseu->ss_stride; i++)
mask |= (u32)subslice_mask[offset + i] << i * BITS_PER_BYTE;- return mask;
-}
-u32 intel_sseu_get_subslices(const struct sseu_dev_info *sseu, u8 slice) -{
- return sseu_get_subslices(sseu, sseu->subslice_mask, slice);
-}
- if (sseu->has_xehp_dss)
return bitmap_weight(sseu->subslice_mask.xehp,XEHP_BITMAP_BITS(sseu->subslice_mask));-static u32 sseu_get_geometry_subslices(const struct sseu_dev_info *sseu) -{
- return sseu_get_subslices(sseu, sseu->geometry_subslice_mask, 0);
-}
- for (i = 0; i < ARRAY_SIZE(sseu->subslice_mask.hsw); i++)
total += hweight8(sseu->subslice_mask.hsw[i]);-u32 intel_sseu_get_compute_subslices(const struct sseu_dev_info *sseu) -{
- return sseu_get_subslices(sseu, sseu->compute_subslice_mask, 0);
-}
-void intel_sseu_set_subslices(struct sseu_dev_info *sseu, int slice,
u8 *subslice_mask, u32 ss_mask)-{
- int offset = slice * sseu->ss_stride;
- memcpy(&subslice_mask[offset], &ss_mask, sseu->ss_stride);
- return total;
}
unsigned int -intel_sseu_subslices_per_slice(const struct sseu_dev_info *sseu, u8 slice) +intel_sseu_get_hsw_subslices(const struct sseu_dev_info *sseu, u8 slice) {
- return hweight32(intel_sseu_get_subslices(sseu, slice));
- WARN_ON(sseu->has_xehp_dss);
- if (WARN_ON(slice >= sseu->max_slices))
return 0;- return sseu->subslice_mask.hsw[slice];
}
static u16 sseu_get_eus(const struct sseu_dev_info *sseu, int slice, @@ -147,32 +116,66 @@ int intel_sseu_copy_eumask_to_user(void __user *to, return copy_to_user(to, eu_mask, len); }
+/**
- intel_sseu_copy_ssmask_to_user - Copy subslice mask into a userspace buffer
- @to: Pointer to userspace buffer to copy to
- @sseu: SSEU structure containing subslice mask to copy
- Copies the subslice mask to a userspace buffer in the format expected by
- the query ioctl's topology queries.
- Returns the result of the copy_to_user() operation.
- */
+int intel_sseu_copy_ssmask_to_user(void __user *to,
const struct sseu_dev_info *sseu)+{
- u8 ss_mask[GEN_SS_MASK_SIZE] = {};
- int ss_stride = GEN_SSEU_STRIDE(sseu->max_subslices);
- int len = sseu->max_slices * ss_stride;
- int s, ss, i;
- for (s = 0; s < sseu->max_slices; s++) {
for (ss = 0; ss < sseu->max_subslices; ss++) {i = s * ss_stride * BITS_PER_BYTE + ss;if (!intel_sseu_has_subslice(sseu, s, ss))continue;ss_mask[i / BITS_PER_BYTE] |= BIT(i % BITS_PER_BYTE);}- }
- return copy_to_user(to, ss_mask, len);
+}
static void gen11_compute_sseu_info(struct sseu_dev_info *sseu,
u32 g_ss_en, u32 c_ss_en, u16 eu_en)
u32 ss_en, u16 eu_en){ u32 valid_ss_mask = GENMASK(sseu->max_subslices - 1, 0); int ss;
- /* g_ss_en/c_ss_en represent entire subslice mask across all slices */
- GEM_BUG_ON(sseu->max_slices * sseu->max_subslices >
sizeof(g_ss_en) * BITS_PER_BYTE);
- sseu->slice_mask |= BIT(0);
- sseu->subslice_mask.hsw[0] = ss_en & valid_ss_mask;
- for (ss = 0; ss < sseu->max_subslices; ss++)
if (intel_sseu_has_subslice(sseu, 0, ss))sseu_set_eus(sseu, 0, ss, eu_en);- sseu->eu_per_subslice = hweight16(eu_en);
- sseu->eu_total = compute_eu_total(sseu);
+}
+static void xehp_compute_sseu_info(struct sseu_dev_info *sseu,
u16 eu_en)+{
int ss;
sseu->slice_mask |= BIT(0);
- /*
* XeHP introduces the concept of compute vs geometry DSS. To reduce* variation between GENs around subslice usage, store a mask for both* the geometry and compute enabled masks since userspace will need to* be able to query these masks independently. Also compute a total* enabled subslice count for the purposes of selecting subslices to* use in a particular GEM context.*/- intel_sseu_set_subslices(sseu, 0, sseu->compute_subslice_mask,
c_ss_en & valid_ss_mask);- intel_sseu_set_subslices(sseu, 0, sseu->geometry_subslice_mask,
g_ss_en & valid_ss_mask);- intel_sseu_set_subslices(sseu, 0, sseu->subslice_mask,
(g_ss_en | c_ss_en) & valid_ss_mask);
bitmap_or(sseu->subslice_mask.xehp,
sseu->compute_subslice_mask.xehp,sseu->geometry_subslice_mask.xehp,XEHP_BITMAP_BITS(sseu->subslice_mask));for (ss = 0; ss < sseu->max_subslices; ss++) if (intel_sseu_has_subslice(sseu, 0, ss))
@@ -182,11 +185,31 @@ static void gen11_compute_sseu_info(struct sseu_dev_info *sseu, sseu->eu_total = compute_eu_total(sseu); }
+static void +xehp_load_dss_mask(struct intel_uncore *uncore,
intel_sseu_ss_mask_t *ssmask,int numregs,...)+{
- va_list argp;
- u32 fuse_val[I915_MAX_SS_FUSE_REGS] = {};
- int i;
- if (WARN_ON(numregs > I915_MAX_SS_FUSE_REGS))
numregs = I915_MAX_SS_FUSE_REGS;- va_start(argp, numregs);
- for (i = 0; i < numregs; i++)
fuse_val[i] = intel_uncore_read(uncore, va_arg(argp, i915_reg_t));- va_end(argp);
- bitmap_from_arr32(ssmask->xehp, fuse_val, numregs * 32);
+}
static void xehp_sseu_info_init(struct intel_gt *gt) { struct sseu_dev_info *sseu = >->info.sseu; struct intel_uncore *uncore = gt->uncore;
- u32 g_dss_en, c_dss_en = 0; u16 eu_en = 0; u8 eu_en_fuse; int eu;
@@ -200,8 +223,10 @@ static void xehp_sseu_info_init(struct intel_gt *gt) intel_sseu_set_info(sseu, 1, 32, 16); sseu->has_xehp_dss = 1;
- g_dss_en = intel_uncore_read(uncore, GEN12_GT_GEOMETRY_DSS_ENABLE);
- c_dss_en = intel_uncore_read(uncore, GEN12_GT_COMPUTE_DSS_ENABLE);
xehp_load_dss_mask(uncore, &sseu->geometry_subslice_mask, 1,
GEN12_GT_GEOMETRY_DSS_ENABLE);xehp_load_dss_mask(uncore, &sseu->compute_subslice_mask, 1,
GEN12_GT_COMPUTE_DSS_ENABLE);eu_en_fuse = intel_uncore_read(uncore, XEHP_EU_ENABLE) & XEHP_EU_ENA_MASK;
@@ -209,7 +234,7 @@ static void xehp_sseu_info_init(struct intel_gt *gt) if (eu_en_fuse & BIT(eu)) eu_en |= BIT(eu * 2) | BIT(eu * 2 + 1);
- gen11_compute_sseu_info(sseu, g_dss_en, c_dss_en, eu_en);
- xehp_compute_sseu_info(sseu, eu_en);
}
static void gen12_sseu_info_init(struct intel_gt *gt) @@ -247,7 +272,7 @@ static void gen12_sseu_info_init(struct intel_gt *gt) if (eu_en_fuse & BIT(eu)) eu_en |= BIT(eu * 2) | BIT(eu * 2 + 1);
- gen11_compute_sseu_info(sseu, g_dss_en, 0, eu_en);
gen11_compute_sseu_info(sseu, g_dss_en, eu_en);
/* TGL only supports slice-level power gating */ sseu->has_slice_pg = 1;
@@ -279,7 +304,7 @@ static void gen11_sseu_info_init(struct intel_gt *gt) eu_en = ~(intel_uncore_read(uncore, GEN11_EU_DISABLE) & GEN11_EU_DIS_MASK);
- gen11_compute_sseu_info(sseu, ss_en, 0, eu_en);
gen11_compute_sseu_info(sseu, ss_en, eu_en);
/* ICL has no power gating restrictions. */ sseu->has_slice_pg = 1;
@@ -291,7 +316,6 @@ static void cherryview_sseu_info_init(struct intel_gt *gt) { struct sseu_dev_info *sseu = >->info.sseu; u32 fuse;
u8 subslice_mask = 0;
fuse = intel_uncore_read(gt->uncore, CHV_FUSE_GT);
@@ -305,7 +329,7 @@ static void cherryview_sseu_info_init(struct intel_gt *gt) (((fuse & CHV_FGT_EU_DIS_SS0_R1_MASK) >> CHV_FGT_EU_DIS_SS0_R1_SHIFT) << 4);
subslice_mask |= BIT(0);
sseu->subslice_mask.hsw[0] |= BIT(0);@@ -316,12 +340,10 @@ static void cherryview_sseu_info_init(struct intel_gt *gt) (((fuse & CHV_FGT_EU_DIS_SS1_R1_MASK) >> CHV_FGT_EU_DIS_SS1_R1_SHIFT) << 4);
subslice_mask |= BIT(1);
sseu->subslice_mask.hsw[0] |= BIT(1);
intel_sseu_set_subslices(sseu, 0, sseu->subslice_mask, subslice_mask);
sseu->eu_total = compute_eu_total(sseu);
/*
@@ -376,8 +398,7 @@ static void gen9_sseu_info_init(struct intel_gt *gt) /* skip disabled slice */ continue;
intel_sseu_set_subslices(sseu, s, sseu->subslice_mask,subslice_mask);
sseu->subslice_mask.hsw[s] = subslice_mask;eu_disable = intel_uncore_read(uncore, GEN9_EU_DISABLE(s)); for (ss = 0; ss < sseu->max_subslices; ss++) {
@@ -434,8 +455,8 @@ static void gen9_sseu_info_init(struct intel_gt *gt) sseu->has_eu_pg = sseu->eu_per_subslice > 2;
if (IS_GEN9_LP(i915)) { -#define IS_SS_DISABLED(ss) (!(sseu->subslice_mask[0] & BIT(ss)))
info->has_pooled_eu = hweight8(sseu->subslice_mask[0]) == 3;+#define IS_SS_DISABLED(ss) (!(sseu->subslice_mask.hsw[0] & BIT(ss)))
info->has_pooled_eu = hweight8(sseu->subslice_mask.hsw[0]) == 3;sseu->min_eu_in_pool = 0; if (info->has_pooled_eu) {
@@ -489,8 +510,7 @@ static void bdw_sseu_info_init(struct intel_gt *gt) /* skip disabled slice */ continue;
intel_sseu_set_subslices(sseu, s, sseu->subslice_mask,subslice_mask);
sseu->subslice_mask.hsw[s] = subslice_mask;for (ss = 0; ss < sseu->max_subslices; ss++) { u8 eu_disabled_mask;
@@ -587,8 +607,7 @@ static void hsw_sseu_info_init(struct intel_gt *gt) sseu->eu_per_subslice);
for (s = 0; s < sseu->max_slices; s++) {
intel_sseu_set_subslices(sseu, s, sseu->subslice_mask,subslice_mask);
sseu->subslice_mask.hsw[s] = subslice_mask;for (ss = 0; ss < sseu->max_subslices; ss++) { sseu_set_eus(sseu, s, ss,
@@ -677,7 +696,7 @@ u32 intel_sseu_make_rpcs(struct intel_gt *gt, */ if (GRAPHICS_VER(i915) == 11 && slices == 1 &&
subslices > min_t(u8, 4, hweight8(sseu->subslice_mask[0]) / 2)) {
subslices > min_t(u8, 4, hweight8(sseu->subslice_mask.hsw[0]) / 2)) {GEM_BUG_ON(subslices & 1);
subslice_pg = false;
@@ -743,14 +762,29 @@ void intel_sseu_dump(const struct sseu_dev_info *sseu, struct drm_printer *p) { int s;
- drm_printf(p, "slice total: %u, mask=%04x\n",
hweight8(sseu->slice_mask), sseu->slice_mask);- drm_printf(p, "subslice total: %u\n", intel_sseu_subslice_total(sseu));
- for (s = 0; s < sseu->max_slices; s++) {
drm_printf(p, "slice%d: %u subslices, mask=%08x\n",s, intel_sseu_subslices_per_slice(sseu, s),intel_sseu_get_subslices(sseu, s));
- if (sseu->has_xehp_dss) {
drm_printf(p, "subslice total: %u\n",intel_sseu_subslice_total(sseu));drm_printf(p, "geometry dss mask=%*pb\n",XEHP_BITMAP_BITS(sseu->geometry_subslice_mask),sseu->geometry_subslice_mask.xehp);drm_printf(p, "compute dss mask=%*pb\n",XEHP_BITMAP_BITS(sseu->compute_subslice_mask),sseu->compute_subslice_mask.xehp);- } else {
drm_printf(p, "slice total: %u, mask=%04x\n",hweight8(sseu->slice_mask), sseu->slice_mask);drm_printf(p, "subslice total: %u\n",intel_sseu_subslice_total(sseu));for (s = 0; s < sseu->max_slices; s++) {u8 ss_mask = sseu->subslice_mask.hsw[s];drm_printf(p, "slice%d: %u subslices, mask=%08x\n",s, hweight8(ss_mask), ss_mask);}- drm_printf(p, "EU total: %u\n", sseu->eu_total); drm_printf(p, "EU per subslice: %u\n", sseu->eu_per_subslice); drm_printf(p, "has slice power gating: %s\n",
@@ -767,9 +801,10 @@ static void sseu_print_hsw_topology(const struct sseu_dev_info *sseu, int s, ss;
for (s = 0; s < sseu->max_slices; s++) {
u8 ss_mask = sseu->subslice_mask.hsw[s];- drm_printf(p, "slice%d: %u subslice(s) (0x%08x):\n",
s, intel_sseu_subslices_per_slice(sseu, s),intel_sseu_get_subslices(sseu, s));
s, hweight8(ss_mask), ss_mask);for (ss = 0; ss < sseu->max_subslices; ss++) { u16 enabled_eus = sseu_get_eus(sseu, s, ss);
@@ -783,16 +818,14 @@ static void sseu_print_hsw_topology(const struct sseu_dev_info *sseu, static void sseu_print_xehp_topology(const struct sseu_dev_info *sseu, struct drm_printer *p) {
u32 g_dss_mask = sseu_get_geometry_subslices(sseu);
u32 c_dss_mask = intel_sseu_get_compute_subslices(sseu); int dss;
for (dss = 0; dss < sseu->max_subslices; dss++) { u16 enabled_eus = sseu_get_eus(sseu, 0, dss);
drm_printf(p, "DSS_%02d: G:%3s C:%3s, %2u EUs (0x%04hx)\n", dss,
str_yes_no(g_dss_mask & BIT(dss)),str_yes_no(c_dss_mask & BIT(dss)),
str_yes_no(test_bit(dss, sseu->geometry_subslice_mask.xehp)),str_yes_no(test_bit(dss, sseu->compute_subslice_mask.xehp)), hweight16(enabled_eus), enabled_eus);} @@ -810,20 +843,44 @@ void intel_sseu_print_topology(struct drm_i915_private *i915, } }
-u16 intel_slicemask_from_dssmask(u64 dss_mask, int dss_per_slice) +void intel_sseu_print_ss_info(const char* type,
const struct sseu_dev_info *sseu,struct seq_file *m)+{
- int s;
- if (sseu->has_xehp_dss) {
seq_printf(m, " %s Geometry DSS: %u\n", type,bitmap_weight(sseu->geometry_subslice_mask.xehp,XEHP_BITMAP_BITS(sseu->geometry_subslice_mask)));seq_printf(m, " %s Compute DSS: %u\n", type,bitmap_weight(sseu->compute_subslice_mask.xehp,XEHP_BITMAP_BITS(sseu->compute_subslice_mask)));- } else {
for (s = 0; s < fls(sseu->slice_mask); s++)seq_printf(m, " %s Slice%i subslices: %u\n", type,s, hweight8(sseu->subslice_mask.hsw[s]));- }
+}
+u16 intel_slicemask_from_xehp_dssmask(intel_sseu_ss_mask_t dss_mask,
int dss_per_slice){
- intel_sseu_ss_mask_t per_slice_mask = {}; u16 slice_mask = 0; int i;
- WARN_ON(sizeof(dss_mask) * 8 / dss_per_slice > 8 * sizeof(slice_mask));
- WARN_ON(DIV_ROUND_UP(XEHP_BITMAP_BITS(dss_mask), dss_per_slice) >
8 * sizeof(slice_mask));
- for (i = 0; dss_mask; i++) {
if (dss_mask & GENMASK(dss_per_slice - 1, 0))
- bitmap_fill(per_slice_mask.xehp, dss_per_slice);
- for (i = 0; !bitmap_empty(dss_mask.xehp, XEHP_BITMAP_BITS(dss_mask)); i++) {
if (bitmap_intersects(dss_mask.xehp, per_slice_mask.xehp, dss_per_slice)) slice_mask |= BIT(i);
dss_mask >>= dss_per_slice;
bitmap_shift_right(dss_mask.xehp, dss_mask.xehp, dss_per_slice,XEHP_BITMAP_BITS(dss_mask));}
return slice_mask;
}
diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.h b/drivers/gpu/drm/i915/gt/intel_sseu.h index ffa375e68959..e7f336a7ffbb 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu.h +++ b/drivers/gpu/drm/i915/gt/intel_sseu.h @@ -25,12 +25,16 @@ struct drm_printer; /*
- Maximum number of subslices that can exist within a HSW-style slice. This
- is only relevant to pre-Xe_HP platforms (Xe_HP and beyond use the
- GEN_MAX_DSS value below).
*/
- I915_MAX_SS_FUSE_BITS value below).
#define GEN_MAX_SS_PER_HSW_SLICE 6
-/* Maximum number of DSS on newer platforms (Xe_HP and beyond). */ -#define GEN_MAX_DSS 32 +/*
- Maximum number of 32-bit registers used by hardware to express the
- enabled/disabled subslices.
- */
+#define I915_MAX_SS_FUSE_REGS 1 +#define I915_MAX_SS_FUSE_BITS (I915_MAX_SS_FUSE_REGS * 32)
/* Maximum number of EUs that can exist within a subslice or DSS. */ #define GEN_MAX_EUS_PER_SS 16 @@ -38,7 +42,7 @@ struct drm_printer; #define SSEU_MAX(a, b) ((a) > (b) ? (a) : (b))
/* The maximum number of bits needed to express each subslice/DSS independently */ -#define GEN_SS_MASK_SIZE SSEU_MAX(GEN_MAX_DSS, \ +#define GEN_SS_MASK_SIZE SSEU_MAX(I915_MAX_SS_FUSE_BITS, \ GEN_MAX_HSW_SLICES * GEN_MAX_SS_PER_HSW_SLICE)
#define GEN_SSEU_STRIDE(max_entries) DIV_ROUND_UP(max_entries, BITS_PER_BYTE) @@ -49,17 +53,24 @@ struct drm_printer; #define GEN_DSS_PER_CSLICE 8 #define GEN_DSS_PER_MSLICE 8
-#define GEN_MAX_GSLICES (GEN_MAX_DSS / GEN_DSS_PER_GSLICE) -#define GEN_MAX_CSLICES (GEN_MAX_DSS / GEN_DSS_PER_CSLICE) +#define GEN_MAX_GSLICES (I915_MAX_SS_FUSE_BITS / GEN_DSS_PER_GSLICE) +#define GEN_MAX_CSLICES (I915_MAX_SS_FUSE_BITS / GEN_DSS_PER_CSLICE)
+typedef union {
- u8 hsw[GEN_MAX_HSW_SLICES];
- /* Bitmap compatible with linux/bitmap.h; may exceed size of u64 */
- unsigned long xehp[BITS_TO_LONGS(I915_MAX_SS_FUSE_BITS)];
+} intel_sseu_ss_mask_t;
struct sseu_dev_info { u8 slice_mask;
- u8 subslice_mask[GEN_SS_MASK_SIZE];
- u8 geometry_subslice_mask[GEN_SS_MASK_SIZE];
- u8 compute_subslice_mask[GEN_SS_MASK_SIZE];
- intel_sseu_ss_mask_t subslice_mask;
- intel_sseu_ss_mask_t geometry_subslice_mask;
- intel_sseu_ss_mask_t compute_subslice_mask; union { u16 hsw[GEN_MAX_HSW_SLICES][GEN_MAX_SS_PER_HSW_SLICE];
u16 xehp[GEN_MAX_DSS];
u16 xehp[I915_MAX_SS_FUSE_BITS];} eu_mask;
u16 eu_total;
@@ -80,8 +91,6 @@ struct sseu_dev_info { u8 max_slices; u8 max_subslices; u8 max_eus_per_subslice;
- u8 ss_stride;
};
/* @@ -99,7 +108,7 @@ intel_sseu_from_device_info(const struct sseu_dev_info *sseu) { struct intel_sseu value = { .slice_mask = sseu->slice_mask,
.subslice_mask = sseu->subslice_mask[0],
.subslice_mask = sseu->subslice_mask.hsw[0],@@ -111,18 +120,27 @@ static inline bool intel_sseu_has_subslice(const struct sseu_dev_info *sseu, int slice, int subslice) {
u8 mask;
int ss_idx = subslice / BITS_PER_BYTE;
if (slice >= sseu->max_slices || subslice >= sseu->max_subslices) return false;
GEM_BUG_ON(ss_idx >= sseu->ss_stride);
mask = sseu->subslice_mask[slice * sseu->ss_stride + ss_idx];
- if (sseu->has_xehp_dss)
return test_bit(subslice, sseu->subslice_mask.xehp);- else
return sseu->subslice_mask.hsw[slice] & BIT(subslice);+}
- return mask & BIT(subslice % BITS_PER_BYTE);
+/*
- Used to obtain the index of the first DSS. Can start searching from the
- beginning of a specific dss group (e.g., gslice, cslice, etc.) if
- groupsize and groupnum are non-zero.
- */
+static inline unsigned int +intel_sseu_find_first_xehp_dss(const struct sseu_dev_info *sseu, int groupsize,
int groupnum)+{
- return find_next_bit(sseu->subslice_mask.xehp, I915_MAX_SS_FUSE_BITS,
groupnum * groupsize);Any specific reason for not replacing I915_MAX_SS_FUSE_BITS with XEHP_BITMAP_BITS here?
It looks like I just missed this one while changing the others. I'll send an update that makes that change.
Matt
Regards, Bala
}
void intel_sseu_set_info(struct sseu_dev_info *sseu, u8 max_slices, @@ -132,14 +150,10 @@ unsigned int intel_sseu_subslice_total(const struct sseu_dev_info *sseu);
unsigned int -intel_sseu_subslices_per_slice(const struct sseu_dev_info *sseu, u8 slice);
-u32 intel_sseu_get_subslices(const struct sseu_dev_info *sseu, u8 slice); +intel_sseu_get_hsw_subslices(const struct sseu_dev_info *sseu, u8 slice);
-u32 intel_sseu_get_compute_subslices(const struct sseu_dev_info *sseu);
-void intel_sseu_set_subslices(struct sseu_dev_info *sseu, int slice,
u8 *subslice_mask, u32 ss_mask);+intel_sseu_ss_mask_t +intel_sseu_get_compute_subslices(const struct sseu_dev_info *sseu);
void intel_sseu_info_init(struct intel_gt *gt);
@@ -151,9 +165,15 @@ void intel_sseu_print_topology(struct drm_i915_private *i915, const struct sseu_dev_info *sseu, struct drm_printer *p);
-u16 intel_slicemask_from_dssmask(u64 dss_mask, int dss_per_slice); +u16 intel_slicemask_from_xehp_dssmask(intel_sseu_ss_mask_t dss_mask, int dss_per_slice);
int intel_sseu_copy_eumask_to_user(void __user *to, const struct sseu_dev_info *sseu); +int intel_sseu_copy_ssmask_to_user(void __user *to,
const struct sseu_dev_info *sseu);+void intel_sseu_print_ss_info(const char* type,
const struct sseu_dev_info *sseu,struct seq_file *m);#endif /* __INTEL_SSEU_H__ */ diff --git a/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c b/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c index 2d5d011e01db..c2ee5e1826b5 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c +++ b/drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c @@ -4,6 +4,7 @@
- Copyright © 2020 Intel Corporation
*/
+#include <linux/bitmap.h> #include <linux/string_helpers.h>
#include "i915_drv.h" @@ -11,14 +12,6 @@ #include "intel_gt_regs.h" #include "intel_sseu_debugfs.h"
-static void sseu_copy_subslices(const struct sseu_dev_info *sseu,
int slice, u8 *to_mask)-{
- int offset = slice * sseu->ss_stride;
- memcpy(&to_mask[offset], &sseu->subslice_mask[offset], sseu->ss_stride);
-}
static void cherryview_sseu_device_status(struct intel_gt *gt, struct sseu_dev_info *sseu) { @@ -41,7 +34,7 @@ static void cherryview_sseu_device_status(struct intel_gt *gt, continue;
sseu->slice_mask = BIT(0);
sseu->subslice_mask[0] |= BIT(ss);
sseu->subslice_mask.hsw[0] |= BIT(ss);@@ -92,7 +85,7 @@ static void gen11_sseu_device_status(struct intel_gt *gt, continue;
sseu->slice_mask |= BIT(s);
sseu_copy_subslices(&info->sseu, s, sseu->subslice_mask);
sseu->subslice_mask.hsw[s] = info->sseu.subslice_mask.hsw[s];for (ss = 0; ss < info->sseu.max_subslices; ss++) { unsigned int eu_cnt;
@@ -147,21 +140,17 @@ static void gen9_sseu_device_status(struct intel_gt *gt, sseu->slice_mask |= BIT(s);
if (IS_GEN9_BC(gt->i915))
sseu_copy_subslices(&info->sseu, s,sseu->subslice_mask);
sseu->subslice_mask.hsw[s] = info->sseu.subslice_mask.hsw[s];for (ss = 0; ss < info->sseu.max_subslices; ss++) { unsigned int eu_cnt;
u8 ss_idx = s * info->sseu.ss_stride +ss / BITS_PER_BYTE; if (IS_GEN9_LP(gt->i915)) { if (!(s_reg[s] & (GEN9_PGCTL_SS_ACK(ss)))) /* skip disabled subslice */ continue;sseu->subslice_mask[ss_idx] |=BIT(ss % BITS_PER_BYTE);
sseu->subslice_mask.hsw[s] |= BIT(ss); } eu_cnt = eu_reg[2 * s + ss / 2] & eu_mask[ss % 2];@@ -188,8 +177,7 @@ static void bdw_sseu_device_status(struct intel_gt *gt, if (sseu->slice_mask) { sseu->eu_per_subslice = info->sseu.eu_per_subslice; for (s = 0; s < fls(sseu->slice_mask); s++)
sseu_copy_subslices(&info->sseu, s,sseu->subslice_mask);
sseu->subslice_mask.hsw[s] = info->sseu.subslice_mask.hsw[s];@@ -208,7 +196,6 @@ static void i915_print_sseu_info(struct seq_file *m, const struct sseu_dev_info *sseu) { const char *type = is_available_info ? "Available" : "Enabled";
int s;
seq_printf(m, " %s Slice Mask: %04x\n", type, sseu->slice_mask);
@@ -216,10 +203,7 @@ static void i915_print_sseu_info(struct seq_file *m, hweight8(sseu->slice_mask)); seq_printf(m, " %s Subslice Total: %u\n", type, intel_sseu_subslice_total(sseu));
- for (s = 0; s < fls(sseu->slice_mask); s++) {
seq_printf(m, " %s Slice%i subslices: %u\n", type,s, intel_sseu_subslices_per_slice(sseu, s));- }
- intel_sseu_print_ss_info(type, sseu, m); seq_printf(m, " %s EU Total: %u\n", type, sseu->eu_total); seq_printf(m, " %s EU Per Subslice: %u\n", type,
diff --git a/drivers/gpu/drm/i915/gt/intel_workarounds.c b/drivers/gpu/drm/i915/gt/intel_workarounds.c index 73b59ea6fd3b..1bf62273e02d 100644 --- a/drivers/gpu/drm/i915/gt/intel_workarounds.c +++ b/drivers/gpu/drm/i915/gt/intel_workarounds.c @@ -948,8 +948,8 @@ gen9_wa_init_mcr(struct drm_i915_private *i915, struct i915_wa_list *wal) * on s/ss combo, the read should be done with read_subslice_reg. */ slice = ffs(sseu->slice_mask) - 1;
- GEM_BUG_ON(slice >= ARRAY_SIZE(sseu->subslice_mask));
- subslice = ffs(intel_sseu_get_subslices(sseu, slice));
- GEM_BUG_ON(slice >= ARRAY_SIZE(sseu->subslice_mask.hsw));
- subslice = ffs(intel_sseu_get_hsw_subslices(sseu, slice)); GEM_BUG_ON(!subslice); subslice--;
@@ -1087,11 +1087,10 @@ static void icl_wa_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) { const struct sseu_dev_info *sseu = >->info.sseu;
- unsigned int slice, subslice;
unsigned int subslice;
GEM_BUG_ON(GRAPHICS_VER(gt->i915) < 11); GEM_BUG_ON(hweight8(sseu->slice_mask) > 1);
slice = 0;
/*
- Although a platform may have subslices, we need to always steer
@@ -1102,7 +1101,7 @@ icl_wa_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) * one of the higher subslices, we run the risk of reading back 0's or * random garbage. */
- subslice = __ffs(intel_sseu_get_subslices(sseu, slice));
subslice = __ffs(intel_sseu_get_hsw_subslices(sseu, 0));
/*
- If the subslice we picked above also steers us to a valid L3 bank,
@@ -1112,7 +1111,7 @@ icl_wa_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) if (gt->info.l3bank_mask & BIT(subslice)) gt->steering_table[L3BANK] = NULL;
- __add_mcr_wa(gt, wal, slice, subslice);
- __add_mcr_wa(gt, wal, 0, subslice);
}
static void @@ -1120,7 +1119,6 @@ xehp_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) { const struct sseu_dev_info *sseu = >->info.sseu; unsigned long slice, subslice = 0, slice_mask = 0;
- u64 dss_mask = 0; u32 lncf_mask = 0; int i;
@@ -1151,8 +1149,8 @@ xehp_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) */
/* Find the potential gslice candidates */
- dss_mask = intel_sseu_get_subslices(sseu, 0);
- slice_mask = intel_slicemask_from_dssmask(dss_mask, GEN_DSS_PER_GSLICE);
slice_mask = intel_slicemask_from_xehp_dssmask(sseu->subslice_mask,
GEN_DSS_PER_GSLICE);/*
- Find the potential LNCF candidates. Either LNCF within a valid
@@ -1177,9 +1175,8 @@ xehp_init_mcr(struct intel_gt *gt, struct i915_wa_list *wal) }
slice = __ffs(slice_mask);
- subslice = __ffs(dss_mask >> (slice * GEN_DSS_PER_GSLICE));
- subslice = intel_sseu_find_first_xehp_dss(sseu, GEN_DSS_PER_GSLICE, slice); WARN_ON(subslice > GEN_DSS_PER_GSLICE);
WARN_ON(dss_mask >> (slice * GEN_DSS_PER_GSLICE) == 0);
__add_mcr_wa(gt, wal, slice, subslice);
@@ -2030,9 +2027,8 @@ engine_fake_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
static bool needs_wa_1308578152(struct intel_engine_cs *engine) {
- u64 dss_mask = intel_sseu_get_subslices(&engine->gt->info.sseu, 0);
- return (dss_mask & GENMASK(GEN_DSS_PER_GSLICE - 1, 0)) == 0;
- return intel_sseu_find_first_xehp_dss(&engine->gt->info.sseu, 0, 0) >
GEN_DSS_PER_GSLICE;}
static void diff --git a/drivers/gpu/drm/i915/i915_getparam.c b/drivers/gpu/drm/i915/i915_getparam.c index ac9767c56619..6fd15b39570c 100644 --- a/drivers/gpu/drm/i915/i915_getparam.c +++ b/drivers/gpu/drm/i915/i915_getparam.c @@ -162,8 +162,7 @@ int i915_getparam_ioctl(struct drm_device *dev, void *data, return -EINVAL;
/* Only copy bits from the first slice */
memcpy(&value, sseu->subslice_mask,min(sseu->ss_stride, (u8)sizeof(value)));
value = intel_sseu_get_hsw_subslices(sseu, 0);diff --git a/drivers/gpu/drm/i915/i915_query.c b/drivers/gpu/drm/i915/i915_query.c index 89c475d525b8..0094f67c63f2 100644 --- a/drivers/gpu/drm/i915/i915_query.c +++ b/drivers/gpu/drm/i915/i915_query.c @@ -31,10 +31,11 @@ static int copy_query_item(void *query_hdr, size_t query_sz,
static int fill_topology_info(const struct sseu_dev_info *sseu, struct drm_i915_query_item *query_item,
const u8 *subslice_mask)
intel_sseu_ss_mask_t subslice_mask){ struct drm_i915_query_topology_info topo; u32 slice_length, subslice_length, eu_length, total_length;
- int ss_stride = GEN_SSEU_STRIDE(sseu->max_subslices); int eu_stride = GEN_SSEU_STRIDE(sseu->max_eus_per_subslice); int ret;
@@ -44,7 +45,7 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, return -ENODEV;
slice_length = sizeof(sseu->slice_mask);
- subslice_length = sseu->max_slices * sseu->ss_stride;
- subslice_length = sseu->max_slices * ss_stride; eu_length = sseu->max_slices * sseu->max_subslices * eu_stride; total_length = sizeof(topo) + slice_length + subslice_length + eu_length;
@@ -60,7 +61,7 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, topo.max_eus_per_subslice = sseu->max_eus_per_subslice;
topo.subslice_offset = slice_length;
- topo.subslice_stride = sseu->ss_stride;
- topo.subslice_stride = ss_stride; topo.eu_offset = slice_length + subslice_length; topo.eu_stride = eu_stride;
@@ -72,9 +73,9 @@ static int fill_topology_info(const struct sseu_dev_info *sseu, &sseu->slice_mask, slice_length)) return -EFAULT;
- if (copy_to_user(u64_to_user_ptr(query_item->data_ptr +
sizeof(topo) + slice_length),subslice_mask, subslice_length))
if (intel_sseu_copy_ssmask_to_user(u64_to_user_ptr(query_item->data_ptr +
sizeof(topo) + slice_length),sseu))return -EFAULT;
if (intel_sseu_copy_eumask_to_user(u64_to_user_ptr(query_item->data_ptr +
-- 2.35.3
PVC splits the mask of enabled DSS over two registers. It also changes the meaning of the EU fuse register such that each bit represents a single EU rather than a pair of EUs.
Signed-off-by: Matt Roper matthew.d.roper@intel.com --- drivers/gpu/drm/i915/gt/intel_gt_regs.h | 1 + drivers/gpu/drm/i915/gt/intel_sseu.c | 31 ++++++++++++++++++------ drivers/gpu/drm/i915/gt/intel_sseu.h | 2 +- drivers/gpu/drm/i915/i915_drv.h | 2 ++ drivers/gpu/drm/i915/i915_pci.c | 3 ++- drivers/gpu/drm/i915/intel_device_info.h | 1 + 6 files changed, 31 insertions(+), 9 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_regs.h b/drivers/gpu/drm/i915/gt/intel_gt_regs.h index 7246eb870c7e..3f2b46c884f9 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_regs.h +++ b/drivers/gpu/drm/i915/gt/intel_gt_regs.h @@ -561,6 +561,7 @@ #define GEN11_GT_VEBOX_DISABLE_MASK (0x0f << GEN11_GT_VEBOX_DISABLE_SHIFT)
#define GEN12_GT_COMPUTE_DSS_ENABLE _MMIO(0x9144) +#define XEHPC_GT_COMPUTE_DSS_ENABLE_EXT _MMIO(0x9148)
#define GEN6_UCGCTL1 _MMIO(0x9400) #define GEN6_GAMUNIT_CLOCK_GATE_DISABLE (1 << 22) diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.c b/drivers/gpu/drm/i915/gt/intel_sseu.c index 6fbc2ac507b7..82e6f324b2ef 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu.c +++ b/drivers/gpu/drm/i915/gt/intel_sseu.c @@ -212,27 +212,44 @@ static void xehp_sseu_info_init(struct intel_gt *gt) struct intel_uncore *uncore = gt->uncore; u16 eu_en = 0; u8 eu_en_fuse; + int num_compute_regs, num_geometry_regs; int eu;
+ if (IS_PONTEVECCHIO(gt->i915)) { + num_geometry_regs = 0; + num_compute_regs = 2; + } else { + num_geometry_regs = 1; + num_compute_regs = 1; + } + /* * The concept of slice has been removed in Xe_HP. To be compatible * with prior generations, assume a single slice across the entire * device. Then calculate out the DSS for each workload type within * that software slice. */ - intel_sseu_set_info(sseu, 1, 32, 16); + intel_sseu_set_info(sseu, 1, + 32 * max(num_geometry_regs, num_compute_regs), + 16); sseu->has_xehp_dss = 1;
- xehp_load_dss_mask(uncore, &sseu->geometry_subslice_mask, 1, + xehp_load_dss_mask(uncore, &sseu->geometry_subslice_mask, + num_geometry_regs, GEN12_GT_GEOMETRY_DSS_ENABLE); - xehp_load_dss_mask(uncore, &sseu->compute_subslice_mask, 1, - GEN12_GT_COMPUTE_DSS_ENABLE); + xehp_load_dss_mask(uncore, &sseu->compute_subslice_mask, + num_compute_regs, + GEN12_GT_COMPUTE_DSS_ENABLE, + XEHPC_GT_COMPUTE_DSS_ENABLE_EXT);
eu_en_fuse = intel_uncore_read(uncore, XEHP_EU_ENABLE) & XEHP_EU_ENA_MASK;
- for (eu = 0; eu < sseu->max_eus_per_subslice / 2; eu++) - if (eu_en_fuse & BIT(eu)) - eu_en |= BIT(eu * 2) | BIT(eu * 2 + 1); + if (HAS_ONE_EU_PER_FUSE_BIT(gt->i915)) + eu_en = eu_en_fuse; + else + for (eu = 0; eu < sseu->max_eus_per_subslice / 2; eu++) + if (eu_en_fuse & BIT(eu)) + eu_en |= BIT(eu * 2) | BIT(eu * 2 + 1);
xehp_compute_sseu_info(sseu, eu_en); } diff --git a/drivers/gpu/drm/i915/gt/intel_sseu.h b/drivers/gpu/drm/i915/gt/intel_sseu.h index 0d3def55e770..79732c1af6c4 100644 --- a/drivers/gpu/drm/i915/gt/intel_sseu.h +++ b/drivers/gpu/drm/i915/gt/intel_sseu.h @@ -33,7 +33,7 @@ struct drm_printer; * Maximum number of 32-bit registers used by hardware to express the * enabled/disabled subslices. */ -#define I915_MAX_SS_FUSE_REGS 1 +#define I915_MAX_SS_FUSE_REGS 2 #define I915_MAX_SS_FUSE_BITS (I915_MAX_SS_FUSE_REGS * 32)
/* Maximum number of EUs that can exist within a subslice or DSS. */ diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h index c5fc402d9c50..1f8422e9511b 100644 --- a/drivers/gpu/drm/i915/i915_drv.h +++ b/drivers/gpu/drm/i915/i915_drv.h @@ -1405,6 +1405,8 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
#define HAS_MBUS_JOINING(i915) (IS_ALDERLAKE_P(i915))
+#define HAS_ONE_EU_PER_FUSE_BIT(i915) (INTEL_INFO(i915)->has_one_eu_per_fuse_bit) + /* i915_gem.c */ void i915_gem_init_early(struct drm_i915_private *dev_priv); void i915_gem_cleanup_early(struct drm_i915_private *dev_priv); diff --git a/drivers/gpu/drm/i915/i915_pci.c b/drivers/gpu/drm/i915/i915_pci.c index 5ad9884874c2..9cee3c69edde 100644 --- a/drivers/gpu/drm/i915/i915_pci.c +++ b/drivers/gpu/drm/i915/i915_pci.c @@ -1079,7 +1079,8 @@ static const struct intel_device_info ats_m_info = { #define XE_HPC_FEATURES \ XE_HP_FEATURES, \ .dma_mask_size = 52, \ - .has_l3_ccs_read = 1 + .has_l3_ccs_read = 1, \ + .has_one_eu_per_fuse_bit = 1
__maybe_unused static const struct intel_device_info pvc_info = { diff --git a/drivers/gpu/drm/i915/intel_device_info.h b/drivers/gpu/drm/i915/intel_device_info.h index a134914237dd..d4ae3a7861cd 100644 --- a/drivers/gpu/drm/i915/intel_device_info.h +++ b/drivers/gpu/drm/i915/intel_device_info.h @@ -157,6 +157,7 @@ enum intel_ppgtt_type { func(has_logical_ring_elsq); \ func(has_media_ratio_mode); \ func(has_mslices); \ + func(has_one_eu_per_fuse_bit); \ func(has_pooled_eu); \ func(has_pxp); \ func(has_rc6); \
On 21/05/2022 00:04, Matt Roper wrote:
This series reworks i915's internal handling of slice/subslice/EU (SSEU) data to represent platforms like Xe_HP in a more natural manner and to prepare for future platforms where the masks will need to grow in size. One key idea of this series is that although we have a fixed ABI to convey SSEU data to userspace (i.e., multiple u8[] arrays with data stored at different strides), we don't need to use this cumbersome format for the driver's own internal storage. As long as we can convert into the uapi form properly when responding to the I915_QUERY ioctl, it's preferable to use an internal storage format that's easier for the driver to work with.
Another key point here is that we're reaching the point where subslice (DSS) masks will soon not fit within simple u32/u64 integer values. Xe_HP SDV and DG2 platforms today have subslice (DSS) masks that are 32 bits, which maxes out the current storage of a u32. With PVC the masks are represented by a pair of 32-bit registers, requiring a bump up to at least 64-bits of storage internally. We could switch to u64 for that in the short term, but since we already know that upcoming architectures intend to provide DSS fuse bits via three or more registers it's best to switch to a representation that's more future-proof but still easy to work with in the driver code. To accomodate this, we start storing our subslice mask for Xe_HP and beyond in a new typedef that can be processed by the linux/bitmap.h operations.
Finally, since no userspace for Xe_HP or beyond is using the legacy I915_GETPARAM ioctl lookups for I915_PARAM_SLICE_MASK and I915_PARAM_SUBSLICE_MASK (since they've migrated to the more flexible I915_QUERY ioctl that can return more than a simple u32 value), we take the opportunity to officially drop support for those GETPARAM lookups on modern platforms. Maintaining support for these GETPARAM lookups don't make sense for a number of reasons:
- Traditional slices no longer exist, and newer ideas like gslices, cslices, mslices, etc. aren't something userspace needs to query since it can be inferred from other information.
- The GETPARAM ioctl doesn't have a way to distinguish between geometry subslice masks and compute subslice masks, which are distinct on Xe_HP and beyond.
- The I915_GETPARAM ioctl is limited to returning a 32-bit value, so when subslice masks begin to exceed 32-bits (on PVC), it simply can't return the entire mask.
- The GETPARAM ioctl doesn't have a way to give sensible information for multi-tile devices.
v2:
- Switch to union of hsw/xehp formats to keep the representation in a natural format for different types of hardware.
- Avoid accessing internals of intel_sseu_ss_mask_t directly outside of intel_sseu.[ch].
- Include PVC SSEU which needs the larger SS mask storage enabled by this series.
v3:
- Correct a BIT(s) typo that should have been BIT(ss), causing incorrect handling on gen9 platforms.
v4:
- Eliminate sseu->{ss,eu}_stride fields and just calculate the proper value in the UAPI code that needs them.
- Handle unwanted ~u8 sign extension at the callsite instead of inside sseu_set_eus.
- Use BITMAP_BITS() macro rather than passing I915_MAX_SS_FUSE_BITS around directly to bitmap operations.
- Improved debugfs / dmesg reporting for Xe_HP dumps
- Various assertion check improvements.
Cc: Tvrtko Ursulin tvrtko.ursulin@linux.intel.com
Matt Roper (6): drm/i915/xehp: Use separate sseu init function drm/i915/xehp: Drop GETPARAM lookups of I915_PARAM_[SUB]SLICE_MASK drm/i915/sseu: Simplify gen11+ SSEU handling drm/i915/sseu: Don't try to store EU mask internally in UAPI format drm/i915/sseu: Disassociate internal subslice mask representation from uapi drm/i915/pvc: Add SSEU changes
For the series:
Acked-by: Tvrtko Ursulin tvrtko.ursulin@intel.com
Almost r-b actually, but I do not feel completely comfortable that I read everything closely enough to not have missed something. So I prefer someone else does a really detailed pass to be sure.
Regards,
Tvrtko
drivers/gpu/drm/i915/gem/i915_gem_context.c | 5 +- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 4 +- drivers/gpu/drm/i915/gt/intel_gt.c | 12 +- drivers/gpu/drm/i915/gt/intel_gt_regs.h | 1 + drivers/gpu/drm/i915/gt/intel_sseu.c | 450 ++++++++++++------- drivers/gpu/drm/i915/gt/intel_sseu.h | 94 ++-- drivers/gpu/drm/i915/gt/intel_sseu_debugfs.c | 30 +- drivers/gpu/drm/i915/gt/intel_workarounds.c | 24 +- drivers/gpu/drm/i915/i915_drv.h | 2 + drivers/gpu/drm/i915/i915_getparam.c | 11 +- drivers/gpu/drm/i915/i915_pci.c | 3 +- drivers/gpu/drm/i915/i915_query.c | 26 +- drivers/gpu/drm/i915/intel_device_info.h | 1 + 13 files changed, 398 insertions(+), 265 deletions(-)
dri-devel@lists.freedesktop.org
-
Balasubramani Vivekanandan -
Matt Roper -
Tvrtko Ursulin