Hi Felix,
On Tue, Mar 09, 2021 at 11:30:19AM -0500, Felix Kuehling wrote:
I think the proper fix would be to not rely on custom hooks into a particular IOMMU driver, but to instead ensure that the amdgpu driver can do everything it needs through the regular linux/iommu.h interfaces. I realize this is more work, but I wonder if you've tried that, and why it didn't work out.
As far as I know this hasn't been tried. I see that intel-iommu has its own SVM thing, which seems to be similar to what our IOMMUv2 does. I guess we'd have to abstract that into a common API.
The common API was added in 26b25a2b98e4 and implemented by the Intel driver in 064a57d7ddfc. To support it an IOMMU driver implements new IOMMU ops: .dev_has_feat() .dev_feat_enabled() .dev_enable_feat() .dev_disable_feat() .sva_bind() .sva_unbind() .sva_get_pasid()
And a device driver calls iommu_dev_enable_feature(IOMMU_DEV_FEAT_SVA) followed by iommu_sva_bind_device().
If I remember correctly the biggest obstacle for KFD is the PASID allocation, done by the GPU driver instead of the IOMMU driver, but there may be others.
Thanks, Jean