On Tue, Jun 30, 2020 at 08:08:46PM +0000, Xiong, Jianxin wrote:
From: Jason Gunthorpe jgg@ziepe.ca Sent: Tuesday, June 30, 2020 12:17 PM To: Xiong, Jianxin jianxin.xiong@intel.com Cc: linux-rdma@vger.kernel.org; Doug Ledford dledford@redhat.com; Sumit Semwal sumit.semwal@linaro.org; Leon Romanovsky leon@kernel.org; Vetter, Daniel daniel.vetter@intel.com; Christian Koenig christian.koenig@amd.com; dri- devel@lists.freedesktop.org Subject: Re: [RFC PATCH v2 0/3] RDMA: add dma-buf support
On Tue, Jun 30, 2020 at 05:21:33PM +0000, Xiong, Jianxin wrote:
> Heterogeneous Memory Management (HMM) utilizes > mmu_interval_notifier and ZONE_DEVICE to support shared > virtual address space and page migration between system memory > and device memory. HMM doesn't support pinning device memory > because pages located on device must be able to migrate to > system memory when accessed by CPU. Peer-to-peer access is > possible if the peer can handle page fault. For RDMA, that means the NIC must support on-demand paging.
peer-peer access is currently not possible with hmm_range_fault().
Currently hmm_range_fault() always sets the cpu access flag and device private pages are migrated to the system RAM in the fault handler. However, it's possible to have a modified code flow to keep the device private page info for use with peer to peer access.
Sort of, but only within the same device, RDMA or anything else generic can't reach inside a DEVICE_PRIVATE and extract anything
useful.
But pfn is supposed to be all that is needed.
Needed for what? The PFN of the DEVICE_PRIVATE pages is useless for anything.
Hmm. I thought the pfn corresponds to the address in the BAR range. I could be wrong here.
No, DEVICE_PRIVATE is a dummy pfn to empty address space.
Jason