RE: [PATCH rdma-core 3/5] pyverbs: Add dma-buf based MR support

-----Original Message----- From: Jason Gunthorpe jgg@ziepe.ca Sent: Monday, November 23, 2020 10:05 AM To: Xiong, Jianxin jianxin.xiong@intel.com Cc: linux-rdma@vger.kernel.org; dri-devel@lists.freedesktop.org; Doug Ledford dledford@redhat.com; Leon Romanovsky leon@kernel.org; Sumit Semwal sumit.semwal@linaro.org; Christian Koenig christian.koenig@amd.com; Vetter, Daniel daniel.vetter@intel.com Subject: Re: [PATCH rdma-core 3/5] pyverbs: Add dma-buf based MR support

On Mon, Nov 23, 2020 at 09:53:02AM -0800, Jianxin Xiong wrote:

...

+cdef class DmaBuf:

• def __init__(self, size, unit=0):

•    """
  

•    Allocate DmaBuf object from a GPU device. This is done through the
  

•    DRI device interface (/dev/dri/card*). Usually this requires the
  

•    effective user id being root or being a member of the 'video' group.
  

•    :param size: The size (in number of bytes) of the buffer.
  

•    :param unit: The unit number of the GPU to allocate the buffer from.
  

•    :return: The newly created DmaBuf object on success.
  

•    """
  

•    self.dmabuf_mrs = weakref.WeakSet()
  

•    self.dri_fd = open('/dev/dri/card'+str(unit), O_RDWR)
  

•    args = bytearray(32)
  

•    pack_into('=iiiiiiq', args, 0, 1, size, 8, 0, 0, 0, 0)
  

•    ioctl(self.dri_fd, DRM_IOCTL_MODE_CREATE_DUMB, args)
  

•    a, b, c, d, self.handle, e, self.size = unpack('=iiiiiiq',
  

• args)

•    args = bytearray(12)
  

•    pack_into('=iii', args, 0, self.handle, O_RDWR, 0)
  

•    ioctl(self.dri_fd, DRM_IOCTL_PRIME_HANDLE_TO_FD, args)
  

•    a, b, self.fd = unpack('=iii', args)
  

•    args = bytearray(16)
  

•    pack_into('=iiq', args, 0, self.handle, 0, 0)
  

•    ioctl(self.dri_fd, DRM_IOCTL_MODE_MAP_DUMB, args);
  

•    a, b, self.map_offset = unpack('=iiq', args);
  

Wow, OK

Is it worth using ctypes here instead? Can you at least add a comment before each pack specifying the 'struct XXX' this is following?

The ioctl call only accept a bytearray, not sure how to use ctypes here. I will add comments with the actual layout of the parameter structure.

Does this work with normal Intel GPUs, like in a Laptop? AMD too?

Yes, the interface is generic and works with most GPUs. Works with AMD, too.

Christian, I would be very happy to hear from you that this entire work is good for AMD as well

Edward should look through this, but I'm glad to see something like this

Thanks, Jason