Does this series look fine now? I would like to try resolving any remaining issues so we Can catch the window for kernel 5.12.
Edward, do you have more opinions on the pyverbs related changes?
Thanks, Jianxin
-----Original Message----- From: Xiong, Jianxin jianxin.xiong@intel.com Sent: Wednesday, December 02, 2020 12:57 PM To: linux-rdma@vger.kernel.org; dri-devel@lists.freedesktop.org Cc: Xiong, Jianxin jianxin.xiong@intel.com; Doug Ledford dledford@redhat.com; Jason Gunthorpe jgg@ziepe.ca; Leon Romanovsky leon@kernel.org; Sumit Semwal sumit.semwal@linaro.org; Christian Koenig christian.koenig@amd.com; Vetter, Daniel daniel.vetter@intel.com; Edward Srouji edwards@nvidia.com Subject: [PATCH rdma-core v4 0/6] Add user space dma-buf support
This is the fourth version of the patch series. Change log:
v4:
- Rework the cmake funciton rdma_cython_module to support both single source (.pyx) and multiple source (.pyx + [.c]*) scenarios instead of using two separate functions
- Rename 'dri_*' to 'drm_*' for the dmabuf allocation interface
- Add option to dmabuf allocation routine to allow allocation from GTT instead of VRAM
- Add proper CPU access flags when allocating dmabufs
- Remove 'radeon' driver support from the dmabuf allocation routines
- Add comand line arguments to the tests for selecting GPU unit and setting the option for allocating from GTT
v3: https://www.spinics.net/lists/linux-rdma/msg98059.html
- Add parameter 'iova' to the new ibv_reg_dmabuf_mr() API
- Change the way of allocating dma-buf object - use /dev/dri/renderD* instead of /dev/dri/card* and use GEM object instead of dumb buffer
- Add cmake function to allow building modules with mixed cython and C source files
- Add new tests that use dma-buf MRs for send/recv and rdma traffic
- Skip dma-buf tests on unsupported systems
- Remove some use of random values in the new tests
- Add dealloc() and close() methods to the new classes
- Replace string.format with f-string in python code
- Fix some coding style issues: spacing, indentation, typo, comments
v2: https://www.spinics.net/lists/linux-rdma/msg97936.html
- Put the kernel header updates into a separate commit
- Add comments for the data structure used in python ioctl calls
- Fix issues related to symbol versioning
- Fix styling issues: extra spaces, unncecessary variable, typo
- Fix an inproper error code usage
- Put the new op into ibv_context_ops instead if verbs_context
v1: https://www.spinics.net/lists/linux-rdma/msg97865.html
- Add user space API for registering dma-buf based memory regions
- Update pyverbs with the new API
- Add new tests
This is the user space counter-part of the kernel patch set to add dma-buf support to the RDMA subsystem.
This series consists of six patches. The first patch updates the kernel headers for dma-buf support. Patch 2 adds the new API function and updates the man pages. Patch 3 implements the new API in the mlx5 provider. Patch 4 adds new class definitions to pyverbs for the new API. Patch 5 adds a set of new tests for the new API. Patch 6 fixes bug in the utility code of the tests.
Pull request at github: https://github.com/linux-rdma/rdma-core/pull/895
Jianxin Xiong (6): Update kernel headers verbs: Support dma-buf based memory region mlx5: Support dma-buf based memory region pyverbs: Add dma-buf based MR support tests: Add tests for dma-buf based memory regions tests: Bug fix for get_access_flags()
buildlib/pyverbs_functions.cmake | 78 ++++++--- debian/libibverbs1.symbols | 2 + kernel-headers/rdma/ib_user_ioctl_cmds.h | 14 ++ kernel-headers/rdma/ib_user_verbs.h | 14 -- libibverbs/CMakeLists.txt | 2 +- libibverbs/cmd_mr.c | 38 +++++ libibverbs/driver.h | 7 + libibverbs/dummy_ops.c | 11 ++ libibverbs/libibverbs.map.in | 6 + libibverbs/man/ibv_reg_mr.3 | 27 ++- libibverbs/verbs.c | 18 ++ libibverbs/verbs.h | 11 ++ providers/mlx5/mlx5.c | 2 + providers/mlx5/mlx5.h | 3 + providers/mlx5/verbs.c | 22 +++ pyverbs/CMakeLists.txt | 11 +- pyverbs/dmabuf.pxd | 15 ++ pyverbs/dmabuf.pyx | 73 ++++++++ pyverbs/dmabuf_alloc.c | 278 +++++++++++++++++++++++++++++++ pyverbs/dmabuf_alloc.h | 19 +++ pyverbs/libibverbs.pxd | 2 + pyverbs/mr.pxd | 6 + pyverbs/mr.pyx | 105 +++++++++++- tests/args_parser.py | 4 + tests/test_mr.py | 264 ++++++++++++++++++++++++++++- tests/utils.py | 30 +++- 26 files changed, 1012 insertions(+), 50 deletions(-) create mode 100644 pyverbs/dmabuf.pxd create mode 100644 pyverbs/dmabuf.pyx create mode 100644 pyverbs/dmabuf_alloc.c create mode 100644 pyverbs/dmabuf_alloc.h
-- 1.8.3.1