This is the seventh version of the patch series. Change log:
v7: * Rebase to the latest rdma-core master (commit a4885eda9addc4) * Rerun kernel-headers/update against linux-rdma for-next so that the kernel commit id in the commit message is correct
v6: https://www.spinics.net/lists/linux-rdma/msg99221.html * Rebase to the latest rdma-core master (commit 14006f2f841b0c) * Update the ABI symbol version to match new package version; also bump the private ABI version because new function has been added to the provider interface * Avoid changing 'struct ibv_context_ops' by replacing SET_OP() with SET_PRIV_OP_IC() * Replace sprintf() with snprintf() * Keep the ops in verbs_set_ops() sorted * Fix some styling issues: extra spaces, struct 0-initialization, error checking control flow
v5: https://www.spinics.net/lists/linux-rdma/msg99015.html * Use a different mr_type for dmabuf so that ibv_dofork_range() is not called inside ibv_dereg_mr() for dmabuf based mr
v4: https://www.spinics.net/lists/linux-rdma/msg98135.html * Rework the cmake funciton rdma_cython_module to support both single source (.pyx) and multiple source (.pyx + [.c]*) scenarios instead of using two separate functions * Rename 'dri_*' to 'drm_*' for the dmabuf allocation interface * Add option to dmabuf allocation routine to allow allocation from GTT instead of VRAM * Add proper CPU access flags when allocating dmabufs * Remove 'radeon' driver support from the dmabuf allocation routines * Add comand line arguments to the tests for selecting GPU unit and setting the option for allocating from GTT
v3: https://www.spinics.net/lists/linux-rdma/msg98059.html * Add parameter 'iova' to the new ibv_reg_dmabuf_mr() API * Change the way of allocating dma-buf object - use /dev/dri/renderD* instead of /dev/dri/card* and use GEM object instead of dumb buffer * Add cmake function to allow building modules with mixed cython and C source files * Add new tests that use dma-buf MRs for send/recv and rdma traffic * Skip dma-buf tests on unsupported systems * Remove some use of random values in the new tests * Add dealloc() and close() methods to the new classes * Replace string.format with f-string in python code * Fix some coding style issues: spacing, indentation, typo, comments
v2: https://www.spinics.net/lists/linux-rdma/msg97936.html * Put the kernel header updates into a separate commit * Add comments for the data structure used in python ioctl calls * Fix issues related to symbol versioning * Fix styling issues: extra spaces, unncecessary variable, typo * Fix an inproper error code usage * Put the new op into ibv_context_ops instead if verbs_context
v1: https://www.spinics.net/lists/linux-rdma/msg97865.html * Add user space API for registering dma-buf based memory regions * Update pyverbs with the new API * Add new tests
This is the user space counter-part of the kernel patch set to add dma-buf support to the RDMA subsystem.
This series consists of six patches. The first patch updates the kernel headers for dma-buf support. Patch 2 adds the new API function and updates the man pages. Patch 3 implements the new API in the mlx5 provider. Patch 4 adds new class definitions to pyverbs for the new API. Patch 5 adds a set of new tests for the new API. Patch 6 fixes bug in the utility code of the tests.
Pull request at github: https://github.com/linux-rdma/rdma-core/pull/895
Jianxin Xiong (6): Update kernel headers verbs: Support dma-buf based memory region mlx5: Support dma-buf based memory region pyverbs: Add dma-buf based MR support tests: Add tests for dma-buf based memory regions tests: Bug fix for get_access_flags()
buildlib/pyverbs_functions.cmake | 78 ++++++--- debian/libibverbs1.symbols | 2 + kernel-headers/rdma/ib_user_ioctl_cmds.h | 14 ++ libibverbs/CMakeLists.txt | 2 +- libibverbs/cmd_mr.c | 38 +++++ libibverbs/driver.h | 8 + libibverbs/dummy_ops.c | 11 ++ libibverbs/libibverbs.map.in | 6 + libibverbs/man/ibv_reg_mr.3 | 27 ++- libibverbs/verbs.c | 19 +++ libibverbs/verbs.h | 7 + providers/mlx5/mlx5.c | 2 + providers/mlx5/mlx5.h | 3 + providers/mlx5/verbs.c | 22 +++ pyverbs/CMakeLists.txt | 11 +- pyverbs/dmabuf.pxd | 15 ++ pyverbs/dmabuf.pyx | 73 ++++++++ pyverbs/dmabuf_alloc.c | 278 +++++++++++++++++++++++++++++++ pyverbs/dmabuf_alloc.h | 19 +++ pyverbs/libibverbs.pxd | 2 + pyverbs/mr.pxd | 6 + pyverbs/mr.pyx | 105 +++++++++++- tests/args_parser.py | 4 + tests/test_mr.py | 266 ++++++++++++++++++++++++++++- tests/utils.py | 30 +++- 25 files changed, 1012 insertions(+), 36 deletions(-) create mode 100644 pyverbs/dmabuf.pxd create mode 100644 pyverbs/dmabuf.pyx create mode 100644 pyverbs/dmabuf_alloc.c create mode 100644 pyverbs/dmabuf_alloc.h