I want to access GPU VRAM via PCIe P2P access, like DirectGMA but with mainline drivers. I can acquire a dmabuf from the GPU and pass it to my PCIe framegrabber. But I don't see a way to get the PCIe bus address of the video memory which I need for the P2P transfer. Is there already any infrastructure in place for doing this? If not, what would be the right way to implement this? Add another callback to the dmabuf struct?
Cheers Dirk
This is work in progress.
I published patches to enable DMA_buf P2P a few months ago, but now I'm waiting for the PCI subsystem to pick up core support for this.
I can prepare you a branch based on current upstream kernel next week if you want to test this.
Regards, Christian.
Am 29.09.2018 09:01 schrieb Dirk Eibach dirk.eibach@googlemail.com: I want to access GPU VRAM via PCIe P2P access, like DirectGMA but with mainline drivers. I can acquire a dmabuf from the GPU and pass it to my PCIe framegrabber. But I don't see a way to get the PCIe bus address of the video memory which I need for the P2P transfer. Is there already any infrastructure in place for doing this? If not, what would be the right way to implement this? Add another callback to the dmabuf struct?
Cheers Dirk
This is work in progress.
I published patches to enable DMA_buf P2P a few months ago, but now I'm waiting for the PCI subsystem to pick up core support for this.
Great news! Can you give me a link to this series so I can already have a look?
I can prepare you a branch based on current upstream kernel next week if you want to test this.
That would be awesome, thanks.
Cheers Dirk
Hello Dirk,
I think Christian is talking about this branch:
https://cgit.freedesktop.org/~deathsimple/linux/log/?h=p2p
His 'home' is, here: https://cgit.freedesktop.org/~deathsimple/linux/
Happy hacking! ;-)
Dieter
Am 29.09.2018 10:17, schrieb Dirk Eibach:
This is work in progress.
I published patches to enable DMA_buf P2P a few months ago, but now I'm waiting for the PCI subsystem to pick up core support for this.
Great news! Can you give me a link to this series so I can already have a look?
I can prepare you a branch based on current upstream kernel next week if you want to test this.
That would be awesome, thanks.
Cheers Dirk _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Hi Christian,
I took the repository Dieter suggested and gave it a spin. I am not sure, if I got the API right.
In userspace I create a bo and get a fd. bo = gbm_bo_create(gbm, w, h, GBM_FORMAT_ARGB8888, GBM_BO_USE_LINEAR); dmabuf_fd = gbm_bo_get_fd(bo);
Then I pass the fd to the grabber driver. There I get a sgt, that I want to pass to the DMA-machine. struct dma_buf_attach_info dma_buf_attach_info = { .dev = &engine->xdev->pdev->dev, .peer2peer = true, }; dma_buf = dma_buf_get(xdma_dma_buf_ioctl.dma_buf_fd); dma_buf_attach_info.dmabuf = dma_buf; dma_buf_attachment = dma_buf_attach(&dma_buf_attach_info); sgt = dma_buf_map_attachment(dma_buf_attachment, DMA_TO_DEVICE);
But the addresses in the sgt do not look like PCIe addresses at at all.
Any ideas?
Cheers Dirk Am Sa., 29. Sep. 2018 um 11:53 Uhr schrieb Dieter Nützel Dieter@nuetzel-hh.de:
Hello Dirk,
I think Christian is talking about this branch:
https://cgit.freedesktop.org/~deathsimple/linux/log/?h=p2p
His 'home' is, here: https://cgit.freedesktop.org/~deathsimple/linux/
Happy hacking! ;-)
Dieter
Am 29.09.2018 10:17, schrieb Dirk Eibach:
This is work in progress.
I published patches to enable DMA_buf P2P a few months ago, but now I'm waiting for the PCI subsystem to pick up core support for this.
Great news! Can you give me a link to this series so I can already have a look?
I can prepare you a branch based on current upstream kernel next week if you want to test this.
That would be awesome, thanks.
Cheers Dirk _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Hi Dirk,
Any ideas?
Could be anything. Most likely cause is that some function things the two device can't communicate directly with each other.
Please provide a full dmesg and "lspci -t -nn -v" as attachment and note which devices should communicate with which.
I'm currently on vacation so it could take a while till I have time to look deeper into this.
Christian.
Am 01.10.2018 um 15:07 schrieb Dirk Eibach:
Hi Christian,
I took the repository Dieter suggested and gave it a spin. I am not sure, if I got the API right.
In userspace I create a bo and get a fd. bo = gbm_bo_create(gbm, w, h, GBM_FORMAT_ARGB8888, GBM_BO_USE_LINEAR); dmabuf_fd = gbm_bo_get_fd(bo);
Then I pass the fd to the grabber driver. There I get a sgt, that I want to pass to the DMA-machine. struct dma_buf_attach_info dma_buf_attach_info = { .dev = &engine->xdev->pdev->dev, .peer2peer = true, }; dma_buf = dma_buf_get(xdma_dma_buf_ioctl.dma_buf_fd); dma_buf_attach_info.dmabuf = dma_buf; dma_buf_attachment = dma_buf_attach(&dma_buf_attach_info); sgt = dma_buf_map_attachment(dma_buf_attachment, DMA_TO_DEVICE);
But the addresses in the sgt do not look like PCIe addresses at at all.
Any ideas?
Cheers Dirk Am Sa., 29. Sep. 2018 um 11:53 Uhr schrieb Dieter Nützel Dieter@nuetzel-hh.de:
Hello Dirk,
I think Christian is talking about this branch:
https://cgit.freedesktop.org/~deathsimple/linux/log/?h=p2p
His 'home' is, here: https://cgit.freedesktop.org/~deathsimple/linux/
Happy hacking! ;-)
Dieter
Am 29.09.2018 10:17, schrieb Dirk Eibach:
This is work in progress.
I published patches to enable DMA_buf P2P a few months ago, but now I'm waiting for the PCI subsystem to pick up core support for this.
Great news! Can you give me a link to this series so I can already have a look?
I can prepare you a branch based on current upstream kernel next week if you want to test this.
That would be awesome, thanks.
Cheers Dirk _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Hi Christian,
Could I simply force it by overwriting peer2peer after the attach call?
Cheers Dirk
Hi Dirk,
Any ideas?
Could be anything. Most likely cause is that some function things the two device can't communicate directly with each other.
Please provide a full dmesg and "lspci -t -nn -v" as attachment and note which devices should communicate with which.
I'm currently on vacation so it could take a while till I have time to look deeper into this.
Christian.
Am 01.10.2018 um 15:07 schrieb Dirk Eibach:
Hi Christian,
I took the repository Dieter suggested and gave it a spin. I am not sure, if I got the API right.
In userspace I create a bo and get a fd. bo = gbm_bo_create(gbm, w, h, GBM_FORMAT_ARGB8888, GBM_BO_USE_LINEAR); dmabuf_fd = gbm_bo_get_fd(bo);
Then I pass the fd to the grabber driver. There I get a sgt, that I want to pass to the DMA-machine. struct dma_buf_attach_info dma_buf_attach_info = { .dev = &engine->xdev->pdev->dev, .peer2peer = true, }; dma_buf = dma_buf_get(xdma_dma_buf_ioctl.dma_buf_fd); dma_buf_attach_info.dmabuf = dma_buf; dma_buf_attachment = dma_buf_attach(&dma_buf_attach_info); sgt = dma_buf_map_attachment(dma_buf_attachment, DMA_TO_DEVICE);
But the addresses in the sgt do not look like PCIe addresses at at all.
Any ideas?
Cheers Dirk Am Sa., 29. Sep. 2018 um 11:53 Uhr schrieb Dieter Nützel <
Dieter@nuetzel-hh.de>:
Hello Dirk,
I think Christian is talking about this branch:
https://cgit.freedesktop.org/~deathsimple/linux/log/?h=p2p
His 'home' is, here: https://cgit.freedesktop.org/~deathsimple/linux/
Happy hacking! ;-)
Dieter
Am 29.09.2018 10:17, schrieb Dirk Eibach:
This is work in progress.
I published patches to enable DMA_buf P2P a few months ago, but now I'm waiting for the PCI subsystem to pick up core support for this.
Great news! Can you give me a link to this series so I can already have a look?
I can prepare you a branch based on current upstream kernel next week if you want to test this.
That would be awesome, thanks.
Cheers Dirk _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
On Sat, Sep 29, 2018 at 9:27 AM Koenig, Christian Christian.Koenig@amd.com wrote:
This is work in progress.
I published patches to enable DMA_buf P2P a few months ago, but now I'm waiting for the PCI subsystem to pick up core support for this.
I can prepare you a branch based on current upstream kernel next week if you want to test this.
Thread hijack, since I just read the lwn article on p2p for storage folks:
https://lwn.net/Articles/767281/
Totally doesn't match what I remember of your dma-buf p2p work. Do we need to rework, or is this some kind of higher-level interface on top of the raw p2p pci stuff? Or do I misremember.
Thanks, Daniel
Regards, Christian.
Am 29.09.2018 09:01 schrieb Dirk Eibach dirk.eibach@googlemail.com: I want to access GPU VRAM via PCIe P2P access, like DirectGMA but with mainline drivers. I can acquire a dmabuf from the GPU and pass it to my PCIe framegrabber. But I don't see a way to get the PCIe bus address of the video memory which I need for the P2P transfer. Is there already any infrastructure in place for doing this? If not, what would be the right way to implement this? Add another callback to the dmabuf struct?
Cheers Dirk _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Am 03.10.2018 um 09:14 schrieb Daniel Vetter:
On Sat, Sep 29, 2018 at 9:27 AM Koenig, Christian Christian.Koenig@amd.com wrote:
This is work in progress.
I published patches to enable DMA_buf P2P a few months ago, but now I'm waiting for the PCI subsystem to pick up core support for this.
I can prepare you a branch based on current upstream kernel next week if you want to test this.
Thread hijack, since I just read the lwn article on p2p for storage folks:
https://lwn.net/Articles/767281/
Totally doesn't match what I remember of your dma-buf p2p work. Do we need to rework, or is this some kind of higher-level interface on top of the raw p2p pci stuff? Or do I misremember.
The P2P from the storage folks are some mix of raw p2p capability detection for PCI and higher level interface.
I'm actually waiting for that stuff to land to extract the p2p capability detection and use it for this DMA-buf stuff as well.
Alternative would be to convince Logan to extract the capability detection in his patch set as well, but so far he wasn't very keen about that.
Christian.
Thanks, Daniel
Regards, Christian.
Am 29.09.2018 09:01 schrieb Dirk Eibach dirk.eibach@googlemail.com: I want to access GPU VRAM via PCIe P2P access, like DirectGMA but with mainline drivers. I can acquire a dmabuf from the GPU and pass it to my PCIe framegrabber. But I don't see a way to get the PCIe bus address of the video memory which I need for the P2P transfer. Is there already any infrastructure in place for doing this? If not, what would be the right way to implement this? Add another callback to the dmabuf struct?
Cheers Dirk _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
On Wed, Oct 03, 2018 at 08:54:44AM +0000, Koenig, Christian wrote:
Am 03.10.2018 um 09:14 schrieb Daniel Vetter:
On Sat, Sep 29, 2018 at 9:27 AM Koenig, Christian Christian.Koenig@amd.com wrote:
This is work in progress.
I published patches to enable DMA_buf P2P a few months ago, but now I'm waiting for the PCI subsystem to pick up core support for this.
I can prepare you a branch based on current upstream kernel next week if you want to test this.
Thread hijack, since I just read the lwn article on p2p for storage folks:
https://lwn.net/Articles/767281/
Totally doesn't match what I remember of your dma-buf p2p work. Do we need to rework, or is this some kind of higher-level interface on top of the raw p2p pci stuff? Or do I misremember.
The P2P from the storage folks are some mix of raw p2p capability detection for PCI and higher level interface.
I'm actually waiting for that stuff to land to extract the p2p capability detection and use it for this DMA-buf stuff as well.
Alternative would be to convince Logan to extract the capability detection in his patch set as well, but so far he wasn't very keen about that.
So no plan to make dma-buf some kind of orchestrator and use the p2p map_sg stuff? That seems to be the new/additional bits I haven't seen yet, and I'm wondered how we should fit dma-buf into that picture. -Daniel
Christian.
Thanks, Daniel
Regards, Christian.
Am 29.09.2018 09:01 schrieb Dirk Eibach dirk.eibach@googlemail.com: I want to access GPU VRAM via PCIe P2P access, like DirectGMA but with mainline drivers. I can acquire a dmabuf from the GPU and pass it to my PCIe framegrabber. But I don't see a way to get the PCIe bus address of the video memory which I need for the P2P transfer. Is there already any infrastructure in place for doing this? If not, what would be the right way to implement this? Add another callback to the dmabuf struct?
Cheers Dirk _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Am 03.10.2018 um 11:06 schrieb Daniel Vetter:
On Wed, Oct 03, 2018 at 08:54:44AM +0000, Koenig, Christian wrote:
Am 03.10.2018 um 09:14 schrieb Daniel Vetter:
On Sat, Sep 29, 2018 at 9:27 AM Koenig, Christian Christian.Koenig@amd.com wrote:
This is work in progress.
I published patches to enable DMA_buf P2P a few months ago, but now I'm waiting for the PCI subsystem to pick up core support for this.
I can prepare you a branch based on current upstream kernel next week if you want to test this.
Thread hijack, since I just read the lwn article on p2p for storage folks:
https://lwn.net/Articles/767281/
Totally doesn't match what I remember of your dma-buf p2p work. Do we need to rework, or is this some kind of higher-level interface on top of the raw p2p pci stuff? Or do I misremember.
The P2P from the storage folks are some mix of raw p2p capability detection for PCI and higher level interface.
I'm actually waiting for that stuff to land to extract the p2p capability detection and use it for this DMA-buf stuff as well.
Alternative would be to convince Logan to extract the capability detection in his patch set as well, but so far he wasn't very keen about that.
So no plan to make dma-buf some kind of orchestrator and use the p2p map_sg stuff?
Not sure about that yet. Didn't had time to take a closer look.
Christian.
That seems to be the new/additional bits I haven't seen yet, and I'm wondered how we should fit dma-buf into that picture. -Daniel
Christian.
Thanks, Daniel
Regards, Christian.
Am 29.09.2018 09:01 schrieb Dirk Eibach dirk.eibach@googlemail.com: I want to access GPU VRAM via PCIe P2P access, like DirectGMA but with mainline drivers. I can acquire a dmabuf from the GPU and pass it to my PCIe framegrabber. But I don't see a way to get the PCIe bus address of the video memory which I need for the P2P transfer. Is there already any infrastructure in place for doing this? If not, what would be the right way to implement this? Add another callback to the dmabuf struct?
Cheers Dirk _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
dri-devel@lists.freedesktop.org
-
Christian König -
Christian König -
Daniel Vetter -
Dieter Nützel -
Dirk Eibach -
Koenig, Christian