On Thu, Jul 02, 2020 at 08:15:40PM +0200, Daniel Vetter wrote:
- rdma driver worker gets busy to restart rx:
thanks to ww_mutex deadlock avoidance this is possible
- lock all dma-buf that are currently in use (dma_resv_lock).
Why all? Why not just lock the one that was invalidated to restore the mappings? That is some artifact of the GPU approach?
No, but you must make sure that mapping one doesn't invalidate others you need.
Otherwise you can end up in a nice live lock :)
Also if you don't have pagefaults, but have to track busy memory at a context level, you do need to grab all locks of all buffers you need, or you'd race. There's nothing stopping a concurrent ->notify_move on some other buffer you'll need otherwise, and if you try to be clever and roll you're own locking, you'll anger lockdep - you're own lock will have to be on both sides of ww_mutex or it wont work, and that deadlocks.
So you are worried about atomically building some multi buffer transaction? I don't think this applies to RDMA which isn't going to be transcational here..
And why is this done with work queues and locking instead of a callback saying the buffer is valid again?
You can do this as well, but a work queue is usually easier to handle than a notification in an interrupt context of a foreign driver.
Yeah you can just install a dma_fence callback but
- that's hardirq context
- if you don't have per-page hw faults you need the multi-lock ww_mutex dance anyway to avoid races.
It is still best to avoid the per-page faults and preload the new mapping once it is ready.
Jason