Since per-process-bo feature is introduced, old lru isn't working for it. old lru order is depending on BO list order, which will be updated by bo list after every command submission. But for per-process-bo, which aren't in bo list, so it have no chance to refresh its order in lru. Which also will resulit in unstable performance for application. per-process-bo means they will be used by every this process, and validated automatically. Their order should be fixed.
Above reason, we introduce a per-process-lru instead of old lru. like struct definition: +struct ttm_process { + struct list_head process_list; + struct rb_root fixed_lru[TTM_NUM_MEM_TYPES][TTM_MAX_BO_PRIORITY]; + struct list_head dynamic_lru[TTM_NUM_MEM_TYPES][TTM_MAX_BO_PRIORITY]; + struct reservation_object *resv; + atomic64_t bo_index; +}; process_list is a list node to add bdev->proces_list. fixed_lru is to store per-process-bo. dyanmic_lru is to store non per-process-bo. resvation is this process pd root bo resvation. bo_index is to counter bo index, every bo beloning to this process will get an index.
Last, the patch set is working in progress, so I don't reorganize and clean up them yet. Sending them to community is to see if any other concern for this new machanism.
Tested result: the solution solves my issue F1 game performance isn't stable, and not introduce extra cpu overhead, the Talos game can prove it.
TODO: 1. patch organization. 2. some corner case need to handle(fini, bo destroy, force_clean) 3. clean up. 4. more palces can be improved if this machanism is accepted, like per-vm-lru don't need move at all if not destroy, maybe add a flag for evictalbe. 5. vm->evicted list in amdgpu can also be replaced by RB tree, which can make sure validation order.
Thanks, David Zhou
Chunming Zhou (11): drm/ttm: add ttm process struct drm/amdgpu: use ttm process in amdgpu vm drm/amdgpu: add kernel process drm/amdgpu: pass process to tbo drm/ttm: add per process lru drm/amdgpu: pass ttm process to buffer object drm/ttm: use RB tree instead of link list drm/ttm: add bo index drm/amdgpu: counter for every bo creation drm/ttm: some good fixes for per-vm-lru drm/ttm: bulk move per vm bo
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 5 +- drivers/gpu/drm/amd/amdgpu/amdgpu_fb.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 24 ++-- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 169 ++++++++++++++++++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 6 + drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 3 + drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 2 + drivers/gpu/drm/ttm/ttm_bo.c | 134 ++++++++++++++++++++--- include/drm/ttm/ttm_bo_api.h | 4 +- include/drm/ttm/ttm_bo_driver.h | 14 ++- 11 files changed, 336 insertions(+), 29 deletions(-)