Hello,
My static analysis tool reports a possible ABBA deadlock in the drm driver in Linux 5.10:
drm_gem_prime_fd_to_handle() mutex_lock(&dev->object_name_lock); --> Line 313 (Lock A) drm_gem_handle_delete() drm_gem_object_release_handle() drm_gem_remove_prime_handles() mutex_lock(&filp->prime.lock); --> Line 16 (Lock B)
drm_gem_prime_handle_to_fd() mutex_lock(&file_priv->prime.lock); --> Line 433 (Lock B) mutex_lock(&dev->object_name_lock); --> Line 466 (Lock A)
When drm_gem_prime_fd_to_handle() and drm_gem_prime_handle_to_fd() are concurrently executed, the deadlock can occur.
I am not quite sure whether this possible deadlock is real and how to fix it if it is real. Any feedback would be appreciated, thanks :)
Reported-by: TOTE Robot oslab@tsinghua.edu.cn
Best wishes, Jia-Ju Bai