From: Nicolai Hähnle Nicolai.Haehnle@amd.com
When ww_mutex_set_context_slowpath runs, we are in one of two situations:
1. The current task was woken up by ww_mutex_unlock.
2. The current task is racing with ww_mutex_unlock: We entered the slow path while lock->base.count <= 0, but skipped the wait in __mutex_lock_common.
In both cases, all tasks that are currently contending for the lock are either racing as in point 2 and blocked on lock->wait_lock, or they have been or will be woken up by ww_mutex_unlock.
Either way, they will check their stamp against ours without having to be woken up again.
This improvement is possible only after the changed behavior of ww_mutex_unlock from the previous patch ("locking/ww_mutex: Fix a deadlock affecting ww_mutexes").
Cc: Peter Zijlstra peterz@infradead.org Cc: Ingo Molnar mingo@redhat.com Cc: Chris Wilson chris@chris-wilson.co.uk Cc: Maarten Lankhorst maarten.lankhorst@canonical.com Cc: dri-devel@lists.freedesktop.org Signed-off-by: Nicolai Hähnle nicolai.haehnle@amd.com --- kernel/locking/mutex.c | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-)
diff --git a/kernel/locking/mutex.c b/kernel/locking/mutex.c index 7fbf9b4..7c09376 100644 --- a/kernel/locking/mutex.c +++ b/kernel/locking/mutex.c @@ -192,8 +192,10 @@ ww_mutex_set_context_fastpath(struct ww_mutex *lock, }
/* - * After acquiring lock in the slowpath set ctx and wake up any - * waiters so they can recheck. + * After acquiring lock in the slowpath set ctx. + * + * Unlike the fast path, other waiters are already woken up by ww_mutex_unlock, + * so we don't have to do it again here. * * Callers must hold the mutex wait_lock. */ @@ -201,19 +203,8 @@ static __always_inline void ww_mutex_set_context_slowpath(struct ww_mutex *lock, struct ww_acquire_ctx *ctx) { - struct mutex_waiter *cur; - ww_mutex_lock_acquired(lock, ctx); lock->ctx = ctx; - - /* - * Give any possible sleeping processes the chance to wake up, - * so they can recheck if they have to back off. - */ - list_for_each_entry(cur, &lock->base.wait_list, list) { - debug_mutex_wake_waiter(&lock->base, cur); - wake_up_process(cur->task); - } }
#ifdef CONFIG_MUTEX_SPIN_ON_OWNER