diff options
| author | Waiman Long <longman@redhat.com> | 2020-11-20 23:14:14 -0500 |
|---|---|---|
| committer | Peter Zijlstra <peterz@infradead.org> | 2020-12-09 17:08:48 +0100 |
| commit | 1a728dff855a318bb58bcc1259b1826a7ad9f0bd (patch) | |
| tree | fc773e33ed49257132e2123517323574c41b8e48 /kernel/locking/lock_events_list.h | |
| parent | 2f06f702925b512a95b95dca3855549c047eef58 (diff) | |
| download | linux-1a728dff855a318bb58bcc1259b1826a7ad9f0bd.tar.bz2 | |
locking/rwsem: Enable reader optimistic lock stealing
If the optimistic spinning queue is empty and the rwsem does not have
the handoff or write-lock bits set, it is actually not necessary to
call rwsem_optimistic_spin() to spin on it. Instead, it can steal the
lock directly as its reader bias is in the count already. If it is
the first reader in this state, it will try to wake up other readers
in the wait queue.
With this patch applied, the following were the lock event counts
after rebooting a 2-socket system and a "make -j96" kernel rebuild.
rwsem_opt_rlock=4437
rwsem_rlock=29
rwsem_rlock_steal=19
So lock stealing represents about 0.4% of all the read locks acquired
in the slow path.
Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Davidlohr Bueso <dbueso@suse.de>
Link: https://lkml.kernel.org/r/20201121041416.12285-4-longman@redhat.com
Diffstat (limited to 'kernel/locking/lock_events_list.h')
| -rw-r--r-- | kernel/locking/lock_events_list.h | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/kernel/locking/lock_events_list.h b/kernel/locking/lock_events_list.h index 239039d0ce21..270a0d351932 100644 --- a/kernel/locking/lock_events_list.h +++ b/kernel/locking/lock_events_list.h @@ -63,6 +63,7 @@ LOCK_EVENT(rwsem_opt_nospin) /* # of disabled optspins */ LOCK_EVENT(rwsem_opt_norspin) /* # of disabled reader-only optspins */ LOCK_EVENT(rwsem_opt_rlock2) /* # of opt-acquired 2ndary read locks */ LOCK_EVENT(rwsem_rlock) /* # of read locks acquired */ +LOCK_EVENT(rwsem_rlock_steal) /* # of read locks by lock stealing */ LOCK_EVENT(rwsem_rlock_fast) /* # of fast read locks acquired */ LOCK_EVENT(rwsem_rlock_fail) /* # of failed read lock acquisitions */ LOCK_EVENT(rwsem_rlock_handoff) /* # of read lock handoffs */ |