mirrors/linux
2
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git synced 2026-08-30 13:23:02 -04:00
Code Issues Packages Projects Releases Wiki Activity

rcu: Add RCU_GET_STATE_NOT_TRACKED for subsystems without expedited GPs

Browse Source 
SRCU and Tasks RCU do not track expedited grace periods. When their
callback state is checked via poll_state_synchronize_rcu_full(), the
uninitialized or zeroed exp field could cause false-positive
completion detection.

This commit adds an RCU_GET_STATE_NOT_TRACKED sentinel value (0x2) that
these subsystems can place into exp to indicate that expedited GP
tracking is not applicable. The expedited sequence check in
poll_state_synchronize_rcu_full() is guarded to skip entries marked with
this sentinel.

This is needed to allow rcu_segcblist_advance() and rcu_accelerate_cbs()
to work with both normal and expedited grace periods via
get_state_synchronize_rcu_full() and poll_state_synchronize_rcu_full().

Reviewed-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit is contained in:
Puranjay Mohan
2026-06-24 06:23:47 -07:00
committed by Paul E. McKenney
parent fb5c815f12
commit 60b54875bd
2 changed files with 13 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
kernel/rcu/rcu.h
View File

@@ -46,16 +46,25 @@
* the number of pending readers that will use
* this inactive index is bounded).
*
* RCU polled GP special control value:
* RCU polled GP special control values:
*
* RCU_GET_STATE_COMPLETED : State value indicating an already-completed
* polled GP has completed. This value covers
* both the state and the counter of the
* grace-period sequence number.
*
* RCU_GET_STATE_NOT_TRACKED : State value indicating that a GP component
* is not tracked by this subsystem and should
* not be checked. Used by SRCU and RCU Tasks
* which do not track expedited GPs, to prevent
* false-positive completion when their
* gp_seq entries are checked via
* poll_state_synchronize_rcu_full().
*/
/* Low-order bit definition for polled grace-period APIs. */
/* Low-order bit definitions for polled grace-period APIs. */
#define RCU_GET_STATE_COMPLETED 0x1
#define RCU_GET_STATE_NOT_TRACKED 0x2
/* A complete grace period count */
#define RCU_SEQ_GP (RCU_SEQ_STATE_MASK + 1)

3
kernel/rcu/tree.c
View File

@@ -3629,7 +3629,8 @@ bool poll_state_synchronize_rcu_full(struct rcu_gp_seq *gsp)
if (gsp->norm == RCU_GET_STATE_COMPLETED ||
rcu_seq_done_exact(&rnp->gp_seq, gsp->norm) ||
gsp->exp == RCU_GET_STATE_COMPLETED ||
rcu_seq_done_exact(&rcu_state.expedited_sequence, gsp->exp)) {
(gsp->exp != RCU_GET_STATE_NOT_TRACKED &&
rcu_seq_done_exact(&rcu_state.expedited_sequence, gsp->exp))) {
smp_mb(); /* Ensure GP ends before subsequent accesses. */
return true;
}