Skip to content
  • Paul E. McKenney's avatar
    rcu: Speed up expedited GPs when interrupting RCU reader · 05f41571
    Paul E. McKenney authored
    
    
    In PREEMPT kernels, an expedited grace period might send an IPI to a
    CPU that is executing an RCU read-side critical section.  In that case,
    it would be nice if the rcu_read_unlock() directly interacted with the
    RCU core code to immediately report the quiescent state.  And this does
    happen in the case where the reader has been preempted.  But it would
    also be a nice performance optimization if immediate reporting also
    happened in the preemption-free case.
    
    This commit therefore adds an ->exp_hint field to the task_struct structure's
    ->rcu_read_unlock_special field.  The IPI handler sets this hint when
    it has interrupted an RCU read-side critical section, and this causes
    the outermost rcu_read_unlock() call to invoke rcu_read_unlock_special(),
    which, if preemption is enabled, reports the quiescent state immediately.
    If preemption is disabled, then the report is required to be deferred
    until preemption (or bottom halves or interrupts or whatever) is re-enabled.
    
    Because this is a hint, it does nothing for more complicated cases.  For
    example, if the IPI interrupts an RCU reader, but interrupts are disabled
    across the rcu_read_unlock(), but another rcu_read_lock() is executed
    before interrupts are re-enabled, the hint will already have been cleared.
    If you do crazy things like this, reporting will be deferred until some
    later RCU_SOFTIRQ handler, context switch, cond_resched(), or similar.
    
    Reported-by: default avatarJoel Fernandes <joel@joelfernandes.org>
    Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.ibm.com>
    Acked-by: default avatarJoel Fernandes (Google) <joel@joelfernandes.org>
    05f41571