• Jens Axboe's avatar
    sbitmap: optimize wakeup check · 5d2ee712
    Jens Axboe authored
    Even if we have no waiters on any of the sbitmap_queue wait states, we
    still have to loop every entry to check. We do this for every IO, so
    the cost adds up.
    
    Shift a bit of the cost to the slow path, when we actually have waiters.
    Wrap prepare_to_wait_exclusive() and finish_wait(), so we can maintain
    an internal count of how many are currently active. Then we can simply
    check this count in sbq_wake_ptr() and not have to loop if we don't
    have any sleepers.
    
    Convert the two users of sbitmap with waiting, blk-mq-tag and iSCSI.
    Reviewed-by: 's avatarOmar Sandoval <osandov@fb.com>
    Signed-off-by: 's avatarJens Axboe <axboe@kernel.dk>
    5d2ee712