Commit a2e16731 authored by Tim Chen's avatar Tim Chen Committed by Linus Torvalds

mm/swap_slots.c: fix race conditions in swap_slots cache init

Memory allocations can happen before the swap_slots cache initialization
is completed during cpu bring up.  If we are low on memory, we could
call get_swap_page() and access swap_slots_cache before it is fully

Add a check in get_swap_page() for initialized swap_slots_cache to
prevent this condition.  Similar check already exists in free_swap_slot.
Also annotate the checks to indicate the likely condition.

We also added a memory barrier to make sure that the locks
initialization are done before the assignment of cache->slots and
cache->slots_ret pointers.  This ensures the assumption that it is safe
to acquire the slots cache locks and use the slots cache when the
corresponding cache->slots or cache->slots_ret pointers are non null.

[ tidy up comment]
[ fix spello in comment]
Link: default avatarTim Chen <>
Reported-by: default avatarWenwei Tao <>
Acked-by: default avatarYing Huang <>
Cc: Minchan Kim <>
Cc: Rik van Riel <>
Cc: Andrea Arcangeli <>
Cc: Johannes Weiner <>
Cc: Michal Hocko <>
Cc: Hillf Danton <>
Cc: Hugh Dickins <>
Signed-off-by: default avatarAndrew Morton <>
Signed-off-by: default avatarLinus Torvalds <>
parent 3a50d14d
......@@ -149,6 +149,13 @@ static int alloc_swap_slot_cache(unsigned int cpu)
cache->nr = 0;
cache->cur = 0;
cache->n_ret = 0;
* We initialized alloc_lock and free_lock earlier. We use
* !cache->slots or !cache->slots_ret to know if it is safe to acquire
* the corresponding lock and use the cache. Memory barrier below
* ensures the assumption.
cache->slots = slots;
slots = NULL;
cache->slots_ret = slots_ret;
......@@ -275,7 +282,7 @@ int free_swap_slot(swp_entry_t entry)
struct swap_slots_cache *cache;
cache = raw_cpu_ptr(&swp_slots);
if (use_swap_slot_cache && cache->slots_ret) {
if (likely(use_swap_slot_cache && cache->slots_ret)) {
/* Swap slots cache may be deactivated before acquiring lock */
if (!use_swap_slot_cache || !cache->slots_ret) {
......@@ -326,7 +333,7 @@ swp_entry_t get_swap_page(struct page *page)
cache = raw_cpu_ptr(&swp_slots);
if (check_cache_active()) {
if (likely(check_cache_active() && cache->slots)) {
if (cache->slots) {
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment