Skip to content

Commit

Permalink
slub: fix unreclaimable slab stat for bulk free
Browse files Browse the repository at this point in the history
SLUB uses page allocator for higher order allocations and update
unreclaimable slab stat for such allocations.  At the moment, the bulk
free for SLUB does not share code with normal free code path for these
type of allocations and have missed the stat update.  So, fix the stat
update by common code.  The user visible impact of the bug is the
potential of inconsistent unreclaimable slab stat visible through
meminfo and vmstat.

Link: https://lkml.kernel.org/r/[email protected]
Fixes: 6a486c0 ("mm, sl[ou]b: improve memory accounting")
Signed-off-by: Shakeel Butt <[email protected]>
Acked-by: Michal Hocko <[email protected]>
Acked-by: Roman Gushchin <[email protected]>
Reviewed-by: Muchun Song <[email protected]>
Cc: Christoph Lameter <[email protected]>
Cc: Pekka Enberg <[email protected]>
Cc: David Rientjes <[email protected]>
Cc: Joonsoo Kim <[email protected]>
Cc: Vlastimil Babka <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
  • Loading branch information
shakeelb authored and torvalds committed Jul 30, 2021
1 parent b5916c0 commit f227f0f
Showing 1 changed file with 12 additions and 10 deletions.
22 changes: 12 additions & 10 deletions mm/slub.c
Original file line number Diff line number Diff line change
Expand Up @@ -3236,6 +3236,16 @@ struct detached_freelist {
struct kmem_cache *s;
};

static inline void free_nonslab_page(struct page *page)
{
unsigned int order = compound_order(page);

VM_BUG_ON_PAGE(!PageCompound(page), page);
kfree_hook(page_address(page));
mod_lruvec_page_state(page, NR_SLAB_UNRECLAIMABLE_B, -(PAGE_SIZE << order));
__free_pages(page, order);
}

/*
* This function progressively scans the array with free objects (with
* a limited look ahead) and extract objects belonging to the same
Expand Down Expand Up @@ -3272,9 +3282,7 @@ int build_detached_freelist(struct kmem_cache *s, size_t size,
if (!s) {
/* Handle kalloc'ed objects */
if (unlikely(!PageSlab(page))) {
BUG_ON(!PageCompound(page));
kfree_hook(object);
__free_pages(page, compound_order(page));
free_nonslab_page(page);
p[size] = NULL; /* mark object processed */
return size;
}
Expand Down Expand Up @@ -4250,13 +4258,7 @@ void kfree(const void *x)

page = virt_to_head_page(x);
if (unlikely(!PageSlab(page))) {
unsigned int order = compound_order(page);

BUG_ON(!PageCompound(page));
kfree_hook(object);
mod_lruvec_page_state(page, NR_SLAB_UNRECLAIMABLE_B,
-(PAGE_SIZE << order));
__free_pages(page, order);
free_nonslab_page(page);
return;
}
slab_free(page->slab_cache, page, object, NULL, 1, _RET_IP_);
Expand Down

0 comments on commit f227f0f

Please sign in to comment.