mirrors/linux
2
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git synced 2026-07-22 03:27:30 -04:00
Code Issues Packages Projects Releases Wiki Activity

block: try slab allocation in bio_alloc_bioset() before mempool

Browse Source 
When the per-CPU bio cache is enabled but empty, bio_alloc_percpu_cache()
returns NULL and bio_alloc_bioset() falls straight through to the mempool
fallback:

    if (unlikely(!bio)) {
        if (!(saved_gfp & __GFP_DIRECT_RECLAIM))
            return NULL;
        ...
    }

For non-sleeping allocations (no __GFP_DIRECT_RECLAIM) this returns NULL
without ever attempting a slab allocation, even when there is plenty of
free memory.

Commit b520c4eef8 ("block: split bio_alloc_bioset more clearly into a
fast and slowpath") introduced this. Before it, a percpu cache miss fell
through to mempool_alloc(), which attempted the underlying slab allocation
first and only failed when that slab allocation failed. The restructuring
dropped the slab attempt that non-sleeping callers of a cache-enabled
bioset (such as the default fs_bio_set used by bio_alloc()) relied on.

Try a slab allocation with optimistic GFP_ flags before falling back to
the mempool whenever the bio is still NULL, so both the cache-empty and
non-cache paths share the same slab attempt. This restores the previous
behavior for non-sleeping allocations.

Fixes: b520c4eef8 ("block: split bio_alloc_bioset more clearly into a fast and slowpath")
Suggested-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260709020145.4011533-1-joseph.qi@linux.alibaba.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
This commit is contained in:
Joseph Qi
2026-07-09 10:01:45 +08:00
committed by Jens Axboe
parent f94de43264
commit 447cfed6d7
1 changed files with 8 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
block/bio.c
View File

@@ -555,6 +555,14 @@ struct bio *bio_alloc_bioset(struct block_device *bdev, unsigned short nr_vecs,
bio = bio_alloc_percpu_cache(bs);
} else {
opf &= ~REQ_ALLOC_CACHE;
}
/*
* For a bioset without a percpu cache, or when the percpu cache was
* empty, try a slab allocation with optimistic GFP_ flags before
* falling back to the mempool.
*/
if (!bio) {
p = kmem_cache_alloc(bs->bio_slab, gfp);
if (p)
bio = p + bs->front_pad;