Jens Axboe de7341ffe4 io_uring: switch normal task_work to a mpscq ...

Like the local task_work list, the normal (tctx) task_work list is an
llist, and hence needs the O(n) llist_reverse_order() pass before
running entries in queue order. On top of that, capped runs - sqpoll
processing IORING_TW_CAP_ENTRIES_VALUE entries at a time - need the
claimed-but-unprocessed leftovers carried in a separate retry_list,
as they can't be pushed back to the shared list.

Switch tctx->task_list to a mpscq, like what was done for the
DEFER_TASKRUN paths as well.

Signed-off-by: Jens Axboe <axboe@kernel.dk>

2026-06-13 06:27:11 -06:00

..

advise.c

io_uring: finish IOU_OK -> IOU_COMPLETE transition

2025-05-21 08:41:16 -06:00

advise.h

…

alloc_cache.c

io_uring: add alloc_cache.c

2025-01-28 15:10:40 -07:00

alloc_cache.h

io_uring/rsrc: use kvfree() for the imu cache

2026-04-21 12:19:01 -06:00

bpf_filter.c

Merge branch 'io_uring-7.0' into for-7.1/io_uring

2026-03-14 08:57:15 -06:00

bpf_filter.h

io_uring: allow registration of per-task restrictions

2026-02-06 07:29:19 -07:00

bpf-ops.c

io_uring/bpf-ops: restrict ctx access to BPF

2026-06-02 11:20:21 -06:00

bpf-ops.h

io_uring/bpf-ops: restrict ctx access to BPF

2026-06-02 11:20:21 -06:00

cancel.c

io_uring/cancel: validate opcode for IORING_ASYNC_CANCEL_OP

2026-04-01 10:21:13 -06:00

cancel.h

io_uring/cancel: de-unionize file and user_data in struct io_cancel_data

2026-02-16 14:16:27 -07:00

cmd_net.c

io_uring/cmd_net: split ioctl code out of io_uring_cmd_sock()

2026-03-09 07:21:53 -06:00

epoll.c

io_uring: finish IOU_OK -> IOU_COMPLETE transition

2025-05-21 08:41:16 -06:00

epoll.h

io_uring/epoll: add support for IORING_OP_EPOLL_WAIT

2025-02-20 07:59:56 -07:00

eventfd.c

io_uring/eventfd: reset deferred signal state

2026-05-03 23:21:40 -06:00

eventfd.h

io_uring: Add SPDX id lines to remaining source files

2026-02-04 07:23:45 -07:00

fdinfo.c

io_uring/rsrc: bump struct io_mapped_ubuf length field to size_t

2026-05-14 08:12:23 -06:00

fdinfo.h

…

filetable.c

io_uring/filetable: clamp alloc_hint to the configured alloc range

2026-02-11 15:20:44 -07:00

filetable.h

io_uring: Trim out unused includes

2026-01-05 17:08:20 -07:00

fs.c

non-consuming variants of do_{unlinkat,rmdir}()

2026-01-16 12:51:50 -05:00

fs.h

…

futex.c

io_uring/futex: ensure partial wakes are appropriately dequeued

2026-04-21 12:19:06 -06:00

futex.h

io_uring: move cancelations to be io_uring_task based

2024-11-06 13:55:38 -07:00

io_uring.c

io_uring: switch local task_work to a mpscq

2026-06-13 06:27:06 -06:00

io_uring.h

io_uring/zcrx: notify user when out of buffers

2026-05-26 10:42:01 -06:00

io-wq.c

io_uring/io-wq: re-check IO_WQ_BIT_EXIT for each linked work item

2026-05-28 08:06:22 -06:00

io-wq.h

io_uring: Add SPDX id lines to remaining source files

2026-02-04 07:23:45 -07:00

kbuf.c

io_uring/kbuf: validate ring provided buffer addresses with access_ok()

2026-06-08 11:03:16 -06:00

kbuf.h

io_uring/kbuf: support min length left for incremental buffers

2026-04-28 16:08:56 -06:00

Kconfig

io_uring/bpf-ops: implement loop_step with BPF struct_ops

2026-03-16 16:15:00 -06:00

loop.c

io_uring/bpf-ops: restrict ctx access to BPF

2026-06-02 11:20:21 -06:00

loop.h

io_uring/bpf-ops: restrict ctx access to BPF

2026-06-02 11:20:21 -06:00

Makefile

io_uring/bpf-ops: implement loop_step with BPF struct_ops

2026-03-16 16:15:00 -06:00

memmap.c

io_uring: take page references for NOMMU pbuf_ring mmaps

2026-04-21 20:14:39 -06:00

memmap.h

io_uring: Add SPDX id lines to remaining source files

2026-02-04 07:23:45 -07:00

mock_file.c

Convert 'alloc_obj' family to use the new default GFP_KERNEL argument

2026-02-21 17:09:51 -08:00

mpscq.h

io_uring/mpscq: add lockless multi-producer, single-consumer FIFO queue

2026-06-13 06:27:00 -06:00

msg_ring.c

io_uring: switch struct io_ring_ctx internal bitfields to flags

2026-03-16 15:32:59 -06:00

msg_ring.h

io_uring/msg_ring: Drop custom destructor

2024-12-27 10:08:21 -07:00

napi.c

io_uring/napi: clear tracked NAPI entries on unregister

2026-05-03 23:21:23 -06:00

napi.h

io_uring/napi: clear tracked NAPI entries on unregister

2026-05-03 23:21:23 -06:00

net.c

io_uring/net: support registered buffer for plain send and recv

2026-06-08 08:32:43 -06:00

net.h

io_uring/net: allow filtering on IORING_OP_CONNECT

2026-05-14 07:23:01 -06:00

nop.c

io_uring/nop: Drop a wrong comment in struct io_nop

2026-06-02 17:43:20 -06:00

nop.h

…

notif.c

io_uring: Add SPDX id lines to remaining source files

2026-02-04 07:23:45 -07:00

notif.h

io_uring/notif: implement notification stacking

2024-04-22 19:31:18 -06:00

opdef.c

io_uring/net: Remove async_size for OP_LISTEN

2026-06-02 17:43:15 -06:00

opdef.h

io_uring: remove iopoll_queue from struct io_issue_def

2026-03-16 16:14:14 -06:00

openclose.c

io_uring/openclose: fix io_pipe_fixed() slot tracking for specific slots

2026-02-11 20:31:21 -07:00

openclose.h

io_uring/bpf_filter: allow filtering on contents of struct open_how

2026-01-27 11:10:46 -07:00

poll.c

io_uring/poll: ensure EPOLL_ONESHOT is propagated for EPOLL_URING_WAKE

2026-04-21 19:18:34 -06:00

poll.h

io_uring: add wrapper type for io_req_tw_func_t arg

2025-11-03 08:31:26 -07:00

query.c

io_uring/zcrx: add shared-memory notification statistics

2026-05-26 10:42:01 -06:00

query.h

io_uring/query: drop unused io_handle_query_entry() ctx arg

2025-11-26 09:37:10 -07:00

refs.h

io_uring: Add SPDX id lines to remaining source files

2026-02-04 07:23:45 -07:00

register.c

io_uring/register: fix ring resizing with mixed/large SQEs/CQEs

2026-04-21 12:19:08 -06:00

register.h

io_uring: unify getting ctx from passed in file descriptor

2026-04-08 13:21:35 -06:00

rsrc.c

io_uring/rsrc: raise registered buffer 1GB limit

2026-05-14 08:12:29 -06:00

rsrc.h

io_uring/rsrc: bump struct io_mapped_ubuf length field to size_t

2026-05-14 08:12:23 -06:00

rw.c

io_uring/rw: add defensive hardening for negative kbuf lengths

2026-04-21 12:19:03 -06:00

rw.h

io_uring: add wrapper type for io_req_tw_func_t arg

2025-11-03 08:31:26 -07:00

slist.h

io_uring: Add SPDX id lines to remaining source files

2026-02-04 07:23:45 -07:00

splice.c

io_uring: don't include filetable.h in io_uring.h

2025-09-08 13:20:46 -06:00

splice.h

io_uring/splice: open code 2nd direct file assignment

2024-10-29 13:43:28 -06:00

sqpoll.c

io_uring: switch normal task_work to a mpscq

2026-06-13 06:27:11 -06:00

sqpoll.h

io_uring/sqpoll: switch away from getrusage() for CPU accounting

2025-10-22 10:51:20 -06:00

statx.c

allow incomplete imports of filenames

2026-01-13 15:18:07 -05:00

statx.h

…

sync.c

io_uring/sync: validate passed in offset

2026-01-21 11:50:59 -07:00

sync.h

…

tctx.c

io_uring: switch normal task_work to a mpscq

2026-06-13 06:27:11 -06:00

tctx.h

io_uring/tctx: have io_uring_alloc_task_context() return tctx

2026-04-08 13:21:30 -06:00

timeout.c

io_uring/timeout: honour caller's time namespace for IORING_TIMEOUT_ABS

2026-05-06 04:58:56 -06:00

timeout.h

io_uring/timeout: migrate reqs from ts64 to ktime

2026-03-09 07:21:54 -06:00

truncate.c

fs: pass on FTRUNCATE_* flags to do_truncate

2026-03-23 12:41:57 +01:00

truncate.h

io_uring: add support for ftruncate

2024-02-09 09:04:39 -07:00

tw.c

io_uring: switch normal task_work to a mpscq

2026-06-13 06:27:11 -06:00

tw.h

io_uring: switch normal task_work to a mpscq

2026-06-13 06:27:11 -06:00

uring_cmd.c

io_uring/uring_cmd: allow non-iopoll cmds with IORING_SETUP_IOPOLL

2026-03-16 16:14:14 -06:00

uring_cmd.h

io_uring/cmd: remove struct io_uring_cmd_data

2025-07-18 12:34:56 -06:00

wait.c

io_uring: switch local task_work to a mpscq

2026-06-13 06:27:06 -06:00

wait.h

io_uring: switch local task_work to a mpscq

2026-06-13 06:27:06 -06:00

waitid.c

io_uring/waitid: fix KCSAN warning on io_waitid->head

2026-01-19 19:55:30 -07:00

waitid.h

io_uring: move cancelations to be io_uring_task based

2024-11-06 13:55:38 -07:00

xattr.c

Convert 'alloc_obj' family to use the new default GFP_KERNEL argument

2026-02-21 17:09:51 -08:00

xattr.h

…

zcrx.c

io_uring/zcrx: kill dead 'sock' member in struct io_zcrx_args

2026-06-11 17:04:00 -06:00

zcrx.h

io_uring/zcrx: add shared-memory notification statistics

2026-05-26 10:42:01 -06:00