In a future patch, mm init will use kho_scratch_overlap() for deciding
the migrate type of pageblocks it initializes. The earliest user
currently is free_area_init(). kho_scratch_overlap()
relies on kho_scratch pointer being initialized. Introduce
kho_memory_init_early() to do this.
kho_populate() would normally be a good place to do this, but
unfortunately, phys_to_virt() does not work at that point on ARM64. So
we need yet another initialization function.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-15-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Support for discovering memory blocks with no preserved memory will be
added in coming patches. These areas will also be marked as scratch to
allow allocations from them. Memblock will switch to looking through the
scratch array to decide the right migratetype.
Expose kho_scratch_overlap() to KHO users. Since it is now used by
non-debug code, move it out of kexec_handover_debug.c and into
kexec_handover.c. Gate the overlap checks in kho_preserve_folio() and
kho_preserve_pages() by IS_ENABLED(CONFIG_KEXEC_HANDOVER_DEBUG) instead.
Since kexec_handover_debug.c is now empty, delete it.
Add a stub for kho_scratch_overlap() to memblock tests to make sure it
compiles. It will be used in memblock by a coming commit.
No functional changes.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-14-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Move the initialization logic of the radix tree into
kho_radix_init_tree() instead of having users open-code it. Makes the
boundaries cleaner and reduces code duplication when a new user of the
radix tree will be added in a future commit.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-13-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
The KHO radix tree allocates memory for table pages from the buddy
allocator using get_zeroed_page(). This is not available in early boot
when memblock is still active.
Using the radix tree in early boot is useful for KHO to track metadata
about its memory. One such example is for tracking free blocks for
memory allocation when scratch runs out of space. This feature will be
added in the following commits.
Add kho_radix_{alloc,free}_node() which allocate and free the table
pages. They use slab_is_available() to decide which allocator to use.
While slab_is_available() indicates availability of the slab allocator,
it gets initialized right after buddy so it serves the same practical
purpose.
Reviewed-by: Pasha Tatashin <pasha.tatashin@soleen.com>
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-11-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
The KHO memory preservation radix tree does not mark the table pages
themselves as preserved. This is done to avoid a circular dependency
where preserving a page can lead of allocating other preserved pages.
This means any walker looking for free ranges of memory outside of
scratch areas will ignore the table
Add a table callback that is invoked for each table page. The callback
is given the physical address of the table page.
This is useful for the upcoming mechanism that discovers blocks of
memory with no preserved pages and lets them be used for boot memory.
Another use case is for users of the radix tree other than KHO itself.
The radix tree does not preserve its own pages due to the circular
dependency described above. But external users of the radix tree would
need to preserve and restore their pages for the radix tree to survive
past early boot. They can use this callback to do so.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-9-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
The memory retrieval logic is spread out across kho_mem_retrieve() and
kho_memory_init(). The incoming scratch area is initialized at
kho_memory_init(), and the error handling is done there too.
Consolidate all this logic into kho_mem_retrieve() to make the code
cleaner.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-7-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
This allows other functions to also use the radix tree.
While at it, add kho_get_mem_map() helper to get the virtual address of
the preserved memory map and use that helper instead of duplicating the
code to get the preserved memory map from the FDT.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-6-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
The KHO radix tree was designed to track preserved pages. So it does not
provide the capability to track any 64-bit key. Instead, it limits the
key width to how much it needs for tracking PFNs and their orders.
Limiting the width reduces the number of levels in the tree.
KHO is not expected to be the only user of the radix tree. With the API
generalized to allow other users, now it is possible to add any key to
the tree.
Check the key width at kho_radix_add_key(), and error out if it exceeds
what the tree can handle. Do this instead of increasing the tree depth
since right now there are no users that need to use wider keys, so this
avoids memory overhead and ABI breakage.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-4-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
The KHO radix tree constants are somewhat hard to understand. The tree
depth essentially comes from the max key width. The max key width comes
from the need to store a 52-bit PFN plus one more bit for the order.
All this is very obscure with the corrent code. The PFN width is defined
as KHO_ORDER_0_LOG2, which makes very little sense to a new reader not
already familiar with what the value means. Then the fact that an extra
bit is needed is hidden in the KHO_TREE_MAX_DEPTH calculation.
Simplify this by removing KHO_ORDER_0_LOG2 and replace it with
KHO_RADIX_KEY_WIDTH. Update the comment to explain why this value is
used. This moves the +1 from KHO_TREE_MAX_DEPTH to KHO_RADIX_KEY_WIDTH,
making things clearer.
Update kho_{encode,decode}_radix_key() to not use KHO_ORDER_0_LOG2.
Instead, refactor the code and comments to make it clearer how the
encoding and decoding is done.
In kho_encode_radix_key(), add a new variable for the shift for physical
address. Use that in calculating where the order bit goes and in
calculating the shifted PFN. Update comments to make this clearer.
In kho_radix_decode_key(), turn order_bit to 0-indexed to simplify the
eventual calculation for order. Touch up comments to make the
computation clearer.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-3-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
The KHO radix tree is a data structure that can track the presence or
absence of an arbitrary key, with nothing inherently tied to KHO memory
preservation tracking. This was one of the design goals of the radix
tree. This was done to enable it to be re-used by other users of KHO.
Despite that, the radix tree APIs are very closely tied to KHO memory
preservation tracking. Adding a key is done by kho_radix_add_page(),
which encodes it as a page tracking operation and takes in PFN and
order. kho_radix_del_page() does the same. These functions encode the
key internally that goes into the radix tree. kho_radix_walk_tree() does
the same by baking the PFN and order into the callback arguments.
Generalize the APIs by taking the key directly and doing the encoding at
the callers. Rename the functions to kho_radix_add_key() and
kho_radix_del_key(). In practice, this removes a line each from the
functions and moves the encoding function call to the callers.
Similarly, update kho_radix_tree_walk_callback_t to take the key
directly.
Now that key encoding is no longer an inherent part of the radix tree
and can be decided by the user, rename kho_radix_{encode,decode}_key()
to kho_{encode,decode}_radix_key(). This moves them out of the
"kho_radix_" name space into the "kho_" namespace. This emphasizes that
this is KHO's way of encoding the key for its radix tree.
Reviewed-by: Pasha Tatashin <pasha.tatashin@soleen.com>
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260801084833.1897543-2-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Pull tracing fixes from Steven Rostedt:
- Move rb_desc->nr_page_va before updating dynamic array
The rb_descr->page_va is a dynamic array counted by nr_page_va. But
the updating of the page_va[] is done before the nr_page_va is
incremented causing a build with CONFIG_UBSAN_BOUNDS to flag it as an
overflow.
Move the increment of the counted by value before the array element
is updated.
- Propagate errors from remote event bulk updates
The return value of trace_remote_enable_event() was not being checked
by remote_events_dir_enable_write() where it would silently fail.
Have it check the return value and propagate that back up to user
space.
- Fix resource leak on mmiotrace trace_pipe close
The mmiotrace tracer was created in 2008 before the trace_pipe had a
close callback to allow tracers to do clean up from trace_pipe open.
The trace_pipe close cleanup callback was added in 2009 but the
mmiotrace tracer was not updated. It had a hack to do the cleanup in
the read call, where it may leak if user space did not read the
entire buffer.
Add a callback to mmiotrace trace_pipe close do to the cleanup
properly.
- Fix a possible NULL pointer dereference in the mmiotrace tracer
If the mmio_pipe_open() fails to find a PCI device, it will set the
hiter->dev pointer to NULL. The read function will blindly
dereference that pointer. Fix the read call to check to see if that
pointer is populated before dereferencing it.
- Fix union collision of module and refcnt for dynamic events
In 'struct trace_event_call', the 'module' pointer and the 'refcnt'
atomic variable share the same memory space in a union. The filter on
module logic only checked if the 'module' was set to determine if the
event belonged to the module. As dynamic events are always builtin,
it doesn't need the 'module' field of the structure and used a
refcount. But the module filtering logic would then mistaken these
dynamic events as a module and call module_name(event->module) on it.
Add a check to see if the event is a dynamic event and if so, do not
check it for being part of the given module.
- Reset the top level buffer in selftests before running instances
The ftracetest selftest initializes each instance before executing
the tests. But it does not reset the top level buffer. Dynamic events
are only added and removed by the top level so any left over dynamic
events will not be removed by the reset in the instances.
Left over dynamic events can cause the tests to incorrectly fail.
Reset the top level buffer before running the instances.
- Make the context_switch counter 64 bit
The code to read user space for a system call trace event or for a
trace_marker will disable migration, enable preemption, read user
space into a per CPU buffer, disable preemption and enable migration
again. It checks if the per CPU context switch counter to see if it
changed, and if it did not, it would know that the per CPU buffer was
not touched by another task.
But the save counter was 32 bit and it would compare it to the 64 bit
context_switch variable. A long running system could have the
context_switch variable greater that 1<<32 in which case the compare
will always fail. The compare will promote the 32 bit int saved value
to 64 bit and compare it to the full 64 bit counter. Since the top 32
bits of the saved value was zero, it would never match.
- Fix a use-after-free of the event_enable trigger
The event_enable trigger allows for enabling one event when another
event is triggered. When the trigger is removed, it must go through a
synchronization phase to make sure it is not triggered again. The
trigger itself is delayed by the "bulk delay" logic that was recently
added. But the code that frees the event_enable data used to rely on
the trigger code to do the synchronization. Now that the code uses
the call RCU functions (and a workqueue), that delay no longer is
there.
Add a callback private_data_free() function that allows triggers to
clean up data after the synchronization phase has completed.
- Move the module_ref counter into the delay callback
Since an event of the event_enable trigger can enable an event for a
module, it ups the module ref count for that event's module. This
prevents the event from trying to enable an event that no longer
exists and cause a use-after-free bug.
The ref counter was set back down when the trigger was removed but
not after thy synchronization phase. This could lead to the module
data being accessed after module was unloaded.
Move the module ref decrement into the private_data_free() callback
of the event_enable trigger.
- Add mutex to protect parser in ftrace filtering
The set_ftrace_filter file uses a parsing descriptor that is
allocated at open and modified by writes. If multiple threads were to
write to the descriptor at the same time, it can corrupt the parser.
Add a mutex around the modifications of the parser descriptor.
- Fix possible corruption in perf syscall tracing
The perf system call trace events can now read user space. To do so,
the reads of user space enable preemption and disables it again.
During this time that preemption is enabled, the task can migrate.
The perf event list head is assigned via a per CPU pointer. It is
done before the user space part is called. If the user space reading
migrates the task to another CPU, then the head pointer is no longer
valid.
Re-assign the head pointer after the reading of user space to keep it
using the correct data.
* tag 'trace-v7.2-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
tracing: perf: Fix stale head for perf syscall tracing
ftrace: Add global mutex to serialize trace_parser access
tracing: Delay module ref count for "enable_event" trigger
tracing: Fix use-after-free freeing trigger private data
tracing: Fix context switch counter truncation
selftests/ftrace: Reset triggers at top level before instance loop
tracing: Fix union collision of module and refcnt for dynamic events
tracing: Fix mmiotrace possible NULL dereferencing of hiter->dev
tracing: Fix resource leak on mmiotrace trace_pipe close
tracing: Propagate errors from remote event bulk updates
tracing/remotes: Fix page_va[] access before counter update in trace_remote_alloc_buffer()
In ftrace, the trace_parser structure is allocated and initialized when
a trace file is opened, and is subsequently used across write and release
handlers to parse user input.
The affected handler paths and their specific functions are:
- Open paths: ftrace_regex_open(), ftrace_graph_open()
- Write paths: ftrace_regex_write(), ftrace_graph_write()
- Release paths: ftrace_regex_release(), ftrace_graph_release()
If userspace opens a trace file descriptor and shares it across multiple
threads, concurrent write calls will race on the parser's internal state,
specifically the 'idx', 'cont', and 'buffer' fields, leading to corrupted
input or undefined behavior.
Fix this by adding a global mutex, parser_lock, to serialize all access
to trace_parser across write and release paths, preventing concurrent
corruption of parser state.
Fixes: e704eff3ff ("ftrace: Have set_graph_function handle multiple functions in one write")
Fixes: 689fd8b65d ("tracing: trace parser support for function and graph")
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260725024721.1983675-1-wutengda@huaweicloud.com
Signed-off-by: Tengda Wu <wutengda@huaweicloud.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Pull bpf fixes from Eduard Zingerman:
- Fix tcp_bpf_sendmsg() error path mistaking a concurrently-freed
sk_psock->cork for the local temporary message and freeing it again
(Chengfeng Ye)
- Reject passing scalar NULL to nonnull arg of a global subprog.
Previously the verifier did not account for the cases directly
passing scalars to a global subprog, e.g.: 'global_func(0);' would
pass even if 'global_func' argument was marked nonnull (Amery Hung)
* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf:
bpf, sockmap: Fix cork use-after-free in tcp_bpf_sendmsg()
selftests/bpf: Test passing scalar NULL to nonnull global subprog
bpf: Reject passing scalar NULL to nonnull arg of a global subprog
Triggers are now delayed from freeing, but can still be triggered until
after the RCU grace period has ended. The freeing of the enable_event data
is put into the private_data_free() callback, but the put of the module
refcount is done immediately.
It is possible that if a module is removed that has an event that would
enable (or disable) it is still active, it can read the data of the module
after it is removed causing a use-after-free bug.
Move the trace_event_put_ref() that releases the module into the delayed
callback so that the module can not be removed until any reference to its
events are finished.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260724132415.1b5005db@gandalf.local.home
Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://sashiko.dev/#/patchset/20260724030523.19081-1-devnexen%40gmail.com
Fixes: 61d445af0a ("tracing: Add bulk garbage collection of freeing event_trigger_data")
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Commit 61d445af0a ("tracing: Add bulk garbage collection of freeing
event_trigger_data") moved the kfree() of event_trigger_data to a kthread
that runs tracepoint_synchronize_unregister() before freeing. That removed
the synchronization the trigger .free callbacks used to get implicitly and
inline from trigger_data_free().
event_hist_trigger_free(), event_hist_trigger_named_free() and
event_enable_trigger_free() free their satellite data (hist_data, cmd_ops,
enable_data) right after trigger_data_free() returns. With the
synchronization now deferred to the kthread, a concurrent tracepoint
handler can still reach that data through the list_del_rcu()'d trigger,
causing a use-after-free.
The histogram teardown must stay synchronous: remove_hist_vars() and
unregister_field_var_hists() have to detach a synthetic event from the
histogram before the trigger-removal write returns, otherwise a following
command races in and the synthetic-event removal fails with -EBUSY, as the
trigger-synthetic-eprobe.tc selftest catches. Make those callbacks wait
with the correct barrier - tracepoint_synchronize_unregister(), matching
the free kthread - before freeing.
The enable trigger has no such synchronous requirement, and a blocking
synchronize there would re-serialize the path that commit deliberately
deferred. Give it an optional private_data_free() callback that the free
kthread runs after its grace period, and free enable_data from there.
Link: https://patch.msgid.link/20260724030523.19081-1-devnexen@gmail.com
Suggested-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Suggested-by: Steven Rostedt <rostedt@goodmis.org>
Fixes: 61d445af0a ("tracing: Add bulk garbage collection of freeing event_trigger_data")
Signed-off-by: David Carlier <devnexen@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
trace_user_fault_read() samples nr_context_switches_cpu() before enabling
preemption and retries the user copy if the counter changes. The helper
returns unsigned long long because rq->nr_switches is u64, but the saved
value is unsigned int.
Once a CPU has performed 2^32 context switches, assigning the counter to
cnt discards its upper bits. The comparison after the copy promotes cnt
back to unsigned long long, but the lost bits remain zero, so it reports a
change even when the task was never scheduled out. Every retry then fails
the same way until the 100-try guard warns and the user copy is abandoned.
This affects long-running systems and workloads with high context-switch
rates. A CPU switching 1,000 times per second takes about 50 days.
Store the sampled count in unsigned long long so the full value is
preserved.
Cc: stable@vger.kernel.org
Fixes: 64cf7d058a ("tracing: Have trace_marker use per-cpu data to read user space")
Link: https://patch.msgid.link/20260717173252.3431565-1-usama.arif@linux.dev
Reported-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Usama Arif <usama.arif@linux.dev>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Reviewed-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
In 'struct trace_event_call', the 'module' pointer and the 'refcnt'
atomic variable share the same memory space in a union. For dynamic
events, the union member is 'refcnt', which acts as an active
reference counter.
When a dynamic event (such as kprobe, uprobe, fprobe, eprobe, or
wprobe) has a non-zero reference count (e.g. due to active event
triggers or perf attachments), its 'call->module' evaluates to a
small non-zero integer instead of NULL.
When filtering or setting events for a specific module (e.g., writing
':mod:<module>' to 'set_event'), the code in
'__ftrace_set_clr_event_nolock()' and 'update_event_fields()' reads
'call->module' directly without checking whether the event is dynamic.
This causes the kernel to treat the small integer (refcnt) as a
'struct module' pointer, leading to a NULL/invalid pointer dereference
(Oops) when dereferencing the module name.
Fix this by ensuring that the 'TRACE_EVENT_FL_DYNAMIC' flag is checked
before treating 'call->module' as a valid pointer in these code paths.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/178425670947.84440.11344393611899824907.stgit@devnote2
Fixes: 4c86bc531e ("tracing: Add :mod: command to enabled module events")
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
A global subprogram argument tagged __arg_nonnull is set up as a
non-nullable PTR_TO_MEM. However the verifier does not check against a
scalar NULL, leading to real NULL pointer dereference. Reject it as
well.
Fixes: 94e1c70a34 ("bpf: support 'arg:xxx' btf_decl_tag-based hints for global subprog args")
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260723221815.367797-1-ameryhung@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
The mmiotrace tracer was added May 12th 2008. At that time, resources
created in pipe_open() could not be freed because there was not
pipe_close function pointer of the tracer. The pipe_close function pointer
was added in December 7th, 2009, but the mmiotrace tracer was not updated.
mmio_pipe_open() allocates a header_iter and takes a pci_dev reference
when trace_pipe is opened. mmio_close() frees them, but it was only
wired to the tracer's .close callback.
tracing_release_pipe() invokes .pipe_close, not .close, when the
trace_pipe file is released. As a result, closing trace_pipe with the
mmiotrace tracer active leaked the header_iter allocation and left a
stale pci_dev reference.
Set .pipe_close to mmio_close, matching how function_graph wires both
callbacks to the same handler.
Note, if the trace_pipe is read to completion, it will clean up the
resources, but if one were to run:
# head -n 1 /sys/kernel/tracing/trace_pipe
VERSION 20070824
Over and over again, it would trigger a massive leak.
Cc: stable@vger.kernel.org
Fixes: c521efd170 ("tracing: Add pipe_close interface)
Link: https://patch.msgid.link/20260715143604.14481-1-gaikwad.dcg@gmail.com
Signed-off-by: deepakraog <gaikwad.dcg@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
remote_events_dir_enable_write() ignores the return value from
trace_remote_enable_event(). If a remote rejects an event state change,
the write therefore reports success even though the affected event remains
in its previous state.
Keep trying all events, but retain and return the first error. This matches
__ftrace_set_clr_event_nolock(), which permits partial updates while
notifying userspace when an operation fails.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260715074455.3897-1-liu.yun@linux.dev
Fixes: 775cb093bc ("tracing: Add events/ root files to trace remotes")
Assisted-by: Codex:gpt-5.6-sol
Signed-off-by: Jackie Liu <liuyun01@kylinos.cn>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
page_va[] is annotated __counted_by(nr_page_va), so nr_page_va must
cover an index before that element is accessed. The allocation loop
writes page_va[id] while nr_page_va is still id and increments it only
afterwards, so every write is one element past the declared count.
The store is out of bounds with respect to the annotation: a build with
CONFIG_UBSAN_BOUNDS on a toolchain that honours __counted_by
(clang >= 20.1, gcc >= 15.1) flags it as an array-index overflow.
Increment nr_page_va before writing the element it now covers. A failed
allocation then leaves the slot counted but NULL; the error path frees
it with free_page(0), which is a no-op.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260713072823.2668323-1-fuad.tabba@linux.dev
Fixes: 96e43537af ("tracing: Introduce trace remotes")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Tested-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Commit b0473dcd4b ("smp: Improve smp_call_function_single()
CSD-lock diagnostics") changed smp_call_function_single() so that,
when CSD lock debugging is enabled, async !wait calls use the
destination CPU csd_data. That improves diagnostics, but it also removes
the single-writer property that made the old csd_lock() safe: multiple
CPUs can now prepare the same destination CPU CSD concurrently.
csd_lock() currently waits for CSD_FLAG_LOCK to clear and then sets the
bit with a non-atomic read-modify-write. Two senders can both see an
unlocked CSD, set the bit, overwrite the callback fields, and enqueue
the same llist node. Re-adding a node that is already the queue head can
make node->next point to itself, leaving the target CPU stuck walking
call_single_queue. Later synchronous work, such as a TLB shootdown, can
then remain queued and trigger soft-lockup warnings or panics.
Keep the single csd_lock() implementation, but when CSD lock debugging is
enabled, acquire CSD_FLAG_LOCK with try_cmpxchg_acquire(). This makes the
destination CPU CSD a real atomic lock in the only configuration where it
can be shared by multiple remote senders, while preserving the existing
non-debug fast path.
Fixes: b0473dcd4b ("smp: Improve smp_call_function_single() CSD-lock diagnostics")
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260716004539.13983-2-paulmck@kernel.org
Commit b0473dcd4b ("smp: Improve smp_call_function_single()
CSD-lock diagnostics") made smp_call_function_single() use the destination
CPU's csd_data when CSD lock debugging is enabled. That lets the debug code
associate a stuck CSD lock with the target CPU, but it also means the CPU
argument is used in per_cpu_ptr() before generic_exec_single() has a chance
to validate it.
This becomes unsafe when smp_call_function_any() cannot find an online CPU
in the supplied mask. In that case the selected CPU can be nr_cpu_ids, and
the !wait path calls get_single_csd_data(cpu) before generic_exec_single()
returns -ENXIO. With csdlock_debug_enabled set, that indexes the per-CPU
offset array with an invalid CPU number.
Use the destination CPU's csd_data only when the CPU number is within
nr_cpu_ids. For invalid CPU numbers, fall back to the local CPU's csd_data
and let generic_exec_single() perform the existing validation and return
-ENXIO.
Fixes: b0473dcd4b ("smp: Improve smp_call_function_single() CSD-lock diagnostics")
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Paul E. McKenney <paulmck@kernel.org>
Acked-by: Muchun Song <muchun.song@linux.dev>
Link: https://patch.msgid.link/20260716004539.13983-1-paulmck@kernel.org
Pull liveupdate fix from Mike Rapoport:
- Fix validation of LIVEUPDATE_SESSION_GET_NAME ioctl argument caused
by a wrong resolution of a merge conflict during the last merge
window
* tag 'liveupdate-fixes-2026-07-22' of git://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux:
liveupdate: fix GET_NAME ioctl argument validation
eprobe_dyn_event_match() checks if the target event system in argv[0]
matches ep->event_system using strncmp(ep->event_system, argv[0], len).
However, if ep->event_system is longer than len (e.g. "eprobes" vs
"ep/event"), strncmp() still returns 0 because the first len characters
match.
Check that ep->event_system[len] is '\0' to ensure exact system name
matching.
Link: https://lore.kernel.org/all/178454235856.290363.14872590900774231133.stgit@devnote2/
Fixes: 7d5fda1c84 ("tracing: Fix event probe removal from dynamic events")
Cc: stable@vger.kernel.org
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
In __set_print_fmt(), LEN_OR_ZERO is defined as (len ? len - pos : 0).
If len is non-zero but smaller than pos, len - pos evaluates to a negative
integer. When passed as a size argument to snprintf(), this negative value
is cast to a large unsigned size_t, bypassing buffer size limits.
Ensure len > pos before subtracting to avoid integer underflow.
Link: https://lore.kernel.org/all/178454234934.290363.15247317871499514139.stgit@devnote2/
Fixes: 5bf652aaf4 ("tracing/probes: Integrate duplicate set_print_fmt()")
Cc: stable@vger.kernel.org
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
If trace_probe_log.argc is 0 in __trace_probe_log_err(), the loop
constructing the command string will not execute and p will remain equal to
command. Writing to *(p - 1) will cause an out-of-bounds access before
command. This should not happen, but better to be treated.
Reject if trace_probe_log.argc is 0.
Link: https://lore.kernel.org/all/178454233992.290363.18323091580600697731.stgit@devnote2/
Fixes: ab105a4fb8 ("tracing: Use tracing error_log with probe events")
Cc: stable@vger.kernel.org
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
In trace_probe_match_command_args(), a stack buffer buf[MAX_ARGSTR_LEN + 1]
(256 bytes) is used to format "<name>=<comm>". However, since name can
be up to 32 bytes (MAX_ARG_NAME_LEN) and comm up to 255 bytes
(MAX_ARGSTR_LEN), the formatted string can exceed 256 bytes and get
truncated by snprintf(), causing spurious argument matching failures.
Instead of formatting into a temporary buffer on stack, compare the
argument name, the '=' delimiter, and the comm expression directly.
Link: https://lore.kernel.org/all/178454233010.290363.10428767141343428804.stgit@devnote2/
Fixes: eb5bf81330 ("tracing/kprobe: Add per-probe delete from event")
Cc: stable@vger.kernel.org
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Pull bpf fixes from Kumar Kartikeya Dwivedi:
- Fix a UAF in socket clone early bailout paths (Matt Bobrowski)
- Reject unhashed UDP sockets on sockmap update to prevent refcount
leaks (Michal Luczaj)
- Account for receive queue data in FIONREAD on sockmap sockets without
a verdict program (Mattia Meleleo)
- Reject negative constant offsets for verifier buffer pointers (Sun
Jian)
- Fix for tracing of kfuncs with implicit arguments (Ihor Solodrai)
* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf:
selftests/bpf: Cover tracing implicit kfunc args
bpf: Fix tracing of kfuncs with implicit args
selftests/bpf: Cover negative buffer pointer offsets
bpf: Reject negative const offsets for buffer pointers
selftests/bpf: Test FIONREAD on a sockmap socket without a verdict program
bpf, sockmap: Account for receive queue in FIONREAD without a verdict program
selftests/bpf: Fail unbound UDP on sockmap update
selftests/bpf: Adapt sockmap update error handling
bpf, sockmap: Reject unhashed UDP sockets on sockmap update
selftests/bpf: Ensure UDP sockets are bound
bpf: Fix UAF in sock clone early bailouts
A kfunc marked with KF_IMPLICIT_ARGS flag takes implicit arguments
(such as bpf_prog_aux) that the verifier injects at load time.
resolve_btfids strips those from the kfunc's BTF-visible prototype and
keeps the real kernel ABI in a counterpart _impl prototype [1].
fentry/fexit/fmod_ret/fsession programs may attach to the BPF kernel
functions, including those with implicit args. However
bpf_check_attach_target() and bpf_check_attach_btf_id_multi() extract
the struct btf_func_model from the wrong BTF prototype of the
kfunc. The btf_func_model is later read to construct the trampoline,
which then causes the injected implicit argument to be clobbered and
the kfunc dereferencing garbage.
Add btf_attach_func_proto() to resolve the real ABI prototype of the
kfunc the way the call site does: by looking up the _impl prototype
for a KF_IMPLICIT_ARGS kfunc. Use it at both attach-target model
construction sites.
To enable this, make two supporting changes:
* pass bpf_verifier_log instead of bpf_verifier_env to
find_kfunc_impl_proto(), so it can be reused from the attach path
* add btf_kfunc_check_flag() to test a flag across all of a kfunc's
hook sets, because a program attaching to a kfunc is not in the
kfunc's call-set
KF_IMPLICIT_ARGS must be consistent across the sets, so
btf_kfunc_check_flag() returns -EINVAL on inconsistency.
btf_kfunc_check_flag() reads the kfunc's flags from the target's
kfunc_set_tab. For a module BTF that table is stable only after the
module is live, so take a module reference around the read, mirroring
how the kfunc call path gates the same lookup with btf_try_get_module().
The remaining call sites of btf_distill_func_proto() are safe as
is. The BPF_TRACE_ITER case distills a registered iterator's
prototype, and bpf_struct_ops_desc_init() distills the
function-pointer members of a struct_ops type. Neither is a kfunc, and
so can't have implicit arguments.
[1] https://lore.kernel.org/all/20260120222638.3976562-1-ihor.solodrai@linux.dev/
Fixes: 64e1360524 ("bpf: Verifier support for KF_IMPLICIT_ARGS")
Reported-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://github.com/sched-ext/scx/issues/3687#issuecomment-4906694106
Link: https://patch.msgid.link/20260713235223.1639022-2-ihor.solodrai@linux.dev
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
LIVEUPDATE_SESSION_GET_NAME was developed in the liveupdate/next branch
while the session type validation change was carried in liveupdate-fixes.
When the conflict between the two branches was resolved, the GET_NAME
operation descriptor picked up the structure and last member from
RETRIEVE_FD.
This makes both its known size and minimum size 16 bytes rather than 72.
A zero-initialized request still succeeds because luo_session_get_name()
writes the full name before luo_ucmd_respond() copies the full GET_NAME
response to userspace. However, copy_struct_from_user() treats the
output-only name field as unknown trailing data and rejects the request
with -E2BIG if any byte in that field is nonzero.
Use the GET_NAME structure and its name field in the descriptor.
Link: https://lore.kernel.org/all/ahWlYXNjGUbkKoHy@sirena.org.uk/
Assisted-by: Codex:gpt-5.6-sol
Reviewed-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Signed-off-by: Jackie Liu <liuyun01@kylinos.cn>
Link: https://patch.msgid.link/20260716012607.22020-1-liu.yun@linux.dev
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
The verifier rejects variable offsets for PTR_TO_TP_BUFFER and PTR_TO_BUF
accesses, but it currently accepts a constant negative offset produced by
pointer arithmetic.
Commit 022ac07508 ("bpf: use reg->var_off instead of reg->off for
pointers") moved constant pointer offsets from reg->off to reg->var_off.
However, __check_buffer_access() continued to check only the instruction
offset. An access with reg->var_off equal to -8 and an instruction offset
of zero therefore passes verification.
For writable raw tracepoints, the access end is also calculated from the
unsigned reg->var_off.value. An eight-byte access starting at -8 wraps
the calculated end to zero, allowing the program to load and attach
without increasing max_tp_access.
After ensuring that reg->var_off is constant, calculate the effective
access start using signed arithmetic and reject it when it is negative.
Use the validated start to calculate the access end for both
PTR_TO_TP_BUFFER and PTR_TO_BUF.
Fixes: 022ac07508 ("bpf: use reg->var_off instead of reg->off for pointers")
Signed-off-by: Sun Jian <sun.jian.kdev@gmail.com>
Acked-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
Cc: stable@vger.kernel.org # 5.2.0
Link: https://patch.msgid.link/20260714093846.18159-2-sun.jian.kdev@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Pull cgroup fixes from Tejun Heo:
- A cpuset that never set its memory nodes could divide by zero when a
task's mempolicy rebinds on CPU hotplug. Rebind against the effective
nodes, which are always populated
- Documentation fixes for memory.stat, io.stat, and the misc and v1
RDMA controllers
* tag 'cgroup-for-7.2-rc3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup:
Docs/admin-guide/cgroup-v2: note blkcg_debug_stats gates io.latency stats
Docs/admin-guide/cgroup-v1: document rdma.peak, rdma.events and rdma.events.local
Docs/admin-guide/cgroup-v2: drop stale misc interface file count
cgroup/cpuset: rebind mm mempolicy to effective_mems, not mems_allowed
Docs/admin-guide/cgroup-v2: fix memory.stat doc details
Pull sched_ext fixes from Tejun Heo:
- Lifecycle fixes for the new sub-scheduler support: two
use-after-frees and an enable-failure path that left a
half-initialized sub-scheduler linked.
- Two dispatch-path locking bugs: a spurious scheduler abort from a
migration race, and a lockdep splat from stale runqueue-lock
tracking.
- Callback and task-state fixes: stale scheduler-owned state on a task
leaving SCX, a weight callback running after disable, and a bogus
warning on core-scheduling forced idle.
- On nohz_full, finite-slice tasks could miss the tick that expires
their slice. Enable it when such a task is picked, with a selftest.
- Smaller fixes: userspace CPU-mask helpers, ratelimited deprecation
warnings, docs and a sparse annotation.
* tag 'sched_ext-for-7.2-rc3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext:
sched_ext: Skip ops.set_weight() for disabled tasks
tools/sched_ext: scx - Fix cmask_subset(), cmask_equal() and cmask_weight()
sched_ext: Fix premature ops->priv publication in scx_alloc_and_add_sched()
sched_ext: Record an error on errno-only sub-enable failure
selftests/sched_ext: Verify nohz_full tick behavior
sched_ext: Enable tick for finite slices on nohz_full
sched_ext: Preserve rq tracking across local DSQ dispatch
sched_ext: Documentation: Fix ops table header reference
sched_ext: Don't warn on core-sched forced idle in put_prev_task_scx()
sched_ext: Pin parent scx_sched across a child sub-scheduler's lifetime
sched_ext: Annotate ksyncs with __rcu in alloc/free_kick_syncs()
sched_ext: Check remote rq eligibility under task's rq lock
sched_ext: Reset dsq_vtime and slice when a task leaves SCX
sched_ext: Avoid flooding the log with deprecation warnings
Pull tracing fixes from Steven Rostedt:
- Free field in error path of synthetic event parse
In __create_synth_event() the field was allocated but was not freed
in the error path
- Fix ring_buffer_event_length() on 8 byte aligned architectures
On architectures with CONFIG_HAVE_64BIT_ALIGNED_ACCESS set to y, the
ring_buffer_event_length() may return the wrong size. This is because
archs with that config set will always use the "big event meta
header" as that is 8 bytes keeping the payload 8 bytes aligned, even
when a 4 byte header could hold the size of the event
But ring_buffer_event_length() doesn't take this into account and
only subtracts 4 bytes for the meta header in the length when it
should have subtracted 8 bytes
- Have osnoise wait for a full rcu synchronization on unregister
osnoise_unregister_instance() used to call synchronize_rcu() before
freeing its copy of the instance but was switched to kfree_rcu(). The
osniose tracer has code that traverses the instances that it uses,
and inst is just a pointer to that instance. By using kfree_rcu()
instead of synchronize_rcu(), the instance that the inst pointer is
pointing to can be freed while the osnoise code is still referencing
it
That is, a rmdir on an instance first unregisters the tracer. When
the unregister finishes, the rmdir expects that the tracer is
finished with the instance that it is using. By putting back the
synchronize_rcu() in osnoise_unregister_instance() the unregistering
of osnoise will now return when all the users of the instance have
finished
- Remove an unused setting of "ret" in tracing_set_tracer()
- Fix ring_buffer_read_page() copying events
The commit that changed ring_buffer_read_page() to show dropped
events from the buffer itself, split the "commit" variable between
the commit value (with flags) and "size" that holds the size of the
sub-buffer. A cut and paste error changed the test of the reading
from checking the size of the buffer to the size of the event causing
reads to only read one event at a time
- Make tracepoint_printk a static variable
When the tracing sysctl knobs were move from sysctl.c to trace.c, the
variable tracepoint_printk no longer needed to be global. Make it
static
- Fix some typos
- Fix NULL pointer dereference in func_set_flag()
The flags update of the function tracer first checks if the value of
the flag is the same and exits if they are, and then it checks if the
current tracer is the function tracer and exits if it isn't. The
problem is that these checks need to be in a reversed order, as if
the tracer isn't the function tracer, then the flag being checked may
not exist. Reverse the order of these checks
- Fix ufs core trace events to not dereference a pointer in TP_printk()
The TP_printk() part of the TRACE_EVENT() macro is called when the
user reads the "trace" file. This can be seconds, minutes, hours,
days, weeks, and even months after the data was recorded into the
ring buffer. Thus, saving a pointer to an object into the ring buffer
and then dereferencing it from TP_printk() can cause harm as the
object the pointer is pointing to may no longer exist
Fix all the trace events in ufs core to save the device name in the
ring buffer instead of dereferencing the device descriptor from
TP_printk()
- Prevent out-of-bound reads in glob matching of trace events
The filter logic of events allows simple glob logic to add wild cards
to filter on strings. But some events have fields that may not have a
terminating 'nul' character. This may cause the glob matching to go
beyond the string. Change the logic to always pass in the length of
the field that is being matched
- Add no-rcu-check version of trace_##event##_enabled()
The trace_##event##_enabled() usually wraps trace events to do extra
work that is only needed when the trace event is enabled. But this
can hide events that are placed in locations where RCU is not
watching, and can make lockdep not see these bugs when the event is
not enabled
The trace_##event##_enabled() was updated to always test to make sure
RCU is watching to catch locations that may call events without RCU
being active
This caused a false positive for the irq_disabled() and related
events. As that use trace_irq_disabled_enabled() to force RCU to be
watching when the event is enabled via the ct_irq_enter() function,
calls the event, and then calls ct_irq_exit() to put RCU back to its
original state
The trace_irq_disabled_enabled() should not trigger a warning when
RCU is not watching because the code within its block handles the
case properly. Make a __trace_##event##_enabled() version for this
event to use that doesn't check RCU is watching as it handles the
case when it isn't
- Fix use-after-free in user_event_mm_dup()
When the enabler is removed from the link list, it is freed
immediately. But it is protected via RCU and needs to be freed after
an RCU grace period. Use queue_rcu_work() so that the event_mutex can
also be taken as user_event_put() takes the mutex on the last
reference is released
- Free type string in error path of parse_synth_field()
There's an error path in parse_synth_field() where the allocated type
string is not freed
- Add selftest that tests deferred event teardown
- Fix leak in error path of trace_remote_alloc_buffer()
If page allocation fails, the desc->nr_cpus is not incremented for
the current CPU and the allocations done for it are not freed
- Fix allocation length in trace_remote_alloc_buffer()
The logic to calculate the struct_len was doing a double count and
setting the value too large. Calculate the size upfront to fix the
error and simplify the logic
- Fix sparse CPU masks in ring_buffer_desc()
If there are sparse CPUs (gaps in the numbering), the
ring_buffer_desc() will fail as it tests the CPU number against the
number of CPUs that are used
* tag 'trace-v7.2-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
ring-buffer: Allow sparse CPU masks in ring_buffer_desc()
tracing/remotes: Fix struct_len in trace_remote_alloc_buffer()
tracing/remotes: Fix leak in trace_remote_alloc_buffer() error path
selftests/user_events: Wait for deferred event teardown after unregister
tracing/synthetic: Free type string on error path
tracing/user_events: Fix use-after-free in user_event_mm_dup()
tracing: Add a no-rcu-check version of trace_##event##_enabled()
tracing: Prevent out-of-bounds read in glob matching
ufs: core: tracing: Do not dereference pointers in TP_printk()
tracing: Fix NULL pointer dereference in func_set_flag()
samples: ftrace: Fix typos in benchmark comment
tracing: Make tracepoint_printk static as not exported
ring-buffer: Fix ring_buffer_read_page() copying only one event per page
tracing: Remove unused ret assignment in tracing_set_tracer()
tracing/osnoise: Call synchronize_rcu() when unregistering
ring-buffer: Fix event length with forced 8-byte alignment
tracing/synthetic: Free pending field on error path
Pull perf events fixes from Ingo Molnar:
- Fix SVM #GP on AMD CPUs that LBR but not BRS (Sandipan Das)
- Fix UAF bug in the perf AUX code (Lee Jia Jie)
- Fix address leakage in the AMD LBR code (Sandipan Das)
- Fix address leakage in the AMD BRS code (Sandipan Das)
* tag 'perf-urgent-2026-07-11' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
perf/x86/amd/brs: Fix kernel address leakage
perf/x86/amd/lbr: Fix kernel address leakage
perf/aux: Fix page UAF in map_range()
perf/x86/amd/core: Avoid enabling BRS from the SVM reload path
Pull timer fix from Ingo Molnar:
- Fix a subtle posix-cpu-timers vs. exec() race, which
unearthed other races in the area (Thomas Gleixner)
* tag 'timers-urgent-2026-07-11' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
posix-cpu-timers: Prevent UAF caused by non-leader exec() race
Pull audit fixes from Paul Moore:
"Two relatively small audit patches to fix potential data races with
the main audit backlog queue as well as possible integer overflows
when logging data as hex strings"
* tag 'audit-pr-20260710' of git://git.kernel.org/pub/scm/linux/kernel/git/pcmoore/audit:
audit: fix potential integer overflow in audit_log_n_hex()
audit: Fix data races of skb_queue_len() readers on audit_queue
If page allocation fails in trace_remote_alloc_buffer(), desc->nr_cpus
is not yet incremented for the current CPU. As a consequence, on error,
half-allocated rb_desc will not be freed in trace_remote_free_buffer().
Increment desc->nr_cpus as soon as the first allocation for the current
CPU has succeeded.
Link: https://patch.msgid.link/20260709160017.1729517-2-vdonnefort@google.com
Fixes: 96e43537af ("tracing: Introduce trace remotes")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When switching a task's sched_class away from sched_ext, we get the
following sequence of events in __sched_setscheduler():
sched_change_begin()
switched_from_scx()
scx_disable_task(p)
ops.disable(p)
__setscheduler_params()
set_load_weight()
reweight_task_scx(p)
ops.set_weight(p)
p->sched_class = next_class;
sched_change_end()
...
Notably, ops.set_weight() is called _after_ ops.disable().
This violates the expected semantics of the callbacks, the expectation
being that ops.disable() can only be followed by ops.exit_task() or
ops.enable().
Skipping the weight adjustment for disabled tasks should be harmless
since the weight will be recalculated in scx_enable_task() if the task
ever rejoins SCX.
Fixes: 637b068282 ("sched: Fold sched_class::switch{ing,ed}_{to,from}() into the change pattern")
Cc: stable@vger.kernel.org # v6.19+
Signed-off-by: Kuba Piecuch <jpiecuch@google.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
map_range() reads rb->aux_pages[], rb->aux_nr_pages and rb->aux_pgoff via
perf_mmap_to_page() while holding only event->mmap_mutex. Those fields are
serialized by rb->aux_mutex, and mmap_mutex is per event.
Thus, two events sharing one rb via PERF_EVENT_IOC_SET_OUTPUT can race
rb_alloc_aux() with map_range(), leading to a page-UAF scenario as follows:
CPU 0 CPU 1
===== =====
rb_alloc_aux() map_range()
[1]: allocate rb->aux_pages[0]
[2]: rb->aux_nr_pages++
[3]: perf_mmap_to_page()
returns rb->aux_pages[0]
[4]: map it as VM_PFNMAP
[5]: rb->aux_pgoff = 1
munmap the page
[6]: free rb->aux_pages[0]
Pages mapped as VM_PFNMAP have no refcount protection, so CPU 1 holds a
mapping to a freed physical frame.
Fix this by taking rb->aux_mutex across the page walk in map_range().
Fixes: b709eb872e ("perf: map pages in advance")
Signed-off-by: Lee Jia Jie <jiajie.lee@starlabs.sg>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: stable@vger.kernel.org
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
scx_alloc_and_add_sched() publishes @sch through ops->priv before allocating
the cgroup path. If that allocation fails, the unwind path clears ops->priv
and frees @sch immediately. scx_prog_sched() callers can dereference
ops->priv from RCU context the moment it is set, so freeing without a grace
period can use-after-free a concurrent kfunc caller.
Move the publication below the cgroup path allocation so that every failure
path after publication frees @sch through kobject_put(), whose release path
defers the freeing by a grace period.
Fixes: 105dcd005b ("sched_ext: Introduce scx_prog_sched()")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
