Consider the following program:
r1 = map_value; /* low 32 bits are zero at runtime */
r6 = 0xdead000000000000;
if w1 != 0 goto l1;
l0: r1 += r6;
r2 = *(u64 *)(r1 + 0);
exit;
l1: r6 = 0;
goto l0;
At the moment is_branch_taken() reports the jump as always taken,
because it does not distinguish between BPF_JMP and BPF_JMP32
comparisons when processing 'if w1 != 0 ...'.
Fixes: cac616db39 ("bpf: Verifier track null pointer branch_taken with JNE and JEQ")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260904083325.2083493-5-eddyz87@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
linked_regs_too_many_regs checks that collect_linked_regs() ties at most
LINKED_REGS_MAX registers for a single jump. Compare r5 instead of r0,
so that the register the jump compares is itself the member that does
not fit, and check that it comes out of the jump unlinked.
W/o the previous patch env->{false,true}_reg{1,2} bring r5's id back and
insn 7 is logged as "R5=scalar(id=1,...)".
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260904083325.2083493-4-eddyz87@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
check_cond_jmp_op() copies the compared registers into
env->{false,true}_reg{1,2} before collect_linked_regs() runs and copies
those snapshots back into both branch states afterwards.
collect_linked_regs() records at most LINKED_REGS_MAX members of a
linked registers group in the jump history and calls clear_scalar_id()
for every member that does not fit. The compared register is not exempt
from that.
As a consequence, sync_linked_regs() might adjust ranges for more
registers than bpf_bt_sync_linked_regs() can propagate precision to.
Collect the linked registers before the snapshots are taken instead.
This might lead to some unnecessary clear_scalar_id's, but from
previous testing situations with many linked registers are
extremely rare.
Fixes: ec1d77cb0e ("bpf: Use bpf_verifier_env buffers for reg_set_min_max")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260904083325.2083493-3-eddyz87@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
reg_not_null() decides that a register holds a non-NULL value by
looking at its type alone. For pointer types that allow arithmetic the
type only guarantees a non-NULL base, in case of an unbound offset
the runtime offset value might still add up to NULL.
Consider the followng program:
r6 = bpf_map_lookup_elem(map, &0); /* present */
if (r6 == 0) return 0;
r7 = bpf_map_lookup_elem(map, &1); /* absent, NULL at runtime */
r8 = r7;
r8 -= r6; /* pointer - pointer: unknown scalar, -r6 */
r8 <<= 1;
r8 >>= 1; /* any non-negative offset is accepted by */
/* check_reg_sane_offset_ptr() */
r6 += r8; /* verifier: map value; runtime: zero */
if (r7 != r6) return 0;
*(u8 *)(r7 + 0); /* r7 is inferred non-NULL, both are zero */
At runtime both registers are zero, the comparison is true and the
load faults with NULL pointer dereference.
Require the offset to be within +-BPF_MAX_VAR_OFF in reg_not_null().
Fixes: cac616db39 ("bpf: Verifier track null pointer branch_taken with JNE and JEQ")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260904083325.2083493-1-eddyz87@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
The netfilter framework is allergic to ip header changing after
validation done by ip/ipv6 stack.
Assert that bpf netfilter programs do not allow skb write access.
Following additional tests are expected to be rejected by verifier:
1. alter skb->len.
2. alter skb->data.
3. prog calls bpf_dynptr_slice_rdwr.
4. alter location returned by dynptr API.
Add following test case for bpf runtime:
- alter skb data via bpf_dynptr_write()
Test checks via __retval() that bpf_dynptr_write() returned nonzero value.
Signed-off-by: Florian Westphal <fw@strlen.de>
Reviewed-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Link: https://lore.kernel.org/r/20260903065845.22762-1-fw@strlen.de
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Kumar Kartikeya Dwivedi says:
====================
Misc bug fixes - part 2
A set of miscellaneous fixes for bugs reported by Nicholas, plus some
new findings by GPT-5.6-Sol and Sashiko. See commit logs for details.
====================
Link: https://patch.msgid.link/20260903214758.2727663-1-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add verifier coverage for the callback restriction on legacy packet
loads. Exercise BPF_LD_ABS directly in a bpf_loop callback and
BPF_LD_IND from a static subprogram called by the callback, ensuring that
callback context follows nested static calls.
Also exercise a callback which reaches BPF_LD_IND through a global
function and its static descendant. A sibling success case calls the same
global chain outside a callback, preserving support for ordinary global
packet loads. Existing success cases continue to cover loads from ordinary
static subprograms.
The failure cases expect the policy-specific rejection instead of reaching
the implicit-return path, triggering a verifier warning, or being accepted
through a function boundary.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903214758.2727663-9-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
check_ld_abs() models a failed BPF_LD_ABS or BPF_LD_IND in a
subprogram as an implicit return with R0 set to zero. It calls
prepare_func_exit() to explore this synthesized path.
When the load is reached directly from a synchronous callback,
prepare_func_exit() enforces the callback return contract and marks R0
precise. R0 is not derived from a real instruction on this path, so
precision backtracking reaches the callback call with R0 still requested
and triggers the "callback unexpected regs" verifier bug. A privileged
program loader can therefore cause a verifier warning and an -EFAULT
BPF_PROG_LOAD.
These legacy packet-load instructions are deprecated. Reject them from
callbacks rather than complicating their implicit-return model. Check all
active frames before constructing the implicit return so nested static
subprograms cannot hide the callback context.
Global functions are verified independently with a fresh frame zero, so
an active-frame check cannot identify a global function called from a
callback. Also check the complete subprogram call graph during stack-depth
validation and reject a function containing a legacy load when any caller
is a callback. This covers global and static descendants without making
has_ld_abs transitive, preserving its per-function BTF return-type check.
Ordinary uses outside callbacks remain supported.
Fixes: ee861486e3 ("bpf: Fix ld_{abs,ind} failure path analysis in subprogs")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://lore.kernel.org/bpf/20260903152147.C0E241F00A3A@smtp.kernel.org
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903214758.2727663-8-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add verifier coverage for the sleepable bpf_get_stack() and
bpf_get_task_stack() implementations. Call each helper while preemption is
disabled and require the verifier to reject it as sleepable.
Both programs load when the prototypes lack might_sleep, so the
expected-failure tests fail. Keep success controls outside the
non-preemptible region to ensure ordinary calls from sleepable uprobes
remain valid.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903214758.2727663-7-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
The faultable variants of bpf_get_stack() and bpf_get_task_stack() pass
may_fault=true into the common stack collection code. Resolving user-space
build IDs may then call build_id_parse_file() and block on filesystem
reads.
Neither helper prototype sets might_sleep. Since prototype selection uses
the sleepability of the whole program, the verifier can still allow these
helpers from a non-sleepable region within that program, such as an
explicit RCU or preemption-disabled region. The task-stack helper can also
be called from a non-sleepable timer callback of a sleepable program.
Mark both faultable prototypes as sleepable. The existing helper context
check then rejects these calls while continuing to allow them in genuinely
sleepable contexts.
Fixes: d4dd9775ec ("bpf: wire up sleepable bpf_get_stack() and bpf_get_task_stack() helpers")
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260903214758.2727663-6-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add an expected failure case which calls
bpf_btf_find_by_name_kind() from a BPF timer callback. Without the
helper prototype being marked sleepable, the verifier accepts the
program and the load unexpectedly succeeds.
Also add a positive control which calls the helper directly from a
syscall program. This verifies that marking the helper sleepable only
rejects it in non-sleepable regions.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903214758.2727663-5-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
When bpf_btf_find_by_name_kind() finds a type in module BTF, it
returns a new BTF object fd through __btf_new_fd(). This reaches
anon_inode_getfd(), which can sleep while allocating or expanding the
current task fd table.
The helper prototype does not set might_sleep, so the verifier allows
the helper in non-sleepable contexts such as BPF timer callbacks. The
fd allocation can then sleep in softirq context and install the fd into
the interrupted task.
Mark the helper as sleepable. This preserves calls from the main body
of a sleepable syscall program while rejecting calls from its
non-sleepable regions.
Fixes: 3d78417b60 ("bpf: Add bpf_btf_find_by_name_kind() helper.")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://lore.kernel.org/bpf/20260903155150.D57251F000E9@smtp.kernel.org
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260903214758.2727663-4-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add a verifier failure case where an rbtree comparator enters two nested
static subprograms and the innermost subprogram unlocks and relocks the
tree. Restoring the lock keeps the surrounding callback state balanced,
so the test specifically exercises whether the callback restriction follows
the nested calls.
Also add a load-only positive control whose comparator calls a harmless
static subprogram. This preserves the intended support for verified static
subprogram calls while holding the tree lock.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903214758.2727663-3-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
bpf_rbtree_add() invokes its comparator while the caller holds the root
lock. The native insertion code retains raw parent and link pointers across
the callback, so the verifier prohibits unlocking, consuming tree nodes,
or changing RCU state from that callback.
in_rbtree_lock_required_cb() only checks the innermost verifier frame.
Static subprogram calls are permitted while holding a spin lock, and such a
call pushes a frame without in_callback_fn set. Consequently, all callback
restrictions disappear in the nested frame. The subprogram can unlock the
tree, remove and drop the node being compared, then relock. Native insertion
resumes with the stale parent pointer and links freed memory into the tree.
Walk all active frames for the rbtree callback instead. Benign static
subprograms remain permitted, while callback restrictions follow execution
into nested frames.
Fixes: a44b1334aa ("bpf: Allow calling static subprogs while holding a bpf_spin_lock")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260903214758.2727663-2-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
mark_fastcall_pattern_for_call() must ensure that matched
"spill; call; fill" instruction series is not interrupted by a jump.
Otherwise the rewrite applied by bpf_remove_fastcall_spills_fills()
is not sound.
Record the instructions targeted by jumps in
insn_aux_data[*].jump_target when the CFG is built and use this flag
to stop growing a pattern at such an instruction. Jumps to the first
spill are fine.
Note that existing insn_aux_data[*].jmp_point field can't be reused,
as it marks subprogram return instructions.
Fixes: 5b5f51bff1 ("bpf: no_caller_saved_registers attribute for helper calls")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260903205820.1743087-1-eddyz87@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
bpf_convert_ctx_accesses() rewrites an atomic on an arena pointer from
BPF_STX | BPF_ATOMIC to BPF_STX | BPF_PROBE_ATOMIC, this patch adjusts
print_bpf_insn() to print such instructions as regular atomics with a
'probe_' prefix (instead of printing them as BUG_XX).
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260903171542.1438050-2-eddyz87@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
bpf_convert_ctx_accesses() rewrites an atomic on an arena pointer from
BPF_STX | BPF_ATOMIC to BPF_STX | BPF_PROBE_ATOMIC, and it runs before
bpf_opt_subreg_zext_lo32_rnd_hi32().
That pass emits an explicit zero extension for a 32-bit cmpxchg even
when bpf_jit_needs_zext() is false. This is done because on some
architectures 32-bit cmpxchg requires explicit zero extension for the
dst register. E.g. on x86-64 'lock cmpxchg' does not change the %eax
if comparison is successful, while BPF semantics declare that each
operation on a 32-bit register zero extends it's upper half.
is_cmpxchg_insn() matches BPF_MODE == BPF_ATOMIC only, so an arena
cmpxchg misses said zero extension adjustment. This patch adjusts
is_cmpxchg_insn() to match BPF_PROBE_ATOMIC alongside BPF_ATOMIC.
Fixes: d503a04f8b ("bpf: Add support for certain atomics in bpf_arena to x86 JIT")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260903171542.1438050-1-eddyz87@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Kumar Kartikeya Dwivedi says:
====================
Misc bug fixes - part 1
A set of miscellaneous fixes for bugs reported by Nicholas. These are
easy ones and should not require any major discussion, hence batched
together. See commit logs and selftests for details.
====================
Link: https://patch.msgid.link/20260903144433.1716731-1-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
A BPF_PROG_TYPE_SYSCALL program is sleepable, but its bpf_timer callbacks
run in a non-sleepable hrtimer softirq context.
Add verifier cases that call bpf_sys_bpf() and bpf_sys_close() from timer
callbacks. Without the syscall helper prototype annotations these programs
load, so their failure expectations expose the bug.
Also add successful controls that call each helper from the syscall program
main body, ensuring that the intended sleepable use remains accepted.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903144433.1716731-11-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
bpf_sys_bpf() executes the bpf(2) syscall body, which can take mutexes,
allocate with GFP_KERNEL, and wait for an RCU grace period.
bpf_sys_close() reaches close_fd() and filp_close(), which can sleep as
well.
Both helpers are limited to BPF_PROG_TYPE_SYSCALL, whose main program is
sleepable. That does not make every callback sleepable: a syscall program
can register a bpf_timer callback, and the verifier checks that callback
in a non-sleepable context while retaining the syscall helper set.
Without .might_sleep on the prototypes, such a callback can invoke
bpf_sys_bpf() from hrtimer softirq context and trigger a
scheduling-while-atomic failure. bpf_sys_close() is exposed through the
same missing context check.
Set .might_sleep on both prototypes so the existing helper-context check
rejects them from timer callbacks and other atomic regions. Calls from the
sleepable main body remain valid.
Fixes: 79a7f8bdb1 ("bpf: Introduce bpf_sys_bpf() helper and program type.")
Fixes: 3abea08924 ("bpf: Add bpf_sys_close() helper.")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903144433.1716731-10-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add a load-time verifier test that dereferences argument 0 of the
sched_process_wait tp_btf program without checking it. The test expects the
nullable-pointer diagnostic, so it is accepted unexpectedly before the fix
and rejected as expected after it.
Add a successful control that checks the argument for NULL before the
dereference. This ensures the nullable marking preserves legitimate access
to the pid when the tracepoint supplies one.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903144433.1716731-9-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
do_wait() passes wo->wo_pid to the sched_process_wait tracepoint.
kernel_wait4() leaves wo_pid NULL for wait4(-1), and
kernel_waitid_prepare() does likewise for waitid(P_ALL).
btf_ctx_access() currently types argument 0 as PTR_TO_BTF_ID |
PTR_TRUSTED. Without PTR_MAYBE_NULL, the verifier accepts an unchecked
dereference. Trusted pointer loads have no fault protection, so a wait for
any child can then cause a NULL pointer dereference in JITed BPF code.
Add sched_process_wait to raw_tp_null_args[] with argument 0 marked
nullable. The verifier rejects an unchecked dereference while preserving
access after the program checks the pointer for NULL.
Fixes: 838a10bd2e ("bpf: Augment raw_tp arguments with PTR_MAYBE_NULL")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903144433.1716731-8-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add a load-only verifier regression for a resilient lock operation in an
rbtree comparison callback. The program holds the rbtree's regular spin
lock and a separate resilient lock, then releases the resilient lock from
the callback. This isolates the missing kfunc policy check without running
a concurrent tree mutation.
Release the resilient lock before the regular lock on the outer
fall-through. The broken verifier therefore accepts the balanced program,
while the fixed verifier rejects the resilient unlock specifically while
verifying the callback.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903144433.1716731-7-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
__bpf_rbtree_add() keeps parent and link pointers live across calls to the
program-supplied comparison callback. The verifier therefore requires the
root's lock to remain held throughout the callback.
The helper path enforces this rule for bpf_spin_lock() and
bpf_spin_unlock(), but the resilient lock kfunc argument path does not.
Since resilient locks may protect BPF rbtree roots, a callback can release
the root lock and let another CPU remove and free the node referenced by
the in-progress tree walk. The walk then resumes using freed pointers.
Reject resilient lock kfuncs in an rbtree comparison callback, matching
the existing policy for the spin lock helpers. Resilient-lock-protected
trees remain valid when their comparison callbacks leave lock state alone.
Fixes: 0de2046137 ("bpf: Implement verifier support for rqspinlock")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903144433.1716731-6-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
tailcall_callback tests a tail call one static subprogram below a callback.
That reaches the later stack-depth rejection, but it does not exercise the
tail-call helper while the current frame is itself a callback.
Add a callback that calls bpf_tail_call directly and expect the existing
"cannot tail call within callback" diagnostic. On an affected kernel, the
load instead reaches the "callback unexpected regs" verifier bug, so the
expected message is absent and the test fails. The existing ordinary
subprogram case remains a success control for legitimate tail calls.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903144433.1716731-5-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
A tail call from a non-zero frame is modeled as a return from that frame.
The verifier makes R0 unknown and calls prepare_func_exit() for the taken
branch.
When the current frame is a synchronous callback, prepare_func_exit()
enforces the callback return-value contract and marks R0 precise. Since the
tail-call path synthesized R0 rather than deriving it from an instruction,
precision backtracking reaches the callback-calling instruction with R0
still requested and triggers the "callback unexpected regs" verifier bug.
A CAP_BPF task can therefore cause a WARN and an -EFAULT BPF_PROG_LOAD.
Tail calls reachable from callbacks are already rejected later by
check_max_stack_depth(). Reject a tail call made directly by a callback
before constructing the inconsistent return state, using the existing
diagnostic. Tail calls from ordinary subprograms keep their current
behavior.
Fixes: e3245f8990 ("bpf: properly verify tail call behavior")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903144433.1716731-4-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add load-only verifier coverage for the signal_generate and
signal_deliver info arguments. The signal_generate case performs a NULL
check before dereferencing info, ensuring that merely making it nullable
cannot satisfy the test when the nonzero SEND_SIG_PRIV sentinel is used.
Both programs load successfully without the verifier fix, contrary to
their expected-failure annotations. With the fix, info is a scalar and
the attempted dereferences are rejected.
Also add success cases showing that plain raw tracepoint and tp_btf
programs can continue to read and compare the context word as a scalar.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903144433.1716731-3-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
The signal_generate and signal_deliver tracepoints declare their info
argument as a struct kernel_siginfo pointer. btf_ctx_access() therefore
treats it as a trusted pointer for tp_btf programs.
Signal delivery also uses SEND_SIG_NOINFO and SEND_SIG_PRIV as special
values for this argument. Those values are zero and one respectively,
and are not pointers. A tp_btf program can currently dereference either
value and fault the kernel. In particular, signal_generate can run from
timer interrupt context, turning the fault into a kernel panic.
Record both tracepoints in raw_tp_null_args[] and mark argument one as
a non-pointer. This preserves scalar access to the cookie while rejecting
direct and helper-mediated pointer use. Merely marking it nullable would
not suffice because SEND_SIG_PRIV is nonzero.
Fixes: 838a10bd2e ("bpf: Augment raw_tp arguments with PTR_MAYBE_NULL")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903144433.1716731-2-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Jiayuan Chen says:
====================
bpf: Fix NULL-ptr-derefs when showing a void BTF type
This series fixes three NULL-ptr-derefs in BTF handling.
Patch 1 handles the syzbot report. A key-less BTF (btf_key_type_id == 0) used
to be rejected for hash maps, until htab and rhtab gained a ->map_check_btf
(to register a dtor) that does not look at the key, so a key-less hash map is
now accepted. Dumping it through bpffs feeds the key type_id 0 into
btf_type_seq_show() and NULL-derefs in btf_type_show(). Reject it again.
Patches 2 and 3 fix two related, pre-existing crashes reachable via
bpf_snprintf_btf(), which renders a type_id taken straight from the BPF
program against the vmlinux BTF. A "const void" (a modifier resolving to
void) NULL-derefs in btf_modifier_show() - void has no ->show op; a
BTF_KIND_VAR NULL-derefs in btf_var_show() - the vmlinux base BTF has no
resolved_ids. Patch 2 falls back to btf_df_show(), the "<unsupported kind:N>"
placeholder already used for FWD/FUNC/FLOAT/DECL_TAG; patch 3 resolves the
var's type directly, mirroring the existing guard in btf_modifier_show().
Patches 4 and 5 add selftests. Patch 4 checks a key-less hash and rhash map is
rejected at creation. Patch 5 extends the snprintf_btf test to render a
"const void" and a BTF_KIND_VAR from the vmlinux BTF and checks they resolve
without crashing.
v3 -> v4: Drop the pin-and-read reproducer from the key-less map test, fold
the void/VAR test into the existing snprintf_btf test, and simplify patch 2's
wording (review comments).
v2 -> v3: Fold in a third fix for the same class of bug, btf_var_show(),
reported while reviewing v2. Address review comments (Fixes attribution,
verbatim syzbot trace, skip instead of fail).
v1 -> v2: AI reported a pre-exist issue. Let's fold it in this series.
v3: https://lore.kernel.org/bpf/20260831110314.150870-1-jiayuan.chen@linux.dev/
v2: https://lore.kernel.org/bpf/20260830073242.148092-1-jiayuan.chen@linux.dev/
v1: https://lore.kernel.org/bpf/20260828093142.179856-1-jiayuan.chen@linux.dev/
====================
Link: https://patch.msgid.link/20260901104924.346187-1-jiayuan.chen@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Extend the snprintf_btf test with type_ids from the vmlinux BTF that
used to NULL-deref in the BTF show path: a "const void", checked to
render the "<unsupported kind:0>" placeholder, and a BTF_KIND_VAR,
checked to resolve and render without error.
The program renders from its own buffer and the test picks a VAR whose
resolved type fits it, so the render stays in bounds.
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Link: https://lore.kernel.org/r/20260901104924.346187-6-jiayuan.chen@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Create a hash and an rhash map with btf_key_type_id == 0 and expect
bpf_map_create() to fail with -EINVAL; a positive control with a real
key type confirms the rejection is about the key-less BTF and not some
unrelated failure.
Such a map used to be accepted and then NULL-deref in btf_type_show()
when dumped through bpffs.
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Link: https://lore.kernel.org/r/20260901104924.346187-5-jiayuan.chen@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
btf_var_show() calls btf_type_id_resolve() unconditionally, which
dereferences btf->resolved_ids. That is NULL for a base BTF - e.g. the
vmlinux BTF that bpf_snprintf_btf() renders against - since base BTF is
not resolved during parsing. btf_modifier_show() guards this with
'if (btf->resolved_ids)', but btf_var_show() does not.
A BPF program that passes the type_id of a BTF_KIND_VAR from the vmlinux
BTF to bpf_snprintf_btf() thus NULL-derefs:
KASAN: probably user-memory-access in range [0x46638-0x4663f]
RIP: 0010:btf_var_show (kernel/bpf/btf.c:2929)
Call Trace:
<TASK>
btf_type_show (kernel/bpf/btf.c:8259)
btf_type_snprintf_show (kernel/bpf/btf.c:8329)
bpf_snprintf_btf (kernel/trace/bpf_trace.c:1047)
bpf_prog_test_run_raw_tp (net/bpf/test_run.c:829)
__sys_bpf (kernel/bpf/syscall.c:4804)
do_syscall_64 (arch/x86/entry/syscall_64.c:84)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
</TASK>
Resolve the var's type directly with btf_type_skip_modifiers() when
resolved_ids is NULL, mirroring btf_modifier_show().
Fixes: c4d0bfb450 ("bpf: Add bpf_snprintf_btf helper")
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Acked-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://lore.kernel.org/r/20260901104924.346187-4-jiayuan.chen@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
btf_modifier_show() resolves the modifier and then calls
btf_type_ops(t)->show() unconditionally. For the void type (type_id 0,
BTF_KIND_UNKN) kind_ops[] has no entry, so ->show is NULL.
A "const void" (a modifier resolving to void) cannot be a map key or
value - map_check_btf() rejects it because void has no size - so the map
dump path does not reach it. But bpf_snprintf_btf() takes a type_id
straight from the BPF program, and passing such a "const void" from the
vmlinux BTF NULL-derefs:
KASAN: null-ptr-deref in range [0x0000000000000028-0x000000000000002f]
RIP: 0010:btf_modifier_show (kernel/bpf/btf.c:2914)
Call Trace:
<TASK>
btf_type_show (kernel/bpf/btf.c:8251)
btf_type_snprintf_show (kernel/bpf/btf.c:8321)
bpf_snprintf_btf (kernel/trace/bpf_trace.c:1047)
bpf_prog_test_run_raw_tp (net/bpf/test_run.c:829)
__sys_bpf (kernel/bpf/syscall.c:4804)
do_syscall_64 (arch/x86/entry/syscall_64.c:94)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
</TASK>
Fall back to btf_df_show() when the resolved type has no show op; it
emits the "<unsupported kind:N>" placeholder already used for kinds like
FWD and FUNC. bpf_snprintf_btf() then returns the length as usual.
Fixes: c4d0bfb450 ("bpf: Add bpf_snprintf_btf helper")
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Acked-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://lore.kernel.org/r/20260901104924.346187-3-jiayuan.chen@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
map_check_btf() allows a key-less BTF (btf_key_type_id == 0) only for
maps that have a ->map_check_btf callback, and leaves the actual
decision to that callback. Hash maps used to have no ->map_check_btf,
so a key-less BTF was rejected outright.
That changed when htab and rhtab gained a ->map_check_btf to register a
dtor - htab in commit 1df97a7453 ("bpf: Register dtor for freeing
special fields") and rhtab in commit 6905f86012 ("bpf: Allow special
fields in resizable hashtab"). Neither looks at the key, so a key-less
hash map now passes map_check_btf() and gets created. Reading it back
through bpffs feeds the key type_id 0 into btf_type_seq_show();
btf_type_by_id() returns the void type, kind_ops[BTF_KIND_UNKN] is NULL,
and btf_type_show() dereferences it:
RIP: 0010:btf_type_show+0x223/0x2e0 kernel/bpf/btf.c:8232
RSP: 0018:ffffc9000399f868 EFLAGS: 00010206
RAX: dffffc0000000000 RBX: 0000000000000000 RCX: 0000000000000000
RDX: 0000000000000005 RSI: 0000000000000000 RDI: 0000000000000028
RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000000
R10: ffffc9000399f970 R11: 0000000000000001 R12: ffffffff9b96b140
R13: ffffc9000399f8e0 R14: ffff88803d393c00 R15: 0000000000000003
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000200000000000 CR3: 000000003d213000 CR4: 0000000000352ef0
DR0: 0000000039ae8f55 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
<TASK>
btf_type_seq_show_flags+0xca/0x120 kernel/bpf/btf.c:8250
htab_map_seq_show_elem+0x12e/0x350 kernel/bpf/hashtab.c:1669
map_seq_show+0x13d/0x1e0 kernel/bpf/inode.c:293
traverse.part.0.constprop.0+0x107/0x650 fs/seq_file.c:112
traverse fs/seq_file.c:99 [inline]
seq_read_iter+0x93f/0x1270 fs/seq_file.c:196
seq_read+0x344/0x4d0 fs/seq_file.c:163
vfs_read+0x1e4/0xb40 fs/read_write.c:572
ksys_pread64 fs/read_write.c:764 [inline]
__do_sys_pread64 fs/read_write.c:772 [inline]
__se_sys_pread64 fs/read_write.c:769 [inline]
__x64_sys_pread64+0x1eb/0x250 fs/read_write.c:769
do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Reject a key-less BTF in htab_map_check_btf() and rhtab_map_check_btf(),
restoring the previous behavior.
Fixes: 1df97a7453 ("bpf: Register dtor for freeing special fields")
Fixes: 6905f86012 ("bpf: Allow special fields in resizable hashtab")
Reported-by: syzbot+37b56485bbbf90ad8489@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/all/6a8f4e88.27659fcc.2ceef7.0008.GAE@google.com/T/
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Acked-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://lore.kernel.org/r/20260901104924.346187-2-jiayuan.chen@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add a test case for a BPF_PSEUDO_FUNC load instruction that references
the entry function of the program it belongs to. W/o the previous
patch the verifier accepts this program thus allowing a runtime call
at a bogus address. See previous patch for detailed description.
Main function needs to be marked with BTF_FUNC_STATIC for the test to
trigger the bug, the patch uses test_verifier harness instead of
test_prog because libbpf has no way to convey this.
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260902233658.1186477-2-eddyz87@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
fixups.c:jit_subprogs() rewrites BPF_PSEUDO_FUNC loads to contain real
function addresses. This function is invoked from bpf_jit_subprogs()
only when env->subprog_cnt > 1. Meaning that for any program like
below:
int main(void *ctx) {
void *ptr = main;
...
bpf_timer_set_callback(..., ptr);
...
}
The 'ptr' won't be ever converted to contain an address.
In combination with e.g. bpf_timer_set_callback() this would lead to a
function call at a bogus address.
Instead of complicating the implementation, just assume that no useful
program needs main to be a sync or async callback and reject
BPF_PSEUDO_FUNC loads for the main subprogram.
Fixes: 69c087ba62 ("bpf: Add bpf_for_each_map_elem() helper")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260902233658.1186477-1-eddyz87@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
The following BPF program was erroneously accepted by the verifier:
static int cb(int i, __u64 *ctx)
{
/* unsafe on a second iteration */
small_arr[*ctx] = i;
*ctx = 100500;
return 0;
}
int main(void *ctx)
{
int nr_loops = 1;
u64 ctx = 0;
if (unlikely(bpf_get_prandom_u32() == 42))
nr_loops = 2;
bpf_loop(nr_loops, cb, &ctx, 0);
return 0;
}
The branch with nr_loops == 1 was explored first and injected a
checkpoint at the entry to 'cb', such that nr_loops in the main's
frame was not marked as precise. This checkpoint pruned the state with
nr_loops == 2 and the program was accepted.
This test case corresponds to the program above.
Entry point is written in assembly to ensure branch processing order.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260831-bug-015-backtrack-cb-args-precise-v1-2-68a8e2a821e0@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
When processing calls to bpf_loop() verifier marks R1 (and R4) as
precise. R1 tracks loop iterations number and because of the
'callback_depth < R1' mechanics in check_helper_call() must be marked
precise. However, precision propagation for R1 was broken,
when bpf_loop() call was verified on a second iteration.
Consider the following verification trace:
- main: bpf_loop(nr_loops, callback ...)
- callback: BPF_EXIT
- main: bpf_loop(nr_loops, callback ...)
- ...
While the first visit of the call to bpf_loop() propagated R1
precision as expected, the second call to mark_chain_precision() in
the check_helper_call() set R1, but it was immediately reset when
backtrack_insn() processed preceding BPF_EXIT in the loop deleted in
this patch.
Because of that, the second visit of the call to bpf_loop() injected
checkpoint with R1 not marked as precise. Which could trick the
verifier into accepting unsafe programs. See the next patch for an
example of such program.
Commit is structured in a way to minimize conflicts when
'bpf' would be eventually merged with 'bpf-next'.
Fixes: ab5cfac139 ("bpf: verify callbacks as if they are called unknown number of times")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260831-bug-015-backtrack-cb-args-precise-v1-1-68a8e2a821e0@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
A test case checking that the verifier properly backtracks both
fallthrough and implicit subprogram exit paths modelled for
BPF_LD | BPF_ABS instruction.
Without the previous patch:
- the verifier did not call bt_subprog_enter() on the implicit
subprogram exit path;
- bpf_pseudo_call() branch in backtrack_insn() executed
'bpf_bt_set_frame_reg(bt, bt->frame - 1, i);' with bt->frame == 0;
- causing a segmentation fault.
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260901-bug-016-backtrack-ld-abs-v1-2-59368f1be435@gmail.com
Nicholas Carlini reported a bug in precision backtracking mechanism
for BPF_LD | BPF_{IND,ABS} instructions. These instructions are
modelled as two branches:
- fallthrough;
- implicit exit from current subprogram.
The implicit exit case was not handled by the backtrack_insn()
function. When backtracking such a path backtrack_insn() did not
call bt_subprog_enter(), which meant that backtracking continued
manipulating precision marks in a caller frame, while looking at
instructions in a callee frame.
This lead to segmentation faults during verification (see the
selftest), or unsound state pruning.
Fixes: ee861486e3 ("bpf: Fix ld_{abs,ind} failure path analysis in subprogs")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260901-bug-016-backtrack-ld-abs-v1-1-59368f1be435@gmail.com
The LLVM commit c7f4a76da024 [1]
"[InstCombine] fold ((x - 1) | (2^k - 1)) + 1 to (x + (2^k - 1)) & -(2^k)"
caused test_task_local_data.bpf.o to fail verification:
The sequence of 8193 jumps is too complex.
processed 188770 insns (limit 1000000) max_states_per_insn 34
total_states 8238 peak_states 12330 mark_read 0
TLD_ROUND_UP(x, 8) expands to ((((x) - 1) | 7) + 1), exactly the pattern
that [1] rewrites, so the accumulation in __tld_fetch_key()
off += TLD_ROUND_UP(metadata[i].size, 8);
is now compiled as (x + 7) & -8 instead of ((x - 1) | 7) + 1. Both are
correct, but they leave the verifier in very different states. Note that
'off' is marked as precise.
Without [1], "size - 1" wraps at zero (size is a __u16), so
the verifier loses all bounds on the increment:
211: (69) r1 = *(u16 *)(r1 +62) ; R1=scalar(...,umax32=0xffff,var_off=(0x0; 0xffff))
212: (04) w1 += -1 ; R1=scalar(smin=0,smax=umax=0xffffffff,smin32=-1,smax32=0xfffe,var_off=(0x0; 0xffffffff))
213: (44) w1 |= 7 ; R1=scalar(smin=umin=umin32=7,smax=umax=0xffffffff,var_off=(0x7; 0xfffffff8))
214: (0c) w6 += w1 ; R6=scalar(smin=umin=umin32=7,smax=umax=0xffffffff,var_off=(0x7; 0xfffffff8))
215: (04) w6 += 1 ; R6=scalar(smin=0,smax=umax=umax32=0xfffffff8,var_off=(0x0; 0xfffffff8))
Note that 'w6' will be used in the next iteration. In the next iteration
after insn 215, the R6 range will be the same as previous iteration.
The iterator loop converges at depth 2.
With [1] the increment stays precisely bounded at [0, 0x10006]:
211: (69) r9 = *(u16 *)(r1 +62) ; R9=scalar(...,umax32=0xffff,var_off=(0x0; 0xffff))
212: (04) w9 += 7 ; R9=scalar(...,umax32=0x10006,var_off=(0x0; 0x1ffff))
213: (54) w9 &= 131064 ; R9=scalar(...,umax32=0x10006,var_off=(0x0; 0x1fff8))
214: (0c) w9 += w6 ; R9=scalar(...,umax32=0x10006,var_off=(0x0; 0x1fff8))
215: (bf) r1 = r10
216: (07) r1 += -8
217: (85) call bpf_iter_num_next
218: (bc) w6 = w9
In the next iteration, we will have
211: (69) r9 = *(u16 *)(r1 +62) ; R9=scalar(...,umax32=0xffff,var_off=(0x0; 0xffff))
212: (04) w9 += 7 ; R9=scalar(...,umax32=0x10006,var_off=(0x0; 0x1ffff))
213: (54) w9 &= 131064 ; R9=scalar(...,umax32=0x10006,var_off=(0x0; 0x1fff8))
214: (0c) w9 += w6 ; R9=scalar(...,umax32=0x2000c,var_off=(0x0; 0x3fff8))
...
so 'off' umax grows by 0x10006 on every iteration and the loop-head
state never repeats:
218: (bc) w6 = w9 ; R6=scalar(...,umax32=0x10006,var_off=(0x0; 0x1fff8))
218: (bc) w6 = w9 ; R6=scalar(...,umax32=0x2000c,var_off=(0x0; 0x3fff8))
218: (bc) w6 = w9 ; R6=scalar(...,umax32=0x30012,var_off=(0x0; 0x3fff8))
...
218: (bc) w6 = w9 ; R6=scalar(...,umax32=0xff95fd6,var_off=(0x0; 0xffffff8))
That last one is iterator depth 4090. Saturating umax would take ~65531
iterations; the verifier gives up long before that.
Note the loop does not diverge from the start. widen_imprecise_scalars()
blows 'off' up to an unbounded scalar while it is still imprecise, and that
alone converges the first three passes through the loop at depth 4.
Once mark_chain_precision() reaches the loop body, maybe_widen_reg() starts
skipping the register, and no widening ever happens again. In the failing
log widening fires exactly 6 times out of 4098 arrivals at the iter_next()
checkpoint, all of them before the umax starts accumulating.
With [1] and this fix, here is one full trip through the loop body,
entered with 'off' (R6) already clamped by the previous iteration:
208: frame1: R6=scalar(...,umax32=4088,var_off=(0x0; 0xff8))
208: (67) r7 <<= 6 ; R7=scalar(...,umax32=3968,var_off=(0x0; 0xfc0))
209: (bf) r1 = r9 ; R1=mem(id=54,sz=4036,imm=4)
210: (0f) r1 += r7
211: (69) r1 = *(u16 *)(r1 +62) ; R1=scalar(...,umax32=0xffff,var_off=(0x0; 0xffff))
212: (04) w1 += 7 ; R1=scalar(...,umax32=0x10006,var_off=(0x0; 0x1ffff))
213: (54) w1 &= 131064 ; R1=scalar(...,umax32=0x10006,var_off=(0x0; 0x1fff8))
214: (0c) w1 += w6 ; R1=scalar(...,umax32=0x10ffe,var_off=(0x0; 0x1fff8))
R6=scalar(...,umax32=4088,var_off=(0x0; 0xff8))
215: (bc) w6 = w1 ; R6=scalar(...,umax32=0x10ffe,var_off=(0x0; 0x1fff8))
216: (26) if w1 > 0xff8 goto pc+1 ; R6=scalar(...,umax32=4088,var_off=(0x0; 0xff8))
217: (05) goto pc-27
This makes the loop body a fixpoint. 'off' (w6) enters at 208 as [0, 4088] with
var_off=(0x0; 0xff8); the increment computed at 212/213 is [0, 0x10006], so
214/215 leave it at [0, 0x10ffe]; then 216 truncates it straight back to
[0, 4088]/(0x0; 0xff8), and only then is the back edge at 217 taken.
Convergence no longer depends on the widening window above. Verification converges
at iterator depth 3.
[1] https://github.com/llvm/llvm-project/pull/216436
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/r/20260828170534.1011183-1-yonghong.song@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
states.c:__clean_func_state() can downgrade scalar zero spill to
STACK_ZERO in the following case:
*(u64 *)(r10 - 8) = 0;
... checkpoint ...
r1 = *(u32 *)(r10 - 4);
... no reads from r10-8 ...
Here 4 bytes at r10-8 are dead and verifier changes scalar spill to a
combination: 0000pppp (p stands for poison). Such a change breaks
precision propagation chains. All places that produce STACK_ZERO
should call bpf_mark_chain_precision() for the zero source.
This patch fixes the bug in a simplest way possible:
avoids converting stack spills of zero to STACK_ZERO.
Two smarter approaches are possible:
- do bpf_mark_chain_precision() from __clean_func_state()
- check slot liveness information in check_stack_write_fixed_off()
I investigated both and the changes required are a bit tricky,
hence go with a simple fix for the time being.
Fixes: be23266b4a ("bpf: 4-byte precise clean_verifier_state")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260827-bug-011-cleanfunc-stack-zero-simple-v1-v1-1-c0e996589a52@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Nicholas Carlini reported a bug when verifier can incorrectly infer
that a pointer is non-null. The bug occurs when two pointers are
compared and one of them has a type w/o PTR_MAYBE_NULL flag,
but which allows a value to be NULL at runtime.
Here is an example:
// `a` is PTR_TO_MEM | MEM_RDONLY | PTR_UNTRUSTED
// `a` is 0 at runtime.
// `b` is PTR_TO_MAP_VALUE | PTR_MAYBE_NULL
void *a = bpf_rdonly_cast(0, 0);
int *b = bpf_map_lookup_elem(...);
if (a == b)
*b = 42; // verifier does not catch null pointer dereference
This happens because of a special case in check_cond_jmp_op(),
which attempts to strip PTR_MAYBE_NULL flags from pointer types,
when processing comparisons like `rA == rB`, if either rA or rB can't
be null.
The non-null property is derived based on the absence of
PTR_MAYBE_NULL flag on rA's or rB's type. But that is not sufficient
for types like PTR_TO_MEM, as in the example.
This patch replaces type_may_be_null() call with reg_not_null(),
which contains an allowlist of types for which absence of
PTR_MAYBE_NULL actually means that the value can't be NULL at runtime.
At the moment, the list in the reg_not_null() omits two types for
which PTR_MAYBE_NULL is applicable: PTR_TO_XDP_SOCK and PTR_TO_BUF.
In order to remain backward compatible, and assuming that only
comparison between pointers of the same type makes sense,
this commit extends reg_not_null(). W/o such an extension e.g.
verifier_jeq_infer_not_null/null_ptr_to_map_value fails.
reg_not_null() can be extended further, but I deem that out of scope
for the fix at hand. Explicit base_type(...) != PTR_TO_BTF_ID
checks in the check_cond_jmp_op() can be removed with migration to
reg_not_null(), but that is a behavioural change, as the special case
would start matching for PTR_TO_BTF_ID that is also is_trusted_reg().
I omit the behavioural change from this commit.
Fixes: befae75856 ("bpf: propagate nullness information for reg to reg comparisons")
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260826-bug-029-bad-non-null-inference-v2-1-136789ace9e9@localhost
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Per-CPU array, hash, and cgroup storage map updates without BPF_F_CPU
or BPF_F_ALL_CPUS use a value buffer whose per-CPU slots are packed in
possible-CPU order. The buffer is sized as:
round_up(value_size, 8) * num_possible_cpus()
The update paths iterate over possible CPUs, but use the logical CPU ID
to calculate the source offset:
value + size * cpu
This only works when possible CPU IDs are contiguous starting at zero.
For example, with a possible CPU mask of 0,2-3, the buffer contains
three slots corresponding to CPUs 0, 2, and 3. CPU2 is therefore
expected to use slot 1 and CPU3 slot 2. Instead, the current code uses
slots 2 and 3 respectively, causing incorrect per-CPU values and an
out-of-bounds read from the update buffer for CPU3.
The corresponding lookup paths already use a dense offset while
iterating over possible CPUs. Do the same for the array, hash, and
cgroup storage update paths, advancing the source offset once for each
possible CPU. BPF_F_ALL_CPUS continues to use the same value for every
CPU.
Fixes: 8eb76cb03f ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags support for percpu_array maps")
Fixes: c6936161fd ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags support for percpu_hash and lru_percpu_hash maps")
Fixes: 47c79f05aa ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags support for percpu_cgroup_storage maps")
Signed-off-by: Hui Su <sh_def@163.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260813155131.1022745-3-sh_def@163.com
BPF_F_CPU stores the target CPU ID in the upper 32 bits of the map
operation flags. bpf_map_check_op_flags() currently compares that ID
with num_possible_cpus(), which is the number of possible CPUs rather
than a bound on CPU IDs.
On an arm64 QEMU guest with a CPU device-tree hole, the possible CPU
mask was 0,2-3. A userspace program using raw bpf() syscalls creates
a BPF_MAP_TYPE_PERCPU_ARRAY and performs update and lookup operations
for each CPU by setting BPF_F_CPU and the CPU ID in the flags.
With the old check, CPU 1 is incorrectly accepted while valid CPU 3 is
rejected with -ERANGE. The CPU 1 update then reaches the per-CPU map
access path and triggers:
Unable to handle kernel paging request at virtual address ...
pc : __pi_memcpy_generic+0x5c/0x22c
lr : bpf_percpu_array_update+0x2dc/0x2e8
Call trace:
__pi_memcpy_generic
bpf_map_update_value
map_update_elem
__sys_bpf
Check the CPU ID against nr_cpu_ids and cpu_possible() instead. This
rejects CPU IDs outside the valid range and CPUs absent from the
possible mask, while allowing valid sparse CPU IDs.
Fixes: 2b421662c7 ("bpf: Introduce BPF_F_CPU and BPF_F_ALL_CPUS flags")
Signed-off-by: Hui Su <sh_def@163.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260813160858.1042834-3-sh_def@163.com
Eduard Zingerman