Commit 814cba835e ("bpf, x86: Fix trampoline stack size for 128-bit
arguments") changed the x86 trampoline to compute the number of
registers from arg_size for every argument, which removed the last user
of BTF_FMODEL_STRUCT_ARG. No other architecture or verifier code looks
at the flag, so remove the macro and the code in __get_type_fmodel_flags()
which sets it.
Keep BTF_FMODEL_SIGNED_ARG at BIT(1) rather than renumbering it to
BIT(0), so BIT(0) is available for a future flag.
No functional change.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/bpf/20260803052726.2821447-1-yonghong.song@linux.dev
Currently, bpf_setsockopt() and bpf_getsockopt() for SOL_IP and SOL_IPV6
only allow a small subset of socket options (such as IP_TOS,
IPV6_TCLASS, and IPV6_AUTOFLOWLABEL). Calling bpf_setsockopt() with
IP_TRANSPARENT or IPV6_TRANSPARENT fails with -EINVAL.
Transparent proxying (TPROXY) and related networking components often
rely on IP_TRANSPARENT and IPV6_TRANSPARENT to enable binding sockets
to non-local IP addresses.
Allow IP_TRANSPARENT for SOL_IP in sol_ip_sockopt() and
IPV6_TRANSPARENT for SOL_IPV6 in sol_ipv6_sockopt().
Signed-off-by: Shivaji Kant <shivajikant@google.com>
Tested-by: Anubhav Singh <anubhavsinggh@google.com>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://lore.kernel.org/bpf/20260801051307.478469-1-shivajikant@google.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Reported by the Sashiko AI review.
arena_vm_fault() returns VM_FAULT_RETRY when it can't take
arena->spinlock, but it never took mmap_lock. The fault path assumes a
VM_FAULT_RETRY handler already dropped mmap_lock and re-takes it on the
retry, so mmap_lock gets taken twice and can deadlock:
do_user_addr_fault()
{
fault = handle_mm_fault(...); // calls arena_vm_fault()
if (fault & VM_FAULT_RETRY)
goto retry; // re-locks mmap_lock
mmap_read_unlock(mm);
}
Return VM_FAULT_SIGBUS instead, for two reasons:
1. We could keep VM_FAULT_RETRY, but then we'd have to drop the fault
lock first and cap the retry ourselves, the way __folio_lock_or_retry()
does.
2. A failed raw_res_spin_lock_irqsave() already means a possible deadlock
was detected, so retrying just hits the same lock again.
So returning VM_FAULT_RETRY here is overkill.
Fixes: b8467290ed ("bpf: arena: make arena kfuncs any context safe")
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260728060517.95183-1-jiayuan.chen@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Amery Hung says:
====================
Generate bpf_func_proto for kfunc
Hi,
This is the second of three patch sets to unify kfunc and helper
argument verification. It:
1) further aligns the kfunc and helper argument checks,
2) makes kfunc argument type classification depend solely on BTF, and
3) generates a bpf_func_proto for each kfunc.
With classification now a pure function of the kfunc's BTF, it is computed
once at add-call time and cached in the generated bpf_func_proto, rather
than re-derived on every verification of the call. Along the way it also
fixes a few issues.
The next patch set will align the argument register compatibility checks
and route helper and kfunc argument verification through a single shared
function.
[1/3] https://lore.kernel.org/bpf/20260715064047.1793790-1-ameryhung@gmail.com/
Changelog
v2 -> v3:
- Drop a patch that introduces SCALAR_MAYBE_ZERO (Eduard)
- Drop patch make helper handle mem+size at mem arg, and instead make
kfunc also handle mem+size at size
- patch 5: New patch replacing temporary mark_ptr_not_null_reg hack
with refine_ptr_not_null_reg (Eduard)
- patch 8: Only allow global subprog to read poisoned stack slots
(Eduard)
- patch 11: Test a precision gap when passing NULL to nullable-mem +
size arg (Eduard)
- patch 15: Reorganize BTF_ID, MEM, MEM+SIZE classification for
clarity (Eduard)
- patch 18: Emded bpf_func_proto in bpf_kfunc_desc and dynamically
resize bpf_kfunc_desc_tab; Record saved_dst_prog_type early in
bpf_prog_load to avoid introducing a fallback logic in
resolve_prog_type (Eduard)
Link: https://lore.kernel.org/bpf/20260724190813.1458271-1-ameryhung@gmail.com/
v1 -> v2:
- patch 2: use reg_arg_name() for the map-mismatch message; derive the
object register correctly on both helper and kfunc paths
- patch 3: reject non-CONST_PTR_TO_MAP regs (base_type check) to fix
map-value type confusion
- patch 15: also reject referenced regs with unsafe modifiers (e.g.
MEM_PERCPU)
- patch 17: reject non-SCALAR_VALUE for KF_ARG_MEM_SIZE
====================
Link: https://patch.msgid.link/20260801074633.1595644-1-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Kfunc argument checking re-derives each argument's kfunc_ptr_arg_type
from BTF on every verification of a call in check_kfunc_args(). Now that
get_kfunc_arg_type() is a function of the kfunc's BTF alone, it no
longer inspects register state. The classification can be computed once
when the call is added and cached. This is a step toward describing
kfuncs with a bpf_func_proto and sharing the helper argument-checking
path.
Generate the classification at bpf_add_kfunc_call() time:
- Extend struct bpf_func_proto to be able to describe a kfunc: widen
arg_type[] and the arg_btf_id[]/arg_size[] union from 5 to
MAX_BPF_FUNC_ARGS, since a kfunc may take up to 12 arguments (5 in
registers, 7 on the stack).
- Embed a bpf_func_proto in struct bpf_kfunc_desc, populated by
gen_kfunc_arg_proto() which runs get_kfunc_arg_type() for each
argument and stores the result in proto.arg_type[]. Grow the
descriptor table's descs[] as a flexible array to not waste memory.
- check_kfunc_args() reads the cached classification from meta->fn
The KF_ARG_PTR_TO_CTX classification depends on the resolved program type,
and for BPF_PROG_TYPE_EXT that is the target program's type, which
resolve_prog_type() reads from prog->aux->saved_dst_prog_type. That field
is normally recorded later during verification in check_attach_btf_id(),
after bpf_add_kfunc_call() has run.
Record saved_dst_prog_type and saved_dst_attach_type from dst_prog at
program load time in bpf_prog_load() so the resolved type is available
at add-call time without reordering check_attach_btf_id(). This keeps
e.g. an freplace of an XDP program calling bpf_xdp_metadata_rx_hash()
classifying its struct xdp_md * argument as context.
The classification result is unchanged; it is only computed earlier and
cached.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-19-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Add kfunc scalar argument types, classify them in get_kfunc_arg_type()
along side with pointer arguments and move scalar type verification
into the main switch in check_kfunc_args(). This keeps BTF-based
classification separate from register validation for every argument,
paving the way for generating the kfunc argument prototype at add-call
time. No functional change intended.
KF_ARG_MEM_SIZE and KF_ARG_CONST_MEM_SIZE now are reachable. Therefore,
remove the fallthrough from KF_ARG_PTR_TO_MEM case and adjust the
register indexing.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-18-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Now that get_kfunc_ptr_arg_type() classifies a kfunc pointer argument
from its BTF alone, express a nullable argument by OR-ing PTR_MAYBE_NULL
into the classified type, and resolve a NULL register after
classification instead of before it.
Previously check_kfunc_args() short-circuited a nullable argument passed
a NULL register with a continue placed before get_kfunc_ptr_arg_type(),
so the NULL never reached classification. That kept a register-state
decision (bpf_register_is_null()) ahead of the BTF-based classification.
This mirrors how helper arguments carry PTR_MAYBE_NULL in their
bpf_arg_type and is a step toward describing kfuncs with a bpf_func_proto:
the nullability now travels with the per-argument classification, so it is
captured when the prototype is generated at add-call time.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-17-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
get_kfunc_ptr_arg_type() decided part of a kfunc pointer argument's type
from the caller's register: a PTR_TO_BTF_ID (or reg2btf_ids) register made
the argument KF_ARG_PTR_TO_BTF_ID, otherwise it fell through to a memory
buffer. Folding register state into argument classification prevents
describing a kfunc's arguments from its BTF alone, which is a prerequisite
for generating a helper-like prototype and eventually sharing the argument
checking (check_func_arg()) between helpers and kfuncs.
Classify pointer arguments from BTF only, and resolve them against the
register in check_kfunc_args():
- A pointer to a struct that is not paired with a __sz/__szk size
argument is classified KF_ARG_PTR_TO_BTF_ID and then checked against
the register. A register carrying a BTF ID (PTR_TO_BTF_ID or a
reg2btf_ids type) must be referenced or trusted and is matched against
the expected type. The only relaxation is when the struct is composed
of scalars, the register may be verified as a fixed-size memory buffer
sized from the BTF type; anything else is rejected.
- A pointer paired with a size argument is always a memory buffer and is
never classified as BTF_ID, so the __sz/__szk case no longer detours
through BTF_ID.
The new design now accepts one previously rejected case: passing
PTR_TO_BTF_ID to a pointer to scalar w/o a following __sz/__szk. The
argument will be classified as KF_ARG_PTR_TO_MEM | MEM_FIXED_SIZE. The
PTR_TO_BTF_ID register will go through check_mem_reg() ->
check_helper_mem_access() -> check_ptr_to_btf_access(). For a pointer to
scalar arg, a kernel btf id will be rejected unless explicitly granted
by btf_struct_access(); a program allocated btf id will be allowed.
The referenced-or-trusted check thus moves into the KF_ARG_PTR_TO_BTF_ID
resolution, alongside the type match.
get_kfunc_ptr_arg_type() no longer needs the register, so drop its regs
and reg parameters; it is now a pure function of the kfunc's BTF.
When a register cannot satisfy a BTF_ID argument, report the register type
passed and, when the expected struct has a reg2btf_ids mapping, the
register type that would be accepted, instead of a confusing "socket".
Update the affected selftest messages accordingly.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-16-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
A kfunc memory-pointer argument comes in two flavors: a fixed-size buffer
whose access size is derived from the pointed-to BTF type, and a
variable-size buffer paired with a following __sz/__szk size argument.
Both were represented by separate kfunc_ptr_arg_type values
(KF_ARG_PTR_TO_MEM vs KF_ARG_PTR_TO_MEM_SIZE) with the pointer classified
as the latter when a size argument followed.
Mirror how helpers describe the same distinction: classify both as
KF_ARG_PTR_TO_MEM and OR in MEM_FIXED_SIZE for the fixed-size case, just
as helpers use ARG_PTR_TO_MEM | MEM_FIXED_SIZE. The switches now key on
base_type(kf_arg_type) so the flag rides along, and the KF_ARG_PTR_TO_MEM
handler either resolves the size from BTF (MEM_FIXED_SIZE) or falls
through to the mem/size-pair check, which validates the buffer against the
following size register and skips it. No functional change.
Currently, KF_ARG_MEM_SIZE and KF_ARG_CONST_MEM_SIZE are only reachable
from ARG_PTR_TO_MEM fallthrough. A patch later will merge scalar
checking into the same switch and remove the fallthrough.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-15-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
get_kfunc_ptr_arg_type() returned KF_ARG_PTR_TO_NULL when a nullable
pointer argument was passed a NULL register. This folded a register-state
decision (bpf_register_is_null()) into what is otherwise BTF-based
argument classification, and it short-circuited before the BTF_ID/MEM
resolution.
Drop KF_ARG_PTR_TO_NULL and handle the NULL case in check_kfunc_args()
instead: a nullable argument that is actually NULL is skipped. Note that
it is okay to skip even when it is a mem+size pair because the size
argument check has been moved to the scalar section. The skip is done
before get_kfunc_ptr_arg_type() so that a NULL passed to a nullable
non-scalar-struct argument is not newly rejected by the BTF_ID/MEM
resolution.
No functional change.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-14-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
check_kfunc_args() already makes sure a scalar value is passed to a
scalar kfunc argument. Drop the check in is_kfunc_arg_mem_size() and
is_kfunc_arg_const_mem_size() to further decouple
get_kfunc_ptr_arg_type() from register state (a prerequisite for
generating a helper-like prototype from kfunc's BTF).
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-13-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
To align helper and kfunc pointer to memory argument handling, move
kfunc constant memorry size argument handling to the kfunc scalar
section. In addition, factor out constant scalar argument handling.
The constant size argument (__szk) of a kfunc memory/size pair was
recorded into meta->arg_constant by a dedicated block in the
KF_ARG_PTR_TO_MEM_SIZE case, duplicating the "only one constant
argument" and "must be a known constant" checks already in the generic
scalar argument handling. That block also did an explicit i++ to skip
the size argument.
This also fixes a precision gap: the old dedicated block did not mark
the size register precise, relying on check_mem_size_reg() for that. But
check_mem_size_reg() is skipped when the buffer is a nullable arg passed
as NULL (e.g. bpf_dynptr_slice(_rdwr) with a NULL buffer), so in that
case the __szk value was recorded and used for regs[R0].mem_size without
marking it precise. Routing the size through the scalar path marks it
precise in all cases.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-11-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
ARG_CONST_SIZE does not require a constant: check_mem_size_reg() accepts
any bounded scalar and verifies the memory access against its maximum
(reg_umax). Rename ARG_CONST_SIZE and ARG_CONST_SIZE_OR_ZERO to
ARG_MEM_SIZE and ARG_MEM_SIZE_OR_ZERO to reflect that. ARG_CONST_ALLOC_
SIZE_OR_ZERO, which does require a constant, is left unchanged.
Pure rename, no functional change.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-10-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Fixed-size memory arguments went through two paths: helpers called
check_helper_mem_access() directly, while kfuncs and global subprogs
used check_mem_reg(). Route the helper MEM_FIXED_SIZE case through
check_mem_reg() too so all three share the same check.
This also fixes a bug in the helper path. When passing a NULL to
PTR_MAYBE_NULL | ARG_PTR_TO_FIXED_SIZE_MEM argument, the program would
be falsely rejected by check_helper_mem_access(). This is not
triggerable since there is no such kind of helper. Also, note that
check_reg_type() still make sure NULL cannot be passed to an argument
not marked with PTR_MAYBE_NULL.
It also tightens the poisoned-stack-slot check. check_mem_reg() encoded
"a STACK_POISON slot may be read" as a negative access size for any
PTR_TO_STACK argument, but that is only sound for global subprogs, where
static stack liveness proved the callee body does not read those slots
(2cb27158ad ("bpf: poison dead stack slots")). Since check_mem_reg() is
also used for kfuncs, kfuncs accidentally inherited it and could read a
poisoned (dead, possibly uninitialized) stack slot. Restrict the negative
size to global subprogs (meta == NULL) so kfuncs, like helpers, require
the whole argument initialized.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-9-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Helper ARG_CONST_SIZE and kfunc KF_ARG_PTR_TO_MEM_SIZE memory arguments
already share check_mem_size_reg(), but the kfunc path reached it
through a thin wrapper, check_kfunc_mem_size_reg(). The wrapper existed
only to invoke check_mem_size_reg() twice. Once for BPF_READ and once for
BPF_WRITE because a kfunc mem argument may be both read and written,
whereas a helper argument carries a single access direction.
Let check_mem_size_reg() take a bitmask of access directions (widening
access_type to u32) and perform each requested access, then pass
BPF_READ | BPF_WRITE from the kfunc call site. This removes the
check_kfunc_mem_size_reg() wrapper so helper and kfunc mem+size arguments
run through exactly the same code.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-7-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpf_map_lookup_elem() is typed to return PTR_TO_MAP_VALUE for every
map, but for some map kinds the looked up value is actually a
different object: an inner map, a socket or an xsk socket.
Until now this reinterpretation happened once the pointer was
converted from its NULL-able form to a concrete value.
Such reinterpretation logic placement led to mark_ptr_not_null_reg()
being called for a temporary register copy in check_mem_reg() and
check_kfunc_mem_size_reg() (check_mem_size_reg() was buggy because of
not calling it). The temporary copy was necessary to pass
reinterpreted parameters as nullable helper and kfunc arguments.
Avoid this complication by refining map lookup result type right away.
The test case verifier_map_in_map/on_the_inner_map_pointer needs an
update because the verifier now prints a concrete NULL-able type for
the lookup.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-6-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Kfuncs used a single '__map' suffix (KF_ARG_PTR_TO_MAP) for two different
things: a verifier-known map matched by map_uid against a bound
timer/wq/task_work object (bpf_wq_init, bpf_task_work_schedule*), and an
opaque 'struct bpf_map *' used only at runtime (bpf_arena_*), which may be a
map fd or a PTR_TO_BTF_ID struct bpf_map (e.g. a bpf_map iterator's ctx->map).
That combined path only accepted the btf map form due to type confusion. The
'if (!reg->map_ptr)' check reads reg->map_ptr, which aliases reg->btf in the
bpf_reg_state union. A PTR_TO_BTF_ID register always has a non-NULL reg->btf,
so the guard silently passed and validation fell through to
process_kf_arg_ptr_to_btf_id(). It also recorded PTR_TO_BTF_ID info in
meta->map, which would be meaningless.
Split the annotation to avoid such type confusion and to align with
helper:
- '__const_map' -> KF_ARG_CONST_MAP_PTR: verifier-known map, handled by
process_map_ptr_arg() like helper ARG_CONST_MAP_PTR.
- '__map' -> KF_ARG_PTR_TO_BTF_ID: opaque struct bpf_map, validated by
process_kf_arg_ptr_to_btf_id(). A map fd still matches via
reg2btf_ids[CONST_PTR_TO_MAP], so bpf_arena_alloc_pages(&map) keeps
working.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-4-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Both the helper ARG_CONST_MAP_PTR and the kfunc KF_ARG_PTR_TO_MAP
recorded the map pointer in meta->map and, when a map was already
bound by a preceding timer/workqueue/task_work argument, rejected a
mismatching map.
Factor the logic into a single process_map_ptr_arg() used by both
paths. The bound-object name (timer, workqueue, or bpf_task_work) is
derived from the bound map's btf_record, and the register numbers in
the message are computed from the map argument position instead of
being hard-coded.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-3-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
The BPF trampoline preserves only 8 bytes of the target's return value
(R0), so attaching an fexit/fmod_ret/fsession program to a function that
returns a >8 byte value is now rejected by the verifier. Add a bpf_testmod
function returning __int128 and an fexit program that targets it. The
program is expected to fail to load with the "with a >8 byte return value
is not supported for this attach type" message.
A 128-bit __int128 argument is passed in a register pair and occupies two
trampoline context slots. Add a bpf_testmod function taking a leading
__int128 argument followed by an int and a long, and an fexit program that
reads those two trailing arguments and the return value, verifying that the
trampoline reserves enough stack for the 128-bit argument and places the
following arguments and the return value at the right context slots.
__int128 is only available on 64-bit targets (where the compiler defines
__SIZEOF_INT128__). The argument test additionally depends on the calling
convention: x86_64 and arm64 pass an __int128 in a register pair as the
trampoline expects, while other architectures pass it differently (e.g.
s390x passes larger arguments by reference), so that subtest runs only on
x86_64 and arm64 and is skipped elsewhere.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260729050209.2587581-1-yonghong.song@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
btf_distill_func_proto() accepts a function argument up to 16 bytes, so a
128-bit scalar such as __int128 reaches the x86 trampoline with
arg_size == 16. But the current implementation assumes an __int128
argument only needs one register, so the register save area is
under-allocated and save_args() overwrites adjacent stack slots.
Compute the register count from arg_size for all arguments to fix it.
Fixes: a9c5ad31fb ("bpf: x86: Support in-register struct arguments in trampoline programs")
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260729050204.2586457-1-yonghong.song@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
btf_distill_func_proto() builds the function model used for the
fentry/fexit/fmod_ret/fsession trampolines and struct_ops. It has
accepted a 16-byte __int128 return value since the trampoline was
introduced: __get_type_size() returns the integer's type size, and the
return-type check only rejected ret < 0.
But the BPF trampoline preserves only 8 bytes of the return value (RAX on
x86, i.e. R0). For an attach type that reads the target's return value the
second half (RDX / R3) is neither saved nor restored, so a program
attached to a function returning a 16-byte value corrupts the value seen
by the real caller and itself observes only half of it. struct_ops
trampolines have the same limitation.
This affects the attach types that read the target's return value: fexit,
fmod_ret and fsession (plus the _multi variants of fexit and fsession),
and struct_ops. fentry/fentry_multi run before the target returns and are
unaffected.
Reject a >8 byte return value for these attach types in
bpf_check_attach_target() and bpf_check_attach_btf_id_multi(), and for
struct_ops in bpf_struct_ops_desc_init().
Fixes: fec56f5890 ("bpf: Introduce BPF trampoline")
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260729050159.2585809-1-yonghong.song@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Replace silent WARN_ON_ONCE with WARN_ONCE that prints the actual error
code from bpf_trampoline_unlink_prog(). This aids debugging of race
conditions during link teardown, while keeping the warning rate limited
to avoid log flooding.
This will be very helpful for speeding up trouble-shooting of some crash
UAF due to bpf_trampoline_unlink_prog failures.
No change to unlink behavior.
Signed-off-by: Xu Xin <xu.xin16@zte.com.cn>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260729141159128mEJmS_aujBKr-cBu1p_UI@zte.com.cn
In bpf_netns_link_update_prog, the checks for old_prog and prog type
are currently performed locklessly before acquiring netns_bpf_mutex.
This creates a race condition that can lead to a UAF issue.
If two threads concurrently execute BPF_LINK_UPDATE on the same netns
link, the following execution path can trigger a UAF:
CPU0 CPU1
bpf_netns_link_update_prog
if (old_prog && old_prog != link->prog)
return -EPERM;
bpf_netns_link_update_prog
if (old_prog && old_prog != link->prog)
...
old_prog = xchg(&link->prog, new_prog);
bpf_prog_put(old_prog);
if (new_prog->type != link->prog->type) <-- trigger UAF
Fix this by moving the old_prog and prog->type checks inside the
netns_bpf_mutex critical section. Meanwhile, use guard() to simplify
lock management and avoid all the goto jumping.
Fixes: 7f045a49fe ("bpf: Add link-based BPF program attachment to network namespace")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/f87b53c0-8f00-45a6-82db-8242fa9b143f@huaweicloud.com [0]
Link: https://lore.kernel.org/bpf/20260728023259.2813482-1-pulehui@huaweicloud.com
Ihor Solodrai says:
====================
resolve_btfids: Discover kfuncs from BTF ID sets
This series develops resolve_btfids in preparation for bringing
kernel-specific BTF transformations in tree, which will reduce kbuild
dependency on pahole's features.
resolve_btfids currently identifies kfuncs by reading the "bpf_kfunc"
decl tags pahole emits into vmlinux BTF. The series switches the
source of truth to the BTF ID sets registered with
BTF_KFUNCS_START()/END() in the kernel, which is the mechanism BPF
verifier uses.
The series is based on patches #5 and #6 from the original
"resolve_btfids: Implement BTF tags emission for kfuncs" series [1],
and includes a few significant additions. Particularly the build time
enforcement of kfunc flags consistency [2].
The series consists of:
- patches #1-#3 implement supporting infrastructure for the proper
kfunc discovery and deduplication
- patches #4-#5 expose btf__find_by_name_kind_own() and use it to
fix a latent bug in _impl func lookup
- patch #6 fixes inconsistent kfunc flags for HID kfuncs
- patch #7 implements kfunc discovery from BTF ID sets
- patch #8 adds enforcement of kfunc flags consistency
The resolve_btfids selftests patches have landed earlier [3].
[1] https://lore.kernel.org/bpf/20260601221805.821394-1-ihor.solodrai@linux.dev/
[2] https://lore.kernel.org/bpf/9b2196dd-443b-4632-ae11-030cdbdc59b4@linux.dev/
[3] https://lore.kernel.org/bpf/20260617210619.1562858-1-ihor.solodrai@linux.dev/
Tested-by: Alan Maguire <alan.maguire@oracle.com>
---
Changes compared to [1]:
- Drop the idempotency (ensure_*) machinery: assume valid input BTF (Andrii)
- Use rbtree to store the kfuncs (Andrii)
- Enforce full KF_* flag consistency as a build error (Eduard)
- Various cleanups and nits (Andrii, Emil, Jiri)
---
====================
Link: https://patch.msgid.link/20260722233518.778854-1-ihor.solodrai@linux.dev
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
collect_kfuncs() currently uses bpf_kfunc decl tags to identify the
list of kfuncs. The decl tags are generated by pahole, which makes
current implementation implicitly rely on those tags being generated.
The authoritative source, used by the the BPF verifier for kfunc
registration, of functions being BPF kfuncs are
BTF_KFUNCS_START()/END() declarations. These are BTF_ID_SET8 under the
hood. Currently resolve_btfids reads kfunc flags from these sets, and
populates them with BTF IDs.
Implement kfunc discovery from BTF_ID_SET8 symbols in resolve_btfids,
removing the dependency on pahole's emmission of decl tags.
Walk BTF_ID_KIND_SET8 sets, and use the address-to-symbol index to
look up set entry's BTF_ID symbol name (before .BTF_ids is patched),
recording the paired flags directly. This makes find_kfunc_flags()
helper unnecessary, so it's removed.
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260722233518.778854-8-ihor.solodrai@linux.dev
Update kfunc flags for hid_bpf_syscall_kfunc_ids set to exactly match
hid_bpf_kfunc_ids set by adding KF_SLEEPABLE flag.
The syscall set omitted the flag because syscall programs are always
sleepable (the verifier rejects a non-sleepable syscall program).
However the upcoming resolve_btfids change enforces per-kfunc flag
consistency across BTF ID sets at build time, which is why this change
is necessary.
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260722233518.778854-7-ihor.solodrai@linux.dev
process_kfunc_with_implicit_args() skips generating <name>_impl
function when one already exists for backwards compatibility.
It uses btf__find_by_name_kind(), which searches the base BTF before
the split BTF. When resolve_btfids processes a module, a same-named
_impl in vmlinux would be found and the module's own counterpart would
not be created.
Fix by using btf__find_by_name_kind_own() for the lookup.
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260722233518.778854-6-ihor.solodrai@linux.dev
btf__find_by_name_kind() searches the base BTF before the split BTF,
so in case of a name collision between base and split it always
returns a base type. Tools that process split BTF may need to restrict
a lookup to the split's own types.
The internal helper btf__find_by_name_kind_own() already does exactly
that. Make it a public API.
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260722233518.778854-5-ihor.solodrai@linux.dev
Several CI runs failed in the libarena parallel tests with -4 (-EINTR) [1],
which says nothing about what actually went wrong.
Two workers can fail like this:
worker 1: gives up, e.g. the rendezvous times out, sets test_abort and
returns its own error (-ETIMEDOUT)
worker 2: sees test_abort and returns -EINTR
-EINTR only means "someone else already gave up", so it carries no
information. Which of the two gets reported depends on the order
pthread_join() collects them, because
err = err ?: (long)thread_ret;
keeps the first non-zero value and drops the rest. When the -EINTR worker
comes first, the error describing the actual failure is lost.
Skip -EINTR entirely: a worker only returns it once another worker has
already reported the real error, so report and log only the real errors.
It is still unclear whether the timeouts come from CI load or from a
problem in the test itself. Report the error accurately first, so the next
failure can be diagnosed.
[1]:
https://github.com/kernel-patches/bpf/actions/runs/29867905253/job/88764463566https://github.com/kernel-patches/bpf/actions/runs/29878191901/job/88794845824
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260723061347.398591-1-jiayuan.chen@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
There is no particular reason to keep bpf_res_spin_lock() disabled in
tracing programs, since it is safe against reentrancy and deadlocks.
Remove the restriction for tracing programs covered by the predicate
is_tracing_prog_type().
This is a prerequisite before the definition of is_tracing_prog_type()
is updated to include raw_tp, fentry, fexit, and fmod_ret. Existing
tracing programs will be updated to use bpf_res_spin_lock() instead when
it is available.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260719113551.1294284-2-memxor@gmail.com
Use kvmalloc_array() instead of open-coding the element-size
multiplication when allocating the TCP BPF iterator batch.
Signed-off-by: Weimin Xiong <xiongwm2026@163.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Use kvmalloc_array() instead of open-coding the element-size
multiplication when allocating the Unix-domain BPF iterator batch.
Signed-off-by: Weimin Xiong <xiongwm2026@163.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Pull block fixes from Jens Axboe:
- Fix a ublk recovery hang, where END_USER_RECOVERY without a
successful START_USER_RECOVERY could be satisfied by a stale
completion latch
- Fix a stack out-of-bounds read in the CDROMVOLCTRL ioctl
- MAINTAINERS email address update for Roger Pau Monne
* tag 'block-7.2-20260724' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux:
MAINTAINERS: update my email address
cdrom: fix stack out-of-bounds read in CDROMVOLCTRL
ublk: wait on ublk_dev_ready() instead of ub->completion
Pull io_uring fixes from Jens Axboe:
- Fix a missing ERESTARTSYS conversion in the read paths, which got
messed up back when some code consolidation was done for read
multishot support
- zcrx UAPI rename, dropping the abbreviated "notif" naming in favor of
"event" for consistency and to be less ambiguous for users. This was
added for 7.2, so let's rename it while we still can. No functional
or code changes, just a strict rename
* tag 'io_uring-7.2-20260724' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux:
io_uring/zcrx: rename notif to event
io_uring/zcrx: rename ZCRX_NOTIF_NO_BUFFERS
io_uring/zcrx: drop "notif" from stats struct names
io_uring/rw: fix missing ERESTARTSYS conversion in read paths
Pull smb server fixes from Steve French:
"This contains eight ksmbd fixes covering POSIX ACL handling, SMB
signing enforcement, DACL parsing and construction hardening, session
lifetime handling, and validation of malformed transform and
compressed SMB2 requests:
- preserve inherited POSIX ACL mask when creating objects.
- enforce the session signing requirement for plaintext SMB requests.
- harden DACL/ACE processing against size overflows, incomplete ACE
copies, and undersized SIDs.
- defer teardown of a previous session until NTLM authentication
succeeds.
- reject undersized encryption-transform and decompressed SMB2
requests before they can reach normal SMB2 request processing"
* tag 'v7.2-rc4-smb3-server-fixes' of git://git.samba.org/ksmbd:
ksmbd: reject undersized decompressed SMB2 requests
ksmbd: validate minimum PDU size for transform requests
ksmbd: defer destroy_previous_session() until after NTLM authentication
ksmbd: validate ACE size against SID sub-authorities
ksmbd: restore DACL size on check_add_overflow() to avoid malformed ACL
ksmbd: bound DACL dedup walk to copied ACEs
ksmbd: enforce signing required by the session
ksmbd: preserve VFS inherited POSIX ACL mask
Pull bpf fixes from Eduard Zingerman:
- Fix tcp_bpf_sendmsg() error path mistaking a concurrently-freed
sk_psock->cork for the local temporary message and freeing it again
(Chengfeng Ye)
- Reject passing scalar NULL to nonnull arg of a global subprog.
Previously the verifier did not account for the cases directly
passing scalars to a global subprog, e.g.: 'global_func(0);' would
pass even if 'global_func' argument was marked nonnull (Amery Hung)
* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf:
bpf, sockmap: Fix cork use-after-free in tcp_bpf_sendmsg()
selftests/bpf: Test passing scalar NULL to nonnull global subprog
bpf: Reject passing scalar NULL to nonnull arg of a global subprog
tcp_bpf_sendmsg() keeps msg_tx across sk_stream_wait_memory(), which
drops and reacquires the socket lock. Its error path tries to decide
whether msg_tx names the local temporary message by comparing it with
the current value of psock->cork.
This comparison is unsafe when two threads send on the same socket:
Thread A Thread B
msg_tx = psock->cork
sk_msg_alloc() fails
sk_stream_wait_memory()
releases the socket lock acquires the socket lock
completes the cork
psock->cork = NULL
frees the cork
reacquires the socket lock
msg_tx != psock->cork
sk_msg_free(msg_tx)
The stale cork is therefore mistaken for the local temporary message
and freed again. KASAN reported:
BUG: KASAN: slab-use-after-free in sk_msg_free+0x49/0x50
Read of size 4 at addr ffff88810c908800 by task poc/90
Call Trace:
sk_msg_free+0x49/0x50
tcp_bpf_sendmsg+0x14f5/0x1cc0
__sys_sendto+0x32c/0x3a0
__x64_sys_sendto+0xdb/0x1b0
Allocated by task 89:
__kasan_kmalloc+0x8f/0xa0
tcp_bpf_sendmsg+0x16b3/0x1cc0
Freed by task 91:
__kasan_slab_free+0x43/0x70
kfree+0x131/0x3c0
tcp_bpf_sendmsg+0xec3/0x1cc0
msg_tx can only name the stack-local tmp or the shared cork. Check for
tmp directly so a changed psock->cork cannot turn a shared message into
an apparent local one.
Fixes: 604326b41a ("bpf, sockmap: convert to generic sk_msg interface")
Signed-off-by: Chengfeng Ye <nicoyip.dev@gmail.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Reviewed-by: Jakub Sitnicki <jakub@cloudflare.com>
Link: https://lore.kernel.org/bpf/87fr18lmzo.fsf%40cloudflare.com/
Link: https://lore.kernel.org/netdev/20260719161630.2901208-1-nicoyip.dev%40gmail.com/ [v1]
Link: https://patch.msgid.link/20260724103856.3399001-1-nicoyip.dev@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
