Commit 5816bf4273 ("lsm,selinux: Add LSM blob support for BPF objects")
made the LSM hook wrappers for BPF object creation clean up the LSM
state internally upon denial, e.g. security_bpf_map_create() internally
calls security_bpf_map_free() when the bpf_map_create hook returns an
error. map_create() however still routes a denial to its free_map_sec
label, which invokes security_bpf_map_free() a second time, so the
bpf_map_free hook fires twice for a single denied map.
In-tree LSMs are unaffected in practice since the blob kfree() inside
security_bpf_map_free() is NULL-safe and idempotent and none of them
implement bpf_map_free, but a BPF LSM program attached to that hook
observes double invocations. Route the denial to free_map instead.
Fixes: 5816bf4273 ("lsm,selinux: Add LSM blob support for BPF objects")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260709073422.379247-1-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Tiezhu Yang says:
====================
Introduce jit_required to prevent a kernel panic
This series introduces a 'jit_required' flag in struct bpf_prog
to track programs that strictly require JIT.
The aim is to prevent a kernel panic by rejecting programs with
inlined helpers when JIT is not available.
v8 -> v9:
-- Initialize 'fp->jit_required' with
'IS_ENABLED(CONFIG_BPF_JIT_ALWAYS_ON)'
in bpf_prog_alloc_no_stats()
-- Simplify __bpf_prog_select_runtime()
-- Simplify the commit description for patch 1
-- Enhance the commit description for patch 2
====================
Link: https://patch.msgid.link/20260708101806.18885-1-yangtiezhu@loongson.cn
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
When an architecture (such as LoongArch, ARM64, and RISC-V) implements
bpf_jit_inlines_helper_call(), the verifier skips rewriting the helper
call offset (insn->imm) in bpf_do_misc_fixups(). This is because the
helper is expected to be inlined by the JIT compiler later. Therefore,
insn->imm remains as the raw helper enum ID.
However, if JIT is disabled at runtime (net.core.bpf_jit_enable=0) or
if JIT compilation fails dynamically (e.g., due to OOM), the program
falls back to the BPF interpreter.
When the interpreter executes (__bpf_call_base + insn->imm) with the
unpatched raw ID, it jumps into an invalid address space, triggering
an instruction alignment fault or a kernel panic.
Although these helpers have valid C implementations in the kernel, the
omission of offset rewriting makes runtime interpreter fallback fatal.
Fix this by setting 'prog->jit_required = 1' when helper call rewriting
is skipped for JIT inlining. This ensures that such programs are safely
rejected if JIT is not available, preventing the runtime kernel panic.
Fixes: 2ddec2c80b ("riscv, bpf: inline bpf_get_smp_processor_id()")
Suggested-by: Alexei Starovoitov <ast@kernel.org>
Suggested-by: KaFai Wan <kafai.wan@linux.dev>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Introduce a 'jit_required' bitfield flag in struct bpf_prog to track
whether a BPF program strictly requires the JIT compiler to run. This
prevents a dangerous runtime fallback to the interpreter for features
that are only implemented in the JIT compiler.
Currently, bpf_prog_has_kfunc_call() is used only for kernel function
calls, replace the kfunc-specific helper with the new 'jit_required'
flag. This makes it easy to support other JIT-only BPF features, such
as inlined helpers.
Suggested-by: Alexei Starovoitov <ast@kernel.org>
Suggested-by: KaFai Wan <kafai.wan@linux.dev>
Suggested-by: Leon Hwang <leon.hwang@linux.dev>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
bpf_find_vma() reads task->mm and calls mmap_read_trylock(mm) without
holding a reference on the mm. On a foreign task, a concurrent exit_mm()
can free the mm_struct between the lockless read and the trylock,
resulting in a use-after-free. mm_struct is not SLAB_TYPESAFE_BY_RCU.
For the current task, task->mm is stable. For a foreign task, pin the mm
under task->alloc_lock and release it with mmput_async(), mirroring commit
d8e27d2d22 ("bpf: fix mm lifecycle in open-coded task_vma iterator").
Use spin_trylock() instead of get_task_mm() so BPF context does not block
on alloc_lock. Reject irqs-disabled contexts and !CONFIG_MMU on the
foreign-task path because dropping the mm reference is not safe there.
Race:
CPU0 (BPF program) CPU1 (exiting task)
============================ ==========================
bpf_find_vma(foreign_task):
mm = task->mm
exit_mm():
task->mm = NULL
mmput(mm) -> frees mm_struct
mmap_read_trylock(mm)
// UAF on mm
Fixes: 7c7e3d31e7 ("bpf: Introduce helper bpf_find_vma")
Signed-off-by: Sanghyun Park <sanghyun.park.cnu@gmail.com>
Reviewed-by: Puranjay Mohan <puranjay@kernel.org>
Acked-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/bpf/20260708072106.199637-2-sanghyun.park.cnu@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Daniel Borkmann says:
====================
Misc BPF fixes from sashiko findings
Addressing some misc/random findings from sashiko that are worth fixing
which popped up as pre-existing issues while working on the signed BPF
loader series.
====================
Link: https://patch.msgid.link/20260708211537.371874-1-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpf_prog_calc_tag() copies the instructions into a plain vmalloc() scratch
buffer to blind the map fds before hashing. The buffer scales with the
program, up to ~8MB at the 1M instruction limit, and is allocated on every
program load, but unlike the rest of the load-time scratch memory it is
not charged to the loader's memcg. Use GFP_KERNEL_ACCOUNT to account it
like the other allocations scoped to the verification/load.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708211537.371874-5-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
The insn_aux_data array is allocated with a plain vzalloc(), while every
other allocation scoped to the verification - verifier states, explored
states, the cfg/scc arrays, liveness masks, jump history - is charged to
the loader's memcg via GFP_KERNEL_ACCOUNT.
At 136 bytes per instruction it is one of the largest verification-time
buffers, in the range of ~130MB for a program at the 1M instruction limit
(worst case), and it lives across the whole verification. The buffer is
also inconsistent with itself: when instruction patching grows it, the
vrealloc() in bpf_patch_insn_data() already passes GFP_KERNEL_ACCOUNT.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708211537.371874-4-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpf_get_btf_vmlinux() serializes the lazy vmlinux BTF parse with
bpf_verifier_lock, the same mutex bpf_check() holds across the whole
verification of an unprivileged program (if enabled; it's disabled
by default). The latter can potentially stall the mutex holder for
a long time (e.g. via userfaultfd), and therefore block first-time
bpf_get_btf_vmlinux() caller from any context, including privileged
program loads.
Give the vmlinux BTF initialization a dedicated btf_vmlinux_lock so
it is independent of the unprivileged verification mutex. The parse
only needs mutual exclusion against itself.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708211537.371874-3-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpf_get_btf_vmlinux() lazily parses the vmlinux BTF under the
bpf_verifier_lock, but publishes the result through a plain store
and re-checks it through a plain lockless load. Nothing orders
the stores initializing the struct btf inside btf_parse_vmlinux()
against the store publishing the pointer: On a weakly ordered
arch, a concurrent first-time caller taking the lockless fast
path could in principle observe the pointer before the parsed
contents are visible. The mutex_unlock() does not help such a
reader given it only synchronizes with a later acquisition of the
same lock. Thus, publish the pointer with smp_store_release()
and read it on the fast path with smp_load_acquire().
Acquire semantics are needed rather than a dependency-ordered
READ_ONCE(): btf_parse_vmlinux() also populates globals outside
the returned object (e.g. bpf_ctx_convert.t). An address
dependency would only order accesses performed through the
pointer and not cover other globals.
Fixes: 8580ac9404 ("bpf: Process in-kernel BTF")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708211537.371874-2-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Probe PMU support before loading bpf_test_rqspinlock.ko,
otherwise the test fails with not obvious error without
proper perf event support:
Failed to load bpf_test_rqspinlock.ko into the kernel: -2
serial_test_res_spin_lock_stress:FAIL:load module AA
unexpected error: -22 (errno 2)
Reported-by: Ilya Leoshkevich <iii@linux.ibm.com>
Signed-off-by: Maxim Khmelevskii <max@linux.ibm.com>
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Link: https://lore.kernel.org/bpf/20260708104729.1248234-2-max@linux.ibm.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
The signature_enforced, signature_too_large, signature_zero_size and
signature_bad_keyring subtests load a program that must be rejected,
but leave the fd open if the kernel unexpectedly accepts the load:
test_progs asserts record the failure and continue, so the fd would
linger for the rest of the run. Just close it.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708184107.369182-1-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Daniel Borkmann says:
====================
Verify BPF signed loader at load time
The BPF signing scheme signs a light skeleton's loader program and lets
the loader vouch for everything else: bpftool bakes the SHA256 of the
metadata map into the loader's instructions, signs the instructions, and
the loader compares the (frozen, exclusive) map against that hash from
within BPF once it runs. The construction is sound as a trusted hash
chain, but the kernel itself never attests the metadata, and that split
has been the recurring objection from the LSM / integrity side since the
scheme was proposed.
This proposal closes both gaps by having the kernel verify the metadata
at BPF_PROG_LOAD time, before the LSM admission hook and before the
verifier, /without/ growing the UAPI. A signed loader binds its metadata
map(s) through the existing fd_array/fd_array_cnt, and exclusive maps
are already bound to the loader's digest via excl_prog_hash. When a
signature is present, the kernel collects the exclusive maps from the
fd_array and appends their frozen contents to the instructions before
PKCS#7 verification, so the signature covers ...
insns || metadata_0 || metadata_1 || [...]
... in fd_array order. The in-loader hash check is dropped from the
gen_loader entirely: generated loaders carry no verification logic
anymore, and signing or verifying a skeleton becomes an ordinary CMS
operation over bytes that sit verbatim in the skeleton, reproducible
offline. A signed program is either BPF_SIG_UNSIGNED or BPF_SIG_VERIFIED
with nothing in between.
There is no new UAPI, we now have a single signature scheme, no LSM
code reaching into BPF internals, no new LSM hook, and unsigned loads
are completely unaffected. It is also less complex since the loader
does not need to deal with BTF, an extra kfunc, etc, as proposed in
an earlier series [0]. Tested against full BPF CI which came back
green. For more details and examples, see the documentation patch in
this series.
[0] https://lore.kernel.org/bpf/20260522023234.3778588-1-kpsingh@kernel.org/
v5 -> v6:
- Added bpf_map_write_active check to close potential race on
arm64 (sashiko)
- Rebased & rest stays as-is
v4 -> v5:
- Squashed patch 2/9 and 3/9 together and dropped Nack (Paul)
- Added Anton's Acked-by on the fd_array cache patch
- Add !attr->signature_size test to reject invalid param
and added BPF selftest for this case
v3 -> v4:
- Fix upper limit in MAX_FD_ARRAY_CNT (Anton)
- Reject !fd_array && attr->fd_array_cnt (Anton)
- Add bpftool patch wrt ignored return value of EVP_Digest() (sashiko)
- Fix setting of gen_loader_fixture_init (sashiko)
- Fix unused map_fd cleanup branch in selftest (bot+bpf-ci)
- Remove now unused map->excl member and adjust selftests
- Added more BPF signed_loader corner case selftest coverage
- Added Paul's Nack wrt bpf_prog_load LSM hook dispute
- Added patch 2 to move bigger allocations below fd_array
resolution (Paul)
v2 -> v3:
- Added first commit to cache and work on objects in fd_array
which was the most recent issue sashiko rightfully complained
- Added more BPF signed_loader selftest coverage to cover that
usage of sparse fd_array or map fds gets rejected
- I left the security_bpf_prog_load as in v2 given preference
from BPF side over adding new hook
v1 -> v2:
- Addressed both sashiko complaints, the TOCTOU bug regarding
fd_array processing, as well as exclusive map checking to
only allow array maps. The validation is now moved into the
verifier before the main verification work happens. This also
gives the opportunity to utilize the verifier log.
====================
Link: https://patch.msgid.link/20260708075343.358712-1-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
With write-only excl member removed from struct bpf_map, ops moves
to offset 32 and inner_map_meta to offset 40. Update the expected
verifier message for the former and retarget the latter at the sha
byte array, so the beyond-member-end rejection path stays covered:
# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t verifier_map_ptr
[...]
#619/5 verifier_map_ptr/bpf_map_ptr: read non-existent field rejected:OK
#619/6 verifier_map_ptr/bpf_map_ptr: read non-existent field rejected @unpriv:OK
#619/7 verifier_map_ptr/bpf_map_ptr: read beyond sha field rejected:OK
#619/8 verifier_map_ptr/bpf_map_ptr: read beyond sha field rejected @unpriv:OK
#619/9 verifier_map_ptr/bpf_map_ptr: read ops field accepted:OK
#619/10 verifier_map_ptr/bpf_map_ptr: read ops field accepted @unpriv:OK
[...]
#620 verifier_map_ptr_mixing:OK
Summary: 2/20 PASSED, 0 SKIPPED, 0 FAILED
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708075343.358712-7-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpftool_prog_sign() signed only the loader instructions. The metadata
blob the loader installs was left to an in-loader hash check, which
the kernel now performs at load time over insns || metadata.
Sign that same concatenation: pass the metadata blob (gen_loader_opts
data) through to bpftool_prog_sign() and feed insns || metadata to
CMS_final(). The excl_prog_hash stays a digest of the instructions
alone; it binds the metadata map to the loader and is matched against
prog->digest by the verifier, independent of what the signature covers.
The signed artifact is now plain data: both bytes the signature
covers are embedded verbatim in the generated skeleton, so signing
and verifying an lskel is an ordinary CMS operation that a signer or
auditor can perform (or reproduce) offline, without analyzing loader
bytecode to establish what the signature actually attests to.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Quentin Monnet <qmo@kernel.org>
Link: https://lore.kernel.org/bpf/20260708075343.358712-6-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpftool_prog_sign() ignores the return value of EVP_Digest(). If the
digest computation fails (context allocation failure, or a digest
fetch failure under OpenSSL), EVP_Digest() returns 0 and leaves the
output buffer untouched, but the function still reports success.
Fixes: 40863f4d6e ("bpftool: Add support for signing BPF programs")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Quentin Monnet <qmo@kernel.org>
Link: https://lore.kernel.org/bpf/20260708075343.358712-5-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
The signed gen_loader used to police its own metadata map from within
BPF: emit_signature_match() read the kernel-cached map->sha[] back
through hardcoded struct bpf_map offsets and compared it against a hash
that compute_sha_update_offsets() baked into the signed instructions,
after a BPF_OBJ_GET_INFO_BY_FD round-trip to populate map->sha[].
The kernel now verifies the metadata at BPF_PROG_LOAD time by folding
the frozen contents of the loader's exclusive fd_array maps into the
signature, so the loader no longer checks anything itself. Generated
loaders thus carry no verification logic of their own anymore: Nothing
in the signing chain depends on emitted loader bytecode doing the right
thing.
On the loading side, skel_internal.h now sets fd_array_cnt for a signed
load so the kernel scans fd_array for the exclusive metadata map -
still frozen, as the kernel requires - and the BPF_OBJ_GET_INFO_BY_FD
round-trip to populate map->sha[] is gone. The struct bpf_map layout
BUILD_BUG_ON()s on the kernel side are removed as well: they only
pinned the ABI for the in-BPF read of map->sha[] that is no longer
needed. Same for the map->excl member. Note: gen_hash is retained; it
still marks a loader as signed so an untrusted host cannot re-dimension
maps or override initial values now covered by the signature.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708075343.358712-4-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
A signed gen_loader program carries the programs, maps and relocations
it installs in a metadata array map. The loader instructions are covered
by the PKCS#7 signature, but the metadata map is not: Today the loader
compares the map contents from within BPF against a hash baked into its
(signed) instructions, using the kernel-cached map hash. The kernel
itself never actually attests that the metadata the loader installs is
the metadata that was signed.
This split is the core of the long-standing objection to the BPF signing
scheme from the LSM / integrity side: the integrity check of a light
skeleton only completes once the loader program runs, that is, after the
security_bpf_prog_load() hook, so at admission time an LSM observes a
program whose payload has not yet been verified. Auditing the chain
link is also not a purely cryptographic operation: whoever signs or
reviews an lskel has to disassemble the loader's preamble to convince
themselves that the embedded hash check is present and correct [0][1].
Two acceptable fixes were identified in those threads: Complete the
integrity check before the admission hook fires, or add a second hook
that collects the verification result after the loader ran [2]. Covering
both the loader and its maps directly with the PKCS#7 signature is what
Blaise Boscaccy's patchsets proposed in several forms. Let's implement
the former, without growing the UAPI, and in particular as a single
unified scheme where the signature spans the raw bytes rather than
derived hashes.
A signed loader binds its metadata map(s) through the existing fd_array,
and an exclusive map is already bound to a program digest (excl_prog_hash).
So when a signature is present, collect the exclusive maps from fd_array
and append their frozen contents to the instructions before verification:
The signature now covers insns || metadata_0 || metadata_1 || [...] in
the fd_array order, and verification completes in bpf_check(), once the
fd_array maps are resolved into used_maps, before the LSM admission hook
and the rest of verification. A program is either BPF_SIG_UNSIGNED or
BPF_SIG_VERIFIED, with nothing in between. While folding the fd_array
maps, a non-exclusive map bound to a signed program is rejected, so every
map folded into the signature is exclusive. A signed loader that fails
to cover its metadata thus does not load, and BPF_SIG_VERIFIED always
means the instructions and every exclusive map are authentic. The maps
must be frozen so the hashed bytes cannot change before the loader runs;
the map <-> program digest binding is enforced by the verifier for every
used map. Binding maps through fd_array_cnt makes the verifier resolve
and excl-check them (excl_prog_sha vs prog->digest) before it would
otherwise compute the digest, so compute prog->digest up front in
bpf_check(), over the unmodified instructions the signature covers, for
a load that folds metadata.
Unsigned programs are not affected by the signature path; for them the
LSM admission hook merely moves below fd_array resolution, with minimal
bounded work in between. Note, signed loaders generated by older libbpf/
bpftool versions need to be regenerated; some of the recent fixes we've
had on the signed loader side require the latter already to close gaps.
Finally, some remarks around the security_bpf_prog_load() placement
given there was discussion on whether a new hook is needed or the existing
security_bpf_prog() hook should be reused [3]: For a new hook it would
mean that just for loading a single BPF program it has to pass through
four layers of LSM hooks:
1) security_bpf (cmd=PROG_LOAD): for gating various bpf subcmds
2) security_bpf_prog_load: historical admission hook (CAP/token,
prog_type, attach point), pre-verification
3) security_bpf_prog_verify_signature: newly asked admission hook,
same role as 2), plus the BPF signature verdict
4) security_bpf_prog: gate handing the prog fd back to userspace,
verification done & signature verified
The use-cases of 2) and 3) conflate, thus BPF community prefers to just
keep a total of 3 LSM hooks (as-is today): 3) makes 2) incoherent given
they are the /same class/ of hook, that is, access-control admission on
the load and split only by _what_ they can see. Worse, with the split,
for a signed BPF program security_bpf_prog_load 2) admits a program whose
signature has not been checked, so a policy gating at 2) is structurally
unable to express "admit only verified" and every such policy is forced
onto 3) *anyway*. In other words, one doesn't get two complementary hooks,
but rather, one real admission hook aka 3) plus a now-degraded /legacy/
hook 2) that can't answer the question operators actually want to ask.
Reusing security_bpf_prog() 4) for admission is no alternative either:
it fires only after the entire verifier (and JIT) pipeline ran, so
denying a not-yet-verified program at that point burns exactly the
work a denial is supposed to avoid, and by then the program has an id
assigned and the kallsyms/perf/audit load events fired. Policies are
free to also consume the signature verdict at 4), but admission control
belongs into security_bpf_prog_load(). Hence the latter remains the only
admission hook, merely moved past signature verification; with moving
large allocations further down into the BPF verifier, there is now only
minimal work between the old and new location: The preparation work in
bpf_check() is reordered such that only the minimally necessary setup
happens up front: Allocating the env, initializing the verifier log and
resolving the fd_array that a signed BPF metadata map needs. The worst
case allocation up until security_bpf_prog_load() is ~90K which is the
env itself (~54K) plus the continuous fd_array cache (at most 32K). The
insn_aux_data array is moved into a later stage in the verification.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/2f71d6c03698eb17d51f7247efde777627ee578a.camel@HansenPartnership.com [0]
Link: https://lore.kernel.org/lkml/ecf0521ed302db672672ebfbc670ecfba36a6e00.camel@HansenPartnership.com [1]
Link: https://lore.kernel.org/bpf/88703f00d5b7a779728451008626efa45e42db3d.camel@HansenPartnership.com [2]
Link: https://lore.kernel.org/bpf/DJOFY21DYUI4.19WKQ3NPZ4H5R@gmail.com [3]
Link: https://lore.kernel.org/bpf/20260708075343.358712-3-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
The fd_array passed to BPF_PROG_LOAD carries the map and module BTF file
descriptors a program binds. The verifier reads it more than once during
a load: process_fd_array() walks it to bind the maps and BTFs, and
check_and_resolve_insns() and the kfunc BTF resolver later read it again
to resolve the program's BPF_PSEUDO_MAP_IDX* and module kfunc refs.
For signed BPF, we need these upfront in memory, thus resolve each fd to
its object once and cache it by fd_array index, then bind that cached
object for the rest of the load. env->fd_array becomes a small per-slot
{map, btf} cache rather than a bpfptr_t; every later reference is then
an in-bounds lookup of an already-resolved object, and an index outside
the cache is rejected instead of read from user memory:
- continuous (fd_array_cnt given): the caller declares the length and
every entry is resolved and bound up front (used also by the BPF
signed loader)
- sparse (no fd_array_cnt): left as the legacy path with no fd_array
cache; each reference reads its fd from the caller's fd_array and
resolves it on the spot. Deduplication in used_maps and the kfunc BTF
table keeps this correct, and only unsigned programs use this shape.
Split these into separate helpers to make it easier to follow.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Anton Protopopov <a.s.protopopov@gmail.com>
Link: https://lore.kernel.org/bpf/20260708075343.358712-2-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Add a TCP congestion-control struct_ops load test for a write through a
BTF pointer produced by bpf_rdonly_cast().
The test expects the verifier to reject the program before the TCP CA
btf_struct_access callback can whitelist the tcp_sock field write.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
check_ptr_to_btf_access() lets program-type btf_struct_access callbacks
validate writes before the default BTF access path rejects non-read
accesses. That bypasses the read-only policy for untrusted BTF pointers
created by helpers such as bpf_rdonly_cast().
Reject non-read accesses through PTR_UNTRUSTED BTF pointers at the
common entry point, before the callback branch to handle all cases.
Fixes: 282de143ea ("bpf: Introduce allocated objects support")
Signed-off-by: Nicholas Dudar <main.kalliope@gmail.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
When building bpf selftest with latest bpf-next, I got the following failure:
In file included from /home/yhs/work/bpf-next/tools/testing/selftests/bpf/libarena/selftests/test_parallel_bitmap.bpf.c:8:
/home/yhs/work/bpf-next/tools/testing/selftests/bpf/libarena/include/libarena/bitmap.h:11:8: error: redefinition of
'bitmap'
11 | struct bitmap {
| ^
/home/yhs/work/bpf-next/tools/testing/selftests/bpf/tools/include/vmlinux.h:51320:8: note: previous definition is here
51320 | struct bitmap {
| ^
The vmlinux.h struct bitmap comes from drivers/md/md-bitmap.c:
struct bitmap {
struct bitmap_counts { ... }
...
}
To fix the issue, I renamed libarena struct bitmap to arena_bitmap to avoid the conflict.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260707220136.910374-1-yonghong.song@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
The verifier loses precision when simulating stack spills for the
following register types:
- PTR_TO_TP_BUFFER
- PTR_TO_INSN
- CONST_PTR_TO_DYNPTR
These types are not allow-listed in the is_spillable_regtype(),
because of that check_stack_write_fixed_off() takes the branch
that marks the slots STACK_MISC.
There are no technical reasons for this limitation.
This commit replaces an explicit list of pointer types in
is_spillable_regtype() with explicit list of non-pointer types.
The function is renamed to is_pointer_regtype() for clarity.
Reported-by: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Suggested-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260707-missing-spillable-types-v1-1-44a92121dc41@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
In the msg_alloc_iov function, the iov pointer is only assigned to
msg->msg_iov after all memory allocations complete successfully.
Therefore, when a calloc failure triggers the unwind_iov cleanup branch,
we should use the local variable iov instead of msg->msg_iov.
Fixes: 753fb2ee09 ("bpf: sockmap, add msg_peek tests to test_sockmap")
Signed-off-by: Feng Yang <yangfeng@kylinos.cn>
Reviewed-by: John Fastabend <john.fastabend@gmail.com>
Link: https://lore.kernel.org/bpf/20260707081434.539327-1-yangfeng59949@163.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
tc, xdp, socket_filter and flow_dissector programs can no longer update
or delete a sockmap. Adjust the tests:
- verifier_sockmap_mutate: the tc, xdp, socket_filter and
flow_dissector cases now expect __failure with "cannot update sockmap
in this context".
- sockmap_basic: drop "sockmap update" / "sockhash update", which load
a SEC("tc") program that copies a sock between maps.
- fexit_bpf2bpf: drop "func_sockmap_update", whose freplace program
updates a sockmap in the tc cls_redirect context.
Remove the now-unused test_sockmap_update.c and freplace_cls_redirect.c.
Signed-off-by: Sechang Lim <rhkrqnwk98@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: John Fastabend <john.fastabend@gmail.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260630145410.3648099-3-rhkrqnwk98@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
sock_map_update_common() and __sock_map_delete() hold stab->lock and call
sock_map_unref() -> sock_map_del_link(), which takes sk_callback_lock for
write. That gives the order stab->lock -> sk_callback_lock.
The reverse order comes from the SK_SKB stream parser.
sk_psock_strp_data_ready() holds sk_callback_lock for read, and after the
verdict tcp_bpf_strp_read_sock() acks the consumed data inline via
__tcp_cleanup_rbuf(). The ACK goes out egress, where a sched_cls program
deletes from the sockmap and takes stab->lock:
WARNING: possible circular locking dependency detected
------------------------------------------------------
syz.9.8824 is trying to acquire lock:
(&stab->lock){+.-.}-{3:3}, at: __sock_map_delete net/core/sock_map.c:421
but task is already holding lock:
(clock-AF_INET){++.-}-{3:3}, at: sk_psock_strp_data_ready net/core/skmsg.c:1173
-> #1 (clock-AF_INET){++.-}-{3:3}:
_raw_write_lock_bh
sock_map_del_link net/core/sock_map.c:167
sock_map_unref net/core/sock_map.c:184
sock_map_update_common net/core/sock_map.c:509
sock_map_update_elem_sys net/core/sock_map.c:588
map_update_elem kernel/bpf/syscall.c:1805
-> #0 (&stab->lock){+.-.}-{3:3}:
_raw_spin_lock_bh
__sock_map_delete net/core/sock_map.c:421
sock_map_delete_elem net/core/sock_map.c:452
bpf_prog_06044d24140080b6
tcx_run net/core/dev.c:4451
sch_handle_egress net/core/dev.c:4541
__dev_queue_xmit net/core/dev.c:4808
...
tcp_bpf_strp_read_sock net/ipv4/tcp_bpf.c:701
strp_data_ready net/strparser/strparser.c:402
sk_psock_strp_data_ready net/core/skmsg.c:1174
tcp_data_queue net/ipv4/tcp_input.c:5661
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
rlock(clock-AF_INET);
lock(&stab->lock);
lock(clock-AF_INET);
lock(&stab->lock);
*** DEADLOCK ***
A tc, xdp, socket_filter or flow_dissector program has no reason to
update or delete a sockmap, and redirect does not go through here. Drop
them from may_update_sockmap() so the verifier rejects it. It also
closes the matching sockhash inversion.
Suggested-by: John Fastabend <john.fastabend@gmail.com>
Signed-off-by: Sechang Lim <rhkrqnwk98@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: John Fastabend <john.fastabend@gmail.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260630145410.3648099-2-rhkrqnwk98@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
When the kernel config does not fully match the BPF selftest config
fragment, some tests may fail to compile. BPF_STRICT_BUILD (defaulting to
1) makes any such failure fatal. Mention the option so that developers are
aware they can set it to 0 to skip broken tests and keep the build going,
which is particularly useful during bringup or when testing on constrained
(e.g. distribution) configurations.
Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260706-b4-bpf_strict_build_docs-v1-1-5324d605c7b0@suse.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Yiyang Chen says:
====================
bpf: Reject MEM_ALLOC BTF accesses past bounds
BTF struct walks can relax the top-level struct-size check for trailing
flexible arrays. That relaxation must not let a PTR_TO_BTF_ID | MEM_ALLOC
access escape the bytes allocated by bpf_obj_new() or bpf_percpu_obj_new().
Patch 1 rejects MEM_ALLOC BTF walks whose access range reaches past the
current struct size before applying the flexible-array relaxation. This now
also applies to struct ID matching used by kfunc and kptr type checks.
Patch 2 adds a linked_list negative loader case for this path.
Changes in v3:
- Pass the flexible-array walk policy through btf_struct_ids_match() callers,
so MEM_ALLOC kfunc/kptr type checks use the same bounds rule.
- Rename the btf_struct_walk() parameter to walk_flex_arrays.
- Rebase onto current bpf-next.
v2:
https://lore.kernel.org/bpf/cover.1782197377.git.chenyy23@mails.tsinghua.edu.cn/
v1:
https://lore.kernel.org/bpf/cover.1782100805.git.chenyy23@mails.tsinghua.edu.cn/
====================
Link: https://patch.msgid.link/cover.1782807039.git.chenyy23@mails.tsinghua.edu.cn
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
BTF struct walks relax the struct-size check for accesses through a
trailing flexible array. That is valid for ordinary BTF type walking, but
PTR_TO_BTF_ID | MEM_ALLOC values point to objects allocated with the static
BTF type size.
When walking a MEM_ALLOC object, reject the access before applying the
flexible-array relaxation if the access range extends past the struct size.
Apply the same policy to struct ID matching so kfunc and kptr type checks
do not walk past the allocated object bounds either.
Fixes: 958cf2e273 ("bpf: Introduce bpf_obj_new")
Fixes: 36d8bdf75a ("bpf: Add alloc/xchg/direct_access support for local percpu kptr")
Signed-off-by: Yiyang Chen <chenyy23@mails.tsinghua.edu.cn>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/4b8c8a81102ba4b595011434c881194f264ddc59.1782807039.git.chenyy23@mails.tsinghua.edu.cn
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
In msg_alloc_iov(), when calloc() fails for an individual iov_base
allocation, the error path frees all previously allocated iov_base
entries but fails to free the iov array itself that was allocated
with calloc() at the beginning of the function. This results in a
memory leak of the iov array.
Add free(iov) in the unwind_iov error path to ensure proper cleanup
of all allocated memory.
Fixes: 753fb2ee09 ("bpf: sockmap, add msg_peek tests to test_sockmap")
Signed-off-by: Malaya Kumar Rout <malayarout91@gmail.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260704122936.102394-1-malayarout91@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Emil Tsalapatis says:
====================
selftests/bpf: libarena cleanup and bitmap struct
Cleanup patches for libarena, along with a new bitmap data type that is
in use by sched-ext. Patch 1 is an NFC that properly renames the buddy
selftests for consistency. Patch 2 fixes the zero variable used in
libarena for can_loop based looping, and afterwrds removes all bpf_for()
instances from the code. Patch 3 fixes an (untriggered) edge case that
could cause spurious selftest failures. Finally, patches 4 to 6
introduce the bitmap data structure along with selftests.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
CHANGELOG
=========
v3 was resent as the first submission was missing the first patch.
v2 -> v3: (https://lore.kernel.org/bpf/20260701185235.4516-1-emil@etsalapatis.com)
- Remove unused macros and fix typo in commit message (AI)
v1 -> v2: (https://lore.kernel.org/bpf/20260618085626.19633-1-emil@etsalapatis.com)
- Added acks by Ihor and Eduard
- Fix missing commit message (Ihor)
- Enforce 64 bits per cell with BITS_TO_LONG_LONG (Sashiko)
- Add test for bmp_copy (Ihor)
- Add atomic versions of _set() and _clear() (Ihor) and add a parallel
selftest for them.
====================
Link: https://patch.msgid.link/20260706181730.21731-1-emil@etsalapatis.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Add a selftest for testing the atomic bitmap set/clear/
test_and_set/test_and_clear operations. The selftest
checks atomicity by spawning two threads, each of which
either only works on even bits or with odd bits. The
test checks that threads do not affect each other's
bits.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260706181730.21731-7-emil@etsalapatis.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Summary: The buddy allocator requires the global BPF buddy allocator
to not be already initialized. However, the test currently merely resets
the allocator before the buddy tests instead of destroying it, and the
test worked because the buddy test happened to run first. Properly
destroy the allocator instead of resetting it.
Fixes: b1487dc1b1 ("selftests/bpf: Add selftests for libarena buddy allocator")
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Acked-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://lore.kernel.org/bpf/20260706181730.21731-4-emil@etsalapatis.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
BPF can_loop based loops require the index variable to stay imprecise.
This means we must initialize them from a currently imprecise variable
instead of directly assigning 0 to them, like so:
static volatile u32 zero = 0;
for (i = zero; i < NUM_LOOPS; i++) {
/* loop body */
}
The libarena implementation of this technique is currently faulty. For
the technique to work, the variable must not be in a map. This includes
the .rodata DATASEC map used for const variables. However, libarena
still defines the zero variable as constant.
Modify the zero variable definition into a volatile variable. This
change adds a complication caused by the compiler optimizing array
derefences from
for (i = zero; i < NUM_LOOPS; i++) {
val = *(ptr + i);
}
into
for (i = zero; i < NUM_LOOPS; i++) {
val = *ptr++;
}
and causing verification failures. Use the barrier_var() clobber macro
to prevent this optimization from taking place. Using barrier_var() is
the only way to break the optimization, as annotating the index as
volatile does not suffice.
After that, remove the bpf_for() invocations introduced in libarena for
parallel spmc testing.
Reported-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260706181730.21731-3-emil@etsalapatis.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Pull mod_devicetable.h header split from Uwe Kleine-König:
"Split <linux/mod_devicetable.h> in per subsystem headers
<linux/mod_devicetable.h> is included transitively in nearly every
driver in an x86_64 allmodconfig build of v7.1:
$ find drivers -name \*.o -not -name \*.mod.o | wc -l
21330
$ find drivers -name \*.o.cmd -not -name \*.mod.o.cmd | xargs grep -l mod_devicetable.h | wc -l
17038
The result of this mixture of different and unrelated subsystem
details is that even when touching an obscure device id struct most of
the kernel needs to be recompiled. Given that each driver typically
only needs one or two of these structures, splitting into per
subsystem headers and only including what is really needed reduces the
amount of needed recompilation.
This split is implemented in the first commit and then after some
preparatory work in the following commits, the last two replace
includes of <linux/mod_devicetable.h> by the actually needed more
specific headers.
There are still a few instances left, but the ones with high impact
(that is in headers that are used a lot) and the easy ones (.c files)
are handled. These remaining includes will be addressed during the
next merge window"
* tag 'device-id-rework' of git://git.kernel.org/pub/scm/linux/kernel/git/ukleinek/linux:
Replace <linux/mod_devicetable.h> by more specific <linux/device-id/*.h> (c files)
Replace <linux/mod_devicetable.h> by more specific <linux/device-id/*.h> (headers)
parisc: #include <linux/compiler.h> for unlikely() in <asm/ptrace.h>
media: em28xx: Add include for struct usb_device_id
LoongArch: KVM: Add include defining struct cpu_feature
ALSA: hda/core: Add include defining struct hda_device_id
usb: dwc2: Add include defining struct pci_device_id
platform/x86: int3472: Add include defining struct dmi_system_id
platform/x86: x86-android-tablets: Add include defining struct dmi_system_id
i2c: Let i2c-core.h include <linux/i2c.h>
of: Explicitly include <linux/types.h> and <linux/err.h>
platform/x86: msi-ec: Ensure dmi_system_id is defined
usb: serial: Include <linux/usb.h> in <linux/usb/serial.h>
driver core: platform: Include header for struct platform_device_id
driver: core: Include headers for acpi_device_id and of_device_id for struct device_driver
media: ti: vpe: #include <linux/platform_device.h> explicitly
mod_devicetable.h: Split into per subsystem headers
Pull ata fixes from Damien Le Moal:
- Quirk the Phison PS3111-S11 SSD with NOLPM due to its defective
link power management (Bryam)
- Strengthen checks on a device concurrent positioning range
information to make sure to reject any invalid report (Bryam)
- Fix probe error handling in the pata_pxa and sata_gemini
drivers (Myeonghun, Wentao)
- Limit buffer size of replies from translated commands to what
libata actually generated (Karuna)
* tag 'ata-7.2-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/libata/linux:
ata: libata-scsi: limit simulated SCSI command copy to response length
ata: pata_pxa: Fix DMA channel leak on probe error
ata: sata_gemini: unwind clocks on IDE pinctrl errors
ata: libata-core: Reject an invalid concurrent positioning ranges count
ata: libata-core: Add NOLPM quirk for PNY CS900 1TB SSD
