Eduard Zingerman 6efbde200b bpf: Handle scalar spill vs all MISC in stacksafe() ...

When check_stack_read_fixed_off() reads value from an spi
all stack slots of which are set to STACK_{MISC,INVALID},
the destination register is set to unbound SCALAR_VALUE.

Exploit this fact by allowing stacksafe() to use a fake
unbound scalar register to compare 'mmmm mmmm' stack value
in old state vs spilled 64-bit scalar in current state
and vice versa.

Veristat results after this patch show some gains:

./veristat -C -e file,prog,states -f 'states_pct>10'  not-opt after
File                     Program                States   (DIFF)
-----------------------  ---------------------  ---------------
bpf_overlay.o            tail_rev_nodeport_lb4    -45 (-15.85%)
bpf_xdp.o                tail_lb_ipv4            -541 (-19.57%)
pyperf100.bpf.o          on_event                -680 (-10.42%)
pyperf180.bpf.o          on_event               -2164 (-19.62%)
pyperf600.bpf.o          on_event               -9799 (-24.84%)
strobemeta.bpf.o         on_event               -9157 (-65.28%)
xdp_synproxy_kern.bpf.o  syncookie_tc             -54 (-19.29%)
xdp_synproxy_kern.bpf.o  syncookie_xdp            -74 (-24.50%)

Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20240127175237.526726-6-maxtram95@gmail.com

2024-02-02 13:22:14 -08:00

..

preload

bpf: make preloaded map iterators to display map elements count

2023-07-06 12:42:25 -07:00

arraymap.c

bpf: Consistently use BPF token throughout BPF verifier logic

2024-01-24 16:21:01 -08:00

bloom_filter.c

bpf: Centralize permissions checks for all BPF map types

2023-06-19 14:04:04 +02:00

bpf_cgrp_storage.c

bpf: Enable bpf_cgrp_storage for cgroup1 non-attach case

2023-12-08 17:08:18 -08:00

bpf_inode_storage.c

Merge tag 'net-next-6.4' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next

2023-04-26 16:07:23 -07:00

bpf_iter.c

bpf: Add __bpf_kfunc_{start,end}_defs macros

2023-11-01 22:33:53 -07:00

bpf_local_storage.c

bpf: bpf_sk_storage: Fix the missing uncharge in sk_omem_alloc

2023-09-06 11:08:14 +02:00

bpf_lru_list.c

bpf: Address KCSAN report on bpf_lru_list

2023-05-12 12:01:03 -07:00

bpf_lru_list.h

bpf: lru: Remove unused declaration bpf_lru_promote()

2023-08-08 17:21:42 -07:00

bpf_lsm.c

bpf: Minor clean-up to sleepable_lsm_hooks BTF set

2024-02-01 18:37:45 +01:00

bpf_struct_ops.c

bpf: Fix error checks against bpf_get_btf_vmlinux().

2024-01-25 20:49:59 -08:00

bpf_task_storage.c

bpf: Teach verifier that certain helpers accept NULL pointer.

2023-04-04 16:57:16 -07:00

btf.c

bpf: treewide: Annotate BPF kfuncs in BTF

2024-01-31 20:40:56 -08:00

cgroup_iter.c

bpf: Let verifier consider {task,cgroup} is trusted in bpf_iter_reg

2023-11-07 15:24:25 -08:00

cgroup.c

bpf: Take into account BPF token when fetching helper protos

2024-01-24 16:21:01 -08:00

core.c

bpf: Consistently use BPF token throughout BPF verifier logic

2024-01-24 16:21:01 -08:00

cpumap.c

net, bpf: Add a warning if NAPI cb missed xdp_do_flush().

2023-10-17 15:02:03 +02:00

cpumask.c

bpf: treewide: Annotate BPF kfuncs in BTF

2024-01-31 20:40:56 -08:00

devmap.c

net, bpf: Add a warning if NAPI cb missed xdp_do_flush().

2023-10-17 15:02:03 +02:00

disasm.c

bpf: change bpf_alu_sign_string and bpf_movsx_string to static

2023-08-04 16:15:50 -07:00

disasm.h

bpf: Relicense disassembler as GPL-2.0-only OR BSD-2-Clause

2021-09-02 14:49:23 +02:00

dispatcher.c

bpf: Use arch_bpf_trampoline_size

2023-12-06 17:17:20 -08:00

hashtab.c

Merge tag 'net-next-6.8' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next

2024-01-11 10:07:29 -08:00

helpers.c

bpf: treewide: Annotate BPF kfuncs in BTF

2024-01-31 20:40:56 -08:00

inode.c

bpf: Support symbolic BPF FS delegation mount options

2024-01-24 16:21:02 -08:00

Kconfig

bpf: Add fd-based tcx multi-prog infra with link support

2023-07-19 10:07:27 -07:00

link_iter.c

bpf: Add bpf_link iterator

2022-05-10 11:20:45 -07:00

local_storage.c

Merge tag 'cgroup-for-6.4' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup

2023-04-29 10:05:22 -07:00

log.c

bpf: emit more dynptr information in verifier log

2023-12-11 19:21:22 -08:00

lpm_trie.c

bpf, lpm: Fix check prefixlen before walking trie

2023-11-09 19:07:38 -08:00

Makefile

bpf: Introduce BPF token object

2024-01-24 16:21:01 -08:00

map_in_map.c

bpf: Optimize the free of inner map

2023-12-04 17:50:26 -08:00

map_in_map.h

bpf: Add map and need_defer parameters to .map_fd_put_ptr()

2023-12-04 17:50:26 -08:00

map_iter.c

bpf: treewide: Annotate BPF kfuncs in BTF

2024-01-31 20:40:56 -08:00

memalloc.c

bpf: Remove unnecessary cpu == 0 check in memalloc

2024-01-04 10:18:14 -08:00

mmap_unlock_work.h

bpf: Introduce helper bpf_find_vma

2021-11-07 11:54:51 -08:00

mprog.c

bpf: Handle bpf_mprog_query with NULL entry

2023-10-06 17:11:20 -07:00

net_namespace.c

net: Add includes masked by netdevice.h including uapi/bpf.h

2021-12-29 20:03:05 -08:00

offload.c

Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net

2023-09-21 21:49:45 +02:00

percpu_freelist.c

bpf: Initialize same number of free nodes for each pcpu_freelist

2022-11-11 12:05:14 -08:00

percpu_freelist.h

bpf: Use raw_spin_trylock() for pcpu_freelist_push/pop in NMI

2020-10-06 00:04:11 +02:00

prog_iter.c

bpf: Refactor bpf_iter_reg to have separate seq_info member

2020-07-25 20:16:32 -07:00

queue_stack_maps.c

bpf: Avoid deadlock when using queue and stack maps from NMI

2023-09-11 19:04:49 -07:00

reuseport_array.c

bpf: Centralize permissions checks for all BPF map types

2023-06-19 14:04:04 +02:00

ringbuf.c

bpf: Fold smp_mb__before_atomic() into atomic_set_release()

2023-10-24 14:26:07 +02:00

stackmap.c

bpf: Add crosstask check to __bpf_get_stack

2023-11-10 11:06:10 -08:00

syscall.c

bpf,lsm: Refactor bpf_map_alloc/bpf_map_free LSM hooks

2024-01-24 16:21:01 -08:00

sysfs_btf.c

bpf: Load and verify kernel module BTFs

2020-11-10 15:25:53 -08:00

task_iter.c

bpf: bpf_iter_task_next: use next_task(kit->task) rather than next_task(kit->pos)

2023-11-19 11:43:44 -08:00

tcx.c

bpf, tcx: Get rid of tcx_link_const

2023-10-23 15:01:53 -07:00

tnum.c

bpf: simplify tnum output if a fully known constant

2023-12-02 11:36:51 -08:00

token.c

bpf,token: Use BIT_ULL() to convert the bit mask

2024-01-29 20:04:55 -08:00

trampoline.c

bpf: Use arch_bpf_trampoline_size

2023-12-06 17:17:20 -08:00

verifier.c

bpf: Handle scalar spill vs all MISC in stacksafe()

2024-02-02 13:22:14 -08:00