mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
synced 2026-07-23 14:27:37 -04:00
9012cf2491
Inline these calls in bpf jit: - bpf_get_smp_processor_id() - bpf_get_current_task() - bpf_get_current_task_btf() s390 has a 8 KiB per-CPU prefix area in the CPU's virtual address space, called the lowcore. It is a struct that contains the cpu number and a pointer to the current task. These are exactly the values returned by the BPF helpers. Emit a load from the lowcore instead of a helper function call. JIT output for `bpf_get_smp_processor_id`: Before: After: --------------- ---------------- brasl %r14,0x3ffe0385460 ly %r14,928 lgr %r14,%r2 JIT output for `bpf_get_current_task`: Before: After: --------------- ---------------- brasl %r14,0x3ffe0362a90 lg %r14,832 lgr %r14,%r2 Benchmark using [1] on KVM(virtme-ng). ./benchs/run_bench_trigger.sh glob-arr-inc arr-inc hash-inc +---------------+--------------------+--------------------+--------------+ | Name | Before | After | % change | |---------------+--------------------+--------------------+--------------| | glob-arr-inc | 244.954 ± 0.654M/s | 278.501 ± 0.834M/s | + 13.70% | | arr-inc | 311.597 ± 1.016M/s | 313.610 ± 0.331M/s | + 0.65% | | hash-inc | 47.421 ± 0.017M/s | 47.600 ± 0.004M/s | + 0.38% | +---------------+--------------------+--------------------+--------------+ [1] https://github.com/anakryiko/linux/commit/8dec900975ef Signed-off-by: Maxim Khmelevskii <max@linux.ibm.com> Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com> Link: https://lore.kernel.org/r/20260414142930.528751-1-max@linux.ibm.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>