mirrors/linux
2
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git synced 2026-08-28 04:03:23 -04:00
Code Issues Packages Projects Releases Wiki Activity

perf tests: Fix flakiness in BPF counters test on hybrid systems

Browse Source 
The `perf stat --bpf-counters test` fails intermittently on hybrid
architectures or systems with dynamic frequency scaling (DVFS). This
happens because the test workload (`sqrtloop`) runs for a fixed 1-second
duration, and the CPU frequency can scale dynamically between idle and
maximum frequency. As the first run runs on a cold CPU and the second run
runs on a warmed-up CPU (or vice versa), the number of instructions
executed in 1 second differs by up to 2.2x, violating the comparison
tolerance.

Also, when running as root, BPF tracepoints and scheduling programs
trigger frequently. Since standard `perf stat -e instructions` measures
both user and kernel space instructions, it counts BPF helper and program
execution overheads, whereas the BPF counters themselves do not self-
measure. This introduces a large kernel-space instruction count
discrepancy between standard and BPF counters.

Fix these issues by:
1. Switching the workload to a strictly deterministic, iteration-based
   workload: `awk 'BEGIN { for (i=0; i<10000000; i++) sum+=i }'`. We pin
the
   workload to a single random allowed CPU using `taskset -c $CPU` via a
bash array.
2. Restricting the counted event to user-space only (`instructions:u` or
`/u`).
3. Tightening the comparison tolerance from 20% to 15%.

These modifications isolate the measurements to user-space instructions of
the deterministic loop, which executes a virtually identical number of
instructions on both runs (with less than 0.001% variation), eliminating
Dynamic Frequency Scaling (DVFS), kernel scheduling noise, and BPF helper
self-measurement overheads.

Fixes: 2c0cb9f560 ("perf test: Add a shell test for 'perf stat --bpf-counters' new option")
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
This commit is contained in:
Ian Rogers
2026-06-22 18:27:53 -07:00
committed by Namhyung Kim
parent 509a2b9a6e
commit b02027776a
1 changed files with 18 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
tools/perf/tests/shell/stat_bpf_counters.sh
View File

@@ -4,21 +4,26 @@
set -e
workload="perf test -w sqrtloop"
# Get the first allowed CPU
CPU=$(taskset -c -p $$ | awk -F': ' '{print $2}' | awk -F'[,-]' '{print $1}')
if [ -z "$CPU" ]; then
CPU=0
fi
workload=(taskset -c "$CPU" awk 'BEGIN { for (i=0; i<10000000; i++) sum+=i }')
# check whether $2 is within +/- 20% of $1
# check whether $2 is within +/- 15% of $1
compare_number()
{
first_num=$1
second_num=$2
# upper bound is first_num * 120%
upper=$(expr $first_num + $first_num / 5 )
# lower bound is first_num * 80%
lower=$(expr $first_num - $first_num / 5 )
# upper bound is first_num * 115%
upper=$(expr $first_num + $first_num / 20 \* 3 )
# lower bound is first_num * 85%
lower=$(expr $first_num - $first_num / 20 \* 3 )
if [ $second_num -gt $upper ] || [ $second_num -lt $lower ]; then
echo "The difference between $first_num and $second_num are greater than 20%."
echo "The difference between $first_num and $second_num are greater than 15%."
exit 1
fi
}
@@ -41,11 +46,12 @@ check_counts()
test_bpf_counters()
{
printf "Testing --bpf-counters "
base_instructions=$(perf stat --no-big-num -e instructions -- $workload 2>&1 | \
base_instructions=$(perf stat --no-big-num -e instructions:u -- "${workload[@]}" 2>&1 | \
awk -v i=0 -v c=0 '/instructions/ { \
if ($1 != "<not") { i++; c += $1 } \
} END { if (i > 0) printf "%.0f", c; else print "<not" }')
bpf_instructions=$(perf stat --no-big-num --bpf-counters -e instructions -- $workload 2>&1 | \
bpf_instructions=$(perf stat --no-big-num --bpf-counters -e instructions:u \
-- "${workload[@]}" 2>&1 | \
awk -v i=0 -v c=0 '/instructions/ { \
if ($1 != "<not") { i++; c += $1 } \
} END { if (i > 0) printf "%.0f", c; else print "<not" }')
@@ -57,7 +63,9 @@ test_bpf_counters()
test_bpf_modifier()
{
printf "Testing bpf event modifier "
stat_output=$(perf stat --no-big-num -e instructions/name=base_instructions/,instructions/name=bpf_instructions/b -- $workload 2>&1)
stat_output=$(perf stat --no-big-num \
-e instructions/name=base_instructions/u,instructions/name=bpf_instructions/bu \
-- "${workload[@]}" 2>&1)
base_instructions=$(echo "$stat_output"| \
awk -v i=0 -v c=0 '/base_instructions/ { \
if ($1 != "<not") { i++; c += $1 } \