The annotation_line__print() has many arguments. But min_percent,
max_lines and percent_type are from struct annotaion_options. So let's
pass a pointer to the option instead of passing them separately to
reduce the number of function arguments.
Actually it has a recursive call if 'queue' is set. Add a new option
instance to pass different values for the case.
Reviewed-by: Ian Rogers <irogers@google.com>
Link: https://lore.kernel.org/r/20250310224925.799005-4-namhyung@kernel.org
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Factor out a function to get the name of member field at the given
offset. This will be used in other places.
Also update the output of typeoff sort key a little bit. As we know
that some special types like (stack operation), (stack canary) and
(unknown) won't have fields, skip printing the offset and field.
For example, the following change is expected.
"(stack operation) +0 (no field)" ==> "(stack operation)"
Reviewed-by: Ian Rogers <irogers@google.com>
Link: https://lore.kernel.org/r/20250310224925.799005-2-namhyung@kernel.org
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
When BPF collects the stats for the latency in usec, it first divides
the time by 1000. But that means it would have 0 if the delta is small
and won't update the total time properly.
Let's keep the stats in nsec always and adjust to usec before printing.
Before:
$ sudo ./perf ftrace latency -ab -T mutex_lock --hide-empty -- sleep 0.1
# DURATION | COUNT | GRAPH |
0 - 1 us | 765 | ############################################# |
1 - 2 us | 10 | |
2 - 4 us | 2 | |
4 - 8 us | 5 | |
# statistics (in usec)
total time: 0 <<<--- (here)
avg time: 0
max time: 6
min time: 0
count: 782
After:
$ sudo ./perf ftrace latency -ab -T mutex_lock --hide-empty -- sleep 0.1
# DURATION | COUNT | GRAPH |
0 - 1 us | 880 | ############################################ |
1 - 2 us | 13 | |
2 - 4 us | 8 | |
4 - 8 us | 3 | |
# statistics (in usec)
total time: 268 <<<--- (here)
avg time: 0
max time: 6
min time: 0
count: 904
Tested-by: Athira Rajeev <atrajeev@linux.ibm.com>
Cc: Gabriele Monaco <gmonaco@redhat.com>
Link: https://lore.kernel.org/r/20250227191223.1288473-1-namhyung@kernel.org
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Add a loop and helper function to avoid repetition, the loop uses
arrays so switch the shell to bash. Add additional topdown group tests
where a topdown event needs to be moved beyond others and the slots
event isn't first in the target group. This replicates issues that
occur on hybrid systems where the other events are for the cpu_atom
PMU. Test with both PMU and software events. Place the slots event
later in the event list.
Signed-off-by: Ian Rogers <irogers@google.com>
Link: https://lore.kernel.org/r/20250307023906.1135613-5-irogers@google.com
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
When running topdown leader smapling test on Intel hybrid platforms,
such as LNL/ARL, we see the below error.
Topdown leader sampling test
Topdown leader sampling [Failed topdown events not reordered correctly]
It indciates the below command fails.
perf record -o "${perfdata}" -e "{instructions,slots,topdown-retiring}:S" true
The root cause is that perf tool creats a perf event for each PMU type
if it can create.
As for this command, there would be 5 perf events created,
cpu_atom/instructions/,cpu_atom/topdown_retiring/,
cpu_core/slots/,cpu_core/instructions/,cpu_core/topdown-retiring/
For these 5 events, the 2 cpu_atom events are in a group and the other 3
cpu_core events are in another group.
When arch_topdown_sample_read() traverses all these 5 events, events
cpu_atom/instructions/ and cpu_core/slots/ don't have a same group
leade, and then return false directly and lead to cpu_core/slots/ event
is used to sample and this is not allowed by PMU driver.
It's a overkill to return false directly if "evsel->core.leader !=
leader->core.leader" since there could be multiple groups in the event
list.
Just "continue" instead of "return false" to fix this issue.
Fixes: 1e53e9d178 ("perf x86/topdown: Correct leader selection with sample_read enabled")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Tested-by: Thomas Falcon <thomas.falcon@intel.com>
Tested-by: Ian Rogers <irogers@google.com>
Link: https://lore.kernel.org/r/20250307023906.1135613-2-irogers@google.com
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Fedora introduced a "MiniDebuginfo" feature, in which an LZMA-compressed
ELF file is placed inside a section named ".gnu_debugdata". This file
contains nothing but a symbol table, which can be used to supplement the
.dynsym section which only contains required symbols for runtime.
It is supported by GDB for stack traces, but it should be useful for
tracing as well. Implement support for loading symbols from
.gnu_debugdata.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Reviewed-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: https://lore.kernel.org/r/20250307232206.2102440-4-stephen.s.brennan@oracle.com
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
When preparing the mem events for the argv copies are intentionally
made. These copies are leaked and cause runs of perf using address
sanitizer to fail. Rather than leak the memory allocate a chunk of
memory for the mem event names upfront and build the strings in this -
the storage is sized larger than the previous buffer size. The caller
is then responsible for clearing up this memory. As part of this
change, remove the mem_loads_name and mem_stores_name global buffers
then change the perf_pmu__mem_events_name to write to an out argument
buffer.
Signed-off-by: Ian Rogers <irogers@google.com>
Tested-by: Thomas Falcon <thomas.falcon@intel.com>
Reviewed-by: Leo Yan <leo.yan@arm.com>
Link: https://lore.kernel.org/r/20250308012853.1384762-1-irogers@google.com
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
With LTO builds, although regular builds could also see this as
all the code is in one file, the datasym workload can realize the
buf1.reserved data is never accessed. The compiler moves the
variable to bss and only keeps the data1 and data2 parts as
separate variables. This causes the symbol check to fail in the
test. Make the variable volatile to disable the more aggressive
optimization. Rename the variable to make which buf1 in perf is
being referred to.
Before:
$ perf test -vv "data symbol"
126: Test data symbol:
--- start ---
test child forked, pid 299808
perf does not have symbol 'buf1'
perf is missing symbols - skipping test
---- end(-2) ----
126: Test data symbol : Skip
$ nm perf|grep buf1
0000000000a5fa40 b buf1.0
0000000000a5fa48 b buf1.1
After:
$ nm perf|grep buf1
0000000000a53a00 d buf1
$ perf test -vv "data symbol"126: Test data symbol:
--- start ---
test child forked, pid 302166
a53a00-a53a39 l buf1
perf does have symbol 'buf1'
Recording workload...
Waiting for "perf record has started" message
OK
Cleaning up files...
---- end(0) ----
126: Test data symbol : Ok
Fixes: 3dfc01fe9d ("perf test: Add 'datasym' test workload")
Signed-off-by: Ian Rogers <irogers@google.com>
Link: https://lore.kernel.org/r/20250226230109.314580-1-irogers@google.com
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Ian told me that there are many memory leaks in the hierarchy mode. I
can easily reproduce it with the follwing command.
$ make DEBUG=1 EXTRA_CFLAGS=-fsanitize=leak
$ perf record --latency -g -- ./perf test -w thloop
$ perf report -H --stdio
...
Indirect leak of 168 byte(s) in 21 object(s) allocated from:
#0 0x7f3414c16c65 in malloc ../../../../src/libsanitizer/lsan/lsan_interceptors.cpp:75
#1 0x55ed3602346e in map__get util/map.h:189
#2 0x55ed36024cc4 in hist_entry__init util/hist.c:476
#3 0x55ed36025208 in hist_entry__new util/hist.c:588
#4 0x55ed36027c05 in hierarchy_insert_entry util/hist.c:1587
#5 0x55ed36027e2e in hists__hierarchy_insert_entry util/hist.c:1638
#6 0x55ed36027fa4 in hists__collapse_insert_entry util/hist.c:1685
#7 0x55ed360283e8 in hists__collapse_resort util/hist.c:1776
#8 0x55ed35de0323 in report__collapse_hists /home/namhyung/project/linux/tools/perf/builtin-report.c:735
#9 0x55ed35de15b4 in __cmd_report /home/namhyung/project/linux/tools/perf/builtin-report.c:1119
#10 0x55ed35de43dc in cmd_report /home/namhyung/project/linux/tools/perf/builtin-report.c:1867
#11 0x55ed35e66767 in run_builtin /home/namhyung/project/linux/tools/perf/perf.c:351
#12 0x55ed35e66a0e in handle_internal_command /home/namhyung/project/linux/tools/perf/perf.c:404
#13 0x55ed35e66b67 in run_argv /home/namhyung/project/linux/tools/perf/perf.c:448
#14 0x55ed35e66eb0 in main /home/namhyung/project/linux/tools/perf/perf.c:556
#15 0x7f340ac33d67 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
...
$ perf report -H --stdio 2>&1 | grep -c '^Indirect leak'
93
I found that hist_entry__delete() missed to release child entries in the
hierarchy tree (hroot_{in,out}). It needs to iterate the child entries
and call hist_entry__delete() recursively.
After this change:
$ perf report -H --stdio 2>&1 | grep -c '^Indirect leak'
0
Reported-by: Ian Rogers <irogers@google.com>
Tested-by Thomas Falcon <thomas.falcon@intel.com>
Reviewed-by: Ian Rogers <irogers@google.com>
Link: https://lore.kernel.org/r/20250307061250.320849-2-namhyung@kernel.org
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
When doing "perf annotate", perf tool provides option to
use specific disassembler like llvm/objdump/capstone. The
order picked is to use llvm first and if that fails fallback
to objdump ie to use PERF_DISASM_LLVM, PERF_DISASM_CAPSTONE
and PERF_DISASM_OBJDUMP
In powerpc, when using "data type" sort keys, first preferred
approach is to read the raw instruction from the DSO. In objdump
is specified in "--objdump" option, it picks the symbol disassemble
using objdump. Currently disasm_line__parse_powerpc() function
uses length of the "line" to determine if objdump is used.
But there are few cases, where if objdump doesn't recognise the
instruction, the disassembled string will be empty.
Example:
134cdc: c4 05 82 41 beq 1352a0 <getcwd+0x6e0>
134ce0: ac 00 8e 40 bne cr3,134d8c <getcwd+0x1cc>
134ce4: 0f 00 10 04 pld r9,1028308
====>134ce8: d4 b0 20 e5
134cec: 16 00 40 39 li r10,22
134cf0: 48 01 21 ea ld r17,328(r1)
So depending on length of line will give bad results.
Add a new filed to annotation options structure,
"struct annotation_options" to save the disassembler used.
Use this info to determine if disassembly is done while
parsing the disasm line.
Reported-by: Tejas Manhas <Tejas.Manhas1@ibm.com>
Signed-off-by: Athira Rajeev <atrajeev@linux.ibm.com>
Tested-By: Venkat Rao Bagalkote <venkat88@linux.ibm.com>
Link: https://lore.kernel.org/r/20250304154114.62093-1-atrajeev@linux.ibm.com
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
The length of PERF_RECORD_KSYMBOL for BPF is a size of JITed code so
it'd be 0 when it's not JITed. The ksymbol is needed to symbolize the
code when it gets samples in the region but non-JITed code cannot get
samples. Thus it'd be ok to ignore them.
Actually it caused a performance issue in the perf tools on old ARM
kernels where it can refuse to JIT some BPF codes. It ended up
splitting the existing kernel map (kallsyms). And later lookup for a
kernel symbol would create a new kernel map from kallsyms and then
split it again and again. :(
Probably there's a bug in the kernel map/symbol handling in perf tools.
But I think we need to fix this anyway.
Reported-by: Kevin Nomura <nomurak@google.com>
Acked-by: Song Liu <song@kernel.org>
Link: https://lore.kernel.org/r/20250305232838.128692-1-namhyung@kernel.org
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
This was detected at the end of a 'perf record' session when build-id
collection was enabled and thus the BPF programs put in place while the
session was running, some even put in place by perf itself were
processed and inserted, with some overlaps related to BPF trampolines
and programs took place.
Using maps__fixup_overlap_and_insert() instead of maps__insert() "fixes"
the problem, in the sense that overlaps will be dealt with and then the
consistency will be kept, but it would be interesting to fully
understand why such overlaps take place and how to deal with them when
doing symbol resolution.
Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Suggested-by: Ian Rogers <irogers@google.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Reviewed-by: Ian Rogers <irogers@google.com>
Link: https://lore.kernel.org/lkml/CAP-5=fXEEMFgPF2aZhKsfrY_En+qoqX20dWfuE_ad73Uxf0ZHQ@mail.gmail.com
Link: https://lore.kernel.org/r/20250228211734.33781-7-acme@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
We can't just replacing the map in the maps_by_address and not touching
on the maps_by_name, that would leave the refcount as 1 and thus trip
another consistency check, this one:
perf: util/maps.c:110: check_invariants:
Assertion `refcount_read(map__refcnt(map)) > 1' failed.
106 /*
107 * Maps by name maps should be in maps_by_address, so
108 * the reference count should be higher.
109 */
110 assert(refcount_read(map__refcnt(map)) > 1);
Committer notice:
Initialize the newly added 'ni' variable, that really can't be
accessed unitialized trips some gcc versions, like:
12 20.00 archlinux:base : FAIL gcc version 13.2.1 20230801 (GCC)
util/maps.c: In function ‘__maps__fixup_overlap_and_insert’:
util/maps.c:896:54: error: ‘ni’ may be used uninitialized [-Werror=maybe-uninitialized]
896 | map__put(maps_by_name[ni]);
| ^
util/maps.c:816:25: note: ‘ni’ was declared here
816 | unsigned int i, ni;
| ^~
cc1: all warnings being treated as errors
make[3]: *** [/git/perf-6.14.0-rc1/tools/build/Makefile.build:138: util] Error 2
Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Reviewed-by: Ian Rogers <irogers@google.com>
Link: https://lore.kernel.org/lkml/Z79std66tPq-nqsD@google.com
Link: https://lore.kernel.org/r/20250228211734.33781-5-acme@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
