Pull new perf tool feature from Arnaldo Carvalho de Melo:
" User visible changes:
- Generate perf.data files from 'perf stat', to tap into the scripting
capabilities perf has instead of defining a 'perf stat' specific scripting
support to calculate event ratios, etc. Simple example:
$ perf stat record -e cycles usleep 1
Performance counter stats for 'usleep 1':
1,134,996 cycles
0.000670644 seconds time elapsed
$ perf stat report
Performance counter stats for '/home/acme/bin/perf stat record -e cycles usleep 1':
1,134,996 cycles
0.000670644 seconds time elapsed
$
It generates PERF_RECORD_ userspace records to store the details:
$ perf report -D | grep PERF_RECORD
0xf0 [0x28]: PERF_RECORD_THREAD_MAP nr: 1 thread: 27637
0x118 [0x12]: PERF_RECORD_CPU_MAP nr: 1 cpu: 65535
0x12a [0x40]: PERF_RECORD_STAT_CONFIG
0x16a [0x30]: PERF_RECORD_STAT
-1 -1 0x19a [0x40]: PERF_RECORD_MMAP -1/0: [0xffffffff81000000(0x1f000000) @ 0xffffffff81000000]: x [kernel.kallsyms]_text
0x1da [0x18]: PERF_RECORD_STAT_ROUND
[acme@ssdandy linux]$
An effort was made to make perf.data files generated like this to not
generate cryptic messages when processed by older tools.
The 'perf script' bits need rebasing, will go up later.
Jiri's cover letter for this series:
The initial attempt defined its own formula lang and allowed triggering user's
script on the end of the stat command:
http://marc.info/?l=linux-kernel&m=136742146322273&w=2
This patchset abandons the idea of new formula language and rather adds support
to:
- store stat data into perf.data file
- add python support to process stat events
Basically it allows to store stat data into perf.data and post process it with
python scripts in a similar way we do for sampling data.
The stat data are stored in new stat, stat-round, stat-config user events.
stat - stored for each read syscall of the counter
stat round - stored for each interval or end of the command invocation
stat config - stores all the config information needed to process data
so report tool could restore the same output as record
The python script can now define 'stat__<eventname>_<modifier>' functions
to get stat events data and 'stat__interval' to get stat-round data.
See CPI script example in scripts/python/stat-cpi.py."
Also a few other changes:
User visible changes:
- Make command line options always available, even when they
depend on some feature being enabled, warning the user about
use of such options (Wang Nan)
- Support --vmlinux in perf record, useful, so far, for eBPF,
where we will set up events that will be used in the record
session (He Kuang)
- Automatically disable collecting branch flags and cycles with
--call-graph lbr. This allows avoiding a bunch of extra MSR
reads in the PMI on Skylake. (Andi Kleen)
Infrastructure changes:
- Dump the stack when a 'perf test -v ' entry segfaults, so far we
would have to run it under gdb with 'set follow-fork-mode child'
set to get a proper backtrace (Arnaldo Carvalho de Melo)
- Initialize the refcnt in 'struct thread' to 1 and fixup its
users accordingly, so that we try to have the same refcount
model accross the perf codebase (Arnaldo Carvalho de Melo)
- More prep work for moving the subcmd infrastructure out of
tools/perf/ and into tools/lib/subcmd/ to be used by other
tools/ living utilities (Josh Poimboeuf)
- Fix 'perf test' hist testcases when kptr_restrict is on (Namhyung Kim)
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Pull perf/urgent tooling fix from Arnaldo Carvalho de Melo:
User visible changes:
- Fix 'perf list' segfault due to lack of support for PERF_CONF_SW_BPF_OUTPUT
in an array used just for printing available events, robustify the code
involved (Arnaldo Carvalho de Melo)
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Pull perf/core improvements from Arnaldo Carvalho de Melo:
User visible changes:
- Add record.build-id config option to 'perf record', to allow configuring
in the ~/.perfconfig file if and how build-ids should be processed, allowing
a permanent setting for options such as -B and -N: (Namhyung Kim)
$ perf record -h -B -N
Usage: perf record [<options>] [<command>]
or: perf record [<options>] -- <command> [<options>]
-B, --no-buildid do not collect buildids in perf.data
-N, --no-buildid-cache do not update the buildid cache
$
Infrastructure changes:
- Move code for options parsing and subcommand handling from tools/perf/
to tools/lib/subcmd/, so that it can be used by other tools/ living
utilities (Josh Poimboeuf)
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Adding processing of event update events, so perf stat report can store
additional info for events - unit,scale,name.
Committer note:
Before:
# perf stat record -e power/energy-cores/ -a
^C
Performance counter stats for 'system wide':
77.41 Joules power/energy-cores/
1.597176695 seconds time elapsed
# perf stat report
Performance counter stats for '/home/acme/bin/perf stat record -e power/energy-cores/ -a':
332,488,114,176 power/energy-cores/
1.597176695 seconds time elapsed
#
After, using the same perf.data file generated in the "Before" case
above:
# perf stat report
Performance counter stats for '/home/acme/bin/perf stat record -e power/energy-cores/ -a':
77.41 Joules power/energy-cores/
1.597176695 seconds time elapsed
#
Reported-by: Kan Liang <kan.liang@intel.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1446734469-11352-17-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Adding processing of stat and stat round events.
The stat data com in stat events, using generic function
process_stat_round_event to store data under perf_evsel object.
The stat-round events comes each interval or as last event in non
interval mode. The function process_stat_round_event process stored data
for each perf_evsel object and print it out.
Committer note:
After this patch:
$ perf stat record usleep 1
Performance counter stats for 'usleep 1':
0.498381 task-clock (msec) # 0.571 CPUs utilized
2 context-switches # 0.004 M/sec
0 cpu-migrations # 0.000 K/sec
149 page-faults # 0.299 M/sec
1,271,635 cycles # 2.552 GHz
928,712 stalled-cycles-frontend # 73.03% frontend cycles idle
663,286 stalled-cycles-backend # 52.16% backend cycles idle
792,614 instructions # 0.62 insns per cycle
# 1.17 stalled cycles per insn
136,850 branches # 274.589 M/sec
<not counted> branch-misses (0.00%)
0.000873419 seconds time elapsed
$
$ perf stat report
Performance counter stats for '/home/acme/bin/perf stat record usleep 1':
0.498381 task-clock (msec) # 0.571 CPUs utilized
2 context-switches # 0.004 M/sec
0 cpu-migrations # 0.000 K/sec
149 page-faults # 0.299 M/sec
1,271,635 cycles # 2.552 GHz
928,712 stalled-cycles-frontend # 73.03% frontend cycles idle
663,286 stalled-cycles-backend # 52.16% backend cycles idle
792,614 instructions # 0.62 insns per cycle
# 1.17 stalled cycles per insn
136,850 branches # 274.589 M/sec
<not counted> branch-misses (0.00%)
0.000873419 seconds time elapsed
$
Reported-by: Kan Liang <kan.liang@intel.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1446734469-11352-16-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We currently don't support storing multiple session in perf.data,
so we can't allow -r option in stat record.
$ perf stat -e cycles -r 2 record ls
Cannot use -r option with perf stat record.
Committer note:
Before this patch we would a perf.data file such as:
$ perf stat -e cycles -r 2 record ls
<SNIP>
Performance counter stats for 'ls' (2 runs):
3,935,236 cycles
0.002353261 seconds time elapsed ( +- 4.76% )
$ perf report -D | grep PERF_RECORD | grep ROUND
0xf0 [0]: failed to process type: 16
Error:
failed to process sample
$
Reported-by: Kan Liang <kan.liang@intel.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1446734469-11352-10-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Allowing storing stat record data into pipe, so report tools
(report/script) could read data directly from record.
Committer note:
Before this patch:
$ perf stat record -o - usleep 1 | perf report -i -
incompatible file format (rerun with -v to learn more)
$ perf stat record -o - usleep 1 | perf script -i -
incompatible file format (rerun with -v to learn more)
$ ls -la perf.data
ls: cannot access perf.data: No such file or directory
$
After:
$ perf stat record -o - usleep 1 | perf report -i -
# To display the perf.data header info, please use
# --header/--header-only options.
#
Error:
The - file has no samples!
$ perf stat record -o - usleep 1 | perf script -i -
Display of symbols requested but neither sample IP nor sample address
is selected. Hence, no addresses to convert to symbols.
0 [0x80]: failed to process type: 64
$ ls -la perf.data
ls: cannot access perf.data: No such file or directory
$
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Tested-by: Kan Liang <kan.liang@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1446734469-11352-7-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Synthesizing needed stat record data for report/script:
- cpu/thread maps
- stat config
Committer note:
New records generated on a perf.data file with this patch:
$ perf report -D | grep PERF_RECORD_
0x568 [0x28]: PERF_RECORD_THREAD_MAP nr: 1 thread: 29097
0x590 [0x12]: PERF_RECORD_CPU_MAP nr: 1 cpu: 65535
0x5a2 [0x40]: PERF_RECORD_STAT_CONFIG
$
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Tested-by: Kan Liang <kan.liang@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1446734469-11352-5-git-send-email-jolsa@kernel.org
[ Adjusted wrt kernel PERF_RECORD_MMAP added when introducing 'perf stat record' ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add 'perf stat record' command support. It creates simple (header only)
perf.data file ATM.
The record command could be specified anywhere among stat options. All
stat command options are valid for stat record command with '-o' option
exception. If specified for record command it denotes the perf data file
name.
Committer note:
Set sample_type to PERF_SAMPLE_IDENTIFIER, which should be harmless
while avoiding that older tools show confusing messages, for instance,
with sample_type = 0, we get:
$ perf stat record usleep 1
Performance counter stats for 'usleep 1':
0.630237 task-clock (msec) # 0.528 CPUs utilized
1 context-switches # 0.002 M/sec
0 cpu-migrations # 0.000 K/sec
52 page-faults # 0.083 M/sec
978,312 cycles # 1.552 GHz
671,931 stalled-cycles-frontend # 68.68% frontend cycles idle
<not supported> stalled-cycles-backend
646,379 instructions # 0.66 insns per cycle
# 1.04 stalled cycles per insn
131,046 branches # 207.931 M/sec
7,073 branch-misses # 5.40% of all branches
0.001193240 seconds time elapsed
$ oldperf evlist
WARNING: The perf.data file's data size field is 0 which is unexpected.
Was the 'perf record' command properly terminated?
non matching sample_type
$
While with sample_type set to PERF_SAMPLE_IDENTIFIER, after we re-run 'perf
stat record usleep' we get:
$ oldperf evlist
WARNING: The perf.data file's data size field is 0 which is unexpected.
Was the 'perf record' command properly terminated?
task-clock
context-switches
cpu-migrations
page-faults
cycles
stalled-cycles-frontend
stalled-cycles-backend
instructions
branches
branch-misses
$
Which at least shows the names of the events in the perf.data file.
Additionally, such files, when passed to 'perf report' will produce:
$ oldperf report --stdio
WARNING: The perf.data file's data size field is 0 which is unexpected.
Was the 'perf record' command properly terminated?
Warning:
Kernel address maps (/proc/{kallsyms,modules}) were restricted.
Check /proc/sys/kernel/kptr_restrict before running 'perf record'.
As no suitable kallsyms nor vmlinux was found, kernel samples
can't be resolved.
Samples in kernel modules can't be resolved as well.
Error:
The perf.data file has no samples!
# To display the perf.data header info, please use --header/--header-only options.
#
$
Which is confusing and can be solved by just adding the kernel mmap record,
which will also remove that warning about the data size field being equal to
zero, after generating the mmap record:
$ perf stat record usleep 1
Performance counter stats for 'usleep 1':
0.600796 task-clock (msec) # 0.478 CPUs utilized
1 context-switches # 0.002 M/sec
0 cpu-migrations # 0.000 K/sec
54 page-faults # 0.090 M/sec
886,844 cycles # 1.476 GHz
582,169 stalled-cycles-frontend # 65.65% frontend cycles idle
<not supported> stalled-cycles-backend
638,344 instructions # 0.72 insns per cycle
# 0.91 stalled cycles per insn
130,204 branches # 216.719 M/sec
7,500 branch-misses # 5.76% of all branches
0.001255897 seconds time elapsed
$ oldperf evlist
task-clock
context-switches
cpu-migrations
page-faults
cycles
stalled-cycles-frontend
stalled-cycles-backend
instructions
branches
branch-misses
$ oldperf report --stdio
Error:
The perf.data file has no samples!
# To display the perf.data header info, please use --header/--header-only options.
#
[acme@zoo linux]$
No warnings, sensible output about what are the events in the perf.data file and also
a "file has no samples" message, which indeed it doesn't.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Tested-by: Kan Liang <kan.liang@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: htp://lkml.kernel.org/r/1446734469-11352-3-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Adding the cpu_map event to pass/store cpu maps as data in
a pipe/perf.data.
We store maps in 2 formats:
- list of cpus
- mask of cpus
The format that takes less space is selected transparently in the
following patch.
The interface is made generic, so we could add the cpumap event data
into another event in the following patches.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Kan Liang <kan.liang@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1445784728-21732-8-git-send-email-jolsa@kernel.org
[ cpu_map_data_cpus -> cpu_map_entries, cpu_map_data_mask -> cpu_map_mask ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
