Hook into session processing so that AUX area decoding can synthesize
events transparently to the tools.
The advantages of transparent decoding are that tools can be used
directly with perf.data files containing AUX area tracing data, which is
easier for the user and more efficient than having a separate decoding
tool.
This will work as follows:
1. Tools will feed auxtrace events to the decoder using
perf_tool->auxtrace() (support for that still to come).
2. The decoder can process side-band events as needed due
to the auxtrace->process_event() hook.
3. The decoder can deliver synthesized events into the
event stream using perf_session__deliver_synth_event().
Note the expectation is that decoding will work on data that is
time-ordered with respect to the per-cpu or per-thread contexts that
were recorded.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/1428594864-29309-9-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add support for reading from the AUX area tracing mmap and synthesizing
AUX area tracing events.
This patch introduces an abstraction for recording AUX area data.
Recording is initialized by auxtrace_record__init() which is a weak
function to be implemented by the architecture to provide recording
callbacks.
Recording is mainly handled by auxtrace_mmap__read() and
perf_event__synthesize_auxtrace() but there are callbacks for
miscellaneous needs including validating and processing user options,
populating private data in auxtrace_info_event, and freeing the
structure when finished.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Namhyung Kim <namhyung@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/1428594864-29309-5-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add two user events for AUX area tracing.
PERF_RECORD_AUXTRACE_INFO contains metadata, consisting primarily the
type of the AUX area tracing data plus some amount of
architecture-specific information. There should be only one
PERF_RECORD_AUXTRACE_INFO event.
PERF_RECORD_AUXTRACE identifies AUX area tracing data copied from the
mmapped AUX area tracing region. The actual data is not part of the
event but immediately follows it.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Namhyung Kim <namhyung@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/1428594864-29309-4-git-send-email-adrian.hunter@intel.com
[ s/MIN/min/g and use cast to fix up wrt -Werror=sign-compare till we adopt min_t() ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This patch supports the addition to the kernel of AUX area buffers that
can be mmapped separately from the perf-events buffer.
The AUX buffer can be configured to contain hardware-produced trace
information. The first implementation will support Intel BTS and Intel
PT.
One auxtrace buffer is mmapped per perf-events buffer. If the requested
auxtrace buffer size is zero, which it will be until further support is
added, then no auxtrace mmapping is attempted.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/1428594864-29309-3-git-send-email-adrian.hunter@intel.com
[ Fixed conflict in evlist.h ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
When converting int values, perf first extractes it to a ulonglong, then
feeds it to babeltrace as a signed value.
For negative 32 bit values (for example, return values of failed
syscalls), the extracted data should be something like 0xfffffffe (-2).
It becomes a large int64 value.
Babeltrace denies to insert it with bt_ctf_field_signed_integer_set_value()
because it is larger than 0x7fffffff, the largest positive value a
32 bit int can be.
This patch introduces adjust_signedness(), which fills high bits of
ulonglong with 1 if the value is negative.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jeremie Galarneau <jgalar@efficios.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Tom Zanussi <tzanussi@gmail.com>
Link: http://lkml.kernel.org/r/1429372220-6406-8-git-send-email-jolsa@kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
[ s/signess/signedness/g ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
0d68bc92c4 breaks compiles on RHEL6/OL6:
cc1: warnings being treated as errors
builtin-kmem.c: In function ‘search_page_alloc_stat’:
builtin-kmem.c:322: error: declaration of ‘stat’ shadows a global declaration
node = &parent->rb_left;
/usr/include/sys/stat.h:455: error: shadowed declaration is here
builtin-kmem.c: In function ‘perf_evsel__process_page_alloc_event’:
builtin-kmem.c:378: error: declaration of ‘stat’ shadows a global declaration
/usr/include/sys/stat.h:455: error: shadowed declaration is here
builtin-kmem.c: In function ‘perf_evsel__process_page_free_event’:
builtin-kmem.c:431: error: declaration of ‘stat’ shadows a global declaration
/usr/include/sys/stat.h:455: error: shadowed declaration is here
Rename local variable to pstat to avoid the name conflict.
Signed-off-by: David Ahern <david.ahern@oracle.com>
Link: http://lkml.kernel.org/r/1429033773-31383-1-git-send-email-david.ahern@oracle.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Pull perf/core improvements and fixes from Arnaldo Carvalho de Melo:
New features:
- Analyze page allocator events in 'perf kmem' (Namhyung Kim)
User visible changes:
- Fix retprobe 'perf probe' handling when failing to find needed debuginfo (He Kuang)
- lazy_line probe fixes in 'perf probe' (Naohiro Aota, He Kuang)
Infrastructure changes:
- Record pfn instead of pointer to struct page in tracepoints (Namhyung Kim)
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
The first argument passed to find_probe_point_lazy() should be CU die,
which will be passed to die_walk_lines() when lazy_line matches.
Currently, when we probe with lazy_line pattern to file without function
name, NULL pointer is passed and causes a segment fault.
Can be reproduced as following:
$ perf probe -k vmlinux --add='fs/super.c;s->s_count=1;'
[ 1958.984658] perf[1020]: segfault at 10 ip 00007fc6e10d8c71 sp
00007ffcbfaaf900 error 4 in libdw-0.161.so[7fc6e10ce000+34000]
Segmentation fault
After this patch:
$ perf probe -k vmlinux --add='fs/super.c;s->s_count=1;'
Added new event:
probe:_stext (on @fs/super.c)
You can now use it in all perf tools, such as:
perf record -e probe:_stext -aR sleep 1
Signed-off-by: He Kuang <hekuang@huawei.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Acked-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1428925290-5623-3-git-send-email-hekuang@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
When perf probe searched in a debuginfo file and failed, it tried with
an alternative, in function get_alternative_probe_event():
memcpy(tmp, &pev->point, sizeof(*tmp));
memset(&pev->point, 0, sizeof(pev->point));
In this case, it drops the retprobe flag and forgets to set it back in
find_alternative_probe_point(), so the problem occurs.
Can be reproduced as following:
$ perf probe -v -k vmlinux --add='sys_write%return'
...
Added new event:
Writing event: p:probe/sys_write _stext+1584952
probe:sys_write (on sys_write%return)
$ cat /sys/kernel/debug/tracing/kprobe_events
p:probe/sys_write _stext+1584952
After this patch:
$ perf probe -v -k vmlinux --add='sys_write%return'
Added new event:
Writing event: r:probe/sys_write SyS_write+0
probe:sys_write (on sys_write%return)
$ cat /sys/kernel/debug/tracing/kprobe_events
r:probe/sys_write SyS_write
Signed-off-by: He Kuang <hekuang@huawei.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Acked-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1428925290-5623-1-git-send-email-hekuang@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Dan Carpenter pointed out that the control flow in pt_pmu_hw_init()
is a bit messy: for example the kfree(de_attrs) is entirely
superfluous.
Another problem is the inconsistent mixing of label based and
direct return error handling.
Add modern, label based error handling instead and clean up the code
a bit as well.
Note that we'll still do a kfree(NULL) in the normal case - this does
not matter as this is an init path and kfree() returns early if it
sees a NULL.
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/20150409090805.GG17605@mwanda
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Pull perf/core improvements and fixes from Arnaldo Carvalho de Melo:
New user visible features:
- Support multiple probes on different binaries on the same command line (Masami Hiramatsu)
User visible changes:
- Fix synthesizing fork_event.ppid for non-main thread (David Ahern)
- Fix cross-endian analysis (David Ahern)
- Fix segfault in 'perf buildid-list' when show DSOs with hits (He Kuang)
Infrastructure changes:
- Fix type for references to data_head/tail (David Ahern)
- Fix error path to do closedir() when synthesizing threads (Arnaldo Carvalho de Melo)
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
To avoid probing in unintended binary, the orphaned -x option must be
checked and warned.
Without this patch, following command sets up the probe in the kernel.
-----
# perf probe -a strcpy -x ./perf
Added new event:
probe:strcpy (on strcpy)
You can now use it in all perf tools, such as:
perf record -e probe:strcpy -aR sleep 1
-----
But in this case, it seems that the user may want to probe in the perf
binary. With this patch, perf-probe correctly handles the orphaned -x.
-----
# perf probe -a strcpy -x ./perf
Error: -x/-m must follow the probe definitions.
...
-----
Reported-by: Jiri Olsa <jolsa@redhat.com>
Acked-by: Jiri Olsa <jolsa@redhat.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20150401102541.17137.75477.stgit@localhost.localdomain
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Support multiple probes on different binaries with just
one command.
In the result, this example sets up the probes on icmp_rcv in
kernel, on main and set_target in perf, and on pcspkr_event
in pcspker.ko driver.
-----
# perf probe -a icmp_rcv -x ./perf -a main -a set_target \
-m /lib/modules/4.0.0-rc5+/kernel/drivers/input/misc/pcspkr.ko \
-a pcspkr_event
Added new event:
probe:icmp_rcv (on icmp_rcv)
You can now use it in all perf tools, such as:
perf record -e probe:icmp_rcv -aR sleep 1
Added new event:
probe_perf:main (on main in /home/mhiramat/ksrc/linux-3/tools/perf/perf)
You can now use it in all perf tools, such as:
perf record -e probe_perf:main -aR sleep 1
Added new event:
probe_perf:set_target (on set_target in /home/mhiramat/ksrc/linux-3/tools/perf/perf)
You can now use it in all perf tools, such as:
perf record -e probe_perf:set_target -aR sleep 1
Added new event:
probe:pcspkr_event (on pcspkr_event in pcspkr)
You can now use it in all perf tools, such as:
perf record -e probe:pcspkr_event -aR sleep 1
-----
Reported-by: Arnaldo Carvalho de Melo <acme@infradead.org>
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20150401102539.17137.46454.stgit@localhost.localdomain
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
commit: f3b623b849 ("perf tools: Reference count struct thread")
appends every thread->node to dead_threads in machine__remove_thread()
and list_del_init() this node in thread__put().
perf_event__exit_del_thread() releases thread wihout using
machine__remove_thread(), and causes a NULL pointer crash when
list_del_init(&thread->node) is called. Fix this by using
machine_remove_thread() instead of using thread__put() directly.
This problem can be reproduced as following:
$ perf record ls
$ perf buildid-list --with-hits
[ 3874.195070] perf[1018]: segfault at 0 ip 00000000004b0b15 sp
00007ffc35b44780 error 6 in perf[400000+166000]
Segmentation fault
After this patch:
$ perf record ls
$ perf buildid-list --with-hits
bc23e7c3281e542650ba4324421d6acf78f4c23e /proc/kcore
643324cb0e969f30c56d660f167f84a150845511 [vdso]
0000000000000000000000000000000000000000 /bin/busybox
...
Signed-off-by: He Kuang <hekuang@huawei.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1428658500-6483-1-git-send-email-hekuang@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Pull perf/core improvements and fixes from Arnaldo Carvalho de Melo:
User visible changes:
- Teach 'perf record' about perf_event_attr.clockid (Peter Zijlstra)
- Improve 'perf sched replay' on high CPU core count machines (Yunlong Song)
- Consider PERF_RECORD_ events with cpumode == 0 in 'perf top', removing one
cause of long term memory usage buildup, i.e. not processing PERF_RECORD_EXIT
events (Arnaldo Carvalho de Melo)
- Add 'I' event modifier for perf_event_attr.exclude_idle bit (Jiri Olsa)
- Respect -i option 'in perf kmem' (Jiri Olsa)
Infrastructure changes:
- Honor operator priority in libtraceevent (Namhyung Kim)
- Merge all perf_event_attr print functions (Peter Zijlstra)
- Check kmaps access to make code more robust (Wang Nan)
- Fix inverted logic in perf_mmap__empty() (He Kuang)
- Fix ARM 32 'perf probe' building error (Wang Nan)
- Fix perf_event_attr tests (Jiri Olsa)
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
The soft maximum number of open files for a calling process is 1024,
which is defined as INR_OPEN_CUR in include/uapi/linux/fs.h, and the
hard maximum number of open files for a calling process is 4096, which
is defined as INR_OPEN_MAX in include/uapi/linux/fs.h.
Both INR_OPEN_CUR and INR_OPEN_MAX are used to limit the value of
RLIMIT_NOFILE in include/asm-generic/resource.h.
And the soft maximum number finally decides the limitation of the
maximum files which are allowed to be opened.
That is to say a process can use at most 1024 file descriptors for its
o pened files, or an EMFILE error will happen.
This error can be fixed by increasing the soft maximum number, under the
constraint that the soft maximum number can not exceed the hard maximum
number, or both soft and hard maximum number should be increased
simultaneously with privilege.
For perf sched replay, it uses sys_perf_event_open to create the file
descriptor for each of the tasks in order to handle information of perf
events.
That is to say each task needs a unique file descriptor. In x86_64,
there may be over 1024 or 4096 tasks correspoinding to the record in
perf.data, which causes that no enough file descriptors can be used.
As a result, EMFILE error happens and stops the replay process. To solve
this problem, we adaptively increase the soft and hard maximum number of
open files with a '-f' option.
Example:
Test environment: x86_64 with 160 cores
$ cat /proc/sys/kernel/pid_max
163840
$ cat /proc/sys/fs/file-max
6815744
$ ulimit -Sn
1024
$ ulimit -Hn
4096
Before this patch:
$ perf sched replay
...
task 1549 ( :163132: 163132), nr_events: 1
task 1550 ( :163540: 163540), nr_events: 1
task 1551 ( <unknown>: 0), nr_events: 10
Error: sys_perf_event_open() syscall returned with -1 (Too many open
files)
After this patch:
$ perf sched replay
...
task 1549 ( :163132: 163132), nr_events: 1
task 1550 ( :163540: 163540), nr_events: 1
task 1551 ( <unknown>: 0), nr_events: 10
Error: sys_perf_event_open() syscall returned with -1 (Too many open
files)
Have a try with -f option
$ perf sched replay -f
...
task 1549 ( :163132: 163132), nr_events: 1
task 1550 ( :163540: 163540), nr_events: 1
task 1551 ( <unknown>: 0), nr_events: 10
------------------------------------------------------------
#1 : 54.401, ravg: 54.40, cpu: 3285.21 / 3285.21
#2 : 199.548, ravg: 68.92, cpu: 4999.65 / 3456.66
#3 : 170.483, ravg: 79.07, cpu: 1349.94 / 3245.99
#4 : 192.034, ravg: 90.37, cpu: 1322.88 / 3053.67
#5 : 182.929, ravg: 99.62, cpu: 1406.51 / 2888.96
#6 : 152.974, ravg: 104.96, cpu: 1167.54 / 2716.82
#7 : 155.579, ravg: 110.02, cpu: 2992.53 / 2744.39
#8 : 130.557, ravg: 112.08, cpu: 1126.43 / 2582.59
#9 : 138.520, ravg: 114.72, cpu: 1253.22 / 2449.65
#10 : 134.328, ravg: 116.68, cpu: 1587.95 / 2363.48
Signed-off-by: Yunlong Song <yunlong.song@huawei.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1427809596-29559-8-git-send-email-yunlong.song@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since there is sem_wait for each task in the wait_for_tasks(), e.g.
sem_wait(&task->work_done_sem).
The sem_wait can continue only when work_done_sem is greater than 0, or
it will be blocked.
For perf sched replay, one task may sem_post the work_done_sem of
another task, which causes the work_done_sem of that task processed in a
reasonable sequence, e.g. sem_post, sem_wait, sem_wait, sem_post...
This sequence simulates the sched process of the running tasks at the
time when perf sched record runs.
As a result, all the tasks are required and their threads must be
successfully created.
If any one (task A) of the tasks fails to create its thread, then
another task (task B), whose work_done_sem needs sem_post from that
failed task A, may likely block itself due to seg_wait.
And this is a dead halt, since task B's thread_func cannot continue at
all.
To solve this problem, perf sched replay should exit once any task fails
to create its thread.
Example:
Test environment: x86_64 with 160 cores
Before this patch:
$ perf sched replay
...
Error: sys_perf_event_open() syscall returned with -1 (Too many open
files)
------------------------------------------------------------ <- dead halt
After this patch:
$ perf sched replay
...
task 1551 ( <unknown>: 0), nr_events: 10
Error: sys_perf_event_open() syscall returned with -1 (Too many open
files)
$
As shown above, perf sched replay finishes the process after printing an
error message and does not block itself.
Signed-off-by: Yunlong Song <yunlong.song@huawei.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1427809596-29559-7-git-send-email-yunlong.song@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
