| perf-amd-ibs(1) |
| =============== |
| |
| NAME |
| ---- |
| perf-amd-ibs - Support for AMD Instruction-Based Sampling (IBS) with perf tool |
| |
| SYNOPSIS |
| -------- |
| [verse] |
| 'perf record' -e ibs_op// |
| 'perf record' -e ibs_fetch// |
| |
| DESCRIPTION |
| ----------- |
| |
| Instruction-Based Sampling (IBS) provides precise Instruction Pointer (IP) |
| profiling support on AMD platforms. IBS has two independent components: IBS |
| Op and IBS Fetch. IBS Op sampling provides information about instruction |
| execution (micro-op execution to be precise) with details like d-cache |
| hit/miss, d-TLB hit/miss, cache miss latency, load/store data source, branch |
| behavior etc. IBS Fetch sampling provides information about instruction fetch |
| with details like i-cache hit/miss, i-TLB hit/miss, fetch latency etc. IBS is |
| per-smt-thread i.e. each SMT hardware thread contains standalone IBS units. |
| |
| Both, IBS Op and IBS Fetch, are exposed as PMUs by Linux and can be exploited |
| using the Linux perf utility. The following files will be created at boot time |
| if IBS is supported by the hardware and kernel. |
| |
| /sys/bus/event_source/devices/ibs_op/ |
| /sys/bus/event_source/devices/ibs_fetch/ |
| |
| IBS Op PMU supports two events: cycles and micro ops. IBS Fetch PMU supports |
| one event: fetch ops. |
| |
| IBS PMUs do not have user/kernel filtering capability and thus it requires |
| CAP_SYS_ADMIN or CAP_PERFMON privilege. |
| |
| IBS VS. REGULAR CORE PMU |
| ------------------------ |
| |
| IBS gives samples with precise IP, i.e. the IP recorded with IBS sample has |
| no skid. Whereas the IP recorded by regular core PMU will have some skid |
| (sample was generated at IP X but perf would record it at IP X+n). Hence, |
| regular core PMU might not help for profiling with instruction level |
| precision. Further, IBS provides additional information about the sample in |
| question. On the other hand, regular core PMU has it's own advantages like |
| plethora of events, counting mode (less interference), up to 6 parallel |
| counters, event grouping support, filtering capabilities etc. |
| |
| Three regular core PMU events are internally forwarded to IBS Op PMU when |
| precise_ip attribute is set: |
| |
| -e cpu-cycles:p becomes -e ibs_op// |
| -e r076:p becomes -e ibs_op// |
| -e r0C1:p becomes -e ibs_op/cnt_ctl=1/ |
| |
| EXAMPLES |
| -------- |
| |
| IBS Op PMU |
| ~~~~~~~~~~ |
| |
| System-wide profile, cycles event, sampling period: 100000 |
| |
| # perf record -e ibs_op// -c 100000 -a |
| |
| Per-cpu profile (cpu10), cycles event, sampling period: 100000 |
| |
| # perf record -e ibs_op// -c 100000 -C 10 |
| |
| Per-cpu profile (cpu10), cycles event, sampling freq: 1000 |
| |
| # perf record -e ibs_op// -F 1000 -C 10 |
| |
| System-wide profile, uOps event, sampling period: 100000 |
| |
| # perf record -e ibs_op/cnt_ctl=1/ -c 100000 -a |
| |
| Same command, but also capture IBS register raw dump along with perf sample: |
| |
| # perf record -e ibs_op/cnt_ctl=1/ -c 100000 -a --raw-samples |
| |
| System-wide profile, uOps event, sampling period: 100000, L3MissOnly (Zen4 onward) |
| |
| # perf record -e ibs_op/cnt_ctl=1,l3missonly=1/ -c 100000 -a |
| |
| Per process(upstream v6.2 onward), uOps event, sampling period: 100000 |
| |
| # perf record -e ibs_op/cnt_ctl=1/ -c 100000 -p 1234 |
| |
| Per process(upstream v6.2 onward), uOps event, sampling period: 100000 |
| |
| # perf record -e ibs_op/cnt_ctl=1/ -c 100000 -- ls |
| |
| To analyse recorded profile in aggregate mode |
| |
| # perf report |
| /* Select a line and press 'a' to drill down at instruction level. */ |
| |
| To go over each sample |
| |
| # perf script |
| |
| Raw dump of IBS registers when profiled with --raw-samples |
| |
| # perf report -D |
| /* Look for PERF_RECORD_SAMPLE */ |
| |
| Example register raw dump: |
| |
| ibs_op_ctl: 000002c30006186a MaxCnt 100000 L3MissOnly 0 En 1 |
| Val 1 CntCtl 0=cycles CurCnt 707 |
| IbsOpRip: ffffffff8204aea7 |
| ibs_op_data: 0000010002550001 CompToRetCtr 1 TagToRetCtr 597 |
| BrnRet 0 RipInvalid 0 BrnFuse 0 Microcode 1 |
| ibs_op_data2: 0000000000000013 RmtNode 1 DataSrc 3=DRAM |
| ibs_op_data3: 0000000031960092 LdOp 0 StOp 1 DcL1TlbMiss 0 |
| DcL2TlbMiss 0 DcL1TlbHit2M 1 DcL1TlbHit1G 0 DcL2TlbHit2M 0 |
| DcMiss 1 DcMisAcc 0 DcWcMemAcc 0 DcUcMemAcc 0 DcLockedOp 0 |
| DcMissNoMabAlloc 0 DcLinAddrValid 1 DcPhyAddrValid 1 |
| DcL2TlbHit1G 0 L2Miss 1 SwPf 0 OpMemWidth 32 bytes |
| OpDcMissOpenMemReqs 12 DcMissLat 0 TlbRefillLat 0 |
| IbsDCLinAd: ff110008a5398920 |
| IbsDCPhysAd: 00000008a5398920 |
| |
| IBS applied in a real world usecase |
| |
| ~90% regression was observed in tbench with specific scheduler hint |
| which was counter intuitive. IBS profile of good and bad run captured |
| using perf helped in identifying exact cause of the problem: |
| |
| https://lore.kernel.org/r/20220921063638.2489-1-kprateek.nayak@amd.com |
| |
| IBS Fetch PMU |
| ~~~~~~~~~~~~~ |
| |
| Similar commands can be used with Fetch PMU as well. |
| |
| System-wide profile, fetch ops event, sampling period: 100000 |
| |
| # perf record -e ibs_fetch// -c 100000 -a |
| |
| System-wide profile, fetch ops event, sampling period: 100000, Random enable |
| |
| # perf record -e ibs_fetch/rand_en=1/ -c 100000 -a |
| |
| Random enable adds small degree of variability to sample period. This |
| helps in cases like long running loops where PMU is tagging the same |
| instruction over and over because of fixed sample period. |
| |
| etc. |
| |
| PERF MEM AND PERF C2C |
| --------------------- |
| |
| perf mem is a memory access profiler tool and perf c2c is a shared data |
| cacheline analyser tool. Both of them internally uses IBS Op PMU on AMD. |
| Below is a simple example of the perf mem tool. |
| |
| # perf mem record -c 100000 -- make |
| # perf mem report |
| |
| A normal perf mem report output will provide detailed memory access profile. |
| However, it can also be aggregated based on output fields. For example: |
| |
| # perf mem report -F mem,sample,snoop |
| Samples: 3M of event 'ibs_op//', Event count (approx.): 23524876 |
| Memory access Samples Snoop |
| N/A 1903343 N/A |
| L1 hit 1056754 N/A |
| L2 hit 75231 N/A |
| L3 hit 9496 HitM |
| L3 hit 2270 N/A |
| RAM hit 8710 N/A |
| Remote node, same socket RAM hit 3241 N/A |
| Remote core, same node Any cache hit 1572 HitM |
| Remote core, same node Any cache hit 514 N/A |
| Remote node, same socket Any cache hit 1216 HitM |
| Remote node, same socket Any cache hit 350 N/A |
| Uncached hit 18 N/A |
| |
| Please refer to their man page for more detail. |
| |
| SEE ALSO |
| -------- |
| |
| linkperf:perf-record[1], linkperf:perf-script[1], linkperf:perf-report[1], |
| linkperf:perf-mem[1], linkperf:perf-c2c[1] |
