andrewimm/spaa
Overview
This skill helps you read and interpret SPAA (.spaa) files to diagnose application performance and memory problems. It explains the format, points out the important header fields and record types, and shows how to find hotspots, exclusive time, and memory allocation patterns. Use it when you need practical guidance to turn raw SPAA traces into actionable insights.
How this skill works
The skill walks you through inspecting the header to determine the profiler, frame order, and primary metric. It explains the record types (header, dso, frame, thread, stack, sample) and shows how to resolve frame IDs to function names. It provides jq/grep patterns to extract heavy stacks, exclusive-time hotspots, memory allocation sites, and temporal sample distributions.
When to use it
- When you have a .spaa profiling trace and need to identify CPU hotspots or hot call stacks.
- When you suspect memory leaks or high allocation churn and want to find top allocation sites.
- When you need to translate frame IDs into readable function names for reporting.
- When you want to filter traces by thread, event type, kernel vs userspace, or time.
- When you need quick, repeatable jq commands to extract prioritized lists from large NDJSON files.
Best practices
- Always read the header first to learn the source_tool, frame_order, and the primary metric to weight results correctly.
- Prioritize exclusive time to find where CPU time is actually consumed rather than where it appears on call paths.
- Build a frame lookup table (index frames by id) before resolving stacks to speed repeated queries.
- Use the primary metric (period, samples, alloc_bytes, live_bytes) appropriate to the profiler and analysis goal.
- Filter by context (tid, event, stack_type) to narrow noise and isolate relevant behavior.
Example use cases
- Find the top 10 CPU-heavy stacks in a perf-generated trace using the header's primary metric.
- Identify functions with the highest exclusive time and map their frame IDs to source lines.
- Detect possible memory leaks by listing stacks with the largest live_bytes values.
- Compare kernel vs userspace time by selecting kernel stack records or kernel-kind frames.
- Plot sample counts over time when raw sample records are present to spot intermittent spikes.
FAQ
Check the header.events[].sampling.primary_metric; use that metric (e.g., period, samples, alloc_bytes, live_bytes) to weight and sort stacks.
What if frame_order is leaf_to_root or root_to_leaf?
Respect frame_order when reconstructing call stacks: leaf_to_root lists callees first, root_to_leaf lists callers first; reverse as needed for readable call traces.
