performance-benchmark-specialist_skill

Overview

This skill provides hands-on performance benchmarking expertise for shell-based tools, focused on rigorous measurement, statistical analysis, workspace generation, and clear reporting. It codifies a repeatable benchmark structure, helper utilities, and performance targets so you can validate optimizations and detect regressions reliably.

How this skill works

The skill supplies a standard benchmark script pattern and a helper library that handles high-precision timing, warmups, repeated runs, statistical calculations (min/max/mean/median/stddev), comparisons, assertions against targets, CSV result storage, and formatted reporting. It creates test workspaces at configurable sizes, runs baseline and optimized phases, computes speedup ratios, and saves results for historical analysis.

When to use it

• Designing repeatable benchmarks for shell scripts or CLI tools
• Validating performance targets (e.g., <100ms cached navigation)
• Comparing baseline versus optimized implementations
• Generating reproducible test workspaces at different scales
• Creating CSV-backed benchmark records for tracking regressions

Best practices

• Define performance targets before implementation and test design
• Always perform warmup runs to reduce noise and use multiple iterations
• Use statistical summaries (median and stddev) rather than single runs
• Test at realistic scale with representative workspaces
• Assert targets automatically and persist results for trend analysis

Example use cases

• Measure a shell-based navigation command to ensure cached lookup <100ms
• Compare an optimization patch by running baseline and optimized phases and reporting speedup
• Build medium and large test workspaces to evaluate scaling behavior
• Automate nightly benchmarks that append CSV results for historical tracking
• Validate cache hit rate and cache build time against defined targets