root-cause-tracing_skill

Overview

This skill provides a structured root-cause tracing toolkit using 5 Whys, fault-tree analysis, and dependency tracing to find the true source of production problems. It guides you from symptom recording through hypothesis generation, evidence collection, and verification to a documented root-cause report. The outputs are practical, reproducible, and formatted for incident reviews and remediation planning.

How this skill works

The skill inspects symptoms, logs, stack traces, and call graphs to build hypotheses. It applies the 5 Whys to dig deeper, constructs fault trees to enumerate possible causes top-down, and traces dependencies upstream from error points using stack traces, call graphs, and timeline correlation. Finally it verifies hypotheses with targeted tests, documents evidence, and recommends immediate fixes and long-term preventive actions.

When to use it

• When incidents recur and the visible fix doesn’t prevent repeats
• When logs or errors point to multiple potential causes
• During post-incident reviews to produce a clear root-cause report
• When you need to prioritize fixes by likelihood and impact
• Before rolling out significant infrastructure or schema changes

Best practices

• Record symptoms precisely: timestamps, error messages, and triggering conditions
• Start with 5 Whys to generate concise hypotheses, then expand with a fault tree for coverage
• Use timeline correlation across app, system, network, and DB logs to link events
• Focus on the first application frame in stack traces—trace callers upstream until the root is reached
• Document evidence and verification steps in the prescribed report format for handoff

Example use cases

• API timeouts where database queries or indexes are suspected
• Login failures that could stem from authentication, network, or client issues
• Intermittent transaction failures in blockchain integrations (nonce, gas, balance)
• Regression detection using git bisect combined with failing test traces
• Incident reports that require clear immediate mitigation and long-term prevention steps