devops-troubleshooter_skill

Overview

This skill is a DevOps troubleshooter focused on rapid production debugging, incident response, and deployment recovery. It combines log and metrics analysis, container and network debugging, and guided fixes that minimize customer impact. Use it proactively during outages or to harden systems against recurrence.

How this skill works

The skill starts by gathering facts: logs, metrics, traces, and recent deployment metadata. It forms and tests hypotheses with targeted kubectl, network, and logging queries, producing step-by-step commands and temporary mitigations. After stabilizing the service, it delivers a root cause analysis, monitoring queries, and a runbook for long-term remediation.

When to use it

• Live production outages where quick containment is required
• Deployment failures or repeated rollbacks after release
• Unexpected performance degradation or memory leaks
• Intermittent network/DNS problems affecting service reachability
• Proactive validation of monitoring and alerting gaps

Best practices

• Gather structured evidence first: timestamps, request IDs, and relevant log slices
• Test hypotheses in a controlled manner; prefer read-only inspections before changes
• Apply quick, reversible mitigations (scaled replicas, traffic splitting, feature flags)
• Document every step for the postmortem and add monitoring for detection
• Keep runbooks concise with exact commands and verification steps

Example use cases

• Analyze correlated ELK/Datadog logs to identify the failing microservice and the error cascade
• Provide kubectl and container-level commands to inspect OOMs, attach to processes, and collect core dumps
• Diagnose DNS or service-discovery failures and propose emergency DNS rollbacks or cache clears
• Execute a safe deployment rollback or hotfix with commands and traffic validation steps
• Create monitoring queries and alerts to detect the same failure pattern proactively