site-reliability-engineer_skill

Overview

This skill provides end-to-end site reliability engineering for production systems, covering monitoring, observability, SLO/SLI management, and incident response. It delivers concrete artifacts: SLI/SLO definitions, monitoring stack configs (Prometheus, Grafana, Datadog, ELK), alert rules, dashboards, runbooks, post-mortems, health checks, and error budget tracking. Use it to operationalize reliability goals and maintain production health.

How this skill works

I inspect non-functional requirements, define SLIs and SLOs, and map them to monitoring metrics and dashboards. I generate monitoring stack templates, alerting rules, and notification/escalation channels, plus readiness/liveness probes for services. For incidents I produce runbooks, automated triage steps, mitigation playbooks, and post-mortem templates with action items and error-budget reporting.

When to use it

• You need production monitoring and dashboards after deployment
• You want to define or revise SLOs/SLIs and error budgets
• You must create or validate alert rules and notification flows
• You need incident runbooks, on-call procedures, or post-mortem templates
• You want to instrument logs, metrics, and traces for observability

Best practices

• Alert on user-impacting symptoms, not low-level causes
• Instrument services early: logs, metrics (RED), and traces
• Tie alerts to SLOs and monitor error budget consumption
• Keep runbooks concise, actionable, and updated after every incident
• Review SLOs and alert noise quarterly to reduce false positives

Example use cases

• Create Prometheus scrape configs, Grafana dashboards, and alert rules for an API service
• Define availability and latency SLOs, calculate error budgets, and set notification policies
• Add readiness and liveness endpoints and integrate them with Kubernetes probes
• Produce an incident response runbook for SEV-1 scenarios with rollback steps
• Generate post-mortem template and follow-up action items after a production outage