kleppmann_skill

Overview

This skill teaches designing data-intensive distributed systems in the style of Martin Kleppmann, focusing on clear trade-offs, reliability, and deep understanding of data semantics. It guides architects and engineers through choices around consistency, failure modes, data modeling, and operational guarantees. Use it to evaluate architectures, pick patterns, and produce practical, robust designs for databases, streams, and pipelines.

How this skill works

The skill inspects system requirements and maps them to Kleppmann-style design patterns: consistency models, event sourcing, CDC, idempotency, and log-based messaging. It asks targeted questions about access patterns, failure scenarios, schema evolution, and operational constraints, then recommends concrete code patterns and operational controls. Outputs include architecture trade-offs, recommended primitives, and example code snippets to illustrate implementation choices.

When to use it

• Selecting a storage or messaging system where trade-offs must be explicit (CAP, PACELC).
• Designing data pipelines, change-data-capture flows, or stream processing topologies.
• Defining consistency and isolation guarantees for a new service or API.
• Migrating schemas or planning long-lived data evolution strategies.
• Implementing retries, idempotency, and failure-handling for distributed operations.

Best practices

• Make trade-offs explicit: document what you gain and what you accept to lose.
• Design for partial failure: assume components fail and plan retries, timeouts, and compensations.
• Prefer idempotent APIs and idempotency keys for safe retries.
• Use append-only or event-sourced stores for auditability and easy replay.
• Choose CDC or log-based replication instead of fragile dual-writes.
• Define and test consistency guarantees; do not treat 'eventual' as a catch-all.

Example use cases

• Designing a payment service that must avoid duplicate charges using idempotency keys and exactly-once-like workflows.
• Building a search index updated from a primary database with CDC to ensure ordered, reliable updates.
• Choosing between a strongly consistent primary-replica DB and a partition-tolerant eventual system for a geo-distributed app.
• Implementing stream processing with windowed aggregations and careful checkpointing for recoverability.
• Planning a schema migration strategy that supports backward compatibility and event replay.