dean_skill

This skill helps you design distributed systems at planet scale with Jeff Dean principles, optimizing tail latency and scalability.

Python

3

GitHub Stars

1

Bundled Files

2 months ago

Catalog Refreshed

4 months ago

First Indexed

Readme & install

Copy the install command, review bundled files from the catalogue, and read any extended description pulled from the listing source.

Installation

Preview and clipboard use veilstrat where the catalogue uses aiagentskills.

npx veilstrat add skill copyleftdev/sk1llz --skill dean

SKILL.md11.1 KB

Overview

This skill captures Jeff Dean–style large-scale system design practices for architects and engineers building systems that serve millions or billions. It focuses on practical, measurable techniques—designing for 10x growth, minimizing tail latency, and instrumenting systems from day one. Use it for architecture tradeoffs, performance tuning, and operational resilience planning.

How this skill works

The skill inspects system requirements and workload numbers, then applies proven Dean patterns: LSM/BigTable-style storage, consistent hashing for sharding, MapReduce-style parallelism, and hedged/backup requests to reduce tail latency. It produces concrete recommendations, code patterns, and instrumentation plans keyed to latency, throughput, and failure modes. Outputs emphasize measurable targets (p50/p90/p99), failure scenarios, and incremental rollout strategies.

When to use it

Designing storage or indexing for high write throughput
Architecting services that must meet strict tail-latency SLAs
Sharding and rebalancing a large distributed cache or database
Planning instrumentation and operational metrics for new systems
Choosing algorithms or topologies for planet-scale workloads

Best practices

Start by quantifying numbers: latencies, bandwidth, IOPS, and growth projections
Design for 10x scale and expect to iterate before 100x
Optimize the common case but measure and mitigate tail latencies (p99/p999)
Instrument everything from day one and track percentiles, errors, and resource saturation
Prefer simple, composable abstractions and idempotent APIs; favor batching and parallel fanout with hedged requests

Example use cases

Designing a globally sharded key-value store using consistent hashing and virtual nodes
Building a write-optimized time-series ingest pipeline with LSM/flush/compaction
Reducing user-facing tail latency by adding hedged requests and backup replicas
Specifying monitoring and alerting: p50/p90/p99 latencies, queue depths, and GC pauses
Creating a MapReduce-style batch job framework for large-scale analytics

FAQ

Pick LSM for write-heavy workloads with high ingest and compaction budget; pick B-tree when reads dominate and update-in-place with lower write amplification is required.

When should I use hedged requests versus more replicas?

Use hedged requests to reduce tail latency without increasing steady-state load; add replicas when capacity or fault tolerance requires more copies rather than short-term latency mitigation.