deadlock-prevention_skill

Overview

This skill helps teams prevent and diagnose deadlocks when acquiring multiple locks or resources. It codifies practical strategies—consistent ordering, timeouts, try-lock with backoff, hierarchical locks, and distributed leases—to avoid circular waits that freeze systems without crashing. Use it to harden code paths that acquire more than one lock or coordinate across services.

How this skill works

The skill inspects locking patterns and recommends fixes: enforce a total order for acquiring locks, add safe timeouts and try-lock retries, or replace fine-grained locks with a single higher-level lock when appropriate. It also offers runtime deadlock detection by tracking wait-for graphs to surface cycles and trigger alerts or aborts before permanent hang. Examples include SQL FOR UPDATE ordering, Redis distributed leases, and in-process hierarchical locks.

When to use it

• When code acquires multiple locks or resources in the same operation
• When nested transactions span tables, services, or regions
• When services coordinate over shared resources (DB rows, keys, files)
• When operations hang under load without errors or crashes
• When locks are held across async/await boundaries

Best practices

• Always enforce a consistent global lock order (alphabetical, ID-based, or hierarchy) across code and services
• Prefer releasing locks in finally blocks and avoid holding locks while awaiting long operations
• Use lock timeouts as a safety net, not the primary fix—timeouts expose issues but don’t eliminate circular waits
• Implement try-lock with exponential backoff to avoid hold-and-wait scenarios under contention
• Document lock ordering and require cross-team coordination for shared resources

Example use cases

• Bank transfers acquiring two account locks — sort IDs and lock lower-first to prevent cycles
• Distributed cache updates — use a lease-based Redis lock with compare-and-release to avoid stale ownership
• Multi-table SQL updates — SELECT ... ORDER BY id FOR UPDATE to ensure consistent DB lock ordering
• Nested resource operations — apply a lock hierarchy (system → region → account) to enforce parent-before-child acquisition
• High-contention workflows — use try-lock with backoff and limited retries to reduce blocking and fallback to coarser locks