property-based_skill

Overview

This skill teaches property-based testing in the style of John Hughes (QuickCheck). It focuses on writing general properties instead of example tests, using randomized generators to explore inputs, and shrinking failures to minimal counterexamples for faster debugging. It is implemented with Python-friendly patterns and practical guidance for integration into CI.

How this skill works

You express invariants or laws (roundtrip, idempotence, algebraic, equivalence) as properties that must hold for all generated inputs. A generator produces thousands of random cases, runs the property against each, and when a failure appears the framework repeatedly applies shrink strategies to find a minimal failing example. Tests are repeatable via seeds and tuned by test count and generator size parameters.

When to use it

• Testing pure functions with clear invariants (sort, normalize, encode/decode).
• Validating algorithms and data structures against algebraic laws and invariants.
• Comparing new implementations to known-correct or naive oracles.
• Fuzzing parsers, serializers, and deserializers for roundtrip safety.
• CI regression checks with reproducible seeds and many iterations.

Best practices

• Prefer properties that state true invariants over enumerating examples.
• Provide custom generators that model real input distributions and support shrinking.
• Always implement shrinking: minimal counterexamples speed triage and fixes.
• Run many iterations (1000+ when practical) and set seeds for CI reproducibility.
• Split complex behavior into multiple orthogonal properties for clearer failures.

Example use cases

• Assert serialize(deserialize(x)) == x for data structures and blob formats.
• Verify sort preserves length and produces an ordered permutation of input.
• Check idempotence: normalize(normalize(s)) == normalize(s) for input sanitizers.
• Compare optimized algorithm output to a simple/reference implementation across random inputs.
• Fuzz parser then shrink the smallest input that causes a crash or mis-parse.