symmetry-validation-suite_skill

Overview

This skill provides a practical, data-driven suite to empirically test whether your hypothesized symmetries hold in data or model behavior. It gives stepwise test protocols, sampling strategies, statistical criteria, and a report template so you can decide whether to enforce invariance/equivariance in architecture. Use it to avoid mis-specified symmetry assumptions that hurt performance.

How this skill works

You enumerate candidate symmetry hypotheses, generate transformed test sets, and run invariance or equivariance checks that compare model outputs before and after transforms. The suite verifies group structure (closure, identity, inverse, associativity) and analyzes whether transformed data remains in-distribution. Results are aggregated into quantitative metrics and a clear pass/fail recommendation with guidance on hard constraints, soft regularization, or augmentation.

When to use it

• Before choosing an equivariant architecture to confirm the symmetry is present
• When debugging model failures suspected to stem from incorrect symmetry assumptions
• To validate invariance or equivariance claims in papers or benchmarks
• When designing data augmentation schemes consistent with true symmetries
• To decide between enforcing hard constraints versus soft regularization

Best practices

• List each symmetry with expected invariance/equivariance and a confidence level before testing
• Use at least ~100 samples and many random transforms per sample for statistical reliability
• Report mean, std, and high-percentile errors (95th/99th) and set thresholds by numerical precision
• Check that transformed inputs remain in-distribution to catch approximate symmetries
• Test group axioms numerically (closure, identity, inverse, associativity) within tolerances

Example use cases

• Validate rotational equivariance for a vision model before switching to an SO(2)/SO(3) equivariant layer
• Check permutation symmetry in a graph model to decide whether to enforce S_n constraints
• Measure whether translations are approximate in a dataset to choose augmentation ranges
• Diagnose whether a suspected reflection symmetry actually breaks at image boundaries
• Compare hard equivariance vs. augmentation by running statistical tests and decision matrix