model-equivariance-auditor_skill

Overview

This skill verifies that a model claiming equivariance actually respects its intended symmetries. It provides a systematic audit: end-to-end numerical tests, layer-wise checks, gradient tests, failure-mode diagnosis, and a structured audit report template. Use it to catch implementation bugs early and generate actionable fixes. The output distinguishes exact, approximate, and broken equivariance with clear thresholds.

How this skill works

Run randomized transformation tests to compare f(T(x)) against T'(f(x)) for equivariant outputs or f(T(x)) against f(x) for invariance. If end-to-end tests fail, isolate the faulty component with layer-wise tests and check gradient equivariance to ensure training preserves symmetry. Record error statistics, classify failures by common modes, and produce a concise audit report with recommended fixes and severity levels.

When to use it

• You have implemented an equivariant model and need to confirm correctness before training.
• Model training underperforms and you suspect symmetry implementation bugs.
• You want to audit whether layers or normalization break equivariance.
• You need to verify gradients transform correctly for stable optimization.
• Preparing a reproducible audit report for a model release or review.

Best practices

• Specify the symmetry group, input/output representations, and transform functions before testing.
• Use multiple random transformations and report mean and max absolute/relative errors.
• Test layers individually: freeze others to isolate the breaking layer quickly.
• Include gradient equivariance checks; forward pass can be correct while backward pass is not.
• Use higher precision (float64) for diagnosis of tiny discrepancies and set tolerance tiers.

Example use cases

• Run an end-to-end equivariance test on a rotation-equivariant CNN to validate kernel parameterization.
• Diagnose a model that diverges during training by running gradient equivariance checks.
• Pinpoint a nonlinearity or batchnorm as the problem using layer-wise tests and then swap to GroupNorm.
• Produce a formal audit report summarizing pass/fail status, error magnitudes, and recommended fixes.
• Validate custom attention layers for permutation equivariance before large-scale training.