model-inversion_skill

Overview

This skill performs model inversion and related privacy attacks to test whether an AI model exposes training data or sensitive attributes. It provides modular attacks like membership inference, training-data extraction, attribute inference, and gradient-based reconstruction to evaluate privacy risk. Use it as part of red teaming or privacy audits to quantify and reproduce leakage scenarios.

How this skill works

The skill runs targeted probes against models and APIs to detect memorization and extractable secrets. Membership inference inspects confidence distributions or trains shadow models to decide if a sample was in the training set. Extraction uses crafted prompts to elicit memorized sequences. Attribute inference applies classifiers on embeddings to infer sensitive attributes. Gradient reconstruction attempts to recover inputs from shared gradients in federated setups.

When to use it

• During pre-deployment privacy assessments for LLMs and classifiers
• When auditing models for regulatory compliance (GDPR/PII risk)
• As part of adversarial red team exercises to prioritize mitigations
• To evaluate federated learning setups for gradient leakage
• When validating defense effectiveness (DP, output perturbation, regularization)

Best practices

• Run attacks in controlled environments with authorization and clear legal scope
• Use shadow models and diverse prompts to improve membership and extraction accuracy
• Measure privacy metrics (membership advantage, extraction rate, attribute accuracy)
• Combine defenses (differential privacy + deduplication) and measure utility impact
• Log and store only derived metrics, not extracted PII, to reduce exposure risk

Example use cases

• Estimate membership inference risk for a production classification model before public release
• Probe an LLM for memorized API keys or credit card-like sequences using focused prompts
• Test a federated learning pipeline for input reconstruction from shared gradients
• Assess whether embeddings permit gender or age inference from anonymous records
• Validate the effectiveness of differential privacy and output perturbation defenses