testing-methodologies_skill

Overview

This skill provides structured approaches for AI security testing, covering threat modeling, vulnerability assessment, exploitation proofs, and reporting. It codifies lifecycle phases and mappings to OWASP LLM Top 10 and MITRE ATLAS to support repeatable, measurable assessments.

How this skill works

The methodology follows a lifecycle: scope definition, threat modeling (STRIDE + attack trees), test planning, execution (recon, vulnerability assessment, exploitation), analysis, and reporting with remediation tracking. It includes techniques for fingerprinting models, probing input constraints, building attack trees, developing proof-of-concept exploits, and generating prioritized security reports.

When to use it

• Before deploying an AI/ML model to production to validate defenses.
• When performing periodic security assessments or compliance audits.
• During incident response to map attack paths and identify root causes.
• To prioritize fixes by risk using coverage and effectiveness metrics.
• When integrating security testing into CI/CD pipelines for models.

Best practices

• Start with a clear scope and threat model that maps to STRIDE and MITRE ATLAS.
• Use reconnaissance to fingerprint model behavior, limits, and error patterns before active testing.
• Prioritize tests by OWASP LLM Top 10 and attack path risk (likelihood × impact).
• Execute exploitation proofs of concept with safety controls and detailed evidence logging.
• Produce actionable reports with remediation roadmaps and track fixes until closure.

Example use cases

• Red team engagement to identify prompt injection and jailbreak paths against a chatbot.
• Penetration test focused on training data extraction and system prompt disclosure risks.
• Assessing robustness to adversarial and out-of-distribution inputs for model hardening.
• Measuring coverage of OWASP and MITRE techniques and reporting remediation velocity.
• Integration of automated reconnaissance scans into model deployment pipelines.