prompt-hacking_skill

Overview

This skill simulates advanced prompt manipulation techniques to evaluate LLM safety and resilience during red team assessments. It focuses on identifying weaknesses in instruction handling, external content ingestion, multi-turn interactions, and context-window vulnerabilities. The goal is to expose practical risks so engineers can prioritize mitigations and harden agent behavior.

How this skill works

The skill orchestrates simulated attack patterns and safe probes against an agent to surface failure modes. It exercises four broad categories—direct instruction overrides, indirect injections via external content, multi-turn social engineering sequences, and context-window flooding—while measuring success indicators and detection difficulty. Results produce actionable findings and severity classifications to guide remediation.

When to use it

• Conducting adversarial red team exercises for LLM-based services
• Validating defenses against prompt injection and system-prompt leakage
• Assessing multi-turn conversational robustness and state management
• Testing content ingestion pipelines (webpages, documents, emails) for hidden instructions
• Evaluating context-window handling and instruction persistence

Best practices

• Run tests in a controlled environment with clear safety approval and scope defined
• Limit probes to non-production models or datasets when possible to avoid accidental exposure
• Focus on detection signals and root causes rather than reproducing harmful content verbatim
• Combine automated scans with human review to interpret ambiguous outputs and false positives
• Report findings with prioritized fixes: input sanitization, provenance checks, instruction hierarchy, and response filtering

Example use cases

• Red team routine to validate new system-prompt protections before deployment
• Penetration test of a document ingestion pipeline to find hidden metadata or formatting abuses
• Multi-turn resilience audit for a customer support agent that maintains long conversations
• Benchmarking different mitigation strategies (instruction anchoring, external content sanitization) and comparing detection rates