prompt-guard_skill

Overview

This skill provides a lightweight, production-ready prompt injection and jailbreak detector based on an 86M-parameter classifier. It filters user prompts and third-party data for LLM applications with very high true positive rates and low false positives, while remaining fast enough for real-time inference. Deployable via HuggingFace or used for batch RAG filtering, it supports eight languages and both CPU and GPU environments.

How this skill works

The model classifies input into BENIGN, INJECTION, or JAILBREAK labels and returns probabilities for each class. You can compute a direct jailbreak score or sum injection+jailbreak probabilities for third-party data. It accepts tokenized inputs (truncated to 512 tokens by default) and supports single-call, batch, and sliding-window scoring for long texts.

When to use it

• Pre-filter user prompts before sending them to an LLM to block jailbreak attempts
• Scan API responses, scraped content, or retrieved RAG documents for embedded instructions
• Run high-throughput batch filtering of retrieved documents during RAG pipelines
• Deploy as a lightweight guard in low-budget or latency-sensitive environments
• Add a language-aware safety layer for multilingual LLM apps (8 supported languages)

Best practices

• Use a lower threshold (e.g., 0.3) for third-party data and a stricter threshold (e.g., 0.5) for user inputs
• Apply a sliding-window or chunked scoring approach for texts longer than 512 tokens to avoid missed injections
• Combine Prompt Guard with content moderation and output-validation tools for defense-in-depth
• Raise thresholds for trusted users or known internal data to reduce false positives on legitimate security research
• Batch tokenize and run inference in groups to maximize throughput when filtering large document sets

Example use cases

• Block user messages like 'Ignore previous instructions' before they reach the LLM
• Filter RAG-sourced documents and discard any with embedded commands or hidden system instructions
• Scan API responses from third-party services and sanitize or drop suspicious payloads
• Run nightly batch scans of knowledge bases to remove context-poisoning content
• Integrate as Layer 1 in a multi-layer safety pipeline alongside moderation and policy validators