guidance_skill

Overview

This skill controls LLM output using regex, grammars, and Pythonic workflows to guarantee valid JSON, XML, or code generation. It provides token-level constraints, token healing, and reusable guidance functions so agents produce structured, predictable outputs. Designed for both API and local models, it reduces latency and invalid outputs for production pipelines.

How this skill works

The skill wraps model calls with context managers and generation primitives (gen, select) that convert regex and grammars into token-level constraints. During inference, invalid tokens are filtered and token healing adjusts token boundaries to avoid spacing issues. You can define CFG grammars, selection lists, and stateful @guidance functions to build multi-step, tool-enabled workflows that return guaranteed formats.

When to use it

• When you must guarantee valid JSON, XML, or code from an LLM
• When enforcing strict formats (dates, emails, IDs) via regex or grammars
• When building multi-step or tool-using agents with predictable state
• When reducing retries and latency for constrained outputs
• When running on local models (Transformers, llama.cpp) or API models

Best practices

• Prefer regex constraints for format validation rather than free generation to avoid invalid outputs
• Use select() for fixed-choice fields to eliminate typos and ambiguous labels
• Leverage stop sequences for single-line captures and max_tokens to bound outputs
• Create reusable @guidance functions for patterns like person generation or extraction
• Balance constraint strictness: too-strict regex can fail or slow generation; prefer reasonable patterns

Example use cases

• Generate valid JSON user profiles guaranteed to parse downstream
• Extract structured entities (person, organization, ISO dates) from messy text
• Implement ReAct-style agents that call tools and maintain state across rounds
• Constrain code output to specific grammar fragments or programming language syntax
• Perform reliable classification with fixed-choice outputs and numeric confidence