dspy-output-refinement-constraints_skill

Overview

This skill refines DSPy module outputs by applying iterative improvement (dspy.Refine) and best-of-N selection (dspy.BestOfN) with custom reward/constraint functions. It helps enforce format, length, and content rules so predictions meet validation criteria before being accepted. Use it to replace deprecated Assert/Suggest patterns and to raise output quality with controlled multi-attempt generation.

How this skill works

You wrap an existing dspy.Module with Refine or BestOfN and supply a reward_fn that scores each prediction from 0.0 to 1.0. Refine runs iteratively and tries to improve outputs until the threshold is met; BestOfN generates N independent attempts and selects the highest-scoring result. Reward functions validate structure, content, and business rules and must return a numeric score without raising exceptions.

When to use it

• You need strict format validation (JSON, CSV, specific schema).
• Outputs must meet length or token limits (min/max words or chars).
• Content must include required terms or avoid prohibited items.
• You want higher-quality results through multiple attempts.
• You need a drop-in replacement for deprecated Assert/Suggest patterns.

Best practices

• Score gradually using a 0.0–1.0 range and weight multiple constraints.
• Keep reward functions robust: catch exceptions and return 0.0 on failure.
• Limit N to reasonable values (Refine: 3–5, BestOfN: 5–10) to control cost.
• Log failed constraints to identify recurring model weaknesses.
• Start with coarse checks, then add finer-grained constraints iteratively.

Example use cases

• Enforce JSON schema for extracted entities (name, age, email) with BestOfN selection.
• Refine document summaries to required length and include mandatory keywords.
• Validate and score e-commerce extraction (product, price, quantity) before ingest.
• Apply multi-constraint scoring (length, capitalization, required terms) for QA answers.
• Replace Assert/Suggest checks by converting rules into reward functions for Refine.