senior-prompt-engineer_skill

Overview

This skill delivers world-class prompt engineering and LLM system design guidance for production AI products. It combines advanced prompt patterns, few-shot and chain-of-thought techniques, RAG optimization, agent orchestration, and evaluation frameworks. Use it to optimize LLM performance, design agentic systems, and scale inference in real-world environments.

How this skill works

The skill inspects prompts, model interactions, retrieval pipelines, and system architecture to propose concrete optimizations and structured output templates. It analyzes few-shot examples, chain-of-thought strategies, and RAG configurations, then recommends changes to prompts, data flows, and deployment settings. It also provides scripts and commands to run prompt optimization, RAG evaluation, and agent orchestration workflows in a production pipeline. Results include measurable performance targets, monitoring recommendations, and deployment-safe patterns.

When to use it

• Designing or refactoring prompts for Claude, GPT-4, or similar LLMs
• Building retrieval-augmented generation (RAG) pipelines or improving retrieval quality
• Designing agentic systems or orchestrating multiple LLMs and tools
• Optimizing latency, throughput, and cost for production inference
• Establishing evaluation frameworks and automated testing for LLM behavior

Best practices

• Use few-shot examples and explicit output schemas to enforce structured outputs
• Instrument models and pipelines with monitoring (latency, error rates, drift) before rollout
• Adopt TDD and CI for prompt logic and evaluation scripts to prevent regressions
• Isolate sensitive data, apply PII anonymization, and enforce encryption in transit and at rest
• Start with canary or feature-flagged releases and ramp traffic after validating metrics

Example use cases

• Optimize customer-support completion prompts to reduce hallucinations and lower response latency
• Design an agent orchestrator that composes retrieval, tool calls, and multi-step reasoning for automation tasks
• Implement RAG with vector stores and fine-tuned retriever for domain-specific QA
• Set up evaluation pipelines to measure P50/P95 latency, throughput, and answer quality during CI runs
• Mentor engineering teams on prompt patterns, production architecture, and model governance