ai_skill

Overview

This skill packages practical patterns and tools for building production AI systems, covering LLM integration, retrieval-augmented generation (RAG), embeddings, fine-tuning, and MLOps. It consolidates modern stack choices, code examples, and architecture patterns to accelerate shipping reliable, auditable AI features. Use it as a hands-on reference when designing pipelines, deploying models, or implementing evaluation and monitoring.

How this skill works

The skill inspects common AI engineering tasks and recommends focused solutions: prompt engineering with DSPy, document ingestion and retrieval with LlamaIndex, vector storage choices (Qdrant, Pinecone), and orchestration with LangGraph. It provides code snippets for RAG pipelines, MCP tool integration, LoRA/QLoRA fine-tuning recipes, and MLOps tracking examples using MLflow. The content emphasizes reproducible patterns: query rewrite → hybrid retrieval → rerank → generate → cite, and integrates evaluation with RAGAS.

When to use it

• Designing a retrieval-augmented generation (RAG) pipeline for product search or knowledge assistants
• Integrating LLMs and toolchains (OpenAI, Anthropic, Claude, custom models) into applications
• Selecting and configuring vector databases and embedding models for semantic search
• Fine-tuning or parameter-efficient tuning (LoRA/QLoRA) for domain-specific tasks
• Setting up experiment tracking, evaluation metrics, and production model serving

Best practices

• Treat prompts as code: use programmatic prompt tooling (DSPy) and optimize with data-driven bootstrapping
• Combine dense and lexical retrieval (dense + BM25) and rerank with cross-encoders for precision
• Choose embedding models by task: higher dims for general purpose, multilingual models for cross-language RAG
• Track experiments, params, and artifacts with MLflow or W&B to enable reproducibility
• Use MCP or a tooling protocol for secure, testable integration of external tools and stateful agents
• Evaluate RAG outputs with faithfulness and relevance metrics (RAGAS) before deploying

Example use cases

• Customer support assistant: ingest docs with LlamaIndex, store vectors in Qdrant, serve via LangGraph-driven agents
• Domain-adapted model: apply LoRA/QLoRA on a base model and track runs with MLflow for A/B comparisons
• Hybrid search system: combine BM25 and dense embeddings, rerank answers, and include citations in generation
• Tool-enabled agents: expose internal search and analytics via MCP tools and orchestrate flows with LangGraph
• Evaluation pipeline: run RAGAS metrics on benchmark queries to validate faithfulness and context precision