axolotl_skill

Overview

This skill provides expert guidance for fine-tuning large language models using Axolotl. It covers YAML configuration patterns, model and trainer APIs, LoRA/QLoRA workflows, advanced optimization methods (DPO, KTO, ORPO, GRPO), and multimodal support across 100+ models. The content is practical, example-driven, and focused on reproducible training and inference setups.

How this skill works

The skill inspects Axolotl configuration patterns, training utilities, and code-level APIs to surface recommended settings, common pitfalls, and runnable examples. It maps YAML options to runtime behavior (FSDP, context parallelism, compression) and explains how to integrate LoRA/QLoRA and reward-style optimizers into Axolotl trainers. It also provides troubleshooting steps for performance bottlenecks, data handling, and multi-GPU setups.

When to use it

• When preparing YAML configs for training or distributed setups with Axolotl
• When implementing LoRA/QLoRA or integrating quantized training pipelines
• When using DPO/KTO/ORPO/GRPO-style optimization for instruction tuning or reward learning
• When debugging multi-GPU scaling, FSDP, or context-parallel issues
• When adapting Axolotl for multimodal models or custom integrations

Best practices

• Validate inter-node bandwidth using NCCL tests before large runs to identify transfer bottlenecks
• Make context_parallel_size a divisor of total GPUs to ensure predictable global batch sizes
• Use FSDP settings (offload, state_dict type, auto-wrap) in YAML for memory-efficient training on many layers
• Enable save_compressed:true to reduce disk usage and keep compatibility with vLLM and post-quantization tools
• Support both single-example and batched inputs in data pipelines; clip or drop overly long sequences before batching

Example use cases

• Create a YAML that enables FSDP with auto_wrap for a Llama-based decoder and run a 100B LoRA fine-tune across 8+ GPUs
• Swap training to QLoRA by adjusting config and enabling compressed saves for downstream fast inference with vLLM
• Implement DPO reward optimization to align model responses using AxolotlTrainer hooks and custom evaluators
• Integrate a custom dataset format by following dataset-format patterns and handling both single and batched input_ids
• Troubleshoot degraded throughput by running NCCL all-reduce perf tests and tuning context_parallel_size