Unsloth

You are setting up and using an MCP server that runs Unsloth, a library that speeds up fine-tuning of large language models while dramatically reducing VRAM usage. This server exposes convenient endpoints to load models, fine-tune with memory-efficient techniques, generate text, and export models for deployment, all through your MCP client workflows.

How to use

Connect your MCP client to the Unsloth MCP server to perform common tasks such as loading models, fine-tuning with 4-bit quantization, generating text, and exporting trained models. You can: load a pretrained model with Unsloth optimizations for faster inference and lower memory usage, fine-tune models with LoRA/QLoRA techniques, generate text from a fine-tuned model, and export to formats like GGUF or Hugging Face. The server is designed to work with models such as Llama, Mistral, Phi, Gemma, and other compatible variants, with options to customize sequence lengths, batch sizes, and learning rates as needed. Ensure you have a running MCP client that can call the listed tools and manage assets across your workspace.

How to install

pip install unsloth

cd unsloth-server
npm install
npm run build

{
  "mcpServers": {
    "unsloth-server": {
      "command": "node",
      "args": ["/path/to/unsloth-server/build/index.js"],
      "env": {
        "HUGGINGFACE_TOKEN": "your_token_here"
      },
      "disabled": false,
      "autoApprove": []
    }
  }
}

Advanced usage and notes

Advanced usage includes loading models with 4-bit quantization, enabling gradient checkpointing to save memory, and adjusting training parameters such as the LoRA rank, learning rate, batch size, and gradient accumulation steps to fit your hardware profile. You can also provide custom datasets hosted on Hugging Face or a local path for fine-tuning. If you work with large models on limited hardware, consider reducing batch size, increasing gradient accumulation steps, enabling 4-bit quantization, and shortening the training sequence length when possible to fit memory constraints.

Troubleshooting

If you encounter CUDA Out of Memory errors, try reducing batch size, using 4-bit quantization, enabling gradient checkpointing, or selecting a smaller model. If import errors occur, verify you have compatible versions of torch, transformers, and Unsloth installed. If a model cannot be found, double-check that you are using a supported model name or have required access to private models.

Requirements

• Python 3.10–3.12
• NVIDIA GPU with CUDA support (recommended)
• Node.js and npm
• Access to a model repository compatible with Unsloth (e.g., Hugging Face) for loading pretrained models and datasets.

Performance and capabilities

Unsloth offers 2x faster fine-tuning, up to 80% less VRAM usage, and extended context lengths (up to 13x longer in some configurations) with maintained model quality. It supports 4-bit quantization and optimized backpropagation, enabling efficient training of large models on consumer hardware.

Available tools

check_installation

Verify that Unsloth is properly installed on your system.

list_supported_models

List all models supported by Unsloth, including Llama, Mistral, Phi, and Gemma variants.

load_model

Load a pretrained model with Unsloth optimizations for faster inference and fine-tuning.

finetune_model

Fine-tune a model using Unsloth optimizations with LoRA/QLoRA techniques.

generate_text

Generate text using a fine-tuned Unsloth model.

export_model

Export a fine-tuned Unsloth model to formats such as GGUF, Ollama, VLLM, or Hugging Face.