vLLM

Exposes vLLM functionality through MCP endpoints for chat, completion, and model management.

python

GitHub Stars

python

Language

5 months ago

First Indexed

2 months ago

Catalog Refreshed

Documentation & install

Readme and setup notes from the catalogue, plus a client-ready config you can copy for your MCP host.

Installation

Add the following to your MCP client configuration file.

Configuration

View docs

{
  "mcpServers": {
    "micytao-vllm-mcp-server": {
      "command": "uvx",
      "args": [
        "vllm-mcp-server"
      ],
      "env": {
        "VLLM_MODEL": "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
        "VLLM_API_KEY": "YOUR_API_KEY",
        "VLLM_BASE_URL": "http://localhost:8000",
        "VLLM_HF_TOKEN": "hf_your_token_here",
        "VLLM_DEFAULT_MAX_TOKENS": "1024",
        "VLLM_DEFAULT_TEMPERATURE": "0.7"
      }
    }
  }
}

You can run a vLLM MCP Server that exposes vLLM capabilities to MCP-compatible clients. This server lets assistants send chat messages, request completions, inspect models, monitor health, and manage containerized deployments with automatic platform detection. It is designed to work with clients like Cursor and Claude, enabling seamless integration of vLLM-powered reasoning into your workflows.

How to use

Once the MCP server is running, you connect your MCP client to the local vLLM MCP endpoint. You can manage the vLLM server lifecycle from the MCP client layer, send chat messages, request text completions, list available models, and check server status and performance metrics. The platform-aware container control automatically chooses the right container image based on your system (Linux/macOS/Windows) and GPU availability, and it sets a sensible default max_model_len for CPU vs GPU usage.

Key usage patterns you will apply include starting the MCP-managed vLLM server, querying models, sending chat prompts with multi-turn history, and stopping the server when you’re done. You can also run benchmarks against the server to gauge performance and use pre-defined system prompts for common tasks.

How to install

Prerequisites: you need Python and Git installed on your machine. You also need a tool to run MCP endpoints from your client, such as uvx or a compatible process launcher.

Install the MCP server package from Python package index or install from source to run locally.

Install with uvx (recommended):

uvx vllm-mcp-server

Install with pip (alternative):

pip install vllm-mcp-server

Install from source for active development:

git clone https://github.com/micytao/vllm-mcp-server.git
cd vllm-mcp-server
pip install -e .

Available tools

vllm_chat

Send chat messages to vLLM with multi-turn conversation support.

vllm_complete

Generate text completions from a given prompt.

list_models

List all available models on the vLLM server.

get_model_info

Retrieve detailed information about a specific model.

vllm_status

Check the health and status of the vLLM server.

start_vllm

Start a vLLM server inside a container with automatic platform detection.

stop_vllm

Stop a running vLLM container.

restart_vllm

Restart the vLLM container.

list_vllm_containers

List all vLLM containers that are present on the system.

get_vllm_logs

Fetch container logs to monitor loading progress.

get_platform_status

Get detailed platform, Docker, and GPU status information.

run_benchmark

Run a GuideLLM benchmark against the server.

Built by

VeilStrat

AI signals for GTM teams