LLM MCP Bridge Server

You can connect any OpenAI-compatible LLM API to this MCP bridge to analyze, compare, and evaluate model quality across providers. It acts as a neutral mediator that lets you run common MCP tools against multiple backends, collect performance metrics, and generate comprehensive quality reports.

How to use

You start by configuring one or more MCP connections that point to your LLM servers. Use the provided tools to check connectivity, list models, chat with performance metrics, run benchmarks, and generate quality reports. You can override the base URL for a specific call to test a different backend without changing your main setup.

How to install

cd llm-mcp-bridge
npm install
npm run build

Additional content

Configuration in your client is driven by environment variables and per-server settings. The bridge supports multiple providers via HTTP endpoints and can run as a local stdio process for quick testing.

Environment variables you may use include the following:

• LLM_BASE_URL: URL of the target OpenAI-compatible API. Example values are shown in each configuration block below.

Local and remote MCP connections (examples)

# Local LM Studio-like server (HTTP)
# URL: http://localhost:1234/v1

# Local Ollama-like server (HTTP)
# URL: http://localhost:11434/v1

# OpenAI-compatible cloud service (HTTP)
# URL: https://api.openai.com/v1

# Groq-compatible service (HTTP)
# URL: https://api.groq.com/openai/v1

Tools and endpoints you can use

You can access a set of MCP tools to interact with and evaluate models. Each tool is designed to perform a specific task, from discovering available models to producing a full quality report.

Examples of common tasks

@llm_status

@llm_get_models

@llm_chat prompt="Explica qué es machine learning" temperature=0.5 maxTokens=256

@llm_benchmark prompts=["Hola", "¿Qué hora es?", "Cuenta hasta 10"]

@llm_quality_report

Available tools

llm_get_models

Obtains a list of models in JSON format.

llm_status

Checks the connection status with the MCP bridge.

llm_list_models

Lists models in a human-friendly format.

llm_chat

Chat with the model and receive performance metrics.

llm_benchmark

Runs benchmarks across multiple prompts.

llm_evaluate_coherence

Evaluates the coherence of model outputs.

llm_test_capabilities

Tests capabilities across different areas.

llm_compare_models

Compares multiple models side-by-side.

llm_quality_report

Generates a comprehensive quality report.