LibreModel

MCP cerver to let claude use llamacpp. Uses API.

javascript

8

GitHub Stars

javascript

Language

6 months ago

First Indexed

2 months ago

Catalog Refreshed

Documentation & install

Readme and setup notes from the catalogue, plus a client-ready config you can copy for your MCP host.

Installation

Add the following to your MCP client configuration file.

Configuration

View docs

{
  "mcpServers": {
    "openconstruct-llama-mcp-server": {
      "command": "node",
      "args": [
        "/home/jerr/llama-mcp/dist/index.js"
      ],
      "env": {
        "LLAMA_SERVER_URL": "http://localhost:8080"
      }
    }
  }
}

You run a Model Context Protocol (MCP) server that connects Claude Desktop to your local LLM via llama-server. This bridge gives you full conversation flow, precise parameter control, health checks, testing tools, and visibility into performance and token usage, all configurable through environment variables.

How to use

Once your MCP server is running, you connect Claude Desktop to your local LLM through the configured MCP endpoint. You can chat with LibreModel, run quick capability tests, and monitor server health from Claude Desktop. The configuration snippet below defines how Claude Desktop should start the MCP bridge.

{
  "mcpServers": {
    "libremodel": {
      "command": "node",
      "args": ["/home/jerr/llama-mcp/dist/index.js"]
    }
  }
}

What you can do inside Claude Desktop

Use the chat tool to converse with LibreModel. You can adjust parameters such as temperature, max_tokens, top_p, and top_k to influence responses. Run quick_test to verify capabilities like creativity or knowledge, and use health_check to verify that the MCP bridge and llama-server are functioning properly.

Understand the flow

Claude Desktop communicates with the MCP server, which translates your requests into llama-server API calls and returns responses suitable for display in the chat interface. This enables seamless interaction with your local model without exposing you to the underlying protocol details.

Security and hosting notes

Keep your llama-server reachable on the configured URL and port, and ensure your network allows traffic between Claude Desktop, the MCP bridge, and llama-server. Use environment variables to point the MCP bridge to the correct llama-server instance and to safeguard access where needed.

Troubleshooting

If Claude Desktop cannot reach the local model, verify that llama-server is running on the specified port and that the model load succeeded. Check firewall settings and network routes between your machine and the configured endpoints.

If Claude Desktop does not recognize the MCP tool, restart Claude Desktop after making changes to the MCP configuration and confirm the path to the MCP index script is correct and absolute.

Available tools

chat

Converse with LibreModel from Claude Desktop using adjustable parameters such as message, temperature, max_tokens, top_p, top_k, and system_prompt

quick_test

Run predefined capability tests to verify creative, mathematical, knowledge-based, or general capabilities of the local model

health_check

Check the MCP bridge and llama-server status to ensure healthy operation