xCOMET

You can use the xCOMET MCP Server to evaluate machine translation quality in an automated workflow. It connects to the xCOMET model to compute a quality score, identify potential errors, and process multiple translation pairs efficiently, helping you iterate on translations faster and with clearer metrics.

How to use

Start by running the MCP server locally or remotely, then connect your MCP client to obtain a quality score and error analysis for each translation pair. Use the HTTP endpoint for remote access or run the server locally to integrate directly into your translation pipeline.

Common usage patterns include evaluating single pairs, detecting errors, and batch evaluating many pairs in one call. You can enable GPU acceleration if you have compatible hardware to speed up inference, and you can preload the model so the first request remains fast.

How to install

Prerequisites: you need Node.js and Python, plus access to a Python environment with the required packages.

# Prerequisites: Node.js and Python should be installed

# Install the MCP server globally (JavaScript runtime)
npm install -g xcomet-mcp-server

# Optional: preinstall dependencies and build if you are using a local clone
# If you cloned a repo, navigate to the directory first
# npm install
# npm run build

Configuration notes

The MCP server supports both local (stdio) and remote (http) operation, with environment variables to customize behavior. You can choose the model size, Python path, and preload behavior as part of your startup configuration.

{
  "mcpServers": {
    "xcomet": {
      "command": "npx",
      "args": ["-y", "xcomet-mcp-server"],
      "env": {
        "XCOMET_MODEL": "Unbabel/XCOMET-XL",
        "XCOMET_PYTHON_PATH": "/path/to/python3",
        "XCOMET_PRELOAD": "true",
        "TRANSPORT": "http",
        "PORT": "3000"
      }
    }
  }
}

Using with a client

HTTP (remote) access: start the server with the http transport, then query the endpoint to evaluate translations.

# Start in HTTP mode on port 3000
TRANSPORT=http PORT=3000 npm start

# Connect to http://localhost:3000/mcp from your MCP client

Model selection and performance tips

Choose a model based on your quality needs and available memory. XL is a strong default; XXL offers higher quality at a higher memory cost. For lighter workloads, the wmt22-comet-da model might be suitable.

{
  "mcpServers": {
    "xcomet": {
      "command": "npx",
      "args": ["-y", "xcomet-mcp-server"],
      "env": {
        "XCOMET_MODEL": "Unbabel/XCOMET-XXL"
      }
    }
  }
}

Available tools

xcomet_evaluate

Evaluate translation quality for a single source-translation pair, returning a quality score and a summary.

xcomet_detect_errors

Detect and categorize translation errors with severity levels (minor, major, critical).

xcomet_batch_evaluate

Evaluate multiple translation pairs in a single request with a single model load for efficiency.