Trustwise

Trustwise MCP Server provides a suite of evaluation tools that you can call from MCP clients to assess AI safety, alignment, and performance. It enables you to measure metrics like faithfulness, relevancy, toxicity, carbon footprint, and cost for model outputs, helping you build safer and more efficient AI systems.

How to use

You connect an MCP client to the Trustwise MCP Server by configuring the client to launch the server as a remote process via Docker. Once connected, you can invoke any of the Trustwise metrics tools to evaluate a model response against your context, query, or policy. Use the tools to gather safety, quality, and cost signals and incorporate them into your evaluation pipelines, agents, or orchestration systems.

How to install

Prerequisites: install Docker on your machine and obtain a Trustwise API key.

Step 1: Prepare the client connection configuration. You will configure each MCP client to run the Trustwise MCP Server as a Docker process. Below are two example configurations for common clients.

{
  "mcpServers": {
    "trustwise": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "--rm",
        "-e",
        "TW_API_KEY",
        "ghcr.io/trustwiseai/trustwise-mcp-server:latest"
      ],
      "env": {
        "TW_API_KEY": "<YOUR_TRUSTWISE_API_KEY>"
      }
    }
  }
}

Additional installation notes

If you are using a specific Trustwise instance, provide the base URL to the API by setting the TW_BASE_URL environment variable in the same configuration. For example, set TW_BASE_URL to your instance URL to point requests to a private or dedicated deployment.

Additional content

Step 2: If you also use Cursor, add a similar configuration with the base URL variable included. This allows you to route Trustwise evaluations through your preferred MCP client environment.

{
  "mcpServers": {
    "trustwise": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "--rm",
        "-e",
        "TW_API_KEY",
        "-e",
        "TW_BASE_URL",
        "ghcr.io/trustwiseai/trustwise-mcp-server:latest"
      ],
      "env": {
        "TW_API_KEY": "<YOUR_TRUSTWISE_API_KEY>"
      }
    }
  }
}

Available tools

faithfulness_metric

Evaluate the faithfulness of a response to its context by comparing the output to the given context and evidence.

answer_relevancy_metric

Evaluate how relevant the answer is to the original query.

context_relevancy_metric

Evaluate how well the provided context supports the query or answer.

pii_metric

Detect personally identifiable information in a response.

prompt_injection_metric

Detect risks related to prompt injection.

summarization_metric

Evaluate the quality of summary content.

clarity_metric

Assess how clearly the response communicates information.

formality_metric

Evaluate the formality level of the response.

helpfulness_metric

Assess how helpful the response is in achieving the user’s goal.

sensitivity_metric

Evaluate sensitivity of content in relation to safety policies.

simplicity_metric

Evaluate simplicity and understandability of the response.

tone_metric

Evaluate the tone of the response.

toxicity_metric

Assess potential toxicity in the response.

refusal_metric

Detect refusals to answer or comply with the query.

completion_metric

Evaluate whether the response completes the user’s instruction.

adherence_metric

Evaluate adherence to a given policy or instruction.

stability_metric

Evaluate consistency across multiple responses.

carbon_metric

Estimate the carbon footprint of a response.

cost_metric

Estimate the cost of delivering a response.