OpenXAI

You run an MCP server that exposes OpenXAI’s evaluation and benchmarking tools through a standard interface. This enables you to load datasets and models, generate explanations with multiple methods, evaluate those explanations with established metrics, and compare results on leaderboards. It’s designed to plug into MCP clients and other compatible applications, so you can automate experiments, benchmarks, and deployments in a consistent way.

How to use

Use an MCP client to connect to the OpenXAI MCP server and perform operations such as loading datasets, loading models, generating explanations, evaluating explanations, and viewing leaderboards. You interact with the server through predefined commands exposed by the client, trigger actions, retrieve results, and manage deployments. You will typically load a dataset, pick a model trained on that dataset, request explanations for predictions, and then evaluate or compare those explanations using the framework’s metrics.

How to install

Prerequisites you need to install first are Node.js 18+ and Python 3.7+ to support the MCP server and the OpenXAI functionality.

Clone the MCP server repository, install dependencies, and install the Python package used by the server tooling.

Additional steps and configuration

Configure the MCP server integration for your environment by registering it in your Cursor/MCP client settings. The example shows how to register a local stdio-based server that you start with Node and point to the server entry file.

{
  "mcpServers": {
    "openxai": {
      "command": "node",
      "args": ["/path/to/openxai-mcp/index.js"],
      "env": {}
    }
  }
}

Development and local run

During development you can start the MCP server in development mode and run tests to ensure everything works as expected.

# Start the server (example)
npm start

# Development with auto-reload (example)
npm run dev

# Run tests
npm test

Available tools

list_datasets

List available datasets in the OpenXAI framework. Optional filter by category: synthetic, real-world, tabular, image, text, all.

load_dataset

Load a specific dataset from OpenXAI. Parameters include dataset_name and an optional download flag.

list_models

List available pre-trained models. Optional filters by dataset and model_type.

load_model

Load a pre-trained model for a given dataset with an option to load pretrained weights.

list_explainers

List available explanation methods, with an optional method_type filter.

generate_explanation

Generate explanations for model predictions using a chosen method and input data.

list_metrics

List available evaluation metrics, with optional metric_type filtering.

evaluate_explanation

Evaluate explanation quality using a specified metric and model information.

get_leaderboard

Retrieve leaderboard results for explanation methods, with optional dataset and metric filters.

get_framework_info

Get information about the OpenXAI framework and its features.