Karenina

You can explore Karenina verification results using an experimental MCP server that translates natural language questions into precise SQL queries against a Karenina SQLite database. This makes it easy to inspect results, compare models, and derive insights without writing SQL yourself.

How to use

After configuring your local Karenina MCP server, you interact with it through an MCP client to ask questions in plain language. The server analyzes the available data schema, identifies relevant views, and then generates exact queries to return results as neatly formatted tables. Typical questions include overall pass rates, model comparisons, and performance by keywords.

How to install

Prerequisites: You need Python and the MCP tooling installed, plus access to your Karenina SQLite database.

# Install prerequisites if needed
# Example placeholders; replace with your environment specifics
python3 -m venv venv
source venv/bin/activate
pip install uvx  # or your preferred MCP tool, if applicable

Configuration and running the server

Start the server in STDIO mode to run locally with direct command input, or run in HTTP mode to expose an API endpoint for remote clients.

# Run in STDIO mode (local interaction)
uv run karenina-mcp

# Run in HTTP mode (remote access on port 8000)
uv run fastmcp run src/karenina_mcp/server.py --transport http --port 8000

HTTP server details

When running in HTTP mode, the server will be accessible at http://localhost:8000 by default. You can customize the host and port as needed.

# Example: expose on all interfaces on port 8000
uv run fastmcp run src/karenina_mcp/server.py --transport http --host 0.0.0.0 --port 8000

Claude integration (configuring clients)

Configure your Claude Code or Claude Desktop to connect to the MCP server so natural language questions are routed correctly. Use the provided command and directory references to point Claude to the MCP server.

{
  "mcpServers": {
    "karenina": {
      "command": "uv",
      "args": ["run", "karenina-mcp"]
    }
  }
}

Tools you can use

The server exposes a couple of core actions you can invoke to work with your data: configure_database to connect to your Karenina SQLite database, and get_schema to retrieve detailed schema information for specific views.

# Initialize the database connection
{ "configure_database": { "db_path": "/path/to/karenina.db" } }

# Inspect the schema for relevant views
{ "get_schema": { "view_names": ["template_results", "question_attributes"] } }

Available tools

configure_database

Initialize the server by connecting to your Karenina SQLite results database and returning the available tables and views.

get_schema

Return detailed schema documentation for specific views, including columns, types, keys, joins, and example queries.