Spark Documentation

You can search and read Apache Spark documentation through an MCP server that powers AI assistants to find exact sections or full pages quickly. It provides full-text search, section filtering, and easy deployment with Docker so you can incorporate Spark docs into your AI workflows without browsing manually.

How to use

You use the Spark documentation MCP server by connecting your MCP client to the server configuration described below. Once connected, you can perform full-text searches for Spark topics and retrieve complete documentation pages. Use the search tool to filter results by section (for example sql-ref, api, streaming) and then read or fetch the full page content for any result.

How to install

Prerequisites you need before installation are Docker installed on your machine and an MCP client that understands standard MCP server configurations. You can also run locally using the development flow if you prefer not to use Docker.

Install and run using Docker (recommended) Set up the Docker-based MCP server, which includes a pre-indexed Spark documentation set.

Step 1: Build the Docker image and run the server.

Docker — quick start

# Build the Docker image (includes pre-indexed documentation)
make docker-build

# Test the server
make docker-run

Local development flow (uv)

# Initialise the environment
make init

# Build the documentation index
make index

# Run the server
make run

Configuration

Configure your MCP client to connect to the Spark documentation server. The following configuration exposes a local Docker-based MCP server you can start directly from your MCP client.

{
  "mcpServers": {
    "spark_doc": {
      "command": "docker",
      "args": ["run", "-i", "--rm", "martoc/mcp-spark-documentation:latest"]
    }
  }
}

Additional configuration examples

If you build and run a locally built Docker image, you can use this variant in your MCP client configuration.

{
  "mcpServers": {
    "spark_doc": {
      "command": "docker",
      "args": ["run", "-i", "--rm", "mcp-spark-documentation"]
    }
  }
}

Notes on development without Docker

For local development without Docker, you can use the uv-based workflow if you are integrating with an MCP client that supports uv. The process involves initializing the environment, building the index, and running the server within the project directory.

Available tools

search_documentation

Search Spark documentation by keyword query with optional section filtering

read_documentation

Retrieve the full content of a specific documentation page