Docs Fetch

You can use the Docs Fetch MCP Server to autonomously fetch and recursively explore web content, gathering clean, content-rich pages from a single domain up to a chosen depth. This helps you train or inform an LLM by letting it learn from documentation and related pages efficiently.

How to use

You use this MCP server with an MCP client to fetch web content and guide the exploration through linked pages. Start with a target URL and an exploration depth to control how far from the root page the server should follow links. The server will return the root page content, a list of linked pages found within the same domain, and the contents of those pages up to your specified depth. You can rely on the server to filter navigation links and focus on content-rich pages, helping the model learn from coherent documentation and related material.

How to install

Prerequisites: you need Node.js and npm installed on your system.

1. Clone the repository that provides the Docs Fetch MCP Server.

2. Install dependencies.

3. Build the project.

4. Configure your MCP client to use the server. The following configuration is used to run the MCP server locally using a stdio connection.

{
  "mcpServers": {
    "docs-fetch": {
      "command": "node",
      "args": [
        "/path/to/docs-fetch-mcp/build/index.js"
      ],
      "env": {
        "MCP_TRANSPORT": "pipe"
      }
    }
  }
}

Configuration and usage notes

The server is designed for local development and integration with an MCP client. It runs a local Node.js process and exports a single MCP tool that fetches page content with optional recursive exploration. Use the provided JSON config to run the server as shown in the installation section. If you need to adjust how the MCP channel is established, modify the MCP_TRANSPORT environment variable accordingly.

Troubleshooting and notes

If you encounter load errors or timeouts, verify that the path to the built index script is correct and that Node has access to the required network resources. Ensure the depth parameter is within the supported range (1-5) and that you are staying within the same domain for link exploration to avoid cross-domain restrictions.

Available tools

fetch_doc_content

Fetches web page content with optional recursive exploration up to a specified depth and returns root content, page links, and extracted content for each explored page.