WaterCrawl

WaterCrawl MCP provides a standardized interface to interact with WaterCrawl’s crawling, scraping, and search capabilities. You can run it locally via npx for quick testing or integrate it with AI assistants to perform web data gathering and processing through a consistent MCP API.

How to use

You run WaterCrawl MCP from your environment using a client that speaks the MCP protocol. The quickest way is to start an inline process with npx and pass your API key. You can attach the MCP as a service to AI assistants and use it to perform tasks like scraping pages, submitting search queries, downloading sitemaps, and managing crawl and search requests. You will configure either a direct npx invocation or a SSE server to allow real-time streaming of results.

How to install

Prerequisites you need before starting are Node.js and npm. Ensure Node.js is installed and accessible from your command line.

Install and run WaterCrawl MCP quickly with npx without installing anything globally.

npx @watercrawl/mcp --api-key YOUR_API_KEY

If you plan to integrate with AI assistants, you can configure the MCP server in the host client with a prebuilt command. For Codeium or Windsurf, you execute the MCP via npx and pass the API key and base URL as part of the MCP server configuration.

Example setup for Codeium/Windsurf client configuration.

{
  "mcpServers": {
    "watercrawl": {
      "command": "npx",
      "args": [
        "@watercrawl/mcp",
        "--api-key",
        "YOUR_API_KEY",
        "--base-url",
        "https://app.watercrawl.dev"
      ]
    }
  }
}

Additional usage modes

You can also run WaterCrawl MCP in SSE mode to enable real-time streaming of results. This is useful for interactive workflows or integration with clients that consume Server-Sent Events.

Example for Claude Desktop using SSE mode.

npx @watercrawl/mcp sse --port 3000 --endpoint /sse --api-key YOUR_API_KEY

Configuration options you’ll commonly use

• Base URL: your WaterCrawl API endpoint, e.g. https://app.watercrawl.dev
• API key: your WaterCrawl API key
• SSE port and endpoint: when using SSE mode to stream results in real time.

Troubleshooting tips

If you run into authentication errors, double-check that YOUR_API_KEY is correct and that the key is active for your project. If the API base URL is incorrect, set the --base-url value to the correct WaterCrawl instance. For SSE mode, ensure the port is open and not blocked by your firewall.

Security notes

Keep your API key confidential. Do not expose the key in client-side code or public repositories. Use environment variables or secure vaults when integrating WaterCrawl MCP into production workflows.

Examples

Two common usage patterns are shown below. The first runs MCP directly with npx, the second launches an SSE server for real-time streaming.

npx @watercrawl/mcp --api-key YOUR_API_KEY

npx @watercrawl/mcp sse --port 3000 --endpoint /sse --api-key YOUR_API_KEY

Notes on updates and compatibility

WaterCrawl MCP is designed to be compatible with standard MCP clients. If you update your client configuration, ensure the API key and base URL remain correct and that the SSE endpoint, if used, continues to point to the configured port and path.

License

ISC license is used for WaterCrawl MCP.

Available tools

scrape-url

Scrape content from a URL with customizable page options, including tag filtering, wait time, timeout, cookies handling, locale, headers, and actions like screenshot or pdf.

search

Perform web searches with configurable language, country, time range, depth, and result limits.

download-sitemap

Download a sitemap from a crawl request in different formats such as json, graph, or markdown.

manage-crawl

List, view details, stop, or download results of crawl requests.

manage-search

List, view details, or stop running search requests.

monitor-request

Monitor a crawl or search request in real-time with a timeout and optional file download.