Crawleo

Crawleo MCP provides real-time web search and deep crawling capabilities for AI assistants using MCP. It enables you to query live web data and extract content from URLs while keeping your data private and easily accessible from your preferred MCP client or IDE integrations.

How to use

You can connect your MCP client to Crawleo to perform real-time web searches or crawl web pages during AI workflows. Use web.search to look up current information in any language or country, returning results in formats like Enhanced HTML, Markdown, or plain text. Use web.crawl to extract the core content from specific URLs, with optional markdown output and screenshots. Configure the connection, supply your API key, and start issuing search or crawl requests from your AI assistant.

How to install

Prerequisites: you need Node.js and npm, or you can run Crawleo as a Docker container or use a hosted MCP endpoint.

# Option 1: Local usage with npm (recommended for local testing)
npm install -g crawleo-mcp

# Or run without installing
npx crawleo-mcp

# Option 3: Docker (build and run)
docker build -t crawleo-mcp .
docker run -e CRAWLEO_API_KEY=your_api_key crawleo-mcp

# Option 4: Remote server (no installation required) - use the hosted endpoint
# No local install needed; configuration is handled in your MCP client

Configuration and usage notes

You can connect to Crawleo in two primary ways: a remote HTTP endpoint or a local Docker-based runtime. The HTTP endpoint is hosted at a fixed URL and is the easiest starting point. The local runtime via Docker requires you to provide an API key and runs Crawleo as a separate process that your MCP client talks to.

Security and keys

Always keep your API key secret. Use HTTPS when communicating with the remote endpoint and avoid exposing keys in logs or shared configurations. If you are using Docker, pass your API key as an environment variable (CRAWLEO_API_KEY) when starting the container.

Available tools

web.search

Real-time web search that returns results in multiple formats (enhanced HTML, raw HTML, Markdown, plain text) and supports device-specific views and optional auto-crawling.

web.crawl

Deep content extraction from specified URLs, with options for markdown output and optional screenshots.