WebScout

Provides web search via DuckDuckGo and advanced fetching with LLM extraction for intelligent content analysis.

python

0

GitHub Stars

python

Language

4 months ago

First Indexed

2 months ago

Catalog Refreshed

Documentation & install

Readme and setup notes from the catalogue, plus a client-ready config you can copy for your MCP host.

Installation

Add the following to your MCP client configuration file.

Configuration

View docs

{
  "mcpServers": {
    "rainyuniverse-mcp-webscout": {
      "command": "mcp-webscout",
      "args": [],
      "env": {
        "PROXY_URL": "http://127.0.0.1:7890",
        "USE_PROXY": "true",
        "PYTHONUTF8": "1",
        "DEEPSEEK_API_KEY": "YOUR_DEEPSEEK_API_KEY",
        "DEFAULT_MAX_LENGTH": "5000"
      }
    }
  }
}

You can run MCP WebScout as a dedicated MCP server to perform web searches and extract intelligent content using LLM-powered analysis. It combines DuckDuckGo search with advanced fetching and extraction to help you obtain concise results and structured content from the web.

How to use

Use an MCP client to connect to the WebScout server and access two core capabilities. First, perform web searches to retrieve relevant results with titles, URLs, and summaries. Second, fetch web pages with optional extraction using an LLM to obtain structured content or summaries based on your prompts.

How to install

Prerequisites: you need Python 3.10 or newer, pip, and Playwright for browser automation. If you plan to use DeepSeek for extraction, you also need a DeepSeek API key. Proxies may be required for users in mainland China.

git clone <repository>
cd mcp-webscout
python -m venv .venv

# Windows
.venv\Scripts\activate

# macOS/Linux
source .venv/bin/activate

pip install -e ".[dev]"

playwright install chromium

cp .env.example .env

Edit the .env file to provide needed configuration such as the DeepSeek API key and proxy settings if you are in a restricted network.

Additional configuration and usage notes

Environment variables you may need to set include DEEPSEEK_API_KEY for LLM extraction, PROXY_URL and USE_PROXY if you route traffic through a proxy, and optional DEFAULT_MAX_LENGTH to limit output size. If you are in mainland China or behind a restricted network, ensure your proxy is correctly configured.

Troubleshooting and tips

If you encounter issues starting the server, verify your Python version, ensure dependencies are installed, and confirm that Playwright browsers are available. For extraction failures, check that your DEEPSEEK_API_KEY is valid and that network access to the DeepSeek service is permitted by your environment.

Available tools

search

Query the web using DuckDuckGo and return formatted results with titles, URLs, and snippets.

fetch

Fetch a URL with optional LLM-driven extraction to produce structured content or summaries.