URL Fetcher

一个基于 FastMCP 的网页内容获取服务器，允许 AI 通过 MCP 协议获取指定网址的网页内容。

python

0

GitHub Stars

python

Language

7 months ago

First Indexed

3 months ago

Catalog Refreshed

Documentation & install

Readme and setup notes from the catalogue, plus a client-ready config you can copy for your MCP host.

Installation

Add the following to your MCP client configuration file.

Configuration

View docs

{
  "mcpServers": {
    "lucoo01-url-fetcher": {
      "command": "python",
      "args": [
        "url_fetcher_server.py"
      ]
    }
  }
}

You can run a FastMCP-based server that fetches and processes web pages on demand. This URL Fetcher MCP Server lets clients request full page content, smart extraction of the main article, metadata, and optional summaries from any HTTP/HTTPS URL, all through MCP-compatible interactions.

How to use

To use the URL Fetcher MCP Server, start the local stdio-based MCP server and connect with an MCP client. You can request the full HTML content of a page, ask for a concise summary, or fetch multiple pages in parallel. The server handles content extraction, metadata retrieval, and error handling to make it easy for your AI workflows to analyze web content without dealing with raw pages.

How to install

Prerequisites: you need Python installed on your system. Preferably use a modern Python 3.x environment.

Step 1 — Install dependencies

pip install -r requirements.txt

-or-

pip install fastmcp>=2.0.0 aiohttp>=3.8.0 beautifulsoup4>=4.11.0

Step 2 — Start the server

python url_fetcher_server.py

The server starts in STDIO mode and waits for MCP clients to connect.

## Step 3 — Configure your MCP client (example)

If you use a configuration tool to register MCP servers, add a stdio entry pointing to the fetcher server. You will reference the Python program that runs the URL Fetcher server.

{ "mcpServers": { "url-fetcher": { "command": "python", "args": ["path/to/your/url_fetcher_server.py"], "env": {}, "autoApprove": [ "fetch_web_content", "fetch_web_content_summary", "fetch_multiple_web_content", "fetch_multiple_web_content_summary", "validate_url" ] } } }


## Available tools

### fetch\_web\_content

Retrieves the complete content of a specified URL, returning the full HTML/text content for analysis.

### fetch\_web\_content\_summary

Fetches a summarized version of a web page, with optional max\_length to control the summary size.

### fetch\_multiple\_web\_content

Fetches the full content of multiple URLs in parallel, improving throughput for batch requests.

### fetch\_multiple\_web\_content\_summary

Fetches summarized content for multiple URLs in parallel, with optional per-item max\_length and concurrency controls.

### validate\_url

Validates the format of a given URL and returns basic information about it.