MCP Webscan Server

You run a focused MCP server that fetches, analyzes, and extracts information from web pages. It converts pages to Markdown, extracts links, crawls sites, checks links, finds matching patterns, and can generate sitemaps, all accessible through an MCP client.

How to use

Start the server locally and connect to it with your MCP client. You will run the server via a standard Node process and then issue commands to perform tasks such as fetching a page, extracting links, crawling a site, or generating a sitemap. The server operates over a stdio transport, so you can integrate it with clients that expect a local process interface. Begin by starting the server, then use the available tooling to accomplish web content scanning and analysis tasks.

How to install

Prerequisites: ensure you have Node.js version 18 or higher and npm installed on your system.

Clone the project repository, install dependencies, and build the project using the following commands.

# Clone the repository
git clone <repository-url>
cd mcp-server-webscan

# Install dependencies
npm install

# Build the project
npm run build

Starting the server

Run the server using the standard start command. The server is designed to operate over stdio so it can be used directly with MCP clients.

npm start

Configuration and environment

Configure the MCP client to connect to the local server through the stdio transport. The example configuration below shows how to hook up the server in a client that supports MCP stdio endpoints.

{
  "mcpServers": {
    "webscan": {
      "command": "node",
      "args": ["path/to/mcp-server-webscan/dist/index.js"],
      "env": {
        "NODE_ENV": "development"
      }
    }
  }
}

Notes and tips

If you modify code or switch between development and production modes, adjust the NODE_ENV value accordingly. When you rebuild, ensure the dist output path referenced in the startup command remains accurate for your environment.

Available tools

fetch_page

Fetches a web page and converts it to Markdown. Parameters: url (required), selector (optional).

extract_links

Extracts all links from a web page with their text. Parameters: url (required), baseUrl (optional).

crawl_site

Recursively crawls a website up to a specified depth. Parameters: url (required), maxDepth (optional, default 2).

check_links

Checks for broken links on a page. Parameters: url (required).

find_patterns

Finds URLs matching a specific pattern. Parameters: url (required), pattern (required).

generate_sitemap

Generates a simple XML sitemap. Parameters: url (required), maxUrls (optional, default 100).