WebClone

WebClone is now an official Model Context Protocol (MCP) server, making website cloning available to AI agents like Claude, CrewAI, and any MCP-compatible framework!

python

1

GitHub Stars

python

Language

6 months ago

First Indexed

2 months ago

Catalog Refreshed

Documentation & install

Readme and setup notes from the catalogue, plus a client-ready config you can copy for your MCP host.

Installation

Add the following to your MCP client configuration file.

Configuration

View docs

{
  "mcpServers": {
    "ruslanmv-webclone": {
      "command": "python",
      "args": [
        "/path/to/webclone/webclone-mcp.py"
      ]
    }
  }
}

WebClone MCP Server lets AI agents like Claude Desktop and other MCP-compatible frameworks control website cloning tasks. It exposes a trusted, local interface that AI agents can use to start clones, fetch files, save authentication data, and inspect site information, enabling automated workflows and integration with broader AI tooling.

How to use

You interact with WebClone as an MCP server from your MCP client. The server provides a local interface that AI agents can call to perform actions such as initiating a website clone, downloading individual files, saving and reusing authentication cookies, listing saved sessions, and analyzing a site before cloning.

How to install

Prerequisites for the MCP server are minimal. You need Python installed on your machine to run the MCP script directly. You also need a way to start the MCP server and point clients to it using the supported command.

Configuration and setup for MCP clients

To enable MCP clients like Claude Desktop to talk to WebClone, add an MCP server entry that points to the local Python script. Use the following configuration as the starting point.

{
  "mcpServers": {
    "webclone": {
      "type": "stdio",
      "command": "python",
      "args": ["/path/to/webclone/webclone-mcp.py"]
    }
  }
}

Usage notes and available actions

Once configured, you can instruct your MCP client to perform these actions through the WebClone MCP server: clone_website to download an entire site, download_file to fetch a specific URL, save_authentication to store login sessions, list_saved_sessions to view existing cookies, and get_site_info to analyze a site before cloning.

Security considerations

Operate the MCP server on a trusted machine or in a secured environment. Treat the local MCP endpoint as you would any other automation surface: limit access, use authenticated clients, and rotate credentials for sensitive sessions.

Troubleshooting

If the MCP server does not start, verify that Python is installed and that the path to webclone-mcp.py is correct. Ensure the client is configured to connect to the same local endpoint and that the script has executable permissions.

Examples with MCP clients

An MCP client can issue commands such as clone_website with a target URL and optional parameters. The client receives structured results that include page counts, asset summaries, and total sizes, enabling you to validate outcomes within automated workflows.

Notes

This MCP integration is designed to work with Claude Desktop, CrewAI, LangChain, and any MCP-compatible framework. The example configuration shows how to wire the local Python-based MCP server into a client’s config.

Appendix: Tools exposed by WebClone MCP server

The MCP server exposes the following actions to clients:

Appendix: Tools details

clone_website — Download an entire website, preserving structure and assets. download_file — Retrieve a specific file or URL. save_authentication — Save login cookies for session reuse. list_saved_sessions — List all stored authentication cookies. get_site_info — Analyze a website to gather metadata before cloning.

Available tools

clone_website

Downloads an entire website, preserving pages, assets, and structure for later analysis or archiving.

download_file

Fetches a specific file or URL through the MCP interface.

save_authentication

Saves login cookies or tokens to enable authenticated access in future runs.

list_saved_sessions

Lists all stored authentication cookies or sessions for quick reuse.

get_site_info

Retrieves metadata and basic analysis about a target site before cloning.