Puppeteer Vision

You can scrape webpages and convert them to well-formatted Markdown using a Puppeteer-powered MCP server. It automatically handles interactive elements like cookies and captchas, extracts the main content, and converts HTML to clean Markdown, making it easy to feed polished content into your LLM workflows.

How to use

Run the server using a simple NPX command to stay on the latest version. Provide your API key and any optional settings as environment variables. You can operate in three communication modes: stdio for direct integration, SSE over HTTP for networked access, or HTTP with session management for full transport features.

To scrape a webpage, use the built-in tool scrape-webpage. It takes the target URL and optional controls for interaction behavior, interaction retries, and network idle waits. The result is a single markdown document representing the page content, with metadata describing the success and size.

If you want to see browser activity while testing, enable headful mode. Otherwise, leave the browser headless for normal operation.

How to install

Prerequisites: ensure you have Node.js and npm installed on your system.

Environment setup: you control configuration with either a .env file or shell environment variables.

Start the server with NPX. This downloads the latest package if needed and runs it in stdio mode by default.

Here is a minimal example of how you can configure and run everything.

# Required
OPENAI_API_KEY=YOUR_OPENAI_API_KEY

# Optional (defaults shown)
VISION_MODEL=gpt-4.1
API_BASE_URL=https://api.openai.com/v1
TRANSPORT_TYPE=stdio
PORT=3001
DISABLE_HEADLESS=false

Configuration and usage notes

Environment variables control how the server uses the vision model and transports. The OPENAI_API_KEY is required. You can choose a vision model with or without a specific API base URL. The transport type selects stdio, SSE, or HTTP modes.

Example local run via NPX, using stdio transport by default and with headless browser enabled:

npx -y puppeteer-vision-mcp-server

Tool usage: scrape-webpage

The server exposes a tool named scrape-webpage for extracting content from a given URL. You can pass controls to auto-interact with page overlays and to wait for network idle before processing.

Key parameters include the page URL, and optional flags for automatic interaction and interaction attempts.

Example MCP configuration

{
  "mcpServers": {
    "puppeteer_scraper": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "puppeteer-vision-mcp-server"] ,
      "env": {
        "OPENAI_API_KEY": "YOUR_OPENAI_API_KEY_HERE",
        "VISION_MODEL": "gpt-4.1",
        "API_BASE_URL": "https://api.openai.com/v1",
        "TRANSPORT_TYPE": "stdio",
        "DISABLE_HEADLESS": "false"
      }
    }
  }
}

Notes on modes and endpoints

In stdio mode, communication is direct via process I/O and no HTTP server starts. In SSE mode, an HTTP server runs on port 3001 by default and can be accessed at /sse. In HTTP mode, a streamable HTTP transport with session management runs on the same port and is accessible at /mcp.

Troubleshooting

If you encounter issues loading pages or handling overlays, enable headful mode to observe interactions in real time and adjust DISABLE_HEADLESS accordingly.

Check that your OPENAI_API_KEY is valid and that the specified TRANSPORT_TYPE matches how you connect from your MCP orchestrator.

Available tools

scrape-webpage

A tool to scrape a webpage URL, automatically handle interactive elements if enabled, extract main content with Readability, and convert to Markdown. The tool returns the content as Markdown with associated metadata indicating success and content size.