LilFetch

MCP server for scraping webpages to clean Markdown, wrapped for easy npx use. Uses crawl4ai and playwright.

javascript

0

GitHub Stars

javascript

Language

7 months ago

First Indexed

3 months ago

Catalog Refreshed

Documentation & install

Readme and setup notes from the catalogue, plus a client-ready config you can copy for your MCP host.

Installation

Add the following to your MCP client configuration file.

Configuration

View docs

{
  "mcpServers": {
    "jphdevsf-lilfetch-mcp": {
      "command": "npx",
      "args": [
        "lilfetch"
      ]
    }
  }
}

You can enable lilFetch MCP Server to scrape html or text from webpages directly within your chat prompts. It runs with Playwright in a headless Chromium instance to handle JavaScript-heavy sites, making it a robust tool for market research, content gathering, and quick context for writing tasks. This guide shows you how to use the server with a client, how to install it, and additional notes to help you get up and running quickly.

How to use

Start the lilFetch MCP server in your environment and connect your MCP client. You can then prompt it with a request to scrape one or more URLs and return the results so you can use them in subsequent queries. For example, you can ask lilFetch to pull the top news headlines from a site and write the results to a markdown file in your repository root. You can reuse the scraped content as context for new file creation, market research summaries, or other tasks that require up-to-date web data.

How to install

Prerequisites before installing the MCP server:

Node.js 14+
Python 3.10+ (auto-detected during setup)
First run time ~200MB download for Playwright browsers and about 1–2 minutes

Option 1: Global Install

Install globally to use anywhere in your workspace by running these steps.

git clone https://github.com/jphdevsf/lilfetch-mcp.git lilfetch-mcp
cd lilfetch-mcp

npm run global-install

Configure in a VS Code workspace

Add the MCP server configuration to your workspace settings so your editor can communicate with lilFetch.

{
  "servers": {
    "lilFetch": {
      "type": "stdio",
      "command": "npx",
      "args": ["lilfetch"]
    }
  }
}

Test It (global install)

Open a new terminal and start the MCP server, then issue a sample prompt within your editor.

npx lilfetch

Option 2: Local Install

If you prefer to test or extend within the repository itself, install locally.

git clone https://github.com/jphdevsf/lilfetch-mcp.git lilfetch-mcp
cd lilfetch-mcp

npm install

Configure in a VS Code workspace (local install)

Add a local MCP configuration that runs the lilFetch wrapper from the repository.

{
  "servers": {
    "lilFetch": {
      "type": "stdio",
      "command": "node",
      "args": ["bin/lilfetch.js"]
    }
  }
}

Test It (local install)

Run the local wrapper to start the server and test with a prompt.

npm run dev
# or directly if you prefer the binary in this repo
./node_modules/.bin/lilfetch

Notes and tips

Ensure the executable bit is set for the local wrapper when using a file-based start command.
If you run into Python or Playwright setup issues, re-create the Python virtual environment and reinstall dependencies as described in the troubleshooting notes.