html2md_skill

This skill converts HTML to clean markdown using Readability and Turndown, automating content extraction for research and agent workflows.

Python

2.5k

GitHub Stars

2

Bundled Files

2 months ago

Catalog Refreshed

4 months ago

First Indexed

Readme & install

Copy the install command, review bundled files from the catalogue, and read any extended description pulled from the listing source.

Installation

Preview and clipboard use veilstrat where the catalogue uses aiagentskills.

npx veilstrat add skill openclaw/skills --skill html2md

_meta.json274 B
SKILL.md3.5 KB

Overview

This skill converts HTML pages into clean, agent-friendly Markdown using Mozilla Readability for main-content extraction and Turndown for HTML→Markdown conversion. It aggressively strips navigation, ads, footers, cookie banners, and social CTAs, and offers token-aware truncation and multiple input/output modes for automation.

How this skill works

The tool fetches or reads HTML, runs Readability to isolate the main article or falls back to a cleaned <body> when necessary, then converts the result with Turndown. Post-processing removes comments, zero-width characters, empty headings, breadcrumbs, and collapses excess whitespace. Optional flags control link/image preservation, JSON output, token budgeting, and input sources (URL, local file, or stdin).

When to use it

Extract readable article content from cluttered web pages for downstream processing
Prepare web text for LLM consumption with a strict token budget (--max-tokens)
Batch or cron jobs where consistent, noise-free markdown is required
Agent workflows that need programmatic output (--json) with title, url, markdown, and token count
Local file or piped HTML conversion during scraping or archival tasks

Best practices

Use --max-tokens when feeding content into an LLM to avoid exceeding context limits
Prefer --json for automated pipelines to receive metadata and token counts programmatically
Validate untrusted URLs before passing them in automated agents to reduce SSRF risk
Combine --no-links or --no-images flags when link or media noise would confuse downstream agents
Call the binary directly (execFileSync) from scripts to avoid shell injection risks

Example use cases

Convert a news article to Markdown and save for research notes: html2md https://example.com/article > article.md
Extract documentation pages in a CI job and enforce a token limit for indexing: html2md --max-tokens 2000 https://docs.example.com
Pipe local HTML from other tools into a clean Markdown stream: cat page.html | html2md --stdin
Fetch and get structured output for an agent: html2md --json https://example.com | jq .markdown
Clean Hacker News-like pages with --no-links --no-images to focus on text content

FAQ

URLs, local files via --file, and piped HTML via --stdin are supported.

How does token budgeting work?

Use --max-tokens N; the skill preserves headings, fills the remaining budget in document order, and appends a truncation note. It approximates 1 token ≈ 4 characters.