firecrawl/cli

Overview

This skill downloads an entire website into organized local files so you can keep documentation, reference pages, or site sections offline. It combines site mapping and scraping into nested directories under .firecrawl/, and supports multiple per-page formats plus screenshots. Use it to bulk-save pages in Markdown, plain text, screenshots, or custom combinations for each page.

How this skill works

The skill first maps the site to discover URLs, then scrapes each discovered page according to the chosen options. It writes outputs into a directory tree under .firecrawl/, creating one or more files per page (for example index.md, links.txt, screenshot.png). All scrape options apply to the download flow, including format selection, content filters, screenshots, and path include/exclude rules.

When to use it

• Save an entire site or a specific section for offline access
• Download documentation or developer guides for local reference
• Bulk-export pages into Markdown, text, screenshots, or multiple formats per page
• Archive a site snapshot for audits, backups, or research
• Automate large downloads in CI or agent-driven workflows using -y

Best practices

• Always pass -y in automated or CI runs to skip the confirmation prompt
• Use --limit to control scope and avoid overloading the target site
• Filter with --include-paths and --exclude-paths to target relevant sections
• Prefer --only-main-content when you want clean article text without navigation
• Combine formats (e.g., markdown,links) when you need both readable content and link inventories
• Use screenshots (--screenshot or --full-page-screenshot) for visual verification or design archives

Example use cases

• Download a product docs site as Markdown + screenshots for offline reading and team distribution
• Archive a developer portal section before a major API change using include/exclude paths
• Bulk-save knowledge base articles into a local folder for internal search indexing
• Create per-page pairs of content and link lists to support data curation or migration
• Automate nightly snapshots of a site section in CI with -y and --limit to keep recent copies