web_archive_analysis_skill

Overview

This skill uses the Wayback Machine to reconstruct a domain's historical technology footprint and detect migrations over time. It queries the CDX API for archived snapshots, fetches representative pages, and builds a timeline of detected frameworks, hosting changes, and frontend stacks. Outputs include snapshot coverage, per-snapshot technology lists, detected migrations, and confidence estimates.

How this skill works

The skill queries the Wayback CDX API with filters (status 200, monthly collapse) to collect available snapshots between specified years. It selects representative snapshots at regular intervals, fetches archived HTML pages, and extracts signals from headers, HTML elements, script/link paths, and common platform patterns. Changes are detected by comparing extracted technologies across snapshots and mapping indicator patterns to known migration types.

When to use it

• Investigating historical platform or CMS migrations for a target domain
• Validating when a site moved from on-prem to cloud/CDN or changed hosting provider
• Building timelines for security research, bug bounty reconnaissance, or pentest planning
• Confirming disappearance or introduction of specific libraries, frameworks, or CDN footprints
• Correlating technology transitions with known incidents or upgrade windows

Best practices

• Cache CDX responses and fetched pages to respect rate limits and reduce repeat queries
• Analyze a blend of recent, 1-3 year, and 5+ year snapshots to capture short- and long-term trends
• Request archived pages with a clear User-Agent and Accept headers to avoid accidental blocks
• Treat archive signals as contextual — corroborate findings with live analysis for high-confidence claims
• Gracefully handle API errors (503, timeouts) with exponential backoff and continue with available data

Example use cases

• Detecting a migration from WordPress to a React/Next.js stack and estimating when it occurred
• Finding when a site adopted CloudFront/Cloudflare headers to infer a cloud/CDN migration timeline
• Tracing removal of jQuery CDN references and appearance of SPA assets to confirm front-end rewrites
• Producing a technology timeline for a target to prioritize vulnerability checks on legacy components
• Correlating hosting changes (Heroku → Vercel) with observed configuration or header differences