interact-with-browser_skill

Overview

This skill provides a CLI for interacting with a headless or headed browser to automate browsing, inspect accessibility trees, and run UI actions. It exposes deterministic refs, traditional selectors, semantic locators, and a rich set of commands for navigation, input, network control, and session management. The tool is optimized for AI-driven workflows by offering JSON output and snapshots that include stable refs for reliable automation. It also supports sessions, custom executables, and serverless usage patterns.

How this skill works

You drive the browser with simple CLI commands like open, click, fill, snapshot, and screenshot. Use snapshot to obtain an accessibility tree with refs; then reference elements by ref (e.g., @e2) for deterministic actions. For dynamic selection you can also use CSS, text, XPath, or semantic locators (find role/label/etc.). Many commands support --json for machine-readable output and options to scope, filter, or limit snapshot output.

When to use it

• Automating UI flows and end-to-end tests from the command line or CI
• Feeding deterministic element refs into an LLM-based agent workflow
• Taking screenshots or PDFs of pages for visual verification or reporting
• Intercepting or mocking network requests during API/UI tests
• Managing multiple isolated sessions for parallel testing or multi-user scenarios

Best practices

• Prefer snapshot + refs for AI-driven automation to avoid brittle selectors
• Use snapshot -i (interactive) or -c (compact) to reduce output size when parsing
• Scope actions with --session to isolate cookies, storage, and auth state
• Use --json when integrating with programs or agents to parse responses reliably
• Clean up screenshots, traces, and browser instances (close) after runs

Example use cases

• Run an agent workflow: open a page, snapshot, identify refs, then click and fill using refs
• Automated login: set headers for auth or load saved state, then navigate protected pages
• Visual testing: capture full-page screenshots or PDFs and compare diffs in CI
• Network testing: route or abort requests to simulate failures and validate fallbacks
• Serverless scraping or testing: run a lightweight Chromium executable in AWS Lambda or Vercel