deflake-e2e_skill

Overview

This skill identifies and fixes flaky end-to-end (E2E) tests by running them repeatedly, inspecting failures, and applying Playwright best practices to stabilize tests. It guides you through running tests 10 times, diagnosing intermittent failures, applying test-side fixes, updating snapshots when appropriate, and verifying the fixes.

How this skill works

The skill runs specified E2E test files (or prompts to run the entire suite) 10 times with automatic retries disabled to expose intermittent failures. For any test that fails at least once, it enables browser debug logs to surface timing, animation, network, or state-leak issues, then recommends and applies test-level fixes and snapshot updates. Finally, it re-runs the tests to confirm stability.

When to use it

• You see intermittent E2E failures in CI or locally that pass on retry.
• You want to validate whether a failing test is genuinely broken or flaky.
• You need a reproducible process to stabilize visual or timing-sensitive tests.
• You maintain a test suite where tests share global state or cause flakiness.
• You need to update failing snapshot baselines after intentional UI changes.

Best practices

• Always run npm install and npm run build before E2E runs; E2E tests exercise the built binary.
• Disable Playwright automatic retries (PLAYWRIGHT_RETRIES=0) so flakiness is visible.
• Run each candidate test 10 times (--repeat-each=10) to classify flaky tests reliably.
• Prefer stable selectors (data-testid, role, text) and explicit expect checks (toBeVisible) before interactions.
• Use page.waitForLoadState('networkidle') and minimal explicit waits only when necessary; isolate tests and reset state.
• Do not change application code—limit changes to test files and snapshot baselines.

Example use cases

• Investigating a UI test that occasionally times out in CI but passes locally.
• Stabilizing a visual snapshot test that shows minor pixel diffs across runs.
• Finding race conditions where an element is interacted with before it becomes ready.
• Confirming a suspected flaky test no longer fails after applying waits or selector fixes.
• Documenting which tests were flaky and summarizing fixes for team review.