tests-maintenance_skill

Overview

This skill maintains the IdeaVim test suite by reviewing disabled tests, ensuring Neovim exclusion annotations are documented, and improving test readability. It produces small, focused changes suitable for CI and PR review. Use it periodically to keep tests accurate and understandable without modifying production code.

How this skill works

It scans test sources for @Disabled, @TestWithoutNeovim, deprecated skip reasons, and meaningless test content. For a single selected test or method, it attempts to run the test (with and without Neovim when needed), re-enables passing tests, or updates annotations with specific reasons and descriptions. It also replaces placeholder text and improves test names or assertions to increase clarity.

When to use it

• Periodic cleanup of test suite to reduce technical debt
• After dependency or platform changes that may affect tests
• When seeing many deprecated or undocumented TestWithoutNeovim annotations
• Before a release to ensure tests are well-documented and runnable
• When CI shows flaky or disabled tests that may be re-enabled

Best practices

• Make one logical change per run: one test file or one test method only
• If updating multiple annotations, group only when they share the exact same skip reason and fix
• Always add a description parameter for non-obvious TestWithoutNeovim reasons
• Prefer replacing vague reasons (DIFFERENT/UNCLEAR) with a specific category and evidence
• Do not change production code or implement feature fixes in test-maintenance PRs

Example use cases

• Re-enable a test that now passes after a platform bug was fixed and document why it was disabled
• Add description to @TestWithoutNeovim(SCROLL) to explain viewport differences with Neovim
• Replace meaningless Lorem-like strings with realistic code snippets to make assertions clearer
• Split a test into two parts so only the Neovim-incompatible portion remains excluded
• Change a deprecated skip reason to INTELLIJ_PLATFORM_INHERITED_DIFFERENCE with a short explanation