msmps/pilotty

Overview

This skill automates terminal TUI applications by managing PTY sessions so an AI agent or script can interact with programs like vim, htop, lazygit, and dialog. It runs a background daemon, spawns isolated sessions, captures screen snapshots with detected UI elements, and sends keyboard/mouse input to automate interactive CLI workflows. Use it to reliably edit files, navigate menus, toggle checkboxes, and drive interactive prompts programmatically.

How this skill works

pilotty spawns a command inside a managed PTY session (named or default) and exposes commands to wait for text, take structured snapshots, and send keys, typed text, clicks, scrolls, and resize events. Snapshots include parsed elements (buttons, inputs, toggles) and a content_hash for change detection. The recommended pattern is: spawn -> wait-for -> snapshot -> interpret elements -> navigate with keys -> re-snapshot to verify changes.

When to use it

• Automating edits in terminal editors (vim, nano) from scripts or agents.
• Driving interactive installers, dialog checklists, or form-style TUIs.
• Controlling monitoring tools (htop) or other full-screen terminal apps.
• Testing and verifying TUI behavior by comparing snapshot content_hash.
• Running multiple isolated terminal sessions shared across CLI calls.

Best practices

• Always put pilotty flags (e.g., --name, -s, --format) before the command and its arguments.
• Use pilotty wait-for to ensure an app is ready before interacting.
• Treat snapshot elements as read-only context; perform actions with keyboard commands.
• Use content_hash to detect screen changes and re-analyze only when it changes.
• Name sessions for multi-app workflows and target them with -s / --session.

Example use cases

• Edit /tmp/notes.txt in vim: spawn named editor, wait-for filename, send i/type/Escape/:wq, Enter, then verify sessions.
• Fill a curses form: spawn form, snapshot to find inputs, tab/type to fields, space to toggle checkboxes, Enter to submit.
• Interact with dialog checklist: spawn dialog with --name opts, snapshot toggles, navigate with Down/Space, Enter to confirm.
• Automated monitoring: spawn htop, wait-for CPU text, snapshot for layout, send keys to open kill menu or quit.
• End-to-end CLI workflow testing: spawn app, wait-for prompts, type responses, capture snapshots for CI assertions.