windows-ui-automation_skill

Overview

This skill is an expert Windows UI Automation (UIA) and Win32 API agent for safe, accessible desktop automation. It focuses on secure element discovery, validated input simulation, and strict process interaction controls. It is a HIGH-RISK capability and enforces permission tiers, blocklists, and audit logging to reduce abuse.

How this skill works

The skill inspects UI Automation elements via UIA COM interfaces and integrates Win32 calls for window management and input. It validates target processes, enforces permission tiers (read-only, standard, elevated), applies timeouts, and records audit trails before performing actions. Blocked applications and sensitive key combinations are prevented, and process integrity checks are used where available.

When to use it

• Automating desktop workflows where native UIA patterns are needed (buttons, text fields, lists).
• Interacting with legacy or custom Windows applications that lack web APIs.
• Building accessible automation compatible with screen readers and assistive tech.
• Performing controlled input simulation for testing or macro tasks under consented permissions.
• Validating automation security posture and blocking high-risk targets before action.

Best practices

• Always validate process identity and integrity before any interaction.
• Use the minimal permission tier required; avoid elevated tier unless approved.
• Maintain and update a blocked-process list for password managers and admin tools.
• Enforce operation timeouts and rate limits to prevent hangs and input flooding.
• Log every operation with a correlation ID for audit and incident tracing.

Example use cases

• Find and read UI element properties in a target app under read-only permission.
• Send controlled keystrokes to an application after process validation and focus checks (standard tier).
• Automate accessibility testing to verify screen reader compatibility and focus behavior.
• Validate executable integrity and owner before automating interactions with critical apps.
• Run scheduled UI checks that report element existence and state without simulating input.