alexgladkov/claude-in-mobile

Overview

This skill provides a fast CLI and automation server for interacting with mobile devices and desktop apps across Android (ADB), iOS Simulator (simctl), Aurora OS (audb), and Desktop (companion JSON-RPC). Use it to capture and analyze screens, perform UI actions (tap, swipe, type), manage apps, transfer files, query device state, run shell commands, and control desktop windows. It’s designed for test automation, debugging, and remote device management from scripts or CI pipelines.

How this skill works

The CLI communicates with platform backends (ADB, simctl, audb, or a desktop companion) to execute device-level operations and return structured output. Commands can capture screenshots (base64 or file), dump UI hierarchies, identify interactive elements, and perform gestures or key events. Platform selectors and companion path flags let you target specific devices, simulators, or desktop instances.

When to use it

• Automating UI flows: tapping, swiping, typing, long-pressing across Android/iOS/Aurora/desktop.
• Capturing or annotating screenshots for visual debugging or sending images to an LLM.
• Inspecting the UI tree or analyzing screen elements to build robust selectors for tests.
• Installing, launching, stopping, or uninstalling apps during CI or device provisioning.
• Transferring files and accessing device logs, clipboard, or system info for troubleshooting.

Best practices

• Specify the device/simulator/companion explicitly with --device, --simulator, or --companion-path to avoid ambiguity.
• Use compressed screenshots (--compress, --max-width, --quality) when sending images to LLMs or over slow links.
• Prefer semantic taps (by text or resource-id) instead of raw coordinates for resilient scripts.
• Throttle automation flows with wait between actions to account for animations and network delays.
• Capture ui-dump before performing actions to verify element bounds and avoid flaky taps.

Example use cases

• Run end-to-end UI tests: install app, launch, navigate UI via tap-text/find, validate screen with analyze-screen.
• Remote debugging: pull device logs, capture annotated screenshots, and inspect current-activity and system-info.
• Automated repro: push a test file, open it in the app, record steps via screenshot/annotate, and reproduce via tap/swipe.
• Desktop automation: list windows, focus and resize windows, launch or stop desktop apps via the companion.
• CI device provisioning: batch install/uninstall apps, reboot devices, and verify screen sizes and performance metrics.