Precision Desktop

Provides DPI-aware coordinate translation, UI element finding, and health checks to align Windows DPI scaling with desktop automation.

python

0

GitHub Stars

python

Language

4 months ago

First Indexed

2 months ago

Catalog Refreshed

Documentation & install

Readme and setup notes from the catalogue, plus a client-ready config you can copy for your MCP host.

Installation

Add the following to your MCP client configuration file.

Configuration

View docs

{
  "mcpServers": {
    "ikoskela-precision-desktop": {
      "command": "python",
      "args": [
        "C:/path/to/precision-desktop/server.py"
      ],
      "env": {
        "WINDOWS_MCP_PATH": "C:/path/to/windows-mcp"
      }
    }
  }
}

You will run precision-desktop as an MCP server that fixes DPI coordinate scaling for Windows desktop automation. It sits alongside your desktop automation MCP to calibrate and convert coordinates so your AI agents click, type, and hover at the correct positions on high-DPI displays.

How to use

Use precision-desktop alongside your existing MCP such as windows-mcp to provide calibrated, DPI-aware coordinate handling. You can calibrate on this machine, convert coordinates between physical and logical spaces, and locate UI elements by name for precise interaction. The server returns physical coordinates ready for clicking and integrates with patching to ensure your companion MCP applies the right DPI-aware changes.

How to install

Prerequisites: Python 3.10 or newer, PowerShell, and a working MCP environment. You will clone the repository, install dependencies, and run the server script.

git clone https://github.com/ikoskela/precision-desktop.git
cd precision-desktop
pip install -e .

Configuration and usage notes

Two MCP configurations are shown to run the server locally from Python. Use one or both as needed.

{
  "mcpServers": {
    "precision_desktop": {
      "command": "python",
      "args": ["C:/path/to/precision-desktop/server.py"]
    }
  }
}

{
  "mcpServers": {
    "precision_desktop": {
      "command": "python",
      "args": ["C:\\path\\to\\precision-desktop\\server.py"]
    }
  }
}

Calibration and ongoing use

Run the precision-desktop server and calibrate on first use. Calibration stores scale factors and offsets, enabling subsequent coordinate conversions. The server provides commands for calibration, verification, and health checks, as well as tools for UI element finding and coordinate conversion.

Security and environment considerations

If you run on Windows, keep the server in a trusted network and protect the calibration data. The calibration is user-specific and stored under state/calibration.json.

Troubleshooting tips

If coordinates land off by a consistent amount after scaling changes, re-run calibration to refresh scale factors and offsets. Ensure the companion MCP is reachable and that UI Automation is available on the target system.

Integration with windows-mcp

Precision-desktop describes patch intents that the companion MCP can apply, such as adding a coordinate_system parameter for automatic conversion or enabling element-based finding and clicking. Use patch_status to verify which patches are active.

Requirements

Windows 10/11 with Windows UI Automation, Python 3.10+, PowerShell, MCP SDK, and a tool to read cursor positions in both physical and logical coordinates for calibration.

License

MIT license.

Calibration and UI Automation details

Calibration computes independent scale factors for X and Y, validates consistency, and persists results to state/calibration.json. UI Automation searches return elements with physical coordinates when accessed from DPI-aware processes, enabling ready-to-use positions for clicking.

UI element finding capabilities

You can locate a single UI element by name within a window or search across a window to list all interactive elements. Results include center coordinates in physical space suitable for direct use with Click-Tool or Move-Tool.

Architecture overview

The server exposes tools for calibration, coordinate conversion, and UI element discovery, and it monitors patch status to ensure DPI-aware changes align with the companion MCP.

Available tools

calibrate

Compute DPI scale factors from 2+ reference points with known physical and logical coordinates.

calibrate_verify

Mark calibration as verified after confirming a test click landed correctly.

get_calibration

Read current calibration state including scale factors, verification status, and age.

convert_coordinates

Convert a coordinate pair between physical and logical coordinate systems.

find_ui_element

Find a single UI element by name using Windows UI Automation and return physical coordinates.

find_all_ui_elements

Find all UI elements matching a name and return their physical coordinates.

list_ui_elements

List all named interactive elements in a window.

find_window

Find a window handle by title substring.

health_check

Run environment checks for calibration freshness, UI Automation availability, and companion MCP status.

patch_status

Check which DPI-aware patches are applied to the companion MCP.