Wayland

Provides screen capture, VLM analysis, and input automation via the Model Context Protocol for Wayland desktops.

python

GitHub Stars

python

Language

6 months ago

First Indexed

3 months ago

Catalog Refreshed

Documentation & install

Readme and setup notes from the catalogue, plus a client-ready config you can copy for your MCP host.

Installation

Add the following to your MCP client configuration file.

Configuration

View docs

{
  "mcpServers": {
    "kurojs-wayland-mcp": {
      "command": "uvx",
      "args": [
        "wayland-mcp"
      ],
      "env": {
        "VLM_MODEL": "qwen/qwen2.5-vl-72b-instruct:free",
        "VLM_PROVIDER": "openrouter",
        "GEMINI_API_KEY": "AIza...",
        "WAYLAND_DISPLAY": "wayland-0",
        "XDG_RUNTIME_DIR": "/run/user/1000",
        "WAYLAND_MCP_PORT": "4999",
        "OPENROUTER_API_KEY": "YOUR_OPENROUTER_API_KEY"
      }
    }
  }
}

You can use the Wayland MCP Server to let AI assistants interact with your Wayland desktop. It provides screen capture with analysis, mouse and keyboard control, and the ability to chain actions, enabling automated workflows on modern Linux desktops.

How to use

You install the server and connect with an MCP client. From the client, request actions such as taking a screenshot, analyzing what’s on screen, moving the mouse, clicking, typing text, or running a sequence of actions. Create workflows by chaining actions in a single string, for example moving the cursor, clicking, typing, and sending a key press in one go. The server interprets these requests, performs the corresponding desktop actions, and returns results such as screenshots, analysis results, or execution status.

How to install

Prerequisites you need before installation:

Python 3.8 or higher
A Wayland compositor (GNOME, KDE Plasma, Hyprland, Sway, or other wlroots-based environments)
grim and slurp for screenshots (usually pre-installed)

Install the MCP server using the quick install command:

uvx wayland-mcp

From source

If you prefer building from source, clone the project repository, install dependencies, and install the package in editable mode.

git clone https://github.com/kurojs/wayland-mcp.git
cd wayland-mcp
pip install -e .

Input control setup

To enable mouse and keyboard automation, run the setup script. This prepares device access and permissions so the server can control input devices.

sudo ./setup.sh

Post-setup note

After running the setup script, log out and back in to apply group membership changes.

Available tools

capture_screenshot

Take a screenshot with optional overlays for measurement and later analysis.

capture_and_analyze

Capture the screen and run visual analysis in one step.

analyze_screenshot

Analyze a provided screenshot with a custom prompt or model.

compare_images

Compare two screenshots to detect visual differences.

move_mouse

Move the cursor to absolute or relative coordinates.

click_mouse

Click the mouse button at the current position.

drag_mouse

Drag from one coordinate to another with precision.

scroll_mouse

Scroll vertically or horizontally.

type

Type a text string into focused input.

press

Press a specific key or key combination.

execute_action

Execute a single action or a chain of actions.

Built by

VeilStrat

AI signals for GTM teams