Computer Use

Controls virtual computers via MCP to manage cloud desktops for Computer Use Agents

python

6

GitHub Stars

python

Language

6 months ago

First Indexed

2 months ago

Catalog Refreshed

Documentation & install

Readme and setup notes from the catalogue, plus a client-ready config you can copy for your MCP host.

Installation

Add the following to your MCP client configuration file.

Configuration

View docs

{
  "mcpServers": {
    "spencerkinney-computer-use-mcp": {
      "command": "python",
      "args": [
        "computer_mcp_server.py"
      ],
      "env": {
        "ORG O?": "PLACEHOLDER_ORGO_KEY",
        "ANTHROPIC_API_KEY": "PLACEHOLDER_ANTHROPIC_KEY"
      }
    }
  }
}

You can manage virtual computers and cloud desktop environments through the MCP (Model Context Protocol) using a local Python-based MCP server. This setup lets you initialize a virtual computer, capture screenshots, simulate mouse and keyboard actions, run commands, and manage the computer’s lifecycle from your code or a client that supports MCP.

How to use

Connect with an MCP client and perform common actions to control a virtual computer. You can initialize the computer, take screenshots for visual feedback, simulate mouse clicks and keyboard input, wait for operations, run terminal commands, and manage the computer’s lifecycle. If you integrate Claude AI, you can provide natural language instructions to automate tasks on the computer.

How to install

Prerequisites: ensure you have Python 3.8 or higher installed on your system.

Create a virtual environment and activate it.

python3 -m venv mcp-env
source mcp-env/bin/activate

Install the required Python packages.

pip install fastmcp pydantic orgo

Set up your API keys for Orgo and Claude (Anthropic). You will provide these keys as environment variables when running the server.

export ORGO_API_KEY="your_orgo_key"
export ANTHROPIC_API_KEY="your_anthropic_key"

Run the MCP server locally. The server starts with the Python command shown here.

python computer_mcp_server.py

Additional notes

The server is intended for local development and experimentation. It exposes capabilities to initialize a computer, capture screenshots, simulate many input events, run shell commands, and manage the computer lifecycle. Ensure your environment variables are kept secure and do not expose API keys in public code.

Security and troubleshooting

Keep API keys secure. Do not hard-code keys in public repositories. Use environment variables or secret management for production use.
If the server fails to start, verify that Python 3.8+ is installed, the virtual environment is activated, and the environment variables ORGO_API_KEY and ANTHROPIC_API_KEY are set before launching the server.

Available tools

initialize_computer

Start virtual computer and prepare the environment for use.

get_screenshot

Capture the current screen image of the virtual computer.

left_click

Simulate a left mouse click at given coordinates.

right_click

Simulate a right mouse click at given coordinates.

double_click

Simulate a double-click at given coordinates.

scroll

Scroll the virtual screen up or down.

type_text

Type text into the active input field on the computer.

press_key

Simulate a single key press on the keyboard.

wait

Pause execution for a specified number of seconds.

execute_bash

Run terminal commands on the virtual computer.

restart_computer

Restart the virtual computer.

shutdown_computer

Shut down the virtual computer.

get_status

Query current status of the computer and sessions.

list_sessions

List active sessions on the virtual computer.

prompt

Claude AI Computer Use: provide natural language instructions for automation.