Sensor

SensorMCP Server enables automated dataset creation and custom object detection model training through natural language interactions. It bridges large language models with computer vision workflows, letting you define object classes, automatically label images, fetch training data, and train specialized detectors in a reproducible, MCP-enabled setup.

How to use

Connect your MCP-enabled language model or chat interface to the SensorMCP Server. Use natural language to define what you want to detect, prepare your labeling and training pipelines, and run training end-to-end without leaving your chat context.

How to install

Prerequisites you need before installing the server:

• uv for package management
• Python 3.13+
• CUDA-enabled GPU recommended for training

Step-by-step installation flow you should follow from a clean environment:

git clone <repository-url>
cd sensor-mcp
uv sync

Create an environment file to provide credentials for image sources (example key shown; replace with your actual key):

UNSPLASH_API_KEY=your_unsplash_api_key_here

You are now ready to start either the MCP integration server or the standalone web server.

Configuration and running modes

SensorMCP Server can run in MCP integration mode or as a standalone web server. Use the following commands to start each mode.

uv run src/zoo_mcp.py

uv run src/server.py

If you want your MCP client to auto-load the server, add this configuration to your MCP client settings:

{
  "mcpServers": {
    "sensor_mcp": {
      "type": "stdio",
      "command": "uv",
      "args": [
        "--directory",
        "/path/to/sensor-mcp",
        "run",
        "src/zoo_mcp.py"
      ]
    }
  }
}

About the project structure and data flow

SensorMCP Server creates a fixed directory structure to keep data organized and reproducible. You will have raw images, auto-labeled image sets, and trained model weights stored under the sensor-mcp/data directory, with subfolders for raw_images, labeled_images, and models.

Security and credentials

Only supply credentials that are required for your data sources. Store sensitive keys in a secure environment file and do not commit them to version control.

Project notes

The server supports labeling with a foundation model, defining custom object ontologies through natural language, downloading training images from Unsplash, and training YOLOv8 variants for your objects.

Available tools

list_available_models

List all supported base models for labeling and all target models available for training, so you can pick the right combination for your workflow.

define_ontology

Define the object classes you want to detect using natural language input, which will be translated into a structured ontology used by labeling and training processes.

set_base_model

Initialize the foundation model used for automatic labeling of images (for example GroundedSAM).

set_target_model

Initialize the target training model (for example various YOLOv8 variants) to train on your defined ontology.

fetch_unsplash_images

Download training images from Unsplash based on your query and the maximum number of images you want to acquire.

import_images_from_folder

Import locally stored images into the training dataset for labeling or training.

label_images

Auto-label the image collection using the base model to generate an initial training dataset.

train_model

Train a custom object detector with the defined ontology using the selected target model and specified epochs on the chosen device.