AI Vision

You run an MCP server that analyzes images and videos using AI models from Google Gemini or Vertex AI. It operates as a modular, transport-agnostic service you can connect to with any MCP-compatible client to get rich media insights, annotations, and comparisons.

How to use

Connect to the MCP server from your MCP client by configuring an MCP server entry that runs locally and exposes the standard MCP transport. You can choose between a Google-based provider setup or a Vertex AI-based setup. The server supports four main analysis tools: image analysis, image comparison, object detection in images, and video analysis. Use these tools to describe content, compare visuals, identify objects with bounding boxes, or summarize video content.

Practical usage patterns include starting the MCP locally and pointing your client at the stdio transport. You provide the provider choice through environment variables, and your client then sends requests for image or video analysis. For production, prefer the Vertex AI provider with proper credentials and a Google Cloud Storage bucket. For quick experiments, the Google provider with an API key works well.

When you start a local MCP server, you run a command that launches the MCP runner with the ai-vision-mcp package. You can configure environment variables to select the image and video providers and to supply credentials as shown in the examples.

How to install

Prerequisites: you need Node.js 18 or newer and a standard Node package manager (npm or yarn). You will install dependencies, build, then run the server in development mode.

# Prerequisites
node -v
npm -v

# Install the MCP server project dependencies
npm install

# Build the TypeScript project
npm run build

# Start the development server (watch mode)
npm run dev

Provider configuration and startup guidance

Choose a provider and set the required environment variables before starting the MCP server. The Google provider uses an API key, while Vertex AI uses service credentials and a GCS bucket.

Google AI Studio Provider (recommended for quick starts) set these variables before starting the MCP server:

export IMAGE_PROVIDER="google"
export VIDEO_PROVIDER="google"
export GEMINI_API_KEY="your-gemini-api-key"

Vertex AI Provider (production) set these variables before starting the MCP server:

export IMAGE_PROVIDER="vertex_ai"
export VIDEO_PROVIDER="vertex_ai"
export VERTEX_CREDENTIALS="/path/to/service-account.json"
export GCS_BUCKET_NAME="your-gcs-bucket"

Available tools

analyze_image

Analyzes an image and returns a detailed description based on a prompt and image source (URL, base64, or local file).

compare_images

Compares multiple images and returns a detailed analysis highlighting differences or similarities based on a prompt.

detect_objects_in_image

Detects objects in an image, produces bounding boxes, and returns detected objects with coordinates and an annotated image if requested.

analyze_video

Analyzes a video source (YouTube URL or local file) and returns a detailed description based on a prompt.