MedGemma Vertex

MCP Medical Imaging routes medical questions and images to MedGemma models hosted on Vertex AI, enabling you to get text-only or multimodal medical insights through a consistent MCP interface.

How to use

You connect your MCP client to MedGemma Vertex by pointing at the MCP HTTP endpoint or running the local MCP server during development. Use the text endpoint to get text-only answers from MedGemma-27B and the multimodal endpoint to process medical images alongside text with MedGemma-4B-IT. Submit your medical questions and, when needed, upload images, and you will receive structured responses that integrate model reasoning with the provided data.

Typical usage patterns include asking diagnostic questions, clarifying medical concepts, or requesting interpretation of medical images. Ensure you supply enough context for accurate results, and handle outputs with your own validation, especially for critical medical decisions.

How to install

Prerequisites you need on your workstation:

1. Install Node.js (version 14+ or as required by your project). Ensure npm is available after installation.

2. Install dependencies and start the server in development mode.

# 1. Install deps
npm install

# 2. Configure env vars (see below)
export VERTEX_PROJECT=your-gcp-project-id
export ENDPOINT_27B=1234567890123456789
export ENDPOINT_4B=9876543210987654321

# 3. Run in dev mode (stdio+HTTP on :4000)
npm run dev

Additional notes

You can connect a client like Claude Desktop by configuring a MedGemma server entry that points to the local MCP endpoint. Example host configuration to forward requests to the running MCP server is as follows.

{
  "id": "medgemma",
  "name": "MedGemma Vertex",
  "transport": "http",
  "url": "http://localhost:4000/mcp"
}

Available tools

answer_med_text

Text-only MedGemma response path for medical questions generated by MedGemma-27B.

answer_med_image

Multimodal MedGemma path that handles medical questions with accompanying images via MedGemma-4B-IT.