Perception

Perception-MCP is a lightweight Model Context Protocol server that lets you ask questions about image, audio, or video files and returns answers powered by multimodal models served through fal.ai. It enables you to plug multimodal querying into your workflows and client tools with a simple runtime.

How to use

To use Perception-MCP with your MCP client, run the local stdio server and connect your client to its standard input/output. You can integrate the server by configuring your MCP client to invoke the provided runtime as shown in the example configuration. The server exposes three capabilities you can ask about: images, audio, and video.

You can configure your client to start the MCP via a rapid runtime command and wire it into your desktop or automation workflow. The typical pattern is to start the server process and then reference it by name in your client’s MCP server mappings. When you query, you specify the topic or question (for example, “What objects are in this image?”) and the server responds with a precise answer generated by the multimodal models.

Example client integration snippet you may use in your client configuration (this shows how a local MCP server is wired into the client’s MCP server registry). The exact fields may vary by client, but the concept is the same: point the client at the server’s stdio interface and provide a command path to start it.

{
  "mcpServers": {
    "perception_mcp": {
      "command": "fastmcp",
      "args": ["run", "perception-mcp", "serve"]
    }
  }
}

How to install

Install Perception-MCP by cloning the repository, then set up its secret configuration file. Ensure you have Python 3.11+ and the uv tool installed, and that you have accounts and API keys for fal.ai and Perplexity as required by the server.

Run these commands to install locally:

git clone --recurse-submodules https://github.com/lintyourcode/perception-mcp.git
cd perception-mcp
cp mcp_agent.secrets_template.yaml mcp_agent.secrets.yaml
$EDITOR mcp_agent.secrets.yaml

Additional notes

Prerequisites include Python 3.11 or newer, the uv tool for running tests, and accounts with API keys for fal.ai and Perplexity. You will also need a client capable of interfacing with an MCP server and wiring the runtime into your workflow.

You can add Perception-MCP to your development environment by starting the server with the provided runtime and then registering it in your MCP client’s server map. The server is designed to respond to queries about image, audio, and video content using state-of-the-art multimodal models accessed through fal.ai.

Available tools

query_image

Ask questions about the contents of an image and receive descriptive, model-driven answers.

query_audio

Ask questions about an audio file to get understanding of its content, transcripts, or features.

query_video

Ask questions about a video to obtain information about scenes, objects, or events present.