Shepherd

You run a Model Context Protocol (MCP) server to let AI assistants query and analyze your Shepherd agent sessions from multiple observability providers. This server bridges your AI workflows with observability data so you can debug, compare, and understand agent behavior across providers like AIOBS and Langfuse.

How to use

You use an MCP client to connect to the Shepherd MCP server, either by running the server locally via a containerized command or by pointing a client at a remote MCP endpoint. Once connected, you can list sessions, inspect details, compare sessions, and explore data from multiple providers in a unified view. Start the MCP server with the command that matches how you prefer to run it, then configure your client to access the server and supply the necessary API keys.

How to install

Prerequisites: you need Python installed (3.8+ is recommended) and a working environment to install Python packages. You also need access keys for AIOBS and Langfuse if you plan to use those providers.

Install the MCP package from PyPI and install optional runtime support via uvx to run locally.

pip install shepherd-mcp

# Or run directly with uvx if you have it installed
uvx shepherd-mcp

Configuration and running locally

Before you start, set the required credentials for AIOBS and Langfuse. You can provide these as environment variables or via a .env file in your project.

Run the MCP server using either the uvx-based command or the Python module approach.

# Option 1: run with uvx (preferred for quick local testing)
uvx shepherd-mcp

# Option 2: run as a Python module
export AIOBS_API_KEY=aiobs_sk_xxxx
export LANGFUSE_PUBLIC_KEY=pk-lf-xxxx
export LANGFUSE_SECRET_KEY=sk-lf-xxxx
python -m shepherd_mcp

Configuration details

The MCP server can read configuration from environment variables for the two providers.

AIOBS (Shepherd) environment variables include the API key and an optional endpoint.

Security and credentials

Keep your API keys secret. Do not share keys in version-controlled files. Use environment-variable-based configuration or secure secret management for production.

Notes and troubleshooting

If you run into connectivity issues, verify that your keys are correct and that the MCP server is running. Check your environment to ensure the keys for AIOBS and Langfuse are accessible to the MCP process.

Examples of usage patterns

# Example: list all AIOBS sessions using a client
# Client command syntax will vary by client; this shows the intended action
List my AI agent sessions from AIOBS

# Example: compare two AIOBS sessions
Compare AIOBS sessions abc123 and def456

# Example: fetch a Langfuse trace and its observations
Get Langfuse trace details for trace-id-123

Additional configuration and environment details

The server supports loading configuration from a .env file located in your project directory or any parent directory. This makes it convenient to manage keys for AIOBS and Langfuse without embedding them in code or scripts.

Available tools

aiobs_list_sessions

List all AI agent sessions from Shepherd. Optional limit to constrain the number of results.

aiobs_get_session

Retrieve detailed information about a specific session, including the full trace, LLM calls, and evaluations.

aiobs_search_sessions

Search and filter sessions using multiple criteria such as provider, model, labels, and time range.

aiobs_diff_sessions

Compare two sessions and show differences across metadata, calls, traces, and evaluations.

langfuse_list_traces

List traces with pagination and filters to explore complete workflows.

langfuse_get_trace

Fetch a specific trace with its observations, generations, spans, and events.

langfuse_list_sessions

List sessions with pagination to group related traces.

langfuse_get_session

Get a specific Langfuse session with metrics and traces.

langfuse_list_observations

List observations such as generations, spans, and events with filters.

langfuse_get_observation

Get a specific observation with full details including input, output, usage, and costs.

langfuse_list_scores

List scores/evaluations with filters to review model/performance assessments.

langfuse_get_score

Fetch a specific score/evaluation with full details.