Voice-AGI

You can run a stateful, voice-controlled AGI system locally by hosting a dedicated MCP server that handles multi-turn conversations, voice-driven tool calls, and local STT/TTS processing. This setup keeps your data private, reduces API costs, and provides rich, context-aware interactions with built-in performance metrics.

How to use

To use this MCP server, run it locally and connect from your MCP client or integration driver. You will interact through voice commands that are transcribed by a local STT pipeline, interpreted by an intent detector, and routed to a set of voice-callable tools. The system remembers previous turns to maintain context, executes appropriate tools, and replies with synthesized speech.

How to install

Prerequisites are required to run this MCP server locally.

Step 1: Install dependencies and prepare the environment.

cd /mnt/agentic-system/mcp-servers/voice-agi-mcp
pip install -r requirements.txt

Configuration

Configure the MCP server by registering it with your Claude Code workspace. Add the following MCP server entry to the Claude Code configuration so it can launch the Voice-AGI MCP server as a local process.

{
  "mcpServers": {
    "voice-agi": {
      "command": "python3",
      "args": ["/mnt/agentic-system/mcp-servers/voice-agi-mcp/src/server.py"],
      "disabled": false
    }
  }
}

Environment and runtime details

The server relies on several local components and environment variables. Typical configuration includes: Ollama URL and model, Whisper model size, Edge TTS voice, latency tracking, and URLs to local MCP services for enhanced memory, agent runtime, and AGI orchestrator.

Global environment variables you’ll configure in your shell or deployment environment include:

Troubleshooting

If you encounter issues, verify components individually and check logs for the voice-agi channel.

# Check logs for, e.g., voice-agi
journalctl -f | grep voice-agi

Notes and tips

This MCP server uses local processing for STT/TTS and uses an Ollama model for intent detection. It supports memory enhancements and a suite of tools you can extend over time. For best latency, balance model sizes and enable latency tracking to identify bottlenecks.

Performance overview

Latency and usage stats are tracked by the system to help you optimize the setup. Typical measurements include STT latency, TTS latency, total round-trip time, and conversation statistics like turns and words per session.

Available tools

search_agi_memory

Search past memories via voice to retrieve relevant context or data.

create_goal_from_voice

Create a new goal based on spoken input to guide memory optimization or tasks.

list_pending_tasks

List current tasks that are pending or in progress.

trigger_consolidation

Run memory consolidation procedures to organize and summarize memory data.

start_research

Initiate autonomous research on a specified topic.

check_system_status

Report on the current health and status of the MCP system.

remember_name

Store and recall the user's name for personalized interactions.

recall_name

Retrieve the stored user name.

start_improvement_cycle

Begin a cycle to improve system consolidation speed or performance.

decompose_goal

Break a high-level goal into actionable tasks.