Waifu Queue

MCP server for conversational AI using an LLM for text generation. 🗣️ Utilizes a Redis queue for asynchronous request processing and handling concurrent users. ⚙️ Built with the FastMCP library for simplified server setup and management. ⚡ Offers an MCP tool to enqueue generation jobs and an MCP resource to track job status. 👍📊

python

GitHub Stars

python

Language

4 months ago

First Indexed

3 weeks ago

Catalog Refreshed

Documentation & install

Readme and setup notes from the catalogue, plus a client-ready config you can copy for your MCP host.

Installation

Add the following to your MCP client configuration file.

Configuration

View docs

{
  "mcpServers": {
    "waifuai-mcp-waifu-queue": {
      "command": "uvicorn",
      "args": [
        "mcp_waifu_queue.main:app",
        "--reload",
        "--port",
        "8000"
      ],
      "env": {
        "REDIS_URL": "redis://localhost:6379",
        "GEMINI_API_KEY": "YOUR_GEMINI_API_KEY",
        "MAX_NEW_TOKENS": "2048",
        "OPENROUTER_API_KEY": "YOUR_OPENROUTER_API_KEY"
      }
    }
  }
}

You run an MCP server that queues and processes text-generation requests for a conversational character using a Redis-backed queue. It lets clients submit prompts, handles asynchronous processing, tracks job status, and returns results when ready, enabling scalable and responsive interactions with Gemini-based or OpenRouter-backed models.

How to use

Submit a text-generation request through the MCP client by calling the generate_text tool with your prompt. The system will enqueue the request, return a unique job ID, and process the prompt in the background. You can then poll or query the job resource to check status and retrieve the generated text once the job finishes.

How to install

# Prerequisites
- Python 3.7+
- Redis server (running locally or accessible remotely)
- Optional: API keys for the providers you plan to use

# 1) Create and activate a Python virtual environment
python -m uv venv .venv

# If you're on Windows, activate with:
.venv\Scripts\activate
# If you're on Unix/macOS, activate with:
source .venv/bin/activate

# 2) Install dependencies
pip install -r requirements.txt
pip install -r requirements-dev.txt

# 3) Start Redis (if not already running locally)
# Typical startup varies by system; on many setups:
redis-server

# 4) Run the worker (processes queued jobs)
python -m mcp_waifu_queue.worker

# 5) Start the MCP server (HTTP API)
uvicorn mcp_waifu_queue.main:app --reload --port 8000

Configuration and run notes

Configure the system via environment variables and local files to control provider choice, model selection, API keys, and Redis connectivity.

Provider selection can be OpenRouter (default) or Gemini. You can override with PROVIDER=openrouter or PROVIDER=gemini.

Security and keys

Store API keys securely and ensure access is restricted to your trusted environment. OpenRouter and Gemini keys can be loaded from environment variables or from local files as fallbacks.

Testing and maintenance

Run the test suite to validate behavior and ensure Redis connectivity is functioning. Regularly monitor worker and server logs for errors and adjust your Redis configuration and timeouts as needed.

Troubleshooting

If jobs remain queued, verify that the worker is running and connected to the same Redis instance configured in your environment. Check Redis connectivity and ensure API keys are valid if Gemini/OpenRouter calls fail.

Notes

The workflow uses a Redis-backed queue to decouple request submission from processing, enabling asynchronous execution of model calls and scalable handling of concurrent prompts.

Available tools

generate_text

Submits a text-generation request to the queued MCP workflow, triggering Gemini or OpenRouter via the background process.

job_resource

Allows retrieval of the status and result for a previously submitted job via the MCP resource URL

Built by

VeilStrat

AI signals for GTM teams