Local RAG Backend

You run a local RAG back-end that ingests documents, stores them, and provides MCP-based search across memory, graph, and ranking components. This server handles a wide range of file formats, unifies document registration, and exposes search tools via MCP so you can build powerful, context-aware retrieval flows in your applications.

How to use

Connect your MCP client to the local MCP server to perform searches, memory operations, and episode management. Use the included MCP tool endpoints to run RAG-specific searches, fact queries, and to fetch or add memory episodes. You can register documents, then query across vector embeddings, graph relations, and full-text results with re-ranking based on relationships.

How to install

Prerequisites: ensure you have Docker and Docker Compose available on your machine. You will also need a way to fetch files from the internet to place configuration and initial data.

mkdir -p path/to/RAG/data/input/
cd path/to/RAG/

curl -Lo docker-compose.yml https://raw.githubusercontent.com/suwa-sh/local-RAG-backend/refs/heads/main/docker-compose.yml
curl -Lo .env https://raw.githubusercontent.com/suwa-sh/local-RAG-backend/refs/heads/main/.env.example

# Edit environment variables to suit your setup
vi .env

docker compose up -d

Document registration and ingestion can be run after you place documents in the input directory. The ingest process will pick up files and feed them into the graph and vector stores for search.

Additional information

Narrow MCP connections are shown for client access. One example MCP connection uses a local Graphiti memory server over HTTP for streaming and search results.

{
  "mcpServers": {
    "graphiti_memory": {
      "type": "http",
      "url": "http://localhost:8000/sse",
      "name": "graphiti_memory",
      "args": []
    }
  }
}

Configuration and environment

The server uses a .env file to configure database connections, LLM endpoints, and embedding models. You can customize the following example values as needed.

# Neo4j database
NEO4J_URI=bolt://localhost:7687
NEO4J_USER=neo4j
NEO4J_PASSWORD=password

# LLM model endpoints
LLM_MODEL_URL=https://api.openai.com/v1
LLM_MODEL_NAME=gpt-4o-mini
LLM_MODEL_KEY=YOUR_OPENAI_API_KEY

# Rerank model
RERANK_MODEL_NAME=gpt-4.1-nano

# Embedding model
EMBEDDING_MODEL_URL=http://host.docker.internal:11434/v1
EMBEDDING_MODEL_NAME=kun432/cl-nagoya-ruri-large:latest
EMBEDDING_MODEL_KEY=YOUR_EMBEDDING_API_KEY

# Tenant identifier
GROUP_ID=default

# Optional chunking settings
CHUNK_SIZE_MAX=2000
CHUNK_SIZE_MIN=200
CHUNK_OVERLAP=0

Troubleshooting and notes

If you encounter issues with memory search, ensure the MCP transport for the graphiti-memory server is reachable and that the HTTP endpoint is available. Logs for ingestion and search are written under data/logs in the project directory, and you can tail them to diagnose problems.

Available tools

search_for_rag

RAG-side search tool that queries vector search, graph search, and full text with re-ranking to produce the most contextually relevant results.

search_memory_facts

Fact-oriented search that retrieves specific factual snippets from memory, useful for verification.

search_memory_nodes

Node-level memory search to locate conceptually related nodes within the knowledge graph.

get_episodes

Retrieve recent memory episodes to understand how concepts have evolved over time.

add_memory

Add new memory with an episode body, associating it with a group and source.