omi-backend-patterns_skill

Overview

This skill documents backend patterns for the Omi system, focused on conversation processing, memory extraction, chat routing, and integrations with LangGraph, Firestore, Pinecone, and Redis. It summarizes module hierarchy, API and database conventions, and the typical conversation-to-memory pipeline. Use it as a practical guide when implementing or extending backend features in Python/FastAPI services.

How this skill works

The skill explains the end-to-end conversation pipeline: audio arrives via WebSocket, transcription runs through supported providers, the system creates a conversation record, and processing is triggered by an API call or timeout. LLMs extract structured artifacts (title, action items, events, and categorized memories), which are stored in Firestore and indexed in Pinecone; Redis and GCS are used for caching and binary storage. Chat routing uses LangGraph to choose between direct LLM responses, agentic tool-enabled paths, or persona-driven responses.

When to use it

• Building or modifying backend endpoints under routers/
• Implementing conversation processing or transcription flows
• Adding memory extraction or LLM-based extractors
• Integrating vector search with Pinecone or semantic storage in Firestore
• Adding or extending LangGraph chat routing or tools

Best practices

• Follow strict import hierarchy: database/ → utils/ → routers/ → main.py; avoid upward imports.
• Keep routers thin; place business logic in utils/ and DB access in database/.
• Use dependency injection for auth and consistent error response shapes.
• Use process_conversation() as the canonical pipeline entry for conversation processing.
• Store structured LLM outputs in Firestore and index embeddings in Pinecone; use Redis for ephemeral caches.

Example use cases

• Add a new FastAPI endpoint that triggers conversation processing and returns a job ID.
• Enhance memory extraction to capture new categories such as medications or travel preferences.
• Register a new LangGraph tool and expose it through the agentic chat path for external data access.
• Implement webhook triggers after conversation processing to notify client apps or wearable devices.
• Migrate a binary storage flow to GCS for audio and profile assets while keeping metadata in Firestore.