hybrid-search-implementation_skill

This skill implements hybrid search by combining vector and keyword results to boost recall in RAG systems and search engines.

Python

20

GitHub Stars

1

Bundled Files

2 months ago

Catalog Refreshed

4 months ago

First Indexed

Readme & install

Copy the install command, review bundled files from the catalogue, and read any extended description pulled from the listing source.

Installation

Preview and clipboard use veilstrat where the catalogue uses aiagentskills.

npx veilstrat add skill nilecui/skillsbase --skill hybrid-search-implementation

SKILL.md17.7 KB

Overview

This skill shows practical patterns and templates for combining vector similarity and keyword-based search to improve retrieval. It targets RAG systems, search engines, and applications where semantic and exact-match signals both matter. The focus is on fusion strategies, database and search-engine examples, and a pluggable pipeline for reranking.

How this skill works

The skill inspects results from separate vector and keyword searches, then fuses them using methods like Reciprocal Rank Fusion (RRF), linear score interpolation, or cross-encoder reranking. It includes concrete implementations for PostgreSQL (pgvector + full-text), Elasticsearch (dense_vector + BM25), and in-process fusion functions. The pipeline returns merged candidate lists, optionally reranks with a neural model, and supports filtering and weighting controls.

When to use it

Building retrieval-augmented generation (RAG) systems requiring high recall
When queries contain both semantic intent and exact tokens (names, codes, ids)
Improving search over domain-specific vocabulary where vector-only or text-only misses matches
When you need a tunable trade-off between precision, recall, and efficiency
Deploying search in environments with existing full-text or vector infrastructure

Best practices

Normalize and scale scores before linear fusion to avoid dominance by one source
Use RRF for robust combination without heavy parameter tuning
Apply a lightweight cascade (filter then rerank) to limit costly cross-encoder calls
Tune vector vs text weights on a validation set with representative queries
Index both embeddings and text with appropriate indexes (HNSW for vectors, GIN/tsvector or BM25 for text)

Example use cases

RAG knowledge retrieval that must return exact policy numbers and semantically related paragraphs
Enterprise document search combining legal phrase matching and semantic paraphrase retrieval
Customer support search that finds exact product codes and related troubleshooting steps
Academic literature search where keyword terms and concept similarity both matter
A search service that first filters by metadata then fuses vector and text rankings for final results

FAQ

Reciprocal Rank Fusion (RRF) is simple, robust, and requires minimal tuning—good default for most workloads.

When should I use a cross-encoder reranker?

Use a cross-encoder when you need the highest re-ranking quality and can afford CPU/GPU costs on a limited candidate set (e.g., 50 candidates).

How do I pick vector vs keyword weights?

Start with 0.5/0.5, then tune on a labeled validation set to maximize metrics like recall@k or MAP. Consider higher keyword weight when exact tokens matter.