Semantic search, keyword search, and hybrid retrieval

Watch the walkthrough

1 · The smallest useful model

• Vector search (Semantic): Converts text into floating-point embeddings (e.g., Qdrant, Pinecone) to capture high-dimensional similarity, context, meaning, and paraphrases rather than exact words. How well it holds up across languages depends on the embedding model — do not assume, measure it.
• Full-text search (Lexical): Matches literal token strings, exact identifiers, names, acronyms, and numbers using algorithms like BM25 and typo tolerance (e.g., Meilisearch, Elasticsearch). When exact spelling carries the meaning, this is what saves you. Note: Engine boundaries are evolving — engines like Meilisearch can also handle vector embeddings natively for unified hybrid search.
• Graph retrieval (Relational / Structural): Navigates explicit nodes, edges, and entity relationships dynamically (e.g., Neo4j, LightRAG). It isn't just "another way to find a chunk" — it is the single structural signal capable of multi-hop connection tracing (e.g., connecting facts spread across different documents), which vector and full-text search cannot do reliably on their own.
• Summary retrieval: A separate index holding one embedded summary per whole document, not per chunk. It's built for "what is this document about," when a single ~500-token chunk is too narrow a unit to represent the whole thing.
• Cumulative union: For this experiment, a source counts as found the moment it turns up in any of the four signals. That tells you about retrieval contribution — nothing about whether the eventual answer is correct.
• Hybrid retrieval: In this system, "hybrid" means running all signals in parallel and concatenating their formatted results into the prompt — not fusing or reranking them. There's no score normalization or reciprocal rank fusion across signals; the only real fusion happens inside LightRAG itself, across its own entity, relationship, and vector passes. A rerank engine exists in the codebase but isn't wired into live retrieval — don't assume "hybrid" implies more sophistication than parallel-and-concatenate.

Primary reading: Qdrant Hybrid Search — pay attention to how it combines multiple search representations, and why that combination needs to be evaluated rather than just assumed to work.

2 · Run one controlled experiment

Reuse the same isolated demo and evaluation set from Lesson 4. Retrieval mode hits /api/v1/search directly with no LLM judge in the loop, which is what keeps this a clean read on retrieval by itself.

cd demo
python3 eval.py verify
python3 eval.py run --mode retrieval --workspace meridian_demo --tag lesson-05-experiment

This gives you two files: an aggregate report, and a per-question JSON report.

reports/lesson-05-experiment.md
reports/lesson-05-experiment.json

For each question you are checking, look at:

• results[].retrieval.indexes.vector.all
• results[].retrieval.indexes.fulltext.all
• results[].retrieval.indexes.summary.all
• results[].retrieval.indexes.graph.all
• results[].retrieval.cumulative["vector+fulltext"].all
• results[].retrieval.cumulative["vector+fulltext+summary"].all
• results[].retrieval.cumulative["vector+fulltext+summary+graph"].all

Judge a hit by source metadata, not by text matching — a document ID that merely shows up inside another document’s body text does not count as a retrieval hit.

The graph column is approximate, and the report says so: LightRAG returns one flat context string rather than a list of documents, so a reference counts as a graph hit if it appears anywhere in that string — which could come from a cross-reference inside another document, not the target document itself. Treat it as a real but noisier signal. It's reported separately and never silently folded into the other three.

3 · Fill the comparison table

Fill the five result columns with all, partial, or miss — use partial when a multi-document question only turns up some of its expected sources. Then, for each row, add one line of interpretation and one specific next step.

4 · Interpret before changing code

5 · Deliverable and next action

1. Fill in all five rows of the comparison table.
2. Write one paragraph: which signal added a source, for which type of query, and what's the evidence for that?
3. Pick one next action, and name the metric you'll rerun to check it worked.

Next: Lesson 6 — Chunking and document structure. Hang onto this table — it's your retrieval baseline for the chunking and query-handling experiments still to come.

Ask me anything about the report fields, metadata matching, query categories, or any row that isn't clicking for you. I'm your teacher for this course — that's exactly what I'm here for.