RAG Formation · Reference

Run and read a RAG baseline

A standalone cheat sheet for learning and testing RAG quality on this site.

$ reference --status
▸ course RAG Formation
▸ mode standalone reference
▸ status Complete
● learn → test → verify

Reference sheet

Run and read a RAG baseline

The guided RAG Formation course content is complete; this reference is available for self-directed practice.

Baseline recipe

Verify → start isolated stack → ingest → retrieval run → answer run → inspect failures → save configuration.

What to record

Item	Why
Tag	Labels the run, for example `lesson-4-baseline`.
Corpus and eval-set version	Stops you from comparing different test data as if it were the same.
Workspace and index names	Confirms that the run used the isolated demo data.
topk and thresholds	These retrieval settings affect the result.
Model and judge model	Changing either model can change the answer or the judge’s verdict.
Concurrency and timeout	These operational settings affect errors and latency.

Metrics to read separately

Retrieval union: did the combined indexes return every expected document?
Strict accuracy: how many judged answers were completely correct?
Source hit rate: did the final answer cite every expected source?
Refusal accuracy: did the system decline every unanswerable question?
Hallucination rate: how often did it answer a question the evidence could not support?
Latency p50 / p95: the typical response time and the slow tail.

Good baseline statement

“With this corpus, eval set, model, and configuration, the system achieved X.”

Bad baseline statement

“The chatbot is 90% accurate.”

Failure reading rule

For every failure, write down three things: the expected source, the actual source, and the judge’s reason. That turns a percentage into an engineering hypothesis.

Continue learning: Open the matching standalone lesson · Course roadmap