RAG Formation · Reference

Latency, cost, and observability

A standalone cheat sheet for learning and testing RAG quality on this site.

$ reference --status
▸ course RAG Formation
▸ mode standalone reference
▸ status Complete
● learn → test → verify

Reference sheet

The guided RAG Formation course content is complete; this reference is available for self-directed practice.

One-line rule

Average latency can hide a slow tail that makes the system unusable in a pilot.

Term	Meaning
p50 and p95	The median and the slow tail. Both matter because a fast average can hide painful outliers. p95 is only computed for answer-mode runs — retrieval-only reports p50 alone.
Cost drivers	Embedding calls, graph LLM calls, answer generation, judge calls, and context size.
The one real lever	`--graph-topk` (default 20) — lowering it trades LightRAG's graph-traversal depth against latency. There is no way to disable LightRAG entirely on a live request.
Observability	Structured traces, metrics, and logs that show what happened without exposing secrets or document text.

Need	Record	Don't claim
Baseline	Command, configuration, observed result — the same `eval_set.yaml` baseline from Lesson 4.	The baseline is a target.
Change	One hypothesis or decision	You can tell what caused the result when several things changed.
Metric	`summary.answer.latency_p50_s`/`.latency_p95_s`, `summary.retrieval.latency_p50_s`, `answer.stats.total_context_chars`/`.graph_context_chars`; never log secrets or raw sensitive content.	One metric tells you everything about quality.
Safety	A failure and a regression case	A fluent answer proves the system is reliable.

cd demo
python3 eval.py run --mode both --workspace meridian_demo --tag lesson-14-latency

Use --mode both, not --mode retrieval alone — the retrieval-only summary never reports p95.

Artifact: An MVP performance budget and telemetry checklist, with each item marked measured or assumed.

Continue learning: Open the matching standalone lesson · Course roadmap