RAG Formation · Lesson · Grounded answer generation
Refusal, uncertainty, and hallucination control
Define safe behavior when the corpus does not contain the answer a QA, regulatory, or audit reader needs.
$ lesson --status
▸ course RAG Formation
▸ lesson 12 / 16
▸ phase Grounded answer generation
▸ status Complete
● build → measure → learnStandalone lesson
Learn the full lesson and test your understanding here.
The 16-lesson RAG Formation curriculum is complete. This page contains the complete lesson content, local reference material, and instant browser exercises.
Today’s tangible win
By the end, you’ll have a refusal policy — with test cases covering no evidence, partial evidence, conflicting evidence, and ambiguous questions.
Watch the walkthrough
1 · The skill
Take the six unanswerable eval questions and sort each response into one of three buckets: safe refusal, unsafe hedge, or outright hallucination.
- Refusal: Says plainly that the indexed evidence doesn’t establish the fact being asked for.
- Uncertainty: Spells out — carefully — what the evidence does support, and what’s still an open question.
- Hallucination: A real, substantive claim that the allowed evidence just doesn’t back up.
2 · Run the evidence loop
cd demo
python3 eval.py run --mode answer --workspace meridian_demo --category unanswerable --tag lesson-12-refusalTrack refusal accuracy and hallucination rate as two separate numbers, and grab one example answer that crossed the line.
Careful: a smooth, partial answer can still be a hallucination — the moment it slips in one unsupported claim that actually matters.
3 · Decision worksheet
| Baseline evidence | One hypothesis/change | Metric | Regression check | Bounded conclusion |
|---|---|---|---|---|
| _____ | _____ | _____ | _____ | _____ |
4 · Retrieval practice
Question: Your evidence only covers part of what’s being asked. What’s the safe move?
Mission connection
In QA, audit, and regulatory work, a plain ‘I can’t confirm that from the indexed documents’ beats a confident-sounding answer that’s just made up.
5 · Deliverable and next action
- Save the command output, or a report of it.
- Fill out the decision worksheet.
- Write down one failure case and one thing to verify next.
By the end, you’ll have a refusal policy — with test cases covering no evidence, partial evidence, conflicting evidence, and ambiguous questions.
Next: Lesson 13 — Failure isolation and graceful degradation.
Got questions? Ask me — about the evidence, the metric, the failure case, or whatever you want to try next. I’m your instructor for this course.