AI Engineering & LLMs (50 Qs)•Part 4: LLM Security, Safety & OWASP Top 10 for LLMs
Overreliance & Hallucination Mitigation: Grounding Citations, Self-Consistency & Confidence Scoring
1. The Danger of Overreliance in Critical Domains
When enterprise software blindly trusts LLM outputs in legal, healthcare, or financial operations, hallucinations create massive legal liability.
- Hallucination: The model produces syntactically fluent, authoritative-sounding statements that are factually false.
2. Production Mitigation Techniques
css
User Question
│
▼
[ RAG Retrieval with Strict Chunk ID Tags ]
│
▼
[ LLM Constrained Prompt: Grounded Generation ]
"Every factual assertion MUST cite a bracketed chunk ID [Chunk-12]."
│
▼
[ Automated Citation Verifier Gate ]
Are all cited claims strictly present in the cited chunk?
├─────────► YES ──► Output with clickable source link
└─────────► NO ──► Refuse: "Data not found in documentation"
3. Self-Consistency / Ensemble Sampling
For high-stakes reasoning (e.g., medical dosage calculation or complex tax logic):
- Sample the prompt 5 times at temperature $T = 0.7$.
- Compare the 5 independent outputs:
- If 5/5 or 4/5 outputs converge on the same conclusion → High Confidence.
- If outputs diverge (2 said Yes, 3 said No) → Low Confidence: Route to Human Expert.