RAG Hallucination Detector
A hallucination detector for RAG systems: a full pipeline (retrieval + generation + verification) with a model matching published state-of-the-art results, plus a research paper auditing an unexploited annotation-quality issue in the field's standard benchmark.
This project tackles a real problem in RAG (Retrieval-Augmented Generation) systems: detecting when an LLM invents information not supported by the context it was given. It has two independent deliverables, each solid on its own. The first is the system: a complete RAG pipeline (FAISS + embeddings retriever, LLM generation, and a token-level hallucination detector) with an interactive demo and a deployable API; the final model (ModernBERT, token-level binary classification) matches the published F1 of LettuceDetect-base and, in its scaled-up variant, slightly surpasses the F1 of LettuceDetect-large — the field's most recent reference system — trained entirely on free GPU. The second is the paper: during evaluation, it turned out that RAGTruth, the field's standard benchmark, labels both false content and true-but-not-mentioned-in-context content as "hallucination," without any published system distinguishing between the two cases when scoring. The project quantifies exactly how much this conflation weighs (9.49% of the test set's character mass), designs a pre-registered ablation experiment to test whether fixing it helps training (result: no, with a dose-response curve ruling out both alternative explanations), and documents the full finding in a preprint verified line by line against the raw data.
The problem
Production RAG systems can generate answers that "sound right" but aren't actually supported by the retrieved documents — a real risk in sensitive domains (legal, healthcare, finance). On top of that, the benchmark the community uses to measure these detectors (RAGTruth) mixes, under a single label, genuinely harmful hallucinations with content that is technically ungrounded but true — and no published system, including the field's reference systems, accounts for this when reporting its metrics.
The solution
A token-level hallucination detector (ModernBERT), trained and validated with full experimental rigor: zero-shot baseline, systematic comparison across 4 different architectures, error analysis, and a pre-registered ablation with the decision fixed in code before seeing the results. The system is integrated into a real RAG pipeline (retrieval + generation + verification), with a working demo and API. In parallel, the benchmark's label-conflation problem was quantified and formally documented, with exhaustive verification of every figure against the raw data and the cited primary sources.