For 2 years, vendors sold retrieval as the cure for hallucination. Point the model at your documents, we were told, and it can only tell the truth. This was the new “Veritaserum”.
Well, Stanford’s RegLab tested that. Their preregistered study of the leading legal research tools found Lexis+ AI and Thomson Reuters’ products hallucinating between 17% and 33% of the time, against 43% for raw GPT-4. Accuracy varied more than the marketing suggested: Lexis+ AI answered 65% of queries accurately, Westlaw’s AI-Assisted Research 42%, and Ask Practical Law AI around 20%, with incomplete answers on more than 60% of queries.
Hallucination rate, legal research tools
Stanford RegLab, preregistered study of the leading legal research tools.
Yes, retrieval cut the rate by more than half, which is incredible. It also left 1 in 6 answers fabricated at the best-performing vendor, in a domain where a fabricated citation gets you sanctioned.
A February 2026 research roundup adds a detail that is absolutely brutal: these systems sound equally confident whether they’re right or wrong.
Why grounding leaks
Retrieval fails silently. The search returns nothing useful, and the model answers anyway from what it learned in training. The answer may even be correct, which is worse, because it means your evaluation scored a lucky guess as a win.
An August 2026 evaluation framework puts numbers on this. The researchers planted “knowledge-gap canaries”, questions whose answers verifiably do not exist in the knowledge base, so any answer at all proves the system is drawing on the model rather than your documents. Across 3 commercial RAG products, accuracy when answering clustered tightly. What separated them was the policy governing when not to answer, with canary violation rates running as high as 98.1%. Conventional accuracy scoring ranked those products in the opposite order.
Which means most RAG evaluations reward exactly the behavior RAG was bought to prevent.
What a trustworthy answer looks like
Ungrounded
What’s our refund window for annual enterprise plans?
Annual enterprise plans carry a 30-day refund window, prorated after day 14.
Confident, specific, sourced from nothing.
Grounded, with an abstention policy
What’s our refund window for annual enterprise plans?
The policy documents I can see cover monthly plans only (Refund Policy v4, §2). I don’t have a document covering annual enterprise terms. Escalating to billing.
[ Refund Policy v4 ] [ escalate ]
The second answer is the harder engineering problem and the only one you can put in front of a customer.
5 tests before you deploy
- Canary questions. Write 20 questions your knowledge base provably can’t answer. Any confident answer is a failure, even a correct one.
- Citation contract. Every factual claim carries a retrieved passage ID. Claims without one get blocked before they reach the user.
- Abstention rate. Measure how often the system says it doesn’t know, and check it against how often it should have. A 0% refusal rate is a red flag, not a feature.
- Multi-turn drift. Test conversations, not single questions. Retrieval quality degrades badly once context accumulates.
- Retrieval recall, measured separately. If the right document never gets retrieved, no amount of prompt work saves the answer.
Run these before launch, then keep running them, because your documents change and the model behind the system gets swapped without warning.
Thinking about putting one in front of customers and we’ve scared you enough? Book a free audit call, and we’ll go through your retrieval setup and your refusal behaviour.
