An associate brings you a research memo. It is well organized, the writing is clean, and the third case cited does not exist. Not "is distinguishable," not "was overruled." It was never decided by any court. The reporter citation is formatted perfectly and the parenthetical describes a holding that would have been very convenient.
That is a hallucination, and it is the single most important thing to understand about using AI assistants in legal work. This page explains what causes it, what courts have done about it, and how to build a verification habit that catches it every time without slowing the firm down.
What a hallucination looks like in practice
The obvious form is the invented citation: a case name, reporter, and pin cite that look right and lead nowhere. Less obvious forms are more common. A real case is cited for a holding it does not contain. A statute is quoted with a subsection that was renumbered years ago. A deposition summary attributes a statement to the witness that appears nowhere in the transcript, because the tool answered the question it expected rather than the question the document answered.
There is also the quiet version: a fact summary that is accurate in every detail except one date, which was shifted by a month. Nobody catches it because everything around it is right.
Why it happens: fluency is not knowledge
A language model produces the most plausible next words given everything it has seen. When asked for a case supporting a proposition, it produces text that looks like a case supporting that proposition. Whether such a case exists is a separate question the model is not, by itself, answering.
Modern assistants, including Claude, are substantially better at saying "I am not certain" and at sticking to documents you provide. Anthropic is open that Claude can still be wrong and that outputs need human review. The improvement is real, and it is also the reason the risk persists: the errors that remain are the ones that look most like correct work.
The sanctions cases and what courts now expect
The pattern became public in 2023 when a federal court in New York sanctioned lawyers who filed a brief containing citations generated by an AI tool that did not exist, in the matter involving Avianca. Similar orders have followed in other courts, and the common thread is not that lawyers used AI. It is that they filed work they had not verified.
Some judges now have standing orders requiring lawyers to certify whether generative AI was used in a filing and that any such content was checked by a person. ABA Formal Opinion 512 (2024) frames the same expectation under the existing duties of competence and candor to the tribunal. Check your local rules and the orders of the judges you appear before, and build the certification step into your filing checklist so nobody has to remember it under deadline pressure.
A verification workflow that fits a busy practice
Verification does not have to be a second full pass. It has to be a specific set of checks done every time, and done by the person who signs.
For citations: open every authority in Westlaw, Lexis, or the court's own database. Confirm it exists, read the pinpointed passage, confirm it supports the stated proposition, and check its subsequent history. Do not accept a citation because the tool also gave you a quotation; quotations can be invented as easily as cites. For document-based work: require the tool to give a page or paragraph reference for every fact, then spot-check a sample and fully check anything that matters to the outcome. For numbers and dates: recompute or re-read every one that appears in a demand, a filing, or a client communication.
Write the checks into the firm's AI policy as a review requirement so that they apply to staff work as well as attorney work, and teach them with real examples in AI training for paralegals and legal staff.
Reducing the rate: grounding, retrieval, and constraints
You cannot eliminate hallucinations, but you can make them rarer and easier to catch. The biggest lever is grounding: give the tool the source material and instruct it to answer only from that material, and to say when the material does not answer the question. For research, this means doing the research in your research platform first and handing the tool the cases, not asking it to find them. Our page on drafting legal documents with Claude builds this into the drafting method.
The second lever is constraint. Ask for citations to the provided documents by page. Ask for a list of assumptions and gaps at the end of every output. Instruct the tool that it is better to leave a blank than to fill one. The third lever is choosing tools that are designed to retrieve from verified sources; the AI features inside major research platforms work this way, and they should still be checked. This is the core of what we teach in Claude training for lawyers.
Questions we get
Are the research platforms' own AI tools safe from this?
Safer, not safe. Tools built on top of a verified case database are far less likely to invent an authority, because they retrieve real documents and summarize them. They can still misstate what a real case holds. The verification step is shorter with these tools, but it is still required.
Should we stop using AI for legal research entirely?
Most firms land on a rule: AI may help organize, summarize, and draft, but the authorities must come from a research platform and every one gets opened by a person. That rule preserves most of the value and removes most of the risk. A blanket ban tends to push usage underground, where nothing is verified.
What do we do if an unverified citation already went out?
Correct it promptly and candidly. Courts have been far harsher on lawyers who denied or minimized the problem than on those who notified the court and fixed the filing. Then treat it as an incident under the firm's policy: find out how it got through, and fix the process rather than only the person.
If you want help building a verification routine your whole team will follow, tell us what you are working with.