I attended an interesting panel discussion at the opening day at LegalWeek 2025. The presentation was called Do My Eyes Deceive Me? GenAI Hallucinations in Legal Research Citation Tools. The presentation was put on by AALL and the panelists were all law librarians — Anna Russell from Cornell, Diana Koppang from Neal, Gerber & Eisenberg, Mandy Lee from Seton Hall, and Paul Callister from the University of Missouri.

Let me say up front: law librarians are some of the most trustworthy and practical voices when it comes to evaluating how GenAI—and, for that matter, other technologies—are actually functioning for legal research. They know how to ask better questions and look at a problem from every angle — something many lawyers don’t do or are not good at.

It’s Not Just Hallucinations

Most of the legal world’s concern with GenAI tools boils down to hallucinations: fake cases cited that don’t exist. That’s a fair concern — and one that’s relatively easy to catch. If a case doesn’t exist, a quick search will tell you that.

But the panel made an important point: the bigger problem isn’t just hallucinations. It’s the very real inaccuracies that don’t always get caught.

These inaccuracies include:

• A real case being cited, but not for the legal proposition it’s supposedly supporting.

• Citations to statutes that have been repealed or significantly amended.

• Precedent being referenced in a case, when it’s actually not.

• Cases cited in a dissent as if they were in the majority opinion, when it isn’t .

These are the kinds of subtle but significant errors that even seasoned attorneys might miss, not because they’re lazy, but because they’ve been conditioned to trust the tools they use. All to often, lawyers stop at merely verifying that a case simply exists in their own pleadings and in reviewing the other sides briefs. That’s clearly not enough.

There simply is no trust

And by not looking critically at the cites the other sides offers, many of the inaccuracies are getting uncovered.

Trust but Verify?

The standard advice giving for using the research results of an LLM is “trust but verify”. But I think that’s exactly the wrong mindset when it comes to GenAI legal tools.

There simply is no trust. If you have to go and read every single case to see if the tool got it right, where’s the trust? Verification is sole job. And yes, it takes time. Yes, it requires close, critical reading. But that’s what research is. And as one panelist said, researching is different from searching.

Lawyers have gotten all too comfortable with search tools — plug in a few keywords, skim the top results, move on. But GenAI doesn’t reward that behavior. In fact, it punishes it. You can’t just check to see if a case GenAI gives you exists.

Over reliance on the tools is a mistake.

And even closed systems that provide their sources (like the newer LLMs with retrieval-augmented generation) aren’t enough. The mere presence of a source for a citation isn’t proof that it’s the right one. You still have to critically read the case. Over reliance on the tools is a mistake.

What Works

The panel noted that a couple of current search engines can catch many of these inaccuracies — more so than general-purpose tools. But even the search engines have limitations:  ask two different systems to “find the best cases,” for example and you’ll likely get two very different lists. I can’t say it enough.You still have to read the cases. Technology won’t save you from the hard work. It only speeds up the front end of research.

Research Is a Process

And just because technology can speed up the beginning phases of some research, they aren’t pancreas.

Real research is iterative. You ask a question, get an answer, refine the question, dig deeper, reframe, drill down. It’s not linear. It’s not passive. And it’s certainly not just trusting whatever pops up first in the results. As the panel articulately put it, research ain’t the same as searching.

Bottom line? If you’re using GenAI tools for legal research, you need to:

• Know exactly what the tool does — and doesn’t do.

• Read every cited case and statute for yourself.

• Don’t assuming these systems think like lawyers. They don’t.

Skip the trust. Just verify.