Please Don’t Use AI as Your Expert Witness

I sincerely love large language models for brainstorming and research. But we need to be really clear about something: large language models can’t weigh evidence or reason the way humans do, so you should not cite an AI response as a reasoned conclusion to bolster your argument.

Large language models calculate responses based on the frequency of language patterns, and the prevalence of opinions—especially on contentious topics—often has little to do with actual truth. If you feed an LLM articles that support a particular position and ask it to craft a response based on them, it will reflect that input, essentially echoing the narrative you’ve curated. This selective feeding can create a kind of echo chamber, where the output feels authoritative but is just a snapshot of the provided data, not a broader truth.

There’s no doubt that LLMs excel at research and surfacing information quickly, like synthesizing trends in discussions about digital literacy or pulling together studies for a literature review. But they can’t evaluate that information for truthfulness. They might assert something is true, but they’re merely mimicking human claims of truth, acting as “stochastic parrots” that predict and string together words based on statistical patterns, not understanding or critical thinking.

Even models labeled as “reasoning” models, from what I can tell, are doing an impressive job of identifying patterned questions and recalculating responses based on new guidelines. It looks like reasoning, but it’s not what we consider human reasoning—no extrapolation or critical judgment is happening. Providers calling these “reasoning models” can mislead users into thinking they’re getting independent insight, when really, it’s just advanced pattern-matching.

This misuse isn’t just a technical issue—it’s ethical. AI can amplify biases from its training, and it can be used to manipulate or deceive when treated as a trusted source, which underscores the need for caution.

With reports of widespread student use of AI and its apparent reasoning, this could signal a growing problem for critical thinking. Treating AI like an expert witness or historian risks undermining our ability to question and reason for ourselves, much like over-relying on Wikipedia as a final source rather than a starting point. We need to use AI for what it’s great at—research, brainstorming, and spotting patterns—while reserving judgment and truth-seeking for human minds.