When an AI Answer Sounds Right and Isn’t

Every utility project runs on documents. Specifications, test reports, submittals, spec sheets, manufacturer drawings. When a sourcing team starts using out-of-the-box AI tools to pull answers out of that paperwork, the appeal is obvious. You ask a question and get an answer in seconds, instead of working through a thousand-page package by hand.

The catch is that an AI tool can hand you an answer that reads as authoritative and is simply wrong. In AI, this is called a hallucination, which is an unfortunate term, because it makes people picture a glitch or an error message. A hallucination is neither. It is a fluent, confident, well-formatted answer that happens to be incorrect.

Here is why it happens. A large language model does not look things up the way a person searches a file. It generates text by predicting which words should come next, based on patterns it learned from an enormous body of writing. Most of the time that prediction lands on something true. Sometimes it lands on something that only sounds true. The model has no internal sense of which is which. A fabricated impedance value and a correct one come out of the same process and arrive with the same confidence.

Consider a common case. An engineer asks an AI assistant for the basic impulse insulation level of a transformer described in a submittal package. The tool returns a clean number in a tidy sentence. It looks right. But that figure may have been carried over from a similar unit, averaged across products the model saw during training, or generated simply to fit the shape of the question. Nothing in the answer tells you which of those occurred.

In most everyday settings, a wrong answer is a minor annoyance. In utility sourcing it is something else. A specification figure feeds procurement decisions, compatibility checks, and eventually the equipment that has to perform on an energized system for decades. An answer that is confidently wrong and goes unverified can move quietly downstream into a purchase order, where catching it costs far more than it would have at the start.

This is not an argument against using these tools. It is an argument for understanding what they are. The useful question to carry into any evaluation is not whether a tool sounds knowledgeable, because they all do. The useful question is whether you can trace a given answer back to the source document it came from, so a person can confirm it before it matters.

A hallucination is not a malfunction. It is a built-in characteristic of how these systems produce language. Knowing that does not mean keeping AI out of the sourcing process. It means reading every answer it gives with a clear understanding of where that answer came from, and treating a source you can check as worth more than a sentence that merely sounds correct.