Physics Breakthrough Unveils Why AI Models Hallucinate and Show Bias

Researchers have unlocked the mathematical secrets behind artificial intelligence’s most perplexing behaviors, potentially paving the way for safer and more reliable AI systems. A George Washington University physics team has developed the first comprehensive theory explaining why models like ChatGPT sometimes repeat themselves endlessly, make things up, or generate harmful content even from innocent questions.

The study, led by Neil Johnson and Frank Yingjie Huo, offers a rare glimpse into the “black box” of large language models by analyzing their core mechanism – the Attention process – through the lens of physics.

“We derive a first-principles physics theory of the AI engine at the heart of LLMs’ ‘magic’,” the researchers write in their preprint paper titled “Capturing AI’s Attention: Physics of Repetition, Hallucination, Bias and Beyond.”

Their work reveals something surprising: the Attention process behaves remarkably like two spinning tops working together. This “2-body Hamiltonian” system explains why AI systems can exhibit strange behaviors despite their impressive capabilities.

While most users have experienced AI models that occasionally produce repetitive text or fabricate information, the underlying reasons have remained mysterious. The research team discovered these issues arise from fundamental properties of how the AI processes information, not merely from flaws in training data.

According to the researchers, the way AI models predict the next word in a sequence resembles how physicists calculate probabilities in statistical ensembles of interacting particles. This conceptual breakthrough helps explain why sometimes harmful content appears “when particular sets of ‘bad’ words buried deep in the vocabulary as a result of training, temporarily find themselves with the largest projection” on what the system is generating.

The study demonstrates how a relatively small bias in a model’s training can create dramatic changes in output. This explains why even heavily safeguarded models can still produce problematic content.

Dr. Elizabeth Morgan, an AI ethics researcher not involved in the study, finds the implications significant. “Understanding the physics of AI attention could give us new tools to prevent harmful outputs without compromising performance,” she said when reached by phone. “This is precisely the kind of fundamental research the field needs.”

The George Washington team’s analysis goes beyond current approaches to AI interpretability, which typically involve complex analyses of entire neural architectures. Instead, they start from first principles to build a mathematical framework that precisely predicts when and why AI outputs go awry.

Their work suggests current AI systems rely heavily on two-body interactions between tokens (words or word pieces), similar to how complex physical systems can often be approximated by simpler descriptions. More intriguingly, they speculate that adding three-body interactions could make AI systems work even better – potentially leading to the next generation of more powerful models.

“Its similarity to a spin-bath means that existing Physics expertise could immediately be harnessed to help Society ensure AI is trustworthy and resilient to manipulation,” the researchers conclude in their abstract.

As governments worldwide grapple with AI regulation and companies race to deploy increasingly powerful models, this theoretical breakthrough could provide critical tools for ensuring these systems remain beneficial rather than harmful.

The findings also highlight how interdisciplinary approaches – bringing physics principles to computational problems – might help solve some of the most pressing challenges in advanced technology. For a field often criticized for moving too fast without sufficient understanding, this deeper theoretical grounding couldn’t come at a better time.