[FRIAM] Hallucinations

[Marcus Daniels]

Three ways some to mind..  I would guess that OpenAI, Google, Anthropic, and xAI are far more sophisticated..

 

1.	Add a softmax penalty to the loss that tracks non-factual statements or grammatical constraints.   Cross entropy may not understand that some parts of content are more important than others. 
2.	Change how the beam search works during inference to skip sequences that fail certain predicates – like a lookahead that says “Oh, I can’t say that..” 
3.	Grade the output, either using human or non-LLM supervision, and re-train.

 

From: Friam <friam-bounces at redfish.com> On Behalf Of Russ Abbott
Sent: Tuesday, September 9, 2025 3:03 PM
To: The Friday Morning Applied Complexity Coffee Group <friam at redfish.com>
Subject: [FRIAM] Hallucinations

 

 

OpenAI just published a paper on hallucinations <https://cdn.openai.com/pdf/d04913be-3f6f-4d2b-b283-ff432ef4aaa5/why-language-models-hallucinate.pdf>  as well as a post summarizing the paper <https://openai.com/index/why-language-models-hallucinate/> . The two of them seem wrong-headed in such a simple and obvious way that I'm surprised the issue they discuss is still alive. 

 

The paper and post point out that LLMs are trained to generate fluent language--which they do extraordinarily well. The paper and post also point out that LLMs are not trained to distinguish valid from invalid statements. Given those facts about LLMs, it's not clear why one should expect LLMs to be able to distinguish true statements from false statements--and hence why one should expect to be able to prevent LLMs from hallucinating. 

 

In other words, LLMs are built to generate text; they are not built to understand the texts they generate and certainly not to be able to determine whether the texts they generate make factually correct or incorrect statements.

 

Please see my post <https://russabbott.substack.com/p/why-language-models-hallucinate-according>  elaborating on this.

 

Why is this not obvious, and why is OpenAI still talking about it?

 

-- Russ Abbott <https://russabbott.substack.com/>   (Click for my Substack)

Professor Emeritus, Computer Science
California State University, Los Angeles

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://redfish.com/pipermail/friam_redfish.com/attachments/20250909/1129432c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5594 bytes
Desc: not available
URL: <http://redfish.com/pipermail/friam_redfish.com/attachments/20250909/1129432c/attachment.p7s>