LLMs Will Always Hallucinate, and We Need to Live With This
arxiv.org
As Large Language Models become more ubiquitous across domains, it becomes important to examine their inherent limitations critically. This work argues that hallucinations in language models are not just occasional errors but an inevitable feature of these systems. We demonstrate that hallucinations stem from the fundamental mathematical and logical structure of LLMs. It is, therefore, impossible to eliminate them through architectural improvements, dataset enhancements, or fact-checking mechanisms. Our analysis draws on computational theory and Godel's First Incompleteness Theorem, which references the undecidability of problems like the Halting, Emptiness, and Acceptance Problems. We demonstrate that every stage of the LLM process-from training data compilation to fact retrieval, intent classification, and text generation-will have a non-zero probability of producing hallucinations. This work introduces the concept of Structural Hallucination as an intrinsic nature of these systems. By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated.
  • Ilixtze@lemmy.mlEnglish
    20·
    2 days ago

    This is a feature not a bug. Right wing oligarchs, a lot of them in tech, have been creaming their pants on the fantasy of shaping general consensus and privatizing culture for decades. LLM hallucination is just a wrench they are throwing on the machinery of human subjectivity.

    • Scrubbles@poptalk.scrubbles.techEnglish
      35·
      2 days ago

      Uh, no. You want to be mad at something like that look into how they’re training models without a care for bias (or adding in their own biases).

      Hallucination is a completely different thing that is mathematically proven to happen regardless of who or what made it. Even if the model only knows about fluffy puppies and kitties it will still always hallucinate to some extent, just in that case it will be hallucinating fluffy puppies and kitties. It’s just random data at the end.

      That isn’t some conspiracy. Now if you expected a model that’s fluffy kitties and puppies and you’re mad because it starts spewing out hate speech - that’s not hallucination. That’s the training data.

      If you’re going to rage about something like that, you might as well rage about the correct thing.

      I’m getting real tired here of the “AI is the boogieman”. AI isn’t bad. We’ve had AI and Models for over 20 years now. They can be really helpful. The bias that is baked into them and how they’re implemented and trained has always been and will continue to be the problem.

      • anomnom@sh.itjust.worksEnglish
        2·
        6 hours ago

        So it’s really both.

        LLMs may always hallucinate, bad actors are also going to poison models they have control over, but even “good” or “neutral” LLMs are useful to fascists. Because part of the fascist playbook it’s to remove meaning and facts from the language they use. Often their appeal to recruits is that they are telling them what they want to hear or feel, sometimes based on a truth or fact, but it doesn’t matter to the fascists, just like it doesn’t matter to the LLMs.

      • Ilixtze@lemmy.mlEnglish
        3·
        1 day ago

        Nah don’t put words in my mouth, I’m mad so much money is wasted on this useless LLM shit. I didn’t use the word “AI” once on my post, so the fact that we’ve had AI for 20 years is beside the point.

        The dream of the tech oligarchs is to privatize and centralize everything. LLM’s is their tool. Fuck techbros and their Large language bullshit

      • AmbiguousProps@lemmy.todayEnglish
        9·
        2 days ago

        The AI we’ve had for over 20 years is not an LLM. LLMs are a different beast. This is why I hate the “AI” generalization. Yes, there are useful AI tools. But that doesn’t mean that LLMs are automatically always useful. And right now, I’m less concerned about the obvious hallucination that LLMs constantly do, and more concerned about the hype cycle that is causing a bubble. This bubble will wipe out savings, retirement, and make people starve. That’s not to mention the people currently, right now, being glazed up by these LLMs and falling to a sort of psychosis.

        The execs causing this bubble say a lot of things similar to you (with a lot more insanity, of course). They generalize and lump all of the different, actually very useful tools (such as models used in cancer research) together with LLMs. This is what allows them to equate the very useful, well studied and tested models to LLMs. Basically, because some models and tools have had actual impact, that must mean LLMs are also just as useful, and we should definitely be melting the planet to feed more copyrighted, stolen data into them at any cost.

        That usefulness is yet to be proven in any substantial way. Sure, I’ll take that they can be situationally useful for things like making new functions in existing code. They can be moderately useful for helping to get ideas for projects. But they are not useful for finding facts or the truth, and unfortunately, that is what the average person uses it for. They also are no where near able to replace software devs, engineers, accountants, etc, primarily because of how they are built to hallucinate a result that looks statistically correct.

        LLMs also will not become AGI, they are not capable of that in any sort of capacity. I know you’re not claiming otherwise, but the execs that say similar things to your last paragraph are claiming that. I want to point out who you’re helping by saying what you’re saying.