• Voroxpete@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    57
    ·
    21 hours ago

    Basically, yes.

    This comes back to the basic problem with LLMs as a business proposition; any digital service can either be cheap and frivolous, or expensive and necessary. You can even, if you do things right, be cheap and necessary. But you cannot ever be expensive and frivolous.

    LLMs were supposed to be expensive and necessary, but it turns out they’re just so unreliable that they simply cannot perform an essential role. They must always be disposable; fun, or useful, but never essential.

    Which leaves only cheap and frivolous as an option. And they’re just not cheap, nor does it seem likely that they ever will be.

    So you get Character.ai filling chapter 11, because people simply will not ever pay what it actually costs to run these models just so that they can have a computer program pretend to be Lois Griffin for them.

    • schipelblorp@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      19
      ·
      20 hours ago

      With the added wrinkle that there are expensive things that may be necessary-ish, but when you exclusively use it for those things, the price jumps up to insanity.

      Now that people are shunting easier requests to local models and sending the harder problems out, the hyper scalers have less income, so they have to charge more.

      Because eventually the only real limitation will be hardware, that kind of AI will be a rarely-used commodity priced only slightly above what it would cost you to recreate it inside your business.

      Not to mention the privacy concerns you’d have sending your expensive and necessary compute to the cloud.

      • Voroxpete@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        18
        ·
        20 hours ago

        Now that people are shunting easier requests to local models and sending the harder problems out, the hyper scalers have less income, so they have to charge more.

        While you’re 100% correct about the fact that this is happening, and it is a serious problem for the hyper-scalers, it’s really, really important to note that even before this started happening, LLMs were already hilariously unprofitable, both on the software and the hardware side. For the AI companies, there is simply no way to charge people what the models actually cost to run and still turn a profit. On the data-centre side, the turnover on burned out GPUs is going to annihilate any hope of profitability. They’re treating as capex what is essentially a consumable and their business models simply do not account for this reality.

        The behaviour you’re identifying here is basically just a swift kick in the guts to someone who was already bleeding to death from several bullet wounds. Seriously unhelpful, but not remotely the impending cause of death.