• AnAmericanPotato@programming.dev
      link
      fedilink
      English
      arrow-up
      1
      ·
      11 hours ago

      It’s still an open question where the eventual sweet spot will be in terms of model size and speed once the dust settles.

      Nobody has the hardware to run frontier models in their personal devices. Even the larger open models are out of reach unless you’re ready to spend $10-20k on hardware. You can’t do shit on 8GB of memory.

      That said, I don’t think there’s any great use case for trillion-parameter models in the long term. You can get good results for cheap from much smaller models with smarter workflows, and eventually that will become as easy and accessible as using cloud products. The big players have done well staying 6-12 months ahead, but that’s really not a lot in the grand scheme and they can’t keep it up indefinitely.

      Their only play is regulatory capture and they’re pushing hard for it.