• AnAmericanPotato@programming.dev
    link
    fedilink
    English
    arrow-up
    1
    ·
    11 hours ago

    It’s still an open question where the eventual sweet spot will be in terms of model size and speed once the dust settles.

    Nobody has the hardware to run frontier models in their personal devices. Even the larger open models are out of reach unless you’re ready to spend $10-20k on hardware. You can’t do shit on 8GB of memory.

    That said, I don’t think there’s any great use case for trillion-parameter models in the long term. You can get good results for cheap from much smaller models with smarter workflows, and eventually that will become as easy and accessible as using cloud products. The big players have done well staying 6-12 months ahead, but that’s really not a lot in the grand scheme and they can’t keep it up indefinitely.

    Their only play is regulatory capture and they’re pushing hard for it.