• frongt@lemmy.zip
    link
    fedilink
    English
    arrow-up
    13
    ·
    5 hours ago

    Definitely. We do this at my job. But a server big enough to provide a comparable service would cost at least $30k. And you likely need more than one. And you still need someone to maintain it.

    • chocrates@piefed.world
      link
      fedilink
      English
      arrow-up
      3
      ·
      4 hours ago

      I’ve heard that GPU’s for this work are exceedingly hard to get ahold of as well. Glm 5, which i think is near frontier model capabilities, takes around 1.5 Tb of vram un-quantized.