Has anyone tried in organization to use self hosted llm models for agentic programming?

Im curious if it makes any sense. My organization spends fortune on tokens from us companies. I want to recommend something…

  • Eager Eagle@lemmy.worldEnglish
    3·
    2 days ago

    it’s an h100, I think, no idea about how many users

    in my personal setup i use quantized versions on a 3080, which is not great, so I still lean a lot on APIs