Has anyone tried in organization to use self hosted llm models for agentic programming?
Im curious if it makes any sense. My organization spends fortune on tokens from us companies. I want to recommend something…
Has anyone tried in organization to use self hosted llm models for agentic programming?
Im curious if it makes any sense. My organization spends fortune on tokens from us companies. I want to recommend something…
How many concurrent users and what hardware if i may ask?
it’s an h100, I think, no idea about how many users
in my personal setup i use quantized versions on a 3080, which is not great, so I still lean a lot on APIs