Can you cite your source on the claim that “inference is currently insanely profitable”? Everything I read suggests that openai and anthropic lose money on their plans.
I suspect it’s profitable in the abstract - and their accountants would be bad at their jobs if they couldn’t work out what utilisation rate you need to pay for the server runtime.
However how aggressively you amortise the cost of the training is the key, especially if you keep releasing new models every 6 months.
Can you cite your source on the claim that “inference is currently insanely profitable”? Everything I read suggests that openai and anthropic lose money on their plans.
I suspect it’s profitable in the abstract - and their accountants would be bad at their jobs if they couldn’t work out what utilisation rate you need to pay for the server runtime.
However how aggressively you amortise the cost of the training is the key, especially if you keep releasing new models every 6 months.