even though it is worthless whenever next model version is released.
For most version releases I actually doubt that, I’m pretty sure they just keep training the model and release checkpoints every few months. Architectural changes, however, will indeed invalidate the previously trained models, and occasionally they have to do that to keep up with the competition.
But if you do architectural changes even just once a year, that means you’d have to depreciate those training costs over just that one year to be truthful. Meaning they might as well be losses rather than depreciating capital investments.
For most version releases I actually doubt that, I’m pretty sure they just keep training the model and release checkpoints every few months. Architectural changes, however, will indeed invalidate the previously trained models, and occasionally they have to do that to keep up with the competition.
But if you do architectural changes even just once a year, that means you’d have to depreciate those training costs over just that one year to be truthful. Meaning they might as well be losses rather than depreciating capital investments.
New models are trained on older models, so even if the older model is not longer at par, without it the new model couldn’t exists