Distillation isn’t stealing the original model, though. It just uses the models to make synthetic training data to train their own thing. They aren’t stealing the model itself.
It also doesn’t seem like as big a deal as Anthropic and Open AI make it look, IMO. Them treating it like a national security issue where the company gets its models stolen from under its nose just comes across like a media company claiming that every download is a copy they would otherwise have sold at full price, and thus they have accrued trillions of dollars in damages.
I could, in theory, take a bunch of google Gemini outputs, and train a GPT-2 model on them. That doesn’t mean that I’ve recreated Gemini, nor does it mean that i’ve stolen it from Google, either.
To top it all off, it’s not like their services were abused. The companies were presumably paid appropriately for the usage.
Distillation isn’t stealing the original model, though. It just uses the models to make synthetic training data to train their own thing. They aren’t stealing the model itself.
Plus, a lot of companies do it. Anthropic’s Claude was calling itself DeepSeek for a while.
It also doesn’t seem like as big a deal as Anthropic and Open AI make it look, IMO. Them treating it like a national security issue where the company gets its models stolen from under its nose just comes across like a media company claiming that every download is a copy they would otherwise have sold at full price, and thus they have accrued trillions of dollars in damages.
I could, in theory, take a bunch of google Gemini outputs, and train a GPT-2 model on them. That doesn’t mean that I’ve recreated Gemini, nor does it mean that i’ve stolen it from Google, either.
To top it all off, it’s not like their services were abused. The companies were presumably paid appropriately for the usage.