Open weight LLMs are not trained any differently than all the others. They just publish the weights. It’s still massive theft run through a data center.
There is no (good) coding LLM trained ethically because securing rights to the amount of code and text required has not been attempted and is likely not possible. The best you can do is a company who hid a “by using this service you give us rights to use your code/writing to train our LLM” into a six page EULA no one reads and that’s just a different kind of unethical.
Some models like Nemotron use open datasets, and utilize hardware more efficiently than OpenAI and such.
Its (IMO) also quite different when the weights are Apache licensed. If absolutely zero profit is involved, that gets close to the “fair use” umbrella, IMO, like other noncommercial derivative works of restrictively licensed content (fanfics/fanart).
You are not wrong about the closed datasets though, for the vast majority of LLMs. Lord knows what the Chinese trainers or OpenAI use to train…
But anything truly “open weights” is still a different order of magnitude in the ethics scale, IMO. I view the primary issue to be profiting off the scraping and training; take that away, and its not nearly as egregious.
Open weight LLMs are not trained any differently than all the others. They just publish the weights. It’s still massive theft run through a data center.
There is no (good) coding LLM trained ethically because securing rights to the amount of code and text required has not been attempted and is likely not possible. The best you can do is a company who hid a “by using this service you give us rights to use your code/writing to train our LLM” into a six page EULA no one reads and that’s just a different kind of unethical.
Some models like Nemotron use open datasets, and utilize hardware more efficiently than OpenAI and such.
Its (IMO) also quite different when the weights are Apache licensed. If absolutely zero profit is involved, that gets close to the “fair use” umbrella, IMO, like other noncommercial derivative works of restrictively licensed content (fanfics/fanart).
You are not wrong about the closed datasets though, for the vast majority of LLMs. Lord knows what the Chinese trainers or OpenAI use to train…
But anything truly “open weights” is still a different order of magnitude in the ethics scale, IMO. I view the primary issue to be profiting off the scraping and training; take that away, and its not nearly as egregious.