A pre-trained model alone can’t really be open source. Without the source code and full data set used to generate it, a model alone is analogous to a binary.
@sunstoned@Ephera That’s nonsense. You could write the scripts, collect the data, publish all, but without the months of GPU training you wouldn’t have the trained model, so it would all be worthless. The code used to train all the proprietary models is already open-source, it’s things like PyTorch, Tensorflow etc. For a model to be open-source means you can download the weights and you are allowed to use it as you please, including modifying it and publishing again. It’s not about the dataset.
@dandi8 but you are the one who is changing it. And who said it’s not feasible? Mixtral model is open-source. WizardLM2 is open-source. Phi3:mini is open-source… what’s your point?
But the license of the model is not related to the license of the data used for training, nor the license for the scripts and libraries. Those are three separate things.
Open-source software (OSS) is computer software that is released under a license in which the copyright holder grants users the rights to use, study, change, and distribute the software and its source code to anyone and for any purpose.
From Mistral’s FAQ:
We do not communicate on our training datasets. We keep proprietary some intermediary assets (code and resources) required to produce both the Open-Source models and the Optimized models. Among others, this involves the training logic for models, and the datasets used in training.
Unfortunately we’re unable to share details about the training and the datasets (extracted from the open Web) due to the highly competitive nature of the field.
The training data set is a vital part of the source code because without it, the rest of it is useless. The model is the compiled binary, the software itself.
If you can’t share part of your source code due to the “highly competetive nature of the field” (or whatever other reason), your software is not open source.
I cannot lool at Mistral’s source and see that, oh yes, it behaves this way because it was trained on this piece of data in particular - because I was not given accesa to this data.
I cannot build Mistral from scratch, because I was not given a vital piece of the recipe.
I cannot fork Mistral and create a competitor from it, because the devs specifically said they’re not providing the source because they don’t want me to.
You can keep claiming that releasing the binary makes it open source, but that’s not going to make it correct.
What makes it open source?
The license.
If I license a binary as open source does that make it open source?
Nope. Second point in the definition: https://opensource.org/osd
My point precisely :)
A pre-trained model alone can’t really be open source. Without the source code and full data set used to generate it, a model alone is analogous to a binary.
@sunstoned @Ephera That’s nonsense. You could write the scripts, collect the data, publish all, but without the months of GPU training you wouldn’t have the trained model, so it would all be worthless. The code used to train all the proprietary models is already open-source, it’s things like PyTorch, Tensorflow etc. For a model to be open-source means you can download the weights and you are allowed to use it as you please, including modifying it and publishing again. It’s not about the dataset.
Just because open source AI is not feasible at the moment is no reason to change the definition of open source.
@dandi8 but you are the one who is changing it. And who said it’s not feasible? Mixtral model is open-source. WizardLM2 is open-source. Phi3:mini is open-source… what’s your point?
But the license of the model is not related to the license of the data used for training, nor the license for the scripts and libraries. Those are three separate things.
https://en.m.wikipedia.org/wiki/Open-source_software
From Mistral’s FAQ:
https://huggingface.co/mistralai/Mistral-7B-v0.1/discussions/8
The training data set is a vital part of the source code because without it, the rest of it is useless. The model is the compiled binary, the software itself.
If you can’t share part of your source code due to the “highly competetive nature of the field” (or whatever other reason), your software is not open source.
I cannot lool at Mistral’s source and see that, oh yes, it behaves this way because it was trained on this piece of data in particular - because I was not given accesa to this data.
I cannot build Mistral from scratch, because I was not given a vital piece of the recipe.
I cannot fork Mistral and create a competitor from it, because the devs specifically said they’re not providing the source because they don’t want me to.
You can keep claiming that releasing the binary makes it open source, but that’s not going to make it correct.