Yes, something like that. If you use a model which is specifically trained on C(++), Assembly and Rust, as well as scientific articles about Linux and the Linux Kernel, has the whole Linux Kernel already baked in, you can probably get faster and more accurate results than if you add the whole Wikipedia in 20 languages.
My LLM doesn’t need to answer all the questions, it’s enough if it answers the ones I want to ask. AI and the (mathematical) algorithms used for it are very old in terms of computer timeframes (think the 70s to 90s). The new part with LLMs is actually in the first “L”: large datasets. I think we have gone too large.
And, with Linux, you have an issue with authorship if you train on copyrighted data. Reproducing code 1:1 from a non-Free textbook might be problematic, so special LLMs trained on Open/Libre texts will be safer in terms of compliance as well.
like preferably a model only trained on one language kind of idea?
Supposedly that makes them dumber in general
For a person I could understand. Now I need to go bother another code monkey I know that is specialized in ML.
It’s something to do with improving the reasoning capabilities, because there are more patterns in the weights that it can activate against
Yes, something like that. If you use a model which is specifically trained on C(++), Assembly and Rust, as well as scientific articles about Linux and the Linux Kernel, has the whole Linux Kernel already baked in, you can probably get faster and more accurate results than if you add the whole Wikipedia in 20 languages.
My LLM doesn’t need to answer all the questions, it’s enough if it answers the ones I want to ask. AI and the (mathematical) algorithms used for it are very old in terms of computer timeframes (think the 70s to 90s). The new part with LLMs is actually in the first “L”: large datasets. I think we have gone too large.
And, with Linux, you have an issue with authorship if you train on copyrighted data. Reproducing code 1:1 from a non-Free textbook might be problematic, so special LLMs trained on Open/Libre texts will be safer in terms of compliance as well.