The machine learning community is finally waking up to the madness, but the detour of the last few years has been costly.
  • Mikina@programming.devEnglish
    4·
    5 days ago

    Some of this may come as news to a lot of the machine learning community

    Does it? I only have pretty basic knowledge in the ML field, from like two courses during my Masters in gamedev around 8 years ago, and I though that it’s a basic fact of most of the ML algorithms, that simply throwing more data at it won’t get it “smarter”, as in from the basic understanding of how ML works, it’s pretty apparent that you can’t get anything like an AGI with the current algorithms.

    You’re basically just approximating a function (which is my understanding of what ML does) of what’s the next word based on previous senteces, your dataset. It kind of makes sense it would converge into absolute mediocrity (not even mediocity, because a lot of data in the datasets is very probably wrong), and not be able to come up with new things.

    But, we’ve never really learned about transformers, since that tech wasn’t yet part of our syllabus, so I might be wrong/overly simplyfing things.