I’m currently taking a beginners python course and one of the chapter project examples breaks down basic sentence structure based on parts like nouns, prepositions, etc. The idea is to make several dictionaries based on these sentence parts with different words associated with them and then randomly pick words from them in the right order to make a cohesive (although not always intelligible) sentence. I figure most LLMs use a much bigger version of that with even more defined dictionaries to better encapsulate certain responses based on the topic.
There’s nothing in that kind of program structure that would even come close to implying it’s alive. LLMs are just a fancy version of Mad Libs.
I recommend trying to make your own Markov chain based chatbot. You can produce grammaticality correct nonsense just by using raw probability and without even attempting to categorize words as nouns or verbs. All you need is a dataset to build your probabilistic model… Sounds familiar?
I’m not a CS major and I don’t know exactly how LLMs work but I think you will find Markov chains interesting and highly relevant.
I’m currently taking a beginners python course and one of the chapter project examples breaks down basic sentence structure based on parts like nouns, prepositions, etc. The idea is to make several dictionaries based on these sentence parts with different words associated with them and then randomly pick words from them in the right order to make a cohesive (although not always intelligible) sentence. I figure most LLMs use a much bigger version of that with even more defined dictionaries to better encapsulate certain responses based on the topic.
There’s nothing in that kind of program structure that would even come close to implying it’s alive. LLMs are just a fancy version of Mad Libs.
With LLM, the dictionaries come from the training data, and training also says what’s the most probable link between a word and the next word.
I recommend trying to make your own Markov chain based chatbot. You can produce grammaticality correct nonsense just by using raw probability and without even attempting to categorize words as nouns or verbs. All you need is a dataset to build your probabilistic model… Sounds familiar?
I’m not a CS major and I don’t know exactly how LLMs work but I think you will find Markov chains interesting and highly relevant.
The sad part is how easily people are blinded. The emperors new clothes is really the best story there is.
Even worse if it’s malicious of course, weaponizing this.
It’s alive! It fighting for survival! It’s doing things on its own! Sigh
We have had better games doing this.
we had better video games doing this in the 90s