Do LLMs “think” in a similar way to humans? Or is it totally different?
Maybe it’s a good idea to listen to someone who publishes papers on this very subject, and is a professor of both philosophy and psychiatry and directs an Institute for Cognitive Science. That person is Dr. Chandra Sripada and his insights are fascinating.
Sean Carroll (interviewer, scientist and science communicator) says this interview made him lean towards the answer being “yes, they think like humans” whereas previously he favored the opposite view.
I call BS. There is no sort of cognition taking place in an LLM. They calculate statistics, that’s all. No more magic than the cash register at the supermarket.
The difference between the cash register and an inference engine is the training data. Also, barring something like mechanical failure, I would trust the calculation of a cash register (that has previously been demonstrated to function with high fidelity, and barring any major incidents such as dropping it onto the floor that might make me question the relevance of those prior tests to its current performance).
But the results of a LLM “calculation” - and I mean this with total sincerity - might be something like:
“What is 1+1=”?
“Answer: 6, 7!! 🤪”
Or in ye olden times, one expected response might be “ur mother!” or some other flippant remark… exactly like a small child, imitating others might do. Monkey see, monkey do => the principles undergirding LLM technology? It does not understand what it sees, hence it does not “know” when to apply what answer, only going by the most popular (whatever the weighting scheme is - probably highest upvoted answer on Reddit?) string of words that seem to be associated with the question.
Can you prove that humans don’t?
This, this this. This is something the prevailing armchair opinion is really missing. Their blatantly-wrong assumptions about human cognition.
It feels as proudly ignorant as my grandmother’s generation who proudly declared Evolution to be patently ridiculous (“my grerat-gramma was no monkey swingin’ from a tree!”)
I am not a researcher in the area, but I am the kind of nerd who watches university colloquia and sometimes does in fact read the paper.
“I don’t hallucinate all the time”
YES, YES YOU DO, do you not understand how the sensory system works? You interpolate between inputs. This is why optical illusions work! Your fovea is a tiny tiny spot and is the only area that’s actually in focus!
Learning by statistical sampling, reward/repress feedback loops, production system-based computation giving rise to “fast” (System I) thinking, “hallucinating” to interpolate between sensory inputs … the list goes on and on
If i could prove it, it would be proof that we do. Our brains aren’t logic machines, they’re a wet mess where cognition emerges irrationally.
So you’re going by the vibes then? Why even bother with your comment? What basis in neuroscience is this founded in?
Neurons are in fact scochastic get over it.
It’s “stochastic”, if you want to be pedantic, and I get the feeling that’s all you’re here for. “In fact”.
The fact that all you can do is reply to a typo is exactly the issue, there’s nothing of substance here
Humans are largely too & tbh it’s a rather weak argument these days, but it’s also reinforcing the opposite view, because you are being really stochastic with this comment itself.
What a dumb question. “Cognition” is an abstract concept that just leads to debate over its definition, except that it’s commonly understood that humans have cognition.
This entire interview also seems predicated on the belief that neural networks work fundamentally the same as human brains, which is an oversimplification of brains. To begin with, neural networks cannot train themselves at inference time (context windows aren’t training). Also, humanity has barely a concept of how the brain works and regularly learns new things about it. What we do know is that they are more complex than a bunch of connected neurons, and that we have no way of modelling something we don’t yet fully understand.
So do neural networks have cognition? If they did, then we’ve had cognitive AIs for longer than I’ve been alive.
The rest of the interview is imaginative fiction based on what they think the model is doing predicated on what Anthropic likes to advertise their models do.
As basic evidence, if I tell you that the word “strawberry” has 3 "r"s, you can remember that. You can recall that no matter how many conversations we have in between when I told you that and when I asked you again.
If I teach you how to count how many "r"s there are in the word “strawberry”, you learn a skill. In the future, if I ask how many "r"s there are in the word “strawberry”, you can count them using your new skill.
If I teach a LLM how many there are or how to count the letters, it can only use that skill or knowlege if it can effectively search for it. Its ability to execute skills (which, to begin with, isn’t it executing the skills but it asking something else to do them) is diminished the more skills it gains access to, and its “knowlege” (aka long-term storage) becomes diluted when it has access to more of it.
This is blatantly untrue of humans, who can process knowlege and skills they learn, efficiently and quickly search them, and actually get faster and more accurate as they develop related skills and learn related knowlege.
Your example falls flat on two counts
Llm’s that only have one glyph per token can easily count this, and yeah we train in bursts because it’s computationally cheaper, this hardly means anything important.
[Citation needed]
There’s no difference between counting tokens containing the letter “r” and tokens solely representing the letter “r”. To begin with, this assumes the way we encode words in our brain is by tokenizing each letter, which we pretty much know to be false (we can process words without seeing evry ltr of th wrd and even think about them without it being mentioned).
No, this is not how neural networks work at all.
We train in bursts because we need a ground truth/scoring function to guide the learning, which means we need a way to guide the model in the right direction. Even with unsupervised learning, the model has a way to determine whether it’s going in the right direction, like with GANs.
Neural networks are also incapable of synthesizing completely new skills from its own weights. Humans have been doing that for the entirety of human history.
These are completely orthogonal things.
We train in bursts (“epochs”) because empirically it leads to more rapid convergence on an acceptable error minimum.
The fact that a scoring function is needed is almost a tautology. You have to have some differentiable function to say what direction is lower error. Typically you segment your training corpus, and use the scoring function to get the error for one batch at a time to a certain threshold before proceeding to the next batch. Then go to the first batch and push the error bar even lower, and so on.
Kinda like tightening nuts on a car wheel. You don’t fully tighten the first one first; you have to wiggle the system into its final configuration.
We’re comparing this to the observable behaviors of humans, though. Humanity engages in unsupervised learning without a scoring function (let alone a differentiable one) and, as a result, can not only train on the fly, but can also find its own ways to self-evaluate its learning. Humans also have hormones which change how the brain works even further.
This is also all disregarding the ability for the human mind to become faster and more accurate on recall when it has more knowledge to search in a field. LLMs can’t recall very effectively, instead becoming less effective the more information it needs to search.