I think most people have gotten over this fallacy when it comes to organic intelligence these days, but people still make it with regards to machine learning. They say LLMs are reducible to an equation (which is an oversimplification) and therefore they can’t have any subjective experience.

But if your math is complex enough to reduce an LLM to an equation (something no human has ever done, but is theoretically possible), then you’d be able to do it to humans too. Unreasonable effectiveness of mathematics, and all that. So yeah, humans are just a mathematical equation. A human brain is just a machine that predicts the most likely action to help you eat food and have sex.

And if you believe talking creatures can lack subjective experience, well congratulations, you’ve invented p-zombies. Which I think is dangerous. We don’t need millions of people walking around who seriously believe in p-zombies.