Superhuman Intelligence might be a threat if any of the companies developing AI right now had any way that their word guessing programs could possibly simulate consciousness. The best argument I’ve heard so far that they could is “no one understands how LLMs work, and no one understands how the human brain works, so you can’t prove they’re not the same.”

Talking about superhuman intelligence now, and fretting and worrying about what we’re going to do in the future where AI is smart enough to be dangerous, ignores all of the ways AI is dangerous now, like giving wronf answers, making it almost impossible to find useful information on the internet because sites that have it are buried under AI-generated SEO slop that’s wrong half the time, consuming way more electricity and water than municipalities can afford to supply them with, the datacenter heat island effect, carbon emissions and other forms of deadly pollution from brand new power plants for AI datacenters, to say nothing of all the jobs that are being destroyed by bosses that think their customers won’t notice the change in product quality.