Although there are a few ways to mitigate the risk, the only way to block it is to get AI to differentiate instructions from data, which is impossible today.
Just a hundred years ago, we had one floating point operation per second in mechanical calculators. We got roughly 100 TFlops (fp16; probably much more accurate than human brain neurons) in a high-end gaming GPU now. That’s 14 orders of magnitudes in a hundred years. I guess, 4 more orders of magnitude in another hundred years should be possible. We don’t even have fusion power yet. A lot changes when power becomes a practically unlimited resource.
Just a hundred years ago, we had one floating point operation per second in mechanical calculators. We got roughly 100 TFlops (fp16; probably much more accurate than human brain neurons) in a high-end gaming GPU now. That’s 14 orders of magnitudes in a hundred years. I guess, 4 more orders of magnitude in another hundred years should be possible. We don’t even have fusion power yet. A lot changes when power becomes a practically unlimited resource.