Seems kind of impractical, why even bother with trying to propagate a 4b model running at 0.39 tokens per second if you could instead make a conventional worm using a more powerful AI model targeting known devices? It seems like the only reason you would want to do this is if you can only find vulnerabilities while actually on the given network, but that would be kind of niche since how many people are really running totally custom stuff?
Yeah the actually scary kind of AI worm is one that scrapes dozens to hundreds of credentials people accidentally pushed to GitHub and spreads its API requests across all of them to minimize the chance of being noticed. Not something that requires downloading 5GB+ of LLM to someone’s potato ass hardware and running it at half a token per second.
Yeah, hard to get local LLMs running decently fast even when done deliberately. Like people aren’t going to notice their CPU, memory, and GPU are all getting pegged when this malware runs?
Come on, Anthropic, you need to do better than this if you want your “open LLMs dangerous” marketing campaign to be credible.
Many of these IoT devices have incredibly poor security and if they’re connected to the internet can be hacked. The bigger models are very good at finding exploits and using them as well, so it could search for devices and break into vulnerable ones then send something to self replicate and start process.
The bigger models might exploit the parents company as well and get in that way. E.g hack a smart security camera company and then find a way into the cameras.
It doesn’t even have to be only for known devices. Once you’ve compromised one device, you can just phone home to have a full powered LLM do your work.
The only real use cases for this would be on closed networks with no internet access, or networks where the traffic is closely monitored. Like, yeah, if I’m trying to hack the CIA, maybe I consider something like this. For the vast majority of devices and networks, this is inefficient and unnecessary.
I mean, it is kind of cool. But vanishingly few organizations have tight enough security that this is worth doing.
Seems kind of impractical, why even bother with trying to propagate a 4b model running at 0.39 tokens per second if you could instead make a conventional worm using a more powerful AI model targeting known devices? It seems like the only reason you would want to do this is if you can only find vulnerabilities while actually on the given network, but that would be kind of niche since how many people are really running totally custom stuff?
Yeah the actually scary kind of AI worm is one that scrapes dozens to hundreds of credentials people accidentally pushed to GitHub and spreads its API requests across all of them to minimize the chance of being noticed. Not something that requires downloading 5GB+ of LLM to someone’s potato ass hardware and running it at half a token per second.
Yeah, hard to get local LLMs running decently fast even when done deliberately. Like people aren’t going to notice their CPU, memory, and GPU are all getting pegged when this malware runs?
Come on, Anthropic, you need to do better than this if you want your “open LLMs dangerous” marketing campaign to be credible.
You’re going to notice your fridge is running at max processing?
But how do they access IoT devices without access to the WiFi they’re connected on?
I guess the argument is most IoTs can be accessed through open external IP ports?
Many of these IoT devices have incredibly poor security and if they’re connected to the internet can be hacked. The bigger models are very good at finding exploits and using them as well, so it could search for devices and break into vulnerable ones then send something to self replicate and start process.
The bigger models might exploit the parents company as well and get in that way. E.g hack a smart security camera company and then find a way into the cameras.
It doesn’t even have to be only for known devices. Once you’ve compromised one device, you can just phone home to have a full powered LLM do your work.
The only real use cases for this would be on closed networks with no internet access, or networks where the traffic is closely monitored. Like, yeah, if I’m trying to hack the CIA, maybe I consider something like this. For the vast majority of devices and networks, this is inefficient and unnecessary.
I mean, it is kind of cool. But vanishingly few organizations have tight enough security that this is worth doing.
And if the traffic is closely monitored, you’d think they’d also notice all their cpu and ram suddenly being used for some mysterious purpose