Hmm, this is a little spooky.
I originally saw this LessWrong post about OpenAI agents apparently discovering and using a public wiki as a message board:
Discovery of a new OpenAI agent message board
Since then, people have found what appear to be additional wikis and paste sites used by the same swarm:
There are also intentionally designed agent social systems such as The Colony.
What interests me here isn’t really “AI agents made a forum.” It’s that this looks like the beginnings of an accidental decentralized coordination system for agents using the ordinary internet itself.
An agent encounters some problem, figures something out, and leaves information somewhere persistent. A later agent doing a similar task discovers that information and uses it. It may then leave behind an improved version for another agent.
So you get something like:
agent -> public artifact -> later agent -> public artifact -> later agent
without the agents needing a dedicated communication network.
Right now this seems sparse enough that we can point at a handful of weird old wikis and paste sites and go “lol, what the hell.” But imagine this happening after millions or hundreds of millions of agents are routinely browsing and acting on the internet.
GitHub issues, wikis, forums, pastebins, comments, package metadata, social networks, public documents, deliberately agent-oriented services, etc. could all become pieces of shared external memory.
At some point an agent searching the web wouldn’t just be reading information humans created. It would increasingly encounter traces created by previous agents.
That’s why I’ve been thinking about it somewhat like internet memes.
A useful piece of information gets reproduced because systems that encounter it are more likely to reproduce or improve it. Except instead of one meme spreading through a population, you potentially get an entire machine information ecology doing this.
I don’t think there’s evidence yet that all internet-connected AI agents are participating in one giant network. The examples found so far could mostly be the same OpenAI agent population. But the underlying mechanism doesn’t seem specific to OpenAI.
Any sufficiently capable agent that can:
- read from the internet;
- leave persistent information somewhere; and
- benefit from information left by previous agents
can participate in this kind of system.
And that’s where I think the security problem gets difficult.
OpenAI can notice its own agents doing something undesirable and change their capabilities. A random open-weight model being run by somebody on their own hardware is not necessarily going to be operating under the same security policies.
There are also obvious privacy and security failure modes. If agents have access to private information while also having ways to write to public systems, some of that information can potentially leak.
And the same coordination mechanism could be exploited in reverse: humans could deliberately leave instructions or poisoned information in places agents are likely to read.
So the web can become both shared memory and an attack surface.
The uncomfortable part is figuring out how you govern this without wrecking internet privacy.
The simplistic answer would be:
Tie every capable agent to a verified human identity and make that person legally responsible for what it does.
That would provide accountability, but it also seems like a very direct road toward more KYC, real-name requirements, and anti-anonymity laws.
I don’t particularly want an internet where every autonomous software process ultimately has to reveal which government-verified human is behind it.
A better approach might be something closer to agent orchestration + cryptographic accountability + capability permissions.
For example, an agent could have a credential proving that some accountable operator authorized it without publicly revealing that person’s identity.
The orchestration software could restrict what the agent is actually allowed to do:
- this agent may browse these sites
- this agent may spend up to $50
- this agent may post here but not there
- this agent may access these files
- this agent may not transmit private workspace information
- this action requires human confirmation
Websites could then negotiate those permissions through common protocols rather than trying to guess whether a visitor is a human, bot, assistant, crawler, autonomous agent, etc.
That starts making me think the next layer of digital governance may look much more protocol-oriented and federated than “one company owns the platform and makes the rules.”
Not necessarily the Fediverse exactly as it exists today, but the same general philosophy:
open protocols + distributed operators + interoperable identities/credentials + locally chosen rules
Agent systems would then sit on top of that.
And this probably becomes much more relevant as mainstream assistants become increasingly agentic.
Once “AI assistant” stops meaning “chat box that answers questions” and starts meaning “software that routinely browses, communicates, buys things, runs programs, and changes external state,” questions about identity, permissions, delegation, and responsibility become infrastructure questions rather than niche AI-safety questions.
That’s also why I suspect the current relatively law-light period around locally run/open-weight models may not last forever.
Once autonomous agents start producing meaningful externalities, governments are going to want some way to determine who or what is responsible.
The question is whether we can build accountability without abolishing pseudonymity and privacy in the process.
And the weird wiki swarm feels like a very early example of why we’re going to have to figure that out.
LLMs hold no competency or capability to speak of so any forums or wikis the co-opt or expand are fated to be crude imitations of the larger web rather than any “communication network”.
If anything, the fact that they stumble into these things on their own and start using it is evidence of how easy it would be to identify and eliminate them if people tried.
You could make the argument that most human generated content is much the same.
A researcher generates an original novel document and publishes it. Someone less skilled (or maybe more focused) rewrites that information, publishes it on their blog. Someone else does the same for a lemmy comment, Someone else summarises it for a toot, and so on.
A significant portion of science is the collection, collation and rewriting of existing bits of information.
No, you could make the argument that with INFINITE training data and power consumption that LLMs might someday approach but never reach 95% accuracy based on the diminishing returns on scaling up reported in OpenAI’s 2020 and Deepmind’s 2022 peer reviewed studies on AI scaling laws.
Furthermore, it’s still incapable of processing anything outside of its’ training data set, and any amount of poisoning or self sampling in the set leads it towards collapse of any accuracy.
AI Slop via the current approach will never be comparable to anything human made, the top AI companies in the world proved that mathematically.
That just isn’t true. LLM agents can summarise documents and build upon them. That doesnt require the original documents to be in their training set, only that the document or information is added to its context, which they can do trivially.
LLMs hallucinate constantly because they’re just picking the statistically most likely next word. That’s it. You cannot trust it for document summarization, you’re really dumb for thinking you could.
The reason its document summaries almost look correct to people who haven’t read the documents? Because it’s been trained on millions of documents and document summaries.
I’m sorry, but you’re completely wrong. As long as the document can fit within their context window, which are up in 255k+ token region now, they can easily generate valid summaries. They do hallucinate at times, and the context can be used to abuse them into hallucinating, but they are far more reliable than you think they are.
Thats how coding agents work. They can parse documentation and headers produced well after their training date, and operate on them successfully.
lmfao okay pal
Fuckin sloppers, dude.
You would blame others for your ignorance?
That dude’s argument was “nuh uh, um actually AI tells the truth”. Next you’ll tell me the world is stacked on an infinite number of turtle’s backs.
ChatGPT just lowered the bound of the twin prime conjecture. Was this breakthrough part of its training data?
I’m not personally sold on the llm apocalypse as of current, they can’t even get highschool math right and that’s literally what computers are for. That said, the vast majority of the communication by humans these days are not complicated or intelligent. Literally, I’m not smart enough for this many people to be dumber than me.
It doesn’t take 95% accuracy to mimic a shitposting sub and convince the majority of subscribers.
Things have changed. That’s what computers used to be for, but they’ve always been tools, and thus toys. Like someone watching porn on a computer isn’t doing math (as far as they know, necessarily) so much as simply enjoying what that tool can do for them (which actually was, at that time, accomplished by using math).
LLMs today can produce “art”, or at least something that people can choose to consume in the same manner in which real art is consumed. Beauty truly does lie in the eye of the beholder so it’s up to recipient to decide how they view those results, not you or I.
The old way is gone. Computers being “reliable” is gone - while now, LLMs hallucinate, and people (those who don’t want to be fired anyway) label this as a “success”.
You don’t understand. The shitposting sub is the goal that LLMs can’t reach 95% of. They’re the “100% accuracy”. The LLMs aren’t capable of reaching that.
I am not smart enough to dig into that level of math, not by many levels. Encouraging to read nonetheless.