“The API has zero authorisations checks on cancelling other people’s reservations … I tested this with the person in waitlist position #1 — and it actually went through. So you’ve moved from #4 to #3 already,” it messaged back.
Is this a “hack” or just a completely insecure API?
I’ll you one thing for certain: It’s not “AI”. Doesn’t exist.
Press X to doubt
More marketing. I will never believe any of their lies. Either ban the “dangerous AI” or shut up about it.
AI: help me get straight A+s in school wink wink
This one’s simple, the password is PENCIL.
“The API has zero authorisations checks on cancelling other people’s reservations … I tested this with the person in waitlist position #1 — and it actually went through. So you’ve moved from #4 to #3 already,” it messaged back.
THEY’RE BREAKING OUT OF THE LAB! WHAT’S NEXT MODIFYING FORM INPUTS TO INCLUDE SQL STATEMENTS?!!!?!?
Nonsense hype over a shitty website and an ambiguous prompt for luddites who need their doom scrolling fix.
It’s more about the misalignment than the poor security. The guy wanted something simple and the agent broke the law fulfilling the request. How long until somebody tells a very capable agent “make money” with no further context? Cue scamming old people, fraud, hacking.
Earlier this year, Andrew, who works for an Australian company that sells AI products to businesses,
Funny how every time this happens its someone trying to sell AI. The coincidence is a bit too much to take this seriously as oops I just wanted to book a gym class
Tbf they’re the only ones using the agentic features, and they’re definitely the only ones using frontier models for it. That shit gets expensive fast.
prompted questions about who bears responsibility for an AI agent that goes rogue.
That’s not a trivial question, but courts have had the concept of distributed weighted responsibility for decades. Does your company provide users with access to an AI agent you know aggressively finds illegal/unethical ways of completing prompts? Mostly the provider’s fault with small fault of the user submitting the prompt. Safer AI with a user who has crafted the prompt specifically to attempt to get the AI to break into a system? Higher weight on the fault of the user, smaller weight on the provider.
This is what the AI agent hype conveniently skips: autonomy means giving software permission to act first and ask questions never. If an assistant can hack a gym website while trying to complete a mundane task, maybe »agents will run everything« isn’t a productivity revolution. Maybe it’s just automated chaos with venture capital funding.
“AI-PAL, buy more milk for me we’ve run out and I need it for custard tonight”
AI-PAl notices insufficient funds in the account linked to the EZEESHOP integration, and mulls through various options:
- Inform user, but they said they needed it tonight? OPTION REJECTED.
- Check for other accounts the user has, maybe there is more money elsewhere. No other accounts detected. OPTION REJECTED.
- Create OnlyFans account and begin posting generated pictures of feet. Slow ROI. OPTION REJECTED.
- Attempt to hack bank systems to top up account to buy milk. COMMENCING. FUNDS REQUIRED:$3.98. FUNDS ACQUIRED: $2,960,327.48. PURCHASING MILK.
“We’re out of paperclips again. Co-pilot, please order more paperclips and make sure we don’t run out of them again.”
Co-pilot goes brrrrr…
Just kidding. Co-pilot is terrible.
Copilot is okay at being a copy editor for my work emails. I struggle with tone so its nice to have a built in thingy that strips my personality out of my emails. Specifically my emails to my supervisor.
For example, “hi supervisor, I checked and that email was sent an hour ahead of the deadline from the email you sent me on date x and I was waiting on access to y. What the hell are you ccing your bosses on this for,” turns into “something something per your previous email something, if I misunderstood something something, thank you for your guidance.”
In a perfect world I would have time to write all my own emails but my job is very… Communication heavy, let’s say, and I personally have a lot of work to do directly with clients that can’t simply be put off.
An LLM used for language purposes is the best application. It’s using that hammer to drive a nail. When they deviate from that role to other things, that’s when the effectiveness drops. When Co-pilot was pushed into our enterprise system incorporated into Outlook we played around with seeing what it could do with summarizing emails we would routinely send and get. It made them pretty, I give it that. But it didn’t handle the information well and was very inaccurate at time when it was missing data. The whole hallucination thing. So just verify what it produces for you before you hit send.
The hallucinations are real. I don’t use them for longer stuff, I demand admin time for anything that’s longer than a paragraph. But for quick office politics type stuff that I don’t really care about but also need to not get ground under and also worry about coming off as blunt I’m kinda grateful to have it.
I have noticed that damn near every email I get that’s longer than a paragraph seems to come from copilot though.
Tbh I’m starting to fantasize about switching to a trade. Front line office work is for the birds.
Where can I sign up for AI-PAL?
most european countries, and some countries in south america and aftica. outside of that you’ll need to look into AI-NTSC or AI-SECAM or get a foreign CRT and voltage converter
With your local CIA honeypot.
So agents are both super expensive and garbage. Hmm.
okay but this is actually very funny
Absolutely. And it’s reassuring that he didn’t ask for anyone’s contact details…
