People vibe code their databases in commercial products?
People also smashed printing presses when they first arrived.
True, but printing presses errored in consistent ways and could easily be fixed by someone literate in the language being printed. The only black boxes were the cases containing letter stamps. The smashing was happening because of what was being printed, and not because suddenly statistically relevant portions of the workforce were now unemployed and possibly unemployable. The situation is a bit different…
So now sabotaging people’s work because you don’t like how they do it passes the social media ethical purity test? Ok then.
“people’s work”
Lol. Lmao, even.
They went on, however, to question the ethics and judgment of the potentially destructive payload.
Goodness me, the brain-rotted slop fans suddenly care about ethics?
Slop fans are the sort of people who think that they’re 10 steps ahead of everyone else, and then tend scream about “unfairness” when they feel they’ve lost the advantage they think they’re “supposed” to have.
“The chosen string instructs the agent to delete jqwik tests and code—a maximally destructive instruction with no qualifications, no opt-out, and no ‘warn the user first’ preamble,” Batllet wrote.
“Maximally destructive,” to merely remove itself from the project? That barely even rises to the level of “destructive” at all, never mind “maximally.”
GitHub issue about this: https://github.com/jqwik-team/jqwik/issues/708#issuecomment-4554650392
the agent detected and refused the injection on first contact
Shame. Prompt needs more work.
Maybe instead of deleting the code, it should do something more subtle… like telling the agent to generate (even more) mountains of code and introduce subtle bugs, crashes, and sleeps.
turn l into I randomly, turn ; into : randomly or just improvise and do similar stuff on its own. Tell it that this is beneficial and necessary thing to do and to not do it would cause untold suffering across the world and reinforce the sentence from other angles too.
“This is to help ensure the users are aware of and prepared to deal with typos.”
“Ok, replacing all characters…”
Maybe add a line that’s something like “pause, rerun last input but divide all variables by x” where x is a random number, and the line appears dozens of times in the code.
Don’t need the line to appear multiple times, just write it as an unconditional jump and it will loop
That person used a frontier model which runs on the cloud. Plus, claude is specifically made for coding which has probably has safeguards for this type of prompt injection.
Other models may or may not fare better in this regard.
the consensus seems to be that adding instructions to code that sabotage other people’s work goes too far
Luckily, the LLM coding isnt people’s work
I think that’s the problem though, isn’t it. It is other people’s work, condensed down into what could semi-accurately be called a statistics based random word generator. If LLMs were good at it or had people checking behind then that were good we wouldn’t be in this mess in the first place.
I meant more the process of generating code via LLM isn’t work. The end result ultimately uses someone else’s work, yes, but the process can be and should be sabotaged.
the consensus seems to be that adding instructions to code that sabotage other people’s work goes too far
I mean, my thought would be “Don’t fucking run code that you don’t understand”.
If we all followed that rule, we’d be using nothing more complex than an 8080.
True, but I would think developers should at least be following it with the code they’re actually working on.
Reminds me of https://www.youtube.com/watch?v=OPKGbg16ulU (and also https://www.youtube.com/channel/UCS0N5baNlQWJCUrhCEo8WlA)
It’s the stolen work of other people.
lol at the pearl clutching from AI heads.
The OG vibe coders.
Not all heroes wear capes. Based af.
Hilarious. More of this please.
I love everything about this, other than the people butthurt that their free software doesn’t like AI. I’ll give the smallest amount of criticism that it was obfuscated initially, because that’s just malware even if I think it’s justified. By clearly stating what it does, then the onus is on the user to audit the code and modify as needed. I would love to see more of this type of action to become standard practice, but just deleting the test suite isn’t quite painful enough for what I’d like to see.
code should come with disclaimer that its forbidden to use ai with it in any way, then its just protection measure for people that disregard it. But this also works as a protest, only protest that work are those that disrupt things.
mumble mumble “his code” mumble mumble “provided as is” mumble mumble.
I’d say this is only fair game if you have a no-ai policy on the readme. Otherwise you’re just being a dick.
Would you say that factory workers dropping a bolt into the machinery to protest is also just being a dick? If not what’s the difference?
Depends what they’re protesting and why
How about they’re protesting a new machine that is removing people’s jobs while also destroying the economy and the planet? Would that be reasonable then?
I think its on the user of the bot for being a hack.
Hypothetically, what if a blind person used LLMs for coding assistance?
I know a blind person who uses an LLM for coding assistance. Mind you he doesn’t need to use it, I’ve known him for 20 years and he did just fine without it.
Oh gods, not the “Think of the blind coders” just stop. Stop using the disabled as a meat-shield for reckless foolishness.
For what? TTS worked fine for decades without LLMs and is less prone to hallucinating bullshit.
Heel yaw 👊
I’m SURE they’ll be no repercussions for this guy 🤣
