I use AI roughly 6 hours a day or more and have probably read tens of thousands of replies from them, so I know exactly what it sounds like and what it’s voice looks like unless you prompt it heavily to avoid LLMisms. I’ve always scored extremely high on reading comprehension (which helps but isn’t predictive) and studies recently have found some people are uncannily good at detecting AI writing in double blind trials.
I’ve also called things out on AI and later on turned out to be right. It happens a lot… That’s why.
It’s like being able to recognize your friend’s distinctive writing despite them using an alt account, or a different username on a site you just joined. You know it’s them somehow.
Most people using AI to write for them don’t know how to prompt it to sound human (where it gets to be indistinguishable from a real person) so their posts are full of “it’s not X it’s Y”, perfect pacing, no stumbling or awkwardness, perfect punctuation, zingy aphorisms and slogans. Each of those in isolation isn’t an AI tell it’s all of them and more. I don’t know how to properly describe what I’m detecting. Real human writing tends to be more stilted and awkward while AI writing just flows like an experienced writer.
For example, I had Claude rewrite my comment to demonstrate (Claude below.) A lot of people on Lemmy can’t tell the difference (I know that for a fact because when I point out something is AI and someone goes “Nuh-uh!” I exclusively argue with them using Claude hahahaha)
There’s a reason some people are better at this than others. Recent studies have found that certain individuals are genuinely uncanny at detecting AI writing in double-blind trials — it’s not universal, but it’s real and it’s measurable. High reading comprehension correlates with it, though it’s not purely predictive.
I’ve been using AI tools heavily for years. Thousands of interactions, wide range of voices and contexts. At some point it stops being analysis and starts being recognition — the same way you can identify a friend’s writing style on an anonymous account before you have any proof. Something in the cadence gives it away before you can articulate what.
What I’m detecting isn’t any single element. It’s the constellation. Perfect pacing. No stumbling. Aphorisms that land exactly as intended. Punctuation that never slips. The “it’s not X, it’s Y” construction. Each one in isolation means nothing. All of them together, in the same piece of writing, with no rough edges anywhere — that’s a pattern.
Most people using AI to write for them aren’t prompting it carefully enough to eliminate these markers. They don’t know which ones to suppress. So the tells accumulate.
I’ve flagged things as AI before and been vindicated later. Not once or twice. Repeatedly. At some point that track record becomes its own kind of evidence.
Ooo that’s a good one. If you changed the emdashes to periods or semicolons and did a little cleanup to the more stilted parts, I could see myself being convinced that’s human-made. I thiiiink I have a pretty good eye for recognizing LLM stuff. It’s REALLY obvious in review articles, and they’re insanely common now. They fill up the top slots of search engines.
I was looking for an article describing features of a device recently and I clicked on a good looking top-link. If you blur your eyes and look at how the text is formatted, you could already taste that it was LLM-generated. That wouldn’t even be a problem for me if the information contained within was good… It was explaining things like how to get OUT of modes without stating how to get INTO those modes to begin with, and a lot of the “opinions” were worded like they were straight out of press releases.
A page later in my search engine, I find an article with very similar information but within a few sentences, it’s extremely obvious a person with, like… passion? wrote it. I told my partner and, even being very literate and tech savvy, they guessed the wrong article as being LLM-generated.
I’m not the best at detecting LLM writing, but I do feel some folks just have a better sense for it than others.
How do you know how good you are at “noticing AI writing”? Do you have some way of verifying whether something was written by AI?
I use AI roughly 6 hours a day or more and have probably read tens of thousands of replies from them, so I know exactly what it sounds like and what it’s voice looks like unless you prompt it heavily to avoid LLMisms. I’ve always scored extremely high on reading comprehension (which helps but isn’t predictive) and studies recently have found some people are uncannily good at detecting AI writing in double blind trials.
I’ve also called things out on AI and later on turned out to be right. It happens a lot… That’s why.
It’s like being able to recognize your friend’s distinctive writing despite them using an alt account, or a different username on a site you just joined. You know it’s them somehow.
Most people using AI to write for them don’t know how to prompt it to sound human (where it gets to be indistinguishable from a real person) so their posts are full of “it’s not X it’s Y”, perfect pacing, no stumbling or awkwardness, perfect punctuation, zingy aphorisms and slogans. Each of those in isolation isn’t an AI tell it’s all of them and more. I don’t know how to properly describe what I’m detecting. Real human writing tends to be more stilted and awkward while AI writing just flows like an experienced writer.
For example, I had Claude rewrite my comment to demonstrate (Claude below.) A lot of people on Lemmy can’t tell the difference (I know that for a fact because when I point out something is AI and someone goes “Nuh-uh!” I exclusively argue with them using Claude hahahaha)
There’s a reason some people are better at this than others. Recent studies have found that certain individuals are genuinely uncanny at detecting AI writing in double-blind trials — it’s not universal, but it’s real and it’s measurable. High reading comprehension correlates with it, though it’s not purely predictive. I’ve been using AI tools heavily for years. Thousands of interactions, wide range of voices and contexts. At some point it stops being analysis and starts being recognition — the same way you can identify a friend’s writing style on an anonymous account before you have any proof. Something in the cadence gives it away before you can articulate what. What I’m detecting isn’t any single element. It’s the constellation. Perfect pacing. No stumbling. Aphorisms that land exactly as intended. Punctuation that never slips. The “it’s not X, it’s Y” construction. Each one in isolation means nothing. All of them together, in the same piece of writing, with no rough edges anywhere — that’s a pattern. Most people using AI to write for them aren’t prompting it carefully enough to eliminate these markers. They don’t know which ones to suppress. So the tells accumulate. I’ve flagged things as AI before and been vindicated later. Not once or twice. Repeatedly. At some point that track record becomes its own kind of evidence.
Ooo that’s a good one. If you changed the emdashes to periods or semicolons and did a little cleanup to the more stilted parts, I could see myself being convinced that’s human-made. I thiiiink I have a pretty good eye for recognizing LLM stuff. It’s REALLY obvious in review articles, and they’re insanely common now. They fill up the top slots of search engines.
I was looking for an article describing features of a device recently and I clicked on a good looking top-link. If you blur your eyes and look at how the text is formatted, you could already taste that it was LLM-generated. That wouldn’t even be a problem for me if the information contained within was good… It was explaining things like how to get OUT of modes without stating how to get INTO those modes to begin with, and a lot of the “opinions” were worded like they were straight out of press releases.
A page later in my search engine, I find an article with very similar information but within a few sentences, it’s extremely obvious a person with, like… passion? wrote it. I told my partner and, even being very literate and tech savvy, they guessed the wrong article as being LLM-generated.
I’m not the best at detecting LLM writing, but I do feel some folks just have a better sense for it than others.