In a study published in the journal Judgment and Decision Making, 1,682 adults were asked to read one of six short stories, three of which were written by humans and three by ChatGPT. Each AI story had a similar theme to one of the human-authored works.
The team told participants whether their story was written by a human or AI, but this information was not always correct. The researchers then asked participants to rate how absorbing and engaging they found the story, and its quality.
Participants who read an AI-generated story rated it as more absorbing and of higher quality than those who read a story written by a human. However, participants gave higher ratings to stories they had been told were written by people.
Dr Deena Skolnick Weisberg, a senior author of the research from Villanova University in Pennsylvania, said: “AI systems can already generate short stories that are seen as being at least as good as – if not better than – human-written stories. We should update our views of AI’s abilities accordingly.”
However, Weisberg, who is herself a creative writer, said that did not mean writing should be left to AI, noting that novels produced by tech were probably going to be different from those written by humans.
“We may need to make room for AI-generated novels, and for AI/human co-written novels, but that doesn’t mean that there’s no longer space for us to appreciate the process of human creativity,” she said.
Here are the stories they used, if you want to decide for yourself. Although it won’t be a blind test since in these documents they tell you which is which:
Story pair 1
Story pair 2
Story pair 3
Having read the compared stories now, I have to say I’m deeply offended at the results. The AI stories are quicker to read because they are chains of cliches which state and restate the theme of the story (as stated in the prompt) outright instead of demonstrating it through the charged and personal perspectives of the characters. The LLM generated text is meaningless word sausage formatted to be easier to read for people who don’t read much. I have never read something more cynical, and I have read plenty of trash by authors who hate their audience and consider them all to be idiots.
Most people are dumb and are functionally illiterate. Of course the incredibly dumbed down text will appeal to them more.
You nailed it. For me, the human-written stories were interesting because it’s not immediately apparent where they’re going. The prose evokes images and emotion, and I want to put the pieces together to identify the theme.
The AI stories blurt out the theme almost immediately, and then my attention wanes because there’s nothing left in the text except for a base retelling of events.
I like how story pair #1’s AI just took the prompt and basically kept repeating “yeah, them’s fish” over and over without any attempt at significance
I love when someone has already answered my question in the comments.