In the 1980s, economist Robert Solow made an observation that reminded economists of today’s AI boom: “You can see the computer age everywhere but in the productivity statistics.”
In that scenario where AI is used to find specific code snippets or other matching text blocks, the false positives aren’t really the problem. The false negatives are the issue.
I’ve run into that myself a few times when trying to use AI. You give it a very clear prompt to find something and it sometimes just falls flat on its face. It’s easy for the AI evangelists to just blame the human who wrote the prompt, or say “you didn’t give it enough context!” But anyone who’s tried using AI and is being objective about it will tell you that’s a weak excuse that doesn’t hold water a good chunk of the time. You can give it plenty of context, and be very clear, and it still doesn’t find all the examples that clearly match the prompt.
Ultimately, you often have better luck using a well-crafted regular expression to search for text than using AI.
And that seems like the crux of the issue (which you also highlight). While there are some very good use cases for AI, it’s being waaaay over-used. And too often its faults are dismissed or glossed over.
In that scenario where AI is used to find specific code snippets or other matching text blocks, the false positives aren’t really the problem. The false negatives are the issue.
I’ve run into that myself a few times when trying to use AI. You give it a very clear prompt to find something and it sometimes just falls flat on its face. It’s easy for the AI evangelists to just blame the human who wrote the prompt, or say “you didn’t give it enough context!” But anyone who’s tried using AI and is being objective about it will tell you that’s a weak excuse that doesn’t hold water a good chunk of the time. You can give it plenty of context, and be very clear, and it still doesn’t find all the examples that clearly match the prompt.
Ultimately, you often have better luck using a well-crafted regular expression to search for text than using AI.
And that seems like the crux of the issue (which you also highlight). While there are some very good use cases for AI, it’s being waaaay over-used. And too often its faults are dismissed or glossed over.