But if you use AI to find issues, then investigate the issue, reproduce it, document it, etc, then you’ve got something meaningful that someone can do something with.
In principle that is sound, but LLMs can’t do that, yet, if ever. Sounds like anthropomorphism. The ‘hallucination’ thing is intrinsic. What you can do is point it at easily verifiable problems like searching for security flaws, and then verify them. Whether that’s cost effective in wasted energy and human time is still an open question, especially as token cost ramps up pre IPO.
They can find issues now though. Will they also find non issues due to missing context or not understanding the whole operational picture, well yes, but you can usually filter those ones out quickly. While i wish we had QA back and it be a norm again in software engineering as a whole, without it I’m happy to have something look for ways my code will break in unexpected manners that testers previously handled. Is it the same as having someone sit their and try and break your shit for the sole purpose of finding a spot where something will go completely off the rails, no its not but with a lack of true QA and rushed timelines that leave little room for quality QA yourself. I dont mind having something else that will try to gage my code from all angels and suggest areas that might have bugs I did not intend for or did not catch when I was reviewing my code. Especially considering I can do this without making users of my code the guinea pigs in lieu of an actual QA department. Sometimes you have to make the best of a shitty situation and honestly its an improvement over no QA which has been a norm for a while now so I’ll take the small wins where I can
In principle that is sound, but LLMs can’t do that, yet, if ever. Sounds like anthropomorphism. The ‘hallucination’ thing is intrinsic. What you can do is point it at easily verifiable problems like searching for security flaws, and then verify them. Whether that’s cost effective in wasted energy and human time is still an open question, especially as token cost ramps up pre IPO.
That’s why you need to investigate and reproduce what the AI found: to verify it’s a real issue. Blindly submitting the issue is lazy noise.
They can find issues now though. Will they also find non issues due to missing context or not understanding the whole operational picture, well yes, but you can usually filter those ones out quickly. While i wish we had QA back and it be a norm again in software engineering as a whole, without it I’m happy to have something look for ways my code will break in unexpected manners that testers previously handled. Is it the same as having someone sit their and try and break your shit for the sole purpose of finding a spot where something will go completely off the rails, no its not but with a lack of true QA and rushed timelines that leave little room for quality QA yourself. I dont mind having something else that will try to gage my code from all angels and suggest areas that might have bugs I did not intend for or did not catch when I was reviewing my code. Especially considering I can do this without making users of my code the guinea pigs in lieu of an actual QA department. Sometimes you have to make the best of a shitty situation and honestly its an improvement over no QA which has been a norm for a while now so I’ll take the small wins where I can