I feel like a big problem is that LLMs only produce previously learned logical patterns, they don’t do logical reasoning of their own. And code review was always bad at catching logic bugs as well.
As a result, even with us diligently doing code reviews at $DAYJOB, we have more logic bugs than I have ever seen in any codebase before.
I’ll spend three weeks discussing an existing feature with other devs and fixing up logic bugs before we realize that the entire feature is conceptually illogical and needs to be removed/replaced.
This has happened twice for me in the past three months, which should tell you a lot about how much time is wasted here.
TL;DR: I feel like we need something different than code review to catch these logic bugs. Maybe more of a presentation of the feature implementation to the other devs, and then a discussion of that.
I feel like a big problem is that LLMs only produce previously learned logical patterns, they don’t do logical reasoning of their own. And code review was always bad at catching logic bugs as well.
As a result, even with us diligently doing code reviews at $DAYJOB, we have more logic bugs than I have ever seen in any codebase before.
I’ll spend three weeks discussing an existing feature with other devs and fixing up logic bugs before we realize that the entire feature is conceptually illogical and needs to be removed/replaced.
This has happened twice for me in the past three months, which should tell you a lot about how much time is wasted here.
TL;DR: I feel like we need something different than code review to catch these logic bugs. Maybe more of a presentation of the feature implementation to the other devs, and then a discussion of that.