And we have to expect that trend will continue. I haven’t tested frontier capabilities since April but I know for a fact my local drafters are producing far fewer obvious smells than I remember seeing in frontier outputs even just last year.
Current frontier is likely still detectable by an experienced engineer, especially if they’ve seen enough gen code to recognize the patterns, but might otherwise be indistinguishable from junior-level work. That is, I wouldn’t be surprised if the current frontier models didn’t output ANY giveaway “slop” code or outright hallucinations. Symptoms might be more abstract like inelegant architectural decisions, obtuse alg/structure selection, or design choices that demonstrate no grasp of overall objectives.
My point was that from a project maintainer’s perspective, quality is more a side effect, and there’s no effective difference between no AI and no detectable AI. The actual point of a no-vibecode policy is (A) to communicate expectations, ensuring contributors know their PR must pass a sniff test, that their name is on it, so blind PR submission is never a safe choice, and (B) to have a bulletproof policy to refer to in retrospect if ever issues relating to AI use in the project arise.
That’s all fine and dandy, I just don’t agree with simultaneously saying “make it indistinguishable if you’re gonna use AI” in addition to “don’t use AI”, since as you say there is no effective difference if you can’t tell, so why should it still be disallowed? Surely we can come up with some middle ground to describe the intent of the rule better than silently implying “just don’t get caught”?
Well the idea is to NOT say “make it indistinguishable” (even if that’s inevitably the practical upshot) because the second you explicitly allow “indistinguishable” you open a can of worms arguing over what that means.
The same result unfortunately seems to be found by any attempt seeking this “middle ground,” because there’s always a cohort implicitly requesting that project maintainers bless their low-effort use of AI. (As to why this is, I’m not sure, but based on what I’ve seen would guess it’s because the AI use lets these individuals cosplay as developers.)
Regardless, that result is a huge influx of PRs which have obviously been submitted by individuals incapable of understanding, fixing, or taking responsibility for what they’ve submitted, which is an immediately intractable problem for project maintainers who already began with limited time to review PRs which, in spite of their varying quality, could at least be assumed to be understood by the contributor. Allowing any amount AI enables some amount of obvious (read: unfiltered/unreviewed) generated code, every bit of which is potentially problematic.
And we have to expect that trend will continue. I haven’t tested frontier capabilities since April but I know for a fact my local drafters are producing far fewer obvious smells than I remember seeing in frontier outputs even just last year.
Current frontier is likely still detectable by an experienced engineer, especially if they’ve seen enough gen code to recognize the patterns, but might otherwise be indistinguishable from junior-level work. That is, I wouldn’t be surprised if the current frontier models didn’t output ANY giveaway “slop” code or outright hallucinations. Symptoms might be more abstract like inelegant architectural decisions, obtuse alg/structure selection, or design choices that demonstrate no grasp of overall objectives.
My point was that from a project maintainer’s perspective, quality is more a side effect, and there’s no effective difference between no AI and no detectable AI. The actual point of a no-vibecode policy is (A) to communicate expectations, ensuring contributors know their PR must pass a sniff test, that their name is on it, so blind PR submission is never a safe choice, and (B) to have a bulletproof policy to refer to in retrospect if ever issues relating to AI use in the project arise.
That’s all fine and dandy, I just don’t agree with simultaneously saying “make it indistinguishable if you’re gonna use AI” in addition to “don’t use AI”, since as you say there is no effective difference if you can’t tell, so why should it still be disallowed? Surely we can come up with some middle ground to describe the intent of the rule better than silently implying “just don’t get caught”?
Well the idea is to NOT say “make it indistinguishable” (even if that’s inevitably the practical upshot) because the second you explicitly allow “indistinguishable” you open a can of worms arguing over what that means.
The same result unfortunately seems to be found by any attempt seeking this “middle ground,” because there’s always a cohort implicitly requesting that project maintainers bless their low-effort use of AI. (As to why this is, I’m not sure, but based on what I’ve seen would guess it’s because the AI use lets these individuals cosplay as developers.)
Regardless, that result is a huge influx of PRs which have obviously been submitted by individuals incapable of understanding, fixing, or taking responsibility for what they’ve submitted, which is an immediately intractable problem for project maintainers who already began with limited time to review PRs which, in spite of their varying quality, could at least be assumed to be understood by the contributor. Allowing any amount AI enables some amount of obvious (read: unfiltered/unreviewed) generated code, every bit of which is potentially problematic.