I totally get it. I’ve been critical about using AI for code purposes at work and have pleaded to stop using it (management is forcing it, less experienced folk want it). So I’ve been given a challenge by one of the proponents to use a very specific tool. This one should be one of the best AI slop generators out there.
So I spent a lot of time thoroughly writing specs for a task in a way the tool should be able to do it. It failed miserably, didn’t even produce any usable result. So I asked the dude that challenged me to help me refine the specs, tweak the tool, make everything perfect. The thing still failed hard.
It was said it was because I was forcing the tool into decisions it couldn’t handle and to give it more freedom. So we did that, it made up the rules themselves and subsequently didn’t follow those rules. Another failure. So we split up the task into smaller pieces, it still couldn’t handle it. So we split it up even further, to a ridiculous level, at which point it would definitely be faster just to create the code manually. It’s also no longer realistic, as we pretty much have the end result all worked out and are just coaching the tool to get there. And even then it’s making mistakes, having to be corrected all the time, not following specs, not following code guidelines or best practices. Another really annoying thing is it keeps on changing code it shouldn’t touch, since we’ve made the steps so small, it keeps messing up work it did previously. And the comments it creates are crazy, either just about every line has a comment attached and functions get a whole story, or it has zero comments. As soon as you say to limit the comments to where they are useful, it just deletes all the comments, even the ones it put in before or we put in manually.
I’m ready to give up on the thing and have the use of AI tools for coding limited if not outright stopped entirely. But I’ll know how that discussion will go: Oh you used tool A? No, you should be using tool B, it’s much better. Maybe the tools aren’t there now, but they are getting better all the time, so we’ll benefit any day now.
When I hear even experienced devs be enthusiastic about AI tools, I really feel like I’m going crazy. They suck a lot and aren’t useful at all (on top of the thousand other issues with AI), why are people liking it? And why have we hedged the entire economy on it?
I’ve started using it as an interactive rubber duck. When I’ve got a problem, I explain it to the AI, after which it gives a response that I ignore because after explaining it, I figured it out myself.
AI has been very helpful for finding my way around Azure deploy problems, though. And other complex configuration issues (I was missing a certificate to use az login). I fixed problems I probably couldn’t have solved without it.
But I’ve lost a lot of time trying to get it to solve complex coding problems. It makes a heroic effort trying to combine aspects of known patterns and algorithms into something resembling a solution, and it can “reason” about how it should work, but it doesn’t really understand what it’s doing.
I use colleagues or people on Discord for this. I get the solution immediately after asking AND those that saw me, or heard me, ask now think I’m an idiot. It’s my neurodivergent kink!
Which is strange, because Azure’s documentation is complete dogshit.
We were trying to solve something at work (send SMTP messages using OAuth authentication, not rocket science) and Azure’s own chatbot kept on making up non-existent server commands, rest endpoints that don’t exist, and phantom permissions that needed to be added to the account.
Seriously; fuck Azure, fuck Copilot. Made a task that should have taken hours, take weeks.
I totally get it. I’ve been critical about using AI for code purposes at work and have pleaded to stop using it (management is forcing it, less experienced folk want it). So I’ve been given a challenge by one of the proponents to use a very specific tool. This one should be one of the best AI slop generators out there.
So I spent a lot of time thoroughly writing specs for a task in a way the tool should be able to do it. It failed miserably, didn’t even produce any usable result. So I asked the dude that challenged me to help me refine the specs, tweak the tool, make everything perfect. The thing still failed hard. It was said it was because I was forcing the tool into decisions it couldn’t handle and to give it more freedom. So we did that, it made up the rules themselves and subsequently didn’t follow those rules. Another failure. So we split up the task into smaller pieces, it still couldn’t handle it. So we split it up even further, to a ridiculous level, at which point it would definitely be faster just to create the code manually. It’s also no longer realistic, as we pretty much have the end result all worked out and are just coaching the tool to get there. And even then it’s making mistakes, having to be corrected all the time, not following specs, not following code guidelines or best practices. Another really annoying thing is it keeps on changing code it shouldn’t touch, since we’ve made the steps so small, it keeps messing up work it did previously. And the comments it creates are crazy, either just about every line has a comment attached and functions get a whole story, or it has zero comments. As soon as you say to limit the comments to where they are useful, it just deletes all the comments, even the ones it put in before or we put in manually.
I’m ready to give up on the thing and have the use of AI tools for coding limited if not outright stopped entirely. But I’ll know how that discussion will go: Oh you used tool A? No, you should be using tool B, it’s much better. Maybe the tools aren’t there now, but they are getting better all the time, so we’ll benefit any day now.
When I hear even experienced devs be enthusiastic about AI tools, I really feel like I’m going crazy. They suck a lot and aren’t useful at all (on top of the thousand other issues with AI), why are people liking it? And why have we hedged the entire economy on it?
I’ve started using it as an interactive rubber duck. When I’ve got a problem, I explain it to the AI, after which it gives a response that I ignore because after explaining it, I figured it out myself.
AI has been very helpful for finding my way around Azure deploy problems, though. And other complex configuration issues (I was missing a certificate to use
az login). I fixed problems I probably couldn’t have solved without it.But I’ve lost a lot of time trying to get it to solve complex coding problems. It makes a heroic effort trying to combine aspects of known patterns and algorithms into something resembling a solution, and it can “reason” about how it should work, but it doesn’t really understand what it’s doing.
I use colleagues or people on Discord for this. I get the solution immediately after asking AND those that saw me, or heard me, ask now think I’m an idiot. It’s my neurodivergent kink!
Which is strange, because Azure’s documentation is complete dogshit.
We were trying to solve something at work (send SMTP messages using OAuth authentication, not rocket science) and Azure’s own chatbot kept on making up non-existent server commands, rest endpoints that don’t exist, and phantom permissions that needed to be added to the account.
Seriously; fuck Azure, fuck Copilot. Made a task that should have taken hours, take weeks.
Nah bro U just prompting wrong trust me bro just one more tool.
/S