A new Coddy Developer Survey found that four in five developers, 80%, say their use of AI has felt more like a dependence than an advantage.

  • Armand1@lemmy.world
    link
    fedilink
    English
    arrow-up
    14
    ·
    3 hours ago

    I can kind of relate. There are a lot of parts of coding I find a little boring.

    • Once the interesting problem is solved and all you need to do is follow through
    • Large scale refactorings in ways too complicated for an IDE.
    • Fixing annoying dependency / migration issues

    When you use an expensive AI agent, like Claude, it can nowadays handle these tasks competently, to the point where I only need to correct small things here and there.

    When you give these tasks to an AI agent, it feels a bit like delegating to a junior, but without the guilt of giving someone a menial or boring task. That allows me to work on stuff I find more interesting. To me, THAT’S why it’s adictive.

    However, this isn’t without cost. First, there’s the societal costs: Environmental, centralization of power, contribution to hardware shortages and a bubble

    Second, there are more personal costs. You’ll come to rely more and more on these tools, and your skills will rust. You may end up avoiding learning things about a codebase because you delegate it away.

    It’s a tempting tool. One which in my experience can genuinely help, but is easy to misuse.

    • orgrinrt@lemmy.world
      link
      fedilink
      English
      arrow-up
      1
      ·
      57 minutes ago

      Yeah I relate strongly with this. Once you get used to having more time and energy for the tough problems, the interesting stuff, it’s hard to go back to the mechanical and menial.

      But I’ve found it’s a lot like riding a bike. You don’t really rust, because the stuff the agents can handle today are the kind of shit that you’ve done so much and from the start, that it has become menial and boring. Which means it’s pretty ingrained into the backbone, at least that’s what I’ve found.

      But damn if I don’t feel bad about it, recognizing and being conscious about the societal and environmental cost the convenience has.

  • Yewb@lemmy.world
    link
    fedilink
    English
    arrow-up
    5
    ·
    3 hours ago

    Ai coding is like the loom for society but you have to know how to use it.

    People claiming its garbage slop are 2 years out of date on current models.

    To all those people I suggest you familiarize yourself with it as whats coming is going to ruin the world but not because its slop because its going to be capable.

    • JonnyW@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 hour ago

      Furniture used to be made by hand. Then factories came along and furniture makers became furniture assemblers. The quality wasn’t the same but you could create it more quickly and for a cheaper price. For many consumers, the trade off between quality and price was fine.

      This draws parallels with what’s happening in the software industry - we’re essentially making software factories and programmers are becoming ‘assemblers’.

      The thing is, there is still a market for handmade furniture. The market is smaller than it used to be and there are less furniture makers but some people are willing to pay more for quality.

      To say AI is going to ruin the world is a bit of a stretch. There will be software developers who are happy working in the factory assembling lesser quality products and there will be software developers who want to create higher quality products that stand out. I can see a market for both of these things. Whether you can achieve the latter with AI agents is the real question. I’m yet to see it.

    • thefloweracidic@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      ·
      2 hours ago

      AI can handle boilerplate and any coding project that has been done ad nauseam. AI can’t handle legacy system quirks, creative problem solving, or massive projects with changing requirements. I don’t think the AI bubble will be around long enough for them to solve these shortcomings or the issues of expanding scope and scale leading to hallucinations.

      To me AI is just impressive enough to convince managers and inexperienced developers it is the future. I’ll believe that when a Fortune 500 company rebuilds their whole entire tech stack via AI, and I think AI could solve climate change before it could do that haha.

      • Yewb@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 hour ago

        I disagree with much of your ai cant deal with complexity.

        I dunno i am just saying be prepared because its going to blind side everyone thinking its a fad

        • thefloweracidic@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          53 minutes ago

          Could you give some examples of AI handling complexity? From what I’ve read from senior developers is that the more scope AI has to chew on the more mistakes it makes. In terms of complexity I’m not talking about algorithms, differential calculus, or mind breaking recursion. I’m talking about the usual hodgepodge of monolithic microservices too many “Were a tech company that does X” companies have built over the past decade that are layered on top of software built before I was born.

          Maybe I’m just creating too complex and specific use case, but my time as a software developer had me working through problems like that.

      • Sims@lemmy.ml
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 hour ago

        AI can’t handle legacy system quirks, creative problem solving, or massive projects with changing requirements.

        I think that disliking ai - for whatever good reason - makes people shun the reality, cherry-pick articles/studies etc, but we have to be honest: We all do that - even when we try to balance it, so that is normal behaviour within a spectrum. However, current anti-ai leaning do end up blinded to how fast things are moving with this technology because it is moving so fast. There are massive burnouts just from those following AI development every day, and they constantly have to expand/narrow their scope. If anyone takes a few days of in their field of interest, it feels like they have to sprint to catch up. It is accelerating and is no longer confined to model inventions from big corps with mega models - developers are building ‘cognitive architectures’ around the current llm’ limitations.

        So to me, these ‘AI can’t …’ type of sentences are always changing as they become yesterdays ‘AI cant …’ sentences within days or weeks. It is a great ‘moving goalpost’ tho, as there are a lot of human level evolutionary abilities lacking from current models. But every second, one+ of millions of people are working towards adding that next ‘AI cant …’ sentence to yesterdays claims, and things are moving fast.

        I think it is vital for everyone - in all corners of human ‘belief-space’ - to keep in active touch with this technology, as much as they can muster, agree they are angry but try to shake some hard assumptions/‘loosen up’, see where it is headed and adjust their expectations/plans for the future accordingly. Anger is an easy but dangerous thing as it can make us refuse to acknowledge something that we need to prepare for. We’ve all been there - many many times, and we all know how we think and act in anger. Now, is a bad time for that. Please familiarise your self with ai tech.

        Not starting an argument/debate (i’m burned out atmo), just a long-winded observation really…

  • Lucidlethargy@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    32
    ·
    5 hours ago

    Addictive? What the fuck are they coding?

    Every time I try to use AI for coding I just get endlessly frustrated and angry.

    • Siegfried@lemmy.world
      link
      fedilink
      English
      arrow-up
      11
      ·
      edit-2
      4 hours ago

      The code LLMs spit is IME super buggy and gives as much work to correct it as it would have cost writing it from scratch… but it generally does the job. The addictive part is where you click a button and avoid thinking how to code.

      This is my experience, but im an amateur and actually enjoy programming things… so, i piss on LLMs taking that from me.

      EDIT: i know two persons that actively use LLMs, one does it to avoid having to ibteract with someone that actually knows how to code, the other learned to code to make money, so he really does not care that much.

  • chilicheeselies@lemmy.world
    link
    fedilink
    English
    arrow-up
    33
    ·
    7 hours ago

    So my preferred agent atm is GitHub Copilot as a jetbrains plugin. The other day when GitHub had a 6 hour outage and I couldn’t log in, my first instinct was “guess I can’t work today”. I realized almost immediately how fucking dumb that initial thought was. Like I had forgotten that I’ve been doing this for over 20 years, and around 17 of those with no AI whatsoever.

    Was I slower that day? Yes absolutely. Was I incapable? Not at all.

    I wouldn’t want to do without it, because on the positive side it has helped me discover libraries and tools I had no idea existed, and it’s made me a better archtitect. But, maybe we should still actually touch the code a little.

    • thefloweracidic@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      2 hours ago

      I’m curious what tasks are you using GitHub Copilot for? I left the industry before AI got huge so I was never pressured to use the tools.

      • Amju Wolf@pawb.social
        link
        fedilink
        English
        arrow-up
        1
        ·
        7 minutes ago

        It can wrode code decently well and fast, and is amazing at finding stuff in a large project.

        But you still 100% need to verify what it does and truly understand it.

  • blipcast@lemmy.world
    link
    fedilink
    English
    arrow-up
    14
    ·
    6 hours ago

    The biggest chilling effect I noticed, even as a developer who refuses to use AI, is that I feel like I can no longer ask my co-workers for advice. When I do, I get the the AI equivalent of “let me Google that for you, was that so hard?” So even without using it, I feel cut off from the normal places I would use to learn and improve my coding skills.

  • cinoreus@lemmy.world
    link
    fedilink
    English
    arrow-up
    28
    ·
    7 hours ago

    Coding with AI is really leaves you in a similar mental state as if you’ve just scrolled tiktok reels during that time.

    • thefloweracidic@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      edit-2
      2 hours ago

      A fun game to play is watching any tech talk even remotely about UX and replacing the words engagement or retention with addiction. The best category for this would be the 2010’s smart phone app design talks.

    • 4grams@awful.systems
      link
      fedilink
      English
      arrow-up
      5
      ·
      8 hours ago

      Well said.

      Ultimately it’s a capitalism problem, it makes the value proposition drift towards addiction in every single case.

  • Olgratin_Magmatoe@slrpnk.net
    link
    fedilink
    English
    arrow-up
    30
    ·
    edit-2
    8 hours ago

    The only utility I’ve had with it is as a slightly more fancy document search. “Give me the syntax for [one line of code I want but can’t remember]” or “Give me the syntax to do a c# style .select() call in [language I am less familiar with]”

    And then I look at the docs and actually put the effort into understanding how the fuck things work.

    Using it for anything more than that is a mistake. You still need to understand the context of what it is you’re doing, you still need to think.

  • jama211@lemmy.world
    link
    fedilink
    English
    arrow-up
    11
    ·
    6 hours ago

    This is a small, fairly biased survey with unpublished proper metrics. Whether you agree or disagree with the result, this is bad science.

  • dhork@lemmy.world
    link
    fedilink
    English
    arrow-up
    103
    ·
    edit-2
    11 hours ago

    I had this experience once. We have a ChatGPT license where I work, and I asked it to configure a switch that I wasn’t familiar with. I simply described in words what network architecture I wanted and it did it! It even made some nice-looking documentation.

    But, then I tried the new configs, and they didn’t work. It turns out there were some key syntax things it got wrong. And the documentation was wrong on top of that, with incorrect diagrams, and when I asked it to fix it it made different errors I the diagrams in different places. On balance, I still saved some time over reading all the manuals and figuring out the syntaxes myself, but only because I made my own documentation with the results that worked. If I had trusted the AI I would be sunk.

    I’ve concluded that AI gives the illusion of competence, like a overly confident new manager. This can be very attractive to a less experienced person. But it’s really guessing, just like we all are. It can just guess after actually “reading” all the manuals. I haven’t used AI to write anything more than simple configurations and helper scripts. If I did want to use AI for more it would be in more of a pair-programming context. I might have a window open where I describe some things and ask for analysis, but I wouldn’t just run anything it does blindly.

    • boudica@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      2 hours ago

      Pair programming/writing/creating is exactly how AI is meant to be used. It is an augmentor not a replacement for human competence. The person using it still has to do the thinking, qc, and directing, not take the first output as final.

    • dragonlover@lemmy.zip
      link
      fedilink
      English
      arrow-up
      28
      ·
      8 hours ago

      My coworkers use Claude like an actual brain subscription, and I have started to write off everything they say as if it came straight from the AI. They have gained so much unearned confidence about shit they have no idea about, and have even argued with the development team about it.

      I got into an argument about how in band and out of band DTMF work with one of them for a solid half hour before they finally admitted they didn’t actually know but were going off what claude said.

      I wanted to punch them for wasting everyone’s time. If you don’t understand stop answering definitively like you’re the expert.

      Some customers have switched to using AI emails too. Customers that used to ask extremely low level questions will now submit a 2 page long email with action items and explinations about why our product does X, Y, Z, and I have to read it twice to figure out their problem isn’t even in the action items because the AI hyper focused on the wrong thing.

      Like ok thanks you dumped the entire app log into Claude and asked it “why no work” and Claude read an error message that’s benign and now the customer is demanding fixes for something that is not and never has been a problem and won’t actually solve the root issue.

    • AliasVortex@lemmy.world
      link
      fedilink
      English
      arrow-up
      55
      ·
      edit-2
      10 hours ago

      But it’s really guessing, just like we all are.

      Beyond the moralistic objections I have to LLMs, I have serious issues with being fed confidently incorrect answers. I’ve had my share of configuration hell and I’m not above throwing code at the wall to see what sticks, but at least I have the good sense to drop a comment or mention in my commit that “hey, there’s a chance this isn’t right and could cause problems”.

      • dhork@lemmy.world
        link
        fedilink
        English
        arrow-up
        33
        ·
        9 hours ago

        I have serious issues with being fed confidently incorrect answers.

        Except I have plenty of experience dealing with this from humans. (Mainly from the aforementioned new managers, because being wrong with confidence seems to be a key trait to get promoted.) It’s been my experience that when you tell an AI “I just tried that and it didn’t work”, it will accept that more readily than a human would.

        Every bit of AI-generated code that I use, even in the smallest and most meaningless context, has to pass my own review first. I have to understand every line, and if I don’t I will ask the bot to explain what it did. By the time I am done with it, I can stand behind it just as if I wrote it all myself. I might note that I got AI help, but if my name is on the commit I will not pass the buck on any errors.

          • DaveyRocket@lemmy.world
            link
            fedilink
            English
            arrow-up
            5
            ·
            7 hours ago

            Then why not just write the fucking code yourself at that point? This is like putting training wheels on a tricycle.

        • Zarobi@aussie.zone
          link
          fedilink
          English
          arrow-up
          5
          ·
          7 hours ago

          I don’t trust A.I. code at all. If I ever do use it, I use it as a research tool like “please google for me how to do this one obscure thing because IDK what search query to use”; then I type out it’s output manually. Usually as I do so, I come across some subtle error that would cause horrible problems, and fix it as I go.

          I tried to use it for a mathematical algorithm once. I might as well have just written return Math.random();

          • chilicheeselies@lemmy.world
            link
            fedilink
            English
            arrow-up
            3
            ·
            6 hours ago

            This is a good thing to do if your goal is to gain a deep understanding of something.

            If it’s just to get it done, I just enforce TDD on my agent and review it’s output. I don’t need to be an expert in everything (and I am very much a generalist). But if you focus on a very specific thing and only that thing, then yeah what you are doing is a great way to truly understand it. It’s slow, but it’s totally valid.

            • Zarobi@aussie.zone
              link
              fedilink
              English
              arrow-up
              1
              ·
              6 hours ago

              It’s still faster than what I did before A.I.; I would spend ages Googling for something obscure and scrutinising one vague StackOverflow post over and over for insights. Also cursing iOS Safari.

            • dil@piefed.zip
              link
              fedilink
              English
              arrow-up
              1
              ·
              6 hours ago

              I like using it to setup github stuff and save time, like I needed to use rembg, I know you can setup terminal scripts as apps so if you open an app it runs the script. Had it set up a basic app to open videos with, create a folder using ffmpeg and turn it into an image sequence then run through that folder using removebg and/or depth anything (have added options for vectorizing, splats, etc.), afterwards sticying the image sequence back together to the original format, bringing back the audio. I was already doing this with comfyui before I realized they could be installed seprately be run through terminal commands, so I tried to get ai to set this up.

              Took about 5 minutes and a penny or 2 using deepseekv4flash with hermes. At it’s core, it’s hella simple, it’s just running existing programs rather than coming up with how to do all the tasks itself. I technically didn’t need it and could manually type these terminal commands myself or figure out how to automate it, but ai setting it up means it actually got done and saved me hours of time.

              After noticing most converters are frontends for ffmpeg and most downloaders yt-dlp, I realized you can easily make a gui for anything using the terminal with ai.

        • mrgoosmoos@lemmy.ca
          link
          fedilink
          English
          arrow-up
          3
          ·
          8 hours ago

          one of the programs that we use everyday at work recently added an AI coding tool. I was going to announce it to the team when I noticed with the usual disclaimers about ensuring you know what the macros are doing, but then just deleted my message.

          we’re not a team of programmers, and there’s only one or two people on my team that I would trust to write code that could potentially cause us days of rework and tons of thousands of dollars lost to the company.

          those other people don’t need an AI coding tool, because they can’t code in the first place, and those aren’t the people that I want modifying thousands of files at once when I know that they barely review the work they’re doing manually already and I have tools in place to semi-automate that review for them.

    • jobbies@lemmy.zip
      link
      fedilink
      English
      arrow-up
      9
      ·
      8 hours ago

      I’ve concluded that AI gives the illusion of competence

      Basically the same conclusion I came to. Which is terrifying when you think of all the devs who depend and all the money thats riding on it.

      there were some key syntax things it got wrong

      And syntax is the kind of thing llm’s should be awesome at. If they can’t even get that right we’re all cooked.

      • dhork@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        6 hours ago

        In fairness, this was all on a switch, where the commands are very tightly tied to the vendor and their underlying in-house shell. So commands may vary by release greatly. I was already explicitly telling the bot what software version and licenses I had, but ultimately I had to resort to the CLI’s help function at times and tell the bot “The command you gave me didn’t work. Here’s where it broke, and here’s the commands it will accept”. Given that information, it could (generally) figure it all out.

        • jobbies@lemmy.zip
          link
          fedilink
          English
          arrow-up
          2
          ·
          2 hours ago

          I dunno if you had this experience but the one I used made up commands if it didn’t know them.

          It basically hallucinated them.

    • Jo Miran@lemmy.ml
      link
      fedilink
      English
      arrow-up
      12
      ·
      9 hours ago

      I’ve concluded that AI gives the illusion of competence, like a overly confident new manager.

      I like to call this “confidently incorrect”, and ChatGPT is probably the worst culprit.

    • OhVenus_Baby@lemmy.ml
      link
      fedilink
      English
      arrow-up
      3
      ·
      6 hours ago

      This to a fucking T. It will be cock sure of accurate reliable results and run you in circles sometimes for hours and even repeat the same things when it doesn’t know. Ask me how I know. 4 hours alone yesterday fixing a production screen problem.

    • Séimhe (sé / é)@lemmy.world
      link
      fedilink
      English
      arrow-up
      16
      ·
      11 hours ago

      Considering the vast amounts of knowledge it has at its disposal, I can only conclude that it’s not very smart at applying it. A person with a fraction of that knowledge will produce better results.

      So it has more access to information, but the results are poor compared to a person.

      • kescusay@lemmy.world
        link
        fedilink
        English
        arrow-up
        38
        ·
        10 hours ago

        The important thing to remember is that it actually has zero access to information, because that’s not how LLMs work.

        At their core, they’re vector databases, and they’re trying to probabilistically come up with the next most likely token in a stream of tokens found in the DB. You can manipulate the stream by injecting text such as the content of existing files (which becomes more tokens) into the stream, but it never actually understands any of it.

        That’s why hallucinations are inherently unavoidable. It’s really all just hallucinations. It’s just that you can sometimes get useful text from their hallucinations if they happen to comport with reality.

        • frongt@lemmy.zip
          link
          fedilink
          English
          arrow-up
          7
          ·
          8 hours ago

          Well, vector fields are information. But they have no understanding. The number 1 might be followed by 2 in 99.999% of cases, but it has no function to explain why, or to contextualize a scenario where that might be wrong.

        • Zexks@lemmy.world
          link
          fedilink
          English
          arrow-up
          3
          ·
          8 hours ago

          Ypu have no idea what youre talking about. They absolutely have access to “information”

          • kescusay@lemmy.world
            link
            fedilink
            English
            arrow-up
            6
            ·
            edit-2
            7 hours ago

            No, they really don’t. That’s not how they work. At least, not if the “information” you’re talking about is real semantic content that real minds can process.

            Every piece of information you think an LLM has access to is actually just converted into a stream of additional tokens that are fed into the model to (hopefully usefully) modify the next tokens it predicts. That’s not the same thing as having actual access to information. Tokens are just numbers with statistically more (or less) likely relationships to each other.

            I’m not trying to downplay LLMs. They’re architecturally interesting and have genuine uses. I’m just trying to head off a bit of technical inaccuracy.

            • MalReynolds@slrpnk.net
              link
              fedilink
              English
              arrow-up
              2
              ·
              6 hours ago

              Absolutely correct and well said. Until you give it a tool to call a websearch (in my case SearxNG), I occasionally break it out (local 27B model) when a search is pulling lots of AI slop, Spy vs Spy style. I make it give me references and it usually indicates a bad search (XY problem)

          • astronaut_sloth@mander.xyz
            link
            fedilink
            English
            arrow-up
            4
            ·
            8 hours ago

            LLMs don’t. There are tools that can fetch new information and then gets fed into the model as more tokens, but that’s just a special case of what kescusay is saying about injecting text.

          • chilicheeselies@lemmy.world
            link
            fedilink
            English
            arrow-up
            2
            ·
            6 hours ago

            You are both right. An LLM inherently has access to stuff the same way a brain in a jar has access to stuff. It’s information comes from fine-tuning the models to return syntax that agent code can interpret as a request to invoke a tool. That tool returns information to the context of the conversation. It doesn’t learn and it can’t truly remember things. Every time you start a session it is brand new. It sees your codebase for the first time every time.

            The information access they have is whatever the agent allows it to access via tool exposure. Be it built in tools, or MCP servers

          • kescusay@lemmy.world
            link
            fedilink
            English
            arrow-up
            2
            ·
            edit-2
            7 hours ago

            As I’ve mentioned elsewhere, not if by “information” you mean semantic content that a mind can process. What they have are vector fields (essentially just numbers) with statistically more or less likely relationships.

            If I say, “take me out to the ballgame” to an LLM, the tokens representing the words in the next verse of the song are statistically “close” in the vector database, so it’s likely to generate them. But that doesn’t mean it actually knows the lyrics… or even has those lyrics recorded in a regular database anywhere.

            That’s why they hallucinate. The model determines that the next token is something nonsensical, but it has no way of understanding that it has made a mistake. In a sense, it actually hasn’t made a mistake. It’s done exactly what it’s designed to do. It’s just that in the case of hallucinations, its output isn’t useful.

  • leaky_shower_thought@feddit.nl
    link
    fedilink
    English
    arrow-up
    13
    ·
    8 hours ago

    it’s like gambling. you hope it churns out good stuff – and some of the time, it does! lucky!

    then you try your luck again. and again with small “fixes”. and again.

    • jaykrown@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      ·
      7 hours ago

      That’s only if you have no idea what you’re doing with it. Depending on your skill level in development, that is not the case at all.

  • magnue@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    6 hours ago

    Doesn’t mean they said it wasn’t helpful, just that the addictive score was higher than the helpful score for 80% of the developers asked.

  • dregan@lemmy.world
    link
    fedilink
    English
    arrow-up
    20
    ·
    edit-2
    10 hours ago

    Maybe it’s just because Anthropic is so many more miles ahead than OpenAI, but I have not had the “it doesn’t work” experience that people are talking about here. I’ve set up agentic pipelines with code architecture guidelines, the orchestrator calls the implementer, tester, reviewer when needed. If the developed code isn’t up to standard or the work item isn’t fully captured, it’ll reject it and send it back to implementation. It invokes the architect if it runs into something unanticipated. The pipeline can run a feature with high accuracy and very little input.

    I’ve even used it to run batteries of frozen experiments on self synthesized data using its own simulation harnesses to discover novel techniques that work in the real world. If it isn’t helpful, I feel like you are doing it wrong.

    • MagicShel@lemmy.zip
      link
      fedilink
      English
      arrow-up
      23
      ·
      9 hours ago

      I use Claude all the time at work. It is good. But it makes massive mistakes, it misses tests, it confidently says something it screwed up will be fixed by something that certainly isn’t the right way to fix the problem.

      I recently explained to a colleague: if you can use 1 AIU (arbitrary quantity of ai usage) and get 10% productivity bump, that doesn’t mean 5 AIU gets you 50% and 10 doubles your speed. The AI will do and say promising things, make you believe it’s on the verge of solving the problems, but it never quite arrives. There’s always one more problem and if you’re very lucky the AI will find it itself, but most likely it will be found when you pass it on to another person and it’s completely useless.

      Let me put it this way: in addition to development, I use Claude to help with production support issues. It wrote some scripts I didn’t have time to and it pulls logs and data from multiple systems — honestly it works great and has saved me so much time. But I’m constantly in meetings and so I set Claude to investigate an incident so I can focus on my meeting and return when I have time, and it gets RCA wrong well over 50% of the time.

      If it is so bad at RCA, how do you imagine it is fixing the bugs in the code it finds? Badly. It misunderstands the cause of problems, and so it fixes the wrong things until it has cobbled together the creakiest of code that passes the test. In fact I think AI is far worse at fixing code than it is at writing it in the first place.

      I’m not anti AI. I’m trying to find ways to make it effective. And my teams are seeing 20-30% productivity gains - I think because they are skeptical about AI rather than trusting. But it has to be used appropriately, and everywhere I look, even within my own company, people are trying to do too much with it and creating huge problems I have to sort through.

      • Wildmimic@anarchist.nexus
        link
        fedilink
        English
        arrow-up
        8
        ·
        edit-2
        7 hours ago

        It is increasing activity, not necessarily productivity. I see the issue that LLMs do not recognize and cannot communicate when they are missing necessary information to push the inference in the direction that is needed.

        It’s a bit like when you go into a diner and order a chicken sandwich, and the cook has read every book containing a chicken sandwich recipe in the world, no common sense, no budget / time limits on getting ingredients and an attention deficit disorder. You will (most of the time) get something that resembles a chicken sandwich, but it will probably never be to your taste. A normal cook has a lot of real world context to make sure that the sandwich is tasty - he knows what area he works in and what a chicken sandwich in this area normally contains, he knows how different ingredients interact with each other, and he is limited to what is available in his kitchen. If he knows you personally, he will be able to fine-tune the sandwich to your taste.

        LLM output does not have ANY of that context. Even things we do not actively think about - because it’s just the way how things are - are simply not available to an LLM. Only way to make sure our hypothetical ADD-ridden cook with eidetic memory makes a chicken sandwich worth a shit is to provide him with as much clear, non-ambiguous information as possible; and since the connection to the real world is missing, there can never be enough information - or rather, we are in the dark about what pieces of information would be needed to shift the probability towards the thing we actually want.

        • h0tbeef@lemmy.zip
          link
          fedilink
          English
          arrow-up
          4
          ·
          7 hours ago

          I’m on your side of the argument here dude, but you really need to fucking chill with the ableism

          • Wildmimic@anarchist.nexus
            link
            fedilink
            English
            arrow-up
            4
            ·
            edit-2
            6 hours ago

            Please explain how this is ableism in anyway. This nonexistant “person” does not resemble any human alive on this planet at all. Stop getting your panties in a bunch. I am very careful not using language that is used to insult or denigrate people. ADD isnt in the class of “r-word” language at all, and i actually fullfill 9/9 of the medical criteria for diagnosing ADD, so it was the first thing i grabbed at because it is so relateable to my own life. (and my chicken sandwiches are quite tasty too.)

        • MagicShel@lemmy.zip
          link
          fedilink
          English
          arrow-up
          1
          ·
          7 hours ago

          Great reply. I agree with a lot what you said, but I want to make two points:

          First, based on a human breakdown and pointing of stories as if AI weren’t being used, using the methodology we have used for years, we are seeing real delivery velocity increases. Now measurement is tricky; story points are arbitrary for a team. Some of those gains are in stuff like “wrote AI tool” or “created AI documentation” which means we’re being more productive at things we never used to need to do. Still, when you factor all of that in as best you can, we are seeing a measurable 20% increase. (The analysis does point at 30%, but I see holes in how those numbers are built.)

          Second:

          we are in the dark about what pieces of information would be needed to shift the probability towards the thing we actually want

          I don’t think we are completely in the dark. I think we are working on figuring out how to improve the context. The problem I see with the current approach is everyone is doing it independently, their methodology is poor because we can’t afford the tokens for exhaustive tests that are invalidated with the next model, and the results are entirely subjective (and frequently written by AI to sound like absolute success).

          So I come up with something that is amazing for my teams, and you try it, but because you’re in a different domain my technique isn’t quite right, and because our lives have provided us with different context, you don’t implement it quite the way I would have, and so your results differ and is it the domain, or the implementation, or did I just get some lucky rolls?


          One of the things I do in my spare time because I’ve been fascinated by AI for years, is I use AI to write stories. I come up with a premise, maybe do some world building, write an opening, and just let AI go and see what happens, steering it when necessary.

          Now there are two ways to steer: you can “reroll” or you can just edit the text directly. And what I’ve seen over years of doing this is the more human text you inject in, the better the results are. Even if you reroll to steer in the direction you want, the AI will eventually start producing utter gibberish. The quality starts to go down and then falls off a cliff.

          Human input slows that or even stops it altogether. Human input is an essential element in getting good results out of AI. Because bullshit built upon bullshit is multiplicative. An AI that is 99% good falls to 50% good very quickly.

          My point is that humans need to interact with the AI to provide that context you correctly point out the AI can never have, and it has to be frequent because you get to a point of saturation where a human is overwhelmed by the volume of text.

          • h0tbeef@lemmy.zip
            link
            fedilink
            English
            arrow-up
            1
            ·
            edit-2
            7 hours ago

            “Asking the AI to write stories for you” does not equal “using AI to write stories”

            • MagicShel@lemmy.zip
              link
              fedilink
              English
              arrow-up
              1
              ·
              7 hours ago

              I use it lots of ways. My posts are long enough without getting into asides like that, don’t you think?

      • Zexks@lemmy.world
        link
        fedilink
        English
        arrow-up
        3
        ·
        edit-2
        8 hours ago

        This is a user issue. If it missed stuff, you didnt describe what you want clearly enough. This right here is exactly why im not worried about it taking over

        • dregan@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          edit-2
          4 hours ago

          It’s probably a legacy code quality issue too. If your code is a mess, the only fix it can find is a kludgy one. The first thing I had it do, once it was good enough to do real work, was a legacy refactor to get architecture up to best practice standards. Heck, even OpenAI’s Harness Engineering specs say that this is essential to do first. I suspect that many of the people saying it comes up with shitty solutions are the same ones that always complain about over-engineering.

          You’ll also get subpar results if you just have it start coding without fully architecting its solution first. Just like a real developer.

        • h0tbeef@lemmy.zip
          link
          fedilink
          English
          arrow-up
          2
          ·
          7 hours ago

          It can’t “take over”, it’s has no sentience or actual intelligence

        • MagicShel@lemmy.zip
          link
          fedilink
          English
          arrow-up
          1
          ·
          7 hours ago

          I’m not worried about it taking over. Describe exactly what you want, clearly is exactly the job of a programmer and why we aren’t going to be replaced by middle management wielding an army of AI.

          The difference is code largely can’t be ambiguous where human language is subject to interpretation. So the question is what it takes to describe what is required in a way that the AI cannot misinterpret ambiguity, and so that agentic AI doesn’t have ambiguity decided confidently wrong by the orchestration layer.

          This is not strictly a user problem. AI will confidently make bad assumptions about ambiguity and propagate those to sub agents and you wind up with garbage. I’m good at this — would never claim to be the best and maybe you are better, but I’m good at it — and it still causes me grief. One criticism I have of the whole endeavor is that proficiency seems stubbornly difficult to transfer.

    • h0tbeef@lemmy.zip
      link
      fedilink
      English
      arrow-up
      3
      ·
      7 hours ago

      Someone told me to try and use it to solve a problem I was having on my computer.

      It told me to issue a terminal command that was half Linux syntax and half Windows syntax.

      That’s the only time I’ve used it. I’d prefer to learn things than to ask the computer to do the thinking for me.

    • WanderingThoughts@europe.pub
      link
      fedilink
      English
      arrow-up
      9
      ·
      9 hours ago

      That’s why they say it empowers people that already have the knowledge but beginners can’t control it. Basically somebody that controls the dog versus a small kid walking a big dog where if the dog goes, the kid gets dragged along behind it.

    • Mika@piefed.ca
      link
      fedilink
      English
      arrow-up
      2
      ·
      8 hours ago

      Pretty much my experience and I didn’t even have to do workflows or complex harnesses, I wrote a “dont ask” mode wrapper that gives rights to read/write a work dir & explanation that blockers & questions need to be written in a specific directory, and I listen to this directory with a GUI app that notifies me, then also a stop hook that verifies that that doc is updated when it stops, and has all the items done/blocked.

      I did this cause I like the flexibility of a normal agentic chat session.

      Recent LLM are smart enough to resolve many problems as is in agentic mode. Really strange to see “it doesnt work” copium instead of fighting for the means of production and looking for a way to have this setup purely local.