Honestly, there are a few thoughts about this (and yes, some will be unpopular what I say).
There is the ethics problem of how LLM is trained. It is a theft machine. Thats why companies love it, because they can steal work and profit
However, people WRONGLY assume AI is exclusively for Vibe coding. When used by Senior/real developers, it is super useful for writing tests and code reviews. Some of the issues I’ve found in our old code from 10 years ago, were never reported (or, we had reports, but always assumed it was something else). There are tools for code review (and have been for a long time), but AI has stepped it up. In well designed / stricter languages, you can avoid a lot of errors that AI is good at detecting, but, it is still super valuable for this stuff.
Also, good for security testing too.
The biggest issue are untrained slop cryptobros, who throw money at it, and can’t test (or understand) their code. Then it wastes other devs time reviewing it and identifying the 50 regressions it causes.
We’ve had cases where I’ve had to argue with customers that Claude is telling them bullshit. From the support side, it has made things WORSE.
I’d argue it isn’t actually a good thing necessarily to ban using it as a tool entirely from senior devs.
The biggest issue at the moment are arrogant junior cryptobros who are too lazy to learn to code, and just want to throw money at the issue. And that wastes everyone’s time.
What we really need is a ethical AI model that only uses completely open code, pays people for their code (instead of just stealing it). and a way to ensure it is only used by developers who can use it properly. Ideally, only allow that code to be used for open source too
I actually wouldn’t want to see Gnome/System76 completely ban it. What I would want to see is for it to be permitted for approved devs with VERY specific guidelines dictating how it can be used.
Have to think about this in an open source context. You have an open ended population ready to submit to your project if they think they can be useful
Now, thanks to GenAI, a bunch of people who aren’t good at various things now thinks they are good at various things. Whether it’s coding, making comics, making videos, what have you. Try to do nuance even if it strictly makes sense, and you are stuck with the slop problem if your project is sufficiently popular.
In terms of more directly on your points, if someone does GenAI to do a security analysis, ok, but I’d want them to do the tedium of trying to identify the false positives and then re-report it in their words of actual understanding. It can catch things, but along the way makes a haystack of sillyness to go through. Same for code review, legitimate issues, but lots of missing (I spent a non trivial amount of time yesterday because a GenAI code review insisted a variable would be unitialized when referenced, when I see an uncoditional assignment just a few lines above, trying to think if there was some catch I wasn’t seeing). So indirectly using it and only subjecting the developer/maintainer to that which you know makes sense.
Problem being is that people have used Claude and have forwarded to me saying “I don’t understand this well enough to judge, but forwarding to you just in case”. I have the same tools doing the same things as you, I don’t need meat proxies that just pass through the stuff without understanding.
The issue for now is LLM generation of code, not code auditing.
And no, I don’t give a fuck whether some genius in theory could paint s new Mona Lisa with it.
The issue us how it is used in practice, most of the time, today.
At $WORK, I have a severely ai-pilled Senior Embedded Software Architect which hasnt managed in one and a half year to set up a working driver for a RS232-controlled stepper motor, from a port of previously working code. A thing that should take a week at most. I had to educate him that in C++ drivers, you need to use locks or mutexes to access variables that are concurrently changed and read from several threads. AI enables catastrophic levels of incompetence.
And FOSS projects need to protect themselves against that.
I’ve found ‘senior’ staff that became talking heads and stopped doing real work the bane of my existence in GenAI world. “I haven’t coded in 20 years, but thanks to GenAI, I’m doing it again!” The reasons you stopped coding 20 years ago are plainly valid. Good for you, you got to transition to a grift based career where you say nothing but sound smart to the right people and get money, please don’t return to coding because CodeGen is now ‘cool’.
Yeah… LLM Generation I agree is the biggest issue by far.
On the Kernel side, code review though apparently has been a big factor apparently, because people are testing kernel modules in seriously dumb tests that would never happen in practice, and then submitting some of the dumbest patches to protect against faults that won’t happen in reality
Oh man, the insane defensive code that it wants to generate. Yes, a decent principle, but when the stack has checked the same thing like 4 times, it’s a bit much.
As a retired senior programmer, I would have loved to use LLMs in the past for very specific things. I wouldn’t use it every day, but like every couple of months I had to do a massive refactor that took weeks to complete. I usually ended up writing codemod and just painfully changing tens of thousands of lines of code. Would be cool to just say “hey LLM, see how I did this one? Do the same thing everywhere else you find it. If you encounter anything that’s too different from my template just leave it for me to review”
So I had a huge refactor and thought “Ok, this should be right up GenAI alley”. And in fact took your very approach of giving an example for a few and said “go at it”.
To my surprise, it actually did it pretty poorly, would not work, when it would have worked, dire performance implications. Failing to address things that technically would survive the rework functionally intact, but now a very bad way of doing things in new context. Problems exacerbated is that when I’m reviewing code, I tend to have a more optimistic assumption of the code than when I’m writing and second guessing myself. So it’s all the more annoying to read code I didn’t write screw up so much.
I will say it does a pretty good job of boilerplate heavy crap. If I’m going to want to make Go structs from JSON, I can just feed a sample and the very tedious work of making the sometimes maddening tedius Go structs gets chewed through pretty nicely.
Yeah I once tried to use it to write complex SQL and it just completely sucked at it. Like insanely inefficient queries that scan multiple tables multiple times instead of more efficient joins and other methods… Shame, I was hoping it would be good for refactoring at least if you gave it good examples
Yeah, and that’s the stuff people don’t talk about
I have used it to do a lot of refactoring too. Vs code has been able to do basic refactoring for a while, but, it is also good for splitting up code into different files as an example (as long as you use a plan, it works well)
But, that’s because it’s also been trained how to do these tasks. And it isn’t really doing much with the code for them
It can actually do everything you mention, and under a watchful eye, generate reasonable code to spec. If you give it the architecture and requirements, it will make it as you see fit and generally faster than a skilled human.
But even beyond the theft, the resources required to do this are insane. The power and hardware needs are off the chart, and we’re still paying for all this with Monopoly venture bucks.
For me it’s the exact extreme opposite than your first point: The LLMs as death of copyright could be an actual food thing.
The issue im struggling with is the ecological and economic harm. It feels a bit as if the steam machine would’ve been invented but you’d throw in babies in addition to coal.
For the plus sides you’re right (as in: I agree so it must be ;) )b- with one minor exception: the vibe coding part is not bad in itself. I actually used vibe coding as a very hard exercise: develop something and find ways to estimate its quality and performance but you are not allowed to look into the code! It’s a fun brain teaser because it forces a pure outcome-perspective. The code itself is irrelevant for this, it’s the mental training that’s really interesting.
Oh except people who publish vibe coded stuff as a cool new project without disclaimer. Or as PRs. Or as … Well anything where do see itself is actually important.
death of copyright could be an actual [g]ood thing
It could. It would assuredly be a boon to the development of de-build chains: decompilers and the like.
But, as long as Disney and Amazon get protections, I’m going to advocate for the protection of works by independent artists and coders, particularly CC-SA, GPL, or AGPL licensed works. This does mean resisting LLMs (and other generative AI) on copyright grounds. (Even if on the other hand I might advocate for the undoing of copyrights generally.)
And, even if that class of issue were settled, I also agree the current state of generative AI is a travesty to be resisted on ecological and labor terms. As just one example of ecological harm, MS undid years of sustainability work in service to Copilot. As one example of harming Workers, Amnesty International has identified human rights abuses in the training of several model brands.
The vibe coding thing IS bad though… The code being stolen isn’t the code generally written by big companies (because that’s in private repos)… It’s also stealing GPL / MIT code too, and not crediting it. And then the vibe coding bros take it, boast they made it, don’t give credit, don’t use it to improve AI, and they sell that code
I’d have no issue if it credited the original projects, but it doesn’t… And thats why nobody can hold them responsible.
The ecological harm is a fair point though. They’re setting up gas plants to run these things, instead of forcing renewables.
Oh the corporate bullshit I’m with you - but I’m opposed to copyright as a mechanism, that’s why I focused on it. Code can’t be “stolen” the same way a movie can’t be stolen in my book. That said:
With the crediting you’re absolutely right, I had not even thought about it and that’s not what I intended to imply. Thank you!
Although I lack the fantasy how this could look like. I’d be very happy if LLMs would be waived the “by” aspect as it’s in possible to link a generation to a dataset - but the training data has to be publicly available under the most generous licence consumed.
The other way around: music can’t be stolen - piracy is not theft and the concept of property is only beneficial to the ones writing the rules and not the actual creators (just check Disneys copyright law adjustments to extend protectiond in heir various “intellectual properties”.
“Theft”/stealing is the wording the music industry established when media downloads started (“you wouldn’t download a car”? Fuck em, yes I would).
Honestly, there are a few thoughts about this (and yes, some will be unpopular what I say).
I’d argue it isn’t actually a good thing necessarily to ban using it as a tool entirely from senior devs.
The biggest issue at the moment are arrogant junior cryptobros who are too lazy to learn to code, and just want to throw money at the issue. And that wastes everyone’s time.
What we really need is a ethical AI model that only uses completely open code, pays people for their code (instead of just stealing it). and a way to ensure it is only used by developers who can use it properly. Ideally, only allow that code to be used for open source too
I actually wouldn’t want to see Gnome/System76 completely ban it. What I would want to see is for it to be permitted for approved devs with VERY specific guidelines dictating how it can be used.
Have to think about this in an open source context. You have an open ended population ready to submit to your project if they think they can be useful
Now, thanks to GenAI, a bunch of people who aren’t good at various things now thinks they are good at various things. Whether it’s coding, making comics, making videos, what have you. Try to do nuance even if it strictly makes sense, and you are stuck with the slop problem if your project is sufficiently popular.
In terms of more directly on your points, if someone does GenAI to do a security analysis, ok, but I’d want them to do the tedium of trying to identify the false positives and then re-report it in their words of actual understanding. It can catch things, but along the way makes a haystack of sillyness to go through. Same for code review, legitimate issues, but lots of missing (I spent a non trivial amount of time yesterday because a GenAI code review insisted a variable would be unitialized when referenced, when I see an uncoditional assignment just a few lines above, trying to think if there was some catch I wasn’t seeing). So indirectly using it and only subjecting the developer/maintainer to that which you know makes sense.
Problem being is that people have used Claude and have forwarded to me saying “I don’t understand this well enough to judge, but forwarding to you just in case”. I have the same tools doing the same things as you, I don’t need meat proxies that just pass through the stuff without understanding.
The issue for now is LLM generation of code, not code auditing.
And no, I don’t give a fuck whether some genius in theory could paint s new Mona Lisa with it. The issue us how it is used in practice, most of the time, today. At $WORK, I have a severely ai-pilled Senior Embedded Software Architect which hasnt managed in one and a half year to set up a working driver for a RS232-controlled stepper motor, from a port of previously working code. A thing that should take a week at most. I had to educate him that in C++ drivers, you need to use locks or mutexes to access variables that are concurrently changed and read from several threads. AI enables catastrophic levels of incompetence.
And FOSS projects need to protect themselves against that.
I’ve found ‘senior’ staff that became talking heads and stopped doing real work the bane of my existence in GenAI world. “I haven’t coded in 20 years, but thanks to GenAI, I’m doing it again!” The reasons you stopped coding 20 years ago are plainly valid. Good for you, you got to transition to a grift based career where you say nothing but sound smart to the right people and get money, please don’t return to coding because CodeGen is now ‘cool’.
Yeah… LLM Generation I agree is the biggest issue by far.
On the Kernel side, code review though apparently has been a big factor apparently, because people are testing kernel modules in seriously dumb tests that would never happen in practice, and then submitting some of the dumbest patches to protect against faults that won’t happen in reality
Oh man, the insane defensive code that it wants to generate. Yes, a decent principle, but when the stack has checked the same thing like 4 times, it’s a bit much.
As a retired senior programmer, I would have loved to use LLMs in the past for very specific things. I wouldn’t use it every day, but like every couple of months I had to do a massive refactor that took weeks to complete. I usually ended up writing codemod and just painfully changing tens of thousands of lines of code. Would be cool to just say “hey LLM, see how I did this one? Do the same thing everywhere else you find it. If you encounter anything that’s too different from my template just leave it for me to review”
So I had a huge refactor and thought “Ok, this should be right up GenAI alley”. And in fact took your very approach of giving an example for a few and said “go at it”.
To my surprise, it actually did it pretty poorly, would not work, when it would have worked, dire performance implications. Failing to address things that technically would survive the rework functionally intact, but now a very bad way of doing things in new context. Problems exacerbated is that when I’m reviewing code, I tend to have a more optimistic assumption of the code than when I’m writing and second guessing myself. So it’s all the more annoying to read code I didn’t write screw up so much.
I will say it does a pretty good job of boilerplate heavy crap. If I’m going to want to make Go structs from JSON, I can just feed a sample and the very tedious work of making the sometimes maddening tedius Go structs gets chewed through pretty nicely.
Yeah I once tried to use it to write complex SQL and it just completely sucked at it. Like insanely inefficient queries that scan multiple tables multiple times instead of more efficient joins and other methods… Shame, I was hoping it would be good for refactoring at least if you gave it good examples
Yeah, and that’s the stuff people don’t talk about
I have used it to do a lot of refactoring too. Vs code has been able to do basic refactoring for a while, but, it is also good for splitting up code into different files as an example (as long as you use a plan, it works well)
But, that’s because it’s also been trained how to do these tasks. And it isn’t really doing much with the code for them
It can actually do everything you mention, and under a watchful eye, generate reasonable code to spec. If you give it the architecture and requirements, it will make it as you see fit and generally faster than a skilled human.
But even beyond the theft, the resources required to do this are insane. The power and hardware needs are off the chart, and we’re still paying for all this with Monopoly venture bucks.
For me it’s the exact extreme opposite than your first point: The LLMs as death of copyright could be an actual food thing.
The issue im struggling with is the ecological and economic harm. It feels a bit as if the steam machine would’ve been invented but you’d throw in babies in addition to coal.
For the plus sides you’re right (as in: I agree so it must be ;) )b- with one minor exception: the vibe coding part is not bad in itself. I actually used vibe coding as a very hard exercise: develop something and find ways to estimate its quality and performance but you are not allowed to look into the code! It’s a fun brain teaser because it forces a pure outcome-perspective. The code itself is irrelevant for this, it’s the mental training that’s really interesting.
Oh except people who publish vibe coded stuff as a cool new project without disclaimer. Or as PRs. Or as … Well anything where do see itself is actually important.
It could. It would assuredly be a boon to the development of de-build chains: decompilers and the like.
But, as long as Disney and Amazon get protections, I’m going to advocate for the protection of works by independent artists and coders, particularly CC-SA, GPL, or AGPL licensed works. This does mean resisting LLMs (and other generative AI) on copyright grounds. (Even if on the other hand I might advocate for the undoing of copyrights generally.)
And, even if that class of issue were settled, I also agree the current state of generative AI is a travesty to be resisted on ecological and labor terms. As just one example of ecological harm, MS undid years of sustainability work in service to Copilot. As one example of harming Workers, Amnesty International has identified human rights abuses in the training of several model brands.
The vibe coding thing IS bad though… The code being stolen isn’t the code generally written by big companies (because that’s in private repos)… It’s also stealing GPL / MIT code too, and not crediting it. And then the vibe coding bros take it, boast they made it, don’t give credit, don’t use it to improve AI, and they sell that code
I’d have no issue if it credited the original projects, but it doesn’t… And thats why nobody can hold them responsible.
The ecological harm is a fair point though. They’re setting up gas plants to run these things, instead of forcing renewables.
Oh the corporate bullshit I’m with you - but I’m opposed to copyright as a mechanism, that’s why I focused on it. Code can’t be “stolen” the same way a movie can’t be stolen in my book. That said:
With the crediting you’re absolutely right, I had not even thought about it and that’s not what I intended to imply. Thank you!
Although I lack the fantasy how this could look like. I’d be very happy if LLMs would be waived the “by” aspect as it’s in possible to link a generation to a dataset - but the training data has to be publicly available under the most generous licence consumed.
But that’s wishful thinking, I know.
Why do you think code can’t be stolen the same way as music?
The other way around: music can’t be stolen - piracy is not theft and the concept of property is only beneficial to the ones writing the rules and not the actual creators (just check Disneys copyright law adjustments to extend protectiond in heir various “intellectual properties”.
“Theft”/stealing is the wording the music industry established when media downloads started (“you wouldn’t download a car”? Fuck em, yes I would).