- cross-posted to:
- hackaday@ibbit.at
- cross-posted to:
- hackaday@ibbit.at
You must log in or register to comment.
Interesting. I thought this will be another post about slop PRs and bug reports but no, it’s about open source project not being promoted by AI and missing on adoption and revenue opportunities.
So I think we definitely see (and will see more) ‘templatization’ of software development. Some ways of writing apps that are easy to understand for AI and are promoted by it will see wider and wider adoption. Not just tools and libraries but also folder structures, design patterns and so on. I’m not sure how bad this will be long term. Maybe it will just stabilize tooling? Do we really need new React state management library every 6 months?
Hard to tell how will this affect the development of proper tools (not vibe coded ones). Commercial tools struggling to get traction will definitely suffer but most of the libraries I use are hobby projects. I still see good tools with good documentation getting enough attention to grow, even fairly obscure ones. Then again, those tools often struggle with getting enough contributors… Are we going to see a split between vibe coded template apps for junior devs and proper tools for professionals? Will EU step in and found the core projects? I still see a way forward so I’m fairly optimistic but it’s really hard to predict what will happen in a couple of years.
The killing part is not necessarily people vibe coding programs into OSS projects, but even if the OSS itself is not vibe coded, people using AI to integrate with it will result in lower engagement and thus killing the ecosystem:
Together, these patterns suggest that AI mediation can divert interaction away from the surfaces where OSS projects monetize and recruit contributors.
From Section 2.3 of the reported paper.
I just wanna say that’s such a good thumbnail
A Matrix guard thing but with cat details?
Btw, how do you call that kind of image? I’ve seen “hero-image” in some newspapers’ html & css.
Oh yeah. If it was drawn by AI, well, it sure fooled me.
Only until AI investor money dries up and vibe coding gets very expensive quickly. Kinda how Uber isn’t way cheaper than a taxi now.
until AI investor money dries up
Is that the latest term for “when hell freezes over”?
Microsoft steeply lowered expectations on the AI Sales team, though they have denied this since they got pummelled in their quarterly and there’s been a lot of news about how investors are not happy with all the circular AI investments pumping those stocks. When the bubble pops (and all signs point to that), investors will flee. You’ll see consolidation, buy-outs, hell maybe even some bullshit bailouts, but ultimately it has to be a sustainable model and that means it will cost developers or they will be pummeled with ads (probably both).
A Majority of CEOs are saying their AI spend has not paid off. Those are the primary customers, not your average joe. MIT reports 95% generative AI failure rate at companies. Altman still hasn’t turned a profit. There are Serious power build-out problems for new AI centers (let alone the chips needed). It’s an overheated reactionary market. It’s the Dot Com bubble all over again.
There will be some more spending to make sure a good chunk of CEOs “add value” (FOMO) and then a critical juncture where AI spending contracts sharply when they continue to see no returns, accelerated if the US economy goes tits up. Then the domino’s fall.
Hah, they wish. It’s a business, and they need a return on investment eventually. Maybe if we were in a zero interest rate world again, but even that didn’t last.
This.
I wouldn’t be surprised if that’s only a temporary problem - if it becomes one at all. People are quickly discovering ways to use LLMs more effectively, and open source models are starting to become competitive with commercial models. If we can continue finding ways to get more out of smaller, open-source models, then maybe we’ll be able to run them on consumer or prosumer-grade hardware.
GPUs and TPUs have also been improving their energy efficiency. There seems to be a big commercial focus on that too, as energy availability is quickly becoming a bottleneck.
So far, there is serious cognitive step needed that LLM just can’t do to get productive. They can output code but they don’t understand what’s going on. They don’t grasp architecture. Large projects don’t fit on their token window. Debugging something vague doesn’t work. Fact checking isn’t something they do well.
So far, there is serious cognitive step needed that LLM just can’t do to get productive. They can output code but they don’t understand what’s going on. They don’t grasp architecture. Large projects don’t fit on their token window.
There’s a remarkably effective solution for this, that helps both humans and models alike - write documentation.
It’s actually kind of funny how the LLM wave has sparked a renaissance of high-quality documentation. Who would have thought?
High-quality documentation assumes there’s someone with experience working on this. That’s not the vibe coding they’re selling.
I am not aware of what they are selling but every vibe coder i know produces obsessive amounts of documentation. It’s kind of baked into the tool (if you use Claude Code at least), it will just naturally produce a lot of documentation.
Complete hands-off no-review no-technical experience vibe coding is obviously snake oil, yeah.
This is a pretty large problem when it comes to learning about LLM-based tooling: lots of noise, very little signal.
They don’t need the entire project to fit in their token windows. There are ways to make them work effectively in large projects. It takes some learning and effort, but I see it regularly in multiple large, complex monorepos.
I still feel somewhat new-ish to using LLMs for code (I was kinda forced to start learning), but when I first jumped into a big codebase with AI configs/docs from people who have been using LLMs for a while, I was kinda shocked. The LLM worked far better than I had ever experienced.
It actually takes a bit of skill to set up a decent workflow/configuration for these things. If you just jump into a big repo that doesn’t have configs/docs/optimizations for LLMs, or you haven’t figured out a decent workflow, then they’ll be underwhelming and significantly less productive.
(I know I’ll get downvoted just for describing my experience and observations here, but I don’t care. I miss the pre-LLM days very much, but they’re gone, whether we like it or not.)
It actually takes a bit of skill to set up a decent workflow/configuration for these things
Exactly this. You can’t just replace experienced people with it, and that’s basically how it’s sold.
Yep, it’s a tool for engineers. People who try to ship vibe-coded slop to production will often eventually need an engineer when things fall apart.
This sounds a lot like every framework, 20 years ago you could have written that about rails.
Which IMO makes sense because if code isn’t solving anything interesting then you can dynamically generate it relatively easily, and it’s easy to get demos up and running, but neither can help you solve interesting problems.
Which isn’t to say it won’t have a major impact on software for decades, especially low-effort apps.
They’ve thought of that as well, soon nobody will be able to afford consumer grade hardware
Yeah true. I’m assuming (and hoping) that the problems with consumer grade hardware being less accessible will be temporary.
I have wristwatches with significantly higher CPU, memory, and storage specs than my first few computers, while consuming significantly less energy. I think the current state of LLMs is pretty rough but will continue to improve.
Can you cite some sources on the increased efficiency? Also, can you link to these lower priced, efficient (implied consumer grade) GPUs and TPUs?
Oh, sorry, I didn’t mean to imply that consumer-grade hardware has gotten more efficient. I wouldn’t really know about that, but I assume most of the focus is on data centers.
Those were two separate thoughts:
- Models are getting better, and tooling built around them are getting better, so hopefully we can get to a point where small models (capable of running on consumer-grade hardware) become much more useful.
- Some modern data center GPUs and TPUs compute more per watt-hour than previous generations.
Can you provide evidence the “more efficient” models are actually more efficient for vibe coding? Results would be the best measure.
It also seems like costs for these models are increasing, and companies like Cursor had to stoop to offering people services below cost (before pulling the rug out from them).
I wish I could, but it would kinda be PII for me. Though, to clarify some things:
- I’m mostly not talking about vibe coding. Vibe coding might be okay for quickly exploring or (in)validating some concept/idea, but they tend to make things brittle pile up a lot of tech debt if you let them.
- I don’t think “more efficient” (in terms of energy and pricing) models are more efficient for work. I haven’t measured it, but the smaller/“dumber” models tend to require more cycles before they reach their goals, as they have to debug their code more along the way. However, with the right workflow (using subagents, etc.), you can often still reach the goals with smaller models.
There’s a difference between efficiency and effectiveness. The hardware is becoming more efficient, while models and tooling are becoming more effective. The tooling/techniques to use LLMs more effectively also tend to burn a LOT of tokens.
TL;DR:
- Hardware is getting more efficient.
- Models, tools, and techniques are getting more effective.
I think this kind of claim really lies in a sour spot.
On the one hand it is trivial to get an IDE, plug it to GLM 4.5 or some other smaller more efficient model, and see how it fares on a project. But that’s just anecdotal. On the other hand, model creators do this thing called benchmaxing where they fine-tune their model to hell and back to respond well to specific benchmarks. And the whole culture around benchmarks is… i don’t know i don’t like the vibe it’s all AGI maximalists wanking to percent changes in performance. Not fun. So, yeah, evidence is hard to come by when there are so many snake oil salesmen around.
On the other hand, it’s pretty easy to check on your own. Install opencode, get 20$ of GLM credit, make it write, deploy and monitor a simple SaaS product, and see how you like it. Then do another one. And do a third one with Claude Code for control if you can get a guest pass (i have some hit me up if you’re interested).
What is certain from casual observation is that yes, small models have improved tremendously in the last year, to the point where they’re starting to get usable. Code generation is a much more constrained world than generalist text gen, and can be tested automatically, so progress is expected to continue at breakneck pace. Large models are still categorically better but this is expected to change rapidly.
deleted by creator
Vibe coding is a black hole. I’ve had some colleagues try and pass stuff off.
What I’m learning about what matters is that the code itself is secondary to the understanding you develop by creating the code. You don’t create the code? You don’t develop the understanding. Without the understanding, there is nothing.
Yes. And using the LLM to generate then developing the requisite understanding and making it maintainable is slower than just writing it in the first place. And that effect compounds with repetition.
TheRegister had an article, a year or 2 ago, about using AI in the opposite way: instead of creating the code, someone was using it to discover security-problems in it, & they said it was really useful for that, & most of its identified things, including some codebase which was sending private information off to some internet-server, which really are problems.
I wonder if using LLM’s as editors, instead of writers, would be better-use for the things?
_ /\ _
A second pair of eyes has always been an acceptable way to use this imo, but it shouldnt be primary or only
They are pretty good at summarisation. If I want to catch up with a long review thread on a patch series I’ve just started looking at I occasionally ask Gemini to outline the development so far and the remaining issues.
If the abominable intelligence is killing every corner of things we consider good its time to start killing the “AI”…
How AI is killing everything.
Which is really it’s purpose, as far as i can see
LLMs definitely kills the trust in open source software, because now everything can be a vibe-coded mess and it’s sometimes hard to check.
LLMs definitely kills the trust in
open sourcesoftware, because now everything can be a vibe-coded mess and it’s sometimes hard to check.I don’t trust proprietary software anyway.
Might make open source more trustworthy, It can’t be any harder to check than closed source.
A week or two back there was a post on Reddit where someone was advertising a project they’d put up on GitHub, and when I went to look at it I didn’t find any documentation explaining how it actually worked - just how to install it and run it.
So I gave Gemini the URL of the repository and asked it to generate a “Deep Research” report on how it worked. Got a very extensive and detailed breakdown, including some positives and negatives that weren’t mentioned in the existing readme.
Yeah, LLMs do a decent job explaining what code does.
I don’t know yet how good Gemini about it,but I think https://deepwiki.com/ this tool will overkill anything for now
yeah it’s to the point now where if I see emojis in the readme.md on the repo I just don’t even bother.
I used to use emojis in my documentation very lightly because I thought they were a good way to provide visual cues. But now with all the people vibe coding their own readme docs with freaking emojis everywhere I have to stop using them.
Mildly annoying.
✨ especially this one ✨
Is the ✨sparkly emoji✨ the <BLINK> of the 21st century? Discuss.
Bring back
<MARQUEE>, dang it.
ttbomk, emojis are legal function-names in both Swift & Julia…
The Swift example was damned incomprehensible, & … well, it was Apple stuff, so making it look idiotic might have been some kind of cultural-exclusivity intention…
The Julia stuff, though, means that you can use Greek symbols, etc, for functions, & get things looking more like what they should…
Also, I think emojis are actually better than my all-text style, for communicating intonation/emotion ( I’m old: learned last century ), & maybe us old geezers ought to adapt a bit, to such things…
That does NOT mean that cartoon “code” is good-enough, whether it’s cartoonish in plaintext or in emojis, though…
I’m just trying to keep the cultural-prejudice & the code-quality being distinct-categories of judgement, you know?
( & cultural-prejudice is an actual thing, though it’s usually called “religious wars”, isn’t it, in geekdom? )
_ /\ _
Check out this one I came across earlier - https://github.com/Jtensetti/fediverse-career-nexus/blob/main/README.md
It’s a federated LinkedIn. ofc it’s vibe coded.
well to be fair you don’t even need to look at the md since right at the top it says it’s built with loveable.
Man… of all the vibe coding tools, Lovable has gotta be one of the most useless, too.
I work with people (all middle managers) who love Loveable because they can type a two sentence description of an app and it will immediately vomit something into existence. But the code it generates is an absolute disaster and the UIs it designs (which is supposed to be its main draw) is some of the most generic crap I’ve ever seen.
0/10, do not recommend.
Who actually tried this?
A handful of people - https://mastodon.social/tags/noltosocial
or anywhere. Job descriptions for example.
Got a job application this with a one line cover letter “Iam interested to work with u are company” it was kinda refreshing to see that instead of a whole page of slop, like most of them are these days.
I think for someone that is very knowledgeable In a project they would probably somehow now if there is vibe coding. I think this will affect brand new projects but not that much of the older codebase. Even think it might enable finding old bugs in old open source codebase.
You are more optimistic than the maintainers of those older projects that have started to ban LLM generated bug reports. They tend to be a waste of time for the maintainers (e.g.: cURL project).
Open source is not only about publishing code: it’s about quality, verifiable, reproducible code at work. If LLMs can’t do that, those “vibe coding” projects will hit a hard wall. Still, it’s quite clear they badly impact the FOSS ecosystem.
This isn’t the problem with the AI, it’s the problem with the user. If you don’t know enough to select the library and make the AI use it, maybe you were never gonna finish the project without AI anyway.
