• 0 Posts
  • 877 Comments
Joined 3 years ago
cake
Cake day: June 16th, 2023

help-circle

  • Of that comes to pass, they can change it.

    The codegen is a particularly big problem for famous open source projects. All of a sudden you have a flood of people trying to get their name on your project and flooding you with low quality stuff. Codegen enables folks without good ideas to generate submissions that demand a lot of review. Being about to shoot down obviously AI content without deeper review is pretty critical to keep the project viable. There may be good things tossed out with the bathwater, but it is a ton of bathwater. Besides, if there’s something real behind it, a good human is likely to emerge to take it forward more credibly.

    Closed source or personal projects can enjoy a bit more judgement. You can evaluate the people independent of the submissions and may be able to have an even more nuanced take.


  • It’s not like it is a legally binding agreement that needs a high level of quality in adjucation. If things seem off then it can be cited and you don’t have to prove it. Conversely, if something passes human review despite being codegen, well then I guess the codegen isn’t that bad

    I’ve seen a pull request that was ultimately nonsensical but plausible and short, and purported to be needed in a configuration none of the maintainers had access to. It was a pretty harmless change either way so they were inclined to accept it. But they sought a little clarification and the clearly LLM response in comment got it shut down. No human stepped up to actually explain an understanding and the LLM sounded like a TV show writer writing tech, sentences that sounded credible but to anyone with understanding was nonsensical bullshit.

    LLM code can sometimes be hard to tell in a short code change, but the prose when things are anything vaguely needing explanation is just so very very obvious.










  • In the same way it’s accurate to call the human brain a prediction machine

    I have seen no evidience presented that the human brain is simply making statistical predictions.

    This seems to be a relatively new thing some people are asserting, because they see the LLM behavior and have decided that absolutely it looks so human like it must act the same, and since we understand that LLM is fundamentally statistical prediction of a token, then obviously human consciousness must operate on the same principle.

    Even the latest models generate the sort of mistakes that stem from the fundamental limitations. A lot has been done in some contexts for making that not matter so much (e.g. in software development, the strategy is that code may have some verifiable goal, and the models can let the mistakes fly, then take the feedback from the goal, and iterate more). So it still can be very useful, but clearly it isn’t human like because of some of the completely dumb behaviors that result from it not actually thinking about it. It is at it’s best when either mistakes don’t matter (particularly fiction) or at least first mistakes don’t matter and can be automatically reconciled with facts.

    By nature, the meme examples will get better because everyone talks about it and suddenly having all that discussion in training data and in search results, the statistics fix. However the operating principle behind them remains the same.