• 4 Posts
  • 569 Comments
Joined 3 years ago
cake
Cake day: June 30th, 2023

help-circle
  • I’d like to propose a combination of the two. A language model that parses natural language but checks back in with the user to see if it has understood. This can be done by converting high-ambiguity sentences to low-ambiguity ones when appropriate and storing the latter as source. When the interpretation program is confused, it can check back in with the user to ask what they meant.

    This is something I’ve been thinking about lately. It’s a huge problem that looking into how a given software project works or specifically what it does is normally beyond the reach of most people, or in the case of software that is very elaborate or wasn’t written to be read, beyond the reach of almost everyone. It could help a lot to have some kind of tiered specification/documentation going from more concise to more detailed that can at least be independently confirmed in an automated way to have been derived from each other.


  • Game Oracle’s initial research, even at a surface level, is eye-opening. It studied almost 10,000 Steam releases between January and October 2025, discovering that games disclosing AI use averaged just 4 reviews in the first post-launch month compared to 7 reviews for games without AI.

    What I want to know is whether this study involved any sort of pre-screening quality filter for the games considered. If that’s not accounted for, there’s definitely going to be a larger volume of very low effort asset-flip-equivalent AI games just because it would be faster and more scalable to make them that way, which would skew the numbers and not show whether people are avoiding games due to the AI label independently of their quality otherwise.

    Edit: I realized I didn’t check the text of the study so I went and did that, looks like this is addressed:

    We mitigated this however by filtering “slop” out of our dataset based on publication frequency and absurd initial prices (>$100; there is a little over 100 games in this group and most are scams). We removed developers whose historical publication rate is greater than 1 game per every 6 months working






  • you are taking a risk either way. You are placing your trust in the dev and the few that can read code.

    There is definitely a trust issue and a need for ways of conveying and building trust in smaller software projects. I think a much better solution there would be discussions about the code and how it works that aren’t hostile interrogations with foregone conclusions in pursuit of a broader anti-AI agenda. If someone just put a lot of effort into making something the details of that process should be on their mind, it should be possible to make them more accessible to people and convey that there is non-artificial understanding behind the project. Automatic hostility and suspicion makes those kinds of conversations harder and less likely.



  • There’s ways for a comic to be good that go beyond just the strength of the joke. I really like the art style, the color scheme, the ways the emotional state of the characters is expressed, and the background details. It just conveys a sense of this setting and the people in it really succinctly. It would probably be bad if it was a comic with the same joke but much worse execution.










  • Except that it’s been demonstrated multiple times that original training data can be extracted from a language model, so it is completely valid to talk about the model as a database, because the training data is stored within it.

    It’s been demonstrated that some more prominent pieces of training data can be reproduced, the majority of it cannot. This shows that those particular pieces of data are represented in some form within the model, it does not show that the way it works is equivalent to database lookups. If I can write down the lyrics of a song from memory, it shows that those lyrics are encoded as data in some form in my brain, but that doesn’t mean it’s valid to talk about my brain as a literal database, especially not in the sense that the limitations in the capabilities of a database can be ascribed to me (or its strengths, I cannot remember the exact lyrics of most songs I’ve heard, even if I can remember some).

    Hill Climbing Algorithm & Artificial Intelligence - Computerphile

    This video literally starts out by describing evolution as a similar optimization algorithm. If you know the basic mechanism of evolution, does that mean you can use that to then say with certainty and specificity what biological life in its vast diversity of techniques is not capable of? The “underlying operating principles” of evolution don’t “understand” chemistry or deception, but they still produce organisms capable of photosynthesis and camouflage. It’s an algorithm that produces other algorithms, which is what puts those resulting algorithms in a different category of comprehensibility than fixed algorithms that were explicitly written by someone. We are very far from having a comprehensive understanding of biological systems, despite knowing how evolution works.

    This is not a tool that can comprehend what it is doing, it can’t be self-aware. It can only process large amounts of input data and attempt to maximize for particular dimensions. This seems vague to humans because the amount of variables being handled at any given time is far more than a human mind can focus on, but that doesn’t make the optimization routine intelligent or conscious. It’s just doing a lot of number crunching really fast, optimizing for specific aspects as directed by its developers.

    This is like saying evolution is only a simple mechanism taking in the world as data, which, yeah, obviously, but that property doesn’t carry forward to what it produces. The bigger problem here though is, again, concepts like comprehension, consciousness, and intelligence are not well defined in computational terms, and it is unclear what statements involving them mean in any practical sense. These sorts of claims are non-falsifiable and don’t make testable predictions about the boundaries of AI capability.


  • Language models are not databases and they are not markov bots (similar function but work directly using statistical word association maps). The big difference is that those things are algorithms someone wrote and can fully comprehend what they do, but machine learning models are large algorithms built by another algorithm processing training data. There is much more uncertainty about what is going on under the hood.

    There is also great uncertainty about what concepts like understanding or thinking might mean in computer science terms. The main thing we can really know is that ultimately a human mind is a computer, which means that understanding and thinking have some yet unknown mathematical representation, and therefore a comparison can be made. We should eventually be able to quantify whether or to what extent a given algorithm thinks. But you said in another comment that you don’t believe minds can be represented mathematically; this should mean that such comparisons would be apples to oranges, but you’re making them anyway for some reason, and implying they have predictive power for the limitations of LLMs.

    Certainly they do have limitations, at least individually and possibly as a technology. There are things given models are bad at, there are things they initially seem to be able to do well as humans but fail in different ways that suggest over-reliance on pattern matching. But these have been determined empirically through testing. The idea that they are “just statistical models” and this knowledge can be used to say what is impossible for them from philosophical first principles keeps getting repeated but has never worked in practice. The reality is that no one knows enough to say for sure where the line is.