

Hate to break it to you but quality of data isn’t the fundamental problem with LLMs. It’s that they are trying to use statistics to encode entire thought processes into hidden variables from conversation snippets. They want to use statistics to go from many individual interactions to a large model, and then use that model to predict individual interactions again. Which you can do with statistics, but it’s predicting the average text that follows the prompt, not the correct text (it has no concept of correctness; whenever it “talks” about it, that’s just the average text that follows, not any particular insight into what’s correct or even how it works).
That’s not to say that the quality of the training data has no impact; it can have a huge impact. I’m just saying that even if the training data was perfect, the LLM will still get things wrong in its output.








I looked at the picture for a good 10 seconds trying to figure out what kind of steam generating device it was before realizing it wasn’t a shitpost.