Images generated by AI models trained on massive datasets often can’t be traced to specific training images, MIT CSAIL researchers found. Removing individual images from the dataset didn’t change outputs, complicating copyright questions.
Amongst other things, they experimented with “what happens to a generation if you remove one image from the training dataset,” particularly the “source” image for the target generation. What makes it interesting is they tested this at scale. There’s (expectedly, somewhat janky) demo code to illustrate the spread. One example I exported myself:
Where each generation (the “counterfactual”) in the top row corresponds to the image that was removed from the dataset in the bottom.
…Personally, I don’t find the conclusion surprising.
These are models.
Lets say you make a model to predict hurricanes. With a small dataset/model, exclude the hurricane most similar to the one you’re trying to predict, and it won’t do a good job. But do the same with a model based on a huge dataset, and it should still model the novel hurricane reasonably well.
Diffusion models are no different.
I think AI Bros have overly anthropomorphized them with terms like “creativity” and such; they don’t have this. But at the end of the day, they can model things that aren’t strictly in their dataset. That’s kind of the basic premise.
Yeah people don’t understand the math behind these things. They are statistical models, like demographics or climate models. They are interesting and potentially useful, but not necessarily useful for the things we are trying to and want to use them for. They can predict certain things and trends very reliably in a general, high-level sense, but they will get the details wrong if you try to use them outside of their area of applicability, just like if you try to use national climate models to determine what day your basement is going to flood, it may come up with a specific date, but just as obviously that’s not actually going to be correct in reality. It doesn’t mean it’s a bad model, it means you’re using it wrong.
And lots of AI bros and AI companies are using these things desperately wrong, and for very wrong reasons. They want to sell things to you. They want to control you and manipulate you. They don’t care if they’re using the model properly, they only care if it’s profitable for them to do it.
The entire term of “Artificial Intelligence” is basically a misnomer, false advertising, even fraud. It is merely an advanced “correlation engine”. It is the output of an algorithm, and has no concept of what anything is. It does not “know” anything. It is the result of the 1’s and 0’s humans input as training data or context, or give it access to query via the internet etc; including all the misinformation, disinformation, errors, and bias. There is no intelligence or critical thought. It does not get the difference between causation and correlation. It just responds based on some internal probability matrix or distribution; correlations compressed into weights/coefficients from all of the training data.
The only thing that makes its responses appear stochastic is the randomization seed and context, or other techniques harnesses use to dogfood its own responses into itself. Without those, you can have the exact same conversation, byte for byte, and the responses become evidently deterministic and “dumb”.
All of the “novel” solutions I’ve seen to date are not actually novel at all, having merely applied some technique documented in a far removed discipline, entirely explained by the properties of a gargantuan working memory and no off switch. We don’t call a computer “super intelligent” because it can process trillions of mathematical calculations a second. Those are merely properties we explicitly engineered into the machine.
…Perhaps. But I’m not sure what any of this has to do with the paper. It doesn’t claim anything like that, and the term “Artificial Intelligence” does not appear in anywhere in the work.
It does illustrate that a toy diffusion model can approximate an image outside its dataset. In fact, the target image being present in the dataset has a remarkably small effect as training scale goes up.
I don’t think this is really all that surprising if you understand how diffusers actually work. They don’t just draw contours and shit. They apply sequential transformations across a matrix of pixels in an iterative fashion, the transformation varies each iteration based on prompt embeddings and the surrounding pixels of each respective transformation, typically starting from a seed of arbitrary static. A lot of images share overwhelmingly identical techniques, with a tiny sliver being responsible for the truly creative deltas. Diffusers effectively learn techniques so it tracks that dropping singular training data here and there doesn’t change much.
Okay so the paper is actually interesting, and they have a explainer website: https://zheng-dai.github.io/AblationBasedCounterfactuals/
https://www.nature.com/articles/s41467-026-75667-5
https://github.com/zheng-dai/counterfactualuniverses
Amongst other things, they experimented with “what happens to a generation if you remove one image from the training dataset,” particularly the “source” image for the target generation. What makes it interesting is they tested this at scale. There’s (expectedly, somewhat janky) demo code to illustrate the spread. One example I exported myself:
Where each generation (the “counterfactual”) in the top row corresponds to the image that was removed from the dataset in the bottom.
…Personally, I don’t find the conclusion surprising.
These are models.
Lets say you make a model to predict hurricanes. With a small dataset/model, exclude the hurricane most similar to the one you’re trying to predict, and it won’t do a good job. But do the same with a model based on a huge dataset, and it should still model the novel hurricane reasonably well.
Diffusion models are no different.
I think AI Bros have overly anthropomorphized them with terms like “creativity” and such; they don’t have this. But at the end of the day, they can model things that aren’t strictly in their dataset. That’s kind of the basic premise.
Yeah people don’t understand the math behind these things. They are statistical models, like demographics or climate models. They are interesting and potentially useful, but not necessarily useful for the things we are trying to and want to use them for. They can predict certain things and trends very reliably in a general, high-level sense, but they will get the details wrong if you try to use them outside of their area of applicability, just like if you try to use national climate models to determine what day your basement is going to flood, it may come up with a specific date, but just as obviously that’s not actually going to be correct in reality. It doesn’t mean it’s a bad model, it means you’re using it wrong.
And lots of AI bros and AI companies are using these things desperately wrong, and for very wrong reasons. They want to sell things to you. They want to control you and manipulate you. They don’t care if they’re using the model properly, they only care if it’s profitable for them to do it.
The entire term of “Artificial Intelligence” is basically a misnomer, false advertising, even fraud. It is merely an advanced “correlation engine”. It is the output of an algorithm, and has no concept of what anything is. It does not “know” anything. It is the result of the 1’s and 0’s humans input as training data or context, or give it access to query via the internet etc; including all the misinformation, disinformation, errors, and bias. There is no intelligence or critical thought. It does not get the difference between causation and correlation. It just responds based on some internal probability matrix or distribution; correlations compressed into weights/coefficients from all of the training data.
The only thing that makes its responses appear stochastic is the randomization seed and context, or other techniques harnesses use to dogfood its own responses into itself. Without those, you can have the exact same conversation, byte for byte, and the responses become evidently deterministic and “dumb”.
All of the “novel” solutions I’ve seen to date are not actually novel at all, having merely applied some technique documented in a far removed discipline, entirely explained by the properties of a gargantuan working memory and no off switch. We don’t call a computer “super intelligent” because it can process trillions of mathematical calculations a second. Those are merely properties we explicitly engineered into the machine.
People forget that a checkers algorithm is also AI.
…Perhaps. But I’m not sure what any of this has to do with the paper. It doesn’t claim anything like that, and the term “Artificial Intelligence” does not appear in anywhere in the work.
It does illustrate that a toy diffusion model can approximate an image outside its dataset. In fact, the target image being present in the dataset has a remarkably small effect as training scale goes up.
I don’t think this is really all that surprising if you understand how diffusers actually work. They don’t just draw contours and shit. They apply sequential transformations across a matrix of pixels in an iterative fashion, the transformation varies each iteration based on prompt embeddings and the surrounding pixels of each respective transformation, typically starting from a seed of arbitrary static. A lot of images share overwhelmingly identical techniques, with a tiny sliver being responsible for the truly creative deltas. Diffusers effectively learn techniques so it tracks that dropping singular training data here and there doesn’t change much.