The ideal is that the training data don’t exist on it. The reality is that the training algorithm is a very very weird compression algorithm and sometimes the model contains just derived properties and sometimes you can perform sample recovery and get entire books back out. That which is recoverable with a prompt less complex than the output is inherently already in the model, because that’s simply how entropy works.
The ideal is that the training data don’t exist on it. The reality is that the training algorithm is a very very weird compression algorithm and sometimes the model contains just derived properties and sometimes you can perform sample recovery and get entire books back out. That which is recoverable with a prompt less complex than the output is inherently already in the model, because that’s simply how entropy works.