Thank you for putting in the time to try to explain this. I’ve also been working with ML for a long time and have the intuition that there is a fundamental limit to what can be achieved with Neural Networks and probably any other Machine Learning technique.

Your cake analogy is probably better, but let me try another one. It’s like having a machine that is trying to solve a gigantic puzzle with random pieces from different puzzles. Yeah, if you have enough puzzle pieces and the machine is fast enough it can give you suboptimal solutions but the puzzle will not ever be solved as it is supposed to be solved.

It destroyed the budget of scientific research in Spain. I was a Spanish researcher then. There was a point when universities couldn’t pay us the grant that the government had already assigned to us. Some of my peers missed rent. I heard stories of Spanish PhD. students stranded all over the world. I quit research (my dream job at the time) because of this crisis and never came back to it.

Happy you felt nothing in your happy little bubble.