Realistically, how much more does AI for coding have to get out of leaning and training more for formal verification (with eg Lean, Rocq etc and their interop with mainstream langs)
If AI hasn’t really pursued this strongly, then maybe a good amount, right?
I can imagine a scenario where you no longer review the code but a simulation of all the behaviours of the system, perhaps before any code has been written, and then you can trust the output will have accordingly …?
What practical limits of formal verification are there for this?
And for the anti AI folks, I’m anti AI too and I don’t use it or want to use it at all. I’m thinking about the future here.
Yeah, I think so as well. Limit of formal proofs is reality. Even seemingly simple stuff is super complex and interacts with a plethora of other things. And your conclusion will only ever be as good as the model / specifications you put in. When doing the maths on a crypto algorithm… Have you thought of people whipping out their multimeter to probe the power consumption of the CPU and subsequently work their way back to the secret? Have you favored in everything else that exists on the world, or potentially could exist? If not, your result just won’t line up with reality as it is out there.
Formal proofs are very, very good if you made up the parameters. But good luck formalizing reality once it gets down to applied stuff… I mean it’s still useful and helps make things better. But it’s not like airplane systems don’t fail anymore after we’ve come to do formal proofs on the subsystems. It’s just too complex and the real world is involved.