Using computers to search for a counter example to a conjecture isn’t exactly new ground and I suspect they did so with the aide of some harness tweaks like some numerical LSP. Like cool, it pushed the envelope but like what the parent said, they grafted on the ability to do a specific task.
That doesn’t change the fact that llm’s are capable of acing math olympiads. So what if it uses tools? You probably would too. I doubt anybody there did it without a calculator.
Aren’t you the least bit curious what tools they gave the LLM and how the LLM used those tools? It’s like back in math class you are asked to solve a quadratic formula but you forgot how. So you use the calculator to try different numbers and the calculator is telling you if you are getting closer. Sure I got the right answer, but it’s hardly a testament to my math skills.
Using computers to search for a counter example to a conjecture isn’t exactly new ground and I suspect they did so with the aide of some harness tweaks like some numerical LSP. Like cool, it pushed the envelope but like what the parent said, they grafted on the ability to do a specific task.
That doesn’t change the fact that llm’s are capable of acing math olympiads. So what if it uses tools? You probably would too. I doubt anybody there did it without a calculator.
https://www.nature.com/articles/d41586-025-02343-x
Aren’t you the least bit curious what tools they gave the LLM and how the LLM used those tools? It’s like back in math class you are asked to solve a quadratic formula but you forgot how. So you use the calculator to try different numbers and the calculator is telling you if you are getting closer. Sure I got the right answer, but it’s hardly a testament to my math skills.