We understand the proof (most math from all of history) -> We understand how the proof was made (computer-assisted proofs like the four-color theorem) -> We have to trust the computer's explanation for how the proof was made (some LLM proofs)
Is that really a problem? The only risk in a Lean proof is the LLM finding a bug in multiple kernels (2 with the standard setup) that would let it add a new theorem without legitimately proving it. If you have reasoning traces or send a second model to look at the proof you can easily detect that and file a bug report.
For the rest you only need to check that the formal problem statement matches the actual problem and the proof is not using extra axioms. That's easy to check manually or with a simple script.
Many mathematicians care about understanding the method and the tools used more than the result. LLMs have a large advantage in memory and reading speed, you can feed them the entire internet and published literature at training time, and then make them check dozens of papers in a few minutes at runtime. They might not be very creative, but they've seen that obscure 1990's theorem in the training data that happened to use an idea that might just work again.
Reading the LLM's 12 MB lean output is a bit like trying to maintain a codebase that has been fully vibe-coded as a single giant file, by hand. Even if it's possible in principle, no one is going to enjoy that.
I understand the spirit of what you're saying, but "unassisted humans" isn't a good yardstick. There's hardly anything "unassisted" humans understand today... We require plenty of assistance from computer tools in most scientific discoveries.
This is based on a 108 page prose paper that the repository links to. Of course, that's a very difficult paper as well, I don't know how many people would be qualified to read and digest it, but they do exist.
That reminds me that people who hear the word for butterfly (Schmetterling) in German for the first time usually think it sounds extremely violent. Ok, then again, "butterfly" sounds slightly disgusting if you think about it. But TIL that they actually have a similar root: the "Schmetter-" doesn't come from smashing, but from "Schmetten", a word derived from the Slavic "smetana" (cream). Apparently butterflies got their name from the popular belief that witches could turn into butterflies to steal milk or cream (and probably also other dairy products?).
As prediction markets already show, forecasts can influence the outcomes they are trying to predict.
What happens when these models become extremely accurate and widely trusted? A forecast like “Will there be a war between countries A and B?” may itself affect whether the war happens.
If the model says there is a 1% chance of war, little changes. But if it says 90%, governments, markets, militaries, and the public may react: capital flees, troops mobilize, diplomatic trust collapses, and each side starts preparing for the other side’s preparation. The prediction helps make itself true.
The same feedback loop could apply to bank runs, market crashes, civil unrest, elections, and corporate failures.
At some point, the most accurate forecaster may become less like an observer and more like an actor with enormous power over the system it predicts.
And if your muscle memory doesn't allow an immediate transition, give yourself 2 weeks. That's how long it took for me at least.
Invest in custom tooling and helper scripts.
reply