OpenAI’s science claims need verification, not applause

TL;DR: OpenAI’s science work is most useful when models produce checkable mathematical or scientific artifacts, not when they ask us to trust a clever answer.

What did OpenAI actually claim?

OpenAI’s announcement, “Working with academic communities to build scientific understanding,” says the company is sharing model results on three hard domains: the Kakeya conjecture, the Exponential Time Hypothesis, and quantum chromodynamics. That is an ambitious spread. One lives in geometric measure theory. One sits inside computational complexity. One is core physics.

The important part is not that those names sound impressive. The important part is OpenAI’s framing: researchers can verify and build on the results. That is the right bar.

Science does not need more fluent explanations of unsolved problems. We have plenty of those. It needs candidate proofs, counterexamples, derivations, simulations, reductions, code, and intermediate lemmas that other people can inspect. The model output has to leave a trail.

That is especially true in math-adjacent work. A model can write something that reads like a proof and still be wrong in a subtle way. It can also generate a useful construction wrapped in bad prose. The difference only shows up when domain experts try to formalize, reproduce, or break it.

So my read is simple: this is not a victory lap for autonomous science. It is a push toward AI as a proposal engine for expert communities.

Why does verification matter more than model confidence?

Because scientific claims compound.

If a coding agent makes a bad change, the test suite may catch it. If a support bot gives a bad answer, a user may complain. If a model suggests a false lemma that later gets cited, generalized, and built into other work, the damage is quieter.

That does not mean models should stay out of science. It means the workflow needs to be designed around distrust.

For the Kakeya conjecture, ETH, and quantum chromodynamics, the verification burden is different in each case. A math result may need proof checking or expert review. A complexity claim may need careful assumptions and reductions. A physics result may need consistency with known theory, numerical methods, and existing literature. One evaluation style will not cover all three.

This is where a lot of “AI for science” hype gets sloppy. People talk as if the model is a scientist. Sometimes it is closer to a strange graduate student with infinite patience, uneven judgment, and no internal sense of embarrassment. Useful, yes. Dangerous if unsupervised.

The practical question is not “can it reason?” It is “can it produce something worth checking, and can we check it cheaply enough?”

What should builders take from this?

The pattern applies outside frontier research.

If you are building AI into any expert workflow, do not measure success by how polished the answer looks. Measure the quality of the artifact it hands to the next step. A contract markup. A SQL query. A materials hypothesis. A clinical note draft. A testable SEO experiment. A circuit design. A proof sketch.

The artifact should be inspectable. It should expose assumptions. It should separate known facts from proposed moves. It should make verification faster than starting from scratch.

That is the operator lesson in OpenAI’s announcement. The model is not the product. The review loop is the product.

For builders, I would try this on one narrow workflow: make the model generate three candidate artifacts, force each one into a structured format, then have a human or toolchain reject, revise, or accept them. The catch most readers miss: the big productivity gain may come less from the model being right on the first pass, and more from making expert review sharper, faster, and harder to fake.