It learned from us. So did we.

Transcript

Rethink your priors.

'LLMs just remix their training data.' Okay. But does learning from existing information mean you can only repeat conclusions someone already reached? That's the leap I don't buy.

There's a strong objection worth taking seriously. Can these systems reliably produce important new explanations, especially when humans haven't already supplied the problem and the test?

That's an open question. It's different from the stronger claim. They cannot discover anything new because they learned from existing data.

My issue with that claim is that it confuses where knowledge comes from with what can be discovered from it.

Having the data isn't the same as having discovered everything it implies.

Humans make discoveries by finding previously unnoticed relationships in existing evidence. Answers can be implied but unworked.

Otherwise learning from textbooks would disqualify humans too.

The real test is whether a system produces something genuinely new that survives independent verification.

FunSearch paired an LLM with an automated evaluator. It found new constructions for the cap set problem, beyond the best known results. Published in Nature.

AlphaEvolve found a way to use 48 scalar multiplications instead of 49, for four-by-four complex matrices.

These are LLM-based systems with search, evaluators and human-designed objectives. Not a chatbot becoming Einstein. But the LLM is part of a process that produced a verifiably new result.

A common reply. A human chose the problem and told it where to look.

A human choosing the problem tells us something about the system's autonomy. It does not, by itself, tell us whether the solution is novel. Those are different questions.

And the famous historical test. Train a model only on what came before a discovery, and see if it gets there.

One experiment trained a small model on pre-1900 text. It did not establish that the model rediscovered relativity. That matters. But failing to demonstrate a capability is not the same as proving it impossible.

That test asks what a model can discover from existing evidence. A system that can also run experiments faces a broader test. We should evaluate both.

Today's chatbots aren't every possible AI. And humans build on prior knowledge too. Don't impose a from-nothing requirement only on machines.

Be skeptical of the result. Demand evidence. Ask how much the human did. Those are good questions. But 'it learned from us' is not proof that it can't discover something we missed. We learned from us too.