No, that’s just a big mistake.
Richard Sutton, one of reinforcement learning‘s founding figures and co‑founder of Oak Lab with former student Khurram Javeed, is making a blunt case: synthetic data and hand‑built simulators cannot be the scalable shortcut to broadly capable agents. His diagnosis rests on what Javeed calls the “Big World Hypothesis“, the idea that the internet is a finite corpus, while the world an agent must master is effectively open‑ended and historically rich.
What Sutton means by “big world”
Synthetic data here means data produced by simulators or programmatic generators intended to replace or augment real‑world training sets. Sutton’s core argument is threefold:
- Scale mismatch: the world is, in his words, “massively bigger than everything we stored on the internet.” Simulators, however elaborate, remain tiny in comparison.
- Other minds are different: “There’s no way we can have synthetic data for other people’s minds, ” he says, modeling internal goals, beliefs, and cultural context is not the same as generating surface behavior.
- Human bottlenecks: as Javeed puts it, “You need human experts who know what’s a good data set and what’s a bad data set for that approach to scale. So it is bottlenecked by humans.”
Those are strong claims, and deliberately provocative. Sutton also praises recent progress: language models were built by “drink[ing] in the internet, ” and he calls them “an amazing scientific breakthrough.” He estimates they capture roughly “like 20% or a quarter of intelligence, ” a subjective assessment meant to separate impressive pattern‑matching from the persistent, embodied learning he argues agents still need.
Where synthetic data helps, and where it hits limits
Synthetic data is far from useless. It speeds early development, supports safer testing, and amplifies rare events: domain randomization in robotics and procedurally generated scenarios are classic examples where simulation meaningfully shortens time‑to‑trial. It’s also invaluable when privacy or safety prevents collecting real examples.
But the limits matter for products that must model people, maintain long historical context, or adapt to shifting social norms. Sutton points to familiar engineering realities: teams train in simulation, then send systems into the real world to close the remaining gaps. The sim reduces cost and concentrates risk, but it rarely removes the need for lived experience and continual correction.
Why Sutton favors agents that learn from life
Instead of trying to pre‑encode more of the world, Sutton argues for agents that learn continually from their own interactions. “Simulators they make themselves, ” he says, internal world models constructed and refined by agents as they act. This reverses the pipeline: rather than designers baking in scenarios, agents build understanding through experience.
Core to that vision is continual learning: keeping models adaptive over time without erasing past competence. Sutton criticizes the common deployment pattern that leaves base models largely static, “Their weights never change, ” and he contrasts that with his claim that “all learning is continual.” Many production systems do use fine‑tuning, adapters, retrieval systems, or replay mechanisms to adapt between major updates, but Sutton’s point targets the default frozen‑model pattern and its limits for open‑ended competence.
He also points to a technical contribution from his team called “Continual Backprop, ” which he says was published in Nature and is part of the path they’re exploring. Sutton presents that method as a practical step toward avoiding catastrophic forgetting and enabling long‑term learning. The claim is framed as his team’s work and a direction, not a turnkey solution to every challenge of lifelong learning.
Why business leaders should care
If Sutton is right, the strategic choices you make now, about data budgets, testing regimes, and model maintenance, will shape how quickly your AI products adapt to real customers.
- Roadmaps should treat simulation as a force multiplier, not a replacement. Use synthetic data to speed iteration and stress‑test edge cases, but budget for real‑world feedback loops and fine‑tuning.
- Deployed models need maintenance and observability. If frozen base models limit adaptation, then invest in monitoring, safe update pipelines, adapters, and rollback mechanisms.
- Prioritize narrow, sandboxed online learning experiments. Small tasks (re‑ranking, slot filling, personalization) are safer proving grounds for continuous adaptation than full embodied agents.
- Reorient expertise toward evaluation and safety. Move subject matter experts away from repetitive labeling toward defining reward signals, adversarial tests, and acceptance criteria.
How to act now, a prioritized 90‑day starter plan
- Instrument key drift metrics (effort: small). Choose three business‑critical metrics (conversion by cohort, failure rate in escalations, user correction rate) and start tracking them daily. You can’t adapt what you don’t measure.
- Launch one sandboxed online adaptation experiment (effort: medium). Pick a narrow task where online signals are abundant (e.g., recommendation ranking). Run a controlled test that allows the model to adapt in a restricted environment with automated rollback and human review gates.
- Define a human review rotation and adversarial test suite (effort: small, medium). Rotate SMEs through evaluation shifts, and maintain a growing set of failure cases that must be resolved in sim and real before a rollout.
These are practical, budget‑sensitive steps that turn Sutton’s critique into manageable engineering bets rather than all‑or‑nothing gambles.
When to favor synthetic data, and when to avoid relying on it
- Favor simulation when the rules are well understood (robotic kinematics, industrial controls) or when you must amplify rare but well‑specified events (fraud scenarios, extreme weather tests).
- Be cautious when the task requires modeling changing human preferences, long social histories, or the hidden states of other agents. Those are the regimes where Sutton argues synthetic data struggles to scale.
Balancing viewpoints
Sutton’s critique is corrective, not a wholesale rejection. Synthetic data has proven value; his point is strategic: don’t expect designer‑authored worlds to replace the agility that comes from agents learning in and from the actual environments where they’ll operate. Many teams will find a hybrid path most practical, simulation for safety and breadth, plus robust real‑world feedback loops for depth and adaptation.
Key questions you should be asking
-
Is synthetic data a scalable substitute for real experience?
Sutton argues it is “a big mistake” to treat synthetic data as a full substitute because simulators remain “microscopic” compared to the world’s complexity and because creating high‑quality synthetic datasets often requires human expertise, which can bottleneck scaling.
-
Do current LLMs already solve the problem by absorbing everything on the internet?
Sutton calls LLMs “an amazing scientific breakthrough” but estimates they cover only “like 20% or a quarter of intelligence”, a subjective judgement that highlights their strengths in pattern‑rich text but also their limits as largely static models without persistent, grounded world models unless extended with continual learning methods.
-
What alternative does Sutton propose?
Agents that learn continually from their own interactions, building internal simulators (“Simulators they make themselves”) and updating over time; Sutton points to a method his team calls “Continual Backprop” (which he says was published in Nature) as part of that approach.
-
How should businesses change their AI roadmaps?
Treat simulation as a force multiplier, not a replacement. Invest in instrumentation, safe online adaptation experiments, model maintenance, and human expertise focused on evaluation and safety rather than rote curation.
Practical AI for business is a craft as much as a technology choice: combine the cost‑efficiency of simulation with the reality‑checking power of continuous, real‑world learning. Start small, measure everything, and give your systems the pipelines they need to keep learning safely once they leave the lab. Sutton’s message is blunt but useful: scale matters, but scale without life, without continual adaptation to messy social reality, will leave systems brittle where it counts.