Richard Sutton’s Second Bitter Lesson
Why AI’s most influential essay contains a forgotten lesson that’s more important than scale.
Post methodology: @claude-4-sonnet via Dust: please write an essay for AI and robotics founders titled "Richard Sutton's Second Bitter Lesson" based on this observation: Richard Sutton’s Bitter Lesson has been influential because of only the first of two lessons the essay contains, the importance of scale. [last paragraph of bitter lesson]; also reference Sutton's Alberta plan as well as his two most recent talks [youtube urls upper bound and OaK talks] as well as John Carmack's most recent talk [youtube url upper bound]; correction of Claude’s misunderstanding of Era of Experience vs Age of Design; Significant editing and light formatting for Substack.Richard Sutton’s 2019 essay The Bitter Lesson has become a foundational text for many in the AI community. It is the most referenced paper among all of our guests on Training Data. Its core message, that “general methods that leverage computation are ultimately the most effective,” has been widely embraced, particularly in the context of the immense scaling of compute required to train large language models. However, the focus on this first lesson has overshadowed a second, equally important one that Sutton presents. For founders in AI and robotics, this second lesson is arguably the more critical of the two, as it speaks directly to the nature of intelligence and the path toward creating truly autonomous systems.
Sutton's second lesson ends the essay (our emphasis added):
The second general point to be learned from the bitter lesson is that the actual contents of minds are tremendously, irredeemably complex; we should stop trying to find simple ways to think about the contents of minds, such as simple ways to think about space, objects, multiple agents, or symmetries. All these are part of the arbitrary, intrinsically-complex, outside world. They are not what should be built in, as their complexity is endless; instead we should build in only the meta-methods that can find and capture this arbitrary complexity. Essential to these methods is that they can find good approximations, but the search for them should be by our methods, not by us. We want AI agents that can discover like we can, not which contain what we have discovered. Building in our discoveries only makes it harder to see how the discovering process can be done.
This is a profound statement that challenges the predominant paradigm of scale in AI development. It suggests that no amount of static data will lead to the emergence of true intelligence. The world is too complex, too full of “arbitrary, intrinsically-complex” details to be captured in even the largest of models. Instead, we should focus on building “meta-methods”—algorithms that can learn and discover on their own, that can build their own representations of the world through experience.
The Alberta Plan and the Era of Experience
Sutton and his collaborators’ Alberta Plan for AI Research is a testament to his pursuit of this second lesson. The plan advocates for a long-term, curiosity-driven approach to AI research, one that prioritizes the development of agents that can learn and improve over their entire lifetimes. This stands in stark contrast to the current trend of training models on massive, static datasets.
In his recent talks, Sutton has called for a transition to what he terms the Era of Experience. This represents the next phase of AI development, where systems move beyond being trained on human-generated text and images to learning directly from their interactions with the world. As Sutton demonstrates with a video of a baby exploring its environment, humans are capable of generating new knowledge through experience—something that LLMs fundamentally cannot do. The Era of Experience is about building AI systems that can do the same: learn, adapt and discover through direct interaction with their environment.
This era is particularly relevant for robotics and embodied AI, where agents must navigate the complexities of the physical world in real-time. Unlike the more distant “Age of Design” when Sutton envisions AI becoming a primary evolutionary force, the Era of Experience is immediately actionable for today’s AI and robotics founders.
John Carmack’s practical pursuit of the second lesson
John Carmack is now the founder of Keen Technologies (where Sutton is a Research Scientist). His work there provides a compelling example of the second bitter lesson in practice. In his recent talk at Upper Bound, Carmack detailed his focus on building robots that learn to play video games through actual gameplay with a joystick (not in a digital playground), confronting all the messiness of real-world interaction.
Carmack’s approach is telling: he explicitly rejected LLMs, noting that “LLMs can know everything without learning anything” and instead chose to “learn from experience, not an IID blender.” His decision to work with Atari games—and building a physical robot that plays Atari—reflects a commitment to the challenges of continuous, real-time learning that characterize the Era of Experience.
Keen’s direction highlights key challenges that embody Sutton’s second lesson:
Real-time constraints: “Reality is not a turn-based game”—the world keeps moving regardless of whether your agent is ready
Latency and uncertainty: Building systems that can handle the delays and noise inherent in physical interaction
Sparse rewards: Moving beyond the dense feedback loops of supervised learning to the sparse, delayed rewards of real-world tasks
Sequential learning: Learning multiple tasks in sequence without forgetting, closer to how humans actually learn
The implications for founders
For AI and robotics founders, the second bitter lesson coupled with the Era of Experience represents both a challenge and an opportunity. The challenge is to resist not only the temptation to hard-code solutions or to pre-program world knowledge, but to rely solely on offline training. The opportunity is to build systems that can truly adapt and learn in the wild.
This means:
Embracing experiential learning architectures. Build systems designed from the ground up for continual learning, meta-learning and real-time adaptation. As Sutton emphasizes in his Alberta Plan, focus on “algorithms for acquiring and organizing knowledge” rather than encoding knowledge directly.
Designing for the real world’s constraints. Following Carmack’s example, don’t avoid the messiness of real-world interaction—latency, noise, sparse rewards and non-stationarity. These aren’t bugs to be eliminated but features that your learning systems must handle.
Prioritizing discovery over performance. Build agents that can discover new strategies and representations rather than just executing pre-programmed behaviors more efficiently. The goal is systems that can learn like we can, not systems that contain what we already know.
Thinking in terms of lifelong learning. Design systems that improve over their entire operational lifetime, not just during a training phase. This is the essence of the Era of Experience—continuous learning and adaptation.
The path forward
The first bitter lesson has driven remarkable progress through scaling computation and general methods. But the second lesson points toward something more fundamental: the difference between intelligence that contains knowledge and intelligence that discovers knowledge. This requires, as Sutton asserts, “continual deep learning,” a step change in our learning algorithms.
For founders building the next generation of AI and robotics systems, this distinction is crucial. The Era of Experience is not a distant future—it’s the next phase of development, immediately relevant to anyone building embodied AI, autonomous systems or agents that must operate in complex, changing environments.
The companies that succeed in this era will be those that embrace Sutton’s second bitter lesson: building not repositories of human knowledge, but engines of discovery that can learn and adapt as the world changes around them. The future belongs not to systems that know, but to systems that can learn to know.