The Bitter Lesson & the Sweet Outcome
Introduction
In his essay, Rich Sutton talks about the “Bitter Lesson” AI Researchers need to face: the distinction between human domain knowledge and the development of AI agents.
The idea that the development of a greater intelligence scales better with the power of general purpose methods (search and learning). Instead of trying to find ways to think about the “contents of the minds” we should build “in only the meta-methods that can find and capture this arbitrary complexity.”
By exemplifying prior “bitter lessons” AI researchers have faced, like the defeat of Chess world champion Kasparov in 1997 or in the example of Computer Go, search and learning based systems that seem to utilize the computational resources available rather than trying to “make systems that worked the way the researchers thought their own minds worked.”
For the future, this implies a couple things. 1. if the development of AI continues to rely on this philosophy, that simply gaining practical intelligent systems is a function of computational resources, then this will see to the exponential development of AI resources (ie data centers). In fact, we already see such a move by both companies and governments. OpenAI’s Stargate is committed to securing 10GW of AI Infrastructure in US by 2029. As of April 29, 2026, OpenAI has already surpassed that milestone, adding more than 3GW in less than 90 days. The US Government has deployed Claude AI for intelligence analysis and “operational support” (essentially for war/defense purposes). The Trump administration secured a $8.9 billion deal for nearly 10% equity stake in Intel. Spearheaded by Oracle and OpenAI, with the $165 billion investment in Project Jupiter, the “plateau” of AI infrastructure development does not seem near.
The greater attention to build AI infrastructure for this purpose will eventually overtake human needs, effectively stealing away from human resources (as we somewhat see with data centers already).
The 2. implication is that this methodology won’t actually allow us to reach true AGI. Simply brute forcing our way to “intelligence” via general purposes isn’t the most effective path forward. After all, evolution doesn’t just optimize for neural quantity, but also for inter-neural connective efficiency. To truly approach AGI, we cannot continue by scaling our compute infrastructure, instead we need a drastic shift in our methodology, one that’s grounded in fundamental truths as how current “intelligent” systems already work. If the universe is converging to some form of intelligence, the best bet we have to get closer to AGI is to utilize the examples we are already given.
To truly reach closer to AGI, we should not simply be focusing on expanding our compute capabilities, but instead be optimizing for efficient computation.
Essentially optimize for # of computations per energy. Mimicking the brain’s ability to “think” and optimize will allow us to both safely and effectively get closer to AGI, after all, our brain is really just a part of the overall “evolution” that the universe is undertaking, so our best bet is to ride along this natural convergence.
We need much better neural research to understand how “intelligence” is built in the brain, then extract the general truth in that system and scale them in our modern compute systems. After all, wasn’t this the methodology behind initial neural networks?
This shall be AI Researcher’s “Sweet Outcome.” We should not be chasing the next big model in the rat race to get more computational abilities, and instead take a step back to look at our overall trajectory; realize that we need to look at this problem from a different perspective, which could most definitely bring us AGI.