Learning with LLMs: Cognitive Shortcut or Cognitive Debt?

New research suggests learning to write should come before introducing generative AI in the classroom.

Post methodology: @gemini-pro @claude-4-sonnet via Dust: write an essay based on [Mind on ChatGPT paper, Dan Rockmore talk and New Yorker article] about implications of using LLMs in education vs calculators. Rockmore contends that learning to write teaches critical thinking that then makes people better users of LLMs, suggesting we should teach writing first before introducing LLMs in the classroom; @claude-4-sonnet: go into more detail on what the MIT study actually tested and rely less on sources you found by search; write a paragraph explaining who the participants were and questions around implications for younger students. Light editing and formatting for Substack.

The integration of any transformative technology into the classroom inevitably sparks a familiar cycle of excitement, anxiety and debate. Decades ago, the pocket calculator prompted fears that students would lose fundamental arithmetic skills. Today, a similar and far more complex debate surrounds the arrival of large language models (LLMs), sophisticated AI that can generate nuanced, coherent text on command. While the calculator was a tool for computation, the LLM is a tool for communication and creation, raising profound questions about the future of education. However, recent neuroscience research from MIT reveals that the calculator analogy fundamentally misrepresents what happens in the brain when students use these technologies. Unlike calculators, which actually allowed more students to engage in mathematical reasoning, LLMs can create what researchers term “cognitive debt”—a measurable reduction in the very neural activity that builds critical thinking skills.

Similar to the findings in our essay on The AI Productivity Paradox, “the mere presence of new technology was not sufficient to drive productivity; complementary factors such as organizational change, skills, and business process innovation were essential.” In the case of education, the complementary changes required for AI to drive accelerated learning involve the groundwork teachers lay before they introduce LLMs in the classroom. In professional settings we assume a baseline of critical thinking skills that people learned in school. But as Agency founder Elias Torres explains in this week’s episode of Training Data, many companies are also approaching AI in the workplace as if it were a calculator and “demanding this level of perfection” while “completely misunderstanding the technology and its capabilities.”

The seductive appeal of technological parallels

Dartmouth professor Dan Rockmore’s New Yorker article What It’s Like to Brainstorm with a Bot captures the genuine excitement that many academics feel about AI collaboration. His account of colleague Luke Chang’s research breakthrough on a long drive—using ChatGPT to solve a vexatious visualization problem—illustrates AI’s potential as what Rockmore calls a “restless partner in the workshop.” Chang’s experience exemplifies the optimistic view: a knowledgeable researcher using AI as a sophisticated thinking tool, maintaining agency over the creative process while leveraging the machine’s pattern-matching capabilities.

Yet this historical parallel obscures crucial differences between LLMs and their predecessors. The calculator analogy, in particular, has become a conceptual trap that prevents us from recognizing the unique neurological challenges that generative AI poses to education.

A comprehensive examination of LLMs’ impact on learning comes from a rigorous study conducted by researchers at MIT’s Media Lab, Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task. This study goes far beyond subjective impressions or theoretical concerns—it provides direct neurological evidence of what happens in students’ brains when they use AI for academic work.

The researchers designed a sophisticated experiment involving 54 participants divided into three groups: an LLM group that used ChatGPT for essay writing, a search engine group that used traditional web search, and a brain-only group that wrote essays without any digital assistance. Each participant wrote essays across four sessions over several months, with their brain activity monitored using electroencephalography (EEG) throughout the writing process.

The experimental design was particularly clever. In the first three sessions, participants remained in their assigned groups. But in the crucial fourth session, the researchers switched conditions: participants who had been using ChatGPT were asked to write without any tools (LLM-to-Brain), while the brain-only group participants were given access to ChatGPT for the first time (Brain-to-LLM). This crossover design allowed researchers to observe how prior AI use affected subsequent unassisted performance, and how participants that demonstrated strong foundational skills adapted to AI assistance.

The topics were drawn from SAT essay prompts on subjects like art, philanthropy and happiness—complex enough to require genuine thinking and analysis. Participants had 20 minutes per essay, and the researchers collected not just the final essays but also detailed interviews about participants’ sense of ownership, their ability to quote from their own work, and their satisfaction with the results.

The neurological reality of cognitive debt

The results revealed a striking pattern that fundamentally challenges the calculator analogy. While participants using ChatGPT produced essays that were rated higher by both human teachers and AI judges, their brains told a different story. These students reported feeling that the task required less cognitive effort, and indeed it did.

Using a sophisticated analysis called dynamic directed transfer function (dDTF), the researchers measured neural connectivity patterns across different frequency bands associated with various cognitive functions. The findings were unambiguous: brain connectivity systematically scaled down with the amount of external support. The Brain-only group exhibited the strongest, widest-ranging neural networks. The Search Engine group showed intermediate engagement. The LLM assistance elicited the weakest overall coupling between brain regions.

Specifically, the LLM group showed significant reductions in alpha and beta frequency bands in the frontal and temporal-parietal regions—areas crucial for attention, working memory, and language processing. The reduction was particularly pronounced in the left hemisphere, which is dominant for language processing. This suggests that ChatGPT was not merely assisting with writing mechanics, but was offloading the core cognitive work that builds thinking skills.

The researchers termed this phenomenon “cognitive debt,” drawing an analogy to technical debt in software engineering. Just as poor programming choices create long-term maintenance costs, the short-term benefits of AI assistance may come at the cost of long-term cognitive development.

The most revealing findings emerged in the fourth session, when participants switched conditions. The LLM-to-Brain participants—those who had become accustomed to AI assistance—showed weaker neural connectivity when forced to write without ChatGPT. Their brain activity patterns suggested they had become dependent on external cognitive support and struggled to re-engage the neural networks necessary for independent writing.

Even more telling, these participants showed bias toward LLM-specific vocabulary in their unassisted writing, suggesting that their thinking patterns had been shaped by the language model’s linguistic preferences. They also demonstrated significantly impaired ability to quote from essays they had written just minutes earlier, indicating problems with memory encoding and retention.

In contrast, the Brain-to-LLM participants—those unassisted writers who used ChatGPT for the first time—showed a very different pattern. Rather than cognitive decline, they demonstrated “higher memory recall” and “re-engagement of widespread occipito-parietal and prefrontal nodes.” These participants seemed able to use AI as a genuine tool while maintaining their cognitive agency and neural engagement.

The critical thinking prerequisites

This brings us to the crucial argument articulated by researchers like Rockmore. In a recent lecture at the Santa Fe Institute, he posits that the very process of learning to write is, in fact, learning how to think critically. The MIT study provides neurological proof for this concept: the neural connectivity patterns seen in traditional writing are the signature of cognitive skills being built and strengthened.

The study’s findings suggest that students need robust foundational thinking skills before they can productively engage with AI tools. The Brain-to-LLM participants succeeded precisely because they brought strong cognitive capabilities to their interaction with ChatGPT. They could evaluate its suggestions, maintain their own voice, and use the tool strategically rather than becoming dependent on it.

The MIT study also measured psychological factors that illuminate the broader implications of AI use in education. Participants in the LLM group consistently reported low “perceived ownership” of their essays compared to those in the brain-only group. This reflected a genuine uncertainty about which ideas were their own and which came from the AI.

Perhaps most concerning, the LLM group participants showed severely impaired ability to quote from essays they had written. When asked to recall specific passages from their work, they performed significantly worse than participants who had written without AI assistance. This suggests that the cognitive offloading wasn’t just affecting the writing process—it was interfering with memory formation and retention.

Implications for educational practice

The MIT study’s findings have implications for how we integrate AI into educational settings and our professional lives. The evidence suggests that the sequence of skill development matters enormously. Students need to develop foundational writing and critical thinking abilities through traditional methods before they can productively engage with AI tools.

This doesn’t mean banning AI from classrooms entirely (which at this point seems impossible) but rather recognizing that premature integration can be counterproductive. The goal should be to cultivate students who can be thoughtful, critical users of AI rather than passive consumers of its output.

The study points toward several key principles:

• Foundation First: Students should master core writing and thinking skills before using AI assistance. The neural pathways built through traditional writing practice appear necessary for maintaining cognitive agency when working with AI.

• Cognitive Engagement: When AI tools are introduced, they should be designed and used in ways that maintain rather than reduce neural engagement. The goal is collaboration, not cognitive offloading.

• Ownership and Memory: Educational approaches should emphasize students’ ability to maintain ownership of their ideas and remember their own work—capabilities that appear threatened by excessive AI dependence.

It is worth noting that the MIT study examined 54 participants aged 18-39 (average age 22.9) recruited from Boston-area universities—all adults who had already developed foundational writing and critical thinking skills through years of traditional education. This demographic limitation raises critical questions about the implications for younger populations, particularly high school and middle school students who are still developing these cognitive foundations. The researchers reference prior studies that have found “younger users exhibiting higher dependence on AI tools and consequently lower cognitive performance scores,” suggesting the cognitive debt effects observed in college students could be significantly more severe in adolescents.

The authors acknowledge that AI tools “offer unprecedented opportunities for enhancing learning,” yet caution that careful consideration is needed. The study suggests that because participants habituated to unaided writing (the Brain-to-LLM group) were able to use AI effectively while maintaining neural engagement, younger students who lack these foundations would be even more vulnerable to the cognitive offloading that leads to reduced brain connectivity. However, further research will need to establish the veracity of this inference.

The stakes of getting this wrong

The MIT study provides empirical evidence for concerns that have been largely theoretical. The “cognitive debt” measured in participants’ brains represents a real cost of AI dependence—one that may compound over time as students become increasingly reliant on external cognitive support.

As Rockmore notes in his New Yorker piece, we risk creating “a generation educated with AI shortcuts” that “may lack the foundational skills necessary for independent thought, creative problem-solving, and intellectual engagement.” The MIT study suggests this isn’t hyperbole; it can be measured neurologically.

The calculator analogy has become dangerous not because it’s entirely wrong, but because it fundamentally misrepresents the cognitive stakes involved. Calculators support mathematical thinking without replacing it. LLMs, as the brain imaging reveals, can replace the very cognitive processes that education is designed to develop.

The path forward requires recognizing that AI integration is not simply a matter of adopting new tools—it’s a fundamental shift in how we think about learning, cognition and intellectual development. We must prioritize the development of foundational cognitive skills that enable students to maintain their neural engagement and cognitive agency when working with AI.

The future of human thinking may well depend on getting this sequence right: first we learn to think, then we learn to think with machines. The MIT study provides the neurological roadmap for making this distinction—and the compelling evidence for why it matters.

---