Andrej Karpathy’s Software 3.0: Software Eating Software Eating Software

Model mashup: The new software stack and the inexorable march towards abstraction.

Post methodology: Gemini 2.5 Pro, GPT-4, Claude 4 Sonnet: write an essay based on andrej karpathy's talk [YouTube link] on the subject of software eating software eating software using his concepts of software 1.0, 2.0 and 3.0; add a section about how contemporary ai products use a combination of 1.0, 2.0 and 3.0 approaches. Output from all three models interleaved and attributed. Light editing and formatting for the Substack editor.

[Gemini 2.5 Pro]

Software is no longer just eating the world, increasingly it’s eating itself. The 21st century has seen the relentless process of industries being digitized and disrupted by software-driven solutions. In a recent talk, Andrej Karpathy offers a compelling framework for how this evolution can be conceptualized in three distinct stages: Software 1.0, Software 2.0, and the nascent Software 3.0. This progression marks a fundamental shift from programming with explicit instructions to programming by specifying desired outcomes, a trajectory that is redefining what it means to create and what it means to be a developer.

[GPT-4]

Software 1.0: Handcrafted Logic

Software 1.0 is the traditional world of programming. Human engineers write explicit instructions in code—algorithms, data structures, and logic painstakingly crafted to meet requirements. Every behavior is designed, debugged, and maintained by people. This approach, while the powerhouse of the modern world, is limited by the complexity that humans can manage and the speed at which they can operate.

• Characteristics:

  • Explicit, human-written code

  • Predictable, interpretable logic

  • Scaling limited by human effort

Despite its limitations, Software 1.0 underpins the vast majority of today’s digital infrastructure.

[Claude 4 Sonnet]

The Eating Begins

Even in Software 1.0, we see the first signs of software consuming itself:

• Compilers eating assembly code and producing machine code

• High-level languages eating the complexity of memory management

• Frameworks eating boilerplate code and common patterns

• APIs eating the need to understand underlying system complexity

[GPT-4]

Software 2.0: Learned Systems

Software 2.0 marks a paradigm shift. Instead of hand-writing rules, developers specify objectives and provide data. The rules—often in the form of neural network weights—are learned by the computer. This is the era of machine learning, where algorithms like deep learning find patterns and solutions inaccessible to human-crafted code.

• Characteristics:

  • Models trained on data, not written by hand

  • Emergent, often opaque logic

  • Scaling with data and compute, not just humans

Software 2.0 enables breakthroughs in fields like vision, language, and robotics. But it also introduces meta-software: instead of writing the solution, engineers write the software that creates the solution. This is software eating software—the first layer of recursion.

[Claude 4 Sonnet]

Software 2.0 Eating Software 1.0

This transition represents the first major "eating" event:

What Gets Consumed:

• Feature Engineering → Automatic feature learning

• Rule-Based Systems → Pattern recognition from data

• Expert Systems → End-to-end learning

• Traditional Computer Vision → Convolutional neural networks

• Statistical NLP → Transformer architectures

The Transformation Process:

1. Identification: Find domains where Software 1.0 struggles with complexity

2. Data Collection: Gather massive datasets representing desired behavior

3. Architecture Design: Create neural network structures

4. Training: Let optimization find the solution

5. Deployment: Replace traditional code with neural networks

[GPT-4]

Software 3.0: Generative and Self-Improving Software

Software 3.0 is the latest and perhaps most radical evolution. Here, large language models (LLMs) and generative AI can produce code, design new models, and even orchestrate entire systems with minimal human intervention. The boundary between code, data, and intention blurs. The software that writes software can now write itself or create new, unforeseen forms of software.

• Characteristics:

  • Generative models (e.g., GPT-4, Claude, Gemini) capable of code synthesis and system design

  • Self-improving loops: AI models optimizing, retraining, or evolving themselves

  • Human engineers become curators, supervisors, or collaborators rather than sole creators

With Software 3.0, we witness software eating the software that eats software. LLMs can generate training data, architect new learning algorithms, and recursively improve both themselves and the systems they build. The feedback loop accelerates, and the locus of creativity shifts from explicit design to collaborative emergence.

[Claude 4 Sonnet]

Software 3.0 Eating Software 2.0

The most recent "eating" event is already underway:

What's Being Consumed:

• Manual Model Architecture Design → Neural Architecture Search

• Hyperparameter Tuning → Automated optimization

• Feature Engineering → Foundation models with emergent capabilities

• Task-Specific Training → Few-shot and zero-shot learning

• Model Deployment → Automated MLOps pipelines

🏗️ The New AI Stack: A Concrete Example

To illustrate how Software 1.0, 2.0, and 3.0 approaches work together in practice, let's examine a contemporary AI-powered customer service platform—a representative example of how modern AI products are architected.

📞 Case Study: An AI Customer Service Agent

Consider a sophisticated customer service system that can handle complex inquiries across multiple channels. This system demonstrates the strategic deployment of all three software paradigms:

Software 1.0 Layer: The Reliable Foundation

🔧 Traditional Components:

• Database management (customer records, transaction history)

• Authentication and security systems

• API integrations with CRM, billing, and inventory systems

• Load balancing and monitoring infrastructure

• Compliance and audit logging

• Escalation workflows and routing logic

These components require deterministic, predictable behavior. When a customer's payment fails, the system must reliably update their account status—there's no room for probabilistic outcomes in financial transactions.

Software 2.0 Layer: The Intelligence Engine

🧠 Learned Components:

• Natural Language Understanding (intent classification, entity extraction)

• Sentiment analysis for conversation tone

• Document retrieval and semantic search

• Language translation for multilingual support

• Voice-to-text and text-to-voice conversion

• Fraud detection based on conversation patterns

These capabilities emerge from training on massive datasets of customer interactions, support tickets, and domain-specific knowledge. The system learns to recognize when someone is asking about a refund versus a technical issue without explicit programming for every possible variation.

Software 3.0 Layer: The Adaptive Interface

🚀 Generative Components:

• Dynamic response generation tailored to context and customer history

• Real-time code generation for custom integrations

• Automated creation of new training scenarios

• Self-improving conversation flows based on success metrics

• Personalized interaction strategies that evolve per customer

• Meta-learning Cross-customer patterns to improve overall performance

This layer can generate novel solutions for unprecedented situations, create personalized responses that have never been written before, and even improve its own training processes.

🔄 The Interaction Between Layers

The magic happens in how these layers interact:

1. Customer Contact: A user messages "I'm really frustrated! My order was supposed to arrive yesterday for my daughter's birthday party today, and it's still not here!"

2. Software 1.0 handles the infrastructure:

   • Routes the message through security validation

   • Looks up order status in the database

   • Checks shipping carrier APIs for tracking information

3. Software 2.0 provides intelligence:

   • Detects high emotional sentiment (frustration, urgency)

   • Identifies entities (order, delivery date, special occasion)

   • Classifies intent (delivery inquiry + complaint)

   • Retrieves relevant policy documents

4. Software 3.0 generates the response:

   • Creates a personalized, empathetic response acknowledging the birthday party context

   • Generates specific compensation options based on the situation

   • Adapts communication style to match the customer's emotional state

   • Learns from this interaction to improve future birthday-related delivery issues

🎯 The Strategic Architecture

This hybrid approach provides several advantages:

Reliability Where It Matters: Financial transactions and data integrity use Software 1.0's predictable logic.

Intelligence Where It Adds Value: Understanding customer intent and retrieving relevant information leverages Software 2.0's pattern recognition.

Creativity Where It Differentiates: Generating personalized, contextually appropriate responses uses Software 3.0's generative capabilities.

[GPT-4]

Implications: The Infinite Recursion of Software

This recursive “software eating software eating software” dynamic has profound implications:

• Acceleration: Each layer amplifies the speed and scope of innovation. What once took months or years to code can now be generated, tested, and deployed in hours.

• Abstraction: Human programmers move up the stack, focusing on higher-order objectives, ethics, and supervision.

• Opacity and Risk: Each new layer introduces more abstraction and less interpretability. Understanding and controlling these systems becomes more challenging.

• Creativity and Collaboration: The role of the engineer shifts from direct author to orchestrator, collaborator, or even prompt engineer—guiding powerful systems toward desirable outcomes.

[Gemini 2.5 Pro]

This progression from 1.0 to 3.0 represents a relentless climb up the ladder of abstraction. We moved from telling the machine how to do something (Software 1.0), to showing it what to do through examples (Software 2.0), to now simply telling it what we want (Software 3.0). Each stage subsumes the previous one. A Software 3.0 system (an LLM) can generate Software 1.0 code or even the configuration for a Software 2.0 model. This is software eating software, a recursive loop where the tools of creation are themselves being created by the tools. The ultimate challenge will be to manage, verify, and steer these increasingly autonomous systems, ensuring that the software they write aligns with human intent and values, even as the process of its creation becomes ever more abstract.

[Claude 4 Sonnet]

🎯 Conclusion: Embracing the Recursive Future

Karpathy's framework reveals that we're not just witnessing technological progress—we're observing a fundamental phase transition in the nature of computation itself. Software isn't just eating the world; it's recursively consuming and transforming itself in an accelerating spiral of abstraction and capability.

The pattern is clear: each generation of software makes the previous generation's complexity invisible while enabling entirely new classes of problems to be solved. We're living through the transition from Software 2.0 to 3.0, and the implications extend far beyond technology into the very fabric of how we work, create, and understand our world.

As we navigate this transition, the key insight from Karpathy's framework is that fighting these changes is futile—the eating will continue. Instead, we must learn to ride the wave, understanding each paradigm's strengths and limitations while preparing for the next inevitable transformation.

The future belongs to those who can dance between paradigms, leveraging the power of each generation while remaining adaptable enough to evolve with the next wave of software eating software eating software.

---

Editor’s note: This article represents a curated conversation between models responding to identical prompts that attempts to preserve the strengths of each while still reading coherently. Software 3.0 is eating the layers below it, but as editors, creators and developers we are eating it as well.