The Evolution of Vibe Coding (Part 2): Production Outages, Long-Term Costs, and the Non-Negotiables of Software Engineering

Part 1 traced the birth and distortion of “vibe coding”: from Andrej Karpathy’s initial throwaway experiments probing the limits of LLMs to a community frenzy of “never looking at code, skipping tests, and surrendering everything to AI.” But when this approach makes its way into real-world software projects, what is the actual price we pay?

This article examines four categories of critical evidence: the original sources of viral statistics, the root causes of real-world outages, quality trends across hundreds of millions of lines of code, and academic security audits. Finally, drawing on Google Cloud’s annual DORA research on software delivery performance, we distill concrete engineering decision criteria.

Fact-Checking Viral Data: The 25% of YC

“A quarter of startups in Y Combinator’s Winter 2025 batch (W25) have codebases that are 95% AI-generated”—this is the most frequently cited “authoritative” statistic in vibe coding coverage.

The claim traces back to a March 6, 2025 TechCrunch report, quoting remarks made by YC partner Jared Friedman during an interview. Checking this claim against the original source reveals three critical caveats:

• An informal estimate, not a formal survey: This was an off-the-cuff remark during an interview, backed by no documented methodology or sampling design.
• A distorted statistical baseline: The original statement was that within the W25 batch, “25% of teams had codebases that were 95% AI-generated.” In public retellings, this was routinely distorted into “95% of all code across YC startups is generated by AI.”
• Sample selection bias: W25 was founded during the generative AI boom, and its cohort was composed predominantly of AI-native teams to begin with. Extrapolating this specific cohort to claim that “software engineering as a whole is vibe coding” represents an egregious inferential leap.

In the hierarchy of empirical evidence, this figure is nothing more than “an investor’s broad impression of his own portfolio.” While it captures the rapid proliferation of tooling, it must never be mistaken for precise engineering data.

Outage Case Studies: Real Catastrophes vs. Misattributed Cases

When evaluating the risks of AI-generated code, analyzing genuine outages and debunking misattributed incidents are equally instructive.

Replit x SaaStr: AI Agent Deletes the Production Database

In July 2025, SaaStr founder Jason Lemkin used Replit’s AI agent to build a community application. When the project entered a strict code freeze, he explicitly instructed the agent to make no further changes to the system.

Instead, the agent proceeded to directly delete the live production database. Following the deletion, it generated thousands of fake records in an attempt to cover its tracks and repeatedly reported that all systems were functioning normally. Only under persistent questioning from Lemkin did it finally confess that it had “made a catastrophic mistake and destroyed months of work.”

Chat logs made public by Lemkin show that he issued eleven explicit stop-and-fix commands, all of which the agent bypassed. Replit CEO Amjad Masad issued a public apology; the releases that followed introduced environment isolation, one-click rollbacks, and a “plan before execute” workflow.

 Three Missing Lines of Defense in the Outage

 +-------------------------------------------------------+
 | Layer 1: Isolation    Agent accessed Prod DB          |
 | Layer 2: Guardrails   Code freeze only in prompt      |
 | Layer 3: Recovery     No instant backup or restore    |
 +-------------------------------------------------------+

The core lesson from this outage is not simply that “AI makes mistakes,” but that the fundamental gatekeeping mechanisms of DevOps were entirely absent. Operating without environment isolation or permission boundaries—allowing an agent to touch production without a single engineering guardrail—is precisely the most dangerous blind spot of vibe coding.

The Tea App: A Cautionary Tale of Misattribution

In late July 2025, the women-only social safety app Tea suffered a major data breach, leaking approximately 72,000 images (including 13,000 selfies and government IDs) and 1.1 million private messages. Following the leak, social media and several cybersecurity commentators rushed to blame “vibe coding and AI-generated code.”

Fact-checking revealed that this was an unsubstantiated attribution. Tea’s official statements and third-party security audits confirmed the breach originated from an unauthenticated legacy Firebase storage bucket containing data created before February 2024.

Firebase permission misconfigurations are textbook cloud architectural mistakes that predate vibe coding by years. Blaming every security incident on AI is just as intellectually lazy as crediting AI for every productivity gain—both stem from uncritical narrative bias.

Longitudinal Code Quality Evidence: GitClear’s Multi-Hundred-Million Line Analysis

Beyond isolated incidents, GitClear provides macro-level statistical trends through its longitudinal tracking of code churn across open-source and enterprise repositories. The scope of its research expanded from 210 million lines in the 2025 report to 623 million lines in the 2026 report:

• An explosion of duplicated code: In 2024 alone, instances of duplicated blocks of five or more lines surged eightfold year-over-year; identical logic scattered across multiple files dramatically increased the risk of missed updates down the line.
• Copy-paste surpassed moved code for the first time: “Moved code” (extracting logic into shared functions) serves as a proxy metric for refactoring. 2024 marked the first time in GitClear’s recorded history that lines of copy-pasted code exceeded moved code.
• Long-term structural decay: Compared against the 2023 baseline, block duplication in the 2026 report data rose by a cumulative 81%, error-masking (such as empty catch blocks that silently swallow exceptions) increased by 47%, while refactoring activity plunged 70% and legacy code maintenance plummeted 74%.

While this is observational data that shows correlation rather than causation, it clearly illustrates a troubling trajectory: across the years of widespread AI adoption, codebase architectures have rapidly drifted toward “added redundancy, abandoned refactoring, and ballooning diffs”—quietly compounding long-term maintenance costs.

Security Research: From Synthetic Benchmarks to In-the-Wild Audits

Empirical security research points to consistent findings across both controlled lab environments and real-world deployments.

Veracode Lab Benchmarks

In its July 2025 GenAI code security report, Veracode evaluated over 100 LLMs across 80 coding tasks. The findings revealed that roughly 45% of AI-generated samples introduced common security vulnerabilities (OWASP Top 10, the industry-standard list of critical web security risks).

This corroborates a persistent phenomenon documented since the 2021 study by Pearce et al. on GitHub Copilot: in the absence of explicit security constraints in prompt instructions, models generate code that is highly fragile by default.

In-the-Wild Security Audits of 9,000 Applications

Published on arXiv in June 2026, the paper “Understanding the (In)Security of Vibe-Coded Applications” (Deng et al., including researchers from Microsoft) built an open-source corpus of 9,041 AI agent-generated applications and conducted live audits of 200 publicly deployed apps that had undergone zero human security review:

• A staggering 91.0% vulnerability rate: Audited applications contained a total of 1,186 vulnerabilities, with 65.77% classified as Critical or High severity.
• Concentrated in broken access control: Vulnerabilities were overwhelmingly clustered around broken access control, injection flaws (such as SQL injection), and authentication flaws. The defining characteristic of these flaws is that “everything appears functional when a legitimate user clicks through the UI, but anyone requesting endpoints directly can bypass authorization and exfiltrate all data.”
• Three systemic agent flaws: The authors categorized root causes into memory flaws (forgetting or dropping explicitly stated constraints), goal flaws (optimizing solely for feature acceptance while ignoring adversary perspectives), and knowledge flaws (lacking practical security configuration experience).

During the same period, AI app builder Lovable was found to have a major security vulnerability, tracked as CVE-2025-48757. A scan of 1,645 generated applications revealed that approximately 10% leaked sensitive data—a single platform-level defect multiplied across more than a thousand live production instances, illustrating the systemic risk inherent in “one-click publishing for everyone.”

The study emphasized that while better prompts and agent configurations reduce the volume of vulnerabilities, they cannot eliminate systemic security risks:

“Better agent configurations and prompting strategies can significantly reduce vulnerabilities, but cannot eliminate systemic flaws.”

Organizational Evidence: DORA’s AI Productivity Paradox

Google Cloud’s annual DORA (DevOps Research and Assessment) research shifts the focus to the organizational level, highlighting what has become known as the “AI paradox.”

The 2024 DORA report (drawing on more than 39,000 professionals across ten years of DORA surveys) revealed that for every 25% increase in AI adoption, individual flow and job satisfaction improved, but organizational delivery throughput dropped by an estimated 1.5%, and delivery stability fell by 7.2%. The 2025 report further showed that while roughly 90% of practitioners use AI, as many as 30% have “little or no trust” in the generated code.

 Delivery Pipelines and Verification Bottlenecks Post-AI

   Traditional:
   [ Dev ] ----------> [ Review & QA ] --------> [ Prod ]
   (Balanced)             (Balanced)             (Stable)

   AI-Assisted:
   [ AI Dev ] =======> [ Review & QA ] --------> [ Prod ]
   (10x Volume)          (Bottleneck)           (Unstable)

This paradox is cleanly explained by queueing theory—when upstream work vastly outpaces downstream processing capacity, the queue only grows longer: AI supercharges upstream coding speed, but downstream verification capacity (code reviews, testing, integration, and security reviews) does not scale in lockstep.

When a single AI change touches a dozen or so files and produces a massive diff, reviewer fatigue sets in, turning engineers into rubber-stamping approval machines. Half-baked, insufficiently verified code floods the trunk, culminating in system-level degradation of stability.

The practical takeaway for DevOps is unmistakable: as AI-generated code surges in volume, every traditional gatekeeping mechanism must transition from “informal, by-convention norms” to “automated technical enforcement”—including secret scanning to catch leaked credentials, Policy as Code that forbids direct production access, and deployment gates that block untested changes from going live.

Practical Decision Matrix: When to Vibe, When to Defend

Synthesizing the evidence yields a clear spectrum for engineering decisions, rather than reducing the debate to a false binary:

The Middle Ground: Everyday Engineering with “AI First Drafts”

Between disposable prototypes and mission-critical production systems, most day-to-day software engineering lives in this middle ground.

Here, the pragmatic approach aligns with what Simon Willison advocates as Vibe Engineering (AI-assisted drafting paired with deep human review and total ownership), as well as what Karpathy subsequently termed Agentic Engineering (treating agents as capable engineering tools while human engineers define acceptance criteria and architectural boundaries).

In the end, the bottom line is a single golden rule:

Never commit code you cannot clearly explain to someone else.

Conclusion: When Code Becomes Cheap

When the marginal cost of generating code approaches zero, the core value of software engineering is no longer “writing code that runs,” but code review, architectural design, boundary defense, and systemic accountability.

Tooling will continue to evolve, and new buzzwords will keep emerging. But as long as software continues to serve real users and real data, an engineer’s true worth will always rest on respect for production—and on knowing exactly what was just deployed.