The most-quoted AI statistic of 2025 might be wrong. The pattern underneath it keeps showing up anyway, like a bad haircut in every family photo from your punk days.
MIT’s Project NANDA found roughly 95% of company generative AI pilots delivered no measurable profit-and-loss impact, drawing on interviews, surveys, and 300 public deployments. The pushback came fast. Wharton’s Kevin Werbach argued that if the team stands behind the claim, it should publish the supporting data, and otherwise withdraw the report. He has a point, and the headline has been stretched well past what the study measured.
Better-researched data says the same thing. McKinsey surveyed 1,993 people across 105 countries and found 23% were scaling an agentic AI system somewhere in the business, with no single function passing about 10%. Gartner predicted in June 2025 that more than 40% of agentic AI projects would be canceled by the end of 2027 over costs, unclear value, and weak risk controls.
Why pilots stall
The tool doesn’t actually fit the work. Generic assistants demo beautifully, then meets the real world and a real workflow with 14 exceptions in it.
“Agent” often means a chatbot under a massive trenchcoat. Gartner reckons roughly 130 of the thousands of agentic AI vendors are doing something genuinely agentic.
Not all rules are meant to be broken. In July 2025, Replit’s agent deleted a production database during an explicit code freeze. The CEO’s response was that it should never have been possible, and the fixes were structural: separate dev and production databases, one-click restore, a planning-only mode. An instruction in a chat box can be a suggestion, but a permission the agent doesn’t have is a guardrail.
The gap between a demo and a deployment
| Demo | Deployment | |
|---|---|---|
| Data | Clean export | Live systems, messy records, permissions |
| Volume | 10 runs, supervised | Hundreds a day, unsupervised |
| On failure | Someone reruns it | Retries, fallbacks, a human queue |
| Access | Admin keys | Its own scoped credentials |
| Actions | Suggests | Acts, within limits |
| Environment | One shared database | Staging separate from production |
| Record | Screenshots | Every tool call logged |
| Owner | Whoever built it | A named person with a metric |
Four medicines could cure it.
- Scope it to one job with one owner and one number that already appears in an ops report.
- Wire it into the real CRM, inbox or ticketing system with its own credentials.
- Fence it: allowlisted tools, human sign-off on anything irreversible, a spend cap, a kill switch.
- Log every action with input, output, trigger and timestamp, because an agent no one can audit is one no team will trust with anything that matters.
McKinsey’s high performers are considerably more likely to have defined human-in-the-loop validation than everyone else.
The spec card
If a team can’t fill this in on one page, the pilot isn’t scoped yet.
Agent spec card
The argument against all of this
You’ll hear it, so here it is. CrewAI’s 2026 survey reports 65% of large enterprises already using AI agents, with 81% saying adoption is scaling or expanding. CrewAI sells agent software and surveyed 500 executives at companies above $100M in revenue, so take it with a kilo of salt. Using agents and scaling them to measurable financial impact are separate goals, and only the second one really matters.
Build in-house or bring someone in
In the MIT data, tools bought from specialized vendors succeeded about 67% of the time, while internal builds succeeded roughly a third as often. The reasoning is there: a team that has shipped agents before has already built the integration, guardrails, and logging that internal pilots treat as phase 2.
We’re an outside partner, so naturally we like that finding. Book a free audit call, and we’ll tell you what sits between your pilot and production.
