This post is for CTOs and operations leaders whose AI pilot impressed the steering committee and then stalled. The demo worked. Nobody can say when it will run unattended on real work.
The pilot proves the model can do the task. Production asks something harder: can the whole system do it thousands of times, on messy input, with nobody watching, for less than it saves? We practice forward deployed engineering (FDE), which means senior engineers embedded in your team writing code inside your environment, and this gap is where we spend most of our time.
Why do enterprise AI pilots fail after a successful demo?
A demo tests the model. Production tests everything around it. That surrounding scaffold (input handling, retries, validation, limits, monitoring) is usually unbuilt, because a pilot gets judged on whether the answer looks right and nobody asks what happens when something goes wrong.
The published numbers point the same way, with caveats. The MIT NANDA study from July 2025, reported by Fortune, found that about 95% of enterprise gen-AI pilots in the study showed no measurable P&L impact. The sample was about 150 interviews, 350 employees and 300 public deployments, and critics call the methodology opaque, so read it as a rough signal. The same study reported that bought tools or partnerships succeeded about 67% of the time, and internal builds about a third as often.
Gartner looks ahead. Its June 2025 press release predicted that over 40% of agentic AI projects will be canceled by the end of 2027, citing cost, unclear value and weak risk controls. It also described "agent washing": only about 130 of the thousands of vendors claiming agentic features had real ones.
McKinsey's State of AI 2025 shows the funnel. 62% of respondents were at least experimenting with agents, 23% were scaling agents in one or two functions, and 39% reported any EBIT impact from AI. That's a steep drop between trying agents and seeing a return.
What changes between the boardroom demo and a Monday morning in production?
Nearly every assumption a demo relies on disappears at once. Inputs are corrupted, events arrive together, limits get hit and nobody is standing next to the screen. The table lists each condition and the first failure it usually causes.
| Condition | In the boardroom demo | On a Monday morning in production | What usually breaks |
|---|---|---|---|
| Input data | Clean, hand-picked samples | Corrupted scans, odd encodings, missing fields, 14-page attachments | Extraction returns plausible but wrong values |
| Users and events | One user, one request at a time | Concurrent webhooks and bulk uploads that back up the queue | Race conditions and duplicate records |
| Retries | Never triggered | Network drops and timeouts trigger automatic retries | The same action executes twice |
| Rate limits | Not tested, volume is tiny | Provider and downstream API limits are hit by 9am | Failed calls and stalled queues, sometimes with half-finished updates |
| Supervision | A human nearby who fixes things silently | Nobody watching, often outside working hours | Errors persist until a customer complains |
| Spend | A few dollars of model calls | Retry loops and long inputs multiply token use | Unplanned invoice, no kill switch |
| Definition of done | The answer looks right | The right record exists once, in the right system, with a trace | No way to prove what happened |
A June 2026 VentureBeat survey of 157 respondents found that about half had shipped an agent that passed evals and then failed a customer. That's a small sample, but it matches what we see: evals on clean cases don't cover this table.
What are the four sources of production failure?
Four sources account for most of the post-pilot incidents we're asked to fix: duplicate or concurrent side effects, unvalidated model output, uncapped spend and limits, and invisible failures. None is an AI research problem. All four are ordinary engineering that the pilot skipped.
1. Duplicate and concurrent side effects
Webhooks are delivered at least once, which means sometimes twice. Queues redeliver. Users double-click. Without an idempotency key, a retry creates a second invoice or a second order. Concurrent runs also race on shared counters and inventory. The fix is a claim-before-act pattern, shown in code in our reference guide to idempotency keys, atomic locks and audit ledgers.
2. Unvalidated model output
A model returns text that looks like data. If that text goes straight into a database or an ERP, one hallucinated product code or negative total becomes a real record. Outputs need a schema, business-rule checks and a defined path for failures, such as a human review queue. We cover the pattern in Anti-Vibe-Coding.
3. Uncapped spend and unmanaged limits
A retry loop, a long document or a runaway agent can burn through a token budget overnight. A demo has no spend cap because it never needed one. Production needs hard per-run and per-day limits, and backoff for rate-limited APIs. It also needs a circuit breaker that stops calling a dependency once it starts failing. We cover the cost side in our post on preventing an overnight LLM token leak.
4. Invisible failures
If nobody can see a failure, nobody fixes it. A pilot often has console logs and one developer who remembers how it works. Production needs structured logs per run, an alert path with a named owner, an audit trail, and online evaluation on real traffic. The LangChain State of Agent Engineering survey of 1,300+ respondents reported online-evaluation adoption of 37.3%, so most teams are still judging live behavior by hand.
For a real example of a workflow that runs unattended, see how Slender Morris moved purchase orders from PDF to ERP to freight booking with validation at each step.
What does a forward deployed engineer ask first?
An FDE starts by asking how the system fails. These seven questions expose the missing scaffold quickly, usually within the first day or two, and each one maps to a concrete fix. Our week-one playbook covers how the answers turn into a plan.
- What happens if the same event arrives twice, or two events arrive at the same moment?
- What is the worst thing this system can write to a production record, and what stops it?
- What is the largest bill a single bad run or a bad night could produce?
- When the model or a downstream API is down or slow, what does the user see?
- If a customer disputes an outcome next month, can you reconstruct exactly what the system did and why?
- Who is paged when it fails at 3am, and how would they know?
- Which real inputs have you never tested, such as the ugliest scan or the longest document?
What is a 10-minute production-readiness checklist?
Give a technical lead ten minutes with the pilot's repository and this checklist. Each unticked box is a likely Monday-morning incident. Five or more unticked boxes usually means the pilot needs hardening before it carries real volume.
- Every side-effecting action carries an idempotency key, and a duplicate returns the stored result.
- Model output passes a schema and business-rule validation before any write.
- Failed validation routes to a queue a human actually reviews.
- Per-run and per-day spend caps exist and the code enforces them. A dashboard someone has to remember to check doesn't count.
- Rate-limited calls use backoff, and repeated failures trip a circuit breaker.
- Secrets live in a secret manager. None are in the repository or the frontend.
- Each run has a trace ID and structured logs you can search.
- An append-only audit record shows who or what did what, and when.
- A test set built from real, ugly inputs runs before every release.
- A named owner receives alerts, and someone has tried the rollback or kill switch at least once.
If you want help closing the gaps, our prototype-to-production hardening work covers exactly this list, and the full six-week approach is in the hardening playbook. If one of your pilots has stalled and you'd like a second pair of eyes, get in touch and we'll tell you honestly whether it needs hardening or a rethink.
