Kaprin
All insightsFDE Field Notes

Why AI Pilots Stall and What an Embedded Engineer Does in Week One

By Kaprin Engineering 7 min read

This is for operations leaders, CTOs and CFOs who have a pilot that looked good in a demo and hasn't shipped, or who are about to start one. We'll cover what a senior embedded engineer, working inside your team through forward deployed engineering (FDE), does in the first five days to move a pilot toward production.

We keep the diagnosis short, since we've covered it at more length in post-pilot reality: why enterprise AI fails on Monday morning. This post is the playbook.

Why do AI pilots stall?

The hard parts of production are access to real systems, agreed success measures, edge-case handling and ownership. The model is rarely the bottleneck. Surveys disagree on the exact rates, but they agree that many pilots never reach measurable value.

Three data points give context, with caveats:

  • Fortune, reporting on the MIT NANDA study (July 2025): about 95% of enterprise gen-AI pilots in the study showed no measurable P&L impact. The sample was about 150 interviews, 350 employees and 300 public deployments, and critics call the methodology opaque. The same report found bought tools or partnerships succeeded about 67% of the time, internal builds about a third as often.
  • Gartner predicts over 40% of agentic AI projects will be canceled by the end of 2027, citing cost, unclear value or weak risk controls.
  • McKinsey's State of AI 2025 reports 62% of respondents at least experimenting with agents, 23% scaling agents in one or two functions, and 39% reporting any EBIT impact from AI.

Read these as rough direction. The common thread is that value shows up after the pilot, in the unglamorous work. Week one is where that work starts.

What does an embedded engineer do on day 1?

On day 1 the engineer gets access and a working environment. That means accounts in the relevant systems, a development or staging setup, a repository, and a way to run the code against realistic data. If access takes more than a day, that's already the first finding of the engagement.

  1. Join your chat, ticketing and calendar. Meet the person who runs the workflow today.
  2. Request accounts for the systems involved (for example ERP, CRM, payments, email, storage).
  3. Set up a repository and deployment pipeline inside your cloud account or private VPC. You own it from the first commit.
  4. Pull a small sample of real records and confirm they can be read safely.
  5. Write down every blocker with an owner and a date.

The blocker list matters more than it sounds, because most weeks that slip, slip here. To prepare, read what you owe the engineer in the first 90 days of an FDE engagement.

What happens on day 2: mapping the workflow and the baseline?

On day 2 the engineer maps the workflow from first input to final output as it runs today, and agrees one baseline metric with you in writing. The map shows every system, person and handoff. The baseline gives every later demo a single number to move.

The engineer maps it by watching the work. They sit with the person who processes orders, builds the report or handles the exception, and note what that person actually does.

Then comes the baseline. Pick one metric that a finance person would accept:

  • Minutes from order received to order entered.
  • Hours spent on daily reporting.
  • Number of failed or duplicated transactions per week.
  • Cost per ticket handled.

Measure it roughly if you have to, but write it down and get your decision-maker to agree to it. We go deeper on why a single number matters in our post on starting from one number.

What goes into the day 3 failure inventory?

The failure inventory is a written list of how the workflow can go wrong, drawn from real data and from the people who do the work. It records inputs, volumes, edge cases and who approves each exception. It becomes the checklist for hardening in weeks 2-6.

CategoryQuestions the engineer asksExample finding
Inputs What formats arrive? What is malformed, missing, duplicated or in the wrong language? A supplier sends documents rotated or scanned at a skew.
Volumes What is the normal rate, the peak, and the worst day of the year? A month-end spike far above the daily average.
Edge cases What happens on retries, partial failures, timeouts, or when two events arrive together? A network drop that causes the same action to be submitted twice.
Approvals Who must say yes to an exception, how fast, and what happens if nobody does? Refunds above a threshold need a manager, who is often away.
Blast radius What is the worst thing a wrong output could do? Can it be undone? An incorrect freight booking that cannot be canceled once dispatched.

The examples in the table illustrate the kind of finding. They aren't results from a specific client. The inventory turns vague worry into a short list of things to design for. Many items won't need a model at all, because a deterministic rule or a lock will do. Our approach to those is in production hardening.

What is a thin slice, and what happens on day 4?

A thin slice is the smallest version of the workflow that runs from input to output on real data. On day 4 the engineer builds it: one input type, one happy path and the first one or two failure cases from the inventory. It's narrow on purpose, and it's real.

The slice reads actual records from the real source, applies the real logic and writes a result to a real or staging destination. It may handle only one customer, one document type or one product line, which is enough to learn how the system behaves.

The engineer also checks something pilots often skip, which is whether the job needs a model at all. If a plain script does it, the script is the answer. If a model is needed, it goes inside deterministic scaffolding: validated outputs, retries that can't double-process, and a cap on spend.

The Slender Morris case study is an example of the kind of workflow this suits: purchase orders arriving as PDFs, flowing into an ERP and onward to freight booking.

What do you get on day 5?

On day 5 you get a live demo of the thin slice on real data, the baseline metric stated again, the failure inventory, and a written plan for week two. The plan names what will be built, what is deferred, and what the engineer needs from you.

A good week-two plan fits on one page:

Week 2 plan
Goal:        Move baseline metric from X to Y on the pilot slice
Build:       1) Handle failure cases A, B, C  2) Add idempotency to step 4
             3) Add monitoring for queue depth and error rate
Defer:       Edge cases D and E (rare; manual fallback agreed)
Need from you: Access to staging payments account by Tuesday;
               decision on refund threshold by Wednesday
Demo:        Friday, same time
Risks:       Supplier format change; month-end volume

The last line of that plan is the most useful one. Naming risks in writing makes them easier to discuss and harder to ignore.

What does a good week one look like?

A good week one ends with working access, a written baseline, a failure inventory and a demo on real data. A weak one ends with a slide deck, a long list of questions and no code running against your systems. You'll know which one you have by Friday afternoon.

This is the first part of the Kaprin rhythm: Scope within 24 hours, Embed on day 1, Harden in weeks 2-6, Handoff by day 45. The full process is on our how we deploy page.

Next step

If you have a stalled pilot and want a second opinion on what week one would look like, send us a short description of the workflow. We'll tell you what we'd do first, even if the answer is that a simpler fix exists.

Questions readers ask

Why do so many AI pilots fail to reach production?

Studies point to unclear value, cost and weak risk controls. In practice, pilots stall because nobody has access to the real systems, nobody agreed what success looks like, and edge cases were never listed. A week-one process addresses all three directly.

What should an embedded engineer deliver in the first week?

A thin slice of the workflow running on real data, one agreed baseline metric, a written failure inventory and a plan for week two. You should also end up with working access and a development environment, so the second week doesn't start with waiting.

Why agree a baseline metric on day 2?

Without a baseline you can't tell whether the system helped. Writing down the current number, such as minutes per order or hours per report, heads off disagreement later and keeps every demo tied to one measurable result.

What is a failure inventory?

It's a written list of what can go wrong in the workflow: odd inputs, peak volumes, edge cases, and who must approve each exception. It is built from real data and from the people who do the work, and it drives the hardening plan for the following weeks.

Do I need clean data before an engineer starts?

Real data is what you need, messy cases included. Clean samples hide the problems that cause pilots to stall. Anonymize where policy requires it, and leave the examples untidied.

Bring us the broken prototype, the SaaS bill, or the manual process

Tell us what is stuck. We reply within 24 hours with a scoped plan, a fixed timeline and the one number we will move.

More insights

FDE = forward deployed engineering. Strict NDA protection. Senior architect review.