Why the pilot worked and the rollout died
A pilot is a controlled question. Can the model do this task when we feed it clean inputs and watch it closely? The answer is almost always yes. Modern models are good. That yes is what gets the budget approved, and it is also what sets the trap.
The demo proves the wrong thing
A demo proves capability on a good day. A rollout has to survive bad days: the malformed input, the upstream system that times out, the edge case nobody documented, the user who pastes something you never anticipated. Capability was never the constraint. Reliability is.
The work that makes a system survive bad days is mostly invisible in a demo. It is retries and fallbacks, monitoring and alerts, the boring plumbing that decides whether the thing runs while everyone is asleep. None of it shows up in the slide that got the project funded.
Where rollouts actually die
- The handoff. The pilot ran on someone's laptop with hand-fed data, and nobody wired it into the real systems, so it never became part of the day.
- The long tail. The model is right ninety percent of the time, and the missing ten percent is exactly the part a person cannot ignore.
- The owner. After launch the team that built it moved on, and no one inside the business could read, run, or change it.
Build for the second proof
The first proof is can it work. The second proof is does it keep working when we stop watching. Scope the first build to reach the second proof quickly, on a narrow slice of real work, wired into real systems, with a person owning it. A small thing that runs beats a broad thing that demos.
If a system cannot survive the week you go on holiday, it was a demo, not a deployment.