What “in production” actually means
“In production” is one of the most overloaded phrases in the field. For some people it means the code is deployed. For us it means something stricter, and the difference is the whole job.
A working definition
A system is in production when it runs without you watching, fails in ways you have already planned for, and keeps earning its place after the launch energy is gone. Each clause is doing work.
- Runs without you watching. It executes on a schedule or on events, not when an engineer pokes it.
- Fails in ways you have planned for. Failure is expected and handled, not a surprise that pages someone at 2am.
- Keeps earning its place. The value compounds, or it gets retired. Systems that merely exist are a cost.
Two systems, same demo
Picture an invoice-matching system, the kind that reads supplier invoices and matches them to purchase orders. Deployed two ways, from the same model, with the same demo.
Version one is an endpoint. It works when someone in finance remembers to feed it. In month two a large supplier changes its invoice layout, the extraction quietly starts reading the net amount where the gross used to be, and nobody notices for three weeks because nothing crashed. The errors surface at month-end reconciliation, the team stops trusting the tool, and the tool dies of neglect while technically still deployed.
Version two is the same model with the unglamorous parts built around it. Every extraction is validated against the purchase order before it posts, so a mismatch becomes an exception in a queue, not a wrong number in the ledger. A weekly report shows the match rate per supplier, so the layout change appears as a visible dip on day one. There is a named owner in finance who reads that report, and a one-page runbook that says what to do when a supplier changes format. Same model. Only one of these is in production.
Where the effort actually goes
The gap between the two versions is not model quality. It is everything wrapped around the model, and the split below is roughly how the remaining effort divides once the demo already works. Illustrative, but the ordering is the point: the model is the smallest slice.
This is why a pilot that proves the model can do the task has covered perhaps a tenth of the distance. The rest is the plumbing that decides whether the thing survives a supplier changing a template, an upstream system timing out overnight, or the one person who understood it going on leave.
Four questions that test the bar
- Who gets alerted when it fails at 2am, and do they know what to do? If the answer is nobody, it is not in production. It is unattended.
- What happens when an upstream format changes? Silent wrong answers are worse than loud failures. The system should notice its own confusion.
- Who inside the business can turn it off, and what happens to the work if they do? A safe off switch is part of the design, not an admission of doubt.
- When did anyone last check its output against reality? A system nobody measures is a system nobody can trust, whatever it did last quarter.
These questions travel well beyond invoices. Ask them of the nightly stock reconciliation, the maintenance report drafter, the dispatch board. Any system that touches operations will be tested by a bad night eventually. The only choice is whether you designed for that night or get introduced to it.
Why the bar matters
The gap between deployed and in production is where most of the value lives and most of the risk hides. A deployed endpoint with no monitoring is a liability you have not noticed yet. A system that meets the stricter bar is an asset that quietly does its job, and it is the only kind that still earns its keep a year later.
We hold the stricter line because it is the only one a business can actually rely on. Everything we build is measured against the week you are not there to save it.