The demo-to-production gap is an architecture problem
The research consensus is brutal and consistent: RAND, BCG, and S&P Global all put the share of enterprise AI initiatives that fail to deliver production value at 80% or worse. What that number does not say is that the models failed. In almost every documented post-mortem, the model did roughly what the demo promised. What failed was everything the demo never had to survive.
The demo is designed to skip the hard parts
A demo runs on curated data, for a friendly audience, inside a workflow that exists only in the slide deck. Production means the messy source systems, the permission model, the audit requirement, the user who didn't ask for this tool, and the Tuesday afternoon when the upstream schema changes. BCG's 10-20-70 finding names the ratio: about 10% of the effort is algorithms, 20% is data and technology, and 70% is people and process integration. The demo demonstrates the 10% and defers the 90%.
That's why the failure patterns are so recognizable. Pilot paralysis: a portfolio of POCs, none with a route to production ownership. Model fetishism: upgrading the model when the blocker is integration architecture. The evaluation-deployment gap: offline benchmarks that say ship while no one has defined what "working" means against live traffic. Each is an organizational symptom of the same root cause — nobody architected the system the model was supposed to live inside.
What "production-grade" actually means
When I assess a stalled AI initiative, the questions that predict the outcome are architectural, and none of them mention model quality:
- Integration: does the system touch the real workflow, with real credentials, under the real permission model — or does a human copy-paste across the seam?
- Data: is there an owned, contracted path from source systems to the model's context — or a one-off extract that was fresh the week of the demo?
- Failure design: when the model is wrong — and it will be — is that detected, bounded, and recoverable, or does it flow silently into a decision?
- Ownership: is there end-to-end ownership from data to outcome, or does the initiative dissolve at the first org-chart boundary it crosses?
Workflow redesign belongs before model selection, not after. If the workflow can't absorb a probabilistic component — with review points, fallbacks, and a definition of acceptable error — no amount of model improvement will save it. IBM's Watson for Oncology remains the canonical case: world-class model marketing, unowned integration into clinical practice, quiet retreat.
The optimistic reading: because the gap is architectural, it's closable with ordinary engineering discipline — contracts on data, explicit permission boundaries, evaluation against live definitions of success, and one team owning the seam. The organizations crossing the gap aren't the ones with the best models. They're the ones that stopped treating the model as the product.