Proofs of concept answer one question: can this work? Production systems have to answer harder ones: can people depend on it, can we operate it, and does it make the workflow better?
Evaluation was never defined
Teams tune prompts against a handful of examples and call the output good. Production needs a representative evaluation set, meaningful failure categories and thresholds tied to business consequence.
The workflow was designed around the model
If the product assumes the AI is always right, every model error becomes a user problem. Better systems decide which outputs can be automated, which need confirmation, and where deterministic rules should constrain behaviour.
Observability stops at infrastructure
CPU and latency dashboards are not enough. Teams need visibility into model quality, retrieval quality, tool failures, token usage, human corrections and shifts in input distribution.
No one owns the AI as a product surface
Models change, data changes, users change. A production AI feature needs explicit product and technical ownership, with authority to adjust UX, evaluation and model strategy together.
A production AI system is a socio-technical workflow with a model inside it, not a model with a user interface attached.