The notebook works. The demo is flawless. Then you ship it and everything falls apart.
There's a recurring pattern in computer vision: models that perform brilliantly in controlled environments completely fail when deployed to production. This isn't a new problem, but it's getting more attention as companies realize that the gap between "works in the lab" and "works at scale" is wider than anyone wants to admit.
The Demos Are Lying to You
The demo problem is simple. You optimize for the happy path. Clean data, perfect lighting, cooperative subjects. Then real users show up with motion blur, bad angles, edge cases you never considered, and suddenly your 95% accuracy drops to 60%.
One developer put it plainly: "CV work looks amazing in demos but falls apart when deployed." The responses revealed the pattern: scaling issues, latency problems, UX trade-offs nobody thought about, and edge cases that only appear when thousands of people use your system.
This isn't just about model accuracy. It's about the entire system around the model.
What Actually Breaks
Data drift is inevitable. Your training data doesn't match production data. Ever. The question is how fast it diverges and whether you're catching it. Most teams aren't. They train on carefully curated datasets and deploy to messy reality.
Latency kills products. A model that takes 2 seconds to respond feels broken to users, even if it's technically accurate. Speed isn't a nice-to-have. It's a product requirement that often means choosing a worse model that responds faster.
Edge cases are infinite. You can't anticipate them all. One team mentioned that even barcode readers—the most commoditized computer vision application—require careful implementation. If barcode scanning is still hard, what does that tell you about more complex vision tasks?
The Production Mindset vs. The Research Mindset
There's a fundamental disconnect in how people approach computer vision:
Researchers optimize for accuracy. Papers, benchmarks, SOTA results. Can you push the numbers higher?
Engineers optimize for reliability. What happens when the model fails? How do you monitor drift? Can you explain why it made this decision?
These aren't the same problem. And most CV projects are led by people with the research mindset trying to build production systems.
One comment captured this perfectly: "You're thinking like a data scientist, not a product developer. If your dataset is a bit overfit to your real-world usage, and is 'incorrect' in an abstract sense, but solves real world issues consistently for your users, is that really a problem?"
The answer is no. Production is about solving user problems, not achieving academic purity.
What Works: Feedback Loops
The teams that succeed treat deployment as the beginning, not the end. They build data collection into the product. They retrain constantly. They accept that the first version will be mediocre and plan for iteration.
The pattern that emerged from the discussion:
- Ship something passable (not perfect)
- Collect real-world data from users
- Label it (with AI assistance to get 90% there)
- Retrain on actual usage patterns
- Repeat
This isn't sexy. It's not a breakthrough algorithm. It's operational discipline. And it's what separates working products from abandoned research projects.
Classical CV vs. Deep Learning
An interesting tension emerged: some practitioners still advocate for classical computer vision techniques instead of throwing neural networks at everything.
The argument: deep learning is nondeterministic. You can't fully predict or explain its behavior. Classical CV methods are deterministic—you know exactly what they'll do in every case.
There's truth here. For constrained problems with clear requirements, classical methods can be more reliable. They're also easier to debug when something goes wrong.
But the counterargument is equally valid: deep learning handles variation better. Real-world data is messy. Classical approaches require perfect conditions. Neural networks tolerate noise.
The smart teams use both. Classical CV for preprocessing and sanity checks. Deep learning for the hard parts. Hybrid systems that play to each approach's strengths.
The Infrastructure Nobody Talks About
Production computer vision isn't just about models. It's about:
- Monitoring systems that detect when performance degrades
- Labeling pipelines that let you improve the model continuously
- Fallback logic for when the model fails
- User feedback mechanisms to catch edge cases
- Version control for models, not just code
One developer noted: "I would say 80% of the effort should go into production as a system and only 20% towards training the models."
That sounds extreme until you've actually shipped a CV product. Then it sounds about right.
Why This Matters Now
Computer vision AI is moving from research to infrastructure. Companies assume they can just integrate a model and it'll work. They're discovering that's not how this works.
The field is littered with failed deployments. Projects that looked promising in demos but couldn't handle production scale. Models that worked on benchmark datasets but failed on real users.
This isn't a technical problem. It's a product development problem. The teams that succeed understand that computer vision in production is fundamentally different from computer vision in notebooks.
The Uncomfortable Truth
Most computer vision projects fail not because the technology doesn't work, but because teams don't plan for production from day one. They optimize for demos and benchmarks, then act surprised when reality is messier.
The gap between research and production isn't closing. If anything, it's widening as models get more complex and deployment environments get more varied.
The solution isn't better algorithms. It's better engineering practices. Treat CV models as components in a larger system. Build for iteration, not perfection. Accept that your first deployment will expose problems you didn't know existed.
And maybe, just maybe, stop judging projects by how they perform in notebooks.