Run inference on-prem at the edge by default and send only the exceptions to the cloud — because a production line needs a pass/fail verdict in well under a second, and 0.3 to 1.0 seconds per unit is only achievable when the decision never leaves the floor. The rule flips when your exception rate times the payload per exception exceeds what your network can carry inside the line's cycle-time budget, or when retraining and aggregation genuinely need centralised data — at which point a hybrid split, not a wholesale move to cloud, is the answer.
The mistake teams make is picking the GPU before modelling the tail latency. They size compute for the average inference and discover too late that the number that actually governs the architecture is what happens to the slow cases — the uncertain parts that trigger a heavier model, a second capture, or a round trip. Edge-versus-cloud is not a hardware preference. It is a decision about where your exception rate becomes a network call.
The three architectures, matched to what governs them
| On-prem edge | Cloud | Hybrid (edge + cloud) | |
|---|---|---|---|
| Per-unit latency | Lowest, deterministic | Network-bound, variable | Edge for verdict, cloud for the tail |
| Bandwidth need | Minimal (metadata out) | Every frame uploaded | Exceptions and samples only |
| Data governance | Stays on site | Leaves the site | Sensitive frames stay, aggregates leave |
| Retraining | Local or manual sync | Centralised, easy | Centralised, edge-deployed |
| Fails when | Compute undersized for peak | Link or cloud hiccups stop the line | Split boundary set wrong |
| Best fit | Real-time pass/fail at line speed | Offline analytics, low rate | High rate + central learning |
Read the "fails when" row: on-prem fails if you under-provision for the peak, cloud fails the moment the link does — and a line that stops when the internet stops is not a line. Hybrid fails only if you draw the split in the wrong place, which is the one failure you control by design.
Modelling the tail, not the average
The decision rule is a short calculation, not a vendor preference. Take your exception rate — the fraction of parts the edge model cannot confidently resolve — multiply by the data you would ship per exception, and check it against the headroom in your cycle time and your link. If the exceptions fit inside the budget, keep the verdict on-prem and let the cloud handle the tail: deep review, logging, aggregation for retraining. If they do not, either lower the exception rate (better edge model, better optics) or shrink the payload (ship features, not full frames) before you consider moving the decision off the floor.
This is why the GPU-first instinct backfires. A faster edge accelerator lowers average inference time but does nothing for a 20 percent exception rate flooding a thin uplink. Model the tail first, then size the hardware the tail implies.
Integration is where the architecture becomes real
Whatever the split, the verdict has to reach the machine. OPC UA is the common path from a vision station into the line's control layer, and the architecture choice has to survive it: an on-prem verdict publishes to the PLC in deterministic time, while a cloud round-trip introduces jitter the controller was not designed to tolerate. If your reject actuator fires on a fixed cycle, the inference has to resolve inside that cycle — which, for most real lines, settles the question in favour of edge for the verdict and cloud for everything that is not time-critical.
The honest boundary: the latency and per-unit figures here (0.3 to 1.0 seconds per unit) are platform specs on suited tasks, and the decision rule is architecture reasoning, not a benchmark of your specific line. The way to size it is to measure your own exception rate and cycle time before choosing hardware.
Internal links: the hardware-agnostic AI vision guide and our solutions.
Frequently asked questions
Should factory vision inference run on-prem or in the cloud? On-prem at the edge for the real-time pass/fail verdict, because line-speed inspection needs sub-second deterministic latency. Use the cloud for exceptions, logging and retraining, not for the time-critical decision.
When does cloud or hybrid make sense? When your exception rate times the payload per exception fits inside your cycle-time and bandwidth budget, or when retraining needs centralised data. Then split edge for the verdict, cloud for the tail — never a wholesale move that stops the line if the link drops.
How does OPC UA affect the choice? An on-prem verdict can publish to the PLC over OPC UA in deterministic time; a cloud round-trip adds jitter a controller firing on a fixed cycle may not tolerate, which usually settles the verdict in favour of edge.
Send us your line's cycle time and your current exception rate, and within two weeks we will return the edge/cloud split your latency budget actually supports, before any hardware spend. Start at https://apac.hypernology.net/contact.