How to choose an AI vision system: a buyer's guide for manufacturing operations

Across 47 production contracts in semiconductor, automotive parts, display panels, PCB, plating, packaging, and laser-engraving, the criterion that has decided which deployments stay in production for five years and which get torn out in eighteen months is not on most buyer's guides. It is not accuracy. It is not speed. It is not resolution. It is whether the system is still maintainable on the day after the engineer who set it up walks out the door.

This guide is for operations directors past the demo stage. The first four sections cover the criteria most buyer's guides skip — including, for years, ours. The remaining ten cover the standard procurement checklist, with the questions that separate a real answer from a vendor brochure.

A vendor demo evaluates the vendor's best scenario: their controlled lighting, their pre-trained samples, their engineer in the room. Your evaluation has to evaluate their worst day. The day the engineer is gone, the line speed is up, the SKU mix has shifted twice this quarter, and the support phone tree no longer remembers which version of the system you are running.

Part 1: The criteria your vendor will not publish

1. The third-engineer test

Ask any experienced controls engineer what they would buy for a five-year deployment, and the answer rarely starts with accuracy. It starts with: who maintains this after I leave?

One controls engineer captured the post-handover problem precisely: "I tend to find the vendor systems better. Not necessarily for me, but for every other poor sod who comes after me that has to maintain the thing." Plants think in decades. Engineers stay 2-3 years. Custom-built systems — OpenCV pipelines, in-house deep learning stacks, bespoke labelling tools — concentrate operating knowledge in one person. When that person leaves, the documentation is incomplete, the model retraining script lives on a personal laptop, and the next engineer is running a system they cannot diagnose.

The third-engineer test cuts through this:

• Can a competent engineer who was not present at setup diagnose, retrain, and modify the system within two weeks of onboarding?
• Is the model retraining process documented, version-controlled, and runnable from a standard interface, or does it require knowledge that lives in someone's head?
• Are the model weights, datasets, and inference configuration portable across machines, or tied to one workstation?

The right answer is operator-level retraining from the line, version control on every deployed model, and a rollback path that does not require the original implementer. HyperQ AI Vision is built so that a process engineer who joined last month can label new defects, retrain on the line, and roll back if performance degrades — without raising a vendor ticket. That is not a feature. That is the only criterion that protects a five-year deployment from the inevitable engineer turnover.

2. The support inversion: pre-sale versus post-sale

Two structural patterns in vision-system procurement that almost no buyer's guide names:

Pattern A: Unlimited pre-sale support, thin post-sale support. Some vendors offer effectively free pre-sale engineering — repeated demos, custom configurations, on-site visits. The model works because rep turnover is high, so the engineer who specced your system has moved on by year two. Your post-sale contact changes, the vendor's institutional knowledge of your specific deployment evaporates, and a support call eighteen months in lands with someone who has never seen your line.

Pattern B: Charged-for setup support. Other vendors historically billed for on-site support visits at four-figure day rates. The salesman shows up free. The engineer who actually fixes things shows up with an invoice. The cost shifts from the procurement line item to the operating budget, where it stays for the life of the system.

Either pattern is recoverable if you know it is there. Neither pattern is on the comparison spreadsheet at procurement time.

The right test is not "do you have local support." The right test is:

• What does a support call look like 18 months after deployment, when the original sales engineer has moved on?
• Who specifically would respond to a P1 line-down ticket at 2 AM in Penang or Rayong, and what is their direct number?
• How is the institutional knowledge of our specific deployment retained on your side when the original implementer leaves your company?
• Can you name two reference customers who have been live for more than three years and ask about their last support interaction?

HyperQ maintains APAC-based engineering support with named engineers per deployment and defined response SLAs by severity tier. The customer relationship is structured around the deployment, not the salesperson. A production stoppage at 2 AM in Southeast Asia is not waiting for a business day in Munich or Boston.

3. Debuggability over accuracy

Every vendor demo opens with an accuracy figure: 99.x% detection on a sample set. The accuracy figure is the wrong leading question.

Two reasons. First, on critical inspection — semiconductor, medical device, automotive safety parts — 99% is not a passing grade. A practitioner observation that gets repeated in vision-engineer discussions: 99% accuracy is rarely acceptable in production; the actual requirement is five or more nines. The spec-sheet number is already below the production threshold for the inspections that matter most.

Second, and more important: accuracy describes the system on a good day. Debuggability describes the system on a bad day. When the model makes a wrong call — and it will — what diagnostic path is available to the process engineer?

A practitioner running a black-box AI camera in production described the failure mode directly: the algorithm is qualitative, debugging is impossible, and one misclassification ruins the model because there is no way to trace why. You get a yes or no, never a why. The output of "AI bad" is the output of "AI good" minus one bit. That is enough to halt a line. It is not enough to fix the line.

The reframe:

• Don't evaluate accuracy at the demo. Evaluate diagnostic transparency at failure.
• Ask: when the system flags a part as defective, can a process engineer see exactly which region of the part triggered the flag, which trained feature responded most strongly, and what the confidence distribution looked like?
• Ask: when the system misses a defect, what audit trail allows root-cause analysis without proprietary tooling we do not own?

HyperQ AI Vision outputs visual explainability as standard: Grad-CAM heatmaps, bounding-box overlays at micrometer-level localization, defect classification with dimensional data, and confidence scores on every decision. A process engineer can connect a reject signal back to a specific tool position on a specific station. That is the difference between automation you trust and a pre-filter that creates more human review than it replaces.

The spec sheet says 99%. The diagnostic interface says whether you will still be running it in year three.

4. The lock-in vectors practitioners name (and brochures hide)

Hardware-locked AI vision platforms create at least four lock-in mechanisms that compound over a deployment lifetime. Most appear nowhere in the procurement comparison.

Hardware-gated feature upgrades. A practitioner pattern that recurs across vendor reviews: when an existing customer requests a new inspection capability, the response is that the feature will appear in the next generation of cameras — meaning the customer must replace the hardware to access the software upgrade they asked for. The release cadence becomes a forced refresh cycle. This is a structural pattern in some hardware-locked vision platforms, and it does not appear in any pre-sale comparison.

Single-instance software constraints. Some inspection software is licensed and engineered such that only one instance can edit a configuration at a time across all users at the site. As deployments scale across multiple lines and shifts, this becomes a real operational bottleneck — and it is rarely documented in evaluation material.

Operating system dependency. Vision software tied to a specific Windows version creates an upgrade-or-orphan decision the day enterprise IT enforces a security update. One controls engineer described the experience: when his team called support about a vision integration on Windows 11, the first question was the OS version, and the conversation ended with "have a good day." OS dependency is an invisible lock-in vector that surfaces only when the OS changes.

Knowledge lock-in. The most expensive of all. When a custom system depends on the original implementer's knowledge — a custom OpenCV pipeline, an in-house training notebook, a labelling tool nobody else can run — replacing that engineer means rebuilding the system from scratch. Vendor systems shift this lock-in to the vendor. Custom systems leave it inside your organization, where it walks out the door with the people who built it.

The procurement reframe:

• Hardware lock-in is recoverable. You buy new cameras and move on.
• Data lock-in is structural. The trained model and the labelled defect library cannot be exported as a CSV.
• Knowledge lock-in is invisible. It surfaces the day someone leaves.

HyperQ AI Vision works with any industrial camera (Basler, FLIR, Sony, others) via GenICam or GigE Vision, runs as software on standard industrial PCs without OS-version coupling, and version-controls every model so the institutional knowledge stays in the system, not in any one engineer's notebook. The customer owns the model weights, the labelled data, and the configuration. That is the architecture that survives engineer turnover.

Part 2: The criteria every buyer's guide already covers

The remaining ten criteria appear on every reasonable evaluation rubric. The questions below are the ones that separate a real answer from a vendor brochure.

5. Camera hardware compatibility — and the physics layer underneath

Does the system work with cameras you already own?

Some vendors require proprietary hardware. That means a capital line item on day one — typically $3,000-$8,000 per camera position for proprietary units, versus $420-$1,200 for standard industrial cameras connecting via GenICam or GigE Vision. The price difference compounds across a multi-line facility. The dependency follows you through every future upgrade.

There is a deeper point underneath the camera question that most buyer's guides skip: roughly 95% of the success of a vision application is determined by lighting and optics, not by the AI model. The line repeated by experienced practitioners — "you can't program around physics" — is operational reality. Bad lighting on a curved reflective surface defeats any model. The right camera matters because the lighting and optics matter, and the camera is what carries the optics.

Ask:

• Which camera brands and models are supported out of the box?
• Can we bring existing GigE or USB3 cameras into the deployment?
• What lighting and optics validation does the deployment include before model training begins?
• What happens to our hardware investment if we switch vendors in three years?

HyperQ AI Vision works with any industrial camera and any existing CCTV infrastructure. A typical deployment saves 30-50% on hardware compared with systems that require dedicated proprietary cameras. The deployment process explicitly validates the lighting and optics before any model training, because no AI fixes a physics problem.

6. Training data requirements

This is where most AI vision evaluations stall. A vendor quotes high detection accuracy, but the fine print requires 10,000 labelled images per defect class before the model reaches production readiness.

Labelling is expensive, slow, and dependent on defect availability. On a line that sees a given defect once a week, 10,000 examples is a multi-year data collection project. For low-frequency failure modes — micro-cracks that appear a handful of times per quarter, surface voids on reflective metal — a 10,000-image requirement is a project cancellation.

Ask:

• How many labelled images does your system need to reach 99% detection at micrometer precision?
• Can the system learn from rare-defect distributions, or does it require balanced datasets?
• Does the system support semi-supervised or unsupervised training for rare defect types?
• Who does the labelling work, and who pays for it?

HyperQ's patented low-data training reaches 99% detection at micrometer-level precision, including on highly reflective metal surfaces, with as few as 1,000 images per class. For common defect types, the 8,000+ pre-trained model library eliminates cold-start labelling entirely: zero-configuration inspection from deployment day. 1,000 images versus 10,000 — or zero versus 10,000 when a pre-trained model exists — is the gap between a project that launches in a quarter and a project that gets shelved.

7. Retraining and customization workflow

Production lines change. New packaging, new suppliers, new surface finishes. A system that requires a formal retraining project every time something shifts will create a maintenance burden your team was not budgeting for.

The deeper problem: most systems require vendor involvement for retraining. A support ticket, a project scope, a timeline, an invoice. Each cycle. Indefinitely. The real cost is not the retraining itself but the dependency and the lag between "something changed on the line" and "the model catches up."

Ask:

• How do we retrain when product specifications change?
• Can operators trigger retraining from the line, or does it require vendor involvement?
• What is the typical turnaround from retraining request to redeployment?
• Does the system version-control deployed models so we can roll back if a retrain degrades performance?

The ability to retrain without raising a support ticket — at operator level, from the line, with version control — is the difference between a system you own and a system you rent. Over three years, that difference compounds into a meaningful capability gap and a meaningful cost gap.

8. Multi-SKU changeover handling

Most manufacturing operations are not single-SKU. Product changeovers introduce variation in shape, color, surface texture, and defect profile. In high-mix environments running hundreds of SKUs across automotive components, electronics, or consumer goods, systems that treat each SKU as a separate manual configuration create downtime at every changeover. That downtime accumulates into real production cost.

Ask:

• How does the system switch inspection parameters when a new SKU runs?
• Is changeover triggered automatically from PLC signals, or does it require operator input?
• How many SKUs can the system manage simultaneously?
• What is the model-load latency when switching between product configurations?

HyperQ's PLC auto-switching loads the correct inspection model in under two seconds from a PLC signal (Siemens, Allen-Bradley, Mitsubishi) with zero operator input. Across 8,000+ product model configurations — Client A, an automotive parts manufacturer, runs 8,000 product variants on six lines at 11,520 units per day with zero-configuration changeover. That eliminates the changeover labor cost attached to every SKU transition.

9. Edge deployment versus cloud processing

A line running 200+ parts per minute cannot tolerate cloud round-trip inference. A facility with ISO 27001 or regional data sovereignty requirements cannot route production imagery to a vendor's cloud. These are hard constraints, not preferences.

Ask:

• What is the inference latency at the edge versus cloud?
• Can the system operate fully offline if network connectivity drops?
• How are model updates pushed to edge nodes across multiple sites?
• What compute hardware does the edge deployment require, and does it run on existing industrial PCs?

A vendor who argues one deployment model fits every situation is not engaging seriously with your environment. The default for production-critical inspection is edge, with cloud reserved for non-real-time analytics on aggregated data.

10. MES and ERP integration depth

An inspection system that cannot push data to your manufacturing execution system or ERP is an island. You get defect counts. You do not get traceability, process correlation, or the data foundation for continuous improvement.

Ask:

• Which MES and ERP platforms does your system integrate with natively?
• What protocols are supported: PROFINET, EtherNet/IP, OPC-UA?
• What data formats and APIs are available for custom integrations?
• Can the system surface defect data at the batch and serial number level?

Established rule-based AOI vendors have a genuine advantage in this category. They have been in the ecosystem longer and have pre-built connectors to platforms like Aegis FactoryLogix, SAP ME, and Aveva MES. Evaluate whether a newer AI vision platform can match that integration depth for your specific MES before assuming it can. Ask for a reference deployment on the same MES platform you run.

11. Process drift monitoring

Process drift is the silent failure mode of AI inspection programs. Your line evolves over months: new product variants, equipment wear, seasonal lighting shifts, a new supplier whose material has slightly different reflectivity. Model performance degrades in ways that are invisible unless you are actively measuring it.

A vendor that scores 99% in a controlled pilot may be running at 91% six months into production. The gap is predictable. The question is whether the system detects it and alerts your team before an escape event reaches your customer.

Ask:

• Does the system continuously monitor confidence scores and flag when performance falls outside a baseline?
• Is drift detection real-time, or does the vendor rely on scheduled quarterly retraining?
• How does the system alert your team when detection accuracy drops below a threshold?
• Can you show detection rate trend data from a customer who has been live for more than twelve months?

A system that cannot detect its own degradation will not tell you when it stops working. Real-time drift monitoring with continuous confidence scoring and configurable alerts is the requirement, not an upgrade tier.

12. Training data and model ownership

Many AI inspection contracts are structured so the vendor retains ownership of the model weights and the labelled training data generated from your line. This creates a switching cost that rarely surfaces during the sales process.

If you want to change vendors in year three, you may be starting from zero. Your defect library built over 36 months of production labelling. Your edge cases that took two years to accumulate. All belonging to someone else. Rebuilding that library with a new vendor is not a migration. It is a greenfield project at full cost.

Ask:

• Who owns the labelled training data generated on our line?
• Who owns the trained model weights?
• If we terminate the contract, do we retain access to the data and models?
• Can we export our training datasets in a standard annotation format?

Get data ownership terms in writing before signing. A vendor confident in their product's long-term value will compete on performance, not on the switching cost of leaving.

13. The demo-versus-production gap

Every AI vision vendor demo follows the same script. Controlled lighting. Product geometry the system has been trained on extensively. A pre-selected sample set of defect types the model handles well. Detection rates measured against that sample, presented with confidence.

What you are watching is the system's performance on the most favorable version of the problem it was built to solve. Asking a vendor to evaluate their own performance on their own sample data is the same as asking a job candidate to provide only the references they chose. The information is real. It is not representative.

To cut through this:

• Test on your production conditions. Run the system on a representative sample of your actual product, including variants, edge cases, and the defect types that matter most. Measure detection rates under your actual line conditions: your lighting rig, your line speed, your surface finishes, your defect distributions.
• Ask for longitudinal data. Request detection rate trend data from reference customers who have been live in production for more than twelve months. Not case studies or pilot results, but actual precision-recall data over time showing how the system performed as the line evolved.
• Request two references. One customer deployed for two or more years (tells you about stability, support quality, and drift management), and one that went live in the last six months (tells you about the current onboarding experience).

The demo is the rehearsal. Your production conditions are the test.

14. The real integration cost

Every AI vision demo ends at the same moment. The camera captures a part. The model scores it. A bounding box appears around the defect. The confidence reads 97.4%. The demo ends. What it does not show is how that inspection result gets from the AI system into your MES, your ERP, or your quality management platform.

Ask any vendor to walk you through their MES integration architecture. What you typically get is a slide that says "open API" or "standard protocols supported." What you do not get is a diagram of the actual data handshake, the latency at each step, the maintenance burden over time, and who pays when something breaks.

Integration overhead across a typical mid-size AI vision deployment adds 15-25% to total project cost. In environments with multiple MES instances or non-standard ERP configurations, it goes higher. The cost breaks down into four components:

• Data model mapping. Translating AI system output into the format your MES expects, often requiring custom middleware.
• QA cycles for middleware. Testing the connection against every production condition, absorbing engineering time that was not in the original project scope.
• Ongoing maintenance. When either system updates, the middleware may break. Recurring cost for the life of both systems.
• Deployment lag. While integration is being built, the AI system runs disconnected from production workflows. That gap delays ROI by weeks or months.

The alternative is native MES/ERP connectivity built into the product architecture from day one. HyperQ AI Vision connects directly to common platforms via PROFINET, EtherNet/IP, and OPC-UA without a bridging layer. Inspection results — defect classification, dimensional data, batch ID, serial number — enter production workflows in real time.

Before committing to any platform, ask:

• How does the inspection result enter my MES — natively, or through middleware I maintain?
• Who builds and maintains that connection?
• What is the latency from inspection event to MES record?
• What happens when the MES platform updates?

These answers determine total deployment cost more reliably than accuracy numbers from a controlled demo.

Part 3: The budget credibility gap

Operations directors get caught in a pattern that has become routine in AI vision procurement: management sees an AI vision demo online, assumes it costs €1,500 and a few hours of engineering, and asks why the proposed deployment quote is forty times higher.

The answer is not that the vendor is gouging. The answer is that the demo on a phone screen is solving a different problem — controlled lighting, a single SKU, a known defect distribution, an engineer in the loop, no integration to anything. A real production deployment for an inspection that matters typically lands between $5,000 for a single-station pilot and $40,000+ per line for a fully integrated, MES-connected, multi-SKU deployment with ownership of model weights and data. Custom in-house builds frequently consume 200-300 hours of internal engineering time before reaching a stable state, and the engineer who built it then becomes the single point of failure for the system.

A line that gets repeated by people who have done this twice: nothing is more expensive than cheap hardware. The €1,500 vision system either fails on the physics layer (lighting, optics, camera quality) or fails on the maintenance layer (the one engineer who built it leaves, and the system is unmaintainable). Either failure mode is more expensive than buying an integrated solution that survives engineer turnover and a Windows update.

The real TCO comparison includes:

• Hardware (cameras, optics, lighting, edge compute)
• Implementation (specification, deployment, integration)
• Labelling and training data acquisition
• Annual licensing and maintenance
• Internal engineering time for ongoing operation
• Cost to onboard the third engineer to the system, two years from now
• Cost of the support call that nobody on the vendor side remembers

That last line is the one most procurement spreadsheets do not have a column for. Adding it is what separates the deployments that pay back from the deployments that quietly drain capital for years.

Score vendors systematically — and weight by your reality

The criteria in this guide work as a scoring rubric. For each vendor under evaluation, assign a response quality rating (strong, partial, weak) across every dimension. Weight the categories by your operational reality. If you run high-mix lines with frequent changeovers, multi-SKU handling and retraining workflow are non-negotiable. If you operate across APAC with limited internal integration engineering, regional support continuity and native MES connectivity move to the top.

The first four criteria — third-engineer test, support inversion, debuggability, lock-in vectors — are the ones that decide whether the system is still running in year five. Weight them accordingly, even though no vendor will lead with them.

The system that wins your evaluation is not the one that performed best in a controlled demo. It is the one that still works — and is still understood — by the third engineer to inherit it.

Stop watching rehearsals. Run the real test.

Your line has specific products, specific defects, and specific constraints that no vendor demo will replicate. The only evaluation that matters is the one that runs on your conditions, with your engineers, on your worst SKU.

Send us your toughest part and the defect type your current system misses. We will run it through HyperQ AI Vision, send back the detection results, the defect classification, the explainability output, and a deployment estimate — within two weeks, with no contract until the spec is met.

Send a sample, run the real test →