HyperQ AI Vision achieves a 99% defect detection rate on production lines. That number comes from a system that has been validated against known good and known bad parts — a process that is itself a form of measurement system analysis. The problem is that most manufacturers who deploy a vision system never run that analysis formally. They look at the accuracy claim and assume the number applies to their line, their parts, their lighting conditions.
It often doesn't. Not because the system is wrong, but because a measurement system's performance is a property of the system-in-context, not of the software in isolation. The same logic that makes Gauge R&R essential for calipers makes it essential for camera systems. The vision sensor is a gauge. It needs to be treated like one.
What Gauge R&R measures, and why it matters here
Gauge Repeatability and Reproducibility (Gauge R&R) is part of Measurement System Analysis (MSA). The purpose is to quantify how much of your observed variation comes from the measurement system itself versus the parts you're measuring. A gauge that introduces significant variation produces confident answers that may be systematically wrong — you're measuring the gauge, not the part.
The standard breakdown has three components:
- Repeatability: The same operator measures the same part multiple times with the same gauge. How consistent are the results?
- Reproducibility: Different operators (or different measurement conditions) measure the same part. How consistent are the results across conditions?
- Bias: The gauge's average reading versus the known reference value. How close is the system to truth?
For a caliper or CMM, these questions are straightforward. For a vision system, they translate directly:
- Repeatability: The same part runs through the camera five times in identical conditions. Does the system return the same verdict each time?
- Reproducibility: The same part runs under varying conditions — different lighting intensity, slight angular variation, different ambient temperature. Does the verdict hold?
- Bias: The system's results versus a golden-sample reference set of known good and known bad parts. What is the false-reject rate and false-accept rate against ground truth?
A vision system that passes repeatability but fails reproducibility is telling you something specific: the model is stable, but your physical setup isn't. The variable is almost always lighting.
Why vision systems skip MSA and what it costs
Quality teams that would never release a caliper without a GR&R study routinely deploy vision systems with only a demo-environment accuracy figure. The reasons are practical: vision system vendors typically present accuracy data from controlled conditions, MSA protocols weren't written with vision systems in mind, and the system "seems to be working" once it's on the line.
The cost shows up later. A vision system running a high false-reject rate — good parts being flagged — drives operators to override or disable the system. A system with an unchecked false-accept rate passes defects confidently; you don't find out until a customer return. An uncapable measurement system doesn't fail loudly. It produces confident wrong answers, and those confident wrong answers get built into your process Cpk.
One quality manager observed it directly after a customer audit: the process capability index looked healthy on paper. The vision system was producing 60% false rejects on a new SKU variant introduced three months earlier. Operators had learned to override it. The actual detection rate on the new variant was close to zero.
The GR&R study would have caught this in the first week.
Running a GR&R study on a vision system
The following walkthrough describes how to structure a measurement system analysis for HyperQ AI Vision or any AI-based inspection system. It adapts the AIAG MSA methodology to the vision context.
Step 1: Define the golden-sample set
A GR&R study requires a set of reference parts with known classification. For a vision system, this means:
- 10-30 parts spanning the full range of the defect class under study
- Parts classified as "good" and "bad" by a qualified human inspector, with documented rationale
- Parts stored in controlled conditions to prevent damage or change between test runs
The golden-sample set is not a convenience sample of easily classified parts. It should include borderline cases — parts near the acceptance boundary — because those are the cases where the measurement system's capability matters most.
Step 2: Repeatability study
Run each golden-sample part through the system 5-10 times under nominally identical conditions (same lighting, same mounting, same ambient conditions). Record the verdict for each run.
A capable vision system should return identical verdicts on clearly good and clearly bad parts 100% of the time. For borderline parts, some variation is expected. The question is whether the variation is systematic (the model consistently disagrees with itself on the same part) or random (small confidence-score variation around the boundary).
For HyperQ AI Vision, the expected repeatability on clearly in-spec and clearly out-of-spec parts is near-perfect. Borderline part variation typically reflects confidence scores in the 45-55% range — the system is uncertain, which is the correct signal when a part is genuinely close to the boundary.
Step 3: Reproducibility study
Vary the conditions that are likely to change in production:
- Lighting intensity variation (±10% of nominal — a realistic range across shift changes and lamp aging)
- Part positioning variation (within the expected placement tolerance on the fixture)
- Ambient temperature variation if the line is near HVAC vents or heat sources
Run each golden-sample part under each condition set. Compare the verdicts.
In practice, reproducibility failures in AI vision systems almost always trace to lighting, not model behavior. A model trained on images captured at 800 lux will behave differently at 600 lux after a lamp dims. This is not a model failure — it is an illumination maintenance failure. The GR&R study identifies it; the fix is a lighting maintenance schedule, not a model retrain.
Hardware-agnostic deployment introduces one reproducibility variable worth monitoring explicitly: if the same vision model runs on cameras with different optics across multiple production sites (a common scenario when deploying on existing infrastructure), the reproducibility study should test across camera types to confirm that model performance is consistent. In most cases, it will be — HyperQ AI Vision is designed to compensate for hardware variation — but this should be verified rather than assumed.
Step 4: Bias assessment
Compare the system's verdict distribution against the golden-sample ground truth:
- False reject rate: good parts classified as defective
- False accept rate: bad parts classified as good
- Agreement rate on borderline parts: cases where the system verdict matches the human inspector
HyperQ AI Vision's 60-80% false-positive reduction compared to rule-based AOI reflects bias tuning toward the "don't reject good parts" direction. A GR&R study on a new deployment will establish the actual false reject and false accept rates for your specific part family and defect classes — which may differ from published benchmarks depending on part complexity and lighting conditions.
The bias assessment on borderline parts is particularly valuable. If your golden-sample set contains 10 borderline parts that three qualified inspectors classified as "fail" and the vision system classifies as "pass" on 8 of them, that is a calibration gap worth investigating before the system goes to full production.
The vision-GR&R worksheet
The following table summarizes the standard GR&R study structure adapted for vision systems. Run this study at initial deployment, after any physical setup change (camera mount, lighting replacement, fixture modification), and quarterly as a drift check.
| Study component | What to measure | Acceptance threshold | Common failure cause |
|---|---|---|---|
| Repeatability — clear parts | Verdict agreement, 5 runs per part, 10+ clear parts | 100% agreement | Model instability (rare); fixture vibration |
| Repeatability — borderline parts | Verdict agreement, 5 runs per part, 5+ borderline parts | >80% agreement | Correct signal: part is near boundary |
| Reproducibility — lighting ±10% | Verdict change vs nominal, 10+ parts | <5% verdict change | Lighting spec too wide; model undertrained on low-light |
| Reproducibility — position variation | Verdict change vs nominal, 10+ parts | <5% verdict change | Fixture tolerance too loose; field-of-view edge effects |
| Bias — false reject rate | Good parts flagged as defective / total good parts | <5% (strict); <10% (acceptable) | Overfit model; threshold too aggressive |
| Bias — false accept rate | Bad parts passed as good / total bad parts | <1% (strict); <2% (acceptable) | Underfit model; borderline defect class not trained |
| Agreement on borderline parts | Vision verdict vs. qualified human inspector | >85% agreement | Ground-truth disagreement among inspectors; recalibrate |
When a study component falls outside threshold, the remediation depends on the root cause:
- Repeatability failure on clear parts → investigate fixture, vibration, or camera mounting
- Reproducibility failure under lighting variation → implement lighting maintenance schedule; adjust maintenance interval
- Bias on false reject → tune confidence threshold; increase good-part training samples
- Bias on false accept → add labeled bad-part samples; check that defect class is in training set
- Agreement failure on borderline → recalibrate the golden-sample set with a three-inspector consensus review
When the GR&R reveals your lighting, not your model
The most common finding in a vision system GR&R study is a reproducibility failure that traces directly to lighting inconsistency. This is worth understanding clearly because the instinct is to retrain the model.
A model trained on consistently lit images performs well under those conditions and degrades predictably as lighting drifts. The GR&R study, if it tests the ±10% lighting range, will show the inflection point: at what illumination level does the model's verdict begin to change on borderline parts?
That inflection point is a maintenance parameter. Set the lighting maintenance trigger at a point that keeps the system within the stable region — not at the point where the lamp has visibly failed. An AI vision system with a lighting maintenance schedule tied to the model's validated operating range is a capable measurement system. The same system without that schedule is not.
The continuous learning and model retraining article covers what to do when the degradation is in the model rather than the environment — when production conditions have shifted enough that the training data is genuinely stale.
Connecting GR&R results to your process Cpk
The point of measurement system analysis is to interpret process capability correctly. A process Cpk calculated against an uncapable measurement system is not your process Cpk — it is a mix of your process capability and your gauge capability, and you cannot tell them apart without the study.
For a vision system running at 99% detection rate on a validated golden-sample set, the contribution to measurement error should be small. For a vision system that has never been formally studied, running on lighting that hasn't been checked in three months, on a part family that was added after the initial deployment — the measurement error contribution could be substantial, and your Cpk number is not telling you what you think it is.
The false reject rate article covers the relationship between false reject rate and process yield in more detail. The GR&R study described here establishes the false reject rate; that article covers what to do with it.
Maintaining capability over time
A GR&R study at deployment establishes a baseline. The question is how long that baseline stays valid.
The two degradation mechanisms for AI vision are physical (lighting aging, lens contamination, fixture drift) and statistical (model drift as production conditions shift). Physical degradation is detectable in a repeatability or reproducibility re-run. Statistical degradation shows up in the false accept rate creeping upward and the false reject rate shifting, often without any obvious physical cause.
A practical maintenance schedule runs:
- Weekly: false reject rate check (flag if rate increases >5% from baseline)
- Monthly: golden-sample re-run against the original GR&R set (flag if agreement rate drops below threshold)
- Quarterly: full reproducibility study, including lighting variation test
This schedule is not a significant labor burden on a running line — the weekly false reject check is a report pulled from the system; the monthly golden-sample run takes under an hour if the sample set is maintained. The quarterly study is the equivalent of a caliper recalibration — it should be a scheduled maintenance item, not an emergency response to a customer complaint.
HyperQ AI Vision includes the inspection audit trail and performance monitoring needed to support this schedule. The data is there; the maintenance discipline is what makes it useful.
Frequently asked questions
Does a 99% detection rate mean the vision system has already passed a GR&R? No. The 99% detection rate is a benchmark against a validated test set under controlled conditions. A GR&R study on your deployment tests repeatability, reproducibility, and bias against your specific parts, your lighting conditions, and your production environment. The benchmark tells you what the system is capable of; the GR&R tells you what it is actually doing in your plant.
What is a typical GR&R outcome for a well-deployed AI vision system? A properly configured HyperQ AI Vision deployment on clearly defined defect classes typically shows near-perfect repeatability on clear parts and reproducibility within ±5% verdict change across ±10% lighting variation. Borderline parts show expected variance. The most common finding that triggers action is a false reject rate above 5-10%, which usually traces to an overly aggressive confidence threshold or a lighting spec that has drifted.
How is AI vision GR&R different from GR&R on a caliper? The structure is the same (repeatability, reproducibility, bias) but the sources of variation differ. A caliper's repeatability failure is usually operator technique or instrument wear. A vision system's repeatability failure is usually fixture or vibration. Reproducibility failure in a caliper comes from operator-to-operator variation; in a vision system, it comes from lighting or environmental variation. The model itself rarely produces repeatability failures on a well-trained deployment.
How often should the GR&R study be re-run? A full re-run is appropriate after any physical setup change (camera remount, lighting replacement, fixture modification) and on a quarterly schedule as a drift check. The interim monitoring (weekly false reject rate, monthly golden-sample run) provides earlier warning between full studies.
What if the golden-sample set itself is unreliable? This is a real problem. If the parts in the golden-sample set were classified by a single inspector without a consensus review, or if they have degraded since the initial study (surface changes, handling damage), the study results are unreliable. The fix is a three-inspector consensus classification of borderline parts before the study runs, and controlled storage of the sample set between runs.
If you have an AI vision deployment that has never been formally validated, send us your golden-sample set — or describe your part family and defect classes. We will run a repeatability and bias assessment on your specific setup and return a capability report within 10 business days. No contract until the system passes your acceptance threshold.