Across 47 production deployments, the acceptance disputes came from a single root cause: the buyer treated FAT documentation as production readiness. A vendor lab result and a factory floor result do not test the same things, and the site acceptance test (SAT) is the procedure that closes the gap. This post defines both tests, lists the 12 SAT line items that matter for automated vision systems, and covers the failure modes that cause SATs to fail even when the hardware is sound.
FAT and SAT: what each test actually proves
A factory acceptance test (FAT) runs at the vendor's facility before shipment. The system inspects vendor-supplied samples under lab-controlled lighting, at a speed the vendor selects, operated by engineers who know its failure modes. FAT confirms the machine was built to specification. It does not confirm it will perform in your facility.
A site acceptance test runs in the buyer's facility, with the buyer's production parts, under the buyer's shift lighting, at full line speed, operated by the buyer's staff. SAT confirms the system performs to specification in that specific operating environment. These tests address different failure modes. Treating one as a substitute for the other is the most common source of post-installation disputes in automated inspection.
The distinction matters most for vision systems because inspection performance is a function of the complete optical environment — not just the algorithm. A model trained under flat-lit controlled conditions behaves differently under the high-bay LED or sodium vapor lighting typical in press shops and extrusion halls. Temperature cycling over an 8-hour run, vibration from adjacent press equipment, and operator variance can each shift model confidence enough to change the measured false-reject rate by a meaningful margin.
| FAT — vendor lab | SAT — customer floor | |
|---|---|---|
| Parts | Vendor-supplied samples | Buyer's production parts |
| Lighting | Lab-controlled | Factory ambient plus dedicated light |
| Speed | Demo or contract speed | Full production line speed |
| Operators | Vendor engineers | Customer production staff |
| Primary failure modes caught | Integration errors, hardware gaps | Environmental variation, operator gaps, integration edge cases |
| Acceptance authority | Both parties jointly | Buyer authority final |
| Escape budget | Negotiated in contract | Validated live |
A system can pass FAT cleanly and fail SAT on lighting drift, vibration sensitivity, or operator handover. None of those conditions surface in the vendor's lab.
The 12-item SAT checklist for inline vision systems
The items below are the minimum acceptance criteria for a production inline automated vision inspection system. Each is stated as a pass/fail or a measurable condition. Any item unresolved at sign-off remains an open risk in the production quality record.
1. Sample plan agreed Buyer and vendor agree on minimum sample size per defect class before the test runs — typically 50-100 samples per defect type, not only good parts. The plan is documented and signed off. A sample plan written only by the vendor tests the vendor's training distribution, not the buyer's production process.
2. Lighting repeatability confirmed Light intensity measured at the inspection station across three shifts and across the warmup period. Maximum allowable drift specified in lux or in model confidence-score delta. High-bay LED and sodium vapor installations drift over an 8-hour cycle. Testing only on day one, at full warmup, misses this.
3. FAR and FRR against contract targets False accept rate (FAR) and false reject rate (FRR) targets must be stated in the purchase contract before SAT begins. The test run produces measured rates on buyer-supplied production samples. If measured FAR exceeds contract FAR, the system does not pass. This is the item most procurement contracts underspecify — vendors often quote detection rate alone, without binding FAR to a specific tested condition.
4. Escape budget validated Maximum allowable defective parts per million (DPPM) exiting the inspection station, verified under live conditions. This is the number the quality manager should sign, not the vendor's claimed detection rate from a product sheet.
5. Cycle rate at full production load System measured at line speed with the full adjacent process load running — conveyor, downstream accumulator, upstream feeder. Not demo speed. Throughput delta from contract speed is recorded and accepted or escalated before sign-off. A system that achieves spec throughput on an isolated demo fixture may fall short under the processing load of a connected line.
6. Camera and fixture mount repeatability Mechanical mount checked for vibration sensitivity. Camera position measured before and after a simulated shift cycle covering thermal change and forklift-traffic vibration. Pixel-level shift tolerance specified and confirmed. A mount that performs at 08:00 may not hold by 16:00 after a die change and two forklift passes near the base.
7. Operator handover without vendor assistance Buyer production operators — not vendor engineers — execute: run start, fault recovery, product changeover, false reject override, and retraining trigger. This item is documented as pass/fail with observer sign-off. The operator manual must be complete enough for this to succeed without vendor coaching on the test day.
8. Edge-case library reviewed jointly A defined set of gray-zone parts — borderline accept/reject — reviewed by buyer and vendor before the test. Each classification recorded. System judgment on gray-zone parts compared against the agreed classification during the run. This item prevents "acceptable" meaning different things to the two parties at sign-off.
9. Environmental variation tested System run across the thermal and humidity range of the facility, or across a documented simulated range. If seasonal extremes apply, worst-case conditions must be tested or contractually deferred with a specified retest obligation and date.
10. End-to-end integration confirmed PLC, SCADA, or MES signals tested live: reject ejector fires at the correct timing, alarm escalation reaches the right terminal, production counters match line output. Integration edge cases — network latency on a loaded segment, counter race conditions on simultaneous rejects — surface here, not on a test bench with a single device active.
11. Override and audit log verified Every manual override logged with operator ID, timestamp, and part ID. Quality manager confirms the log is accessible, searchable, and exportable to the QMS format in use. If the system operates within ISO 9001 or IATF 16949 scope, audit readiness is not negotiable.
12. Trained model weights in escrow Vendor delivers trained model weights or equivalent in an exportable format the buyer retains independently. Buyer can redeploy or retrain without a vendor service call. This item protects against vendor lock-in, supports regulatory audit traceability, and is the only checklist item that guarantees the buyer's long-term operational independence from the vendor.
Where vision SATs most often fail
The items that generate post-SAT disputes are rarely the technical specifications. They are the operational gaps that neither party flagged during contract negotiation.
The most common failure point is the sample plan. FAT uses clean samples the vendor has trained on. SAT exposes the system to production variation: oil film from the press, handling marks from the bin, profile drift from a warmed-up die, batch variation in raw material finish. If the sample plan was written only by the vendor, the FAT performance numbers describe the vendor's sample distribution, not the buyer's process.
The second failure point is operator handover (Item 7). A vendor engineer who knows the system's confidence model can recover from a camera misalignment or a model drift event in under a minute. A production operator following a manual written by that engineer, under shift pressure, typically cannot — without training and documented procedure tested before go-live rather than after the first production fault.
The third failure point is integration edge cases: the reject ejector fires 40ms late under network load, the PLC signal drops after a configuration change, the MES counter double-counts on simultaneous rejects. These are integration failures, not vision system failures — but they surface at SAT and are the buyer's responsibility to catch before production starts.
A system that passed FAT and failed SAT on these grounds is not defective. It is incompletely validated.
How AI-based vision changes the SAT requirements
Rule-based vision systems have deterministic behavior: change a threshold or a mask, and the output changes predictably. SAT for a rule-based system is primarily a parametric check — verify the thresholds, run the sample plan, document the outputs.
AI-based vision systems produce probabilistic outputs. The same part may score differently across shifts if factory lighting has drifted, if a new defect variant appears outside the training distribution, or if the model was retrained mid-week. This changes two items in the checklist in practical ways.
First, the sample plan must include defect variants specific to the buyer's process that may not appear in the vendor's training set. For surfaces with batch-to-batch variation — extrusion profiles with die warmup drift, pressed parts with lubricant variation, display panels with coating uniformity shifts — the coverage gaps in the training set become the escape risk. These variants need to be in the SAT sample plan or explicitly flagged as known coverage gaps with an agreed monitoring protocol.
Second, the retraining trigger must be agreed and tested at SAT: what confidence-drift threshold or FRR-drift threshold triggers a retraining event, who owns the decision, and what the buyer's operators do while retraining runs. For HyperQ AI Vision deployments, operator-initiated retraining runs in 30 minutes on a standard pattern inspection. That reduces operational risk on Item 7. It does not eliminate the requirement to test the retraining workflow at SAT itself — the SAT should include a simulated retraining event with the buyer's operators running it end to end, without vendor guidance.
The 10x lower training data requirement — 1,000 images rather than the 10,000 typical for conventional deep-learning deployments — means a retraining triggered at SAT does not require a multi-day sample collection cycle. That is practically significant when the buyer's production parts at SAT differ from the vendor's training samples, which is common in applications such as extrusion and profile inspection where batch-to-batch surface variation is a normal operating condition.
Where a SAT alone is not sufficient
SAT validates the system on acceptance day. It does not guarantee performance across the full product family, across seasonal variation, or after a process change. Two additional provisions should be negotiated as standard.
Conditional acceptance period. A 30-90 day operational window during which FAR and FRR are monitored against SAT baselines. Any drift beyond defined thresholds triggers a vendor response obligation: cause documentation, remediation plan, and a partial retest if required.
Requalification trigger. Any firmware update, model update, or camera hardware substitution requires a partial SAT rerun — at minimum Items 3, 4, 5, and 12 — before the change enters production. This is standard in medical device inspection and in IATF 16949 environments. It should be standard in any inspection system where a defect escape carries downstream cost or liability.
The contract negotiation for a vision inspection system should include both provisions explicitly. They are not implied by a standard warranty clause, and vendors do not add them unless asked.
Frequently asked questions
Q: What is the difference between FAT and SAT?
FAT confirms the machine was built to specification in the vendor's facility. SAT confirms it performs to specification in the buyer's operating environment. FAT is build verification; SAT is operational validation. A system with only FAT documentation has not been validated for production. Both tests are contractually and operationally necessary; neither substitutes for the other.
Q: Who writes the SAT checklist — the vendor or the buyer?
Both parties contribute, but the buyer should set the acceptance criteria for Items 3 (FAR/FRR targets), 4 (escape budget), 5 (cycle rate), and 7 (operator handover). These depend on the buyer's process risk and downstream assembly tolerances; the vendor cannot set them without buyer input. The vendor typically owns documentation for Items 6 (mechanical mount), 10 (integration), and 12 (model escrow).
Q: How long does a vision system SAT take?
For a single-camera system on one product family: 1-3 days, including sample collection, test run, and documentation sign-off. Multi-camera systems with multiple product families take 3-7 days. The most time-consuming element is usually the defect sample plan — collecting sufficient examples of each class under live production conditions. Buyers who prepare defect samples before the SAT date compress this significantly.
Q: What is a reasonable FAR target for an inline vision system?
The answer is process-specific. For automotive safety components or items with a regulatory escape consequence, FAR approaches zero — but reaching this without an unacceptable FRR requires documented 99%+ detection under production conditions, not lab conditions. For lower-risk cosmetic inspection, an escape budget of 50-100 DPPM may be acceptable. The SAT should validate the actual achieved rate against the agreed target, not against the vendor's specification sheet.
Q: Can SAT results be used for regulatory audit purposes?
Yes. In ISO 9001, IATF 16949, and medical device environments, SAT documentation is part of the quality record for the inspection station. Items 1 (sample plan), 4 (escape budget), 11 (audit log), and 12 (model escrow) form the traceability chain an external auditor expects. Retain all SAT documentation for the operating life of the inspection line.
Q: What happens if the system passes FAT but fails SAT?
The system is not accepted. The contractual remedy depends on the purchase agreement — typically the vendor remediates and re-runs the SAT within an agreed period at no charge. Failed items should be documented per checklist line item. This prevents disputes about whether a failure is a vendor deficiency or a site condition the buyer is responsible for. Documentation specificity at this stage is what protects both parties.
Send your current FAT report and production-floor specification — part list, line speed, shift count, and lighting type — and we will identify the SAT items most likely to fail in your environment before the vendor arrives on site. You receive a site-specific SAT risk checklist within 5 business days, no contract required. Start the site assessment