Synthetic-to-real domain adaptation in computer vision systems: towards high-precision industrial applications

Abstract

The increasing demand for automated quality control in manufacturing highlights the limitations of conventional visual inspection systems that rely on scarce real-world data. In particular, defect detection and measurement of sheet metal parts are hindered by small datasets, domain shifts, and high annotation costs. To address this, we propose a domain-adaptive instance segmentation pipeline that leverages photorealistic synthetic datasets, generated from CAD models, in combination with a small number of real annotated images. Our approach builds upon Mask RCNN with attention-based enhancements, focusing not only on segmentation accuracy but also on downstream dimensional measurements critical for industrial tolerances. We conduct a comparative study between models trained solely on synthetic data and those trained on mixed synthetic and real datasets, using sheet metal parts with annotated slots and holes as benchmarks. Results demonstrate that synthetic-only training can achieve sub-millimetre measurement errors when the rendering pipeline is sufficiently realistic, while hybrid training improves robustness in challenging real-world scenarios such as complex backgrounds or reflective surfaces. Fine-grained per-feature and per-class analyses reveal that slot segmentation benefits more from mixed data, whereas circular holes are well captured by synthetic-only training. Overall, this work provides quantitative evidence that hybrid synthetic-to-real training reduces annotation costs, improves reliability, and offers a practical pathway toward scalable, high-precision inspection in industrial environments.