Handbook of Machine and Computer Vision (2^nd Ed.) The Guide for Developers and Users

Preface Second Edition xxiii

Preface First Edition xxv

List of Contributors xxvii

1 Processing of Information in the Human Visual System 1
Frank Schaeffel

1.1 Preface 1

1.2 Design and Structure of the Eye 1

1.3 Optical Aberrations and Consequences for Visual Performance 3

1.4 Chromatic Aberration 10

1.5 Neural Adaptation to Monochromatic Aberrations 11

1.6 Optimizing Retinal Processing with Limited Cell Numbers, Space, and Energy 11

1.7 Adaptation to Different Light Levels 12

1.8 Rod and Cone Responses 14

1.9 Spiking and Coding 16

1.10 Temporal and Spatial Performance 17

1.11 ON/OFF Structure, Division of the Whole Illuminance Amplitude 18

1.12 Consequences of the Rod and Cone Diversity on Retinal Wiring 18

1.13 Motion Sensitivity in the Retina 19

1.14 Visual Information Processing in Higher Centers 20

1.14.1 Morphology 21

1.14.2 Functional Aspects – Receptive Field Structures and Cortical Modules 22

1.15 Effects of Attention 23

1.16 Color Vision, Color Constancy, and Color Contrast 23

1.17 Depth Perception 25

1.18 Adaptation in the Visual System to Color, Spatial, and Temporal Contrast 26

1.19 Conclusions 26

Acknowledgements 28

References 28

2 Introduction to Building a Machine Vision Inspection 31
Axel Telljohann

2.1 Preface 31

2.2 Specifying a Machine Vision System 32

2.2.1 Task and Benefit 32

2.2.2 Parts 33

2.2.2.1 Different Part Types 33

2.2.3 Part Presentation 33

2.2.4 Performance Requirements 34

2.2.4.1 Accuracy 34

2.2.4.2 Time Performance 34

2.2.5 Information Interfaces 34

2.2.6 Installation Space 35

2.2.7 Environment 35

2.2.8 Checklist 35

2.3 Designing a Machine Vision System 36

2.3.1 Camera Type 36

2.3.2 Field of View 37

2.3.3 Resolution 38

2.3.3.1 Camera Sensor Resolution 38

2.3.3.2 Spatial Resolution 38

2.3.3.3 Measurement Accuracy 38

2.3.3.4 Calculation of Resolution 39

2.3.3.5 Resolution for a Line Scan Camera 39

2.3.4 Choice of Camera, Frame Grabber, and Hardware Platform 40

2.3.4.1 Camera Model 40

2.3.4.2 Frame Grabber 40

2.3.4.3 Pixel Rate 40

2.3.4.4 Hardware Platform 41

2.3.5 Lens Design 41

2.3.5.1 Focal Length 42

2.3.5.2 Lens Flange Focal Distance 43

2.3.5.3 Extension Tubes 43

2.3.5.4 Lens Diameter and Sensor Size 43

2.3.5.5 Sensor Resolution and Lens Quality 43

2.3.6 Choice of Illumination 44

2.3.6.1 Concept: Maximize Contrast 44

2.3.6.2 Illumination Setups 44

2.3.6.3 Light Sources 45

2.3.6.4 Approach to the Optimum Setup 45

2.3.6.5 Interfering Lighting 46

2.3.7 Mechanical Design 46

2.3.8 Electrical Design 46

2.3.9 Software 46

2.3.9.1 Software Library 47

2.3.9.2 Software Structure 47

2.3.9.3 General Topics 48

2.4 Costs 48

2.5 Words on Project Realization 49

2.5.1 Development and Installation 49

2.5.2 Test Run and Acceptance Test 49

2.5.3 Training and Documentation 50

2.6 Examples 50

2.6.1 Diameter Inspection of Rivets 50

2.6.1.1 Task 50

2.6.1.2 Specification 51

2.6.1.3 Design 51

2.6.2 Tubing Inspection 55

2.6.2.1 Task 55

2.6.2.2 Specification 55

2.6.2.3 Design 56

3 Lighting in Machine Vision 63
Irmgard Jahr

3.1 Introduction 63

3.1.1 Prologue 63

3.1.2 The Involvement of Lighting in the Complex Machine Vision Solution 63

3.2 Demands on Machine Vision lighting 67

3.3 Light used in Machine Vision 70

3.3.1 What is Light? Axioms of Light 70

3.3.2 Light and Light Perception 73

3.3.3 Light Sources for Machine Vision 76

3.3.3.1 Incandescent Lamps/Halogen Lamps 77

3.3.3.2 Metal Vapor Lamps 78

3.3.3.3 Xenon Lamps 79

3.3.3.4 Fluorescent Lamps 81

3.3.3.5 LEDs (Light Emitting Diodes) 82

3.3.3.6 Lasers 85

3.3.4 The Light Sources in Comparison 86

3.3.5 Considerations for Light Sources: Lifetime, Aging, Drift 86

3.3.5.1 Lifetime 86

3.3.5.2 Aging and Drift 88

3.4 Interaction of Test Object and Light 91

3.4.1 Risk Factor Test Object 91

3.4.1.1 What Does the Test Object do With the Incoming Light? 92

3.4.1.2 Reflection/Reflectance/Scattering 92

3.4.1.3 Total Reflection 95

3.4.1.4 Transmission/Transmittance 96

3.4.1.5 Absorption/Absorbance 97

3.4.1.6 Diffraction 99

3.4.1.7 Refraction 100

3.4.2 Light Color and Part Color 101

3.4.2.1 Visible Light (VIS) – Monochromatic Light 101

3.4.2.2 Visible Light (VIS) – White Light 103

3.4.2.3 Infrared Light (IR) 104

3.4.2.4 Ultraviolet (UV) Light 106

3.4.2.5 Polarized Light 107

3.5 Basic Rules and Laws of Light Distribution 109

3.5.1 Basic Physical Quantities of Light 110

3.5.2 The Photometric Inverse Square Law 111

3.5.3 The Constancy of Luminance 113

3.5.4 What Light Arrives at the Sensor – Light Transmission Through the Lens 114

3.5.5 Light Distribution of Lighting Components 115

3.5.6 Contrast 118

3.5.7 Exposure 120

3.6 Light Filters 121

3.6.1 Characteristic Values of Light Filters 121

3.6.2 Influences of Light Filters on the Optical Path 123

3.6.3 Types of Light Filters 124

3.6.4 Anti-Reflective Coatings (AR) 126

3.6.5 Light Filters for Machine Vision 127

3.6.5.1 UV Blocking Filter 127

3.6.5.2 Daylight Suppression Filter 128

3.6.5.3 IR Suppression Filter 128

3.6.5.4 Neutral Filter/Neutral Density Filter/Gray Filter 129

3.6.5.5 Polarization Filter 130

3.6.5.6 Color Filters 130

3.6.5.7 Filter Combinations 131

3.7 Lighting Techniques and Their Use 131

3.7.1 How to Find a Suitable Lighting? 131

3.7.2 Planning the Lighting Solution – Influence Factors 133

3.7.3 Lighting Systematics 135

3.7.3.1 Directional Properties of the Light 135

3.7.3.2 Arrangement of the Lighting 138

3.7.3.3 Properties of the Illuminated Field 138

3.7.4 The Lighting Techniques in Detail 140

3.7.4.1 Diffuse Bright Field Incident Light (No. 1, Table 3.14) 140

3.7.4.2 Directed Bright Field Incident Light (No. 2, Table 3.14) 142

3.7.4.3 Telecentric Bright Field Incident Light (No. 3, Table 3.14) 143

3.7.4.4 Structured Bright Field Incident Light (No. 4, Table 3.14) 145

3.7.4.5 Diffuse Directed Partial Bright Field Incident Light (Nos. 1 and 2, Table 3.14) 148

3.7.4.6 Diffuse/Directed Dark Field Incident Light (Nos. 5 and 6, Table 3.14) 152

3.7.4.7 The Limits of the Incident Lighting 154

3.7.4.8 Diffuse Bright Field Transmitted Lighting (No. 7, Table 3.14) 155

3.7.4.9 Directed Bright Field Transmitted Lighting (No. 8, Table 3.14) 157

3.7.4.10 Telecentric Bright Field Transmitted Lighting (No. 9, Table 3.14) 158

3.7.4.11 Diffuse/Directed Transmitted Dark Field Lighting (Nos. 10 and 11, Table 3.14) 161

3.7.5 Combined Lighting Techniques 162

3.8 Lighting Control 163

3.8.1 Reasons for Light Control – The Environmental Industrial Conditions 164

3.8.2 Electrical Control 164

3.8.2.1 Stable Operation 164

3.8.2.2 Brightness Control 166

3.8.2.3 Temporal Control: Static-Pulse-Flash 167

3.8.2.4 Some Considerations for the Use of Flash Light 168

3.8.2.5 Temporal and Local Control: Adaptive Lighting 171

3.8.3 Geometrical Control 173

3.8.3.1 Lighting from Large Distances 173

3.8.3.2 Light Deflection 175

3.8.4 Suppression of Ambient and Extraneous Light – Measures for a Stable Lighting 175

3.9 Lighting Perspectives for the Future 176

References 177

4 Optical Systems in Machine Vision 179
Karl Lenhardt

4.1 A Look at the Foundations of Geometrical Optics 179

4.1.1 From Electrodynamics to Light Rays 179

4.1.2 Basic Laws of Geometrical Optics 181

4.2 Gaussian Optics 183

4.2.1 Reflection and Refraction at the Boundary between two Media 183

4.2.2 Linearizing the Law of Refraction – The Paraxial Approximation 185

4.2.3 Basic Optical Conventions 186

4.2.3.1 Definitions for Image Orientations 186

4.2.3.2 Definition of the Magnification Ratio β 186

4.2.3.3 Real and Virtual Objects and Images 187

4.2.3.4 Tilt Rule for the Evaluation of Image Orientations by Reflection 188

4.2.4 Cardinal Elements of a Lens in Gaussian Optics 189

4.2.4.1 Focal Lengths f and f ′ 192

4.2.4.2 Convention 192

4.2.5 Thin Lens Approximation 193

4.2.6 Beam-Converging and Beam-Diverging Lenses 193

4.2.7 Graphical Image Constructions 195

4.2.7.1 Beam-Converging Lenses 195

4.2.7.2 Beam-Diverging Lenses 195

4.2.8 Imaging Equations and Their Related Coordinate Systems 195

4.2.8.1 Reciprocity Equation 196

4.2.8.2 Newton’s Equations 197

4.2.8.3 General Imaging Equation 198

4.2.8.4 Axial Magnification Ratio 200

4.2.9 Overlapping of Object and Image Space 200

4.2.10 Focal Length, Lateral Magnification, and the Field of View 200

4.2.11 Systems of Lenses 202

4.2.12 Consequences of the Finite Extension of Ray Pencils 205

4.2.12.1 Effects of Limitations of the Ray Pencils 205

4.2.12.2 Several Limiting Openings 207

4.2.12.3 Characterizing the Limits of Ray Pencils 210

4.2.12.4 Relation to the Linear Camera Model 212

4.2.13 Geometrical Depth of Field and Depth of Focus 214

4.2.13.1 Depth of Field as a Function of the Object Distance p 215

4.2.13.2 Depth of Field as a Function of β 216

4.2.13.3 Hyperfocal Distance 217

4.2.13.4 Permissible Size for the Circle of Confusion d ′ 218

4.2.14 Laws of Central Projection–Telecentric System 219

4.2.14.1 Introduction to the Laws of Perspective 219

4.2.14.2 Central Projection from Infinity – Telecentric Perspective 228

4.3 Wave Nature of Light 235

4.3.1 Introduction 235

4.3.2 Rayleigh–Sommerfeld Diffraction Integral 236

4.3.3 Further Approximations to the Huygens–Fresnel Principle 238

4.3.3.1 Fresnel’s Approximation 239

4.3.4 Impulse Response of an Aberration-Free Optical System 241

4.3.4.1 Case of Circular Aperture, Object Point on the Optical Axis 243

4.3.5 Intensity Distribution in the Neighborhood of the Geometrical Focus 244

4.3.5.1 Special Cases 246

4.3.6 Extension of the Point Spread Function in a Defocused Image Plane 248

4.3.7 Consequences for the Depth of Field Considerations 249

4.3.7.1 Diffraction and Permissible Circle of Confusion 249

4.3.7.2 Extension of the Point Spread Function at the Limits of the Depth of Focus 250

4.3.7.3 Useful Effective f -Number 251

4.4 Information Theoretical Treatment of Image Transfer and Storage 252

4.4.1 Physical Systems as Linear Invariant Filters 252

4.4.1.1 Invariant Linear Systems 255

4.4.1.2 Note to the Representation of Harmonic Waves 259

4.4.2 Optical Transfer Function (OTF) and the Meaning of Spatial Frequency 260

4.4.2.1 Note on the Relation Between the Elementary Functions in the Two Representation Domains 261

4.4.3 Extension to the Two-Dimensional Case 261

4.4.3.1 Interpretation of Spatial Frequency Components (r, s) 261

4.4.3.2 Reduction to One-Dimensional Representations 262

4.4.4 Impulse Response and MTF for Semiconductor Imaging Devices 265

4.4.5 Transmission Chain 267

4.4.6 Aliasing Effect and the Space-Variant Nature of Aliasing 267

4.4.6.1 Space-Variant Nature of Aliasing 274

4.5 Criteria for Image Quality 277

4.5.1 Gaussian Data 277

4.5.2 Overview on Aberrations of the Third Order 277

4.5.2.1 Monochromatic Aberrations of the Third Order (Seidel Aberrations) 278

4.5.2.2 Chromatic Aberrations 278

4.5.3 Image Quality in the Space Domain: PSF, LSF, ESF, and Distortion 278

4.5.3.1 Distortion 280

4.5.4 Image Quality in the Spatial Frequency Domain: MTF 281

4.5.4.1 Parameters that Influence the Modulation Transfer Function 282

4.5.5 Other Image Quality Parameters 283

4.5.5.1 Relative Illumination (Relative Irradiance) 283

4.5.5.2 Deviation from Telecentricity (for Telecentric Lenses only) 284

4.5.6 Manufacturing Tolerances and Image Quality 284

4.5.6.1 Measurement Errors due to Mechanical Inaccuracies of the Camera System 285

4.6 Practical Aspects: How to Specify Optics According to the Application Requirements? 285

4.6.1 Example for the Calculation of an Imaging Constellation 287

References 289

5 Camera Calibration 291
Robert Godding

5.1 Introduction 291

5.2 Terminology 292

5.2.1 Camera, Camera System 292

5.2.2 Coordinate Systems 292

5.2.3 Interior Orientation and Calibration 293

5.2.4 Exterior and Relative Orientation 293

5.2.5 System Calibration 293

5.3 Physical Effects 293

5.3.1 Optical System 293

5.3.2 Camera and Sensor Stability 294

5.3.3 Signal Processing and Transfer 294

5.4 Mathematical Calibration Model 295

5.4.1 Central Projection 295

5.4.2 Camera Model 295

5.4.3 Focal Length and Principal Point 297

5.4.4 Distortion and Affinity 297

5.4.5 Radial Symmetrical Distortion 297

5.4.6 Radial Asymmetrical and Tangential Distortion 299

5.4.7 Affinity and Nonorthogonality 299

5.4.8 Variant Camera Parameters 299

5.4.9 Sensor Flatness 301

5.4.10 Other Parameters 301

5.5 Calibration and Orientation Techniques 302

5.5.1 In the Laboratory 302

5.5.2 Using Bundle Adjustment to Determine Camera Parameters 302

5.5.2.1 Calibration Based Exclusively on Image Information 302

5.5.2.2 Calibration and Orientation with Additional Object Information 304

5.5.2.3 Extended System Calibration 307

5.5.3 Other Techniques 307

5.6 Verification of Calibration Results 308

5.7 Applications 309

5.7.1 Applications with Simultaneous Calibration 309

5.7.2 Applications with Precalibrated Cameras 311

5.7.2.1 Tube Measurement within a Measurement Cell 311

5.7.2.2 Online Measurements in the Field of Car Safety 312

5.7.2.3 High Resolution 3D Scanning with White Light Scanners 312

5.7.2.4 Other Applications 313

References 314

6 Camera Systems in Machine Vision 317
Horst Mattfeldt

6.1 Camera Technology 317

6.1.1 History in Brief 317

6.1.2 Machine Vision versus Closed Circuit TeleVision (CCTV) 317

6.2 Sensor Technologies 319

6.2.1 Spatial Differentiation: 1D and 2D 319

6.2.2 CCD Technology 320

6.2.2.1 Interline Transfer 321

6.2.2.2 Progressive Scan Interline Transfer 321

6.2.2.3 Interlaced Scan Readout 322

6.2.2.4 Enhancing Frame Rate by Multitap Sensors 324

6.2.2.5 SONY HAD Technology 325

6.2.2.6 SONY SuperHAD (II) and ExViewHAD (II) Technology 325

6.2.2.7 CCD Image Artifacts 326

6.2.2.8 Blooming 326

6.2.2.9 Smear 326

6.2.3 CMOS Image Sensor 328

6.2.3.1 Advantages of CMOS Sensor 328

6.2.3.2 CMOS Sensor Shutter Concepts 331

6.2.3.3 Performance Comparison of CMOS versus CCD 336

6.2.3.4 Integration Complexity of CCD versus CMOS Camera Technology 336

6.2.3.5 CMOS Sensor Sensitivity Enhancements 337

6.2.4 MATRIX VISION Available Cameras 338

6.2.4.1 Why So Many Different Models? How to Choose Among These? 338

6.2.4.2 Resolution and Video Standards 338

6.2.4.3 Sensor Sizes and Dimensions 344

6.3 Block Diagrams and Their Description 344

6.3.1 Block Diagram of SONY Progressive Scan Analog Camera 345

6.3.1.1 CCD Read Out Clocks 345

6.3.1.2 CCD Binning Mode 345

6.3.1.3 Spectral Sensitivity 348

6.3.1.4 Analog Signal Processing 348

6.3.1.5 Camera and Frame Grabber 350

6.3.2 Block Diagram of Color Camera with Digital Image Processing 350

6.3.2.1 Bayer TM Complementary Color Filter Array 351

6.3.2.2 Complementary Color Filters Spectral Sensitivity 351

6.3.2.3 Generation of Color Signals 351

6.4 mvBlueCOUGAR-X Line of Cameras 354

6.4.1 Black and White Digital Camera mvBlueCOUGAR-X Camera Series 355

6.4.1.1 Gray Level Sensor and Processing 355

6.4.2 Color Camera mvBlueCOUGAR-X Family 356

6.4.2.1 Analog Processing 356

6.4.2.2 Analog Front End (AFE) 357

6.4.2.3 A/D Conversion 357

6.4.2.4 One-Chip Color Processing 359

6.4.2.5 Inputting Time Stamp Data into Data Stream 361

6.4.2.6 Statistics Engine for White Balance and Auto Features 361

6.4.2.7 Image Memory 361

6.4.2.8 Lookup Table (LUT) and Gamma Function 362

6.4.2.9 Shading Correction 365

6.4.2.10 Reducing Noise by Adaptive Recursive Frame Averaging 366

6.4.2.11 Color Interpolation 367

6.4.2.12 Color Correction 368

6.4.2.13 RGB → YUV Conversion 370

6.4.3 Controlling Image Capture 371

6.4.4 Acquisition and Trigger Modes 371

6.4.4.1 Sequencer 374

6.4.4.2 Latency and Jitter Aspects 375

6.4.4.3 Action Commands 375

6.4.4.4 Scheduled Action Command 377

6.4.5 Data Transmission 377

6.4.5.1 GigE Vision and GVSP 378

6.4.5.2 USB3 Vision 380

6.4.6 Pixel Data 380

6.4.7 Camera Connection 381

6.4.8 Operating the Camera 381

6.4.9 HiRose Jack Pin Assignment 382

6.4.10 Sensor Frame Rates and Bandwidth 382

6.5 Configuration of a GigE Vision Camera 384

6.6 Qualifying Cameras and Noise Measurement (Dr. Gert Ferrano Mv) 386

6.6.1 Explanation of the Most Important Measurements 388

6.6.1.1 Linearity Curve 388

6.6.1.2 Photon Transfer Curve 388

6.7 Camera Noise (by Henning Haider AVT, Updated by Author) 391

6.7.1 Photon Noise 391

6.7.2 Dark Current Noise 391

6.7.3 Fixed Pattern Noise (FPN) 392

6.7.4 Photo Response Non Uniformity (PRNU) 392

6.7.5 Reset Noise 392

6.7.6 1/f Noise (Amplifier Noise) 392

6.7.7 Quantization Noise 392

6.7.8 Noise Floor 393

6.7.9 Dynamic Range 393

6.7.10 Signal to Noise Ratio 393

6.7.11 Example 1: SONY IMX-174 Sensor (mvBlueFOX3-2024) 394

6.7.12 Example 2: CMOSIS CMV2000 (mvBlueCOUGAR-X104) 394

6.8 Useful Links and Literature 394

6.9 Digital Interfaces 395

7 Smart Camera and Vision Systems Design 399
Howard D. Gray and Nate Holmes

7.1 Introduction to Vision System Design 399

7.2 Definitions 400

7.3 Smart Cameras 403

7.3.1 Applications 403

7.3.2 Component Parts 404

7.3.2.1 Processors 404

7.3.2.2 FPGA Processing 406

7.3.2.3 Memory and Storage 407

7.3.2.4 Operating Systems 408

7.3.2.5 Image Sensors 409

7.3.2.6 Inputs and Outputs 410

7.3.2.7 Other Interfaces 412

7.3.2.8 Timers and Counters 413

7.3.3 Programming and Configuring 413

7.3.3.1 Scripting 413

7.3.3.2 High-Level Languages 414

7.3.3.3 Third-Party Tools 416

7.3.4 Environment 416

7.3.4.1 Power Dissipation 416

7.3.4.2 Ingress Protection 417

7.4 Vision Sensors 418

7.4.1 Applications 419

7.4.2 Component Parts 420

7.4.3 Programming and Configuring 420

7.4.4 Environment 421

7.5 Embedded Vision Systems 421

7.5.1 Applications 424

7.5.1.1 Multi-Camera Applications 424

7.5.1.2 Closed Loop Control Applications 424

7.5.2 Component Parts 425

7.5.3 Programming and Configuring 425

7.5.4 Environment 425

7.6 Conclusion 425

References 426

Further Reading 429

8 Camera Computer Interfaces 431
Nate Holmes

8.1 Overview 431

8.2 Camera Buses 432

8.2.1 Software Standards 433

8.2.1.1 GenICam 433

8.2.1.2 Iidc 2 434

8.2.2 Analog Camera Buses (Legacy) 435

8.2.2.1 Analog Video Signal 436

8.2.2.2 Interlaced Video 436

8.2.2.3 Progressive Scan Video 436

8.2.2.4 Timing Signals 437

8.2.2.5 Analog Image Acquisition 437

8.2.2.6 S-Video 438

8.2.2.7 Rgb 438

8.2.2.8 Analog Connectors 439

8.2.3 Parallel Digital Camera Buses (Legacy) 439

8.2.3.1 Digital Video Transmission 439

8.2.3.2 Taps 440

8.2.3.3 Differential Signaling 441

8.2.3.4 Line Scan 441

8.2.3.5 Parallel Digital Connectors 441

8.2.4 IEEE 1394 (FireWire) (Legacy) 442

8.2.4.1 IEEE 1394 for Machine Vision 445

8.2.5 Camera Link 449

8.2.5.1 Camera Link Signals 450

8.2.5.2 Camera Link Connectors 451

8.2.6 Camera Link HS 451

8.2.7 CoaXPress 452

8.2.8 USB (USB3 Vision) 452

8.2.8.1 USB for Machine Vision 454

8.2.9 Gigabit Ethernet (GigE Vision) 455

8.2.9.1 Gigabit Ethernet for Machine Vision 456

8.2.9.2 GigE Vision Device Discovery 456

8.2.9.3 GigE Vision Control Protocol (GVCP) 456

8.2.9.4 GenICam 457

8.2.9.5 GigE Vision Stream Protocol (GVSP) 457

8.2.9.6 Packet Loss and Resends 457

8.2.10 Future Standards Development 458

8.3 Choosing a Camera Bus 459

8.3.1 Bandwidth 459

8.3.2 Resolution 459

8.3.3 Frame Rate 460

8.3.4 Cables 460

8.3.5 Line Scan 460

8.3.6 Reliability 460

8.3.7 Summary of Camera Bus Specifications 461

8.3.8 Sample Use Cases 461

8.3.8.1 Manufacturing Inspection 461

8.3.8.2 LCD Inspection 462

8.3.8.3 Security 463

8.4 Computer Buses 463

8.4.1 Isa/eisa 463

8.4.2 PCI/CompactPCI/PXI 464

8.4.3 Pci-x 466

8.4.4 PCI Express/CompactPCI Express/PXI Express 467

8.4.5 Throughput 469

8.4.6 Prevalence and Lifetime 471

8.4.6.1 Cost 471

8.5 Choosing a Computer Bus 471

8.5.1 Determine Throughput Requirements 471

8.5.2 Applying the Throughput Requirements 473

8.6 Driver Software 473

8.6.1 Application Programming Interface 475

8.6.2 Supported Platforms 477

8.6.3 Performance 477

8.6.4 Utility Functions 478

8.6.5 Acquisition Mode 479

8.6.5.1 Snap 479

8.6.5.2 Grab 479

8.6.5.3 Sequence 480

8.6.5.4 Ring 481

8.6.6 Image Representation 482

8.6.6.1 Image Representation in Memory 482

8.6.7 Bayer Color Encoding 485

8.6.7.1 Image Representation on Disk 487

8.6.8 Image Display 487

8.6.8.1 Understanding Display Modes 488

8.6.8.2 Palettes 489

8.6.8.3 Nondestructive Overlays 490

8.7 Features of a Machine Vision System 491

8.7.1 Image Reconstruction 491

8.7.2 Timing and Triggering 492

8.7.3 Memory Handling 494

8.7.4 Additional Features 496

8.7.4.1 Look-Up Tables 497

8.7.4.2 Region of Interest 499

8.7.4.3 Color Space Conversion 499

8.7.4.4 Shading Correction 501

8.8 Summary 501

References 502

9 Machine Vision Algorithms 505
Carsten Steger

9.1 Fundamental Data Structures 505

9.1.1 Images 505

9.1.2 Regions 506

9.1.3 Subpixel-Precise Contours 508

9.2 Image Enhancement 509

9.2.1 Gray Value Transformations 509

9.2.2 Radiometric Calibration 512

9.2.3 Image Smoothing 517

9.2.4 Fourier Transform 528

9.3 Geometric Transformations 532

9.3.1 Affine Transformations 532

9.3.2 Projective Transformations 533

9.3.3 Image Transformations 534

9.3.4 Polar Transformations 538

9.4 Image Segmentation 540

9.4.1 Thresholding 540

9.4.2 Extraction of Connected Components 548

9.4.3 Subpixel-Precise Thresholding 550

9.5 Feature Extraction 552

9.5.1 Region Features 552

9.5.2 Gray Value Features 556

9.5.3 Contour Features 559

9.6 Morphology 560

9.6.1 Region Morphology 561

9.6.2 Gray Value Morphology 575

9.7 Edge Extraction 579

9.7.1 Definition of Edges in One and Two Dimensions 579

9.7.2 1D Edge Extraction 583

9.7.3 2D Edge Extraction 589

9.7.4 Accuracy of Edges 596

9.8 Segmentation and Fitting of Geometric Primitives 602

9.8.1 Fitting Lines 603

9.8.2 Fitting Circles 607

9.8.3 Fitting Ellipses 608

9.8.4 Segmentation of Contours into Lines, Circles, and Ellipses 609

9.9 Camera Calibration 613

9.9.1 Camera Models for Area Scan Cameras 614

9.9.2 Camera Model for Line Scan Cameras 618

9.9.3 Calibration Process 622

9.9.4 World Coordinates from Single Images 626

9.9.5 Accuracy of the Camera Parameters 629

9.10 Stereo Reconstruction 631

9.10.1 Stereo Geometry 632

9.10.2 Stereo Matching 639

9.11 Template Matching 643

9.11.1 Gray-Value-Based Template Matching 644

9.11.2 Matching Using Image Pyramids 649

9.11.3 Subpixel-Accurate Gray-Value-Based Matching 652

9.11.4 Template Matching with Rotations and Scalings 653

9.11.5 Robust Template Matching 654

9.12 Optical Character Recognition 672

9.12.1 Character Segmentation 672

9.12.2 Feature Extraction 674

9.12.3 Classification 676

References 690

10 Machine Vision in Manufacturing 699
Peter Waszkewitz

10.1 Introduction 699

10.1.1 The Machine Vision Market 699

10.2 Application Categories 701

10.2.1 Types of Tasks 701

10.2.2 Types of Production 703

10.2.2.1 Discrete Unit Production Versus Continuous Flow 703

10.2.2.2 Job-Shop Production Versus Mass Production 704

10.2.3 Types of Evaluations 704

10.2.4 Value-Adding Machine Vision 705

10.3 System Categories 706

10.3.1 Common Types of Systems 707

10.3.2 Sensors 707

10.3.3 Vision Sensors 708

10.3.4 Compact Systems 709

10.3.5 Vision Controllers 710

10.3.6 PC-Based Systems 710

10.3.6.1 Library-Based Systems 711

10.3.6.2 Application-Package-Based Systems 712

10.3.6.3 Library-Based Application Packages 713

10.3.7 Excursion: Embedded Image Processing 713

10.3.8 Summary 714

10.4 Integration and Interfaces 715

10.4.1 Standardization 715

10.4.2 Interfaces 716

10.5 Mechanical Interfaces 716

10.5.1 Dimensions and Fixation 717

10.5.2 Working Distances 718

10.5.3 Position Tolerances 718

10.5.4 Forced Constraints 719

10.5.5 Additional Sensor Requirements 719

10.5.6 Additional Motion Requirements 720

10.5.7 Environmental Conditions 721

10.5.8 Reproducibility 722

10.5.9 Gauge Capability 723

10.6 Electrical Interfaces 725

10.6.1 Wiring and Movement 726

10.6.2 Power Supply 726

10.6.3 Internal Data Connections 727

10.6.4 External Data Connections 729

10.7 Information Interfaces 729

10.7.1 Interfaces and Standardization 730

10.7.2 Traceability 730

10.7.3 Types of Data and Data Transport 731

10.7.4 Control Signals 731

10.7.5 Result and Parameter Data 732

10.7.6 Mass Data 733

10.7.7 Digital I/O 733

10.7.8 Field Bus 733

10.7.9 Serial Interfaces 734

10.7.10 Network 734

10.7.10.1 Standard Ethernet–TCP/IP 734

10.7.10.2 OPC UA and Industry 4.0 735

10.7.10.3 Ethernet-Based Field Bus/Real-Time Ethernet 735

10.7.11 Files 736

10.7.12 Time and Integrity Considerations 736

10.8 Temporal Interfaces 738

10.8.1 Discrete Motion Production 738

10.8.2 Continuous Motion Production 740

10.8.3 Line-Scan Processing 743

10.9 Human–Machine Interfaces 745

10.9.1 Interfaces for Engineering Vision Systems 746

10.9.2 Runtime Interface 747

10.9.2.1 Using the PLC HMI for Machine Vision 749

10.9.3 Remote Maintenance 750

10.9.3.1 Safety Precaution: No Movements 751

10.9.4 Offline Setup 751

10.10 3D Systems 753

10.10.1 Dimensionality and Representation 753

10.10.1.1 Dimensionality 753

10.10.1.2 2.5D and 3D 754

10.10.1.3 Point Clouds and Registration 755

10.10.1.4 Representation 757

10.10.2 3D Data Acquisition 757

10.10.2.1 Passive Methods 758

10.10.2.2 Active Methods 759

10.10.3 Applications 764

10.10.3.1 Identification 765

10.10.3.2 Completeness Check 765

10.10.3.3 Object and Pose Recognition 766

10.10.3.4 Shape and Dimension Applications 767

10.10.3.5 Surface Inspection 769

10.10.3.6 Robotics 770

10.10.4 Conclusion 771

10.11 Industrial Case Studies 772

10.11.1 Glue Check Under UV Light 772

10.11.1.1 Task 772

10.11.1.2 Solution 773

10.11.1.3 Equipment 773

10.11.1.4 Algorithms 774

10.11.1.5 Key Points 774

10.11.2 Completeness Check 774

10.11.2.1 Task 774

10.11.2.2 Solution 774

10.11.2.3 Key Point: Mechanical Setup 775

10.11.2.4 Equipment 775

10.11.2.5 Algorithms 775

10.11.3 Multiple Position and Completeness Check 776

10.11.3.1 Task 776

10.11.3.2 Solution 776

10.11.3.3 Key Point: Cycle Time 778

10.11.3.4 Equipment 778

10.11.3.5 Algorithms 779

10.11.4 Pin-Type Verification 779

10.11.4.1 Task 779

10.11.4.2 Solution 779

10.11.4.3 Key Point: Self-Test 781

10.11.4.4 Equipment 781

10.11.4.5 Algorithms 781

10.11.5 Robot Guidance 781

10.11.5.1 Task 781

10.11.5.2 Solution 782

10.11.5.3 Key Point: Calibration 782

10.11.5.4 Key Point: Communication 783

10.11.5.5 Equipment 784

10.11.5.6 Algorithms 784

10.11.6 Type and Result Data Management 784

10.11.6.1 Task 784

10.11.6.2 Solution 785

10.11.6.3 Key Point: Type Data 785

10.11.6.4 Key Point: Result Data 785

10.11.6.5 Equipment 786

10.11.7 Dimensional Check for Process Control 786

10.11.7.1 Task 786

10.11.7.2 Solution 787

10.11.7.3 Equipment 787

10.11.7.4 Algorithms 788

10.11.8 Ceramic Surface Check 788

10.11.8.1 Task 788

10.11.8.2 Solution 788

10.11.8.3 Equipment 789

10.12 Constraints and Conditions 789

10.12.1 Inspection Task Requirements 789

10.12.2 Circumstantial Requirements 790

10.12.2.1 Cost 791

10.12.2.2 Automation Environment 791

10.12.2.3 Organizational Environment 792

10.12.3 Refinements 793

10.12.4 Limits and Prospects 794

References 796

Appendix 801

Index 805