Color Constancy The Wiley-IS&T Series in Imaging Science and Technology Series

This book provides a comprehensive introduction to the field of color constancy, describing all the major color constancy algorithms, as well as presenting cutting edge research in the area of color image processing. Beginning with an in-depth look at the human visual system, Ebner goes on to:

• examine the theory of color image formation, color reproduction and different color spaces;
• discuss algorithms for color constancy under both uniform and non-uniform illuminants;
• describe methods for shadow removal and shadow attenuation in digital images;
• evaluate the various algorithms for object recognition and color constancy and compare this to data obtained from experimental psychology;
• set out the different algorithms as pseudo code in an appendix at the end of the book.

Color Constancy is an ideal reference for practising engineers, computer scientists and researchers working in the area of digital color image processing. It may also be useful for biologists or scientists in general who are interested in computational theories of the visual brain and bio-inspired engineering systems.

Series Preface xi

Preface xiii

1 Introduction 1

1.1 What is Color Constancy? 1

1.2 Classic Experiments 3

1.3 Overview 7

2 The Visual System 9

2.1 Eye and Retina 9

2.2 Visual Cortex 16

2.3 On the Function of the Color Opponent Cells 30

2.4 Lightness 31

2.5 Color Perception Correlates with Integrated Reflectances 32

2.6 Involvement of the Visual Cortex in Color Constancy 35

3 Theory of Color Image Formation 39

3.1 Analog Photography 41

3.2 Digital Photography 46

3.3 Theory of Radiometry 47

3.4 Reflectance Models 52

3.5 Illuminants 56

3.6 Sensor Response 60

3.7 Finite Set of Basis Functions 63

4 Color Reproduction 67

4.1 Additive and Subtractive Color Generation 68

4.2 Color Gamut 69

4.3 Computing Primary Intensities 69

4.4 CIE XYZ Color Space 70

4.5 Gamma Correction 79

4.6 Von Kries Coefficients and Sensor Sharpening 83

5 Color Spaces 87

5.1 RGB Color Space 87

5.2 sRGB 87

5.3 CIE L∗u∗v∗Color Space 89

5.4 CIE L∗a∗b∗Color Space 92

5.5 CMY Color Space 93

5.6 HSI Color Space 93

5.7 HSV Color Space 96

5.8 Analog and Digital Video Color Spaces 99

6 Algorithms for Color Constancy under Uniform Illumination 103

6.1 White Patch Retinex 104

6.2 The Gray World Assumption 106

6.3 Variant of Horn’s Algorithm 113

6.4 Gamut-constraint Methods 115

6.5 Color in Perspective 121

6.6 Color Cluster Rotation 128

6.7 Comprehensive Color Normalization 129

6.8 Color Constancy Using a Dichromatic Reflection Model 134

7 Algorithms for Color Constancy under Nonuniform Illumination 143

7.1 The Retinex Theory of Color Vision 143

7.2 Computation of Lightness and Color 154

7.3 Hardware Implementation of Land’s Retinex Theory 166

7.4 Color Correction on Multiple Scales 169

7.5 Homomorphic Filtering 170

7.6 Intrinsic Images 175

7.7 Reflectance Images from Image Sequences 188

7.8 Additional Algorithms 190

8 Learning Color Constancy 193

8.1 Learning a Linear Filter 193

8.2 Learning Color Constancy Using Neural Networks 194

8.3 Evolving Color Constancy 198

8.4 Analysis of Chromatic Signals 204

8.5 Neural Architecture based on Double Opponent Cells 205

8.6 Neural Architecture Using Energy Minimization 209

9 Shadow Removal and Brightening 213

9.1 Shadow Removal Using Intrinsic Images 213

9.2 Shadow Brightening 215

10 Estimating the Illuminant Locally 219

10.1 Local Space Average Color 219

10.2 Computing Local Space Average Color on a Grid of Processing Elements 221

10.3 Implementation Using a Resistive Grid 230

10.4 Experimental Results 237

11 Using Local Space Average Color for Color Constancy 239

11.1 Scaling Input Values 239

11.2 Color Shifts 241

11.3 Normalized Color Shifts 246

11.4 Adjusting Saturation 249

11.5 Combining White Patch Retinex and the Gray World Assumption 251

12 Computing Anisotropic Local Space Average Color 255

12.1 Nonlinear Change of the Illuminant 255

12.2 The Line of Constant Illumination 257

12.3 Interpolation Methods 259

12.4 Evaluation of Interpolation Methods 262

12.5 Curved Line of Constant Illumination 265

12.6 Experimental Results 267

13 Evaluation of Algorithms 275

13.1 Histogram-based Object Recognition 275

13.2 Object Recognition under Changing Illumination 279

13.3 Evaluation on Object Recognition Tasks 282

13.4 Computation of Color Constant Descriptors 290

13.5 Comparison to Ground Truth Data 299

14 Agreement with Data from Experimental Psychology 303

14.1 Perceived Color of Gray Samples When Viewed under Colored Light 303

14.2 Theoretical Analysis of Color Constancy Algorithms 305

14.3 Theoretical Analysis of Algorithms Based on Local Space Average Color 312

14.4 Performance of Algorithms on Simulated Stimuli 316

14.5 Detailed Analysis of Color Shifts 319

14.6 Theoretical Models for Color Conversion 320

14.7 Human Color Constancy 324

15 Conclusion 327

Appendix A Dirac Delta Function 329

Appendix B Units of Radiometry and Photometry 331

Appendix C Sample Output from Algorithms 333

Appendix D Image Sets 339

Appendix E Program Code 349

Appendix F Parameter Settings 363

Bibliography 369

List of Symbols 381

Index 385

Permissions 391

MARC EBNER, Lecturer (Privatdozent), Universität Würzburg, Lehrstuhl für Informatik, Am Hubland, 97074 Würzburg, Germany

MARC EBNER, is currently a lecturer at the Department of Computer Science, Programming Languages and Programming Methodology, University of Würzburg, Germany. He has been at the university since 1999, recently having completed his habilitation dissertation, on which this book is based. He teaches courses on computer graphics and virtual reality and his research interests are in colour constancy, computer vision, self-reproducing programs, neutral networks, and evolutionary algorithms. Previous to this post, he has gained qualifications from Stuttgart University, New York University and Tubingen University. To date, he has authored 8 published journal articles, 29 refereed conference papers.