3D Video From Capture to Diffusion

The chapters in this book cover a large spectrum of areas connected to 3D video, which are presented both theoretically and technologically, while taking into account both physiological and perceptual aspects. Stepping away from traditional 3D vision, the authors, all currently involved in these areas, provide the necessary elements for understanding the underlying computer-based science of these technologies. They consider applications and perspectives previously unexplored due to technological limitations.

This book guides the reader through the production process of 3D videos; from acquisition, through data treatment and representation, to 3D diffusion. Several types of camera systems are considered (multiscopic or multiview) which lead to different acquisition, modeling and storage-rendering solutions. The application of these systems is also discussed to illustrate varying performance benefits, making this book suitable for students, academics, and also those involved in the film industry.

Foreword xv

Notations xix

Acknowledgments xxiii

Introduction xxv
Laurent LUCAS, Celine LOSCOS and Yannick REMION

PART 1. 3D ACQUISITION OF SCENES 1

Chapter 1. Foundation 3
Laurent LUCAS, Yannick REMION and Celine LOSCOS

1.1. Introduction 3

1.2. A short history 5

1.3. Stereopsis and 3D physiological aspects 14

1.4. 3D computer vision 17

1.5. Conclusion 20

1.6. Bibliography 20

Chapter 2. Digital Cameras: Definitions and Principles 23
Min H. KIM, Nicolas HAUTIERE and Celine LOSCOS

2.1. Introduction 23

2.2. Capturing light: physical fundamentals 24

2.3. Digital camera 28

2.4. Cameras, human vision and color 33

2.5. Improving current performance 35

2.6. Conclusion 38

2.7. Bibliography 38

Chapter 3. Multiview Acquisition Systems 43
Frederic DEVERNAY, Yves PUPULIN and Yannick REMION

3.1. Introduction: what is a multiview acquisition system? 43

3.2. Binocular systems 45

3.3. Lateral or directional multiview systems 54

3.4. Global or omnidirectional multiview systems 61

3.5. Conclusion 66

3.6. Bibliography 66

Chapter 4. Shooting and Viewing Geometries in 3DTV 71
Jessica PREVOTEAU, Laurent LUCAS and Yannick REMION

4.1. Introduction 71

4.2. The geometry of 3D viewing 72

4.3. The geometry of 3D shooting 75

4.4. Geometric impact of the 3D workflow 80

4.5. Specification methodology for multiscopic shooting 84

4.6. OpenGL implementation 86

4.7. Conclusion 87

4.8. Bibliography 88

Chapter 5. Camera Calibration: Geometric and Colorimetric Correction 91
Vincent NOZICK and Jean-Baptiste THOMAS

5.1. Introduction 91

5.2. Camera calibration 91

5.3. Radial distortion 95

5.4. Image rectification 98

5.5. Colorimetric considerations in cameras 103

5.6. Conclusion 109

5.7. Bibliography 110

PART 2. DESCRIPTION/RECONSTRUCTION OF 3D SCENES 113

Chapter 6. Feature Points Detection and Image Matching 115
Michel DESVIGNES, Lara YOUNES and Barbara ROMANIUK

6.1. Introduction 115

6.2. Feature points 116

6.3. Feature point descriptors 123

6.4. Image matching 128

6.5. Conclusion 131

6.6. Bibliography 132

Chapter 7. Multi- and Stereoscopic Matching, Depth and Disparity 137
Stephanie PREVOST, Cedric NIQUIN, Sylvie CHAMBON and Guillaume GALES

7.1. Introduction 137

7.2. Difficulties, primitives and stereoscopic matching 138

7.3. Simplified geometry and disparity 139

7.4. A description of stereoscopic and multiscopic methods 141

7.5. Methods for explicitly accounting for occlusions 147

7.6. Conclusion 153

7.7. Bibliography 154

Chapter 8. 3D Scene Reconstruction and Structuring 157
Ludovic BLACHE, Muhannad ISMAEL and Philippe SOUCHET

8.1. Problems and challenges 157

8.2. Silhouette-based reconstruction 158

8.3. Industrial application 162

8.4. Temporally structuring reconstructions 166

8.5. Conclusion 170

8.6. Bibliography 171

Chapter 9. Synthesizing Intermediary Viewpoints 173
Luce MORIN, Olivier LE MEUR, Christine GUILLEMOT, Vincent JANTET and Josselin GAUTIER

9.1. Introduction 173

9.2. Viewpoint synthesis by interpolation and extrapolation 173

9.3. Inpainting uncovered zones 181

9.4. Conclusion 189

9.5. Bibliography 189

PART 3. STANDARDS AND COMPRESSION OF 3D VIDEO 193

Chapter 10. Multiview Video Coding (MVC) 195
Benjamin BATTIN, Philippe VAUTROT, Marco CAGNAZZO and Frederic DUFAUX

10.1. Introduction 195

10.2. Specific approaches to stereoscopy 196

10.3. Multiview approaches 202

10.4. Conclusion 207

10.5. Bibliography 208

Chapter 11. 3D Mesh Compression 211
Florent DUPONT, Guillaume LAVOUE and Marc ANTONINI

11.1. Introduction 211

11.2. Compression basics: rate-distortion trade-off 212

11.3. Multiresolution coding of surface meshes 213

11.4. Topological and progressive coding 215

11.5. Mesh sequence compression 218

11.6. Quality evaluation: classic and perceptual metrics 221

11.7. Conclusion 223

11.8. Bibliography 224

Chapter 12. Coding Methods for Depth Videos 229
Elie Gabriel MORA, Joel JUNG, Beatrice PESQUET-POPESCU and Marco CAGNAZZO

12.1. Introduction 229

12.2. Analyzing the characteristics of a depth map 231

12.3. Depth coding methods 232

12.4. Conclusion 245

12.5. Bibliography 245

Chapter 13. StereoscopicWatermarking 249
Mihai MITREA, Afef CHAMMEM and Francoise PRETEUX

13.1. Introduction 249

13.2. Constraints of stereoscopic video watermarking 250

13.3. State of the art for stereoscopic content watermarking 255

13.4. Comparative study 259

13.5. Conclusions 267

13.6. Bibliography 268

PART 4. RENDERING AND 3D DISPLAY 271

Chapter 14. HD 3DTV and Autostereoscopy 273
Venceslas BIRI and Laurent LUCAS

14.1. Introduction 273

14.2. Technological principles 275

14.3. Design of mixing filters 280

14.4. View generation and interleaving 282

14.5. Future developments 285

14.6. Conclusion 286

14.7. Bibliography 287

Chapter 15. Augmented and/or Mixed Reality 291
Gilles SIMON and Marie-Odile BERGER

15.1. Introduction 291

15.2. Real-time pose computation 292

15.3. Model acquisition 299

15.4. Conclusion 304

15.5. Bibliography 305

Chapter 16. Visual Comfort and Fatigue in Stereoscopy 309
Matthieu URVOY, Marcus BARKOWSKY, Jing LI and Patrick LE CALLET

16.1. Introduction 309

16.2. Visual comfort and fatigue: definitions and indications 310

16.3. Signs and symptoms of fatigue and discomfort 312

16.4. Sources of visual fatigue and discomfort 315

16.5. Application to 3D content and technologies 321

16.6. Predicting visual fatigue and discomfort: first models 324

16.7. Conclusion 324

16.8. Bibliography 325

Chapter 17. 2D–3D Conversion 331
David GROGNA, Antoine LEJEUNE and Benoit MICHEL

17.1. Introduction 331

17.2. The 2D–3D conversion workflow 332

17.3. Preparing content for conversion 333

17.4. Conversion stages 337

17.5. 3D–3D conversion 343

17.6. Conclusion 343

17.7. Bibliography 344

PART 5. IMPLEMENTATION AND OUTLETS 347

Chapter 18. 3D Model Retrieval 349
Jean-Philippe VANDEBORRE, Hedi TABIA and Mohamed DAOUDI

18.1. Introduction 349

18.2. General principles of shape retrieval 350

18.3. Global 3D shape descriptors 352

18.4. 2D view oriented methods 353

18.5. Local 3D shape descriptors 353

18.6. Similarity between 3D shapes 356

18.7. Shape recognition in 3D video 359

18.8. Evaluation of the performance of indexing methods 361

18.9. Applications 363

18.10. Conclusion 366

18.11. Bibliography 366

Chapter 19. 3D HDR Images and Videos: Acquisition and Restitution 369
Jennifer BONNARD, Gilles VALETTE, Celine LOSCOS and Jean-Michel NOURRIT

19.1. Introduction 369

19.2. HDR and 3D acquisition 370

19.3. 3D HDR restitution 380

19.4. Conclusion 382

19.5. Bibliography 383

Chapter 20. 3D Visualization for Life Sciences 387
Aassif BENASSAROU, Sylvia PIOTIN, Manuel DAUCHEZ and Dimitri PAPATHANASSIOU

20.1. Introduction 387

20.2. Scientific visualization 387

20.3. Medical imaging 390

20.4. Molecular modeling 397

20.5. Conclusion 401

20.6. Bibliography 402

Chapter 21. 3D Reconstruction of Sport Scenes 405
Sebastien MAVROMATIS and Jean SEQUEIRA

21.1. Introduction 405

21.2. Automatic selection of a region of interest (ROI) 406

21.3. The Hough transform 410

21.4. Matching image features to the geometric model 412

21.5. Conclusion 415

21.6. Bibliography 417

Chapter 22. Experiments in Live Capture and Transmission of Stereoscopic 3D Video Images 421
David GROGNA and Jacques G.VERLY

22.1. Introduction 421

22.2. Retransmissions of various shows 422

22.3. Retransmissions of surgical operations 423

22.4. Retransmissions of “steadicam” interviews 428

22.5. Retransmission of a transatlantic video presentation 433

22.6. Retransmissions of bicycle races 435

22.7. Conclusion 437

22.8. Bibliography 439

Conclusion 441
Laurent LUCAS, Celine LOSCOS and Yannick REMION

List of Authors 443

Index 447

Céline Loscos is Professor at the URCA, within the CReSTIC laboratory, and teaches computer science at the University Institute of Technology (IUT) in Champagne Ardenne, France.

Yannick Remion's research interests include dynamic animation, simulation and co-operation between image processing and computer graphics as well as 3D vision.