Augmented Reality Principles and Practice Usability Series

Today?s Comprehensive and Authoritative Guide to Augmented Reality

By overlaying computer-generated information on the real world, augmented reality (AR) amplifies human perception and cognition in remarkable ways. Working in this fast-growing field requires knowledge of multiple disciplines, including computer vision, computer graphics, and human-computer interaction. Augmented Reality: Principles and Practice integrates all this knowledge into a single-source reference, presenting today?s most significant work with scrupulous accuracy. Pioneering researchers Dieter Schmalstieg and Tobias Höllerer carefully balance principles and practice, illuminating AR from technical, methodological, and user perspectives.

Coverage includes

• Displays: head-mounted, handheld, projective, auditory, and haptic
• Tracking/sensing, including physical principles, sensor fusion, and real-time computer vision
• Calibration/registration, ensuring repeatable, accurate, coherent behavior
• Seamless blending of real and virtual objects
• Visualization to enhance intuitive understanding
• Interaction?from situated browsing to full 3D interaction
• Modeling new geometric content
• Authoring AR presentations and databases
• Architecting AR systems with real-time, multimedia, and distributed elements

This guide is indispensable for anyone interested in AR, including developers, engineers, students, instructors, researchers, and serious hobbyists.

Preface xix

Acknowledgments xxv

About the Authors xxvii

Chapter 1: Introduction to Augmented Reality 1

Definition and Scope 3

A Brief History of Augmented Reality 4

Examples 13

Related Fields 28

Summary 31

Chapter 2: Displays 33

Multimodal Displays 34

Visual Perception 39

Requirements and Characteristics 40

Spatial Display Model 56

Visual Displays 58

Summary 84

Chapter 3: Tracking 85

Tracking, Calibration, and Registration 86

Coordinate Systems 87

Characteristics of Tracking Technology 90

Stationary Tracking Systems 96

Mobile Sensors 99

Optical Tracking 105

Sensor Fusion 117

Summary 120

Chapter 4: Computer Vision for Augmented Reality 121

Marker Tracking 123

Multiple-Camera Infrared Tracking 132

Natural Feature Tracking by Detection 138

Incremental Tracking 149

Simultaneous Localization and Mapping 156

Outdoor Tracking 164

Summary 176

Chapter 5: Calibration and Registration 179

Camera Calibration 180

Display Calibration 183

Registration 190

Summary 194

Chapter 6: Visual Coherence 195

Registration 196

Occlusion 199

Photometric Registration 205

Common Illumination 216

Diminished Reality 227

Camera Simulation 231

Stylized Augmented Reality 236

Summary 237

Chapter 7: Situated Visualization 239

Challenges 241

Visualization Registration 245

Annotations and Labeling 248

X-Ray Visualization 254

Spatial Manipulation 260

Information Filtering 265

Summary 270

Chapter 8: Interaction 271

Output Modalities 272

Input Modalities 279

Tangible Interfaces 286

Virtual User Interfaces on Real Surfaces 294

Augmented Paper 295

Multi-view Interfaces 297

Haptic Interaction 304

Multimodal Interaction 304

Conversational Agents 306

Summary 309

Chapter 9: Modeling and Annotation 311

Specifying Geometry 312

Specifying Appearance 317

Semi-automatic Reconstruction 319

Free-Form Modeling 322

Annotation 325

Summary 328

Chapter 10: Authoring 329

Requirements of AR Authoring 330

Elements of Authoring 333

Stand-Alone Authoring Solutions 335

Plug-In Approaches 339

Web Technology 341

Summary 342

Chapter 11: Navigation 345

Foundations of Human Navigation 346

Exploration and Discovery 347

Route Visualization 347

Viewpoint Guidance 350

Multiple Perspectives 354

Summary 360

Chapter 12: Collaboration 361

Properties of Collaboration Systems 362

Co-located Collaboration 364

Remote Collaboration 370

Summary 377

Chapter 13: Software Architectures 379

AR Application Requirements 380

Software Engineering Requirements 382

Distributed Object Systems 385

Dataflow 389

Scene Graphs 395

Developer Support 400

Summary 407

Chapter 14: The Future 409

What May Drive Business Cases 410

An AR Developer’s Wish List 411

Taking AR Outdoors 415

Interfacing with Smart Objects 417

Confluence of Virtual Reality and Augmented Reality 418

Augmented Humans 419

AR as a Dramatic Medium 420

AR as a Social Computing Platform 421

Summary 422

References 423

Index 473

Dieter Schmalstieg is professor and head of the Institute for Computer Graphics and Vision at Graz University of Technology (TUG), Austria. His current research interests are augmented reality, virtual reality, real-time graphics, 3D user interfaces, and visualization. He received Dipl.-Ing. (1993), Dr. techn. (1997) and Habilitation (2001) degrees from Vienna University of Technology. He is author and coauthor of more than two hundred peer-reviewed scientific publications, associate editor of IEEE Transactions on Visualization and Computer Graphics, member of the editorial advisory board of computers and graphics and of Springer Virtual Reality, member of the steering committee of the IEEE International Symposium on Mixed and Augmented Reality, chair of the EUROGRAPHICS working group on Virtual Environments (1999-2010), advisor of the K-Plus Competence Center for Virtual Reality and Visualization in Vienna, and member of the Austrian Academy of Science. In 2002, he received the START career award presented by the Austrian Science Fund. In 2012, he received the IEEE Virtual Reality Technical Achievement Award for seminal contributions to the field of augmented reality. Since 2008, he is also director of the Christian Doppler Laboratory for Handheld Augmented Reality.

Tobias Höllerer is professor of computer science at the University of California, Santa Barbara, where he leads the Four Eyes Laboratory, conducting research in the four I's of Imaging, Interaction, and Innovative Interfaces. Dr. Höllerer holds a Diplom in informatics from the Technical University of Berlin, as well as an M.S. and Ph.D. in computer science from Columbia University. He is a recipient of the U.S. National Science Foundation's CAREER award for his work on "Anywhere Augmentation." enabling mobile computer users to place annotations in 3D space wherever they go. In 2013, he was named an ACM Distinguished Scientist. Dr. Höllerer is author of more th

• The first comprehensive, coherent, and authoritative guide to Augmented Reality: integrates key advances in all areas of the field
• An indispensable single-source reference for students
• Lead author Dieter Schmalstieg is a leading AR pioneer: this book reflects both his deep grasp of AR theory and his longtime work with industry
• Includes numerous case studies plus a hands-on guide to Android AR development with Qualcomm QCAR