Virtual Reality Technology (3^rd Ed.)

Thorough overview of virtual reality technology, with equal coverage given to hardware, software, and applications

The Third Edition of the first comprehensive technical book on the subject of virtual reality,Virtual Reality Technology, provides updated and expanded coverage of VR technology, including where it originated, how it has evolved, and where it is going. Its primary objective is to be a complete, up-to-date textbook, as well as a source of information on a rapidly developing field of science and technology with broad societal impact.

The two highly qualified authors cover all of the latest innovations and applications that are making virtual reality more important than ever before. Unlike other books on the subject, the book also includes a chapter on Human Factors, which are very important in designing technology around the human user.

Virtual Reality Technology provides readers with a website consisting of a 2-part laboratory manual using the VRML (Virtual Reality Modeling Language) and Java 3D programming languages, which are freely available, leading to cost-effective laboratory setups at universities.

With comprehensive coverage of the subject, Virtual Reality Technology discusses sample topics such as:

• Input and output interfaces, including touch and force feedback, and computing architecture, with emphasis on the rendering pipeline and task distribution
• Object modeling, including physical and behavioral aspects, and programming for virtual reality in a general sense
• Human factors issues such as user performance and sensorial conflict aspects of VR, plus traditional and emerging VR applications
• Input devices, such as trackers, navigation, and gesture interfaces, covering specific technologies like magnetic, ultrasonic, optical, and hybrid inertial trackers

Virtual Reality Technology provides thorough and complete coverage of an in-demand sect of technology, making it a highly valuable resource for senior and graduate students in computer science, engineering, and science, along with a variety of professionals across many different industries, including but not limited to engineering, gaming, and marketing.

Book Table of Contents

Dedication

Foreword

Preface to the 3rd Edition

1.  Introduction

1.1 THE THREE I’S OF VIRTUAL REALITY

1.2 EARLY VIRTUAL REALITY

1.3 FIRST COMMERCIAL VIRTUAL REALITY

1.4 VIRTUAL REALITY AT THE TURN OF THE MILENIUM

1.5 VIRTUAL REALITY IN THE 21ST CENTURY

1.5 COMPONENTS OF CLASSICAL AND MODERN VIRTUAL REALITY SYSTEMS

1.6 REVIEW QUESTIONS

REFERENCES

2.  Input Devices: Trackers, Navigation, Gesture and Neural Interfaces

2.1 THREE-DIMENSIONAL POSITION TRACKERS

2.1.1 Tracker performance parameters

2.1.2  Electromagnetic trackers

Trackers using alternating current (AC) magnetic fields

Trackers using direct current (DC) magnetic fields

Comparison of AC and DC Magnetic Trackers Accuracy Degradation

2.1.3  Optical Trackers

Passive Camera-based Optical Trackers

Active Camera-based Optical Trackers

Time-of-Flight Optical Trackers

Consumer-Grade Optical Trackers

Inside-looking-out Optical Trackers

Eye Trackers for Head-Mounted Displays

2.1.4 Optical Tracker Accuracy Degradation

2.1.5  Hybrid Inertia Trackers

Hybrid inertial-optical trackers

2.1.6  Global Positioning System Trackers

2.2 NAVIGATION AND MANIPULATION INTERFACES

2.2.1  Desktop Navigation Interfaces

2.2.2 Hand-held Navigation and Manipulation Interfaces

The FlightStick3

Game controllers are navigation and manipulation interfaces

2.2.3 Locomotion interfaces

The Cybershoes Arcade

The Omni Treadmill

2.3    GESTURE INTERFACES

2.3.1   Sensing Gloves

The 5DT Ultra Glove Ultra 14

The CyberGlove II

The Cyberglove-HT

2.3.2 Natural hand tracking

Natural vs Sensing Glove-based Hand Tracking

2.4 NEURAL INTERFACES

2.4.1 Research-grade Brain-Computer Interfaces

2.4.2 Consumer-grade Brain-Computer Interfaces

2.5 CONCLUSIONS

2.6 REVIEW QUESTIONS

REFERENCES

3.  Output Devices: Graphics Displays

3.1 THE HUMAN VISION SYSTEM

3.2 GRAPHICS DISPLAY CHARACTERISRICS

3.3 DISPLAY TECHNOLOGIES

3.3.1 Displays using LCD technology

3.3.2 Displays using OLED technology 

3.4 PERSONAL GRAPHICS DISPLAYS

3.4.1 Smart phones as VR displays

Smart phones as input devices

Smart phone use in HMDs

3.4.2 Head-Mounted Displays

Fixed-resolution HMD optics

Foveated HMD optics

Foveated HMDs which reduce native resolution

Foveated HMDs which increase native resolution

Head-Mounted Displays weight and weight distribution

The FOVE 0 HMD

The Oculus Quest 2 HMD

The HTC VIVE Focus 3

The Pimax Vision 8K X HMD

3.4.3 Desk Supported Personal Displays

Autostereoscopic monitors

3D Monitors

3.5 LARGE VOLUME DISPLAYS

3.5.1   Liquid Crystal Tiled Large-Volume Displays

Tiled wall displays

Tiled CAVE displays

3.5.2 Projector-based Large-Volume Displays

Dome-type large volume displays

Tiled Display Scene Continuity

Geometrical continuity

Visual continuity

Temporal continuity

3.6 MICRO-LED WALLS AND HOLOGRAPHIC DISPLAYS

3.6.1 Micro-LED Walls

3.6.2 Holographic displays

3.7 CONCLUSIONS

3.8 REVIEW QUESTIONS

REFERENCES

4.  Output Devices: Three-Dimensional Sound,

     Haptic Displays, Olfactory Displays

4.1 THREE-DIMENSIONAL SOUND DISPLAYS

4.1.1   Localized vs. Non-Localized sound

4.1.2 The Human Auditory System

Azimuth Cues

Elevation Cues

Range cues

4.1.3 Head-Related Transfer Function

4.1.4 Generic Convolvotron Architecture

Spatialized sound for all-in-one HMDs

4.1.5 Speaker-based 3-D Sound

4.1.6   Wearable Sound Interfaces

4.2 HAPTIC DISPLAYS

4.2.1   The Human Haptic System

Human Haptic Sensing System

Human Sensory-Motor Control

4.2.2   Tactile Feedback Interfaces

The CyberTouch II glove

The HaptX DK2 touch feedback glove

The Tactsuit X whole body tactile feedback

Temperature Feedback Interfaces

4.2.3  Force Feedback Interfaces

The Touch X Arm

The Delta.3 Force Feedback Arm

The Armeo Power

SenseGlove Nova Force Feedback Glove

4.3 OLFACTORY DISPLAYS

4.3.1 The Human Olfactory System

4.3.2 Olfactory Systems for VR

Ambient Olfactory Displays

Air Cannon-based Olfactory Display

Wearable Olfactory Displays

4.4 CONCLUSIONS

4.5 REVIEW QUESTIONS

REFERENCES

5.  Computing Architectures for VR

5.1 THE RENDERING PIPELINE

5.1.1 The Graphics Rendering Pipeline

Traditional Graphics Rendering Pipeline

Modern Graphics Rendering Pipeline

A graphics pipeline example

Anti-aliasing

Graphics Pipeline Bottlenecks

Graphics Pipeline Optimization

5.1.2 The Haptic Rendering Pipeline

5.2 GAMING DESKTOP ARCHITECTURES

5.2.1 The 12th Generation Core CPU and Chip Set

5.2.2 Cooling of Gaming PCs

5.2.3 Graphics Accelerator Cards for PCs

The NVIDIA GTX 1080 Founder’s Edition graphics card

NVIDIA GeForce RTX 4090 graphics card

The NVIDIA GeForce RTX 4090 graphics card

5.3 GRAPHICS BENCHMARKS

5.3.2 The SPECviewperf 2020 benchmarks

5.3.3 A Frame Capture Analysis Tool for Virtual Reality

5.4 DISTRIBUTED VR ARCHITECTURES

5.4.1 Split Rendering

5.4.2 Co-located Rendering Pipelines

5.4.3 Multi-pipeline synchronization

Synchronization of haptic and graphics pipelines

Synchronization of co-located graphics pipelines

Pipeline synchronization using the NVIDIA Quadro Sync II card

5.4.4 Cloud Rendering

A Cloud Rendering Architecture Example

Cloud Rendering Latency

5.5 CONCLUSIONS

5.6 REVIEW QUESTIONS

REFERENCES

6.  Modeling Virtual Environments

6.1 GEOMETRIC MODELING

6.1.2 Virtual Object Shape

Using a 3D Authoring Software

Importing CAD files

Creating surfaces with 3D scanners

Using online 3D object databases

6.1.3 Object Visual Appearance

Scene Illumination

Texture Mapping

6.2 KINEMATICS MODELING

6.2.1 Homogeneous transformation matrices

6.2.2 Object Position

6.2.3 Transformation Invariants

6.2.4 Object hierarchies

6.2.5 Viewing the 3-D World

6.3 PHYSICAL MODELING

6.3.1 Collision Detection

Collision detection for large virtual environments

6.3.2 Collision Response Involving Object Surfaces

Topology-preserving collision response

Topology-altering collision response

Surface cutting and stitching

Object morphing

6.3.3 Contact Force Modeling

Contact Forces when Deforming Elastic Objects

Contact Forces when Deforming Plastic Objects

Contact Forces when Interacting with Virtual Walls

6.3.4 Force Smoothing and Mapping

6.3.5 Haptic Texturing

Haptic textures produced by non-wearable interfaces

Haptic textures produced by wearable interfaces

6.4 BEHAVIOR MODELING

6.4.1 Simple behavior models

6.4.2 Enhanced behavior models

6.4.4 Crowd behavior models

6.5 MODEL MANAGEMENT

6.5.1 Level-of-Detail Management

Discrete Level-of-Detail Management

Continuous Level-of-Detail Management

Adaptive Level-of-Detail Management Using Foveated Rendering

Adaptive Level-of-Detail Management Guaranteeing Frame Time

6.5.2 Cell Segmentation

Automatic Cell Segmentation

3-D Cell Segmentation

6.6 CONCLUSION

6.7 REVIEW QUESTIONS

REFERENCES

7.  Virtual Reality Programming

7.1 SCENE GRAPHS AND TOOLKITS

7.1.1 Scene Graphs

Internal scene graphs

Distributed scene graphs

7.1.2 Toolkits

7.2 JAVA3D

7.2.1 Java 3D Model Geometry and Appearance

7.2.2 Java3D Scene Graph

7.2.3 Java3D Sensors and Behaviors

7.2.4 Java3D Networking

7.3 THE VIZARD TOOLKIT

7.3.1 Vizard Model Geometry and Appearance

Model Geometry

Model appearance

7.3.2 Vizard Scene Graph

7.3.3 Vizard Sensors and Behaviors

Vizard Physics Engine

Vizard OpenHaptics Plug-in

7.3.4 Vizard Networking

7.4 THE OPENHAPTICS TOOLKIT

7.4.1   OpenHaptics Integration with the Graphics Pipeline

7.4.2   OpenHaptics QuickHaptics Micro API

7.4.3   OpenHaptics Haptic Device to Screen Mapping

7.4.5 OpenHaptics Unity Plugin

7.5 UNITY 3D GAME ENGINE

7.5.1 The Game Engine

7.5.2 Game Production Pipeline

The pre-production pipeline stage

The production pipeline stage

The post-production pipeline stage

7.5.3 Unity Game Programming

Creating a New Project in Unity

The Unity Editor

Unity Game Objects

Physics Programming in Unity

Scripting in Unity

7.5.4 Artificial Intelligence in Unity Gaming

Non-Player Characters, Finite State Machines and Machine Learning

Unity Implementation

7.6 CONCLUSION

7.7 REVIEW QUESTIONS

REFERENCES

8.  Human Factors in Virtual Reality

8.1 METHODOLOGY AND TECHNOLOGY

8.1.1 The Experimental Protocol

8.1.2 Institutional Review and Participant Consent

8.1.2 Data Collection and Analysis

Objective performance measures

Modality-specific objective performance measures

Task-specific objective performance measures

Subjective performance criteria

8.1.3 Usability Engineering Methodology

8.2 USER PERFORMANCE STUDIES

8.2.1 Testbed Evaluation of Universal VR Tasks

Influence of navigation and text display techniques on user performance

Influence of locomotion techniques on user performance

8.2.3 User Performance Dependency on Feedback Modality

User performance during object manipulation with substituted and redundant haptic feedback

Participant sense of presence during tasks involving sensorial substitution

Olfactory and haptic feedback contribution to participant quality of experience in VR

8.2.4 Sensorial Illusion

Illusion of weight

Illusion of Scent

8.3 HEALTH AND SAFETY ISSUES IN VIRTUAL ENVIRONMENTS

8.3.1 Direct Effects of Virtual Environments on Users

Direct effects on the Visual System

Direct Effects on the Auditory, Musculoskeletal and Olfactory Systems

8.3.2 Indirect Effects of Virtual Environments on Users

Neural conflict between inner ear, visual and proprioceptive sensorial cues

Ways to measure cybersickness severity

Influence of user’s characteristics

Influence of VR system characteristics

Temporal factors influencing cybersickness

Adaptation and Aftereffects

Guidelines for proper VR usage

8.4 SOCIETAL IMPLICATIONS OF VIRTUAL REALITY

8.4.1 Virtual Reality Impact on Professional Life

8.4.2 Virtual Reality Impact on Private Life

8.5 CONCLUSION

8.6 REVIEW QUESTIONS

REFERENCES

9.  Applications of Virtual Reality

9.1 MEDICAL APPLICATIONS OF VIRTUAL REALITY

9.1.1   Medical Education

Anatomy trainers using Virtual Reality

Surgical Skill Acquisition when Using VR Trainers

9.1.2   Virtual Rehabilitation

Virtual Rehabilitation Game Design Principles

Virtual Rehabilitation for Management of Chronic Pain

Virtual Rehabilitation for Cognitive Impairments

Autism Spectrum Disorder

Dementias

Remote Therapeutic Monitoring Using Home Virtual Rehabilitation

Chronic stroke

Cerebral Palsy

9.2 VIRTUAL REALITY IN EDUCATION, ARTS AND ENTERTAINMENT

9.2.1   Virtual Reality in Education

Constructionist Learning in Virtual Reality

Constructionist 3D Dynamic Geometry

The Plasma Engineering Playground

Constructivist Learning in Virtual Reality

9.2.2   Virtual Reality and the Arts

The Virtual Florentine Pietà

The House of Greek Epigrams in ancient Pompeii

Visitor fixation vs. feedback modality – The Honk Kong Heritage 

Olfactory Heritage

9.2.3   Entertainment applications of Virtual Reality

Gaming on Personal Computers

Location-based Virtual Reality Entertainment

Virtual Reality Arcades

Virtual Reality Lounges

Cloud Gaming

General Cloud Gaming

Virtual Reality Cloud Gaming

9.3 MILITARY VIRTUAL REALITY APPLICATIONS

9.3.1   Army Use of Virtual Reality

Small Arms Tactical Trainers

Armored Personnel Carrier Driver Trainer

9.3.2   Air Force Use of Virtual Reality

Aircraft Deployable Tactical Trainers

Mixed-Reality Flight Simulators

Transfer of Training from VR and MR Flight Simulators

9.3.3 Virtual Reality Applications in the Navy

Naval Ship and Bridge Virtual Reality Simulator

Shipboard Firefighting Trainer

9.4 CONCLUSIONS          

9.5 REVIEW QUESTIONS

REFERENCES

Index

Laboratory Manual (online on book site)

GRIGORE C. BURDEA is Professor Emeritus at Rutgers, the State University of New Jersey, author of several books on Virtual Reality and recipient of the prestigious IEEE Virtual Reality Career Award. He is Fellow of the IEEE Virtual Reality Academy and Founder of the International Society on Virtual Rehabilitation. PHILIPPE COIFFET was a Director of Research at the French National Scientific Research Center and Member of the National Academy for Technology of France. He authored 20 books on Robotics and Virtual Reality, which have been translated into several languages.