Virtual Reality: Auditory

Virtual reality (VR) systems provide users with a computer-mediated experience of a sensory world that exists for them as a virtual world within which they can act and interact. Popular fiction has portrayed the VR experience as generating such a realistic illusion that users can forget that they are experiencing a virtual world that is generated by computer. In practice, however, such strong illusions do not typically result unless users are presented with a coordinated interactive display of information via multiple sensory modalities. Therefore, VR systems often include auditory displays as well as visual displays so that users can hear as well as see what occurs in the virtual environments that are generated for them through computer simulations. Of course, haptic display technology can also be an extremely important component of a VR system because it supports the user experience of touching and feeling virtual objects; but here again, the illusion of reality will be best supported if an appropriate sound is heard, for example, whenever the user's hand makes contact with a virtual object. This entry covers spatial auditory displays, perception and action in auditory virtual reality, and research in auditory virtual reality.

Consider the potential advantage users might enjoy if they are able to receive coordinated multimodal stimulation in a telerobotic application, such as remote operation of a vehicle through unknown terrain. Driving a vehicle remotely will almost certainly be facilitated if users are able to listen to sound arriving from all directions around them while focusing their gaze on the environment to be navigated via a relatively constrained visual field. For example, imagine how difficult it would be to drive a car on ice and drifting snow without being able to hear or feel the change in the traction of the tires. Such auditory and haptic feedback could well guide user behavior, allowing successful navigation when vision alone is not enough.

Of course, there are many VR systems serving applications that do not include auditory displays, such as VR-based scientific visualization systems. These are systems that operate upon sets of data [Page 1035]that exist in three or more dimensions and can represent the datasets as solid objects that users can explore visually, without any sonic component. On the other hand, VR-based entertainment systems as a rule will include audio, and many use sophisticated audio signal processing to create realistic spatial impressions of virtual sound sources positioned in the virtual environments in which users find themselves. For example, immersive VR-based games use spatial auditory display technology to provide users with an enhanced awareness of their situation. Indeed, just as in everyday life, auditory events occurring outside of the field of view often lead users to direct their gaze to bring the source of the auditory event into focal vision.