Augmented Reality

[Page 61]Definition of AR

AR has been defined by Larry Johnson (2011) and colleagues as “the addition of a computer-assisted contextual layer of information over the real world, creating a reality that is enhanced or augmented” (p. 16). As described by Iulian Radu and Blair MacIntyre, users of an AR system may control the virtual environment through specific interactions with objects tracked in a physical space. Typically, AR uses two primary forms of observational interface: a head-mounted display that features an egocentric viewpoint, or a tablet or smart-phone monitor that offers a window-on-the-world viewpoint, as Paul Milgram and colleagues have observed.

AR technology inserts 2D or 3D overlays into an observational interface that applies either see-through technology, which projects overlays onto a direct view of the real world, or video-based technology, which captures the real-world imagery with a camera and mixes virtual elements into the observed display. The inserted objects, avatars, effects, and overlays must be precisely overlaid and not jitter or drift as the user moves around and changes perspective. This is particularly challenging for see-through technology because the user can move his or her head rapidly and the system must predict his location very accurately in order to have a stable insertion. AR engineers trigger these overlays by using different approaches for detecting information from the observed environment. In one case, markers may be physically placed on points in an observed environment, and then the AR system generates the overlays as it detects the markers and estimates the camera pose by matching camera images to models of the markers.

By contrast, markerless systems, such as those developed by Taragay Oskiper, Supun Samarasekera, and Rakesh Kumar, use multiple sensors (e.g., video cameras, inertial measurement units, GPS, compass, etc.) to estimate the precise pose (3D location and 3D orientation) of the camera/head and use the computed pose to render the overlays, providing more flexibility. Recent technologies such as image-based object and landmark recognition based on library matching and gesture-based triggers offer even greater flexibility. A technology trend has been to go from local small area, indoor augmented reality to wide area, outdoor augmented reality.

Another key technology required for true augmented reality is occlusion reasoning, where the inserted objects may occlude or be occluded by objects appearing naturally in the scene based on their geometric location with respect to these objects. Occlusion reasoning for AR has typically been done by building 3D models of the static scene, but as Mikhail Sizintsev and colleagues have shown, occlusion based on dynamic and moving real objects may be handled by computer live 3D structure of the scene.

...