[-empyre-] Reemergence of the Augment - a look through the literature of AR (part1)

[Rodney Berry]

Hi all,

Here's my first $.02 worth, so I'll take the opportunity to say Hi and that
it's great to be 'here' in this discussion, and to welcome others who feel
like joining in or just lurking . I'll try to keep my subsequent posts a bit
shorter :)

thanks Patrick for hauling me into this. I'll try to get others from the AR
community to weigh in on this stuff later on if possible.


best wishes,


Rod Berry


        ♪ ~~~~~~~~~~~~~~~~~~~~~~~~~~


A good place to start is to think about what is being augmented and with
what? This gets us down to our basic ideas about reality etc. and how these
come up in what people have written about augmented reality.

Writings about augmented reality typically describe reality in terms of the
underlying technologies. Above all others, the most heavily cited is Milgram
and Kishino’s *A TAXONOMY OF MIXED REALITY VISUAL DISPLAYS (Milgram, P., and
F. Kishino. 1994. A Taxonomy of Mixed Reality Visual Displays. IEICE
Transactions on Information Systems, . Dec, 94).* *
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.102.4646&rep=rep1&type=pdf
*<http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.102.4646&rep=rep1&type=pdf>

More specifically, the canonical moment of the paper is Milgram’s *Virtuality
Continuum*, a diagram that shows an axis running between reality and
virtuality, between which we find *mixed reality* in its variant
forms, *augmented
reality* and *augmented virtuality*.

http://upload.wikimedia.org/wikipedia/en/d/dc/Virtuality_Continuum_2.jpg

The diagram re-appears with slight variations in more than 400 papers by
other authors dealing with *augmented reality* and its variants. Indeed,
failure to at least mention the virtuality continuum is almost certainly the
kiss of death for any paper submission in the field.


Milgram’s continuum places augmented reality down the real end of the
continuum where the real world is firmly in place and visible, but is
augmented by images and objects from the virtual world. There are a number
of implicit assumptions present in this representation:

   1. Real is real and virtual is virtual and never the twain should meet.
   They necessarily exclude one another
   2. Information presented by the computer is virtual and the directly
   perceived elements are real.
   3. This is all true regardless of what the *content* might be, or who is
   viewing it.

It is interesting that an item from the very beginning of a fifteen page
journal paper should become so enshrined in a research community.
Considering that the remainder of the paper is devoted to ironing out the
inadequacies of this simple model, I wonder (as must Milgram) if people
actually bothered to read the rest of the paper and the taxonomy that tries
to unpack and clarify the above model.

After presenting the virtuality continuum, Milgram goes on show how this
idea of real and virtual does not adequately distinguish between the
varieties of configurations that one might encounter traversing its length.
Milgram attempts to qualify it somewhat by dividing real and virtual into
three aspects:

   1. In the realm of objects, an operational definition of *real* as any
   object that has an objective existence. A virtual object is one that exists
   *in essence or effect*, but not *formally or actually*.
   2. In the realm of image quality, perhaps more in the sense of realism of
   the image.
   3. Fuzzier than the first two, but closer to the use of virtual in
   optics, whether the final image has no luminance at its apparent location,
   such as a transparent hologram or an image in a mirror.

They go on to present a formalised taxonomy for merging real and virtual
worlds. The taxonomy is a space made of three axes:

   1. Extent of World Knowledge – How much do we know about the world being
   displayed? Or, more correctly, how much does the computer know about the
   world being displayed? At one end of the extent of World Knowledge axis, we
   find the completely unmodelled world. For example, the computer mixing the
   TV weather presenter over the CG weather map knows almost nothing about
   either item. However, some fancy new weather shows allow the presenter to
   point with a coloured pointer to move icons around the screen. In this case
   the system has basic knowledge of the location of the pointer, at least in a
   2D image of the world, so we could say that the world is partially modelled.
   At the extreme end of axis we would find a completely modelled world where
   the computer knows and tracks the location of every object in the real world
   and has its own constantly updating 3D model of the weather presenter and
   anything else in the scene.
   2. Reproduction Fidelity – How realistically can we display this world?
   This is largely concerned with realism in terms of the quality of the image
   of both the real and the virtual objects. On one end of the axis we would
   find a simple, low resolution mono video image using simple wireframe
   objects. At the other end would be some kind of super high definition 3D TV
   image with lushly-rendered 3D objects.
   3. Extent of Presence Metaphor - To what extent does the user feel that
   he is present in the world? This is linked to Reproduction Fidelity and also
   concerned with realism, but more in terms of the subjective experience of
   the user and her sense of ‘being there’. This axis ranges from a simple mono
   screen (exocentric view) at one end, extending through panoramic imaging to
   fully immersive head-mounted ‘classical’ virtual reality (egocentric view)
   at the other extreme.

The goal of such a taxonomy is that every type of augmented or mixed reality
system imaginable could be found to exist at a particular point in this
space. In doing this, I believe that Milgram is quite successful in this aim
while thinking about suitability of a particular system for a particular
task.


However, I don’t think this will help designers in thinking about what they
could do with this technology without a particular task in mind. In short,
the space of possible augmented reality systems does not encompass the space
of possible things we might want to express with such systems, or what such
systems might want to mean. This is not helped by researchers enshrining the
mixed reality continuum as something that defines and delineates what mixed
reality is and means.


Anyway, that's enough for now, I'll try to continue with this and pull in
some other technical and more philosophical literature specifically about AR
and mixed reality in general.


Rod.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.cofa.unsw.edu.au/pipermail/empyre/attachments/20110408/242d3940/attachment.html>