Reality-Based Interaction



Over the past two decades, HCI researchers have developed a broad range of new interfaces that diverge from the "window, icon, menu, pointing device" (WIMP) or Direct Manipulation interaction style. Development of this new generation of post-WIMP interfaces has been fueled by advances in computer technology and improved understanding of human psychology. Some examples of these post-WIMP interaction styles are: virtual, mixed and augmented reality, tangible interaction, ubiquitous and pervasive computing, context-aware computing, handheld, or mobile interaction, perceptual and affective computing as well as lightweight, tacit or passive interaction. Although some may see these interaction styles as disparate innovations proceeding on unrelated fronts, we propose that they share salient and important commonalities, which can help us understand, connect, and analyze them. We believe that all of these new interaction styles draw strength by building on users' pre-existing knowledge of the everyday, non-digital world to a much greater extent than before.

RBI Themes

RBI Themes
We use the term "real world" to refer to aspects of the physical, non-digital world. However, the terms real world and reality are problematic and can have many additional, interpretations, including cultural and social reality. For that matter, many would also consider keyboards and mice to be as much a part of today's reality as any non-digital artifact. Thus, to clarify, our framework focuses specifically on the following four themes from the real world:

  • Na´ve Physics: people have common sense knowledge about the physical world.

  • Body Awareness & Skills: people have an awareness of their own physical bodies and possess skills for controlling and coordinating their bodies.

  • Environment Awareness & Skills: people have a sense of their surroundings and possess skills for negotiating, manipulating, and navigating within their environment.

  • Social Awareness & Skills: people are generally aware of others in their environment and have skills for interacting with them.

To a greater extent than in previous generations, these four themes play a prominent role in emerging interaction styles. They provide a basis for interaction with computers that is markedly closer to our interaction with the non-digital world.

More Information