Reality-Based Interaction

Team

Introduction

Over the past two decades, HCI researchers have developed a broad range of new interfaces that diverge from the "window, icon, menu, pointing device" (WIMP) or Direct Manipulation interaction style. Development of this new generation of post-WIMP interfaces has been fueled by advances in computer technology and improved understanding of human psychology. Some examples of these post-WIMP interaction styles are: virtual, mixed and augmented reality, tangible interaction, ubiquitous and pervasive computing, context-aware computing, handheld, or mobile interaction, perceptual and affective computing as well as lightweight, tacit or passive interaction. Although some may see these interaction styles as disparate innovations proceeding on unrelated fronts, we propose that they share salient and important commonalities, which can help us understand, connect, and analyze them. We believe that all of these new interaction styles draw strength by building on users' pre-existing knowledge of the everyday, non-digital world to a much greater extent than before.

RBI Themes

RBI Themes
We use the term "real world" to refer to aspects of the physical, non-digital world. However, the terms real world and reality are problematic and can have many additional, interpretations, including cultural and social reality. For that matter, many would also consider keyboards and mice to be as much a part of today's reality as any non-digital artifact. Thus, to clarify, our framework focuses specifically on the following four themes from the real world:

  • Naïve Physics: people have common sense knowledge about the physical world.

  • Body Awareness & Skills: people have an awareness of their own physical bodies and possess skills for controlling and coordinating their bodies.

  • Environment Awareness & Skills: people have a sense of their surroundings and possess skills for negotiating, manipulating, and navigating within their environment.

  • Social Awareness & Skills: people are generally aware of others in their environment and have skills for interacting with them.

To a greater extent than in previous generations, these four themes play a prominent role in emerging interaction styles. They provide a basis for interaction with computers that is markedly closer to our interaction with the non-digital world.

More Information

Acknowledgements

We thank Andrew Afram, Eric Bahna, Tia Bash, Georgios Christou, Michael Poor, and Andrew Pokrovski, who are students and alumni of the HCI group at Tufts. We thank our collaborators Caroline Cao and Holly Taylor of Tufts, Leonidas Deligiannidis of Wentworth Institute of Technology, Hiroshi Ishii of the MIT Media Lab and the students in his Tangible Interfaces class, Sile O'Modhrain of Queen's University Belfast, and Frank Ritter of Pennsylvania State University. We also thank the participants in our CHI 2006 workshop on "What is the Next Generation of Human-Computer Interaction?" for their thoughts and discussion about this area, which have helped us refine our work, and Ben Shneiderman in particular for discussions on this topic. And the participants in the CHI 2007 workshop on "Tangible User Interfaces in Context and Theory," organized by Alan Blackwell, George Fitzmaurice, Lars Erik Holmquist, Hiroshi Ishii, and Brygg Ullmer. And we thank the National Science Foundation (NSF Grant No. IIS-0414389) and the Natural Sciences and Engineering Research Council of Canada for support of this research. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation.