Interactive augmented reality

Abstract

Augmented reality can provide a new experience to users by adding virtual objects where they are relevant in the real world. The new generation of mobile phones offers a platform to develop augmented reality application for industry as well as for the general public. Although some applications are reaching commercial viability, the technology is still limited. The main problem designers have to face when building an augmented reality application is to implement an interaction method. Interacting through the mobile's keyboard can prevent the user from looking on the screen. Normally, mobile devices have small keyboards, which are difficult to use without looking at them. Displaying a virtual keyboard on the screen is not a good solution either as the small screen is used to display the augmented real world. This thesis proposes a gesture-based interaction approach for this kind of applications. The idea is that by holding and moving the mobile phone in different ways, users are able to interact with virtual content. This approach combines the use of input devices as keyboards or joysticks and the detection of gestures performed with the body into one scenario: the detection of the phone's movements performed by users. Based on an investigation of people's own preferred gestures, a repertoire of manipulations was defined and used to implement a demonstrator application running on a mobile phone. This demo was tested to evaluate the gesture-based interaction within an augmented reality application. The experiment shows that it is possible to implement and use gesture-based interaction in augmented reality. Gestures can be designed to solve the limitations of augmented reality and offer a natural and easy to learn interaction to the user.