Gesture Avatar: A Technique for Operating Mobile User Interfaces Using Gestures

Finger-based touch input has become a major interaction modality for mobile user interfaces. However, due to the low precision of finger input, small user interface components are often difficult to acquire and operate on a mobile device. It is even harder when the user is on the go and unable to pay close attention to the interface. In this paper, we present Gesture Avatar, a novel interaction technique that allows users to operate existing arbitrary user interfaces using gestures. It leverages the visibility of graphical user interfaces and the casual interaction of gestures. Gesture Avatar can be used to enhance a range of mobile interactions. A user study we conducted showed that compared to Shift (an alternative technique for target acquisition tasks), Gesture Avatar performed at a much lower error rate on various target sizes and significantly faster on small targets (1mm). It also showed that using Gesture Avatar while walking did not significantly impact its performance, which makes it suitable for mobile uses