We describe OctoPocus, an example of a dynamic guide that combines on-screen feedforward and feedback to help users learn, execute and remember gesture sets. OctoPocus can be applied to a wide range of single-stroke gestures and recognition algorithms and helps users progress smoothly from novice to expert performance. We provide an analysis of the design space and describe the results of two experiments that show that OctoPocus is significantly faster and improves learning of arbitrary gestures, compared to conventional Help menus. It can also be adapted to a markbased gesture set
Finger-based touch input has become a major interaction modality for mobile user interfaces. However, due to the low precision of finger input, small user interface components are often difficult to acquire and operate on a mobile device. It is even harder when the user is on the go and unable to pay close attention to the interface. In this paper, we present Gesture Avatar, a novel interaction technique that allows users to operate existing arbitrary user interfaces using gestures. It leverages the visibility of graphical user interfaces and the casual interaction of gestures. Gesture Avatar
Multi-Touch technology provides a successful gesture based Human Computer Interface. The contact and gesture recognition algorithms of this interface are based on full hand function and, therefore, are not accessible to many people with physical disability. In this paper, we design a set of command-like gestures for users with limited range and function in their digits and wrist. Trajectory and angle features are extracted from these gestures and passed to a recurrent neural network for recognition. Experiments are performed to test the feasibility of gesture recognition system and determine
Collaborative Information Retrieval (CIR) is the process by which people working together can collaboratively search for, share and navigate through information. Computer support for CIR currently makes use of single-user systems. CIR systems could benefit from the use of multi-user interaction to enable more than one person to collaborate using the same data sources, at the same time and in the same place. Multi-touch interaction has provided the ability for multiple users to interact simultaneously with a multi-touch surface. This paper presents a generalised architecture for multi-touch CIR
Creating multiple prototypes facilitates comparative reasoning, grounds team discussion, and enables situated exploration. However, current interface design tools focus on creating single artifacts. This paper introduces the Juxtapose code editor and runtime environment for designing multiple alternatives of both application logic and interface parameters. For rapidly comparing code alternatives, Juxtapose>introduces selectively parallel source editing and execution.
DATE:
TEAM MEMBERS:
Björn HartmannLoren YuAbel AllisonYeonsoo YangScott R. Klemmer
This project is aimed at creating a language / framework independent Gesture Recognition toolkit that takes OSC messages formatted with TUIO specification as input and outputs recognized gestures via OSC protocol. I will use the gesture recognition toolkit AMELiA to describe models specifically for the domain of multitouch gestures. This project will enable multitouch application developers to easily define a gesture and utilize it within their application, creating more engaging experiences.
This paper explores the interactive possibilities enabled when the barrel of a digital pen is augmented with a multi-touch sensor. We present a novel multi-touch pen (MTPen) prototype and discuss its alternate uses beyond those of a standard stylus, such as allowing new touch gestures to be performed using the index finger or thumb and detecting how users grip the device as a mechanism for mode switch-ing. We also discuss the hardware and software implemen-tation challenges in realizing our prototype, and showcase how one can combine different grips (tripod, relaxed tripod, sketch, wrap) and
DATE:
TEAM MEMBERS:
Jim SpadacciniHyunyoung SongHrvoje BenkoFrancois GuimbretiereShahram IzadiXiang CaoKen Hinckley
Creating and editing large graphs and node-link diagrams are crucial activities in many application areas. For them, we consider multi-touch and pen input on interactive surfaces as very promising. This fundamental work presents a user study investigating how people edit node-link diagrams on an interactive tabletop. The study covers a set of basic operations, such as creating, moving, and deleting diagram elements. Participants were asked to perform spontaneous gestures for 14 given tasks. They could interact in three different ways: using one hand, both hands, as well as pen and hand
DATE:
TEAM MEMBERS:
Mathias FrischJens HeydekornRaimund Dachselt
In this paper we describe two projects that utilize reality-based interaction to advance collaborative scientific inquiry and discovery. We discuss the relation between reality-based and embodied interaction, and present findings from an experimental study that illustrate benefits of reality-based tabletop interaction for collaborative inquiry-based learning.
New mobile devices with large multi-touch displays, such as the iPad, have brought revolutionary changes to ways users interact with computers. Instead of traditional input devices such as keyboards, touchpads and mice, multi-touch gestures are used as the primary means of interacting with mobile devices. Surprisingly, body-motion gestures are evolving to become a new, natural, and effective way for game players to interact with game consoles in a very similar fashion: in Kinect for Xbox 360, a controller-free gaming experience is made possible by using body-motion gestures to play games.
This paper outlines research showing a suprizing agreement in the guesability of multitouch gestures on tabletop surfaces between users. It also provides more evidence that crowd sourcing gesture mapping will lead to more complete intuitive gesture set and potential convergence into a standard gesture library.
DATE:
TEAM MEMBERS:
Jacob WobbrockMeredith MorisAndrew Wilson
Direct-touch interaction on mobile phones revolves around screens that compete for visual attention with users‟ real-world tasks and activities. This paper investigates the impact of these situational impairments on touch-screen interaction. We probe several design factors for touch-screen gestures, under various levels of environmental demands on attention, in comparison to the status-quo approach of soft buttons. We find that in the presence of environmental distractions, ges-tures can offer significant performance gains and reduced attentional load, while performing as well as soft buttons
DATE:
TEAM MEMBERS:
Andrew BragdonEugene NelsonYang LiKen Hinckley