One of the primary research themes in the IS&UE RCE is developing and evaluating 3D user interfaces for virtual, augmented, and mixed reality. In particular, we are focused on exploring how to bring 3D user interface techniques and concepts into mainstream video games by leveraging the existing body of work in 3DUI and VR and devising new strategies and methodologies for bringing spatial 3D interaction to gamers. Additionally, we are interested in the continued learning and understanding of how humans interact with and are affected by 3D interfaces.
With the release of a variety of new motion controllers for both PC and console gaming, 3D user interfaces are becoming commonplace in modern games. The focus of this work is to explore how to best utilize 3D spatial interaction in the video game domain by examining existing interaction techniques and creating novel ones as well as understanding how these interfaces affect users.
The Bespoke 3DUI XNA Framework is an open-source software platform for research in 3D user interaction. The Bespoke 3DUI XNA Framework distinguishes itself from other platforms, in that it provides 3D user interface machinery in a game development framework. This combination leverages lowcost, widely available game technologies, enabling researchers to investigate 3DUI techniques, and providing game developers a foundation for prototyping 3DUIs in commercial video games.
In recent years the popularity of music and rhythm-based games has experienced tremendous growth. However, almost all of these games require custom hardware to be used as input devices, and these devices control only one or two similar instruments. One Man Band, a prototype video game for musical expression uses novel 3D spatial interaction techniques using accelerometer-based motion controllers. One Man Band provides users with 3D gestural interfaces to control both the timing and sound of the music played, with both single and collaborative player modes. One Man Band further investigates the ability to detect different musical gestures without explicit selection of mode, giving the user the ability to seamlessly transition between instrument types with a single input device.
With the rise in popularity of 3D spatial interaction in console gaming, such as the Nintendo Wii, it is important to determine whether existing menuing technique findings still hold true when using a 3D pointing device such as the Wii Controller. This project compares linear menus with two other menu techniques: radial menus and rotary menus. We measure effectiveness through task completion time and the number of task errors. User study results indicate that radial menus are an effective menu technique when used with a 3D pointing device.
Non-isomorphic rotational mappings have been shown to be an effective technique for rotation of virtual objects in 3D desktop environments. However, it is unclear how non-isomorphic 3D rotation techniques transition into immersive virtual environments. This project aims to determine how various components, such as head tracking and stereo, affect user performance when rotating virtual 3D objects.
This work was performed in collaboration with Brown University.
The Turn Table peripheral was designed for the FIEA PC video game Sultans of Scratch.
In the game the player must emulate scratching a left and right disc, in addition to slamming a slider knob, in order to match on screen queues to earn points in a tag team style match between other players. The peripheral is composed of an Arduino microprocessor board that relays information about disc rotation and slider location to the video game. Disc rotation is capture with two optical mice that provide counter and counter-clockwise change in rotation values. This paired with a potentiometer's voltage output for the slider's location, enables the game to make the player feel like an actual DJ.
This is a novel, real-time, markerless vision-based tracking system, employing a rigid orthogonal configuration of two pairs of opposing cameras. Our system uses optical flow over sparse features to overcome the limitation of vision-based systems that require markers or a pre-loaded model of the physical environment. We show how opposing cameras enable cancellation of common components of optical flow leading to an efficient tracking algorithm. Experiments comparing our device with an electromagnetic tracker show that its average tracking accuracy is 80% over 185 frames, and it is able to track large range motions even in outdoor settings.
This is a proof-of-concept implementation of a handheld stereo projection display system for virtual worlds. We utilize a single pico projector coupled with a six DOF tracker to generate realtime stereo imagery that can be projected on walls or a projection screen.
The Virtual Experience Test (VET) is a survey instrument used to measure holistic virtual environment experiences based upon the five dimensions of experiential design: sensory, cognitive, affective, active, and relational. Experiential Design (ED) is a holistic approach to enhance presence in virtual environments that goes beyond existing presence theory (i.e. a focus on the sensory aspects of VE experiences) to include affective and cognitive factors.
To explore the 3D file browsing technique design space, we analyzed the existing literature and developed three representative 3D file browsing techniques that cover many of their characteristics. Block3D uses a priority weighting scheme to elevate and display files in a grid-based structure. Cluster3D uses sets of animated racks to display files. LTreeCube3D visualizes files and directories using groups of semi-transparent cubes within a larger cube-like structure. An experiment was conducted exploring the effect these 3D file browsing technique have on users in a manual file searching task. The results indicate that users completed the manual file search task significantly faster using Block3D than both LTreeCube3D and Cluster3D. Although subjective ranking showed users preferred the Block3D technique, user feedback also showed merits of the other techniques.