Benjamin Lok
University of North Carolina at Chapel Hill
Advisor: Dr. Frederick P. Brooks, Jr.
Committee:
· Dr. Edward Johnson (Cognitive Psychology – Professor Emeritus)
· Dr. Anselmo Lastra (Computer Science)
· Dr. Gregory Welch (Computer Science)
· Professor Mary Whitton (Computer Science)
Citation:
Lok, B. “Interacting with Dynamic Real Objects in Virtual Environments,” Ph.D. Dissertation, Department of Computer Science, University of North Carolina at Chapel Hill, 2002.
Abstract:
Suppose one has a virtual model of a car engine and wants to use an immersive virtual environment (VE) to determine whether both a large man and a petite woman can readily replace the oil filter. This real world problem is difficult to solve efficiently with current modeling, tracking, and rendering techniques. Hybrid environments, systems that incorporate real and virtual objects within the VE, can greatly assist in studying this question.
We present algorithms to generate virtual representations, avatars, of dynamic real objects at interactive rates. Further, we present algorithms to allow virtual objects to interact with and respond to the real-object avatars. This allows dynamic real objects, such as the user, tools, and parts, to be visually and physically incorporated into the VE. The system uses image-based object reconstruction and a volume-querying mechanism to detect collisions and to determine plausible collision responses between virtual objects and the real-time avatars. This allows our system to provide the user natural interactions with the VE and visually faithful avatars.
But is incorporating real objects even useful for VE tasks? We conducted a user study that showed that for spatial cognitive manual tasks, hybrid environments provide a significant improvement in task performance measures. Also, participant responses show promise of our improving sense-of-presence over customary VE rendering and interaction approaches.
Finally, we have begun a collaboration with NASA Langley Research Center to apply the hybrid environment system to a satellite payload assembly verification task. In an informal case study, NASA LaRC payload designers and engineers conducted common assembly tasks on payload models. The results suggest that hybrid environments could provide significant advantages for assembly verification and layout evaluation tasks.
Downloads:
Final dissertation (4/24/02) in:
Defense (4/4/02) Slides in
Generating virtual representations of real objects from multiple camera images in real-time. The reconstructed models are visually incorporated with the virtual objects. Notice the correct occlusion between the participant’s hand (real) and the teapot handle (virtual).
We allow the real object avatars to affect virtual objects. Here a participant parts virtual curtains to look out a window in a VE.
We conducted a study to see the effects of handling real objects on a cognitive task.
Four NASA payload designers tried out the system on a assembly verification task.