Benjamin Lok

University of Florida

lok@cise.ufl.edu

Citation:

Lok, B. "Online Model Reconstruction for Interactive Virtual Environments," Proceedings of ACM 2001 Symposium on Interactive 3D Graphics, Chapel Hill, NC, 18-21, March 2001, pp. 69-72, 248. 

Abstract:

We present a system for generating real-time 3D reconstructions of the user and other real objects in an immersive virtual environment (IVE) for visualization and interaction.  For example, when parts of the user's body are in his field of view, our system allows him to see a visually faithful graphical representation of himself, an avatar.  In addition, the user can grab real objects, and then see and interact with those objects in the IVE.  Our system bypasses an explicit 3D modeling stage, and does not use additional tracking sensors or prior object knowledge, nor do we generate dense 3D representations of objects using computer vision techniques.  We use a set of outside-looking-in cameras and a novel visual hull technique that leverages the tremendous recent advances in graphics hardware performance and capabilities. We accelerate the visual hull computation by using projected textures to rapidly determine which volume samples lie within the visual hull.  The samples are combined to form the object reconstruction from any given viewpoint.  Our system produces results at interactive rates, and because it harnesses ever-improving graphics hardware, the rates and quality should continue to improve.  We further examine real-time generated models as active participants in simulations (with lighting) in IVEs, and give results using synthetic and real data.

Additional Keywords: Avatars, Visual Hull, Virtual Reality, Head Mounted Displays, Frame Buffer Tricks, HCI (Human-Computer Interface), Image-Based Rendering

CG Keywords: I.3.7 [Computer Graphics]: Three-Dimensional Graphics and Realism – Virtual Reality; I.3.3 [Computer Graphics]: Bitmap and Framebuffer Operations, I.3.6 [Computer Graphics]: Methodology and Techniques – Interaction Techniques

Downloads:

    Final paper (1/15/03) in:

        Microsoft Word (1.3 Mb)

        Adobe PDF (324 kb)

        HTML

     Presentation in:

          Microsoft Powerpoint (4.1 Mb)

Data set of a 3d model rendered from five positions.  This is the input to the reconstruction algorithm.  Our goal is to generate an arbitrary view of the user in real-time for avatars and interacting with the virtual environment.

The depth buffer of a novel view of the object.

We dynamically reconstruct and light the model from a novel viewpoint.

The resulting models can be used for lighting and  inputs into simulations such as collision detection.