JOURNAL OF MULTIMEDIA (JMM)
ISSN : 1796-2048
Volume : 1    Issue : 7    Date : November/December 2006

A Framework for Constructing Real-time Immersive Environments for Training Physical Activities
Sang-Hack Jung and Ruzena Bajcsy
Page(s): 9-17
Full Text:
PDF (695 KB)


Abstract
This paper describes a framework for constructing a three-dimensional immersive environment that
can be used for training physical activities. The system is designed to capture three dimensional
full-body human motion in real time and visualize the data either locally or remotely through three-
dimensional display system so that the data can be viewed from arbitrary viewpoints. In the
proposed system, an immersive environment is constructed through realistic reconstruction of a
scene by applying a stereo algorithm on a set of images that is captured from multiple viewpoints.
Specifically, twelve camera clusters that consist of four camera quadruples are used to capture the
scene, where each camera cluster is processed by a pc independently and synchronously so that
partial reconstructions from each viewpoint are merged to form a complete 3D description of the
scene. This paper discusses in detail system architectures that enable synchronous operations
across multiple computers while achieving parallel computations within each multi-processor
system. A set of experiments is performed to learn tai-chi lessons in this environment where
students are instructed to follow pre-recorded teacher’s movements while observing both their own
motions and the teacher in real-time. The effect of learning in the virtual environment is discussed
by comparing the performance of the trainee groups under different control environments.

Index Terms