eCite Digital Repository

3D helping hands: a gesture based MR system for remote collaboration


Tecchia, F and Alem, L and Huang, W, 3D helping hands: a gesture based MR system for remote collaboration, Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry (VRCAI 2012), 2-4 December 2012, Singapore, pp. 323-328. ISBN 978-1-4503-1825-9 (2012) [Refereed Conference Paper]

Copyright Statement

Copyright 2012 ACM

Official URL:


There is currently a strong need for collaborative systems with which two or more participants interact over a distance on a task involving tangible artifacts (e.g., a machine, a patient, a tool). The present paper focuses on the specific category of remotecollaboration systems where hand gestures are used by a remote helper to assist a physically distant worker to perform manual tasks. Existing systems use a combination of video capturing, 2D monitors or 2D projectors, however displaying a video of the remote workspace and allowing helpers to gesture over the video does not provide helpers with sufficient understanding of the spatial relationships between remote objects and between their hands and the remote objects. In this paper we introduce our tele-presence Mixed Reality system for remote collaboration on physical tasks based on real-time capture and rendering of the remote workspace and of the helper’s hands. We improve on previous 2D systems introducing 3D capturing and rendering, and exploiting the possibility offered by the use of real 3D data to increase the feeling of immersion offered by the system using head tracking, stereoscopic rendering, inter-occlusion handling and virtual shadowing. We performed initial usability test of our system to verify if users are satisfied with the spatial awareness the system provides.

Item Details

Item Type:Refereed Conference Paper
Research Division:Information and Computing Sciences
Research Group:Library and information studies
Research Field:Human information interaction and retrieval
Objective Division:Expanding Knowledge
Objective Group:Expanding knowledge
Objective Field:Expanding knowledge in the information and computing sciences
UTAS Author:Huang, W (Dr Tony Huang)
ID Code:90141
Year Published:2012
Deposited By:Information and Communication Technology
Deposited On:2014-03-27
Last Modified:2017-11-20
Downloads:1 View Download Statistics

Repository Staff Only: item control page