Abstract:Recent research advance has significantly improved the visual realism of immersive 3D video communication. In this work we present a method to further enhance this immersive experience by adding the hand touch capability ("remote hand clapping"). In our system, each meeting participant sits in front of a large screen with haptic feedback. The local participant can reach his hand out to the screen and perform hand clapping with the remote participant as if the two participants were only separated by a virtual glass. A key challenge in emulating the remote hand touch is the realistic rendering of the participant's hand and arm as the hand touches the screen. When the hand is very close to the screen, the RGBD data required for realistic rendering is no longer available. To tackle this challenge, we present a dual representation of the user's hand. Our dual representation not only preserves the high-quality rendering usually found in recent image-based rendering systems but also allows the hand to reach to the screen. This is possible because the dual representation includes both an image-based model and a 3D geometry-based model, with the latter driven by a hand skeleton tracked by a side view camera. In addition, the dual representation provides a distance-based fusion of the image-based and 3D geometry-based models as the hand moves closer to the screen. The result is that the image-based and 3D geometry-based models mutually enhance each other, leading to realistic and seamless rendering. Our experiments demonstrate that our method provides consistent hand contact experience between remote users and improves the immersive experience of 3D video communication.

Haptic Rendering of Virtual Hand Moving Objects

Physically-Based Haptic Rendering for Virtual Hand Interaction.

Data Analysis on Virtual Stiffness in 6dofs Haptic Rendering System

A Realistic Force Rendering Algorithm for CyberGrasp

Contact Force Rendering of Virtual Hand Interaction Based on Human Fingers' Force Characteristics

Realistic Virtual Hand Modeling with Applications for Virtual Grasping

Haptic rendering method based on generalized penetration depth computation.

Realistic force generation for virtual hand manipulating objects

Dual Quaternion Based Virtual Hand Interaction Modeling

Grasp Identification and Multi-Finger Haptic Feedback for Virtual Assembly

Haptic Sketches on the Arm for Manipulation in Virtual Reality

Haptic Feedback Relocation from the Fingertips to the Wrist for Two-Finger Manipulation in Virtual Reality

Modelling Contact Forces For 3d Interactive Peg-In-Hole Virtual Reality Operations

A Framework for Virtual Hand Haptic Interaction

A Position-Control Based Approach to Haptic Rendering of Stiff Objects

High-Fidelity Grasping In Virtual Reality Using A Glove-Based System

RemoteTouch: Enhancing Immersive 3D Video Communication with Hand Touch

HaptGlove—Untethered Pneumatic Glove for Multimode Haptic Feedback in Reality–Virtuality Continuum

3D Hand Motion Generation for VR Interactions Using a Haptic Data Glove

Six-degree-of-freedom Haptic Rendering Using Translational and Generalized Penetration Depth Computation.

A Virtual Grasping Method of Dexterous Virtual Hand Based on Leapmotion