Abstract:Extended reality applications with haptic interaction over the Tactile Internet enable humans to perceive and manipulate remote environments in real time. However, due to the different requirements of each modality in terms of data rate, latency, and so on, as well as the complexity of characterizing the continuous relationship among modalities, when it comes to tele-manipulation with fine haptic experience in real time, how to effectively transmit these multi-modal streams while ensuring high-quality user experience when network congestion occurs remains a significant challenge. To address this challenge, this study proposes a cross-modal transmission strategy consisting of an active packet loss scheme and a cross-modal restoration procedure. The active packet loss scheme obtains tolerated packet loss probabilities by quantifying the quality of experience, along with predicting the restoration effect using a cross-modal reconstruction model that characterizes the relationship among modalities through real-time haptic rendering. When network congestion is detected, packets from the sender with a high tolerated packet loss probability are actively discarded with high priority. The receiver adopts the cross-modal reconstruction model from the active packet loss scheme to restore the lost signals through other modalities in real-time. We developed a visuo-haptic interaction testbed with 1 kHz force feedback to evaluate our cross-modal transmission strategy. Experimental results from both visual and haptic active packet loss tests show that the transmission gain of the overall mean opinion score exceeds 80 percent at a packet loss rate of 30 percent, indicating that our method enables a significant number of packets to be discarded, while maintaining a high-quality user experience, thereby alleviating network congestion.

A window-based fast-pacing mice flow transmission mechanism for Virtual Reality Environment

Mice flow transmission mechanism: Delay-sensitive Fast-Pacing TCP

Walking Through a Very Large Virtual Environment in Real-time.

Web-based virtual walkthrough of panoramas

BALANCING FIDELITY AND PERFOR MANCE IN VIRTUAL WALKTHROUGH

Simulation of Optical Communication Technology Based on Wireless Sensor Networks in Museum VR Design

Efficient communication approach in HLA-based distributed simulation for VR application

Freeway: Adaptively Isolating the Elephant and Mice Flows on Different Transmission Paths

Deadline-aware and Energy-Efficient Dynamic Flow Scheduling in Data Center Network.

Virtual Reality Wireless Mobile Walkthrough Framework

A novel high speed transport protocol based on explicit virtual load feedback

Implementation Of Progressive Data Transmission In Distributed Virtual Environment

Cross-Modal Transmission with Active Packet Loss and Restoration for Tactile Internet

An Efficient TCP Flow State Management Algorithm in High-Speed Network

High Speed Traffic Archiving System For Flow Granularity Storage And Querying

Architecture and Key Technologies of Virtual Reality Video Transmission and Streaming

FoVR: Attention-based VR Streaming through Bandwidth-limited Wireless Networks

VasNet_mr: Interactive Exploration of Vascular Network Blood Flow Distribution with Mixed Reality

Poster - A Hierarchical VR Streaming System through a WiFi Connection.

A novel spread estimation based abnormal flow detection in high-speed networks

Towards Tile Based Distribution Simulation in Immersive Video Streaming