Haptic-feedback smart glove as a creative human-machine interface (HMI) for virtual/augmented reality applications

Minglu Zhu,Zhongda Sun,Zixuan Zhang,Qiongfeng Shi,Tianyiyi He,Huicong Liu,Tao Chen,Chengkuo Lee

DOI: https://doi.org/10.1126/sciadv.aaz8693

IF: 13.6

2020-05-08

Science Advances

Abstract:A haptic-feedback glove with triboelectric sensors and piezoelectric stimulators is developed for interaction in virtual space.

multidisciplinary sciences

What problem does this paper attempt to address?

Multi-modal Sensing-Based Interactive Glove System for Teleoperation and VR/AR

Xinwei Yao,Ming Chen,Chuan Cao,Lei Zhang,Wenzhen Yang,Mukherjee Mithun,Hujun Bao

DOI: https://doi.org/10.1007/978-3-031-46839-1_15

2023-01-01

Abstract:Haptic interaction is a fundamental approach to our perception surrounding people; at this stage, haptic interaction is mainly used in virtual reality (VR) and remote medical procedures (teleoperation). The traditional large volume, complex operation seriously affects the function of immersive. This paper develops a multi-modal sensing interactive glove system for teleoperation and VR/AR. The device is small in size and convenient to wear; we integrate temperature sensing and tactile sensing into our glove system and deliver real-time and accurate environmental signals to the wearer through the acquisition and processing of information and program-controlled regulation to better realize the fusion of virtual and real. This paper first introduces the structure and working principle of the system and analyzes its performance. Then a sensory experiment based on human hand skin was designed, and volunteers were selected to experience it.
HaptGlove—Untethered Pneumatic Glove for Multimode Haptic Feedback in Reality–Virtuality Continuum

Jiaming Qi,Feng Gao,Guanghui Sun,Joo Chuan Yeo,Chwee Teck Lim

DOI: https://doi.org/10.1002/advs.202301044

IF: 15.1

2023-06-30

Advanced Science

Abstract:An untethered and lightweight pneumatic haptic glove is developed to enable realistic touch sensation in the reality–virtuality continuum. Using soft pneumatic actuators and stretchable fiber sensors, the glove renders both kinesthetic and cutaneous feedback with imperceptible delay. Significantly improved realism and user immersion are observed in user studies, demonstrating the viability in virtual training, education, and entertainment. Novel haptics technologies are urgently needed to bridge the gap between entirely physical world and fully digital environment to render a more realistic and immersive human–computer interaction. Current virtual reality (VR) haptic gloves either deliver limited haptic feedback or are bulky and heavy. The authors develop a haptic glove or HaptGlove, an untethered and lightweight pneumatic glove, that allows users to "physically" interact in a VR environment and enables both kinesthetic and cutaneous sensations naturally and realistically. Integrated with five pairs of haptic feedback modules and fiber sensors, HaptGlove provides variable stiffness force feedback and fingertip force and vibration feedback, allowing users to touch, press, grasp, squeeze, and pull various virtual objects and feel the dynamic haptic changes. Significant improvements in VR realism and immersion are observed in a user study with participants achieving 78.9% accuracy in sorting six virtual balls of different stiffnesses. Importantly, HaptGlove facilitates VR training, education, entertainment, and socialization in a reality–virtuality continuum.

materials science, multidisciplinary,nanoscience & nanotechnology,chemistry
Soft Modular Glove with Multimodal Sensing and Augmented Haptic Feedback Enabled by Materials' Multifunctionalities.

Zhongda Sun,Chengkuo Lee,Minglu Zhu

DOI: https://doi.org/10.1021/acsnano.2c04043

IF: 17.1

2022-08-23

ACS Nano

Abstract:Immersive communications rely on smart perception based on diversified and augmented sensing and feedback technologies. However, the increasing of functional components also raises the issue of increased system complexity. Here, we propose a modular soft glove with multimodal sensing and feedback functions by exploring and utilizing the multiple properties of glove materials. With a single design of basic structure, the main functional unit possesses triboelectric-based sensing of static and dynamic contact, vibration, strain, and pneumatic actuation. Additionally, the same unit is also capable of offering pneumatic tactile haptic feedback and electroresistive thermal haptic feedback. Together with a machine learning algorithm, the proposed glove not only performs real-time detection of dexterous hand motion and direct feedback but also realizes intelligent object recognition and augmented feedback, which significantly enhance the communication and perception of more comprehensive information. In general, this glove utilizes a facile designed sensing and feedback device to achieve dual-way and multimodal communication among humans, machines, and the virtual world via smart perceptions.

Engineering,Medicine,Materials Science
Adaptive tactile interaction transfer via digitally embroidered smart gloves

Yiyue Luo,Chao Liu,Young Joong Lee,Joseph DelPreto,Kui Wu,Michael Foshey,Daniela Rus,Tomás Palacios,Yunzhu Li,Antonio Torralba,Wojciech Matusik

DOI: https://doi.org/10.1038/s41467-024-45059-8

IF: 16.6

2024-01-29

Nature Communications

Abstract:Abstract Human-machine interfaces for capturing, conveying, and sharing tactile information across time and space hold immense potential for healthcare, augmented and virtual reality, human-robot collaboration, and skill development. To realize this potential, such interfaces should be wearable, unobtrusive, and scalable regarding both resolution and body coverage. Taking a step towards this vision, we present a textile-based wearable human-machine interface with integrated tactile sensors and vibrotactile haptic actuators that are digitally designed and rapidly fabricated. We leverage a digital embroidery machine to seamlessly embed piezoresistive force sensors and arrays of vibrotactile actuators into textiles in a customizable, scalable, and modular manner. We use this process to create gloves that can record, reproduce, and transfer tactile interactions. User studies investigate how people perceive the sensations reproduced by our gloves with integrated vibrotactile haptic actuators. To improve the effectiveness of tactile interaction transfer, we develop a machine-learning pipeline that adaptively models how each individual user reacts to haptic sensations and then optimizes haptic feedback parameters. Our interface showcases adaptive tactile interaction transfer through the implementation of three end-to-end systems: alleviating tactile occlusion, guiding people to perform physical skills, and enabling responsive robot teleoperation.

multidisciplinary sciences
Design of a lightweight and easy‐to‐wear hand glove with multi‐modal tactile perception for digital human

Zhigeng Pan,Hongyi Ren,Chang Liu,Ming Chen,Mithun Mukherjee,Wenzhen Yang

DOI: https://doi.org/10.1002/cav.2258

IF: 1.01

2024-06-01

Computer Animation and Virtual Worlds

Abstract:The gloves proposed in this paper integrate 16 sensors capable of detecting temperature, finger pressure, and bending data of the hand. Additionally, we have designed a small circuit board. The incorporation of a novel traversal polling method for the streamlined collection of multi‐channel sensor readings is innovative. The experimental results indicate that the data glove demonstrates a high degree of precision in acquiring real‐time hand interaction information, as well as effectively displaying hand posture in real‐time using Unity3D. Within the field of human–computer interaction, data gloves play an essential role in establishing a connection between virtual and physical environments for the realization of digital human. To enhance the credibility of human‐virtual hand interactions, we aim to develop a system incorporating a data glove‐embedded technology. Our proposed system collects a wide range of information (temperature, bending, and pressure of fingers) that arise during natural interactions and afterwards reproduce them within the virtual environment. Furthermore, we implement a novel traversal polling technique to facilitate the streamlined aggregation of multi‐channel sensors. This mitigates the hardware complexity of the embedded system. The experimental results indicate that the data glove demonstrates a high degree of precision in acquiring real‐time hand interaction information, as well as effectively displaying hand posture in real‐time using Unity3D. The data glove's lightweight and compact design facilitates its versatile utilization in virtual reality interactions.

computer science, software engineering
A Skin-Integrated Multimodal Haptic Interface for Immersive Tactile Feedback

Ya Huang,Jingkun Zhou,Pingchuan Ke,Xu Guo,Chun Ki Yiu,Kuanming Yao,Shaoyu Cai,Dengfeng Li,Yu Zhou,Jian Li,Tsz Hung Wong,Yiming Liu,Lei Li,Yuyu Gao,Xingcan Huang,Hu Li,Jiyu Li,Binbin Zhang,Zhenlin Chen,Huanxi Zheng,Xingyu Yang,Haichen Gao,Zichen Zhao,Enming Song,Hui Wu,Zuankai Wang,Zhaoqian Xie,Kening Zhu,Xinge Yu

DOI: https://doi.org/10.1038/s41928-023-01074-z

IF: 33.255

2023-01-01

Nature Electronics

Abstract:Haptic feedback can be used to improve immersion in virtual and augmented reality systems. However, wearable haptic devices typically provide limited feedback, such as simple vibration, and are restricted to a narrow bandwidth of tuneable frequency or to single tactile perceptions. Here, we report a skin-integrated wireless haptic interface that is based on actuator arrays and offers multimodal and complex feedback mechanisms. Different feedback modes-mechanical, electrotactile and thermal-in the interface are used to selectively activate different cutaneous receptors, providing users with diverse haptic sensations. The selective stimulating mechanism is inspired by the neural tactile sensing system and can reproduce tactile information of fine roughness, macro roughness, slipperiness, force and temperature. A haptic interface that uses thermal, mechanical and electrotactile modes of stimulation to target different receptors in the skin can provide users with diverse haptic sensations, reproducing the tactile information of fine roughness, macro roughness, slipperiness, force and temperature.
Minimalist Design for Multi-Dimensional Pressure-Sensing and Feedback Glove with Variable Perception Communication

Hao Ling,Jie Li,Chuanxin Guo,Yuntian Wang,Tao Chen,Minglu Zhu

DOI: https://doi.org/10.3390/act13110454

IF: 2.6

2024-11-16

Actuators

Abstract:Immersive human–machine interaction relies on comprehensive sensing and feedback systems, which enable transmission of multiple pieces of information. However, the integration of increasing numbers of feedback actuators and sensors causes a severe issue in terms of system complexity. In this work, we propose a pressure-sensing and feedback glove that enables multi-dimensional pressure sensing and feedback with a minimalist design of the functional units. The proposed glove consists of modular strain and pressure sensors based on films of liquid metal microchannels and coin vibrators. Strain sensors located at the finger joints can simultaneously project the bending motion of the individual joint into the virtual space or robotic hand. For subsequent tactile interactions, the design of two symmetrically distributed pressure sensors and vibrators at the fingertips possesses capabilities for multi-directional pressure sensing and feedback by evaluating the relationship of the signal variations between two sensors and tuning the feedback intensities of two vibrators. Consequently, both dynamic and static multi-dimensional pressure communication can be realized, and the vibrational actuation can be monitored by a liquid-metal-based sensor via a triboelectric sensing mechanism. A demonstration of object interaction indicates that the proposed glove can effectively detect dynamic force in varied directions at the fingertip while offering the reconstruction of a similar perception via the haptic feedback function. This device introduces an approach that adopts a minimalist design to achieve a multi-functional system, and it can benefit commercial applications in a more cost-effective way.

engineering, mechanical,instruments & instrumentation
A Seamless, Large‐Area Silk‐based Interface for Immersive On‐Palm Tactile Feedback

Chuan Cao,Haitao Liu,Ye Zhang,Weiqiang Jia,Lei Zhang,Hujun Bao

DOI: https://doi.org/10.1002/admt.202301599

IF: 6.8

2024-01-01

Advanced Materials Technologies

Abstract:Wearable haptic devices are developed as key components to enhance the highly immersive metaverse experience. However, conventional haptic devices enabling the sensation and interaction with virtual objects are often constrained by their bulky design, tethered operation, and physical interference with the skin, especially when integrated with human hands. In this study, a cross-interface modification process that achieves the conversion of multilayered silk fibroin films from rigid to soft is reported. This modification is favorable for applications involving large angles and multiple degrees of freedom motions. By applying electrical current through silk-based electrodes and utilizing a self-designed circuit, virtual tactile sensations can be generated on the entire palm. The high flexibility, stretchability, tailorable modulus, and water permeability of silk substrates enable the electrotactile devices to maintain their functionalities without impeding fine movements and natural tactile sensations in the hand. This compact electrotactile system serves as a user's new interactive terminal through wireless communications. The intimate contact interfaces reduce the unstable adhesion commonly encountered in electrotactile applications, ensuring consistent sensation during hand motions. These materials design methods and integrated systems represent a significant advancement in fabricating and integrating large-area soft electronics. Embark on a journey through the evolution of wearable haptic devices, unlocking the potential for an immersive metaverse experience. Explore the limitations of conventional devices and discover a cross-interface modification process in silk materials, transforming rigid silk fibroin films into soft, and versatile layers. Dive into the intricacies of electrotactile technology, enabling virtual tactile sensations on the entire palm. image
A Liquid Metal Based Multimodal Sensor and Haptic Feedback Device for Thermal and Tactile Sensation Generation in Virtual Reality

Jinhyeok Oh,Suin Kim,Sangyeop Lee,Seongmin Jeong,Seung Hwan Ko,Joonbum Bae

DOI: https://doi.org/10.1002/adfm.202007772

IF: 19

2020-12-16

Advanced Functional Materials

Abstract:<p>Virtual reality (VR) has been widely used for training, gaming, and entertainment, and the value of VR is continually increasing as a contact‐free technology. For an immersive VR experience, measuring finger movements and providing appropriate feedback to the hand are as important as visual information, given the necessity of the hands for activities in daily life. Thus, a hand‐worn VR device with motion sensors and haptic feedback is desirable. In this paper, a multimodal sensing and feedback glove is developed with soft, stretchable, lightweight, and compact sensor and heater sheets manufactured by direct ink writing (DIW) of liquid metal, eutectic gallium‐indium (eGaIn). In the sensor sheet, ten sensors and three vibrators are embedded to measure finger movements and provide vibro‐haptic feedback. The other heater sheet provides thermo‐haptic sensation in accurate and rapid manner via model‐based feedback control even under stretched conditions. The multimodal sensing and feedback glove allows users to feel the contact status and discriminate materials with different temperature. Performance of the proposed multimodal glove is verified under VR environments including touching and pushing two blocks of different materials and grabbing a heated metal ball submerged in hot water.</p>

materials science, multidisciplinary,chemistry, physical,physics, applied, condensed matter,nanoscience & nanotechnology
Augmented tactile-perception and haptic-feedback rings as human-machine interfaces aiming for immersive interactions

Zhongda Sun,Minglu Zhu,Xuechuan Shan,Chengkuo Lee

DOI: https://doi.org/10.1038/s41467-022-32745-8

IF: 16.6

2022-09-05

Nature Communications

Abstract:Abstract Advancements of virtual reality technology pave the way for developing wearable devices to enable somatosensory sensation, which can bring more comprehensive perception and feedback in the metaverse-based virtual society. Here, we propose augmented tactile-perception and haptic-feedback rings with multimodal sensing and feedback capabilities. This highly integrated ring consists of triboelectric and pyroelectric sensors for tactile and temperature perception, and vibrators and nichrome heaters for vibro- and thermo-haptic feedback. All these components integrated on the ring can be directly driven by a custom wireless platform of low power consumption for wearable/portable scenarios. With voltage integration processing, high-resolution continuous finger motion tracking is achieved via the triboelectric tactile sensor, which also contributes to superior performance in gesture/object recognition with artificial intelligence analysis. By fusing the multimodal sensing and feedback functions, an interactive metaverse platform with cross-space perception capability is successfully achieved, giving people a face-to-face like immersive virtual social experience.

multidisciplinary sciences
Triboelectric bending sensor based smart glove towards intuitive multi-dimensional human-machine interfaces

Yang Luo,Zihan Wang,Jiyu Wang,Xiao Xiao,Qian Li,Wenbo Ding,H.Y. Fu

DOI: https://doi.org/10.1016/j.nanoen.2021.106330

IF: 17.6

2021-11-01

Nano Energy

Abstract:<p>Recent advances in human-machine interface (HMI) lead to a renewed interest in creating intuitive and immersive interaction. Here, we designed a simple-structured and high-resolution bending angle triboelectric sensor named bending-angle triboelectric nanogenerator (BA-TENG) to construct a glove-based multi-dimensional HMI. With the assistance of a customized print circuit board (PCB), the glove-based HMI exhibits high sensitivity and low crosstalk in real-time multi-channel finger motion sensing. The signal-to-noise ratio (SNR) is improved by 19.36 dB. By systematically extracting and analyzing the multi-dimensional signal features of the BA-TENG, intuitive multi-dimensional HMIs were realized for smart-home, advanced robotic control, and a virtual keyboard with user recognition functionality. The classification accuracy of the virtual keyboard for seven users reached 93.1% by leveraging the advanced machine learning technique. The proposed BA-TENG-based smart glove reveals its potential as a solution for minimalist-design and intuitive multi-dimensional HMI, promising in diversified areas, including the Internet of things (IoT), assistive technology, and intelligent recognition systems.</p>

materials science, multidisciplinary,chemistry, physical,physics, applied,nanoscience & nanotechnology
A self-sensing soft pneumatic actuator with closed-Loop control for haptic feedback wearable devices

Meng Yu,Xiang Cheng,Shigang Peng,Yingze Cao,Yamei Lu,Bingyang Li,Xiangchao Feng,Yan Zhang,Haoyu Wang,Zhiwei Jiao,Pengfei Wang,Liangyu Zhao

DOI: https://doi.org/10.1016/j.matdes.2022.111149

2022-11-01

Abstract:The intuitive and effective haptic feedback interaction is essential for human–machine interfaces. However, most current commercial haptic feedback solutions are rigid, simplex, and insufficient to fully immerse users in virtual and teleoperated environments. Here, we report the design, fabrication and performance of a low-cost and lightweight self-sensing actuator (SenAct) that provides accurate force and vibration feedback to recreate the tactile perception. The SenAct shows good performance such as fast response (10 ms), high robustness (>10,000 cycles), and large output force (up to 1.55 N) with high controllable resolution (up to 0.02 N) based on the real-time closed-loop control. It is capable of providing the corresponding haptic feedback to the wearer through external force or vibration signal input. With this prototype, we presented a haptic feedback glove that supports precise operation by enhancing immersion and comprehensive sensation. Furthermore, we demonstrated it could shape appropriate interaction in different scenarios, including touching or grasping objects in virtual reality and teleoperation, illustrating its potential applications in human-in-loop interaction system and metaverse.

materials science, multidisciplinary
Pneumatic actuator and flexible piezoelectric sensor for soft virtual reality glove system

Kahye Song,Sung Hee Kim,Sungho Jin,Sohyun Kim,Sunho Lee,Jun-Sik Kim,Jung-Min Park,Youngsu Cha

DOI: https://doi.org/10.1038/s41598-019-45422-6

IF: 4.6

2019-07-18

Scientific Reports

Abstract:The desire to directly touch and experience virtual objects led to the development of a tactile feedback device. In this paper, a novel soft pneumatic actuator for providing tactile feedback is proposed and demonstrated. The suggested pneumatic actuator does not use an external air compressor but it is operated by internal air pressure generated by an electrostatic force. By using the actuator, we designed a glove to interact with virtual reality. The finger motions are detected by attached flexible piezoelectric sensors and transmitted to a virtual space through Bluetooth for interconnecting with a virtual hand. When the virtual finger touches the virtual object, the actuators are activated and give the tactile feedback to the real fingertip. The glove is made of silicone rubber material and integrated with the sensors and actuators such that users can wear them conveniently with light weight. This device was tested in a virtual chess board program, wherein the user picked up virtual chess pieces successfully.

multidisciplinary sciences
Skin-Integrated Haptic Interfaces Enabled by Scalable Mechanical Actuators for Virtual Reality

Yiming Liu,Chun Ki Yiu,Zhao Zhao,Shiyuan Liu,Xingcan Huang,Wooyoung Park,Jingyou Su,Jingkun Zhou,Tsz Hung Wong,Kuanming Yao,Ling Zhao,Ya Huang,Jiyu Li,Pu Fan,Binbin Zhang,Yuan Dai,Zhengbao Yang,Yuhang Li,Xinge Yu

DOI: https://doi.org/10.1109/jiot.2022.3203417

IF: 10.6

2022-12-24

IEEE Internet of Things Journal

Abstract:The very recent concept of metaverse highlights the importance of virtual reality (VR) and augmented reality (AR), which associates with a wide variety of applications in entertainment, medical treatment, and human–machine interfaces. The current VR/AR technologies mainly rely on visual interaction, while immersive experience in VR and AR highly demands sensational feedback, such as haptic and temperature with noticeable quality in wearable or even skin-integrated formarts. In this article, we report a wearable and flexible haptic interface based on electromagnetic vibrotactile actuators with high wearability and stability. By adopting double layers of copper (Cu) coils at the top and bottom of the magnetic disc, an enhanced electromagnetic field can be generated. Additionally, the intensity of the haptic feedback can be modulated according to sensed pressure in the virtual world by adjusting the value of power input and frequency. The actuator exhibits high stability and tolerance upon environmental, cyclic, and impact resistance tests. Finally, the actuators are developed into the soft VR interfaces for mounting on forearms, fingers, and hands to verify their superiority over conventional haptic actuators in the aspects of performance and applications.

computer science, information systems,telecommunications,engineering, electrical & electronic
Haptic interface with multimodal tactile sensing and feedback for human–robot interaction

Mingyu Kang,Cheol-Gu Gang,Sang-Kyu Ryu,Hyeon-Ju Kim,Da-Yeon Jeon,Soonjae Pyo

DOI: https://doi.org/10.1186/s40486-024-00199-w

2024-03-14

Micro and Nano Systems Letters

Abstract:Abstract Novel sensing and actuation technologies have notably advanced haptic interfaces, paving the way for more immersive user experiences. We introduce a haptic system that transcends traditional pressure-based interfaces by delivering more comprehensive tactile sensations. This system provides an interactive combination of a robotic hand and haptic glove to operate devices within the wireless communication range. Each component is equipped with independent sensors and actuators, enabling real-time mirroring of user’s hand movements and the effective transmission of tactile information. Remarkably, the proposed system has a multimodal feedback mechanism based on both vibration motors and Peltier elements. This mechanism ensures a varied tactile experience encompassing pressure and temperature sensations. The accuracy of tactile feedback is meticulously calibrated according to experimental data, thereby enhancing the reliability of the system and user experience. The Peltier element for temperature feedback allows users to safely experience temperatures similar to those detected by the robotic hand. Potential applications of this system are wide ranging and include operations in hazardous environments and medical interventions. By providing realistic tactile sensations, our haptic system aims to improve both the performance and safety of workers in such critical sectors, thereby highlighting the great potential of advanced haptic technologies.

English Else
A Reconfigurable Data Glove for Reconstructing Physical and Virtual Grasps

Hangxin Liu,Zeyu Zhang,Ziyuan Jiao,Zhenliang Zhang,Minchen Li,Chenfanfu Jiang,Yixin Zhu,Song-Chun Zhu

DOI: https://doi.org/10.48550/arXiv.2301.05821

2023-02-02

Abstract:In this work, we present a reconfigurable data glove design to capture different modes of human hand-object interactions, which are critical in training embodied artificial intelligence (AI) agents for fine manipulation tasks. To achieve various downstream tasks with distinct features, our reconfigurable data glove operates in three modes sharing a unified backbone design that reconstructs hand gestures in real time. In the tactile-sensing mode, the glove system aggregates manipulation force via customized force sensors made from a soft and thin piezoresistive material; this design minimizes interference during complex hand movements. The virtual reality (VR) mode enables real-time interaction in a physically plausible fashion: A caging-based approach is devised to determine stable grasps by detecting collision events. Leveraging a state-of-the-art finite element method (FEM), the simulation mode collects data on fine-grained 4D manipulation events comprising hand and object motions in 3D space and how the object's physical properties (e.g., stress and energy) change in accordance with manipulation over time. Notably, the glove system presented here is the first to use high-fidelity simulation to investigate the unobservable physical and causal factors behind manipulation actions. In a series of experiments, we characterize our data glove in terms of individual sensors and the overall system. More specifically, we evaluate the system's three modes by (i) recording hand gestures and associated forces, (ii) improving manipulation fluency in VR, and (iii) producing realistic simulation effects of various tool uses, respectively. Based on these three modes, our reconfigurable data glove collects and reconstructs fine-grained human grasp data in both physical and virtual environments, thereby opening up new avenues for the learning of manipulation skills for embodied AI agents.

Robotics,Artificial Intelligence,Human-Computer Interaction
3D Multimodal Sensing and Feedback Finger Case for Immersive Dual‐Way Interaction

Tao Chen,Zhiwei Dai,Ming Liu,Yudong Zhao,Hao Ling,Lining Sun,Haidong He,Chengkuo Lee,Minglu Zhu

DOI: https://doi.org/10.1002/admt.202301681

IF: 6.8

2024-01-20

Advanced Materials Technologies

Abstract:A soft finger case with three dimensional multi‐modal sensing and feedback functions is proposed. The device is featured with multi‐dimensional tactile and thermal feedback, triboelectric nanogenerator (TENG) tactile sensing with dynamic and static responses and control of movement. The integrated TENG tactile sensing unit in the device is capable of recognizing the texture of an object's surface, providing users with a more natural and intuitive way of interacting with digital objects. With the rapid development of virtual reality (VR) technology, artificial intelligence , and the Internet of Things , a new era of immersive interaction with the digital world has emerged. In this context, a novel soft finger case with 3D multi‐modal sensing and feedback functions (3D‐SFC) is presented. The device leverages multiple material properties to create a modulated fingertip haptic feedback for the perception of orientation and force, triboelectric nanogenerator (TENG) sensing with dynamic and static responses and control of movement, and thermal feedback based on resistive heating of TENG electrodes. The integrated TENG tactile sensing unit in the device is capable of recognizing the texture of an object's surface, providing users with a more natural and intuitive way of interacting with digital objects. The spatial consistency of multi‐modal sensing and feedback enhances human‐machine interaction in various fields such as healthcare, manufacturing and assembly, and gaming, etc. The proposed device represents a new intelligent sensing solution for achieving cross‐space sensory interaction, opening up new possibilities for the future of VR and digital interaction.

materials science, multidisciplinary
Suspended Three-Dimensional Display Haptic Interaction Based on Leap Motion

Li Wei,Li Haifeng,Xu Liang,Xi Tong

DOI: https://doi.org/10.3788/aos202040.0915004

2020-01-01

Abstract:Three-dimensional (3D) display has gained wide attention for its natural and intuitive performance. Herein, we proposed a finger sleeve haptic interaction method based on a suspended 3D display system. The fingertip coordinates were extracted by Leap Motion and a binocular camera was used to assist the coordinate transformation between light field display arca and interaction area. Then, the position relation between the hand and display arca was examined and the signal was transmitted to vibrate the finger sleeve, thus the haptic interaction was finally realized. The results show that the proposed method can be used to complete the coordinate transformation between light field display arca and interaction arca with a transformation precision of within 5 mm. Furthermore, when the user interacts with a 3D display system, the effective and real-time haptic interaction enhances their real sensory experiences.
Soft Robotic Glove with Sensing and Force Feedback for Rehabilitation in Virtual Reality

Fengguan Li,Jiahong Chen,Guanpeng Ye,Siwei Dong,Zishu Gao,Yitong Zhou

DOI: https://doi.org/10.3390/biomimetics8010083

2023-02-15

Abstract:Many diseases, such as stroke, arthritis, and spinal cord injury, can cause severe hand impairment. Treatment options for these patients are limited by expensive hand rehabilitation devices and dull treatment procedures. In this study, we present an inexpensive soft robotic glove for hand rehabilitation in virtual reality (VR). Fifteen inertial measurement units are placed on the glove for finger motion tracking, and a motor-tendon actuation system is mounted onto the arm and exerts forces on fingertips via finger-anchoring points, providing force feedback to fingers so that the users can feel the force of a virtual object. A static threshold correction and complementary filter are used to calculate the finger attitude angles, hence computing the postures of five fingers simultaneously. Both static and dynamic tests are performed to validate the accuracy of the finger-motion-tracking algorithm. A field-oriented-control-based angular closed-loop torque control algorithm is adopted to control the force applied to the fingers. It is found that each motor can provide a maximum force of 3.14 N within the tested current limit. Finally, we present an application of the haptic glove in a Unity-based VR interface to provide the operator with haptic feedback while squeezing a soft virtual ball.
Triboelectric-Inertial Sensing Glove Enhanced by Charge-Retained Strategy for Human-Machine Interaction.

Bo Yang,Jia Cheng,Xuecheng Qu,Yuning Song,Lifa Yang,Junyao Shen,Ziqian Bai,Linhong Ji

DOI: https://doi.org/10.1002/advs.202408689

2024-01-01

Abstract:As technology advances, human-machine interaction (HMI) demands more intuitive and natural methods. To meet this demand, smart gloves, capable of capturing intricate hand movements, are emerging as vital HMI tools. Moreover, triboelectric-based sensors, with their self-powered, cost-effective, and material various characteristics, can offer promising solutions for enhancing existing glove systems. However, a key limitation of these sensors is that charge leakage in the measurement circuit results in capturing only transient signals, rather than continuous changes. To address this issue, a charge-retained circuit that effectively prevents triboelectric signal attenuation is developed, enabling accurate measurement of continuous finger movements. This innovation forms the foundation of a highly integrated smart glove system, enhancing HMI functionality by combining continuous triboelectric signals with inertial sensor data. The system showcases a diverse range of applications, including complex robotic control, virtual reality interaction, smart home lighting adjustments, and intuitive interface operations. Furthermore, by leveraging artificial intelligence (AI) techniques, the system achieves accurate recognition of complex sign language with an impressive 99.38% accuracy. This work presents a charge-retained approach for continuous sensing with triboelectric-based sensors, offering valuable insights for developing future multifunctional HMI and sign language recognition systems. The proposed smart glove system, with its dual-mode sensing and AI integration, holds great potential for revolutionizing various domains and enhancing user experiences.

Haptic-feedback smart glove as a creative human-machine interface (HMI) for virtual/augmented reality applications

Multi-modal Sensing-Based Interactive Glove System for Teleoperation and VR/AR

HaptGlove—Untethered Pneumatic Glove for Multimode Haptic Feedback in Reality–Virtuality Continuum

Soft Modular Glove with Multimodal Sensing and Augmented Haptic Feedback Enabled by Materials' Multifunctionalities.

Adaptive tactile interaction transfer via digitally embroidered smart gloves

Design of a lightweight and easy‐to‐wear hand glove with multi‐modal tactile perception for digital human

A Skin-Integrated Multimodal Haptic Interface for Immersive Tactile Feedback

Minimalist Design for Multi-Dimensional Pressure-Sensing and Feedback Glove with Variable Perception Communication

A Seamless, Large‐Area Silk‐based Interface for Immersive On‐Palm Tactile Feedback

A Liquid Metal Based Multimodal Sensor and Haptic Feedback Device for Thermal and Tactile Sensation Generation in Virtual Reality

Augmented tactile-perception and haptic-feedback rings as human-machine interfaces aiming for immersive interactions

Triboelectric bending sensor based smart glove towards intuitive multi-dimensional human-machine interfaces

A self-sensing soft pneumatic actuator with closed-Loop control for haptic feedback wearable devices

Pneumatic actuator and flexible piezoelectric sensor for soft virtual reality glove system

Skin-Integrated Haptic Interfaces Enabled by Scalable Mechanical Actuators for Virtual Reality

Haptic interface with multimodal tactile sensing and feedback for human–robot interaction

A Reconfigurable Data Glove for Reconstructing Physical and Virtual Grasps

3D Multimodal Sensing and Feedback Finger Case for Immersive Dual‐Way Interaction

Suspended Three-Dimensional Display Haptic Interaction Based on Leap Motion

Soft Robotic Glove with Sensing and Force Feedback for Rehabilitation in Virtual Reality

Triboelectric-Inertial Sensing Glove Enhanced by Charge-Retained Strategy for Human-Machine Interaction.