Abstract:Researchers present a silent speech interface optimized for human‐robot collaboration. Using an unobtrusive, conformal, transparent, and ion‐conductive electromyography electrode array, the system captures speech‐relevant muscle activities, enabling silent speech control of a robotic manipulator. Integrated with an optical hand‐tracking system, it facilitates natural robot control and collaborative tasks in noisy environments, enhancing human‐robot interaction. Silent speech interfaces offer an alternative and efficient communication modality for individuals with voice disorders and when the vocalized speech communication is compromised by noisy environments. Despite the recent progress in developing silent speech interfaces, these systems face several challenges that prevent their wide acceptance, such as bulkiness, obtrusiveness, and immobility. Herein, the material optimization, structural design, deep learning algorithm, and system integration of mechanically and visually unobtrusive silent speech interfaces are presented that can realize both speaker identification and speech content identification. Conformal, transparent, and self‐adhesive electromyography electrode arrays are designed for capturing speech‐relevant muscle activities. Temporal convolutional networks are employed for recognizing speakers and converting sensing signals into spoken content. The resulting silent speech interfaces achieve a 97.5% speaker classification accuracy and 91.5% keyword classification accuracy using four electrodes. The speech interface is further integrated with an optical hand‐tracking system and a robotic manipulator for human‐robot collaboration in both assembly and disassembly processes. The integrated system achieves the control of the robot manipulator by silent speech and facilitates the hand‐over process by hand motion trajectory detection. The developed framework enables natural robot control in noisy environments and lays the ground for collaborative human‐robot tasks involving multiple human operators.

Flexible Self-Powered Low-Decibel Voice Recognition Mask

Applying a Triboelectric Nanogenerator by Using Facial Mask for Flexible Touch Sensor

MicPro: Microphone-based Voice Privacy Protection

Self-powered Speech Recognition System for Deaf Users

All-weather, natural silent speech recognition via machine-learning-assisted tattoo-like electronics

Decoding Silent Speech Commands from Articulatory Movements Through Soft Magnetic Skin and Machine Learning

Wavoice: A mmWave-assisted Noise-resistant Speech Recognition SystemJust Accepted

Wavoice: an Mmwave-Assisted Noise-Resistant Speech Recognition System

Wavoice: an Mmwave-Assisted Noise-Resistant Speech Recognition System.

RF-Mic: Live Voice Eavesdropping Via Capturing Subtle Facial Speech Dynamics Leveraging RFID

A Machine Learning-Combined Flexible Sensor for Tactile Detection and Voice Recognition

Decoding Silent Speech Cues From Muscular Biopotential Signals for Efficient Human‐Robot Collaborations

Triboelectrically self-sensing respiratory ventilator masks for monitoring, diagnosis, therapy, and human-machine interaction

Non-Acoustic Speech Sensing System Based on Flexible Piezoelectric.

A Stretchable High-Density Electromyography Mems Sensor for Efficient Silent Speech Recognition

Triboelectric Nanogenerator Based Smart Electronics Via Machine Learning

Anti-interference Self-Powered Acoustic Fabric for Complex Acoustic Environments

3D printed triboelectric nanogenerator as self-powered human-machine interactive sensor for breathing-based language expression

RF-Mic

A Super-Flexible and High-Sensitive Epidermal Semg Electrode Patch for Silent Speech Recognition

Ultrathin Eardrum‐Inspired Self‐Powered Acoustic Sensor for Vocal Synchronization Recognition with the Assistance of Machine Learning