All-weather, natural silent speech recognition via machine-learning-assisted tattoo-like electronics
Youhua Wang,Tianyi Tang,Yin Xu,Yunzhao Bai,Lang Yin,Guang Li,Hongmiao Zhang,Huicong Liu,YongAn Huang
DOI: https://doi.org/10.1038/s41528-021-00119-7
IF: 14.6
2021-08-13
npj Flexible Electronics
Abstract:Abstract The internal availability of silent speech serves as a translator for people with aphasia and keeps human–machine/human interactions working under various disturbances. This paper develops a silent speech strategy to achieve all-weather, natural interactions. The strategy requires few usage specialized skills like sign language but accurately transfers high-capacity information in complicated and changeable daily environments. In the strategy, the tattoo-like electronics imperceptibly attached on facial skin record high-quality bio-data of various silent speech, and the machine-learning algorithm deployed on the cloud recognizes accurately the silent speech and reduces the weight of the wireless acquisition module. A series of experiments show that the silent speech recognition system (SSRS) can enduringly comply with large deformation (~45%) of faces by virtue of the electricity-preferred tattoo-like electrodes and recognize up to 110 words covering daily vocabularies with a high average accuracy of 92.64% simply by use of small-sample machine learning. We successfully apply the SSRS to 1-day routine life, including daily greeting, running, dining, manipulating industrial robots in deafening noise, and expressing in darkness, which shows great promotion in real-world applications.
materials science, multidisciplinary,engineering, electrical & electronic