Abstract:In recent years, several Japanese companies have attempted to improve the efficiency of their meetings, which has been a significant challenge. For instance, voice recognition technology is used to considerably improve meeting minutes creation. In an automatic minutes-creating system, identifying the speaker to add speaker information to the text would substantially improve the overall efficiency of the process. Therefore, a few companies and research groups have proposed speaker estimation methods; however, it includes challenges, such as requiring advance preparation, special equipment, and multiple microphones. These problems can be solved by using speech sections that are extracted from lip movements and voice information. When a person speaks, voice and lip movements occur simultaneously. Therefore, the speaker’s speech section can be extracted from videos by using lip movement and voice information. However, when this speech section contains only voice information, the voiceprint information of each meeting participant is required for speaker identification. When using lip movements, the speech section and speaker position can be extracted without the voiceprint information. Therefore, in this study, we propose a speech-section extraction method that uses image and voice information in Japanese for speaker identification. The proposed method consists of three processes: i) the extraction of speech frames using lip movements, ii) the extraction of speech frames using voices, and iii) the classification of speech sections using these extraction results. We used video data to evaluate the functionality of the method. Further, the proposed method was compared with state-of-the-art techniques. The average F-measure of the proposed method is determined to be higher than that of the conventional methods that are based on state-of-the-art techniques. The evaluation results showed that the proposed method achieves state-of-the-art performance using a simpler process compared to the conventional method.

A Lost Speech Reconstruction Method Using Linguistic Information

Lost Speech Reconstruction Method Using Speech Recognition Based On Missing Feature Theory And Hmm-Based Speech Synthesis

A Packet Loss Concealment Algorithem Using Speech Recognition and Synthesis

Speech Bandwidth Extension Method Using Speech Recognition And Speech Synthesis

Speech Reconstruction from Silent Lip and Tongue Articulation by Diffusion Models and Text-Guided Pseudo Target Generation

Speech Acquisition and Recovery Based on Piezoelectric Effect in the Mmwave Band

Speech Reconstruction from Silent Tongue and Lip Articulation By Pseudo Target Generation and Domain Adversarial Training

Speech-Section Extraction Using Lip Movement and Voice Information in Japanese

Consecutive Packet Loss Concealment Algorithm Based on SELP Vocoder

Speech Recognition Methods over Internet

Speech Reconstruction for Mfcc-Based Low Bit-Rate Speech Coding

Fuzzy Cluster Analysis and Its Evaluation Method

SVoice

Let There Be Sound: Reconstructing High Quality Speech from Silent Videos

Synthesis of wideband speech from narrow‐band speech based on piecewise linear mapping

Artificial Bandwidth Extension For Speech Signals Using Speech Recogniton

High-quality Speech Recovery Through Soundproof Protections Via Mmwave Sensing

On Deep Speech Packet Loss Concealment: A Mini-Survey

SVoice: Enabling Voice Communication in Silence Via Acoustic Sensing on Commodity Devices.

VoiceFixer: A Unified Framework for High-Fidelity Speech Restoration