Abstract:The text-based speech editor allows the editing of speech through intuitive cutting, copying, and pasting operations to speed up the process of editing speech. However, the major drawback of current systems is that edited speech often sounds unnatural due to cut-copy-paste operation. In addition, it is not obvious how to synthesize records according to a new word not appearing in the transcript. This paper proposes a novel end-to-end text-based speech editing method called context-aware mask prediction network (CampNet). The model can simulate the text-based speech editing process by randomly masking part of speech and then predicting the masked region by sensing the speech context. It can solve unnatural prosody in the edited region and synthesize the speech corresponding to the unseen words in the transcript. Secondly, for the possible operation of text-based speech editing, we design three text-based operations based on CampNet: deletion, insertion, and replacement. These operations can cover various situations of speech editing. Thirdly, to synthesize the speech corresponding to long text in insertion and replacement operations, a word-level autoregressive generation method is proposed. Fourthly, we propose a speaker adaptation method using only one sentence for CampNet and explore the ability of few-shot learning based on CampNet, which provides a new idea for speech forgery tasks. The subjective and objective experiments on VCTK and LibriTTS datasets show that the speech editing results based on CampNet are better than TTS technology, manual editing, and VoCo method. We also conduct detailed ablation experiments to explore the effect of the CampNet structure on its performance. Finally, the experiment shows that speaker adaptation with only one sentence can further improve the naturalness of speech. Examples of generated speech can be found at <a class="link-external link-https" href="https://hairuo55.github.io/CampNet" rel="external noopener nofollow">this https URL</a>.

Just Speak It: Minimize Cognitive Load for Eyes-Free Text Editing with a Smart Voice Assistant

EyeSayCorrect: Eye Gaze and Voice Based Hands-free Text Correction for Mobile Devices

Commanding and Re-Dictation

Supporting Text Entry in Virtual Reality with Large Language Models

Improving Error Correction and Text Editing Using Voice and Mouse Multimodal Interface

Using Confidence Scores to Improve Eyes-free Detection of Speech Recognition Errors

Context-Aware Prosody Correction for Text-Based Speech Editing

Toward Interactive Dictation

CampNet: Context-Aware Mask Prediction for End-to-End Text-Based Speech Editing

Rewriting the Script: Adapting Text Instructions for Voice Interaction

Context-Aware Mask Prediction Network for End-to-End Text-Based Speech Editing

DiffEditor: Enhancing Speech Editing with Semantic Enrichment and Acoustic Consistency

Speech Editing -- a Summary

FluentEditor: Text-based Speech Editing by Considering Acoustic and Prosody Consistency

Research on Error Handling Techniques for Speech Interaction

Listen, Chat, and Edit: Text-Guided Soundscape Modification for Enhanced Auditory Experience

Using Large Language Models to Accelerate Communication for Users with Severe Motor Impairments

Effidit: Your AI Writing Assistant

Voice Attribute Editing with Text Prompt

Emotion Selectable End-to-End Text-based Speech Editing

“The less I type, the better”: How AI Language Models can Enhance or Impede Communication for AAC Users