Abstract:Deep learning (DL) based semantic communication methods have been explored for the efficient transmission of images, text, and speech in recent years. In contrast to traditional wireless communication methods that focus on the transmission of abstract symbols, semantic communication approaches attempt to achieve better transmission efficiency by only sending the semantic-related information of the source data. In this paper, we consider semantic-oriented speech transmission which transmits only the semantic-relevant information over the channel for the speech recognition task, and a compact additional set of semantic-irrelevant information for the speech reconstruction task. We propose a novel end-to-end DL-based transceiver which extracts and encodes the semantic information from the input speech spectrums at the transmitter and outputs the corresponding transcriptions from the decoded semantic information at the receiver. In particular, we employ a soft alignment module and a redundancy removal module to extract only the text-related semantic features while dropping semantically redundant content, greatly reducing the amount of semantic redundancy compared to existing methods. We also propose a semantic correction module to further correct the predicted transcription with semantic knowledge by leveraging a pretrained language model. For the speech to speech transmission, we further include a CTC alignment module that extracts a small number of additional semantic-irrelevant but speech-related information, such as duration, pitch, power and speaker identification of the speech for the better reconstruction of the original speech signals at the receiver. We also introduce a two-stage training scheme which speeds up the training of the proposed DL model. The simulation results confirm that our proposed method outperforms current methods in terms of the accuracy of the predicted text for the speech to text transmission and the quality of the recovered speech signals for the speech to speech transmission, and significantly improves transmission efficiency. More specifically, the proposed method only sends 16% of the amount of the transmitted symbols required by the existing methods while achieving about a 10% reduction in WER for the speech to text transmission. For the speech to speech transmission, it results in an even more remarkable improvement in terms of transmission efficiency with only 0.2% of the amount of the transmitted symbols required by the existing method while preserving the comparable quality of the reconstructed speech signals.

A Low-Complexity Transceiver Design for Terahertz Communication based on Deep Learning

Nonlinearity-aware Optoelectronic Terahertz Discrete Multitone Signal Transmission with a Zero-Bias Diode

Transfer Generative Adversarial Networks (T-GAN)-based Terahertz Channel Modeling

ChannelGAN: Deep Learning-Based Channel Modeling and Generating

Semantic-preserved Communication System for Highly Efficient Speech Transmission

Deep Learning Equalizer Connected with Viterbi-Viterbi Algorithm for PAM D-Band Radio over Fiber Link

Joint Transceiver Optimization for Wireless Communication PHY Using Neural Network.

Joint Transceiver Optimization for Wireless Communication PHY with Convolutional Neural Network

Deep Learning-Assisted Demodulation for TeraHertz Communications under Hybrid Distortions

Deep Learning based Modeling of Wireless Communication Channel with Fading

Model-based Deep Learning for End-to-end Optimization in the Fiber-Terahertz Communication System

Task-Oriented Communication with Reliability-Driven Retransmission Request

Hybrid Beamforming for RIS-Empowered Multi-hop Terahertz Communications: A DRL-based Method

Terahertz Wireless Communications for 2030 and Beyond: A Cutting-Edge Frontier

Quantum optical techniques for quality data transmission process in cognitive networks

Deep Learning Enabled Semantic Communication Systems for Video Transmission

Toward Intelligent Wireless Communications: Deep Learning - Based Physical Layer Technologies

Autoencoder with Fitting Network for Terahertz Wireless Communications: A Deep Learning Approach

Sensing Integrated DFT-Spread OFDM Waveform and Deep Learning-Powered Receiver Design for Terahertz Integrated Sensing and Communication Systems

Model-Driven Deep Learning for Physical Layer Communications

Deep Learning-based Design of Uplink Integrated Sensing and Communication