Abstract:Speech compression attempts to yield a compact bitstream that can represent a speech signal with minimal distortion by eliminating redundant information, which is increasingly challenging as the bitrate decreases. However, existing neural speech codecs do not fully exploit the information from previous speech sequences, and learning encoded features blindly leads to the ineffective removal of redundant information, resulting in suboptimal reconstruction quality. In this work, we propose an end-to-end speech codec with intra-inter broad attention, named IBACodec, that efficiently compresses speech across different types of datasets, including LibriTTS, LJSpeech, and more. By designing an intra-inter broad transformer that integrates multi-head attention networks and LSTM, our model captures broad attention with direct context awareness between the intra- and inter-frames of speech. Furthermore, we present a dual-branch conformer for channel-wise modeling to effectively eliminate redundant information. In subjective evaluations using speech at a 24 kHz sampling rate, IBACodec at 6.3 kbps is comparable to SoundStream at 9 kbps and better than Opus at 9 kbps, with about 30 % fewer bits. Objective experimental results show that IBACodec outperforms state-of-the-art codecs across a wide range of bitrates, with an average ViSQOL, LLR, and CEP improvement of up to 4.97 %, 38.94 %, and 25.39 %, respectively.

Research of The Key Technology of iLBC Speech Codec

Key Technology Research And Analysis Of H.264/Avc Suitable For Wireless Network

A Audio Coding Algorithm for Interactive Communications

Research on real time low bit-rate speech coding

Research on Low Delay Low Bit Rate Speech Coding Algorithm

Studying Performance For The Gain Filters Of 8kbit/S Ld-Acelp

Research on Speech Coding Algorithm at Very Low Bit Rate for Narrow-Band Communication

An Intra-BRNN and GB-RVQ Based END-TO-END Neural Audio Codec

IBACodec: End-to-end speech codec with intra-inter broad attention

LSCodec: Low-Bitrate and Speaker-Decoupled Discrete Speech Codec

An Enhanced Interleaving Frame Loss Concealment Method for Voice Over IP Network Services

A CELP-Speech Information Hiding Algorithm Based on Vector Quantization

8kbit/S Low Delay Speech Coding Algorithm With Adaptive Codebook

Techniques Of Very Low Bit-Rate Speech Coding

Low Frame-rate Speech Codec: a Codec Designed for Fast High-quality Speech LLM Training and Inference

High quality 8kb/s ACELP coder and its real-time realization

Steganography in Low Bit-Rate Speech Streams Based on Quantization Index Modulation Controlled by Keys

The Study On The Layered Coding System For Very Low Bit Rate Videophone

Algorithm of 8 kbit/s low delay speech coding-LD-ACELP

A New and Efficient Audio Compression Based on Ebcot and Rvlc

Efficient LSP Quantization Algorithm for Very Low Bit Rate Speech Coding