Abstract:Audio super-resolution aims to enhance low-resolution signals by creating high-frequency content. In this work, we modify the architecture of AERO (a state-of-the-art system for this task) for music super-resolution. SPecifically, we replace its original Attention and LSTM layers with Mamba, a State Space Model (SSM), across all network layers. Mamba is capable of effectively substituting the mentioned modules, as it offers a mechanism similar to that of Attention while also functioning as a recurrent network. With the proposed AEROMamba, training requires 2-4x less GPU memory, since Mamba exploits the convolutional formulation and leverages GPU memory hierarchy. Additionally, during inference, Mamba operates in constant memory due to recurrence, avoiding memory growth associated with Attention. This results in a 14x speed improvement using 5x less GPU. Subjective listening tests (0 to 100 scale) show that the proposed model surpasses the AERO model. In the MUSDB dataset, degraded signals scored 38.22, while AERO and AEROMamba scored 60.03 and 66.74, respectively. For the PianoEval dataset, scores were 72.92 for degraded signals, 76.89 for AERO, and 84.41 for AEROMamba.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is audio super - resolution, that is, enhancing low - resolution audio signals by generating high - frequency content to improve audio quality. Specifically, the author proposes a new architecture named AEROMamba, aiming to improve the performance of the existing state - of - the - art model AERO in the music super - resolution task. ### Main problems and goals: 1. **Improve audio quality**: - Low - resolution audio signals usually sound dull, so it is necessary to improve the clarity and sound quality of the audio by reconstructing the high - frequency part. 2. **Optimize the use of computing resources**: - Existing methods (such as AERO) require a large amount of GPU memory during training and inference, and there is a problem of memory growth, especially when using the attention mechanism. Therefore, the author hopes to find a more efficient alternative. 3. **Improve the model architecture**: - The AERO model uses the Attention mechanism and the Bidirectional Long - Short - Term Memory Network (BiLSTM). Although these modules are effective, they are less efficient. The author proposes to replace these modules with the State Space Model (SSM) Mamba to improve the efficiency and performance of the model. ### Specific improvement measures: - **Introduce Mamba layer**: Mamba is a sequence - modeling method based on the state - space model. It can not only select important information like the attention mechanism, but also operate as a recurrent network, thus avoiding the memory growth problem brought by the attention mechanism. - **Reduce GPU memory usage**: By using the Mamba layer, the GPU memory required for the training process is reduced by 2 to 4 times, and only constant memory is required during the inference process, avoiding memory growth. - **Accelerate the inference speed**: After using the Mamba layer, the inference speed is increased by 14 times, and the GPU resources used are reduced by 5 times. ### Experimental results: - On the MUSDB dataset, the subjective score of the degraded signal is only 38.22, while the scores of AERO and AEROMamba are 60.03 and 66.74 respectively. - On the PianoEval dataset, the score of the degraded signal is 72.92, and the scores of AERO and AEROMamba are 76.89 and 84.41 respectively. In conclusion, the main purpose of this paper is to improve the AERO model by introducing the Mamba layer, so as to achieve more efficient audio super - resolution, and at the same time significantly improve audio quality and computational efficiency.

AEROMamba: An efficient architecture for audio super-resolution using generative adversarial networks and state space models

AERO: Audio Super Resolution in the Spectral Domain

SSAMBA: Self-Supervised Audio Representation Learning with Mamba State Space Model

Audio Mamba: Bidirectional State Space Model for Audio Representation Learning

Wave-U-Mamba: An End-To-End Framework For High-Quality And Efficient Speech Super Resolution

Frequency-Assisted Mamba for Remote Sensing Image Super-Resolution

Leveraging Joint Spectral and Spatial Learning with MAMBA for Multichannel Speech Enhancement

MambaFoley: Foley Sound Generation using Selective State-Space Models

MambaFormerSR: A Lightweight Model for Remote-Sensing Image Super-Resolution

$\text{S}^{3}$Mamba: Arbitrary-Scale Super-Resolution via Scaleable State Space Model

SepMamba: State-space models for speaker separation using Mamba

Hi-Mamba: Hierarchical Mamba for Efficient Image Super-Resolution

RawBMamba: End-to-End Bidirectional State Space Model for Audio Deepfake Detection

S^3Mamba: Arbitrary-Scale Super-Resolution Via Scaleable State Space Model

Selective State Space Model for Monaural Speech Enhancement

Audio Mamba: Selective State Spaces for Self-Supervised Audio Representations

MambaIRv2: Attentive State Space Restoration

Audio Mamba: Pretrained Audio State Space Model For Audio Tagging

Mamba State-Space Models Are Lyapunov-Stable Learners

CleanUMamba: A Compact Mamba Network for Speech Denoising using Channel Pruning