Abstract:MOTIVATION:While generative models have shown great success in sampling high-dimensional samples conditional on low-dimensional descriptors (stroke thickness in MNIST, hair color in CelebA, speaker identity in WaveNet), their generation out-of-distribution poses fundamental problems due to the difficulty of learning compact joint distribution across conditions. The canonical example of the conditional variational autoencoder (CVAE), for instance, does not explicitly relate conditions during training and, hence, has no explicit incentive of learning such a compact representation.RESULTS:We overcome the limitation of the CVAE by matching distributions across conditions using maximum mean discrepancy in the decoder layer that follows the bottleneck. This introduces a strong regularization both for reconstructing samples within the same condition and for transforming samples across conditions, resulting in much improved generalization. As this amount to solving a style-transfer problem, we refer to the model as transfer VAE (trVAE). Benchmarking trVAE on high-dimensional image and single-cell RNA-seq, we demonstrate higher robustness and higher accuracy than existing approaches. We also show qualitatively improved predictions by tackling previously problematic minority classes and multiple conditions in the context of cellular perturbation response to treatment and disease based on high-dimensional single-cell gene expression data. For generic tasks, we improve Pearson correlations of high-dimensional estimated means and variances with their ground truths from 0.89 to 0.97 and 0.75 to 0.87, respectively. We further demonstrate that trVAE learns cell-type-specific responses after perturbation and improves the prediction of most cell-type-specific genes by 65%.AVAILABILITY AND IMPLEMENTATION:The trVAE implementation is available via github.com/theislab/trvae. The results of this article can be reproduced via github.com/theislab/trvae_reproducibility.

Transfer Learning for Underrepresented Music Generation

MGU-V: A Deep Learning Approach for Lo-Fi Music Generation Using Variational Autoencoders With State-of-the-Art Performance on Combined MIDI Datasets

Deep learning for music generation: challenges and directions

Improving Automatic Jazz Melody Generation by Transfer Learning Techniques

Music Generation System for Adversarial Training Based on Deep Learning

MG-VAE: Deep Chinese Folk Songs Generation with Specific Regional Style

Conditional Out-of-distribution Generation for Unpaired Data Using Transfer VAE.

Modulated Variational auto-Encoders for many-to-many musical timbre transfer

Deep Learning-Based Music Generation

Transfer Learning of Artist Group Factors to Musical Genre Classification

The Usage of Artificial Intelligence Technology in Music Education System Under Deep Learning

A Training-Free Approach for Music Style Transfer with Latent Diffusion Models

MIDI-VAE: Modeling Dynamics and Instrumentation of Music with Applications to Style Transfer

From West to East: Who can understand the music of the others better?

Melody generation based on deep ensemble learning using varying temporal context length

GGA-MG: Generative Genetic Algorithm for Music Generation

Using a Bi-directional LSTM Model with Attention Mechanism trained on MIDI Data for Generating Unique Music

An automatic music generation and evaluation method based on transfer learning

A Study of Transfer Learning in Music Source Separation

Language Models are Drummers: Drum Composition with Natural Language Pre-Training

MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategies