# MelGAN

## Generative Adversarial Networks for Conditional Waveform Synthesis.

# Table of contents

- Abstract
- Results
  - Spectrogram Inversion on Unseen speakers
  - TTS examples
  - Unconditional Music Synthesis
  - Music Translation
  - Samples along Training
  - Ablation

# Abstract

Previous works have found that generating coherent raw audio waveforms with GANs is challenging. In this paper, we show that it is possible to train GANs reliably to generate high quality coherent waveforms by introducing a set of architectural changes and simple training techniques. Two subjective evaluation metrics (Mean Opinion Score and MUSHRA) suggest that our model is state-of-the-art for mel-spectrogram inversion. We show qualitative results of our model on speech synthesis, music domain translation and unconditional music synthesis, to establish the generality of the proposed techniques. We also evaluate different components of the model, proposing a set of guidelines for designing general purpose discriminators and generators for conditional sequence synthesis tasks. Our model is non-autoregressive, fully convolutional, with significantly fewer parameters as compared to competing models and generalizes to unseen speakers for mel-spectrogram inversion. Our pytorch implementation runs at more than 100x faster than realtime on GTX 1080Ti GPU and more than 2x faster than realtime on CPU, without any hardware specific optimization tricks.

# Results

## Spectrogram Inversion on Unseen Speakers

Original

Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element.

Reconstructed

Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element.

## End-to-end text-to-speech examples

Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element.

Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element.

Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element.

## Unconditional Music Synthesis

Original

Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element.

Reconstructed

Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element.

Sampled

Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element. Your browser does not support the audio element.

## Music Translation

Example for source domain: Bach Solo Cello

|  | Beethoven accompanied violin | Beethoven solo piano
--- | --- | --- | ---
| Original | Mor et al. 2019 | Ours | Mor et al. 2019 | Ours
1 | Your browser does not support the audio element. | Your browser does not support the audio element. | Your browser does not support the audio element. | Your browser does not support the audio element. | Your browser does not support the audio element.
2 | Your browser does not support the audio element. | Your browser does not support the audio element. | Your browser does not support the audio element. | Your browser does not support the audio element. | Your browser does not support the audio element.
3 | Your browser does not support the audio element. | Your browser does not support the audio element. | Your browser does not support the audio element. | Your browser does not support the audio element. | Your browser does not support the audio element.
4 | Your browser does not support the audio element. | Your browser does not support the audio element. | Your browser does not support the audio element. | Your browser does not support the audio element. | Your browser does not support the audio element.
5 | Your browser does not support the audio element. | Your browser does not support the audio element. | Your browser does not support the audio element. | Your browser does not support the audio element. | Your browser does not support the audio element.

## Samples along Training

50 epochs - 1.35 hours

Your browser does not support the audio element.

100 epochs - 2.71 hours

Your browser does not support the audio element.

200 epochs - 5.42 hours

Your browser does not support the audio element.

400 epochs - 10.84 hours

Your browser does not support the audio element.

800 epochs - 21.68 hours

Your browser does not support the audio element.

1600 epochs - 43.36 hours

Your browser does not support the audio element.

3200 epochs - 86.72 hours

Your browser does not support the audio element.

## Ablation

original

Your browser does not support the audio element. Your browser does not support the audio element.

Baseline

Your browser does not support the audio element. Your browser does not support the audio element.

l1_observed_no_feat_match

Your browser does not support the audio element. Your browser does not support the audio element.

l1_observed_space

Your browser does not support the audio element. Your browser does not support the audio element.

no_dilations

Your browser does not support the audio element. Your browser does not support the audio element.

no_group_disc

Your browser does not support the audio element. Your browser does not support the audio element.

no_multiscale_disc

Your browser does not support the audio element. Your browser does not support the audio element.

no_patch_gan

Your browser does not support the audio element. Your browser does not support the audio element.

no_weight_norm

Your browser does not support the audio element. Your browser does not support the audio element.

spectral_norm

Your browser does not support the audio element. Your browser does not support the audio element.