https://csteinmetz1.github.io/steerable-nafx/

Steerable discovery of neural audio effects

Christian J. Steinmetz and Joshua D. Reiss

Centre for Digital Music, Queen Mary University of London

Paper * GitHub * Colab

[steerable-]

Abstract

---------------------------------------------------------------------

Applications of deep learning for audio effects often focus on
modeling analog effects or learning to control effects to emulate a
trained audio engineer. However, deep learning approaches also have
the potential to expand creativity through neural audio effects that
enable new sound transformations. While recent work demonstrated that
neural networks with random weights produce compelling audio effects,
control of these effects is limited and unintuitive. To address this,
we introduce a method for the steerable discovery of neural audio
effects. This method enables the design of effects using example
recordings provided by the user. We demonstrate how this method
produces an effect similar to the target effect, along with
interesting inaccuracies, while also providing perceptually relevant
controls.

Demo

---------------------------------------------------------------------

Examples

---------------------------------------------------------------------
Below are a number of example effects generated following the process
described in the paper.
Pre-trained models from different effect types are available for
usage in the Colab notebook.
---------------------------------------------------------------------

Reverb

     Description       c0 c1
Vocal

Clean vocal            -  -
Default reverb         0  0
Shorter reverb         -2 1
Longer reverb          -1 5
Distortion reverb      -7 10
Electric Guitar

Clean electric guitar  -  -
Large room             -7 10
Small room             1  1

Compressor

    Description      c0  c1
Drum kit

Clean drum kit       -   -
Default compression  0   0
Bassy compression    0.2 -1
More compression     0   0

Analog Delay

    Description       c0  c1
Gated Synth

Clean gated synth     -  -
Default delay         0  0
Gritty delay          -3 -3
Metallic delay        10 0
Beat

Clean beat            -  -
Wide delay            0  0
Rumble delay          -7 5
Train in the station  10 -5.5

Guitar Amplifier

     Description       c0 c1
Electric Guitar

Clean electric guitar  -  -
Amp slapback           0  0
Soft fuzz slap         -1 -1
Tunnel                 10 -10

Sound matching (Synth to Synth)

  Description     c0 c1
Piano

Clean piano       -  -
Long cascade      0  0
Fuzzy cascade     -1 0
Heavenly Tunnel   6  6

Paper

---------------------------------------------------------------------
[paper_sm]

Bibtex

---------------------------------------------------------------------


  @inproceedings{steinmetz2021steerable,
    title={Steerable discovery of neural audio effects},
    author={Steinmetz, Christian J. and Reiss, Joshua D.},
    booktitle={5th Workshop on Machine Learning for Creativity and Design at NeurIPS},
    year={2021}}


---------------------------------------------------------------------

*Accepted to the NeurIPS 2021 Workshop on Machine Learning for
Creativity and Design
Send feedback and questions to Christian Steinmetz.

[qm] [aim] [ukri]

Supported by the EPSRC UKRI Centre for Doctoral Training in
Artificial Intelligence and Music (EP/S022694/1).