Digital Synthesis: Trends and Developments

Combining Approaches

In the 1990s and early 2000s, most known synthesis algorithms existed and provided more and more convincing results, due to the increasing computational power. In order to overcome the drawbacks of individual synthesis approaches, paradigms are combined and novel, hybrid approaches are created.

Deep Learning & Neural Nets

Deep learning and neural nets have been used as helper tools in sound synthesis for many years. However, the direct use for the generation of sound is rather new and currently the hot topic in sound synthesis and processing.

Control and Mapping

Although the control of sound synthesis is not a new topic, it remains one of the most active ones. Synthesis algorithms are able to produce a large variety of sounds in real-time since the 1990s, but their integration into musical instruments is a much wider topic, with many possibilities to explore and stil lags behind.

Digital Synthesis: Applications

Applications

While every digital sound synthesis approach stands for itself due to its unique characteristics, they all come with inherent strengths and limitations with regard to specific tasks and applications. [Fig.1] shows a list of synthesis goals with suitable approaches by Misra et al. (2009). In general, frequency-domain methods are less suited for time-critical tasks, involving transients and textures. Also, the table in Fig.1 suggests the versatility of granular and concatenative synthesis.

/images/Sound_Synthesis/taxonomy_42.png
[Fig.1] Synthesis goals and suitable approaches (Misra et al, 2009)

Many methods for sound synthesis - analog or digital - have gained their status as an indipendent musical instrument, with characteristic sound properties. Some shaped the development of popular music and spawned new musical genres. This includes sampling - with a close link to rap music, subractive synthesis in many ways and as backbone of techno music, and FM synthesis - with the DX7 literally playing a part in most 1980s pop hits. Such influential synthesizers have again been synthesized. Virtual analog - also analog modeling - emulates vintage subtractive synthesizers in hard- and software and software synthesizers simulate classic FM synthesizers.


References

  • John Chowning. Turenas: the realization of a dream. Proc. of the 17es Journées d’Informatique Musicale, Saint-Etienne, France, 2011.
    [BibTeX▼]
  • Bilbao, Stefan. Numerical Sound Synthesis. Wiley Online Library, 2009. ISBN 9780470749012. doi:10.1002/9780470749012.
    [BibTeX▼]
  • Ananya Misra and Perry R Cook. Toward Synthesized Environments: A Survey of Analysis and Synthesis Methods for Sound Designers and Composers. In Proceedings of the International Computer Music Conference (ICMC 2009). 2009.
    [BibTeX▼]
  • Tae Hong Park. An interview with max mathews. Computer Music Journal, 33(3):9–22, 2009.
    [BibTeX▼]
  • Julius O. Smith. Viewpoints on the History of Digital Synthesis. In Proceedings of the International Computer Music Conference, 1–10. 1991.
    [BibTeX▼]
  • Curtis Roads and Max Mathews. Interview with max mathews. Computer Music Journal, 4(4):15–22, 1980.
    [BibTeX▼]
  • Max V Mathews. The Digital Computer as a Musical Instrument. Science, 142(3592):553–557, 1963.
    [BibTeX▼]
  • Digital Synthesis: History & Taxonomy

    Family Tree

    The synthesis experiments at Bell Labs are the origin of most methods for digital sound synthesis. [Fig.1] illustrates the relations for a subset of synthesis approaches, starting with Mathews. The foundation for many further developments was laid when John Chowning brought the software MUSIC VI to Stanford from a visit at Bell Labs (Chowning, 2011). After migrating it to a PDP-6 computer, Chowning worked on his groundbreaking digital compositions, using the FM method and spatial techniques.

    /images/Sound_Synthesis/bilbao_history.png
    [Fig.1] Evolution and family tree (Bilbao, 2009).

    Taxonomy

    Digital methods for sound synthesis can be grouped according to their underlying principle of operation. In 1991, Smith proposed four basic categories, shown in [Fig.2].

    /images/Sound_Synthesis/synthesis_taxonomy_SMITH.png
    [Fig.2] Taxonomy of synthesis algorithms (Smith, 1991).

    Processed Recording

    Already a technique in the analog domain, more precisely in Musique Concrète, this family of synthesis approaches makes direct use of previously recorded sound for synthesis. This can be the playback of complete sounds or the extraction of short segments, such as grains or a single period of a sound.

    Spectral Models

    Spectral models use mathmatical means for expressing the spectra of sounds and their devopment over time. They are usually receiver-based, since they model the sound as it is heard, not as it is produced. This paradigm already existed in the mechanical world, as used by Hermann von Helmholtz in the 19th century and is based on even older signal models.

    Physical Models

    Physical Models are based on virtual acoustical and mechanical units, realized through buffers and LTI systems. Oscillators, resonating bodies and acoustic conductors are thus combined as in the mechanical domain. Physical modeling is regarded a source-based approach, since it deals with the actual sound production.

    Abstract Algorithm

    If it is not processed sound, a spectral model or a physical model, it is an abstract algorithm. Algorithms from this category transfer methods from other domains, like message transmission, to the musical domain.

    Missing Recent Approaches

    Although a few categorisations could be debated, the above introduced taxonomy is still valid but misses some recent developments. Methods based on neural networks and deep learning for sound generation may be considered a fifth taxon.


  • John Chowning. Turenas: the realization of a dream. Proc. of the 17es Journées d’Informatique Musicale, Saint-Etienne, France, 2011.
    [BibTeX▼]
  • Bilbao, Stefan. Numerical Sound Synthesis. Wiley Online Library, 2009. ISBN 9780470749012. doi:10.1002/9780470749012.
    [BibTeX▼]
  • Ananya Misra and Perry R Cook. Toward Synthesized Environments: A Survey of Analysis and Synthesis Methods for Sound Designers and Composers. In Proceedings of the International Computer Music Conference (ICMC 2009). 2009.
    [BibTeX▼]
  • Tae Hong Park. An interview with max mathews. Computer Music Journal, 33(3):9–22, 2009.
    [BibTeX▼]
  • Julius O. Smith. Viewpoints on the History of Digital Synthesis. In Proceedings of the International Computer Music Conference, 1–10. 1991.
    [BibTeX▼]
  • Curtis Roads and Max Mathews. Interview with max mathews. Computer Music Journal, 4(4):15–22, 1980.
    [BibTeX▼]
  • Max V Mathews. The Digital Computer as a Musical Instrument. Science, 142(3592):553–557, 1963.
    [BibTeX▼]
  • Faust: OSC Control

    For standalone standalone and embedded plugins, OSC control can be a useful option. Faust compiles with OSC functionality with the -osc flag:

    $ faust2jaqt -osc sine.dsp
    

    Starting the program in the command line gives information on the standard faust OSC ports:

    $ ./sine
    Faust OSC version 1.22 - 'sine' is running on UDP ports 5510, 5511, 5512, sending on localhost
    

    The running program can now be controlled through any OSC sender, using the proper IP address, port and paths:

    /images/Sound_Synthesis/sine_osc_example.png

    Faust: Compile for Puredata

    $ faust2puredata sine.dsp
    

    On linux systems this will create a file sine~.pd_linux, on MacOS a file sine~.pd_darwin. Place the file in your PD search paths and use it, as in the help file sine~-help.pd. Parameters are set with messages.

    /images/Sound_Synthesis/sine_pd_example.png

    Faust: Compile for SuperCollider

    Faust can be used to compile SuperCollider extensions. For the sine.dsp example in the introduction:

    $ faust2supercollider sine.dsp
    

    This will produce two files:

    • the class file sine.sc
    • the binary sine.so

    Note that faust2supercollider depends on ruby, which you may need to install. If missing, the .sc files will be empty. There are now warnings or errors, so this can be confusing.

    Both files need to be placed in the system's SuperCollider extension directory and the class library needs to be recompiled. The class name in SuperColldier is generated by Faust:

    FaustSine : UGen
    {
    *ar { | frequency(100.0), gain(0.0) |
    ^this.multiNew('audio', frequency, gain)
    }
    
    *kr { | frequency(100.0), gain(0.0) |
    ^this.multiNew('control', frequency, gain)
    }
    
    name { ^"FaustSine" }
    
    
    info { ^"Generated with Faust" }
    }
    

    The new class can be used like this:

    {FaustSine.ar(100,1)}.play;
    

    Faust: The sine Example

    This first example implements a sine wave generator with controllable frequency and amplitude. It uses two horizontal sliders for control.

    import("stdfaust.lib");
    
    // input parameters with GUI elements
    freq  = hslider("frequency",100, 10, 1000, 0.001);
    gain  = hslider("gain",0, 0, 1, 0.001);
    
    // a sine oscillator with controllable freuency and amplitude:
    process = os.osc(freq)*gain;
    

    Faust: Quick Introduction

    Faust is a functional audio programming language, developed at GRAME, Lyon. It is a community-driven, free open source project. Faust is specifically suited for quickly designing musical synthesis and processing software and compiling it for a large variety of targets.

    Tools for Working with Faust

    Working with Faust offers a large toolbox. Downloads are found here:

    https://faust.grame.fr/downloads/index.html

    Faust IDE

    The Faust IDE is the fastest way to develop and test .dsp code: https://faustide.grame.fr/

    Faust Compiler

    The Faust compiler is the center-piece of the Faust development tools. Faust code is written in *.dsp files, which are converted to C++ code and then compiled for the desired system. The Faust compiler can be called directly from the command line or by other programs, such as the IDE FaustWorks. When called from the command line, Faust is compiled to the desired target using a faust2* command, which actually calls a build script.

    Depending on your operating system and build target, additional libraries or tools may be required. Targets of interest can be:

    Linux

    • faust2alsaconsole : ALSA command line program
    • faust2alqt : ALSA application with Qt UI
    • faust2alsa : ALSA application with GTK UI

    Plugins

    • faust2ladspa : LADSPA plug-in
    • faust2lv2 : LV2 plug-in
    • faust2faustvst : VST plug-in
    • faust2au : Audio Unit plugin

    Music Programming Environments

    • faust2supercollider : SuperCollider external
    • faust2puredata : PureData external
    • faust2max6 : MaxMSP 6 (and later) external and patch
    • faust2csound : CSOUND Opcode

    Jack

    • faust2jackconsole : JACK command line program
    • faust2jack : JACK application with GTK UI
    • faust2jaqt : JACK application with Qt UI

    MAC / IOS

    • faust2ios : iOS app
    • faust2caqt : CoreAudio application with Qt UI
    • faust2caqtios : iOS app with Qt UI

    PI $ Co

    • faust2rpialsaconsole : Raspberry Pi ALSA command line program
    • faust2bela : BELA program

    Microcontroller

    • faust2esp32 : ESP32 board
    • faust2teensy

    JUCE

    • faust2juce : JUCE Procects

    • faust2unity

      ... and more ...

    Compiling the first example as a PD external would be:

    $ faust2puredata sine.dsp
    

    FaustWorks

    FaustWorks is an integrated development environment for Faust. It includes an editor and manages complilation.

    Online Editor

    First steps and small programs can be experienced using the online editor for Faust:

    https://faust.grame.fr/tools/editor/index.html



    Contents © Henrik von Coler 2020 - Contact