FM Synthesis: History & Backgroud

A Brief History

As mentioned in Introduction II, John Chowning brought a copy of the MUSIC IV software from Bell Labs to Stanford, where he founded the CCRMA, and started experiments in sound synthesis. Although frequency modulation was already a method used in analog sound synthesis, it was Chowning who developed the concept of frequency modulation (FM) synthesis with digital means in the late 1960s.

The concept of frequency modulation, already used for transmitting radio signals, was transferred to the audible domain by John Chowning, since he saw the potential to create complex (as in rich) timbres with a few operations (Chowning, 1973).

For one sinusoid modulating the frequency of a second, frequency modulation can be written as:

\[ y(t) = \sin(2 \pi f_c + I_m \sin(2 \pi f_m t) ) \]

\(f_c\) denotes the so called carrier frequency, \(f_m\) the modulation frequency and \(I_m\) the modulation index. [Fig.1] shows a flow chart for this operation in the style of MUSIC IV.

/images/Sound_Synthesis/modulation/fm-chowning-flow.png
[Fig.1] Flow chart for FM with two operators (Chowning, 1973).

In many musical applications, the use of dynamic spectra is desirable. The parameters of the above shown FM algorithm are therefor controlled with temporal envelopes, as shown in [Fig.2]. Especially the change of the modulation index over time is important, since it results in percussive sound qualities. In musical applications, multiple carriers and modulators, referred to as operators, are connected in different configurations, for generating richer timbres.

/images/Sound_Synthesis/modulation/fm-chowning-flow-2.png
[Fig.2] Flow chart for dynamic FM with two operators (Chowning, 1973).

FM synthesis is considered an abstract algorithm. It does not come with a related analysis approach to generate desired sounds but they need to be programmed or designed. However, there are attempts towards an automatic parametrization of FM synthesizers (Horner, 2003).


John Chowning, composer by profession, combined the novel FM synthesis approach with digital spatialization techniques to create quadraphonic pieces of electronic music on a completely new level. In Turenas, completed in 1972, artificial doppler shifts and direct-to-reverberation techniques are used to intensify the perceived motion and distance of panned sounds in the loudspeaker setup. The sounds used in this piece are only generated by means of FM, resulting in a characteristic quality like the synthetic bell-like sounds beginning at 1:30 or the re-occuring short precussive events.


References

  • John Chowning. Turenas: the realization of a dream. Proc. of the 17es Journées d’Informatique Musicale, Saint-Etienne, France, 2011.
    [BibTeX▼]
  • Andrew Horner. Auto-programmable FM and wavetable synthesizers. Contemporary Music Review, 22(3):21–29, 2003.
    [BibTeX▼]
  • John M Chowning. The synthesis of complex audio spectra by means of frequency modulation. Journal of the audio engineering society, 21(7):526–534, 1973.
    [BibTeX▼]
  • Additive & Spectral: Faust Examples

    Adding Partials 'Manually'

    A simple example, well suited for approaching the idea of additive synthesis in Faust is given by Romain Michon within a CCRMA workshop:

    import("music.lib");
    import("filter.lib");
    
    freq = hslider("freq",300,20,2000,0.01) : smooth(0.999);
    gain = hslider("gain",0.3,0,1,0.01) : smooth(0.999);
    t = hslider("attRel (s)",0.1,0.001,2,0.001);
    gate = button("gate") : smooth(tau2pole(t));
    
    process = osc(freq),osc(freq*2),osc(freq*3) :> /(3) : *(gain)*gate;
    

    Within the process function, three oscillators are called in parallel by comma-separating them. The :>_ operator collects their outputs, which are subsequently devided by 3 and amplified.

    Fourier Series in a Loop

    The example fourier_series.dsp in the seminar's Faust repository makes use of the parallel directive within a loop, allowing the use of more partials.

    // fourier_series.dsp
    //
    // Generate a square wave through Fourier series.
    // - without control
    //
    // Henrik von Coler
    // 2020-05-06
    
    import("stdfaust.lib");
    
    // define a fundamental frequency
    f0            = 100;
    
    // define the number of partials
    n_partial = 50;
    
    // partial function with one argument ()
    partial(partIDX) = (4/ma.PI) * os.oscrs(f)*volume
    // arguments
    with {
    f = f0 * (2*partIDX+1);
    volume = 1/(2*partIDX+1);
    };
    
    // the processing function,
    // running 50 partials parallel
    // mono output
    process = par(i, n_partial, partial(i)) :> +;
    

    The Faust Website Examples

    The Faust website lists two examples for additive Synthesis. Here, each partial is represented in the graphical user interface with individual control for temporal envelope parameters. This allows playing a triggered sound with a dynamic timbre.

    Expressive Timbral Control

    For using additive synthesis in an expressive way, metaparameters are essential. It is desirable to control the behaviour od all partials and thus the timbre with few meaningful controls.

    The following example, found in the semiar's Faust repository, controlls the decrease in energy towards higher partials with a single parameter:

    // exponential.dsp
    //
    // Additive synthesizer with controllable
    // exponential spectral decay.
    //
    // - continuous
    // - stereo output
    //
    // Henrik von Coler
    // 2020-05-05
    
    import("stdfaust.lib");
    
    // define a fundamental frequency
    f0           = 100;
    
    // define the number of partials
    n_partial = 50;
    
    slope     = hslider("s", 1, 0.1, 7, 0.01);
    
    
    // partial function
    partial(partCNT,s) = os.oscrs(f) * volume
    // arguments
    with {
    f = f0 * (partCNT+1);
    volume =  0.3 *  exp(s * -partCNT);
    };
    
    // the processing function,
    // running 50 partials parallel
    // summing them up and applying a global gain
    process = par(i, n_partial,  partial(i,slope)) :>_ * hslider("Master Gain",0,0,1, 0.1) <: _,_;
    

    Digital Synthesis: Trends and Developments

    Combining Approaches

    In the 1990s and early 2000s, most known synthesis algorithms existed and provided more and more convincing results, due to the increasing computational power. In order to overcome the drawbacks of individual synthesis approaches, paradigms are combined and novel, hybrid approaches are created.

    Deep Learning & Neural Nets

    Deep learning and neural nets have been used as helper tools in sound synthesis for many years. However, the direct use for the generation of sound is rather new and currently the hot topic in sound synthesis and processing.

    Control and Mapping

    Although the control of sound synthesis is not a new topic, it remains one of the most active ones. Synthesis algorithms are able to produce a large variety of sounds in real-time since the 1990s, but their integration into musical instruments is a much wider topic, with many possibilities to explore and stil lags behind.

    Digital Synthesis: Applications

    Applications

    While every digital sound synthesis approach stands for itself due to its unique characteristics, they all come with inherent strengths and limitations with regard to specific tasks and applications. [Fig.1] shows a list of synthesis goals with suitable approaches by Misra et al. (2009). In general, frequency-domain methods are less suited for time-critical tasks, involving transients and textures. Also, the table in Fig.1 suggests the versatility of granular and concatenative synthesis.

    /images/Sound_Synthesis/taxonomy_42.png
    [Fig.1] Synthesis goals and suitable approaches (Misra et al, 2009)

    Many methods for sound synthesis - analog or digital - have gained their status as an indipendent musical instrument, with characteristic sound properties. Some shaped the development of popular music and spawned new musical genres. This includes sampling - with a close link to rap music, subractive synthesis in many ways and as backbone of techno music, and FM synthesis - with the DX7 literally playing a part in most 1980s pop hits. Such influential synthesizers have again been synthesized. Virtual analog - also analog modeling - emulates vintage subtractive synthesizers in hard- and software and software synthesizers simulate classic FM synthesizers.


    References

  • John Chowning. Turenas: the realization of a dream. Proc. of the 17es Journées d’Informatique Musicale, Saint-Etienne, France, 2011.
    [BibTeX▼]
  • Bilbao, Stefan. Numerical Sound Synthesis. Wiley Online Library, 2009. ISBN 9780470749012. doi:10.1002/9780470749012.
    [BibTeX▼]
  • Ananya Misra and Perry R Cook. Toward Synthesized Environments: A Survey of Analysis and Synthesis Methods for Sound Designers and Composers. In Proceedings of the International Computer Music Conference (ICMC 2009). 2009.
    [BibTeX▼]
  • Tae Hong Park. An interview with max mathews. Computer Music Journal, 33(3):9–22, 2009.
    [BibTeX▼]
  • Julius O. Smith. Viewpoints on the History of Digital Synthesis. In Proceedings of the International Computer Music Conference, 1–10. 1991.
    [BibTeX▼]
  • Miller S. Puckette. The patcher. In Proceedings of the International Computer Music Conference (ICMC). 1988.
    [BibTeX▼]
  • Emmanuel Favreau, Michel Fingerhut, Olivier Koechlin, Patrick Potacsek, Miller S. Puckette, and Robert Rowe. Software developments for the 4x real-time system. In Proceedings of the International Computer Music Conference (ICMC). 1986.
    [BibTeX▼]
  • Curtis Roads and Max Mathews. Interview with max mathews. Computer Music Journal, 4(4):15–22, 1980.
    [BibTeX▼]
  • Max V. Mathews. The Technology of Computer Music. MIT Press, 1969.
    [BibTeX▼]
  • Max V Mathews. The Digital Computer as a Musical Instrument. Science, 142(3592):553–557, 1963.
    [BibTeX▼]
  • Digital Synthesis: History & Taxonomy

    Family Tree

    The synthesis experiments at Bell Labs are the origin of most methods for digital sound synthesis. [Fig.1] illustrates the relations for a subset of synthesis approaches, starting with Mathews. The foundation for many further developments was laid when John Chowning brought the software MUSIC VI to Stanford from a visit at Bell Labs (Chowning, 2011). After migrating it to a PDP-6 computer, Chowning worked on his groundbreaking digital compositions, using the FM method and spatial techniques.

    /images/Sound_Synthesis/bilbao_history.png
    [Fig.1] Evolution and family tree (Bilbao, 2009).

    Taxonomy

    Digital methods for sound synthesis can be grouped according to their underlying principle of operation. In 1991, Smith proposed four basic categories, shown in [Fig.2].

    /images/Sound_Synthesis/synthesis_taxonomy_SMITH.png
    [Fig.2] Taxonomy of synthesis algorithms (Smith, 1991).

    Processed Recording

    Already a technique in the analog domain, more precisely in Musique Concrète, this family of synthesis approaches makes direct use of previously recorded sound for synthesis. This can be the playback of complete sounds or the extraction of short segments, such as grains or a single period of a sound.

    Spectral Models

    Spectral models use mathmatical means for expressing the spectra of sounds and their devopment over time. They are usually receiver-based, since they model the sound as it is heard, not as it is produced. This paradigm already existed in the mechanical world, as used by Hermann von Helmholtz in the 19th century and is based on even older signal models.

    Physical Models

    Physical Models are based on virtual acoustical and mechanical units, realized through buffers and LTI systems. Oscillators, resonating bodies and acoustic conductors are thus combined as in the mechanical domain. Physical modeling is regarded a source-based approach, since it deals with the actual sound production.

    Abstract Algorithm

    If it is not processed sound, a spectral model or a physical model, it is an abstract algorithm. Algorithms from this category transfer methods from other domains, like message transmission, to the musical domain.

    Missing Recent Approaches

    Although a few categorisations could be debated, the above introduced taxonomy is still valid but misses some recent developments. Methods based on neural networks and deep learning for sound generation may be considered a fifth taxon.


  • John Chowning. Turenas: the realization of a dream. Proc. of the 17es Journées d’Informatique Musicale, Saint-Etienne, France, 2011.
    [BibTeX▼]
  • Bilbao, Stefan. Numerical Sound Synthesis. Wiley Online Library, 2009. ISBN 9780470749012. doi:10.1002/9780470749012.
    [BibTeX▼]
  • Ananya Misra and Perry R Cook. Toward Synthesized Environments: A Survey of Analysis and Synthesis Methods for Sound Designers and Composers. In Proceedings of the International Computer Music Conference (ICMC 2009). 2009.
    [BibTeX▼]
  • Tae Hong Park. An interview with max mathews. Computer Music Journal, 33(3):9–22, 2009.
    [BibTeX▼]
  • Julius O. Smith. Viewpoints on the History of Digital Synthesis. In Proceedings of the International Computer Music Conference, 1–10. 1991.
    [BibTeX▼]
  • Miller S. Puckette. The patcher. In Proceedings of the International Computer Music Conference (ICMC). 1988.
    [BibTeX▼]
  • Emmanuel Favreau, Michel Fingerhut, Olivier Koechlin, Patrick Potacsek, Miller S. Puckette, and Robert Rowe. Software developments for the 4x real-time system. In Proceedings of the International Computer Music Conference (ICMC). 1986.
    [BibTeX▼]
  • Curtis Roads and Max Mathews. Interview with max mathews. Computer Music Journal, 4(4):15–22, 1980.
    [BibTeX▼]
  • Max V. Mathews. The Technology of Computer Music. MIT Press, 1969.
    [BibTeX▼]
  • Max V Mathews. The Digital Computer as a Musical Instrument. Science, 142(3592):553–557, 1963.
    [BibTeX▼]
  • Faust: Compile for Puredata

    Faust can compile Puredata objects:

    $ faust2puredata sine.dsp
    

    On linux systems this will create a file sine~.pd_linux, on MacOS a file sine~.pd_darwin. Place the file in your PD search paths and use it, as in the help file sine~-help.pd. Parameters are set with messages.

    /images/Sound_Synthesis/sine_pd_example.png

    Faust: Compile for SuperCollider

    Faust can be used to compile SuperCollider extensions. For the sine.dsp example in the introduction:

    $ faust2supercollider sine.dsp
    

    This will produce two files:

    • the class file sine.sc
    • the binary sine.so

    Note that faust2supercollider depends on ruby, which you may need to install. If missing, the .sc files will be empty. There are now warnings or errors, so this can be confusing.

    Both files need to be placed in the system's SuperCollider extension directory and the class library needs to be recompiled. The class name in SuperColldier is generated by Faust:

    FaustSine : UGen
    {
    *ar { | frequency(100.0), gain(0.0) |
    ^this.multiNew('audio', frequency, gain)
    }
    
    *kr { | frequency(100.0), gain(0.0) |
    ^this.multiNew('control', frequency, gain)
    }
    
    name { ^"FaustSine" }
    
    
    info { ^"Generated with Faust" }
    }
    

    The new class can be used like this:

    {FaustSine.ar(100,1)}.play;
    

    Faust: Controlling Parameters

    In Faust, control parameters are always declared with a graphic user interface element. For some targets, these elements are ignored. For GUI targets, however, they are automatically included in the final software. With the native Faust GUI, as used in standalone applications, an example with two parameters may look like this:

    text


    This example implements a sine wave generator with controllable frequency and amplitude. The following image shows the top level flow chart generated by Faust:

    text


    The frequency of the oscillator is controlled with a horizontal slider, whereas the element for controlling the gain is in knob style. Additional parameters (see the Faust documentation for details) define parameter ranges, initial value more.

    Load this example in the Faust online IDE for a quick start:

    import("stdfaust.lib");
    
    // input parameters with GUI elements
    freq  = hslider("frequency",100, 10, 1000, 0.001);
    gain  = hslider("gain[style:knob]",0, 0, 1, 0.001);
    
    // a sine oscillator with controllable frequency and amplitude:
    process = os.osc(freq)*gain;
    


    Contents © Henrik von Coler 2021 - Contact