Digital Synthesis: History & Taxonomy


Digital sound synthesis dates back to the first experiments of Max Mathews at Bell Labs in the 1950s. Mathews created the MUSIC programming language for generating musical sounds through additive synthesis on an IBM 704. The Silver Scale, realized by Newman Guttman in 1957, is (probably) the first ever digitally synthesized piece of music (Roads, 1980):

MUSIC and its versions (I, II, III, ...) are direct or indirect ancestors to most recent languages for sound processing. Even the name MAX/MSP is a tribute to the poineer. Although the first experiments sound amusing from todays perspective, Mathews already grasped the potential of the computer as a musical instrument:

“There are no theoretical limitations to the performance of the computer as a source of musical sounds, in contrast to the performance of ordinary instruments.” (Mathews, 1963)

Mathews created the first digital musical pieces himself, but in order to fully explore the musical potential, he was joined by composers, artists and other researchers, such as Newman Guttman, James Tenney and Jean Claude Risset. Later, the Bell Labs were visited by renowned composers of various genres, including John Cage, Edgard Varèse and Laurie Spiegel (Park, 2009).

Family Tree

The synthesis experiments at Bell Labs are the origin of most methods for digital sound synthesis. [Fig.1] illustrates the relations for a subset of synthesis approaches, starting with Mathews. The foundation for many further developments was laid when John Chowning brought the software MUSIC VI to Stanford from a visit at Bell Labs (Chowning, 2011). After migrating it to a PDP-6 computer, Chowning worked on his groundbreaking digital compositions, using the FM method and spatial techniques.

[Fig.1] Evolution and family tree (Bilbao, 2009).


Digital methods for sound synthesis can be grouped according to their underlying principle of operation. In 1991, Smith proposed four basic categories, shown in [Fig.2].

[Fig.2] Taxonomy of synthesis algorithms (Smith, 1991).

Processed Recording

Already a technique in the analog domain, more precisely in Musique Concrète, this family of synthesis approaches makes direct use of previously recorded sound for synthesis. This can be the playback of complete sounds or the extraction of short segments, such as grains or a single period of a sound.

Spectral Models

Spectral models use mathmatical means for expressing the spectra of sounds and their devopment over time. They are usually receiver-based, since they model the sound as it is heard, not as it is produced. This paradigm already existed in the mechanical world, as used by Hermann von Helmholtz in the 19th century and is based on even older signal models.

Physical Models

Physical Models are based on virtual acoustical and mechanical units, realized through buffers and LTI systems. Oscillators, resonating bodies and acoustic conductors are thus combined as in the mechanical domain. Physical modeling is regarded a source-based approach, since it deals with the actual sound production.

Abstract Algorithm

If it is not processed sound, a spectral model or a physical model, it is an abstract algorithm. Algorithms from this category transfer methods from other domains, like message transmission, to the musical domain.

Missing Recent Approaches

Although a few categorisations could be debated, the above introduced taxonomy is still valid but misses some recent developments. Methods based on neural networks and deep learning for sound generation may be considered a fifth taxon.

  • John Chowning. Turenas: the realization of a dream. Proc. of the 17es Journées d’Informatique Musicale, Saint-Etienne, France, 2011.
  • Bilbao, Stefan. Numerical Sound Synthesis. Wiley Online Library, 2009. ISBN 9780470749012. doi:10.1002/9780470749012.
  • Ananya Misra and Perry R Cook. Toward Synthesized Environments: A Survey of Analysis and Synthesis Methods for Sound Designers and Composers. In Proceedings of the International Computer Music Conference (ICMC 2009). 2009.
  • Tae Hong Park. An interview with max mathews. Computer Music Journal, 33(3):9–22, 2009.
  • Julius O. Smith. Viewpoints on the History of Digital Synthesis. In Proceedings of the International Computer Music Conference, 1–10. 1991.
  • Curtis Roads and Max Mathews. Interview with max mathews. Computer Music Journal, 4(4):15–22, 1980.
  • Max V Mathews. The Digital Computer as a Musical Instrument. Science, 142(3592):553–557, 1963.

  • Contents © Henrik von Coler 2020 - Contact