Using JACK Audio

The JACK API implements an audio server, allowing the connection of various software clients and hardware interfaces. In short, it turns the whole system into a digital audio workstation (DAW). It is the the standard way of working on Linux (pro) audio systems, but is also available for Mac and Windows. JACK needs a back end to connect to the actual audio hardware. On Linux systems, this is usually ALSA (Mac uses Core Audio and Windows ASIO).


Starting a JACK Server

A JACK server can be started via various graphical tools, such as QjackCtl or Carla. Many audio programs also boot a JACK server o their own, if launched. The recommended way of starting a JACK server in our case is the terminal, using the command jackd. It takes several arguments. The following line starts a server (in the background) with the ALSA interface named PCH, using a sample rate of 48kHz and a buffer size of 128 samples.

$ jackd -d alsa -d hw:PCH -r 48000 -p 128 &

Finding ALSA Devices

One way of finding the ALSA name of your interface is to type the following command:

$ aplay -l

The output shows all ALSA capable devices, their name listed after the card x:. PCH is usually the default onboard sound card:

**** List of PLAYBACK Hardware Devices ****
card 0: HDMI [HDA Intel HDMI], device 3: HDMI 0 [HDMI 0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: HDMI [HDA Intel HDMI], device 7: HDMI 1 [HDMI 1]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: HDMI [HDA Intel HDMI], device 8: HDMI 2 [HDMI 2]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: HDMI [HDA Intel HDMI], device 9: HDMI 3 [HDMI 3]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: HDMI [HDA Intel HDMI], device 10: HDMI 4 [HDMI 4]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: PCH [HDA Intel PCH], device 0: CX20751/2 Analog [CX20751/2 Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

Connecting JACK Clients

As almost everything, JACK connections can be modified from the terminal. All available JACK ports can be listed with the following command:

$ jack_lsp

Two ports can be connected with the following command:

$ jack_connect client1:output client2:input

Disconnecting two ports is done as follows:

$ jack_disconnect client1:output client2:input

If possible, a GUI-based tool, such as QjackCtl, can be more handy for connecting clients. It can be started via the a Desktop environment or from the command line:

$ qjackctl

/images/basics/qjackctl_connect.png

QjackCtl with hardware connections and two clients.


Storing/Restoring Connections

Several tools allow to store and restore JACK connections. Some of them work in a dynamic way, detecting spawned clients and connecting them accordingly. Others just allow a single operation for restoring connections.

aj-snapshot

The command line tool aj-snapshot is automatically installed alongside JACK. It can store and restore both JACK and ALSA connections, which can be handy when working with MIDI and is the most feature-rich and robust solution.

Once all connections are set, they can be stored to in an XML file, specified by a single argument:

aj-snapshot connections.snap

The above stored connections can be restored with the flag -r. An additional x deletes all prior connections, thus restoring the original state in the file:

aj-snapshot -xr connections.snap

The tool can also be started as a demon, looking for new clients and setting the related connections:

aj-snapshot -d connections.snap

Note

In some spatial audio projects, hardware devices and clients can have many a large number of ports. aj-snapshot does not handle that well and takes an excessive amount of time for deleting existing connections.


jmess

jmess is another command line tool, storing and restoring only JACK connections. It does not come with a demon mode but is a lot faster than aj-snapshot.


jack-matchmaker

jack matchmaker is a Python-based command line tool for dynamically restoring previously saved JACK connections.


QjackCtl Patchbay

The QjackCtl Patchbay offers a graphical solution for storing JACK and ALSA connections. Once activated, it patches new clients dynamically.

Filters in PD

When starting with subtractive synthesis, filters are the first thing to look for. PD offers a couple of builtin filters but additional externals come with nice sounding implementations.


"User-Friendly" Filters

lop~, hip~ and bp~ are the basic non-resonant filters in PD. The PD Floss Manuals on filters give a nice introduction to these builtin one-pole filters. The PD help files also come with examples. Due to the lacking resonance, these filters are not the most interesting ones, musically. They are also called "user-friendly", since they can not become unstable.

With the example one-pole-filters.pd from the repository, different characteristics of the one-pole filters can be compared, using a band-limited sawtooth as input signal. Filter cutoff and quality are controlled with control rate signals:

/images/Sound_Synthesis/subtractive/pd-one-pole-filters.png

Resonant Lowpass Filters

Additional filters can be installed with Deken. Filters and the relevant extensions can be found in the list of external filters . The iemlib, for example, features many useful resonant filters. One is the 8th order resonant lowpass vcf_lp8~. The moog~ filter object from the flatspace ggee library is another good sounding implementation, trying to emulate the Moog Ladder sound. The example resonant-lowpass.pd compares the sound of these filters with a square wave input. For both implementations, all parameters are controlled with audio rate signals. The slider values are thus converted to signals with the line~ object, which is basically a linear interpolation.

/images/Sound_Synthesis/subtractive/pd-resonant-lowpass.png

Exercises

Exercise I

Control the parameters of the resonant lowpass example with temporal envelopes.

Exercise II

Trigger the envelope with a metronome sequencer.

Exercise III

Create a square wave from the sawtooth and use it as input signal (http://write.flossmanuals.net/pure-data/square-waves/).

Physical Modeling: Introduction

Physical modeling emulates actual physical processes with digital means. Oscillators, resonators and acoustic impedance are modeled with buffers and filters, respectively LTI systems. Although first realized when computers had sufficient power, the foundations are much older. Hiller et al. (1971) were the first to transport the 1774 wave equation by d'Alambert to the digital domain for synthesizing sounds of plucked strings.


Early Hardware

Although physical modeling algorithms sound great, offer good means for control and enable the design of interesting instruments, they had less impact on the evolution of music genres and digital instruments. Hardware synths for physical modeling from the 1990s, like the Korg Prophecy or the Yamaha VL1 did not become a success, in the first place. With prices of about 10.000 $, they were way too expensive. There are many more practical reasons for the lack of success (Bilbao et al, 2019). But the technique also had a bad timing, trying to establish in the 1990s. Cheaper and larger memory made sampling instruments more powerful and virtual analog synthesizers sounded more attractive, followed by the second wave of analog synths.


Yamaha VL1 (1994)


Software Instruments

Today, some physical modeling software emerged for high quality piano and organ synthesis (Amazona article). Other implementations aim at strings:

  • Pianoteq Pro 6

  • Organteq Alpha

  • Strum GS 2

  • AAS Chromophone 2


Modular

Since simple physical models are nowadays easily implemented on small embedded systems, various modules exist on the market. It a modular setup, this is especially interesting, since arbitrary excitation signals can be generated and patched. These are just two examples:

/images/Sound_Synthesis/physical_modeling/mysteron.jpg
/images/Sound_Synthesis/physical_modeling/rings.jpg

Physical Models in Experimental Music

Eikasia

Unlike FM synthesis, subtractive synthesis or sampling, physical modeling does not come with genre-defining examples from popular music. However, the technique has been used a lot in the context of experimental music (Chafe, 2004). Eikasia (1999) by Hans Tutschku was realized using the IRCAM software Modalys:


S-Morphe-S

In his 2002 work S-Morphe-S, Matthew Burtner used physical models of singing bowls, excited by a saxophone:


References

2019

  • Stefan Bilbao, Charlotte Desvages, Michele Ducceschi, Brian Hamilton, Reginald Harrison-Harsley, Alberto Torin, and Craig Webb. Physical modeling, algorithms, and sound synthesis: the ness project. Computer Music Journal, 43(2-3):15–30, 2019.
    [details] [BibTeX▼]

2004

  • Chris Chafe. Case studies of physical models in music composition. In Proceedings of the 18th International Congress on Acoustics. 2004.
    [details] [BibTeX▼]

1995

  • Vesa Välimäki. Discrete-time modeling of acoustic tubes using fractional delay filters. Helsinki University of Technology, 1995.
    [details] [BibTeX▼]
  • Gijs de Bruin and Maarten van Walstijn. Physical models of wind instruments: A generalized excitation coupled with a modular tube simulation platform*. Journal of New Music Research, 24(2):148–163, 1995.
    [details] [BibTeX▼]

1993

  • Matti Karjalainen, Vesa Välimäki, and Zoltán Jánosy. Towards High-Quality Sound Synthesis of the Guitar and String Instruments. In Computer Music Association, 56–63. 1993.
    [details] [BibTeX▼]

1992

  • Julius O Smith. Physical modeling using digital waveguides. Computer music journal, 16(4):74–91, 1992.
    [details] [BibTeX▼]

1971

  • Lejaren Hiller and Pierre Ruiz. Synthesizing musical sounds by solving the wave equation for vibrating objects: part 1. Journal of the Audio Engineering Society, 19(6):462–470, 1971.
    [details] [BibTeX▼]
  • Lejaren Hiller and Pierre Ruiz. Synthesizing musical sounds by solving the wave equation for vibrating objects: part 2. Journal of the Audio Engineering Society, 19(7):542–551, 1971.
    [details] [BibTeX▼]

Concept of Subtractive Synthesis

Functional Units

Subtractive synthesis is probably the best known and most popular method of sound synthesis. The basic idea is to start with signals with rich spectral content which are then shaped afterwards by filters. Although the possibilities of subtractive synthesis are quasi-unlimited, especially when combined with other methods, the principle can be explained with three groups of functional units:

  • Generators

  • Manipulators

  • Modulators


[Fig.1] gives an overview how these functional units are arranged in a subtractive synthesizer. Modulators and generators overlap, since they are interchangeable in many aspects. This section uses the terminology from the (modular) analog domain, with Voltage Controlled Oscillators (VCO), Voltage Controlled Filters (VCF) and Voltage Controlled Amplifiers (VCA).


/images/Sound_Synthesis/subtractive/subtractive-figure0.png
Fig.1

Functional units in subtractive synthesis.


Generators

  • Oscillators (VCO)

  • Noise Generators

  • ...

Frequently used oscillators in subtractive synthesis are the basic waveforms with high frequency energy, such as the sawtooth, square wave or the triangular wave (See the section on additive synthesis). Noise generators can be used for adding non-harmonic components.

Manipulators

  • Filters (VFC)

  • Amplifiers (VCA)

  • ...

The most important manipulators are filters and amplifiers, respectively attenuators. Filters will be explained in detail in the following sections.

Modulators

  • LFO (Low Frequency Oscillators)

  • Envelopes (ADSR)

  • ...

Modulators are such units which control the parameters of generators and manipulators over time. This includes periodic modulations, such as the LFO, and envelopes, which are triggered by keyboard interaction.


A Typical Bass/Lead Patch

Like with all methods for sound synthesis, the dynamic change of timbre is an essential target for generating vivid sounds. [Fig.2] shows a more specific signal flow which is a typical subtractive synth patch for generating lead or bass sounds.

  • The signal from a VCO is manipulated by a VCF and then attenuated by a VCA.

  • The VCO has a sawtooth or square waveform.

  • The cutoff frequency of the VCF and the amplitude of the VCA are controlled with individual envelopes.

  • If ENV2 has a faster decay than ENV1, the sound will have a crisp onset and a low decay, resulting in the typical thump.


/images/Sound_Synthesis/subtractive/subtractive-figure1.png
Fig.2

Subtractive patch for bass and lead synth.

AM & Ringmodulation: Formula & Spectrum

Amplitude Modulation vs Ringmodulation

Both amplitude modulation and ringmodulation are a multiplication of two signals. The basic formula is the same for both:

$y[n] = x[n] \cdot m[n]$

However, for ringmodulation the modulation signal is symmetric:

$y[n] = \sin\left(2 \pi f_c \frac{n}{f_s}\right) \cdot \left(\sin\left[2 \pi f_m \frac{n}{f_s}\right]\right)$

Whereas for amplitude modulation, the signal ist asymetric:

$y[n] = \sin\left(2 \pi f_c \frac{n}{f_s}\right) \cdot \left( 1+ \sin\left[2 \pi f_m \frac{n}{f_s}\right]\right)$

This differnce has an influence on the resulting spectrum and on the sound, as the following examples show.

AM Spectrum

The spectrum for amplitude modulation can be calculated as follows:

$Y[k] = DFT(y[n])$

$\displaystyle Y[k] = \sum_{n=0}^{N-1} y[n] \cdot e^{-j 2 \pi k \frac{n}{N}}$

$\displaystyle = \sum_{n=0}^{N-1} \sin\left(2 \pi f_c \frac{n}{f_s}\right) \cdot \left( 1+ \sin\left[2 \pi f_m \frac{n}{f_s}\right]\right) \cdot e^{-j 2 \pi k \frac{n}{N}}$

$\displaystyle =\sum_{n=0}^{N-1} \left( \sin\left(2 \pi f_c \frac{n}{f_s}\right) + 0.5 \left( \cos\left(2 \pi (f_c - f_m)\frac{n}{f_s}\right) - \cos\left(2 \pi (f_1 + f_m)\frac{n}{f_s}\right) \right) \right) \cdot e^{-j 2 \pi k \frac{n}{N}}$

$\displaystyle= \delta[f_1] + 0.5 \delta[f_c - f_m] + 0.5 \ \delta[f_c + f_m]$

AM creates a spectrum with a peak at the carrier frequency and two peaks below and above it. Their position is defined by the difference between carrier and modulator.

Ringmod Spectrum

$\mathcal{F} [ y(t)] = \int\limits_{-\inf}^{\inf} y(t) e^{-j 2 \pi f t} \mathrm{d}t$

$= \int\limits_{-\inf}^{\inf} \left( \sin(2 \pi f_c t) \sin(2 \pi f_s t) \right) e^{-j 2 \pi f t} \mathrm{d}t$

$= \frac{1}{2 j} \int\limits_{-\inf}^{\inf} \left( (-e^{-j 2 \pi f_c t} +e^{j 2 \pi f_c t}) (-e^{-j 2 \pi f_s t} +e^{j 2 \pi f_s t}) \right) \ e^{-j 2 \pi f t} \mathrm{d}t$

$= \frac{1}{2 j} \int\limits_{-\inf}^{\inf} \left( e^{j 2 \pi (f_c+f_s) t} - e^{j 2 \pi (f_c-f_s) t} - e^{j 2 \pi (-f_c+f_s) t} + e^{j 2 \pi (-f_c-f_s) t} \right) e^{-j 2 \pi f t}$

$= \frac{1}{2 j} \left[ \delta(f_c+f_s) -\delta(f_c-f_s) - \delta(-f_c+f_s) + \delta(-f_c-f_s) \right]$

Ringmodulation creates a spectrum with
two peaks below and above the carrier frequency. Their position is defined by the difference between carrier and modulator.
The modulator is supressed, since it is symmetric.

Additive & Spectral: Introduction

The sine wave can be considered the atomic unit of timbre and thus of musical sounds. Additive synthesis and related approaches build musical sounds from scratch, using these integral components. When a sound is composed of several sinusoids, they are referred to as partials, regardless of their properties. Partials which are integer multiples of a fundamental frequency are called harmonics or overtones, when related to the first harmonic.

Fourier Series

According to the Fourier theorem, any periodic signal can be represented by an infinite sum of sinusoids with individual

  • amplitude \(a_i\)

  • frequency \(f_i\)

  • phase \(\varphi_i\)

\(\displaystyle y = \sum\limits_{i=1}^{\infty} a_i \ sin(2 \pi f_i \ t +\varphi_i )\)

When applying this principle to musical sounds, a simplified model can be used to generate basic timbres. All sinusoidal components become integer multiples of a fundamental freuquency \(f_0\), so called harmonics, with a maximum number of partials \(N_{part}\). In an even further reduced model, the phases of the partials can be ignored:

\(\displaystyle y (t) = \sum\limits_{n=1}^{N_{part}} a_n(t) \ sin(2 \ \pi \ n \ f_0 (t) \ t)\)

---

As following sections on spectral modeling show, a more advanced model is needed to synthesize musical sounds which are indistinguishable from the original. This includes the partials' phase, inharmonicities as deviations from exact integer multiples, noise components and transients. However, depending of the number of partials and the driving function for their parameters, this limited formula can generate convincing harmonic sounds.


A Brief History

Early Mechanical

Early use of the Fourier representation, respectively additive synthesis, for modeling musical sounds has been made by Hermann von Helmholtz. He built mechanical devices, based on tuning forks, resonant tubes and electromagnetic excitation for additive synthesis. Von Helmholtz used these devices for investigating various aspects of harmonic sounds, including spectral distribution and relative phases.

/images/Sound_Synthesis/helmholtz_fork.jpg

Tuning forks with resonant tubes (von Helmholtz, 1870, p.183).


Early Analog

The history of Elektronische Musik started with additive synthesis. In his composition Studie II, Karlheinz Stockhausen composed timbres by superimposing sinusoidal components. In that era this was realized through single sine wave oscillators, tuned to the desired frequency and recorded on tape.

Studie II is the attempt to fully compose music on a timbral level in a rigid score. Stockhausen therefor generated tables with frequencies and mixed tones for creating source material. [Fig.1] shows an excerpt from the timeline, which was used to arrange the material. The timbres are recognizable through their vertical position in the upper system, whereas the lower system represents articulation, respectively fades and amplitudes.

/images/Sound_Synthesis/studie4.jpg
Fig.1

From the score of Studie II.


Early Digital

Max Mathews

As mentioned in Introduction II, Max Mathews used additive synthesis to generate the first digitally synthesized pieces of music in the 1950s. In the early 1960s, Mathews had advanced the method to synthesize dynamic timbres, as in Bycicle Built for Two:


Iannis Xenakis

In his Electroacoustic compositions, Iannis Xenakis made use of the UPIC system for additive synthesis (Di Scipio, 1998), as for example is Mycenae-Alpha (1977):


References

1998

  • Agostino Di Scipio. Compositional models in xenakis's electroacoustic music. Perspectives of New Music, pages 201–243, 1998.
    [details] [BibTeX▼]

1870

  • Hermann von Helmholtz. Die Lehre von den Tonempfindungen als physiologische Grundlage für die Theorie der Musik, 3. umgearbeitete Ausgabe. Braunschweig: Vieweg, 1870.
    [details] [BibTeX▼]

Faust: Sequential Composition

Sequential composition is used for passing signals directly from one block to another block. In Faust, this is done with the : operator. The following example illustrates this with a square wave signal, which is processed with a low pass filter:

text


The square wave has a fixed frequency of $50\ \mathrm{Hz}$. The lowpass filter has two arguments, the first being the filter order, the second the cutoff frequency, which is controlled with a horizontal slider. Both blocks are defined and subsequently connected in the process function with the : operator. The adjustable cutoff parameter is additionally smoothed with si.smoo to avoid clicks.

Load this example in the Faust online IDE for a quick start:

import("stdfaust.lib");

freq  = hslider("frequency",100, 10, 1000, 0.001);

sig  = os.square(50);
filt = fi.lowpass(5,freq);

process = sig:filt;

Faust: Quick Introduction

Faust is a functional audio programming language, developed at GRAME, Lyon. It is a community-driven, free open source project. Faust is specifically suited for quickly designing musical synthesis and processing software and compiling it for a large variety of targets. The language works well with physical models and features many components for different physical approaches.


Tools for Working with Faust

Faust offers a large toolbox for different levels of expertise. Downloads are found here:

https://faust.grame.fr/downloads/index.html


Faust IDE

The Faust IDE is the fastest way to develop and test .dsp code: https://faustide.grame.fr/


Faust Compiler

The Faust compiler is the center-piece of the Faust development tools. Faust code is written in *.dsp files, which are converted to C++ code and then compiled for the desired system. The Faust compiler can be called directly from the command line or by other programs, such as the IDE FaustWorks. When called from the command line, Faust is compiled to the desired target using a faust2* command, which actually calls a build script.

Depending on your operating system and build target, additional libraries or tools may be required. Targets of interest can be:

Linux

  • faust2alsaconsole : ALSA command line program

  • faust2alqt : ALSA application with Qt UI

  • faust2alsa : ALSA application with GTK UI

Plugins

  • faust2ladspa : LADSPA plug-in

  • faust2lv2 : LV2 plug-in

  • faust2faustvst : VST plug-in

  • faust2au : Audio Unit plugin

Music Programming Environments

  • faust2supercollider : SuperCollider external

  • faust2puredata : PureData external

  • faust2max6 : MaxMSP 6 (and later) external and patch

  • faust2csound : CSOUND Opcode

Jack

  • faust2jackconsole : JACK command line program

  • faust2jack : JACK application with GTK UI

  • faust2jaqt : JACK application with Qt UI

MAC / IOS

  • faust2ios : iOS app

  • faust2caqt : CoreAudio application with Qt UI

  • faust2caqtios : iOS app with Qt UI

PI $ Co

  • faust2rpialsaconsole : Raspberry Pi ALSA command line program

  • faust2bela : BELA program

Microcontroller

  • faust2esp32 : ESP32 board

  • faust2teensy

JUCE

  • faust2juce : JUCE Procects

  • faust2unity

    ... and more ...

Compiling the first example as a PD external would be:

$ faust2puredata sine.dsp

FaustWorks

FaustWorks is an integrated development environment for Faust. It includes an editor and manages compilation. However, the software has not been maintained in a while.


Faust Libraries

Faust comes with a large set of libraries: Faust Library Website

They can be included individually with the import(delays.lib) command. Most examples in this class import all standard libraries with the import("stdfaust.lib"); command.

Network Audio

OSI Model

The OSI Model groups different services, functions and applications of telecommunication systems into seven hierarchically arranged layers:

OSI Model

Layer

Name

Description

7

Application Layer

End user layer, HCI layer

6

Presentation Layer

data conversion, syntax

5

Session Layer

connection management, sockets

4

Transport Layer

end-to-end connections (TCP, UDP)

3

Network Layer

packet routing

2

Data Link Layer

data formats (bits to frames, MAC addresses)

1

Physical layer

bit stream transmission over medium/hardware (Ethernet, WiFi, ...)


Network based audio systems can be based on different layers. This affects their capabilities and application areas. A comprehensive list can be found here: comparison on Wikipedia


Layer 1 Solutions

Layer 1 solutions only rely on the hardware used in telecommunication systems and use their own routing mechanisms. As a consequence, they usually need specific routers and are often used for direct peer-to-peer connections. The most widespread solution is the open AES50 format, which is found in devices by Behringer and Midas.

Layer 2 Solutions

Layer 2 solutions use the standard Ethernet protocol for transmitting data. Standard routers and hardware can thus be used for routing. Among the well known formats are AVB and AES51, as well as several proprietary solutions.

Layer 3 Solutions

Layer 3 solutions feature an IP header in their packages. Example solutions are DANTE, AES67, RAVENNA and AVB.

Layer 4 Solutions

Some solutions are based on Layer 4 protocols like TCP or UDP 1. Since UDP is faster due to the missing handshake and error-correction. Although this makes it prone to package loss, it is the preferred method for achieving acceptable latency at the cost of dropouts, depending on the quality of the connection.

Examples for Layer 4 solutions can be found in the free and open software community, including NetJack2 2, Zita-njbridge 3 and JackTrip.


1

This needs more references, since it is not unambiguous on which layer they are working.

2

https://github.com/jackaudio/jackaudio.github.com/wiki/WalkThrough_User_NetJack2

3

http://kokkinizita.linuxaudio.org/linuxaudio/index.html

Using SSH for Remote Access

SSH (Secure Shell Protocol) is necessary when working with the server but can also be helpful for configuring the Access Points. For remote machines - like the SPRAWL Server - SSH can be used for command-line operations and command execution.

Connecting to an SSH Server

For connecting to a remote machine, it needs to run an SSH server. On the client side, an SSH connection can be established without additional installations from the terminal on Linux and MAC machines and - since version 10 - from Windows. For older Windows versions, users can use Putty.

$ ssh username@address

X11 Forwarding

With X11 Forwarding, SSH can also be used to run applications with a GUI, remotely. Simply add the -X argument to do so:

$ ssh -X username@address

Remote Commands

SSH can also be used to send single commands, without starting a remote session. This example launches the jack_simple_client, which plays a continuing sine tone on the remote machine.

$ ssh -t username@address 'jack_simple_server'

Exercise

Log into the server with SSH.