The MIDI protocol was released in 1982 as a means for connecting electronic musical instruments. First synths to feature the new technology were the Prophet-600 and the Jupiter-6. Although limited in resolution from a recent point of view, it is still a standard for conventional applications - yet to be replaced by the newly released MIDI 2.0. Besides rare mismatches and some limitations, MIDI devices can be connected without complications. Physically, MIDI has been introduced with the still widespread 5-pin connector, shown below. In recent devices, MIDI is usually transmitted via USB.
MIDI jack (5-pin DIN).
MIDI transmits binary coded messages with a speed of $31250\ \mathrm{kbit/s}$. Timing and latency are thus not a problem when working with MIDI. However, the resolution of control values can be a limiting factor. Standard MIDI messages consist of three Bytes, namely one status Byte (first bit green) and two data bytes (first bit red). The first bit declares the Byte either a status Byte (1) or a data Byte (0).
Standard MIDI message with three Bytes.
Some of the most common messages are listed in the table below. Since one bit is used as the status/data identifier, 7 bits are left for encoding. This results in the typical MIDI resolution of \(2^7 = 128\) values for pitch, velocity or control changes.
Voice Message Status Byte Data Byte1 Data Byte2 ------------- ----------- ----------------- ----------------- Note off 8x Key number Note Off velocity Note on 9x Key number Note on velocity Polyphonic Key Pressure Ax Key number Amount of pressure Control Change Bx Controller number Controller value Program Change Cx Program number None Channel Pressure Dx Pressure value None Pitch Bend Ex MSB LSB
If you are stuck with MIDI for some reason but need a higher resolution, the Pitch Bend parameter can help. Each MIDI channel has one Pitch Bend, each with two combined data Bytes, resulting in a resolution of \(128^2 = 16384\) steps.
SysEx messages can be freely defined by manufacturers. They are often used for dumping or loading settings and presets, but can also be used for arbitrary control purposes. SysEx messages can have any length and are not standardized.
Supercollider (SC) is a server-client-based tool for sound synthesis and composition. SC was started by James McCartney in 1996 and is free software since 2002. It can be used on Mac, Linux and Windows systems and comes with a large collection of community-developed extensions. The client-server principle aims at live coding and makes it a powerful tool for distributed and embedded systems, allowing the full remote control of synthesis processes.
There are many ways of approaching SuperCollider, depending on the intended use case. Some tutorials focus on sequencing, others on live coding or sound design. This introduction aims at programming remotely controlled synthesis and processing servers, which involves signal routing and OSC capabilities.
Binaries, source code and build or installation instructions can be found at the SC GitHub site. If possible, it is recommended to build the latest version from the repository:
https://supercollider.github.io/download
SuperCollider comes with a large bundle of help files and code examples but first steps are usually not easy. There are a lot of very helpful additional resources, providing step by step introductions.
Code snippets in this example are taken from the accompanying repository: SC Example. You can simple copy and paste them into your editor.
SuperCollider is based on a client-server paradigm. The server is running the actual audio processing, whereas clients are used to control the server processes via OSC messages. Multiple clients can connect to a running server. The dedicated ScIDE allows convenient features for live coding and project management:
Server, client and ScIDE.
sclang is the SuperCollider language. It represents the client side when working with SC. It can for example be started in a terminal by running:
$ sclang
Just as with other interpreted languages, such as Python, the terminal will then change into sclang mode. At this point, the class library is complied, making all SC classes executable. Afterwards, SC commands can be entered:
sc3> postln("Hello World!")
Working with SC in the terminal is rather inconvenient. The SuperCollider IDE (ScIDE) is the environment for live coding in sclang, allowing the control of the SuperCollider language:
ScIDE
When booting the ScIDE, it automatically launches sclang and is then ready to interpret. Files opened in the IDE can be executed as a whole. Moreover, single blocks, respectively single lines can be evaluated, which is especially handy in live coding, when exploring possibilities or prototyping. In addition, the IDE features tools for monitoring various server properties.
Global variables are either single letters - s is preserved for the default server - or start with a tilde: ~varname). Local variables, used in functions, need to be defined explicitly:
var foo;
Some of the examples in the SC section of this class are in the repository, whereas other only exist as snippets on these pages. In general, all these examples can be explored by copy-pasting the code blocks from the pages into the ScIDE. They can then be evaluated in blocks or line-wise but can not be executed as complete files. These features help to run code in the ScIDE subsequently:
The Faust oscillators.lib comes with many different implementations of oscillators for various waveforms. At some point one might still need a behavior not included and lower level approaches are necessary.
This example shows how to use a phasor to read a wavetable with a sine waveform. This implementation has an additional trigger input for resetting the phase of the oscillator on each positive zero crossing. This can come handy in various applications, especially for phase-sensitive transients, as for example in kick drums.
The example is derived from Barkati et. al (2013) and part of the repository:
import("stdfaust.lib"); // some basic stuff sr = SR; twopi = 2.0*ma.PI; // define the waveform in table ts = 1<<16; // size = 65536 samples (max of unsigned short) time = (+(1) ~ _ ) , 1 : - ; sinewave = ((float(time) / float(ts)) * twopi) : sin; phase = os.hs_phasor(ts,freq,trig); // read from table sin_osc( freq) = rdtable(ts ,sinewave , int(phase)) ; // generate a one sample impulse from the gate trig = pm.impulseExcitation(reset); reset = button ("reset"); freq = hslider("freq", 100, 0, 16000, 0.00001); // offset = hslider("offset", 0, 0, 1, 0.00001); process = sin_osc(freq);
Spectral spatialization in this system is based on a virtual sound source with a position an space and spatial extent, as shown in [Fig.1]. The source center is defined by two angles (Azimuth, Elevation) and the Distance. The Spread defines the diameter of the virtual source. This model is compliant with many theoretical frameworks from the fields of electroacoustic music and virtual acoustics.
| [Fig.1] | (1, 2) Virtual sound source with position an space and spatial extent. |
The virtual source from [Fig.1] is realized as a cloud of point sources in an Ambisonics system using the IRCAM software Panoramix. 24 point sources can be controlled jointly. The following figures show the viewer of Panoramix, the left half representing the top view, the right half the rear view.
[Fig.2] shows a dense point cloud of a confined virtual sound source without elevation:
| [Fig.2] | Confined virtual sound source. |
The virtual sound source in [Fig.3] has a wider spread and is elevated:
| [Fig.3] | Spread virtual sound source with elevation. |
For small distances and large spreads, the source is enveloping the listener, as shown in [Fig.4]:
| [Fig.4] | Enveloping virtual sound source. |
In a nutshell, the synthesizer outputs the spectral components of a violin sound to 24 individual outputs. Different ways of assigning spectral content to the outputs are possible, shown as Partial to Source Mapping in [Fig.5]. In these experiments, each output represents a Bark scale frequency band. For the point cloud shown above, the distribution of spectral content is thus neither homogenous nor stationary.
| [Fig.5] | Dispersion - routing partials to point sources. |
The typical DMI model connects the musical interface with the sound generation through a mapping stage. [Fig.1] shows the extended DMI model for spatial sound synthesis. The joint control of spatial and timbral characteristics offers new possibilities yet makes the mapping and the resulting control more complex.
| [Fig.1] | Mapping in the extended DMI model. |
We chose Puredata as a graphical interface for mapping controller parameters to sound synthesis and spatialization. Especially in the early stage of development this solution offers maximum flexibility. [Fig.2] shows the mapping GUI as it was used by the participants in the mapping study:
| [Fig.2] | Puredata patch for user-defined mappings. |
In the first stage of the user study, participants had 30 minutes to create their own mapping, following this basic instruction:
The objective of this part is to create an enjoyable mapping, which offers the most expressive control over all synthesis and spatialization parameters.
A set of rules allowed one to many mappings and excluded many to one mappings:
The mapping matrix shows how some control parameters are preferred for specific tasks, considering the final mappings of all participants:
| [Fig.1] | Mapping matrix: how often was a control parameter mapped to a specific rendering parameter? |
More advanced physical models can be designed, based on the principles explained in the previous sections.
The simple lowpass filter in the example can be replaced by more sophisticated models. For instruments with multiple strings, coupling between strings can be implemented.
Model of a wind instrument with several waveguides, connected with scattering junctions (de Bruin, 1995):
The functional principle of Faust is very well suited for programming physical models for sound synthesis, since these are usually described in block diagrams. Working with physical modeling in Faust can happen on many levels of complexity, from using ready instruments to basic operations.
For a quick start, fully functional physical modeling instruments can be used from the physmodels.lib library. These *_ui_MIDI functions just need to be called in the process function:
import("all.lib"); process = nylonGuitar_ui_MIDI : _;
The same algortithms can also be used on a slightly lower level, combining them with custom control and embedding them into larger models:
import("all.lib"); process = nylonGuitarModel(3,1,button("trigger")) : _;
The physmodels.lib library comes with many building blocks for physical modeling, which can be used to compose instruments. These blocks are instrument-specific, as for example:
The bidirectional utitlities and basic elements in Faust's physical modeling library offer a more direct way of assembling physical models. This includes waveguides, terminations, excitation and others:
Taking a look at the physmodels.lib library, even the bidirectional utilities and basic elements are made of standard faust functions:
https://github.com/grame-cncm/faustlibraries/blob/master/physmodels.lib
chain(A:As) = ((ro.crossnn(1),_',_ : _,A : ro.crossnn(1),_,_ : _,chain(As) : ro.crossnn(1),_,_)) ~ _ : !,_,_,_; chain(A) = A;
// karplus_strong.dsp // // Slightly modified version of the // Karplus-Strong plucked string algorithm. // // see: 'Making Virtual Electric Guitars and Associated Effects Using Faust' // (Smith, ) // // - one-pole lowpass in the feedback // // Henrik von Coler // 2020-06-07 import("all.lib"); //////////////////////////////////////////////////////////////////////////////// // Control parameters as horizonal sliders: //////////////////////////////////////////////////////////////////////////////// freq = hslider("freq Hz", 50, 20, 1000, 1) : si.smoo; // Hz // initial filter for the excitation noise initial_filter = hslider("initial_filter Hz",1000,10,10000,1) : si.smoo; lop = hslider("lop Hz",1000,10,10000,1) : si.smoo; level = hslider("level", 1, 0, 10, 0.01); gate = button("gate"); gain = hslider("gain", 1, 0, 1, 0.01); //////////////////////////////////////////////////////////////////////////////// // processing elements: //////////////////////////////////////////////////////////////////////////////// diffgtz(x) = (x-x') > 0; decay(n,x) = x - (x>0)/n; release(n) = + ~ decay(n); trigger(n) = diffgtz : release(n) : > (0.0); P = SR/freq; // Resonator: resonator = (+ : delay(4096, P) * gain) ~ si.smooth(1.0-2*(lop/ma.SR)); //////////////////////////////////////////////////////////////////////////////// // processing function: //////////////////////////////////////////////////////////////////////////////// process = noise : si.smooth(1.0-2*(initial_filter/ma.SR)):*(level) : *(gate : trigger(P)): resonator <: _,_;
// waveguide_string.dsp // // waveguide model of a string // // - one-pole lowpass termination // // Henrik von Coler // 2020-06-09 import("all.lib"); // use '(pm.)l2s' to calculate number of samples // from length in meters: segment(maxLength,length) = waveguide(nMax,n) with{ nMax = maxLength : l2s; n = length : l2s/2; }; // one lowpass terminator fc = hslider("lowpass",1000,10,10000,1); rt = rTermination(basicBlock,*(-1) : si.smooth(1.0-2*(fc/ma.SR))); // one gain terminator with control gain = hslider("gain",0.99,0,1,0.01); lt = lTermination(*(-1)* gain,basicBlock); idString(length,pos,excite) = endChain(wg) with{ nUp = length*pos; nDown = length*(1-pos); wg = chain(lt : segment(6,nUp) : in(excite) : out : segment(6,nDown) : rt); // waveguide chain }; length = hslider("length",1,0.1,10,0.01); process = idString(length,0.15, button("pluck")) <: _,_;
plucked_string_444.png
import numpy as np from numpy import linspace, sin, zeros from math import pi %matplotlib notebook import matplotlib.pyplot as plt from tikzplotlib import save as tikz_save from IPython.display import display, Markdown, clear_output import IPython.display as ipd import ipywidgets as widgets from ipywidgets import * fs = 48000 L = 500 # a function for appending the array again and again # arbitrary 300 times ... def appender(x): y = np.array([]) for i in range(300): y = np.append(y,x*0.33) return y x = np.random.standard_normal(L) y = appender(x) t = np.linspace(0,len(y)/fs,len(y)) f = np.linspace(0,1,len(y)) fig = plt.figure() ax = fig.add_subplot(2, 1, 1) line, = ax.plot(t,y) ax2 = fig.add_subplot(2, 1, 2) Y = abs(np.fft.fft(y)) Y = Y[0:5000] f = f[0:5000] line2, = ax2.plot(f,Y) def update(L = widgets.IntSlider(min = 10, max= 1500, step=1, value=500)): x = np.random.standard_normal(L) y = appender(x) t = np.linspace(0,len(y)/fs,len(y)) f = np.linspace(0,1,len(y)) Y = abs(np.fft.fft(y)) Y = Y[0:5000] f = f[0:5000] line.set_ydata(y) line2.set_ydata(Y) fig.canvas.draw_idle() ipd.display(ipd.Audio(y, rate=fs)) interact(update);
<IPython.core.display.Javascript object>
interactive(children=(IntSlider(value=500, description='L', max=1500, min=10), Output()), _dom_classes=('widge…
Karplus-Strong makes use of the random buffer.
# this implementation serves for a better # understanding and is not efficient # # - wait for process to be finished in # interactive use fs = 48000 L = 500 # the feedback gain gain = 0.99 # the number of samples used for smoothing smooth = 10 def karplus_strong(L,gain,smooth): x = np.random.standard_normal(L) y = np.array([]) for i in range(96000): k = i%L tmp = 0; for j in range(smooth): tmp += x[(k+j) %L] tmp = tmp/smooth x[k] = gain*tmp y = np.append(y,tmp) return y y = karplus_strong(L,gain,smooth) t = np.linspace(0,len(y)/fs,len(y)) f = np.linspace(0,1,len(y)) fig = plt.figure() ax = fig.add_subplot(2, 1, 1) line, = ax.plot(t,y) ax2 = fig.add_subplot(2, 1, 2) Y = abs(np.fft.fft(y)) Y = Y[0:5000] f = f[0:5000] line2, = ax2.plot(f,Y) def update(b = widgets.ToggleButtons( options=['Recalculate','Recalculate'],disabled=False), L = widgets.IntSlider(min = 10, max= 1500, step=1, value=500), gain = widgets.FloatSlider(min = 0.8, max= 1, step=0.01, value=0.99), smooth = widgets.IntSlider(min = 1, max= 20, step=1, value=10)): print(b) y = karplus_strong(L,gain,smooth) t = np.linspace(0,len(y)/fs,len(y)) f = np.linspace(0,1,len(y)) Y = abs(np.fft.fft(y)) Y = Y[0:5000] f = f[0:5000] line.set_ydata(y) line2.set_ydata(Y) fig.canvas.draw_idle() ipd.display(ipd.Audio(y, rate=fs)) interact(update);
<IPython.core.display.Javascript object>
interactive(children=(ToggleButtons(description='b', options=('Recalculate', 'Recalculate'), value='Recalculat…