Raspberry Pi

The class Sound Synthesis at TU Berlin makes use of the Raspberry PI as a development and runtime system for sound synthesis in C++ (von Coler, 2017). Firtly, this is the cheapest way of setting up a computer pool with unified hard- and software. In addition, the PIs can serve as standalone synthesizers and sonification tools. All examples can be found in a dedicated software repository:

https://gitlab.tubit.tu-berlin.de/henrikvoncoler/SoundSynthesis_PI

The full development system is based on free, open source software. The examples are based on the JACK API for audio input and output, RtAudio for MIDI, as well as the liblo for OSC communication and libyaml-cpp for data and configuration files.

The advantage and disadvantage of this setup is that every element needs to be implemented from scratch. In this way, synthesis agorithms can be understood in detail and customized limitlessly. For quick solutions it makes sense to switch to a framework with more basic elements.

The source code can also be used on any linux system, provided the necessary libraries are installed.

The Gain Example

The gain example is the entry point for coding on the PI system:

https://gitlab.tubit.tu-berlin.de/henrikvoncoler/SoundSynthesis_PI/tree/master/examples/gain_example

References

  • Henrik von Coler and David Runge. Teaching sound synthesis in c/c++ on the raspberry pi. In Proceedings of the Linux Audio Conference. 2017.
    [BibTeX▼]
  • Links and Course Material

    TU Website

    The official TU website with information on the schedule and assessment:

    https://www.ak.tu-berlin.de/menue/lehre/sommersemester_2021/network_systems_for_music_interaction/

    TUB Cloud Drive

    The TUB cloud drive is used for additional materials like audio, scores and papers:

    https://tubcloud.tu-berlin.de/s/xeEjq6nWWTYy5BB

    Student Wiki

    This Wiki can be used for sharing knowledge:

    http://teaching-wiki.hvc.berlin

    SPRAWL GIT Repository

    The repository contains SHELL scripts, SuperCollider code and configuration files for the server and the clients of the SPRAWL system:

    https://github.com/anwaldt/SPRAWL

    JackTrip GIT Repository

    Jacktrip is the open source audio over IP software used in the SPRAWL system:

    https://github.com/jacktrip/jacktrip


    Back to NSMI Contents

    Using the Terminal for Doing Stuff

    People working a lot with MAC or Linux systems might be used to doing things from the terminal or console. For novices, it usually appears more frightening than it actually is. Especially when working on remote servers - but also for embedded devices for audio processing - the terminal is the standard tool.


    Directories

    You can maneuver through the system's directories using the cd command. Changing to an absolute path /foo/bar can be done like this:

    $ cd /foo/bar
    

    New directories are created with the mkdir command. For creating a new directory mydir in your recent location, type:

    $ mkdir mydir
    

    The content of the recent directory can be listed with the ls command. The following arguments improve the results:

    $ ls -lah
    
    drwxrwxr-x  2 username group 4,0K Mar 25 12:25 .
    drwxrwxr-x 16 username group 4,0K Mar 25 12:25 ..
    -rwxrwxr-x  1 username group 9,1M Feb  3 17:47 jacktrip
    -rwxrwxr-x  1 username group 334K Feb  3 17:47 sprawl-compositor
    

    Exercise

    Create a directory with your name in the student user home directory /home/student/students/YOURNAME.


    Create and Edit Files

    A file can be created by touching it:

    $ touch filename
    

    The default editor on most systems is nano. It is a minimal terminal based tool and does not require X forwarding. To edit an existing file or create a new file, type:

    $ nano filename
    

    Terminal Only

    Inside nano, you have a lot of defined keystrokes for editing, file handling and other tasks. See the nano cheat sheet for a full list: https://www.nano-editor.org/dist/latest/cheatsheet.html

    Exercise

    Create a text file in your personal directory and fill it with some text.


    GUI Based

    When working with X forwarding, simple text editors with GUI features can be used. On the SPRAWL server, this includes mouspad or the minimal Python IDE idle.


    Starting Applications

    System wide binaries must be located in a directory that is listed in $PATH (see the final section on this page for details). They can be started by simply typing the name:

    $ foo
    

    A local binary named foo can be started with the following command:

    $ ./foo
    

    You can terminate your command with an ampersand (&) to run a process in the background. You can continue to work in the terminal, afterwards:

    $ ./foo &
    [1] 5459
    

    If you start a command this way, it gives you an id of the background process in brackets and the actual process ID (PID).

    You can get the process back into foreground with the fg command followed by the background process id:

    $ fg 1
    

    Check for Running Applications

    At some point, users may want to know whether a process is running or which processes have been started. The command top lets you monitor the system processes with additional information on CPU and memory usage, updated with a fixed interval:

    $ top
    

    htop is a slightly polished version, using colored results:

    $ htop
    

    You can get a list of all running processes, including auxiliary ones, by typing:

    $ ps aux
    

    Usually, these are way to many results. If you want to check whether an instance of a specific program is running, you can use grep after the ps aux to filter the results:

    $ ps aux | grep foo
    

    Shell Variables

    Sometimes it is convenient to store information in variables for later use. Some common variables that are used in Unix like operating systems like Linux, BSD or MacOS are for example PATH and DISPLAY.

    Shell variables are usually uppercase. To get the content of a variable it is prefixed by a dollar sign. The command echo is used to print the content:

    $ echo $PATH
    /home/username/.local/bin:/home/username/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin
    $ echo $DISPLAY
    :0
    

    Defining a variable is done with an equal sign. It happens quite often that the program that should use the variable, opens another environment. To access the variable in that sub-environment, it has to be exported before:

    $ NAME=username
    $ echo $NAME
    username
    $ bash
    $ echo $NAME
    
    $ exit
    exit
    $ export NAME
    $ bash
    $ echo $NAME
    username
    

    A Brief History

    Beginnings of Computer Music

    Digital sound synthesis dates back to the first experiments of Max Mathews at Bell Labs in the 1950s. Mathews created the MUSIC programming language for generating musical sounds through additive synthesis on an IBM 704. The Silver Scale, realized by Newman Guttman in 1957, is (probably) the first ever digitally synthesized piece of music (Roads, 1980):

    MUSIC and its versions (I, II, III, ...) are direct or indirect ancestors to most recent languages for sound processing. Although the first experiments sound amusing from todays perspective, Mathews already grasped the potential of the computer as a musical instrument:

    “There are no theoretical limitations to the performance of the computer as a source of musical sounds, in contrast to the performance of ordinary instruments.” (Mathews, 1963)

    Mathews created the first digital musical pieces himself, but in order to fully explore the musical potential, he was joined by composers, artists and other researchers, such as Newman Guttman, James Tenney and Jean Claude Risset. Later, the Bell Labs were visited by renowned composers of various genres, including John Cage, Edgard Varèse and Laurie Spiegel (Park, 2009).


    Chowning & CCRMA

    The synthesis experiments at Bell Labs are the origin of most music programming languages and methods for digital sound synthesis. The foundation for many further developments was laid when John Chowning brought the software MUSIC VI to Stanford from a visit at Bell Labs in the 1060s. After migrating it to a PDP-6 computer, Chowning worked on his groundbreaking digital compositions, using the FM method and spatial techniques.


    Puckette & IRCAM

    Most of the active music programming environments, such as Puredata, Max/MSP, SuperCollider or CSound, are descendants of the MUSIC languages. Graphical programming languages like Max/MSP and Puredata were actually born as patching and mapping environments. Their common ancestor, the Patcher, developed by Miller Puckette at CCRMA in the 1980s, was a graphical environment for connecting MAX real-time processes and for controlling MIDI instruments.


    References

  • John Chowning. Turenas: the realization of a dream. Proc. of the 17es Journées d’Informatique Musicale, Saint-Etienne, France, 2011.
    [BibTeX▼]
  • Bilbao, Stefan. Numerical Sound Synthesis. Wiley Online Library, 2009. ISBN 9780470749012. doi:10.1002/9780470749012.
    [BibTeX▼]
  • Ananya Misra and Perry R Cook. Toward Synthesized Environments: A Survey of Analysis and Synthesis Methods for Sound Designers and Composers. In Proceedings of the International Computer Music Conference (ICMC 2009). 2009.
    [BibTeX▼]
  • Tae Hong Park. An interview with max mathews. Computer Music Journal, 33(3):9–22, 2009.
    [BibTeX▼]
  • Julius O. Smith. Viewpoints on the History of Digital Synthesis. In Proceedings of the International Computer Music Conference, 1–10. 1991.
    [BibTeX▼]
  • Curtis Roads and Max Mathews. Interview with max mathews. Computer Music Journal, 4(4):15–22, 1980.
    [BibTeX▼]
  • Max V Mathews. The Digital Computer as a Musical Instrument. Science, 142(3592):553–557, 1963.
    [BibTeX▼]
  • OSC: Open Sound Control

    Open Sound Control (OSC) is the de facto standard for exchanging control data between audio applications in distributed systems and on local setups with multiple components: http://opensoundcontrol.org/ Almost any programming language and environment for computer music offers means for using OSC, usually builtin.

    OSC is based on UDP/IP protocol in a client-server way. A server needs to be started for listening to messages sent from a client. For bidirectional communication, each participant needs to implement both a server and a client. Servers are start listening on a freely chosen port, whereas clients send their messages to an arbitrary IP address and port.

    Typical OSC Messages

    A typical OSC message consists of a path and an arbitrary number of arguments. The following message sends a single floating point value, using the path /synthesizer/volume/:

    /synthesizer/volume/ 0.5
    

    The path can be any string with slash-separated sub-strings, as paths in an operating system. OSC receivers can sort the messages according to the path. Parameters can be integers, floats and strings. Unlike MIDI, OSC offers only the transport protocol but does not define a standard for musical parameters. Hence, the paths used for a certain software are completely arbitrary and can be defined by the developers.

    First Sounds with SuperCollider

    Boot a Server

    Synthesis and processing happens inside an SC server. So the first thing to do when creating sound with SuperCollider is to boot a server. The ScIDE offers menu entries for doing that. However, using code for doing so increases the flexibility. In this first example we will boot the default server:

    // boot the server
    s.boot;
    

    A First Node

    In the SC server, sound is generated and processed inside nodes. These nodes can later be manipulated, arranged and connected. A simple node can be defined inside a function curly brackets:

    // play a sine wave
    (
    {
        // calculate a sine wave with frequency and amplitude
        var x = SinOsc.ar(1000);
    
        // send the signal to the output bus '0'
        Out.ar(0, x);
    
    }.play;
    
    )
    

    In the ScIDE, there are several ways to get information on the active nodes on the SC server. The node tree can be visualized in the server menu options or printed from sclang, by evaluating:

    s.queryAllNodes
    

    After creating just the sine wave node, the server will show the following node state:

    NODE TREE Group 0
       1 group
          1001 temp__1
    

    The GUI version of the node tree looks as follows. This representation is updated in real time, when left open:

    /images/basics/sc-nodes-1.png

    Note

    The server itself does not know any variable names but addresses all nodes by their ID. IDs are assigned in an ascending order. The sine wave node can be accessed with the ID 1001.


    Removing Nodes

    Any node can be removed from a server, provided its unique ID:

    s.sendMsg("/n_free",1003)
    

    All active nodes can be removed from the server at once. This can be very handy when experiments get out of hand or a simple sine wave does not quit. It is done by pressing Shift + . or evaluating:

    // free all nodes from the server
    s.freeAll
    

    Running SC Files

    SuperCollider code is written in text files with the extensions .sc or .scd. On Linux and Mac systems, a complete SC file can be executed in the terminal by calling the language with the file as argument:

    $ sclang sine-example.sc

    The program will then run in the terminal and still launch the included GUI elements.

    Getting Started with Puredata

    About

    Puredate (PD) is the free and open source version of Max/MSP, also developed and maintained by Miller Puckette. PD is one of the best options for people new to computer music, due to the obvious signal flow. It is a very helpful for exploring the basics of sound synthesis and programming but can also be used for advanced applications: https://puredata.info/community/member-downloads/patches In addition, PD offers simple and flexible means for creating control and GUI software.

    Installing Extensions

    The official standard version of Pure Data is referred to Pd-vanilla. It features a Deken, a framework to for installing externals from within Pure Data. Some of the examples in this section make use of such extensions. The GUI version of Deken is located in the menu under Help->Find Externals. After downloading an extension to the default location of the operating system, the paths need to be added, manually in Edit->Preferences->Path.

    This site is very helpful when figuring out which extension a missing object belongs to: http://write.flossmanuals.net/pure-data/audio-filters/

    The Sine Example

    This first example creates a sine wave oscillator. Its frequency can be controlled with a slider:

    /images/basics/pd-sine.png
  • Miller S. Puckette. Pure Data. In Proceedings of the International Computer Music Conference (ICMC). Thessaloniki, \\ Greece, 1997.
    [BibTeX▼]
  • Miller S. Puckette. The patcher. In Proceedings of the International Computer Music Conference (ICMC). Computer Music Association, 1988.
    [BibTeX▼]
  • Using JACK Audio

    The JACK API implements an audio server, allowing the connection of various software clients and hardware interfaces. In short, it turns the whole system into a digital audio workstation (DAW). It is the the standard on Linux (pro) audio systems, but is also available for Mac and Windows. JACK needs a backend to connect to the actual audio hardware. On Linux systems, this is usually ALSA (Mac uses Core Audio and Windows ASIO).


    Starting a JACK Server

    A JACK server can be started via various graphical tools, such as QjackCtl or Carla. Many audio programs also boot a JACK server o their own, if launched. The recommended way of starting a JACK server in our case is the terminal, using the command jackd. It takes several arguments. The following line starts a server (in the background) with the ALSA interface named PCH, using a sample rate of 48kHz and a buffer size of 128 samples.

    $ jackd -d alsa -d hw:PCH -r 48000 -p 128 &
    

    Finding ALSA Devices

    One way of finding the ALSA name of your interface is to type the following command:

    $ aplay -l
    

    The output shows all ALSA capable devices, their name listed after the card x:. PCH is usually the default onboard sound card:

    **** List of PLAYBACK Hardware Devices ****
    card 0: HDMI [HDA Intel HDMI], device 3: HDMI 0 [HDMI 0]
      Subdevices: 1/1
      Subdevice #0: subdevice #0
    card 0: HDMI [HDA Intel HDMI], device 7: HDMI 1 [HDMI 1]
      Subdevices: 1/1
      Subdevice #0: subdevice #0
    card 0: HDMI [HDA Intel HDMI], device 8: HDMI 2 [HDMI 2]
      Subdevices: 1/1
      Subdevice #0: subdevice #0
    card 0: HDMI [HDA Intel HDMI], device 9: HDMI 3 [HDMI 3]
      Subdevices: 1/1
      Subdevice #0: subdevice #0
    card 0: HDMI [HDA Intel HDMI], device 10: HDMI 4 [HDMI 4]
      Subdevices: 1/1
      Subdevice #0: subdevice #0
    card 1: PCH [HDA Intel PCH], device 0: CX20751/2 Analog [CX20751/2 Analog]
      Subdevices: 1/1
      Subdevice #0: subdevice #0
    

    Connecting JACK Clients

    As almost everything, JACK connections can be modified from the terminal. All available JACK ports can be listed with the following command:

    $ jack_lsp
    

    Two ports can be connected with the following command:

    $ jack_connect client1:output client2:input
    

    Disconnecting two ports is done as follows:

    $ jack_disconnect client1:output client2:input
    

    If possible, a GUI-based tool, such as QjackCtl, can be more handy for connecting clients. It can be started via the a Desktop environment or from the command line:

    $ qjackctl
    

    /images/basics/qjackctl_connect.png

    QjackCtl with hardware connections and two clients.


    Storing/Restoring Connections

    Several tools allow to store and restore JACK connections. Some of them work in a dynamic way, detecting spawned clients and connecting them accordingly. Others just allow a single operation for restoring connections.

    aj-snapshot

    The command line tool aj-snapshot is automatically installed alongside JACK. It can store and restore both JACK and ALSA connections, which can be handy when working with MIDI and is the most feature-rich and robust solution.

    Once all connections are set, they can be stored to in an XML file, specified by a single argument:

    aj-snapshot connections.snap
    

    The above stored connections can be restored with the flag -r. An additional x deletes all prior connections, thus restoring the original state in the file:

    aj-snapshot -xr connections.snap
    

    The tool can also be started as a demon, looking for new clients and setting the related connections:

    aj-snapshot -d connections.snap
    

    Note

    In some spatial audio projects, hardware devices and clients can have many a large number of ports. aj-snapshot does not handle that well and takes an excessive amount of time for deleting existing connections.


    jmess

    jmess is another command line tool, storing and restoring only JACK connections. It does not come with a demon mode but is a lot faster than aj-snapshot.


    jack-matchmaker

    jack matchmaker is a Python-based command line tool for dynamically restoring previously saved JACK connections.


    QjackCtl Patchbay

    The QjackCtl Patchbay offers a graphical solution for storing JACK and ALSA connections. Once activated, it patches new clients dynamically.

    Using SSH for Remote Access

    SSH (Secure Shell Protocol) is necessary when working with the server but can also be helpful for configuring the Access Points. For remote machines - like the SPRAWL Server - SSH can be used for command-line operations and command execution.

    Connecting to an SSH Server

    For connecting to a remote machine, it needs to run an SSH server. On the client side, an SSH connection can be established without additional installations from the terminal on Linux and MAC machines and - since version 10 - from Windows. For older Windows versions, users can use Putty.

    $ ssh username@address
    

    X11 Forwarding

    With X11 Forwarding, SSH can also be used to run applications with a GUI, remotely. Simply add the -X argument to do so:

    $ ssh -X username@address
    

    Remote Commands

    SSH can also be used to send single commands, without starting a remote session. This example launches the jack_simple_client, which plays a continuing sine tone on the remote machine.

    $ ssh -t username@address 'jack_simple_server'
    

    Exercise

    Log into the server with SSH.



    Contents © Henrik von Coler 2021 - Contact