Week 8 and 9 Overview
What is Sound?
Sound Signals:
What is Sound?
- An object will emit sound if it vibrates between a rate of about 20 - 20,000 cycles/second or Hertz (Hz) and there exists a means for some of those vibrations to be converted to sound pressure variations.
- Sound is the vibration of air or variations of the ambient air pressure at a rate within the limited range of the human auditory system.
Sound Signals:
- Sounds are typically measured via transducers that convert air vibrations into electrical signals.
- Electrical sound signals are continuous-time representations of air pressure variations.
- These representations can subsequently be stored and/or processed using signal processing techniques.
Ideal Sinusoidal Signals
- A sinusoid is the simplest example of a periodic signal.
- The period or duration of a single cycle of a periodic waveform is given by the inverse of its frequency T = 1 / f0.
Sound Spectra
By Fourier theory, any waveform can be represented by a summation of a (possibly infinite) number of sinusoids, each with a particular amplitude and phase.
By Fourier theory, any waveform can be represented by a summation of a (possibly infinite) number of sinusoids, each with a particular amplitude and phase.
- A periodic waveform can be represented by a (possibly infinite) set of harmonically related sinusoids (whose frequencies are related by integer multiples).
- Conversely, a sound composed of non-harmonically related sinusoids cannot be periodic.
- The Fourier transform of a signal provides a "recipe'' for recreating that signal in terms of sinusoidal components.
- We can often learn much more about the "content'' of a complex waveform by viewing its spectrum or frequency-domain representation instead of (or in addition to) its time-domain representation.
- An ideal sinusoidal signal has only a single non-zero Fourier transform value.
- An ideal noise signal has a flat spectrum.
Audio Processing in MSP
MSP Signals
- Max objects are processed/updated at 1 millisecond intervals (referred to as the "control rate").
- Between control increments, MSP objects must perform their computations to satisfy the audio sample rate (typically 44100 samples per second on most Macintosh built-in audio devices).
- MSP objects perform "vectorized" computations ... they compute a set number of output samples before sending that data to the next "object" in the audio network. The number of samples per "vector" can be controlled in the DSP Status window (opened from the Options menu) via the "Signal Vector Size" parameter.
- Audio samples to be sent to the digital-to-analog converter (DAC) or read from the analog-to-digital converter (ADC) are "bundled" in buffers as well. The size of these input/output buffers is set via the "I/O Vector Size" parameter in the DSP Status window.
MSP Sine Wave (Cycle~)
- Audio output in MSP is accomplished with the dac~ object (digital-to-analog converter).
- The dac~ can be turned "on" or "off" with 1 or 0, or "start" and "stop", messages, respectively.
- Input signals to the dac~ must be in the range -1.0 < x < 1.0 or the output will clip/distort.
- Audio signal operators are specified by "tilde" symbols (i.e., *~, /~).
- The cycle~ object is a table lookup oscillator with inlets for frequency and phase offset.
- By default, the cycle~ object uses an internal table of 512 samples of a single period of a sine wave and linear interpolation to generate values between these 512 samples.
- The cycle~ object is an example of an object that produces audio-rate output but which can take either control- or audio-rate input (though not both at the same time).
MSP Amplitude Smoothing (Line~)
- Sudden jumps in signal amplitude will typically produce audible "clicks".
- The line~ object can be used to smooth amplitude (or other parameter) changes. It takes target/duration input pairs.
- The ezdac~ object combines a toggle button with the functionality of the dac~ object.
MSP Wireless
- The send~ and receive~ objects can be used to transmit audio signals without patch-cord connections.
- There are several distinctions between the send~ and receive~ objects and the Max send and receive objects:
- send~ and receive~ CANNOT be abbreviated with s~ and r~ identifiers.
- send~ and receive~ must always have a typed in identifier argument (though it can be changed with the set message).
- A receive~ object can only receive audio signals from a send~ object of the same name.
- One can dynamically change audio signal routings by changing the name of a receive~ object with a set message.
MSP Piecewise Linear Envelopes
- The noise~ object generates psuedo-random numbers in the range -1.0 < x < 1.0.
- The line~ object generates signal-rate linear envelopes.
- Up to 64 value-time pairs can be sent to a line~ object to produce multi-segment envelopes.
- The function object allows one to graphically create piece-wise linear envelopes that can be used by the line~ object.
- The right outlet of the line~ object sends a "bang" when the final destination point has been reached ... this can be used to re-trigger a new envelope.
MSP Resonance Filtering
- The reson~ MSP object implements a second-order digital resonance filter with parameters for gain, center frequency, and Q.
- The quality factor, Q = center frequency / 3dB bandwidth, will typically range between 1-100.
Time Domain and Frequency Domain Visualization Within MSP
Max Messages to MSP Signal Convertors
- The sig~ object takes a control-rate float or int value and outputs that value at the audio rate.
- The gain~ object provides an interpolating logarithmic signal fader.
- The number~ object can be used to monitor incoming signal values or generate a constant audio-rate signal.
MSP Signal to Max Message Convertors
- The average~ object computes a running average of an input signal over a specified number of samples. It can provide three different average types, "bipolar'', "absolute'', and "rms''.
- The avg~ object computes the average of the absolute value of an input signal between bangs.
- The meter~ object provides a graphical signal level meter.
- The snapshot~ object outputs the current audio sample value when a bang is received.
- The peakamp~ object reports the maximum absolute value of audio data between bangs or at regular intervals.
- The spike~ object reports intervals of zero to non-zero transitions.
- The zerox~ object counts zero-crossings, which can be used to distinguish transients from pitched signals.
Signal Feature Extraction in MSP
- The fiddle~ object, written by Miller Puckette, provides an estimate of the fundamental frequency and amplitude of an input sound. If specified, it will attempt to track more than one voice at the same time.
- A large number of objects are available from Tristan Jehan for estimating signal parameters and features.
- The brightness~ object (by Tristan Jehan) provides a ``brightness'' indicator (spectral centroid measure).
- The noisiness~ object (by Tristan Jehan) estimates noise in terms of spectral flatness.
- The beat~ object (by Tristan Jehan) is a signal beat and tempo detector.
- The analyzer~ (by Jehan) objects includes all of the above fuctionalities within it plus more
two modules within the Jamoma user lib exist that have efficiently wrapped the analyzer~ object:
They both can be downloaded from the Jamoma User Library on GitHub:
https://github.com/jamoma/JamomaUserLibraries
one uder the rdp library and the other under Orphans/MGT
- jmod.analyzer~
- jmod.rdp.analyzer~
They both can be downloaded from the Jamoma User Library on GitHub:
https://github.com/jamoma/JamomaUserLibraries
one uder the rdp library and the other under Orphans/MGT
The Zsa library encompasses a lot of recent trends in low level sound descriptor analysis in one library:
For basic yet robust envelope following take a look at the jmod.nav.env~ object within the jamoma user library, downloadable from here: https://github.com/navid/NavNav
NOTE: after downloading the package make sure to unzip the matrms~ zip file within the destributed folder or otherwise the module would not function
NOTE: after downloading the package make sure to unzip the matrms~ zip file within the destributed folder or otherwise the module would not function
Signal Matrixes
Research / homeWork:
A handful of jitter objects exist that convert signals into matrixes or matrixes into signals and etc. Open this patch, explore the help files and play around.
Also, refer to jitter tutorial #48 ehich contains some basic examples of these objects in use.
A handful of jitter objects exist that convert signals into matrixes or matrixes into signals and etc. Open this patch, explore the help files and play around.
Also, refer to jitter tutorial #48 ehich contains some basic examples of these objects in use.
Example Patch:
- the rand~ object is creating random MSP signals at the rate (Hz) specified in the UI
- The MSP signals are scaled using the *~ object
- Using jit.poke~ the audio signals are used to fill both planes of a 2 dimensional matrix.
- The resulted matrix is inputed into the right inlet of jit.rpos to modulate or remap the position of our target matrix (driven from the camera).
- Note that in this scenario the matrix produced by jit.poke was specifically created to best satisfy its intended destination (the right inlet of jit.repos)
- Notes: also look at and read through Jitter Tutorial #39 on the topic of spatial mapping to better comprehend the functionalities of jit.repos
Dynamic Mapping and Interaction Design in Jamoma
This patch extracts features from the camera to modulate sampled and synthesized sound and also demonstrates:
- How to route OSC messages from Jamoma Modules
- How to properly initialize and auto-calibrate complex Jamoma patches
- How to use presets to ramp from one mapping to another
- etc.
Audio-Vision
Please read the following section from Michel Chion's Audio-Vision:
- chapter one "Projection of sound on image" (Page 3-24)
- Please read all of chapter 4 "The Audio-Visual Scene" (pages 66-94)
- and then look at the final chapter "introduction to audiovisual analysis" (pages 185-192)