Compendium

Sound Synthesis — Deep Taxonomy

29 min read

Sound Synthesis — Deep Taxonomy

A practitioner’s deep reference to every commercially significant synthesis method: the mathematics underneath each engine, the canonical hardware and software implementations, the patches and producers that established each idiom, and the tradeoffs that determine which method to reach for. Companion to the Tier-1 sound-synthesis-and-electronic-music note, which surveys the field at a higher altitude. This note drills down into the algorithms.

See also

1. The synthesis taxonomy

Every synthesis method makes a different bet about which representation of a sound wave is the most musically tractable. Group them by domain:

  • Time-domain generators produce sample streams directly. Subtractive, FM, phase distortion, wavetable, granular, physical modeling.
  • Frequency-domain generators specify a target spectrum and inverse-transform. Additive, spectral resynthesis.
  • Hybrid engines mix domains: Phase Plant, Harmor, Kyma, modern wavetables with per-partial spectral processing.

A second axis is causal model vs signal model. Subtractive and additive are signal models — they shape arbitrary spectra. Physical modeling is a causal model — it simulates the source (string, tube, membrane). The two camps converged in the 2010s with modal synthesis and DDSP-style neural physical models.

A third axis is control rate vs audio rate. LFOs and envelopes are control-rate (≤200 Hz); audio-rate modulation (operator FM, sync, ring mod) creates sidebands and is where most of the interesting timbres live.

2. Subtractive synthesis

Subtractive starts with a harmonically rich source and removes content. The musical decisions are the filter type, the resonance, the envelope on the cutoff, and the saturation in the filter path.

2.1 The Moog ladder filter

Robert Moog patented the transistor ladder filter in 1969 (US Patent 3,475,623). Four cascaded transistor pairs each contribute a 6 dB/octave low-pass section, summing to 24 dB/octave. The defining sonic feature is self-oscillation at high resonance — the ladder becomes a sine oscillator when feedback is critical — and gain-droop at high resonance, which removes low end as you turn the resonance up. Modern emulations that don’t model the droop sound wrong on a sweep.

Canonical Moog instruments using the ladder:

  • Minimoog Model D (1970) — 3 VCOs, 24 dB ladder, no patch memory; the lead sound of Sun Ra, Kraftwerk, Parliament-Funkadelic, Stevie Wonder. Reissued 2016, current 2022 production.
  • Moog Voyager (2002–2015) — programmable Minimoog with TouchSurface XYZ controller; Bob Moog’s last design before his death in 2005.
  • Moog Sub 37 / Subsequent 37 / Subsequent 25 — paraphonic 2-voice with Multidrive (overdrive into the filter); 256 patches.
  • Moog One (2018) — 8/16-voice polyphonic with three filter types per voice (ladder, state-variable, Moog/Oberheim hybrid).
  • Moog Matriarch / Grandmother / Mother-32 — semi-modular Eurorack-compatible ladder synths.

2.2 The Oberheim SEM filter

Tom Oberheim’s Synthesizer Expander Module (1974) used a 12 dB/oct state-variable filter (Curtis CEM 3320 later) with continuously variable low-pass to notch to high-pass to band-pass response. The SEM is brighter than the Moog and lets you sweep through filter modes without clicks. SEM-style filters define the Oberheim sound on Van Halen “Jump,” Rush “Tom Sawyer,” Styx “Mr. Roboto.”

Canonical SEM-bearing instruments:

  • Oberheim SEM (1974) — single-voice module; stacked SEMs became the 2-voice, 4-voice, 8-voice systems.
  • Oberheim OB-X (1979), OB-Xa (1980), OB-8 (1983) — 4/8-voice polyphonics with discrete SEM-style filters (OB-X) then CEM-3320 (OB-Xa, OB-8).
  • Oberheim OB-X8 (2022) — Tom Oberheim’s authentic revival; selectable OB-X, OB-Xa, and OB-8 voice modes; 8-voice analog.
  • Oberheim Matrix-12 (1985) — 12-voice flagship with two SEM-style filters per voice and a deep modulation matrix.

2.3 The Prophet line and the SSM/CEM era

Dave Smith’s Sequential Circuits Prophet-5 (1978) was the first programmable polyphonic synthesizer with patch memory. Five VCOs (two per voice plus LFO/aux) and a 24 dB SSM 2040 (Rev 1, 2) or CEM 3320 (Rev 3) ladder. The Prophet-5’s tuning stability and 40-patch memory made it the default studio polysynth for late-70s and early-80s pop. Rev 1 + 2 are warmer (drift); Rev 3 is tighter.

Canonical Prophet-family hardware:

  • Sequential Prophet-5 (1978, reissued 2020) — 5-voice; reissue offers Rev 1/2 (SSM) and Rev 3 (CEM) modes.
  • Sequential Prophet-10 (1981, reissued 2020) — 10-voice Prophet-5.
  • Sequential Prophet-6 (2015) — modern 6-voice all-discrete, 2-pole + 4-pole switchable filter.
  • Sequential Prophet T8 (1983) — Prophet-5 voice with 76-key weighted keyboard.
  • Sequential Prophet VS (1986) — vector-synthesis hybrid (digital + analog filter).
  • Sequential Pro 3 (2020) — 3-voice paraphonic mono with three filters and 32-step sequencer.
  • Sequential Trigon-6 (2022) — Moog-style ladder filter in a Prophet-6 chassis.
  • Sequential Take 5 (2021) — affordable 5-voice with Prophet-6 voice and Pro 3 mod matrix.

Dave Smith died in 2022; Sequential continues under Focusrite ownership.

2.4 Filter topology cheat sheet

  • Sallen-Key (Korg MS-20, 12 dB self-oscillating) — Hideki Matsutake’s circuit; brittle, screams at resonance.
  • State-variable (SEM, Oberheim) — simultaneous LP/HP/BP/notch outputs; smooth, vocal.
  • Ladder (Moog, ARP 2600) — fat, with characteristic resonance droop.
  • Diode ladder (Roland TB-303, EMS VCS3) — squelchier than transistor ladder; defines acid.
  • OTA-based (CEM/SSM chips on Prophet, Memorymoog, Pro-One) — Curtis 3320, 3372, 3389 and Solid State Microtechnology 2040, 2044; cleaner than discrete ladder.
  • Steiner-Parker (Arturia Brute series) — Nyle Steiner’s 1974 design; aggressive, unique self-oscillation character.
  • Korg MS-20 / Polysix — Korg-35 chip filter; gritty, prone to self-destruction near full resonance.

2.5 Modern subtractive software

  • u-he Diva (2011) — circuit-level analog modeling with selectable Moog, MS-20, Roland, Oberheim, Sequential voicing; CPU-hungry but the gold standard for “analog in a plugin.”
  • u-he Repro-1 (Pro-One) and Repro-5 (Prophet-5) — single-purpose Sequential clones with per-component modeling.
  • Arturia V Collection 10 (2024) — every Moog/ARP/Oberheim/Sequential model plus DX7, CS-80, Synclavier, Fairlight, more.
  • TAL-J-8 (Jupiter-8), TAL-U-No-LX (Juno-60), TAL-Mod — Togu Audio Line’s affordable Roland emulations.
  • Cherry Audio Mercury-6, Polymode, GX-80 — Prophet-5, Minimoog, CS-80 emulations at $49–99.
  • Native Instruments Monark, Reaktor Blocks — Moog Model D and modular building blocks.
  • Softube Model 72 (Minimoog), Model 84 (Juno-106) — high-fidelity Eurorack-cousin emulations.

3. Additive synthesis

Additive sums sinusoids — each with its own frequency, amplitude, and phase — to construct a complex tone. Fourier’s theorem guarantees any periodic waveform can be built this way; the cost is parameter explosion. A 64-partial sound with time-varying envelopes is 64 × N control points per voice. The musical question is how to abstract the parameter space.

3.1 Canonical additive instruments

  • Hammond B-3 organ (1935, drawbar design 1937) — Laurens Hammond’s tonewheel additive: 9 drawbars per manual at 16’, 5⅓’, 8’, 4’, 2⅔’, 2’, 1⅗’, 1⅓’, 1’ (subharmonic, fundamental, octave, octave + fifth, two octaves, etc.). 91 spinning tonewheels generate sines; drawbars mix them. The original commercial additive instrument.
  • Kawai K5000S / K5000W / K5000R (1996) — 64 partials per source, four sources per patch, macro envelopes that group partials, formant filters; the most ambitious commercial digital additive. Discontinued; cult following.
  • Synclavier II (1980) — New England Digital’s $200,000 system; FM + additive partial synthesis + sample resynthesis. Used by Frank Zappa, Stevie Wonder, Sting; resurrected as Synclavier Go! plugin (2022).
  • VirSyn Cube 2 — dedicated additive softsynth with 512 partials and morphing.
  • Camel Audio Alchemy (2007, acquired by Apple 2015) — additive + spectral + granular + virtual analog; bundled with Logic Pro since 10.2 (2015).
  • Image-Line Harmor — additive/spectral hybrid with re-synthesis from audio files and bitmap images; partial-by-partial filtering and pitch.
  • Native Instruments Razor (Reaktor ensemble) — Errorsmith’s 320-partial additive with morphable noise; “Razor leads” defined a Skrillex-era brostep timbre.
  • MetaSynth (1998, U&I Software) — paint-the-spectrum picture-to-sound additive; Aphex Twin used it on Windowlicker’s spectrogram-encoded face.

3.2 The DX7 algorithm 32 trick

The Yamaha DX-7 (1983) implemented FM, but in algorithm 32 all six operators are parallel sines with no modulation — pure additive. Configure operator frequencies to harmonic-series ratios (1, 2, 3, 4, 5, 6 of the fundamental), set per-operator envelopes, and you have a 6-partial additive synth with the DX-7’s clean digital sines. Many “DX-7 bell” patches exploit this directly.

3.3 Differentiable additive (DDSP)

Google Magenta’s DDSP (Differentiable Digital Signal Processing, 2020, Engel et al.) trains a neural network to drive a fixed additive synth from features extracted from a target instrument recording. The synth is differentiable, so backprop optimizes partial envelopes to match. Recovers near-physical timbres without the per-partial tuning burden; the technique underlies several 2023–2026 timbre-transfer tools (Mawf, Neutone DDSP models).

4. FM synthesis — Chowning’s algebra

John Chowning at Stanford CCRMA discovered FM as an audio synthesis method in 1967 and published “The Synthesis of Complex Audio Spectra by Means of Frequency Modulation” in the Journal of the Audio Engineering Society in 1973. Stanford licensed it to Yamaha; the Yamaha DX-7 (1983) sold over 200,000 units and defined the digital-keys, slap-bass, electric-piano, and glassy-bell sound of mid-80s pop.

4.1 Theory

Modulate the frequency of a carrier sine with another sine. If the modulator is at audio rate, sidebands appear at carrier ± k·modulator for integer k, with amplitudes given by Bessel functions of the first kind J_k(I), where I is the modulation index (modulator amplitude / modulator frequency).

Key relationships:

  • C:M ratio (carrier-to-modulator frequency ratio) determines the spectrum’s harmonic structure. Integer ratios (1:1, 1:2, 2:3) produce harmonic spectra. Non-integer ratios (1:1.414, 1:π) produce inharmonic spectra — bells, gongs, metallic tones.
  • Modulation index I controls how many sidebands are audible and how strong. Vary I over time (with an envelope on the modulator amplitude) and the spectrum evolves — the DX-7 electric piano is a 1:1 ratio with a decaying modulation index from ~3 to 0.
  • Feedback on an operator creates self-modulation: an operator modulating itself converges toward a sawtooth as feedback increases.

4.2 The DX-7 architecture

Six sine operators arranged in one of 32 algorithms (factory routings). Each algorithm specifies which operators are carriers (output to the mix) and which are modulators (feed into other operators’ frequency input). Algorithm 1 is a stack: 6→5→4→3→carriers 1+2. Algorithm 32 is all six in parallel.

Each operator has:

  • Frequency ratio or fixed frequency (the C in C:M).
  • Output level 0–99 (the modulation index when used as a modulator, the mix level when used as a carrier).
  • EG (envelope) with four rates and four levels (R1/L1, R2/L2, R3/L3, R4/L4); a non-ADSR envelope that gave the DX-7 its expressive timbre evolution.
  • Keyboard scaling (level + rate) so the timbre tracks pitch.
  • Velocity sensitivity on level + rate.

The DX-7’s 12-bit sine LUT and lack of analog output filtering give it a slightly gritty, “digital” edge; the DX-7 II FD (1986) added 16-bit conversion.

4.3 The FM family

  • Yamaha DX-7 (1983) — 6 operators, 32 algorithms, 16-voice polyphony.
  • Yamaha DX-7 II FD (1986) — micro-tuning, dual mode, floppy disk; 12-bit→16-bit.
  • Yamaha DX-9 (1983) — budget DX-7 with 4 operators, 8 algorithms; mostly forgotten.
  • Yamaha DX-1 (1983) — wood-paneled flagship with two DX-7 engines, weighted keys; $13,900 at launch.
  • Yamaha DX-5 (1985) — DX-1 voice in a more affordable chassis.
  • Yamaha TX-816, TX-802, TX-7, TX-81Z — rack and module FM family. The TX-81Z (1987) added 8 alternative operator waveforms (not just sine), enabling timbres impossible on a DX-7.
  • Yamaha SY77 (1989), SY99 (1991) — RCM (Realtime Convolution & Modulation) — 6-op FM + sample playback + filter (the missing piece on DX-7).
  • Yamaha FS1R (1998) — 8 operators, 88 voiced+unvoiced operator waveforms, formant-sequenced “fseq” data; the most powerful Yamaha hardware FM ever, criminally rare.
  • Yamaha DX-200, FS1R Editor, Reface DX (2015), MODX/MONTAGE (2016–2024) — modern Yamaha FM continuations; MONTAGE M (2023) is the current flagship with FM-X + AWM2 + 16-part multitimbral.
  • Korg Volca FM (2016), Volca FM2 (2021), opsix (2020), opsix mkII (2023) — Korg’s modern FM (the opsix has 6 operators and 8 operator waveforms including ring mod, comb, wavefolder; the opsix mkII adds spectral filter).
  • Elektron Digitone (2018) — 4-operator FM with multi-mode filter and sequencer; Elektron parameter-locking made FM patches programmable per-step.

4.4 Software FM

  • Native Instruments FM8 (2006) — 8 operators, 14 algorithms (plus user algorithms); the DX-7’s heir in plugin form, with operator waveforms beyond sine.
  • Arturia DX7 V — sample-accurate DX-7 recreation in V Collection.
  • Plogue OPS7 — DX-7 in software.
  • dexed (free, open source) — DX-7 emulator with sysex import (loads original DX-7 patches).
  • Tone2 RayBlaster — impulse-modeled FM-adjacent.

4.5 PM vs FM

The DX-7 actually uses phase modulation (PM), not true FM — modulating the phase of a sine instead of its frequency. The spectra are identical for stationary modulators, but PM is easier to implement cleanly and the modulator envelope behaves more musically (no DC offset issues). All “FM” digital synthesizers since the DX-7 use PM under the hood.

5. Phase distortion synthesis

Casio’s answer to Yamaha FM in 1984. Phase distortion (PD) reads a fixed sine LUT with a non-uniformly-incremented phase pointer — speed up the read through certain regions, slow down through others — producing a distorted waveshape with controllable harmonic content. Two carrier waveshapes per voice can be modulated; the spectra resemble FM but the parameter space is friendlier (a single “PD index” per oscillator).

  • Casio CZ-101 (1984), CZ-1000, CZ-3000, CZ-5000, CZ-1 (1986) — 8 waveshapes, ring modulation, noise, 8-stage envelopes; the CZ-101 was 4-voice, $499, a bedroom-studio gateway.
  • Casio VZ-1, VZ-8M, VZ-10M (1988) — interactive phase distortion (iPD), a more flexible PD that could approximate FM.
  • Plogue Chipsynth CZ (2020) — accurate CZ emulation plugin.
  • VirSyn Reflect (2020s) — PD-influenced softsynth.

6. Wavetable synthesis

A wavetable is an array of single-cycle waveforms. The synth scans through the array on a “position” parameter, smoothly interpolating between adjacent waveforms; modulating the position with an LFO or envelope produces evolving timbres impossible with subtractive.

6.1 The PPG era

Wolfgang Palm’s PPG Wave 2 (1981), 2.2 (1982), 2.3 (1984) was the first commercial wavetable synth: 8 voices, 30 wavetables of 64 waveforms each, 8-bit DACs, analog SSM 2044 filter per voice. The PPG sound is on Tangerine Dream, Depeche Mode “Construction Time Again,” Trevor Horn productions (Frankie Goes to Hollywood, ABC), Tears for Fears “Songs from the Big Chair.” Discontinued 1987 when PPG went bankrupt; Wolfgang Palm later founded Waldorf.

6.2 The Waldorf line

  • Waldorf Microwave (1989) — rack PPG-style wavetable.
  • Waldorf Wave (1993) — flagship; weighted keys, 48 voices, $20,000 at launch.
  • Waldorf Microwave II / XT (1997–2000) — 10-voice, 64 wavetables.
  • Waldorf Q (1999), Q+ (2006) — virtual-analog + wavetable.
  • Waldorf Blofeld (2007) — desktop wavetable + virtual analog at $499; massive bedroom-studio uptake.
  • Waldorf Iridium (2020), Iridium Keyboard (2021), Iridium Core (2023) — current flagship with wavetable + waveform + particle (granular) + resonator (modal) + kernel (FM) engines.
  • Waldorf Quantum (2018), Quantum Mk2 (2023) — Iridium with analog filters per voice.

6.3 Modern software wavetables

The dominant pop/EDM synthesis paradigm in 2024–2026:

  • Xfer Records Serum (2014) — Steve Duda’s wavetable plugin; built-in wavetable editor with FFT, formula, and audio import; the de facto standard for EDM, future bass, hyperpop. Serum 2 announced 2024, released 2025 with revised UI.
  • Vital (2020, free + paid) — Matt Tytel’s open-source Serum competitor; cross-platform, MPE-aware, free tier covers most use cases.
  • Native Instruments Massive X (2019) — successor to Massive (2007); 170+ wavetables, 10 modular modulators, performance routing.
  • Native Instruments Massive (2007) — original; defined the brostep/dubstep wobble bass alongside Sylenth1.
  • u-he Hive 2 — affordable, CPU-light wavetable.
  • Arturia Pigments 5 (2024) — multi-engine including wavetable, virtual analog, sample, harmonic; one of the most flexible plugin synths.
  • Kilohearts Phase Plant — modular wavetable + sample + analog + granular + FM in a snap-together patcher.
  • Ableton Wavetable (Live 10+, 2018) — Live’s built-in wavetable; lighter than Serum, good integration with Live’s Operator and Drift.
  • Bitwig Polymer, Phase-4, Grid — Bitwig’s modular and dedicated wavetable engines.

6.4 Spectral wavetables

Modern wavetables process per-frame in the frequency domain. Serum can apply filters, formants, FM modulation, and noise to a wavetable in spectral form before resynthesizing. This blurs the boundary between wavetable and additive — the wavetable is effectively a frozen additive spectrum, and per-partial operations are first-class citizens.

7. Vector synthesis

Vector synthesis (Sequential Prophet VS, 1986) places four sound sources at the corners of an X/Y joystick plane and crossfades between them via joystick position. Korg Wavestation (1990) extended this to wavetable-style sequenced vectors. The VS engine appears in:

  • Sequential Prophet VS (1986) — 4 digital oscillators + analog filter per voice; 8-voice.
  • Korg Wavestation (1990), Wavestation A/D, Wavestation SR — wave sequencing + vector synthesis; Korg Legacy Collection plugin (and KORG Collection Wavestation, 2017).
  • Korg Wavestate (2020) — modern Wavestation with deeper wave sequence engine.
  • Yamaha SY22 / SY35, TG33 (1990) — vector synthesis on Yamaha.

8. Physical modeling synthesis

Physical models simulate the source: a string equation, a tube resonator with reflection coefficients, a membrane PDE. The output is causally tied to physical parameters (string tension, tube length, bow pressure) rather than spectral parameters (filter cutoff, partial amplitude). The classic algorithms:

8.1 Karplus-Strong (1983)

Kevin Karplus and Alex Strong, “Digital Synthesis of Plucked-String and Drum Timbres,” Computer Music Journal 1983. Excite a delay line of length N samples with a burst of noise; feed the delay output back through a low-pass filter into the input; the noise decays into a pitched tone at fundamental f_0 = sample_rate / N, with realistic harmonic decay because the low-pass attenuates each round-trip. The algorithm models the wave equation on a stiff string with damping; with variations (all-pass for inharmonic stiffness, comb filtering for plectrum position) it captures plucked-string realism with minimal CPU.

8.2 Waveguide synthesis

Julius O. Smith III at Stanford CCRMA generalized Karplus-Strong to bidirectional delay lines representing left- and right-moving waves on a string or in a tube, with reflection filters at boundaries representing instrument body, bridge, embouchure, valves. Stanford patented waveguide synthesis; Yamaha licensed it for:

  • Yamaha VL1 (1994), VL1-m (1994), VL7 (1994) — 2-voice solo wind/string physical modeling; required a breath controller (BC-3, WX-7) for expression. Underpowered hardware (2 voices for $5,000) but the timbres — clarinet, sax, flute, violin — were uncanny for 1994.
  • Yamaha VP1 (1994) — 16-voice polyphonic physical modeling prototype; only 100 built.
  • Korg Prophecy (1995) and Korg Z1 (1997) — monosynth and polysynth with several physical models (string, tube, brass, reed) alongside virtual analog.
  • Technics WSA1 (1995) — Acoustic Modeling Synthesis hybrid.

8.3 Modal synthesis

Decomposes a resonant object into a sum of damped sinusoids (modes), each with frequency, amplitude, and damping. The mode parameters can be measured (by impulse-response analysis) or computed (by FEA of the object’s shape). Modal synthesis is computationally cheap once modes are extracted and is the basis for:

  • Mutable Instruments Rings (2016, Eurorack) — 80-partial modal resonator; the “ringing metal” sound on modular techno.
  • Mutable Instruments Elements (2014) — modal + exciter + reverb.
  • Madrona Labs Aalto, Kaivo, Sumu — boutique modal/granular/additive softsynths by Randy Jones.
  • Applied Acoustics Systems Chromaphone 3 (2018) — modal resonators (string, beam, drumhead, membrane, plate) excited by mallets, noise bursts, audio.

8.4 Pianoteq

Modartt Pianoteq (2006, current v8.3 in 2024) is a fully physical-model piano: no samples, just a real-time string/hammer/soundboard simulation. 50 MB install vs 50+ GB for sampled pianos like Synthogy Ivory or Native Instruments Noire. Trade-off: less raw realism on hyper-detailed individual notes, but seamless legato, velocity layers, repetition, and per-string sympathetic resonance — and modeling parameters (string length, hammer hardness, unison detuning) are user-tweakable. Used by Hans Zimmer (“Inception”), Trent Reznor + Atticus Ross, many film composers needing a piano that runs in 50 MB on a laptop.

8.5 MAX/MSP, Pd, gen~

Cycling ‘74 Max (current Max 8.6 in 2024) and Miller Puckette’s Pure Data (Pd) are the canonical environments for building physical models from primitives. The gen~ extension in Max compiles sample-accurate DSP to native code; entire physical models (waveguide flute, mass-spring lattice, FM bell) are constructed and modified live. Used in IRCAM compositions, computer-music PhDs, and by sound designers like Diego Stocco.

8.6 Modalys (IRCAM)

IRCAM’s Modalys is a research-grade physical modeling environment with primitives for strings, membranes, tubes, plates, and connections (bows, hammers, reeds). Used for instrument prototyping and computer-music composition since the 1990s; integrates with OpenMusic and Max.

9. Granular synthesis

Granular synthesis splits a source (sample or synthesized signal) into short overlapping grains (typically 5–200 ms) and re-sequences them with control over grain pitch, position, density, pan, and envelope. Dennis Gabor proposed the time-frequency atom in 1947; Curtis Roads developed granular synthesis at CalArts in the 1970s and wrote the standard reference Microsound (MIT Press, 2001).

9.1 Grain parameters

  • Grain size: 5–20 ms = noise-like cloud, 20–100 ms = textural smear, 100–500 ms = audible time-stretch, >500 ms = sample triggering.
  • Grain density: grains per second; low = sparse pointillism, high = continuous texture.
  • Position scan: where in the source the grain reads from; static = freeze, slow scan = time-stretch, fast = playback.
  • Pitch / size decoupling: independent control of grain pitch (how fast the sample plays back inside the grain) and grain size (envelope length).
  • Window function: Gaussian, Hann, Tukey, triangular, rectangular; affects spectral spread (rectangular grains have ringing sidebands; Gaussian grains are spectrally clean).

9.2 Software granular instruments

  • Ableton Granulator II (2014, free Max for Live device) — Robert Henke’s granular sampler; the canonical introduction.
  • Native Instruments Form (2017) — wavetable + granular hybrid in Reaktor.
  • Output Portal (2018) — granular effect plugin.
  • Twisted Tools Ultraloop, Soundmorph Twist — granular sample mangling.
  • Tone2 Nemesis 2 — granular delay.
  • Glitchmachines Polygon, Quadrant — granular drum and texture instruments.
  • Madrona Labs Kaivo — physical model + granular.
  • Spitfire LABS Granular (free), Spitfire BBC Symphony Orchestra granular patches — orchestral granular textures.
  • MeldaProduction MGranularMB — multiband granular FX.

9.3 Hardware granular

  • Tasty Chips GR-1 (2017), GR-Mega (2022) — dedicated hardware granular samplers; 24-voice GR-1 has its own dedicated screen for grain visualization.
  • 1010music Bitbox MX, Blackbox — desktop samplers with granular modes.
  • Mutable Instruments Clouds (2014, discontinued 2017) — Eurorack granular module; cult classic; spawned the entire “Clouds-clone” niche of Eurorack (Music Thing Modular Mikrophonie, Plum Audio Beads, etc.).
  • Mutable Instruments Beads (2021, the Clouds successor) — granular Eurorack with more grains, looper mode, and stereo signal path; later open-sourced and now built by clones (After Later Audio Beads, Plum Audio).
  • Tiptop Audio Z-DSP — DSP cartridge platform with granular cartridges.
  • Make Noise Morphagene — tape-style granular Eurorack.

9.4 Granular albums / artists

  • Curtis Roads — Point Line Cloud (2004), Sonal Atoms — academic granular composition.
  • Aphex Twin — Drukqs (2001) uses granular extensively.
  • Autechre — Confield (2001), Quaristice (2008) — algorithmic granular textures.
  • Tim Hecker — Ravedeath, 1972 (2011) — granular processing of pipe organ.
  • Ben Frost — A U R O R A (2014) — granular textures behind orchestral writing.
  • Daniel Lopatin (Oneohtrix Point Never) — granular reprocessing of pop fragments.

10. Spectral synthesis and resynthesis

Spectral methods work in the frequency domain (STFT, sinusoidal model). Resynthesis takes an audio file, analyzes its time-varying spectrum, and lets you manipulate that spectrum (transpose individual partials, freeze a moment, replace noise floors with synthetic ones) before re-synthesizing.

10.1 Tools

  • SPEAR (Sinusoidal Partial Editing Analysis and Resynthesis, Michael Klingbeil, free) — graphical partial-by-partial editing of a recorded sound; the academic reference.
  • Spear (don’t confuse) / Loris / SDIF — Lippold Haken’s CCRMA tools for partial tracking.
  • AudioSculpt (IRCAM) — graphical spectral editing.
  • iZotope Iris 2 (2014, discontinued 2020 but still licensed) — paint-the-spectrum sampler: load a sound, select frequency regions with a graphical mask, play polyphonically.
  • MAGIX SpectraLayers Pro 11 (2024) — spectral editing for stem extraction and repair.
  • Image-Line Harmor — additive/spectral hybrid (as in §3.1).
  • Acoustica MetaSynth (Mac) — picture-to-spectrum; Aphex Twin’s Windowlicker face image is encoded in the spectrogram of the track itself via MetaSynth.
  • Soundgrain (Olivier Bélanger) — free granular/spectral environment built on Csound.
  • Csound and PVOC opcodes — Barry Vercoe’s Csound has had phase-vocoder primitives since the 1990s.
  • Phase Plant Spectral Filter, Snapin — Kilohearts spectral effects.

10.2 Spectral freeze and morph

Convergent technique: capture the spectrum at a moment (one STFT frame) and hold it indefinitely, producing an infinite drone with the spectral fingerprint of the source. Used heavily in ambient (Tim Hecker, Stars of the Lid, William Basinski). Tools: GRM Tools Freezing, Eventide Blackhole + Tricerachorus chained, Output Portal, Paulstretch (Nasca Octavian Paul, free, 1000× time stretch via spectral phase randomization).

11. Sample-based synthesis

A sample is a recording loaded into RAM and played back at a controllable rate (pitch-shifted by rate change). Modern samplers add per-zone filtering, envelopes, layering, round-robins, velocity layers, key switches, articulation modeling.

11.1 The line

  • Fairlight CMI (Series I 1979, II 1982, III 1985) — Peter Vogel and Kim Ryrie’s 80,000 Australian sampler; defined the orchestra-hit, choir-pad, found-sound sample idiom (Kate Bush, Trevor Horn, Peter Gabriel “Sledgehammer”).
  • E-mu Emulator I (1981), II (1984), III (1987) — affordable Fairlight competitor; 8-bit; defined hip-hop sampling.
  • Akai S612 (1985), S900 (1986), S1000 (1988), S3000 (1992), S5000/S6000 (1998), MPC60 (1988), MPC2000XL, MPC3000, MPC4000 — Akai’s S-series and MPC line dominated 1990s–2000s sample-based production. The MPC60 (Roger Linn–designed) and MPC3000 are J Dilla, DJ Premier, Pete Rock, Madlib’s defining instruments.
  • Ensoniq Mirage (1984), EPS, ASR-10 — Ensoniq’s affordable samplers; Mirage was first sub-$2k sampler.
  • Native Instruments Kontakt (2002, current Kontakt 7 in 2024) — the modern sample-library platform; runs every major orchestral library, vintage keys, and signature instrument.
  • UVI Falcon (2017) — multi-engine sampler/synth platform.
  • Ableton Sampler / Simpler (Live) — built-in.
  • Logic Sampler / Quick Sampler (2020, Logic Pro 10.5+) — replaced EXS24.
  • TX16Wx, Plogue sforzando, DecentSampler — free samplers.

11.2 Modern sample libraries

See orchestration-deep for the orchestral sampling landscape (Spitfire, EastWest, VSL, Orchestral Tools, Cinematic Studio Series).

12. AM, RM, sync — the “almost FM” operators

12.1 Amplitude modulation (AM)

Multiply a carrier by a modulator that swings between 0 and 1. Sidebands at carrier ± modulator. Half the sideband energy of true ring mod. Subtle tremolo at low modulator frequency; harmonic-adding effect at audio-rate modulation.

12.2 Ring modulation (RM)

Multiply two bipolar signals (DC-balanced). Sidebands at sum and difference frequencies; the carrier itself is suppressed. Bode Frequency Shifter (Don Buchla / Harald Bode, 1964) is the classic hardware; defines Doctor Who Daleks (“EX-TER-MIN-ATE”), Black Sabbath “Iron Man” intro guitar (Tony Iommi through ring mod), most cinematic alien voices.

12.3 Hard sync

A “slave” oscillator is forcibly reset to phase zero every time the “master” completes a cycle. The slave’s perceived pitch tracks the master; its timbre is determined by where it is in its own cycle when it resets. Sweeping the slave’s frequency under a fixed master produces the classic Cars “Let’s Go” / Van Halen “Eruption” / Prince “Let’s Go Crazy” sync lead.

12.4 Frequency modulation via cross-mod patches

Pre-FM-chip synths like the Minimoog and Prophet-5 implement audio-rate frequency modulation by routing one oscillator’s output into another oscillator’s pitch input. The math is the same as Chowning FM; the audio quality differs because analog VCO FM tracks pitch poorly (modulator depth depends on absolute frequency, not ratio).

12.5 Wavefolding

Wavefolding (Buchla, Serge) folds a waveform back on itself when it exceeds a threshold, generating odd harmonics. Buchla 259 Complex Waveform Generator and Serge Wave Multiplier are canonical; modern Eurorack equivalents include Make Noise Wogglebug + DPO, Intellijel Plonk, ALM Akemie’s Castle (which is also FM). Wavefolders define West Coast / Buchla idioms (Suzanne Ciani, Morton Subotnick).

13. Eurorack — modular reborn

Doepfer A-100 (1996) introduced the 3.5 mm patch cable / ±12 V / 1 V per octave / Eurocard form factor that became the global modular standard. By 2026 there are ~5,000 unique Eurorack modules from 400+ manufacturers, a $200M+/year market, and a culture distinct from preset-synth users (patch programming vs patch selection, generative vs deterministic, sound exploration vs song production).

13.1 Module families

Oscillators / sound sources:

  • Make Noise DPO (Dual Prismatic Oscillator) — west-coast complex oscillator with wavefolder.
  • Mutable Instruments Plaits (2018, the Braids successor) — 16 synthesis models in one module (analog VA, FM, wavetable, granular, modal, additive, noise, percussion). Now open-source and cloned (After Later Audio, Plum Audio, Hikari, Befaco).
  • Intellijel Dixie II+, Rubicon 2, Tete + Tetrapad — analog and analog-style sources.
  • Verbos Complex Oscillator, Harmonic Oscillator — west-coast Buchla-influenced.
  • Sputnik Modular 5-Step Voltage Source — Buchla-style.
  • Noise Engineering Loquelic Iteritas, Manis Iteritas, Basimilus Iteritas Alter — digital aggressive oscillators.
  • Joranalogue Generate 3 — analog complex osc.

Filters:

  • Make Noise QPAS, MMG — west-coast filters.
  • Mutable Instruments Ripples 2020 — Moog-style 4-pole multimode.
  • Doepfer A-124 Wasp Filter, A-106-6 XP — classic Doepfer cheap filters.
  • Erica Synths Polivoks — aggressive Russian-style filter.
  • Intellijel Polaris, Morgasmatron — multimode workhorses.

Envelopes, LFOs, modulation:

  • Make Noise Maths (5-channel function generator) — the most-patched module in Eurorack; envelope, LFO, slew, mixer, attenuverter, logic, all in one.
  • Mutable Instruments Stages (open-source) — 6 segment generators; envelopes, LFOs, sequencer-steps, all configurable.
  • Doepfer A-143 family — quad ADSR, quad LFO.
  • Tiptop Audio Z8000 — matrix sequencer / modulator.

Sequencers:

  • Make Noise René (Cartesian sequencer, 2D), Tempi, 0-Coast, Strega.
  • Intellijel Metropolix — 8-track CV sequencer.
  • WMD/SSF Metron — gate sequencer.
  • Mutable Instruments Marbles (open-source) — generative random sequencer.
  • Vermona randomRHYTHM mkIII — analog 2-channel rhythm sequencer.

Effects:

  • Mutable Instruments Clouds, Beads — granular (as in §9.3).
  • Mutable Instruments Rings, Elements — modal (as in §8.3).
  • Make Noise Mimeophon — color stereo delay.
  • Strymon Magneto — quad tape delay Eurorack.
  • Erica Synths Plasma Drive — high-voltage distortion.
  • 4ms Spectral Multiband Resonator (SMR) — 6-band filter bank.

13.2 Notable manufacturers

  • Make Noise (Asheville, NC) — Tony Rolando; west-coast Buchla-influenced; Black & Gold series.
  • Mutable Instruments (Émilie Gillet, France) — every module open-source; closed shop 2022; designs now built by clones.
  • Doepfer (Dieter Doepfer, Germany) — originator of A-100; affordable workhorses.
  • Intellijel (Vancouver) — high-end, deeply featured.
  • Noise Engineering (Los Angeles) — digital aggressive.
  • WMD (William Mathewson Devices) — closed shop 2023; legacy modules still in production at Steady State Fate.
  • Verbos Electronics — Mark Verbos; Buchla-style.
  • Tiptop Audio — DSP cartridge platform Z-DSP, Buchla 200 clones.
  • Erica Synths (Latvia) — drum modules, Polivoks circuits, Syntrx (EMS Synthi clone).
  • 4ms — Spectral Multiband Resonator, looper, DLD delay.
  • ALM Busy Circuits (UK) — Pamela’s NEW Workout (clock + LFO), Akemie’s Castle (FM dual oscillator).
  • Frap Tools — Italian high-end; CGM mixer.
  • Xaoc Devices (Poland) — Drezno, Lipsk, Erfurt — digital “Leibniz” subsystem.

13.3 Patch programming culture

A Eurorack patch is a directed graph of signal and modulation; the patcher’s craft is choosing which CV controls which destination, and accepting (or designing for) the generative behavior that emerges. Generative artists like Hainbach, Lightbath (Mike Schenk), Caterina Barbieri, Suzanne Ciani, Kaitlyn Aurelia Smith built bodies of work on modular generative systems. The aesthetic prizes accidents, drift, and emergent complexity over surgical control.

14. Neural audio and the 2024–2026 frontier

The 2020s machine-learning explosion has produced new synthesis paradigms that don’t fit the classical taxonomy:

14.1 Differentiable DSP (DDSP)

Google Magenta’s DDSP (Engel et al., 2020) wraps classical DSP modules (additive synth, subtractive filter, noise) in differentiable form so gradient descent can fit their parameters to target audio. Train a small network to map pitch + loudness extracted from a violin recording into DDSP additive partials, and you have a real-time controllable violin model that runs on a Raspberry Pi. Used by Mawf (Magenta’s iOS app), Neutone (open-source plugin host for neural models), and several research instruments.

14.2 RAVE (Realtime Audio Variational autoEncoder)

IRCAM’s RAVE (Caillon + Esling, 2021) is a variational autoencoder trained on audio that compresses ~44 kHz raw audio into a 16-dimensional latent space at 50 Hz; decoding the latent back gives audio. Train RAVE on a violin corpus and the latent space is “violin-shaped” — interpolating between two latent points gives a continuous timbral morph between two violin sounds. Used in live coding (TidalCycles + RAVE bridge), Eurorack (Nervous Squirrel’s RAVE module), and the IRCAM nn external for Max/MSP.

14.3 Jukebox, MusicLM, MusicGen, Stable Audio

Generative end-to-end models that produce raw audio from text or audio prompts:

  • OpenAI Jukebox (2020) — first large-scale text-to-music; coarse audio quality but stylistically remarkable.
  • Google MusicLM (2023) — hierarchical text-to-music; high quality 24 kHz output.
  • Meta MusicGen (2023) — transformer-based music generator; open weights; runs on consumer GPUs.
  • Stability AI Stable Audio 2 (2024) — text-to-audio for music + sound effects; commercial.
  • Suno AI v3 / v4 (2024–2025) — full-track generation with vocals from text prompt; subscription product.
  • Udio (2024) — competitor to Suno with similar capabilities.

These don’t replace traditional synthesis but are increasingly used for sketching, sound design, and album-track ideation. Legal status of training data + copyright of outputs is unsettled as of 2026 (Suno + Udio sued by RIAA in mid-2024).

14.4 Neural audio codecs as synthesis primitives

Meta’s EnCodec (2022) and Google’s SoundStream (2021) are neural audio codecs that encode 24 kHz audio into discrete tokens at ~75 Hz; these tokens are also the operating layer for MusicGen + MusicLM. The codec’s decoder is effectively a 75 Hz-controllable synth — sequence tokens with a language model and you have a generative music engine.

14.5 Neural vocoders and singing synthesis

  • DiffSinger (2022) — diffusion-based singing voice synthesis; high quality.
  • NEUTRINO — Japanese neural singing voice synth; open source.
  • VOCALOID 6 (Yamaha, 2022) — sample-based singing synthesis successor; widely used in J-pop.
  • Synthesizer V (Dreamtonics, 2020+) — neural singing synth; replaced Vocaloid for many producers.
  • CeVIO AI — Japanese singing + voice synthesis.

15. Unified perspective

FM, AM, PM, ring mod, sync, wavetable, granular, additive, and spectral synthesis are not unrelated families — they are different parametrizations of the same operation: control how a sound’s spectrum evolves over time.

  • Subtractive controls the spectrum’s shape via a filter on a fixed-spectrum source.
  • Additive controls each partial’s amplitude directly.
  • FM/PM/AM/RM generates sidebands by modulating one signal with another.
  • Wavetable controls the spectrum by selecting from a precomputed library.
  • Granular controls the spectrum by re-sequencing time-frequency atoms.
  • Spectral resynthesis controls the spectrum by editing its STFT directly.
  • Physical modeling controls the spectrum by simulating its acoustic cause.

The 2020s convergence is in plugins like Phase Plant, Pigments, Bitwig Grid, and the Iridium/Quantum hardware: every method available as a generator or processor in the same patch, with cross-modulation between them. The deep-synthesis future is multi-engine modular with neural-network-driven control.

Adjacent

Graph View