Sound Matching with a Differentiable Karplus-Strong Algorithm

Chapter one

The Karplus-Strong Algorithm

Original (1983) Extended

Sound off

analytic $|H(e^{j\omega})|$ measured output spectrum

Fundamental frequency $f_0 = f_s / (N + \tfrac12)$ Pluck frequency

1.1  Origins

In 1983 Kevin Karplus and Alex Strong published an algorithm that turns a short burst of noise into a convincing plucked string, using almost no arithmetic. Fill a buffer with random numbers, play it out in a loop, and on every pass replace each sample with the average of itself and the one before it. No multiplies at all.

It is worth being precise about what they thought they had built. As the authors describe it, this was wavetable synthesis with a self-modifying wavetable — not physical modelling. The physical reading came afterwards, when Alex Strong showed it to David Jaffe, who brought it to Julius Smith; Smith recast the loop as a delay line with an explicit additive input and recognised the moving average as a point-to-point transfer function on an ideal string. Only then did the delay line become a string and the averager become propagation loss.

In that recasting the whole algorithm is one equation:

$$H(z) = \frac{1}{1 - \tfrac{1}{2}\, z^{-N}\left(1 + z^{-1}\right)}$$

The curve above is that expression, evaluated on the unit circle. Its teeth are the string's harmonics, and the loop filter $H_L(z) = \tfrac12(1 + z^{-1})$ sets how fast each one dies: its magnitude is $|\cos(\omega/2)|$, so energy near Nyquist is removed almost entirely on every trip round the loop while low partials survive. That single fact is the whole timbre of the algorithm — bright at the attack, mellow as it rings.

Notice there is no gain term: the averaging itself is the only loss in the loop. Written exactly that way the algorithm is in fact only marginally stable, because the averager has unity gain at DC — the pole sits right on the unit circle and any DC offset in the noise burst would ring forever. Implementations keep a loop gain a hair below one, and use a zero-mean burst, as this one does.

Now the catch. Pitch comes entirely from the length of the delay line, and that length is a whole number of samples — there is no interpolation filter here to absorb a remainder. Drag the frequency control above and watch the substitution beneath it: the value you ask for is divided into the sample rate, then rounded, and the pitch you actually get is whatever that rounded delay produces.

(Strictly, the two-point average is linear phase with a delay of exactly half a sample at every frequency, so the loop rings at $f_s/(N + \tfrac12)$ and Karplus and Strong's tuning rule is $N = f_s/f_0 - \tfrac12$. The readout accounts for it.)

At low notes the delay line is long and the grid is fine, so the rounding error is negligible — under a couple of cents, well below what anyone can hear. Push up towards the top of the range and the delay line gets short, the grid gets coarse, and the tuning goes badly wrong: past 1 kHz the error exceeds twenty cents, a fifth of a semitone, and plainly audible.

Fixing that is where the story goes next.

1.2  Extensions

The figure has changed. Four new knobs appeared, and the transfer function behind the curve is now the extended algorithm we use throughout the paper:

$$H_{tKSA}(z) = \frac{H_D(z)\,H_P(z)}{1 - H_L(z)\,H_I(z)\,z^{-L_{int}}}$$

Each new factor separates something the original moving average had entangled.

Tuning. The interpolation filter $H_I(z)$ is a fifth-order Lagrange interpolator, whose fractional delay is far more uniform across frequency than the allpass filters used originally. It absorbs both the non-integer remainder of $L = f_s/f_0$ and the loop filter's own phase delay:

$$D = L - \lfloor L \rfloor - P_L(f_0), \qquad P_L(f_0) = -\frac{\angle H_L(e^{j\omega_0})}{\omega_0}$$

The quantisation error you just heard is gone: $f_0$ no longer has to land on an integer number of samples.

Decay and damping, separately. The moving average gives you one fixed brightness curve and one fixed decay rate. Välimäki and colleagues replaced it with a one-pole filter,

$$H_L(z) = g\,\frac{1 - a_1}{1 - a_1 z^{-1}}$$

whose DC gain is exactly $g$. So $g$ sets how long the note rings overall, and $a_1$ sets how much faster the high partials die than the low ones — two knobs for two physically distinct things. Sweep decay and the whole comb rises or falls together; sweep damping and watch the top of the spectrum tilt while the fundamental stays put.

Where you pluck. Plucking a real string at a fraction $p$ of its length suppresses every harmonic with a node at that point. Jaffe and Smith captured this with a feedforward comb, $H_P(z) = 1 - z^{-p}$, applied once to the excitation. Sweep pluck position and you can see the notches march through the harmonic series — set it to $0.25$ and the fourth harmonic and its multiples drop out.

How hard you pluck. A harder pluck is not just louder, it is brighter. The dynamics filter $H_D(z) = \frac{1-R}{1-Rz^{-1}}$ models intensity as a change in spectral tilt rather than a change in gain, which is what the ear actually uses to judge playing strength.

Together these turn a resonator into an instrument — and, more to the point for what follows, into a synthesiser with six meaningful, physically interpretable parameters per note. The rest of this page is about whether a neural network can be made to recover them.

Want to play it rather than read about it? The playable synthesiser has both algorithms on a piano roll.

Chapter two

Instrument Modelling & Gradient Descent

2.1  A guitar note is an event, not a frame

Everything so far assumed we already knew the parameters. Sound matching is the inverse problem: given a recording, recover the parameters that reproduce it. For a physical model like the KSA this is unusually appealing, because the parameters that come back out are interpretable — they tell you something about the string.

Listen to a real recording first. This is an acoustic guitar note from NSynth, plucked once and palm-muted at around three seconds.

Here is a real one: an acoustic guitar note from NSynth, plucked once, left to ring, and released at around three seconds. Beneath it is a two-event extended Karplus-Strong at the same pitch, waiting for you to make it match. Each event carries the four timbral controls from Chapter 1, held from its onset until the next — which is exactly how the synthesiser treats a set of events. Drag a marker to move an event in time, click one to edit it, and click anywhere else on either picture to listen.

Two onsets and eight timbral values, and the resemblance you can reach by hand is already close. That is the appeal of a physical model: the parameters mean something, and there are few enough of them to reason about. The rest of this page is about getting a network to find them without you.

Two things happen in that clip, and both are events: an excitation and a termination. Nothing about the string changes in between. This matters for how a model should be shaped. Most DDSP systems predict parameters on a fixed frame grid — a new $f_0$, a new decay, a new damping every few milliseconds. For a plucked string that is both wasteful and actively harmful: the KSA is a recursive, energy-dissipating system, so an error in a frame-wise decay estimate accumulates through the feedback loop instead of staying local.

So we predict a set of events instead. Each event carries an onset time, an $f_0$, and the timbral parameters from Chapter 1, held constant for the note's duration. Provided no extended techniques are present — no vibrato, no pitch bends — this is not an approximation, it is simply the right representation.

Learning that event representation from recordings requires a training signal that can pass through resynthesis. Backpropagating an audio loss therefore requires the synthesiser itself to be differentiable. There are two ways to do it, and the difference between them turns out to matter a great deal.

2.2  Frequency sampling versus the time domain

A naive time-domain implementation of a recursive filter computes sample by sample, which builds a computational graph 64,000 nodes deep. Two approaches restore parallelism.

The frequency sampling method evaluates the filter response on a grid of $M$ frequencies,

$$z_M = \left[e^{j\pi\frac{0}{M-1}}, e^{j\pi\frac{1}{M-1}}, \ldots, e^{j\pi}\right]$$

turning the recursion into an elementwise division, with fractional delays becoming a smooth closed-form phase shift $z_M^{-D} = e^{-j2\pi k M^{-1} D}$. The extended algorithm becomes

$$H_{fKSA}(z_M) = \frac{H_D(z_M) H_P(z_M)}{1 - H_L(z_M) H_I(z_M) z_M^{-L}}$$

This is fast and clean, and it has one serious problem. A Hermitian symmetry constraint forces a real impulse response of length $N_{FFT}$. If the true impulse response is longer than that, the tail does not get truncated — it wraps around and adds to the beginning of the frame. The string's decay folds back onto its own attack.

Time-domain interpolation avoids this entirely by computing exact analytic gradients through the recursive filter via the adjoint method, with fractional delay handled by the Lagrange interpolator directly. No frames, so no wrap-around.

For a single event, however, both approaches work well. The frequency-domain synthesiser can treat the whole sound as one linear time-invariant system. For this visual comparison the pluck sits at the start of the clip; as in the paper's single-event CGA/FGA path, the 4 s signal is zero-padded to an 8 s FFT, so its decay has enough room not to wrap. Both panels optimise the same loss, $\mathcal{L} = 0.05\,\mathcal{L}_{MSS} + \mathcal{L}_{SOT}$. Choose one initial $g$ and $a_1$ below — dragging either square moves both markers to exactly the same parameter values — then run the two descents from that shared start.

Multiple events make the frequency-domain case more complicated. Their parameters change over time, so the signal is no longer one LTI system that can be transformed once. fKSA must divide it into overlapping frames, and $N_{FFT}$ becomes an additional design and computational choice: long enough to contain a resonant decay, but costly enough to matter during training. tKSA has no corresponding frame-size choice.

The multi-event demonstration below makes that consequence concrete. A target is rendered with $g = 0.99$, $a_1 = 0.2$; both implementations start from the same point and descend the real joint loss $\mathcal{L} = 0.05\,\mathcal{L}_{MSS} + \mathcal{L}_{SOT}$, sampled offline with the paper's own code at its 16 kHz working rate. There are two plucks, one at the start of the clip and one three seconds in. Their excitations are summed first and their parameters form one piecewise timeline, exactly as in the model. A single Adam tick then updates every marker together.

The four squares are conditional views of that one four-parameter objective. Each square varies one event's $g$ and $a_1$ while holding the other at its current marker, so dragging either event immediately changes the landscapes and gradients for the other. Rows are the two implementations, columns the two events; the spectrograms on the right are always the complete coupled signal. This is also why tKSA and fKSA need separate rows: frame-wise circular convolution deforms the joint objective itself.

Run it and watch all four squares. Moving one marker changes the direction seen by its partner because both notes contribute to the same magnitude spectra and therefore the same MSS/SOT loss. The interaction is especially troublesome for a short fKSA frame: a resonant response that has not decayed within $N_{FFT}$ is circular, so its tail wraps back to the beginning of every inverse-FFT frame. With centred overlapping windows that aliased energy can appear before an attack and contaminate the frames used by the neighbouring event.

Then lengthen the frame and watch the circular artefact retreat. More of each resonant decay fits before periodic repetition, the pre-attack leakage falls, and the conditional fKSA slices increasingly resemble the time-domain ones. Nothing is wrong with frequency sampling as such; the damage comes from representing a response longer than the FFT period as though it were periodic.

Look at how the loss ranges and slopes change as well. At short frames time-aliasing can flatten or redirect the conditional surface, and a move by one event can alter the gradient received by the other. That is the training problem the independent-note picture concealed: the model does not optimise isolated plucks, but all event parameters through one shared audio loss.

The headline result is that time-domain Lagrange interpolation matches frequency sampling on gradient accuracy while avoiding this artefact — which makes it a viable, artefact-free alternative for differentiable physical models with time-varying, highly resonant behaviour.

2.3  Onset time and $f_0$

Timbre parameters, then, are recoverable. Pitch and timing are a different story, and this is the part of the work that surprised us.

To measure it we ask a deliberately minimal question of each gradient — not how large it is, but whether it even points the right way:

$$\mathrm{sgn}\left(\frac{\partial \mathcal{L}(\hat{x}f[n], x[n])}{\partial f}\right) = \mathrm{sgn}(f - f_x)$$

A parameter scoring 50% is indistinguishable from a coin flip. Coarse gradient accuracy initialises across the whole parameter range; fine accuracy initialises within $\pm 10\%$ of the target, asking whether the gradient is still trustworthy once you are already close.

Single-Event Gradient Accuracy

Single-event gradient accuracy (%) comparing $\mathcal{L}_{MSS}$ and $\mathcal{L}_{SOT}$ across tKSA and fKSA.

The numbers are stark. Every timbral parameter is reliable under $\mathcal{L}_{MSS}$ — decay and damping score at or near 100%, dropping only to 94.8% for fKSA in the fine regime. Onset time sits at 51–56%: a coin flip. $f_0$ manages 58–70%, better than chance but nowhere near usable.

Multi-Event Onset Gradient Accuracy

Multi-event onset time gradient accuracy (%) for $K$ equally spaced plucks. fKSA uses $N_{FFT}=16384$, hop=256. Any: at least one gradient points toward a target; Joint: all $K$ simultaneously toward their Hungarian-matched targets.

And it gets worse with more notes. The Joint column asks whether all $K$ onsets point toward their matched targets simultaneously — which is what actually has to happen for joint training to work. At five events it has collapsed to at most 4.1%. There is, in practice, no usable onset gradient for a multi-event signal.

You can watch the failure happen. Below, the landscape is over onset time (horizontal) and $f_0$ (vertical). The tKSA and fKSA trajectories keep their blue and red identities, while the right-hand column mirrors Section 2.2 with separate target, tKSA and fKSA spectrograms.

Start the marker away from the target and run it. Watch the two axes behave differently — and note what the markers are following. They do not roll downhill on the heatmap beneath them. They follow the true gradient that backpropagation delivers to the model, precomputed by autograd through the synthesiser itself.

That distinction is the whole finding. The loss surface does have usable slope toward the right onset: finite-differencing the heatmap scores 69.4% on the sign test. But onset reaches the synthesiser through a Straight-Through Estimator — the forward pass snaps the burst to the nearest whole sample, while the backward pass routes gradients through the continuous phase rotation of Eq. 9 — and the gradient that comes back out scores 54.4%, against Table 1's 51.2%. A coin flip. The valley is there; the gradient cannot see it.

The $f_0$ axis is better behaved, and the loss-mix slider shows why. Under pure $\mathcal{L}_{MSS}$ the pitch gradient is genuinely directional, at 71.1% — Table 1 reports 68–70%. Slide toward $\mathcal{L}_{SOT}$ and it falls to 63.1%. Since training weights SOT at $1.0$ against MSS at $0.05$, it is the weaker of the two that dominates what the model receives — and $\mathcal{L}_{SOT}$ was adopted precisely to supply the directional frequency information a point-wise loss lacks. Surprisingly, given its design, it is close to blind to $f_0$.

And no reweighting rescues onset, for a structural reason. $\mathcal{L}_{MSS}$ is a vertical, point-wise distance: it compares each time-frequency bin with the same bin in the target, so a correctly shaped note in the wrong place is simply wrong twice — once where it is, once where it should be — with no indication of which direction closes the gap. $\mathcal{L}_{SOT}$ transports magnitude along the frequency axis within each STFT frame, but never between frames. Neither operator can express the sentence “this note should move earlier.”

There is a second, sneakier problem, and this figure explains it:

Pitch and decay are not independent in this algorithm. A signal at harmonic $k$ passes through the loop filter exactly $f_0$ times per second, so

$$\mathrm{RT60}_k = \frac{-3}{f_0 \log_{10}\lvert H_L(e^{j\omega_k})\rvert}$$

Move $f_0$ and you move the decay time with it. So a gradient step that improves pitch also, unavoidably, changes how long the note rings — and the loss sees both changes mixed together. This is why we supervise the joint analytical decay time RT60 rather than $g$ and $a_1$ independently: it is the quantity that is actually perceptually coherent.

Two failures, then, and they compound: no directional information for onset, and a pitch axis entangled with decay. Together they explain why joint audio-and-parameter training collapses, which is what Chapter 3 is about.

Chapter three

Neural Network Training

3.1  Three ways to supervise

The model first turns a recording into the compact event set introduced in Section 2.1, then resynthesises those events through the KSA decoder:

The dual frontend combines a mel spectrogram with learned filterbanks before a set-prediction transformer produces ten possible event slots. The question is how to train those shared features and parameter heads when the audio gradients from Chapter 2 are unevenly reliable.

Chapter 2 leaves us with a specific problem. Audio losses are excellent for timbre and useless for onsets. If we train end-to-end on audio alone, the onset and $f_0$ heads receive gradients that are barely better than noise, and — because they share an encoder with everything else — they drag the rest of the model down with them.

There are two ways out, and we test both, plus their combination.

The first is to stop using audio for those parameters. Because we can generate synthetic data with known ground truth, we can supervise the parameters directly. Following the set-prediction literature, Hungarian matching assigns each of the $N = 10$ predicted event slots to a ground-truth event, and the matched loss is

$$\mathcal{P}_{Loss} = w_e \mathcal{L}_e + w_t \mathcal{L}_t + w_{f_0} \mathcal{L}_{f_0} + w_g \mathcal{L}_g + w_{\mathrm{nt60}} \mathcal{L}_{\mathrm{nt60}} + w_p \mathcal{L}_p + w_d \mathcal{L}_d$$

with existence handled by a sigmoid focal loss to cope with class imbalance, and $f_0$ by negative log-likelihood over 128 log-spaced bins rather than a regression — pitch is better posed as a classification.

The second is to hand those two parameters to something that already works: CREPE for $f_0$, spectral flux with backtracking for onsets. The differentiable synthesiser then only has to learn timbre, which is precisely what audio losses are good at.

That gives us three regimes:

To isolate the mechanism we also run stop-gradient ablations on $\mathcal{P}$+Audio: detach the audio-loss gradient from the onset head, from the $f_0$ head, or from both, leaving them supervised only by $\mathcal{P}_{Loss}$. If Chapter 2 is right, detaching should help — which is a strange thing to expect from removing a training signal.

3.2  What happened

Two datasets. Synth is 4-second mono clips at 16 kHz containing 1–10 plucks at constant $f_0$ over D1–D6, parameters drawn from Beta distributions with a 30% palm-mute probability, reseeded every epoch — 8,360 training samples. Real is the acoustic guitar subset of NSynth, restricted to non-reverberant samples in the same D1–D6 range, split 8,360 / 930 / 290.

Synthetic Parameter Recovery

Synthetic parameter recovery. All models are trained and evaluated on synthetic data. $\mathcal{P}$-Only uses tKSA with $\mathcal{P}_{\text{Loss}}$ supervision; $\mathcal{P}$+Audio models use differentiable decoders trained end-to-end with both $\mathcal{P}_{\text{Loss}}$ and audio losses (MSS, SOT); Audio-Only models use differentiable decoders with audio losses and external detectors for onsets and $f_0$. Rows beneath each $\mathcal{P}$+Audio model are stop-gradient ablations that detach the audio-loss gradient from the named head ($f_0$, onset, or both), leaving it supervised only by $\mathcal{P}_{\text{Loss}}$. Per column, the best, second-best, and third-best values are shown in bold, underline, and italic respectively.

The prediction from Chapter 2 holds, and more sharply than we expected. $\mathcal{P}$+Audio does not merely underperform — it collapses. Precision, recall and F1 all fall to 0.01, and RMS to 0.33: the model is emitting near-silence. Detaching the onset head alone lifts F1 from 0.01 to 0.31; detaching both onset and $f_0$ takes it to 0.83 and brings $\Delta$Cents from 117 down to 12.7. Removing a training signal makes the model dramatically better, which only makes sense if that signal was actively misleading — exactly what a 51% directional accuracy means.

$\mathcal{P}$-Only wins nearly everything on synthetic data, including timbre. Beyond having a clean path to the optimum, it benefits from the RT60 reparameterisation, which supervises damping and decay jointly instead of pretending they are independent.

Real-World Instrument Modelling

Real-world instrument modeling results evaluated on the NSynth acoustic guitar test set. $\mathcal{P}$-Only uses tKSA trained on synthetic data with $\mathcal{P}_{\text{Loss}}$ only; $\mathcal{P}$+Audio models are trained with both $\mathcal{P}_{\text{Loss}}$ and audio losses; Audio-Only models are trained with audio losses and external detectors. HpN and HpN+ are Harmonics-plus-Noise baselines using the original DDSP encoder and our TCN encoder respectively. KAD is reported in CLAP and EnCodec embedding spaces. Rows beneath each $\mathcal{P}$+Audio model are stop-gradient ablations that detach the audio-loss gradient from the named head. Per column, the best, second-best, and third-best values are shown in bold, underline, and italic respectively.

On real audio the ordering changes. $\mathcal{P}_{Loss}$ generalises poorly — it frequently misses onsets, most likely because the CNN overfits to synthetic transients and predicts $f_0$ erratically on real ones. Audio-Only with external detectors is the strongest KSA configuration on real recordings, and notably it beats both Harmonics-plus-Noise baselines on CLAP distance, which we attribute to the KSA's natural modelling of sharp transients.

We should be straightforward about the rest: HpN and HpN+ achieve better reconstruction fidelity on most metrics. That gap is structural rather than incidental. A single feedback loop captures neither the resonant guitar body, nor the interaction between the string and the player's hands, nor string–fret collisions — all contributions an HpN model can absorb directly. What the KSA offers instead is a handful of physically interpretable parameters and a faithful transient. No formal listening test was conducted, so we would rather you judged the trade-off yourself, below.

3.3  Listen

Every model in both tables, on the same examples. Use the toggles to isolate an implementation, reveal the stop-gradient ablations, or bring in the Harmonics-plus-Noise baselines.

Chapter four

References

1. K. Karplus and A. Strong, “Digital Synthesis of Plucked-String and Drum Timbres,” Computer Music Journal, 1983.
2. D. A. Jaffe and J. O. Smith, “Extensions of the Karplus-Strong Plucked-String Algorithm,” Computer Music Journal, 1983.
3. V. Välimäki et al., “Physical Modeling of Plucked String Instruments with Application to Real-Time Sound Synthesis,” JAES, 1996.
4. P. Tablas de Paula, J. O. Smith III, V. Välimäki and J. D. Reiss, “Four Decades of Digital Waveguides,” J. Audio Eng. Soc., 2026, in press.
5. J. Engel et al., “DDSP: Differentiable Digital Signal Processing,” ICLR, 2020.
6. B. Hayes et al., “A Review of Differentiable Digital Signal Processing for Music and Speech Synthesis,” 2024.
7. S. Schwär and M. Müller, “Multi-scale Spectral Loss Revisited,” IEEE Signal Processing Letters, 2023.
8. B. Torres, G. Peeters and G. Richard, “Unsupervised Harmonic Parameter Estimation Using Differentiable DSP and Spectral Optimal Transport,” ICASSP, 2024.
9. C.-Y. Yu and G. Fazekas, “Accelerating Automatic Differentiation of Direct Form Digital Filters,” 2025.
10. P. Tablas de Paula, D. Marttila and J. D. Reiss, “Differentiable Karplus-Strong,” DMRN+20, 2025.