[HN Gopher] Show HN: Web Audio Spring-Mass Synthesis
___________________________________________________________________
Show HN: Web Audio Spring-Mass Synthesis
Hi, I'm the author of this little Web Audio toy which does physical
modeling synthesis using a simple spring-mass system. My current
area of research is in sparse, event-based encodings of musical
audio (https://blog.cochlea.xyz/sparse-interpretable-audio-codec-
pa...). I'm very interested in decomposing audio signals into a
description of the "system" (e.g., room, instrument, vocal tract,
etc.) and a sparse "control signal" which describes how and when
energy is injected into that system. This toy was a great way to
start learning about physical modeling synthesis, which seems to be
the next stop in my research journey. I was also pleasantly
surprised at what's possible these days writing custom Audio
Worklets!
Author : cochlear
Score : 64 points
Date : 2025-03-14 21:27 UTC (1 days ago)
(HTM) web link (blog.cochlea.xyz)
(TXT) w3m dump (blog.cochlea.xyz)
| danbmil99 wrote:
| Very cool! I've often wondered whether one could procedurally
| generate sounds of objects interacting in a physics engine? This
| approach seems like a good place to start.
| dberst wrote:
| You (or anyone interested in sound/physics/code) may find this
| video / series interesting
|
| https://youtu.be/RKT-sKtR970?si=qvTXK6YSMsPaE-GE
| cochlear wrote:
| Same here! Not a physics engine per se, but I've been eyeing
| Taichi Lang (https://github.com/taichi-dev/taichi) as a
| potential next stop for running this on a much larger scale.
|
| My assumption has been that any physics engine that does soft-
| body physics would work in this regard, just at a much higher
| sampling rate than one would normally use in a gaming scenario.
| This simulation is actually only running at 22050hz, rather
| than today's standard 44100hz sampling rate.
| ta988 wrote:
| This is fun, but if you reduce the tension and click a lot on it,
| it gets stuck with almost all the points at the top of the screen
| except one. Also weirdly in Firefox android it clicks a lot, not
| on chrome.
| cochlear wrote:
| Thanks for the bug report, I definitely do see now that things
| fall apart at certain extreme values. I'll look into it and see
| if I can fix. My assumption is that it's some sort of numeric
| overflow or underflow resulting in NaN values, and that I'll
| probably just need to set the slider boundaries more carefully.
| Thanks again!
| SoleilAbsolu wrote:
| Interesting, I'm into physical modeling and learning DSP.
|
| I did notice glitching in latest Firefox on a Mac, like I'd get
| running a DAW with too small a buffer...While the tab was open I
| got similar crackles and slightly delayed audio stream playing
| YouTube vids in other tabs.
| cochlear wrote:
| Thanks for the bug report, I'll look into this and see if I can
| make it better! There's definitely more optimization juice to
| be squeezed, I think, and it'd probably be smart to allow the
| number of nodes in the simulation to be adjusted.
|
| Working with AudioWorklets (https://developer.mozilla.org/en-
| US/docs/Web/API/AudioWorkle...) has been really cool, and I've
| been surprised at what's possible, but I _haven't_ yet figured
| out how to get good feedback about when the custom processor
| node is "falling behind" in terms of not delivering the next
| buffer quickly enough.
| xavriley wrote:
| This is cool - there's some similar work here
| https://arxiv.org/pdf/2402.01571 which uses spiking neural
| networks (essentially Dirac pulses). I think the next step for
| this would be to learn a tonal embedding of the source alongside
| the event embedding so that you don't have to rely on physically
| modelled priors. There's some interesting work on guitar amp tone
| modelling that's doing this already
| https://zenodo.org/records/14877373
| cochlear wrote:
| How funny, I actually corresponded with one of the authors of
| the "Spiking Music..." paper when it first showed up on arxiv.
| I'll definitely give the amp-modeling paper a read, looks to be
| right up my alley!
|
| Now that I understand the basics of how this works, I'd like to
| use a (much) more efficient version of the simulation as an
| infinite-dataset generator and try to learn a neural operator,
| or NERF like model that, given a spring mesh configuration, a
| sparse control signal, and a time, can produce an approximation
| of the simulation in a parallel and sample-rate-independent
| manner. This also (maybe) opens the door to spatial audio, such
| that you could approximate sound-pressure levels at a
| particular point in time _and_ space. At this point, I'm just
| dreaming out-loud a bit.
| blovescoffee wrote:
| This is possible but very very hard! Actually getting the
| model to converge on something that sounds reasonable will
| make you pull your hair out. It's definitely a fun and
| worthwhile project though. I attempted something similar a
| few years ago. Good luck!
| akomtu wrote:
| Here's a simple, but high quality model of piano and violin:
| https://mattdailis.github.io/simulating-strings/resources/ph...
| chaosprint wrote:
| Great demo.
|
| I used to do some web audio and tonejs works, but later switched
| to rust and glicol for sound synthesis.
|
| For example, this handwritten dattorro reverb:
|
| https://glicol.org/demo#handmadedattorroreverb
|
| This karplus-stress-tester may also be interesting to you.
|
| https://jackschaedler.github.io/karplus-stress-tester/
|
| In short, I think to study more powerful physics synthesis, you
| need to consider the technology stack of
|
| - rust -> wasm - audioworklet - sharedarraybuffer
|
| Visual can rely on wgpu. Of course, webgl is enough in this case
| imho.
|
| If it is purely desktop, you can consider using the physics
| library in bevy.
| cochlear wrote:
| You're the author of Glicol, right? I've definitely had my eye
| on trying it out for a while. The karplus-stress-tester is
| great; I'm currently using message ports, because they seemed
| most accessible at first, but I'm happy to know there are
| other, and better options. I have done quite a bit of hand-
| optimizing of the code here, and while I think there's probably
| juice left to squeeze, it has become apparent to me that wasm
| is probably my next stop.
|
| I've written one other AudioWorklet at this point, which just
| runs "inference" on a single-layer RNN given a pre-trained set
| of weights: https://blog.cochlea.xyz/rnn.html. It has similarly
| mediocre performance.
|
| Thanks for all the great tips, and for your work on Glicol!
| matt123456789 wrote:
| First of all - very cool, played with this for a few minutes
| before discovering the following bug:
|
| In Safari (iOS 18.3.1) if you set the Mass slider to 0 and
| increase Tension, not only does the app crash, but a repeated
| clicking noise starts, and persists, even after the tab is closed
| - even after Safari itself is closed! Seems to be a Safari bug. I
| have reproduced it 3x.
|
| If you reproduce this and want the noise to go away you have to
| start another app that tries to play sound.
| trial3 wrote:
| amazing! there are a couple settings here where it sounded like
| how i wish the String engine on my old OP-1 sounded. the random
| setting is really fun
| lfsh wrote:
| If you like this you might also like Anukari's 3D Physics
| Synthesizer.
|
| It is also a spring-mass synth but with Midi and audio and much
| more options.
|
| https://anukari.com/
___________________________________________________________________
(page generated 2025-03-15 23:02 UTC)