[HN Gopher] Ask HN: Create audio software akin to physics engines?
___________________________________________________________________
Ask HN: Create audio software akin to physics engines?
Hi everyone, I used to think that alot of physics was bs until I
found a tutorial on Box 2D physics. The tutorials used real
physics concepts, like momentum, coefficient of friction and others
to simultate physical (mechanical properties of items on a
computer). I found the same in physically based rendering/ray
tracing where real Physics concepts from how light behaves were
used to create virtual simulations that resembled those in reality.
Here is my question, are there any resources on how someone can
simulate sound using software. Like howling of wind, the sound from
musical instruments. Are there any good study resources, sample
tools or theories (what is the "PBR or ray tracing of audio" ?)
What is this field called? I am assuming it has something to do
with the physics field of acoustics? The I would like to build a
software tool for creating such audio simulation i.e a "box2d
physics" library but for audio is the elevator pitch. Thanks
Author : max_
Score : 67 points
Date : 2024-06-19 07:58 UTC (1 days ago)
| compressedgas wrote:
| https://en.wikipedia.org/wiki/Digital_waveguide_synthesis
| Reubend wrote:
| This is called "physical modelling". Creating sound from a
| physical simulation is popular for things like plucked strings,
| but it's very difficult for more complicated instruments like
| harmonicas
| duped wrote:
| Read this book https://ccrma.stanford.edu/~jos/pasp/. It's a
| hefty tome but kind of the bible of the various approaches to
| audio signal processing and synthesis.
|
| There's no one method to rule them all, they all have tradeoffs.
| So there is no "ray tracing" of audio (well actually, it's ray
| tracing, but for a lot of reasons it's prohibitive to actually do
| that in a way that sounds good).
|
| > What is this field called?
|
| Audio synthesis. Popular journals with papers covering the topic
| are DAFX (digital audio effects) and jAES (journal of the audio
| engineering society). To a lesser extent, the IEEE transactions
| on audio signal processing.
|
| > I am assuming it has something to do with the physics field of
| acoustics?
|
| You can find plenty of sources in acoustics journals/text books -
| but this is like comparing the needs of mechanical and electrical
| engineers to game engine designers. At a surface level there is
| crossover, and cross pollination of techniques/tools, but the
| needs are fundamentally different.
| thrtythreeforty wrote:
| PASP is awesome but it just scratches the surface. You _will_
| need to dive into papers to make any headway - there is a ton
| of research but there is no "toolkit" for doing it because
| it's comparatively a niche thing to do. You will probably also
| want to reference the author's other books on digital filters
| and computer audio. There is also no consensus on the "best"
| techniques, so you read papers until you find a technique you
| can adapt to the modeling framework you've built, implement,
| repeat.
|
| Physical modeling can be done on a modern computer in realtime
| (for example, my VST instrument models a 6-string guitar, with
| a litany of modeling features, at about 3-4x realtime at 96kHz,
| and I have plans to expand that to a whole piano at roughly the
| same computational cost). But you will have to be quite good at
| DSP engineering to be able to meet the realtime requirements -
| this is similarly high-performance as graphics programming or
| any other kind of modeling code. You can also build toy models
| in Python or MATLAB and run them offline, but this is not
| nearly as much fun.
| an_aparallel wrote:
| There is STK Toolkit by Julius O. Smith - which has a
| stunning violin model. But because I havent learnt C++ i dont
| know how stunted it is - as the IP is owned by Yamaha - but
| the code is still available, i dont know how that works.
| solardev wrote:
| There's quite a bit of this in the video & PC game world. Early
| on, technologies like Aureal A3D
| (https://en.wikipedia.org/wiki/Aureal_Semiconductor#A3D) or
| Creative EAX
| (https://en.wikipedia.org/wiki/Environmental_Audio_Extensions)
| tried to do simple simulations of reverb and spatial audio. The
| gave way to OpenAL (https://en.wikipedia.org/wiki/OpenAL), I
| guess more recently it's Wwise
| (https://www.audiokinetic.com/en/wwise/wwise-spatial-audio/).
|
| The Wwise page is especially interesting, with a few paragraphs
| on audio positioning and reflections simulated inside level
| geometry.
| jfkw wrote:
| Engine Simulator is an interesting example of this type of work:
| https://www.youtube.com/@AngeTheGreat
| bpanon wrote:
| Trumpet Simulator is also very impressive though source not
| presently available
|
| https://www.youtube.com/watch?v=rGNUHigqUBM
| eigenvalue wrote:
| Yeah, was also going to mention this YouTube channel. It's
| really good and he does everything from scratch.
| rapjr9 wrote:
| There is the synthesis of sounds and there are a variety of
| models for that, such as FM synthesis, additive harmonic
| synthesis, physical modeling, wavetable synthesis, granular
| synthesis and more. Physical modeling is somewhat like ray
| tracing, modeling the materials and sound paths in math to
| imitate a real instrument (or any object) making sound.
|
| There is modeling of acoustic environments, which is very much
| like the ray tracing of audio, dealing with reflections, damping,
| phase, refraction, dispersion and reverberation.
|
| There is the physics of audio perception, how the ear works, how
| the brain perceives sound, psychoacoustics. What makes your ear
| "think" you are indoors or outdoors?
|
| There are encoding standards, AAC, WAV, MP3, FLAC, which can also
| get into psychoacoustics.
|
| There are algorithms, like DSP signal processing algorithms for
| audio which do reverb, EQ, echo, psychoacoustics, sound
| projection/direction, flanging, surround sound, and a lot more
| for shaping sounds.
|
| There is the study of how to build optimal acoustic spaces, like
| concert halls and music studio performance rooms. More
| psychoacoustics, but some math and sensors also.
|
| There is the art of trying to use all the above to accomplish
| something, a song, a soundtrack, a concert, a jingle.
|
| So there are a lot of different aspects to audio, and I've
| probably left some out of the above list (voice, foley, sound
| effects, subsonic, supersonic,...). I'd suggest you first work on
| defining your problem better, what do you actually want to solve,
| and for what audience? There are many tools that do a lot of this
| already. If you start out with the most basic audio physics, how
| sound travels through a variety of materials, and what happens at
| transitions between materials you could probably spend many years
| working on just that (as a comprehensive system) before you even
| get to trying to generate sounds. I don't think there is a
| complete system that can be used to predict all aspects of sound
| propagation through materials (including air) because materials
| can be extremely complex, for example air has air currents,
| variations in humidity, air pressure is variable in time and
| space, and the sounds in air are also strongly shaped by
| reflections and the qualities of the surfaces it reflects from.
| Imagine the complexity of modeling all the trees, leaves, plants
| and rocks in a forest. You can "record" the effects of one
| specific space in a forest using impulse response recordings and
| apply it to sounds, but to model it from scratch and be able to
| recreate any forest location would probably be a lifetimes work,
| perhaps several lifetimes, unless you can find a clever way to do
| it (maybe generate artifical fractal landscapes based on
| parameters measured from real places, and then do acoustic ray
| tracing?)
|
| Acoustic "ray tracing" is not like light ray tracing because
| sound is much lower frequency and diffracts, it bends around
| objects instead of bouncing off, and also bounces off. The phase
| of the interacting "rays" or waveforms as they reach the ear also
| matters a lot more with audio than with light which makes the
| computations more complex. In a light ray tracer light travels in
| straight lines from sources to virtual camera. In a sound ray
| tracer sound can bounce and refract and disperse from anywhere to
| reach the virtual ear. What is on the other side of a wall can
| matter to what a room sounds like.
|
| Don't let my talking of complexity deter you though, look and
| maybe you'll find something nobody has thought of before. It
| might be a matter of plugging together some tools that already
| exist (e.g., a 3D definition of a room or outdoor space, sound
| source models, choosing a virtual ear location, and let the
| computer crunch for a few days.)
| FractalHQ wrote:
| Download a free version of Chromophone or Logic Pro and play with
| physical modeling synths yourself for endless fun!
| Slow_Hand wrote:
| The field you're looking for in audio is known as physical
| modeling, it's a facet of the broader field of audio synthesis
| (along with subtractive synthesis, FM synthesis, granular
| synthesis, etc).
|
| Physical modeling is primarily used to simulate musical
| instruments or create non-existent instruments like a 40 ft wide
| frame drum or a saxophone made of rubber.
|
| I don't see it used that often to simulate non-musical noises,
| but I wouldn't be surprised if there were other people out there
| doing it.
| kretaceous wrote:
| I can't be of much help here but
|
| > I used to think that alot of physics was bs until I found a
| tutorial on Box 2D physics.
|
| > The tutorials used real physics concepts, like momentum,
| coefficient of friction and others to simultate physical
| (mechanical properties of items on a computer).
|
| Can you share these said tutorials if possible? Thanks!
| max_ wrote:
| Unfortunately I cannot find the links. It was about 10 years
| ago.
|
| But it was about a Box2D physics JavaScript implementation
| hn30000 wrote:
| Some good posts here. Another term to look into is auralization
| (often discussed in reference to room acoustics). I have done
| some research into architectural acoustics and real-time audio
| rendering tools and am happy to discuss further (though it is a
| big field and I definitely don't know it all), anyone reading
| this should feel free to shoot me a message.
| an_aparallel wrote:
| Hi - your email is not in your profile :) I'd love to chat!
| an_aparallel wrote:
| The type of synthesis you're refering to is "physical modelling"
| - but from a physics and mathematics perspective - my
| understanding is that this is refered to as DSP (Digital Signal
| Processing - which all the resources pointed to here are)
| computational physics, numeric analysis, fluid dynamics - and
| most likely a few other names i'm yet to stumble upon.
|
| The closest thing to what you mention is: Modalys by IRCAM
| (Institute for Research and Coordination in Acoustics/Music)
|
| https://forum.ircam.fr/projects/detail/modalys/
|
| From the readme:
|
| Modalys is IRCAM's flagship physical model-based sound synthesis
| environment, used to create virtual instruments from elementary
| physical objects such as strings, plates, tubes, membranes,
| plectra, bows, or hammers.
|
| It is also possible to create objects with more complex shapes
| out of 3D meshes, or using measurements, and Modalys does all the
| hard computational work for you, bringing them to life and making
| them sound.
|
| You can run Modalsys inside Ableton's flagship Suite.
|
| I'm very passionate about this area of music/DSP...one day i hope
| to get involved in it :-)
| rzzzt wrote:
| There is a book on DSP (theory) by Steven W. Smith which you
| can read online: http://www.dspguide.com/pdfbook.htm
| an_aparallel wrote:
| still working up to the math i need to get this one :)
| glial wrote:
| I found it surprisingly approachable -- yes it discusses
| convolutions, but it also teaches you what that means in a
| very step-by-step manner, with nice illustrations.
| an_aparallel wrote:
| im still pre-calculus! :P And i feel i need absolute
| fluency before i start haha. Thanks for the encouragement
| :-)
| janosdebugs wrote:
| Unreal Engine and Steam Audio may be worth looking into before
| you invest significant amounts of time into this.
| d--b wrote:
| > I used to think that alot of physics was bs until I found a
| tutorial on Box 2D physics.
|
| Haha. Thanks Newton for Angry Birds.
| coolhand2120 wrote:
| A few companies have commercialized the concept of physical audio
| modeling. Audio modeling allows for complex control surfaces
| (e.g.: https://roli.com/products/seaboard/rise2) that use MPE
| (Multi polyphonic expression).
|
| https://audiomodeling.com/. Creates the SWAM technology and
| provides a whole bunch of physically modeled instruments, like
| horns, winds, and strings.
|
| For more fundamental sounds check out Ableton Suite
|
| https://www.ableton.com/en/live/compare-editions/#software-i...
|
| Ableton provides a number of physically modeled instruments.
| Tension = string, collision = mallet/percussion, and a few more.
|
| Both are quite expensive and require a somewhat powerful machine.
|
| While these sound cool, a sampler, that is a synth that plays
| recorded versions of the real instrument, sounds much more "real"
| than a modeled version. That's thanks to the Fourier transform.
| https://www.andreinc.net/2024/04/24/from-the-circle-to-epicy...
|
| A good place to start on sound in general is at Bartosz
| Ciechanowski's blog on sound: https://ciechanow.ski/sound/
| steve1977 wrote:
| Tension and Collision were made by Applied Acoustics as far as
| I know, these are their ,,originals" (although in terms of
| value you probably get more with Ableton)
|
| https://www.applied-acoustics.com/chromaphone-3/
| https://www.applied-acoustics.com/string-studio-vs-3/
| binkethy wrote:
| Well, howling of wind can be achieved with simply low-pass
| filtering white noise and varying the cutoff frequency.
|
| What you are referring to is known as physical modeling, and
| there are many techniques employed, depending upon the sort of
| system one seeks to model.
| iainctduncan wrote:
| It's called Physical Modelling, and it's a big sub-branch of
| audio synthesis. A very good intro is Perry Cook's "Real Sound
| Synthesis" book.
|
| To understand the physics behind the software, the field you need
| to read is called Acoustics. A great intro (and the textbook we
| used in grad school) is "The Science of Sound" by Rossing et. al.
|
| (And +1 to Julius Smith book recommendations).
| iKlsR wrote:
| Can you share the tutorial you found?
| max_ wrote:
| Unfortunately it was a very long time ago about 10+ years ago.
|
| I am unable to recover the link.
|
| All I remember was that it was during the HTML5 wave & this
| tutorial was particularly about the JavaScript implementation
| of Box2D js
| nodenoise wrote:
| This is essentially sound design from first principles. There's a
| good book here: https://www.amazon.com/Designing-Sound-Press-
| Andy-Farnell/dp... Note that the software used (Pure Data) can be
| replaced by another high-level language (SuperCollider:
| https://supercollider.github.io/) pretty easily. I know of no
| "tool" to do what you want because there are few things that are
| universal to different kinds of natural and unnatural sound.
| (Note: study acoustics and psycho-acoustics to better understand
| why the former is true.)
| theoriginaldave wrote:
| I know companies like Siemens sell very enhanced acoustic
| modeling and simulation tools. The software is commercial, but if
| they use standards or open libraries you might be able to make
| use of them, at least as a starting point for research.
| https://plm.sw.siemens.com/en-US/simcenter/simulation-test/a...
| musha68k wrote:
| Expressive amongst some others do sell some impressive physical
| modelling tech:
|
| https://youtu.be/oTxgHJMpKsw?si=ibIX9pxxl3Sh--xy
|
| If you want to delve deeper on the open source side with csound,
| super collider and more:
|
| Csound - wg (waveguide) opcodes (e.g. wgpluck, wgbow)
|
| https://csound.com/docs/manual/SiggenWavguide.html
|
| SuperCollider - Stk (Synthesis ToolKit) plugins (e.g. StkBowed,
| StkPlucked)
|
| https://github.com/supercollider/sc3-plugins
|
| Faust - Functional audio processing
|
| https://faustlibraries.grame.fr/libs/physmodels/
|
| https://youtu.be/u8WTnQPzL2w?si=DY5J-ktLIYLdM_fB
|
| Pure Data (Pd) - pmpd (Physical Modeling for Pure Data) library
|
| https://puredata.info/downloads/pmpd
| stefanha wrote:
| > the sound from musical instruments
|
| Pianoteq (https://en.wikipedia.org/wiki/Pianoteq) is a physically
| modeled collection of instruments (mostly pianos). Runs even on a
| Raspberry Pi and sounds like the real deal without gigabytes of
| prerecorded samples. Super impressive what physical modeling can
| achieve.
| jayd16 wrote:
| nVidia has had some implementation of path traced audio in their
| VR SKDs for quite a while. I don't think its that's popular,
| though.
|
| https://www.youtube.com/watch?v=Ozhywx2YbzM
| thrtythreeforty wrote:
| > Like howling of wind, the sound from musical instruments
|
| The typical technique used to model these is a "digital
| waveguide" - treating the wind instrument or string as a one-
| dimensional tube. The wave propagating down the tube is sampled
| at your sample rate _spatially_ , so the computer model consists
| of a pair of queues of samples propagating in each direction. A
| typical guitar string might have a pair of buffers 300-600 audio
| samples long, and you move pointers into those buffers every
| cycle to model the propagation. I'll link to PASP as others are
| doing for a diagram:
|
| https://ccrma.stanford.edu/~jos/pasp/Ideal_Acoustic_Tube.htm...
|
| At the end of the waveguide (typically where the string is
| attached to the bridge, or the sound hole of a tube is), the
| traveling wave encounters an impedance _discontinuity_ , exactly
| like an electrical wave on a transmission line would, except the
| units are different. This discontinuity causes the energy in the
| wave to reflect, and as this repeats, the string vibrates.
|
| That's the core technique. There is so much research on top of
| this, backed by hard math, to model other nonlinear behaviors of
| strings, to add other components to the model, etc. I find it
| fascinating how physical modeling is very well studied within a
| very small circle of researchers, and nearly nobody else has
| heard of the concepts.
|
| Happy to discuss further, either here or over email, if you or
| anyone else has questions!
| an_aparallel wrote:
| I also recommend Bart Hopkin's book "Musical Instrument Design" -
| which is a practical crash course in DIY musical instrument
| making / mixed in with a below informal instroduction to
| acoustics/physics.
| moffkalast wrote:
| Man do I have a brilliant youtube channel to point you at:
| https://www.youtube.com/@AngeTheGreat/videos
| tlarkworthy wrote:
| The pluckable string demo on ycombinator the other day is pretty
| cool https://news.ycombinator.com/item?id=40442595
|
| Just simulating a string vibrating has a huge variety of sounds
| depending on how its harmonics pile up and where you sample from
| it (like where the pickup is on an electric guitar).
| joe_away629 wrote:
| Search "physical modeling" on dafx.de: https://www.dafx.de/paper-
| archive/search.php?q=physical+mode...
|
| You might also like to search "analog modeling" or "virtual
| analog" for models of analog electronic circuits (distortion
| pedals, filters, etc.)
|
| Book about modeling, eg., car engines in Pure Data (eg., for
| games): https://mitpress.mit.edu/9780262014410/designing-sound/
|
| You might check out the source code to Emilie Gillet's (of
| Mutable Instruments) Rings and Elements modules:
| https://github.com/pichenettes/eurorack
|
| "Designing the Make Noise Erbe-Verb" Reverb Design Lecture":
| https://www.youtube.com/watch?v=Il_qdtQKnqk&t=18s
|
| Sean Costello's (of Valhalla DSP) blog about mostly reverbs:
| https://valhalladsp.wordpress.com/
| hammock wrote:
| "Audio engines" exist.. they are called VSTs or plugins. The ones
| with the most physical modeling are echo and reverb plugins..
| modeling a physical environment and then reproducing it
| digitally. Or noise reduction plugins - reverse engineering
| various "noise" sources and then subtracting them. Very math
| intensive.
|
| There are also analog versions of this.. guitar pedals that
| create echo and reverb that "model" a room sound via physical
| internal parts
| a1o wrote:
| If you need for game reasons, I would advise to inspect the code
| of OpenAL soft which has lots of interesting concepts implemented
| and also to read the documentation on Steam Audio - this one is
| also open source too, so you can take a look on valve's
| implementation here: https://github.com/ValveSoftware/steam-audio
|
| I would at least take a look on how HRTF works as you need to
| account how humans experience sounds too when simulating audio
| things.
| firebot wrote:
| There's MIDI, of course. Then you have things like fruity loops
| and similar.
|
| Then of course there's all kinds of synths for FL studio.
|
| Audio is ultimately analog, and requires a DAC, digital to analog
| conversion. I would think you might have some fun with some
| hardware synths.
| mitthrowaway2 wrote:
| It's a bit old now, but you might be interested in _Phya:
| Physical audio for virtual worlds_
|
| https://phyacode.wordpress.com/2016/01/30/phya/
|
| https://repository.gatech.edu/bitstream/1853/50024/1/Menzies...
___________________________________________________________________
(page generated 2024-06-20 23:01 UTC)