[HN Gopher] Show HN: Audiocube - A 3D DAW for Spatial Audio
___________________________________________________________________
Show HN: Audiocube - A 3D DAW for Spatial Audio
I've recently released my solo project Audiocube I wanted to make
a 3D DAW, where spatial audio, physics, and virtual acoustics are
all directly integrated into the engine. This makes it easy to
create music in 3D, and experiment with new techniques which aren't
possible in traditional DAWs and plugins. I'd love to get any
feedback on this software (Mac/Windows) to make it better. You can
download it for free through the website. Thanks, Noah
Author : noahfk
Score : 130 points
Date : 2025-01-30 13:12 UTC (9 hours ago)
(HTM) web link (www.audiocube.app)
(TXT) w3m dump (www.audiocube.app)
| tines wrote:
| Is there an audio format that stores the 3D origins of each
| sound, so that you could theoretically play it through Airpods
| (or some other spatially-aware headphones) and hear the sound
| change as you tilt your head?
| justinsaccount wrote:
| Something like https://en.wikipedia.org/wiki/Dolby_Atmos and
| https://en.wikipedia.org/wiki/DTS,_Inc.#DTS:X
| hyperific wrote:
| Another way to do this would be to make an app that uses
| positional information from the headphones and feeds the sound
| of your choice into a head related transfer function (HRTF) to
| give the illusion of spatial audio.
| bee_rider wrote:
| I sort of expected this sort of thing to be a nice use for VR,
| since it already needs to do the tracking. But VR generally
| didn't seem to catch on, so...
| PaulHoule wrote:
| I have a home theater which is 5.1 which is great for movies
| [1] and some music [2]. There are 'virtual surround' systems
| based on [3] of which Airpods are the most famous. Most of
| those don't work for me except in the sense that I learn
| 'sounds like I have a cold' means 'the sound is supposed to
| be above me'. The exception is my Hololens where if I use the
| web browser to play music and turn around it _really_ sounds
| like the music is coming from behind me. I have yet to be
| impressed by spatial audio on my Meta Quest 3.
|
| [1] Oddly the surround sound soundtracks that impress me the
| most are old VHS tapes encoded with Dolby Pro Logic
|
| [2] let's see... _Close to the Edge_ by Yes, _Supernature_ by
| Goldfrapp, _Maximum-Minimum_ by Kraftwerk
|
| [3] https://en.wikipedia.org/wiki/Head-
| related_transfer_function
| _flux wrote:
| You could also just emulate what the music would sound like
| with a stereo system and support head tracking with that. dear
| reality does something like that with their dearVR digital
| audio workstation plugins.
|
| And then I recall there was this one old Kickstarter project
| that made headphones with that idea. It had an IR transmitter
| for head tracking.
|
| I think it might be quite difficult to track head movement with
| just earbuds and not have the tracking drift, though.
| noahfk wrote:
| I'm working on a mobile app that lets you playback projects
| created in Audiocube and gives you full control over where your
| listening position.
|
| It's definitely possible to have head tracking, I think it will
| just depend on what headphones and hardware offer it. It would
| be cool to give you that kind of VR listening experience but
| just on headphones
| colanderman wrote:
| Dolby Atmos is literally that, I think there are several
| physical formats that can support it.
|
| Ambisonics, especially higher-order ambisonics, also is capable
| of spatial panning, though it encodes the 3D soundfield rather
| than individual sound sources. Again, there are a few physical
| formats.
| mrandish wrote:
| In addition to Dolby Atmos there is MPEG-H 3D Audio.
|
| https://en.wikipedia.org/wiki/MPEG-H_3D_Audio
|
| https://ieeexplore.ieee.org/document/7056445
| glenneroo wrote:
| This is probably a very odd and roundabout way of doing
| something akin to what you are asking for, but my recently
| released VR game DodgeALL[0] has a form of modding support for
| (wav/mono) audio files which can be uploaded (via SideQuest) to
| the headset. These are then assigned to "obstacles" which are
| launched at the player i.e. you could move your head around and
| hear spatialized audio files as they fly around the arena. You
| don't even need special headphones, the ones on the Quest do a
| good enough job, and Meta's Spatial audio SDK does a wonderful
| job of spatializing audio. I was originally using SteamAudio[1]
| but switched mainly due to moving from PC-VR to Standalone.
|
| Sorry if this sounds like some self-promotion - I specifically
| built this functionality[2] for my previous audio engineer so
| that he could quickly design and test sounds without the hassle
| of having to build/install an APK with every new audio file.
|
| [0]: https://www.DodgeALL.com (only for Quest 2/3/3s/Pro
| headsets)
|
| [1]: https://valvesoftware.github.io/steam-audio/
|
| [2]:
| https://docs.google.com/document/d/1IFQjkroomHh8FocjfA-a55JP...
| JofArnold wrote:
| This is great (in theory). During lockdown I got an ambisonic mic
| (Rode NF-SFW1) and used it to create Dolby Atmos experiences. The
| workflow - including sending it to Dolby's tool every time - was
| such a pain. Adding additional 3d elements was especially
| annoying and limiting.
|
| Unfortunately that's no longer my hobby so can't test this for
| you but definitely scratches an itch for past me. Nice
| noahfk wrote:
| Thanks!
|
| Yeah, frustration with the existing solutions was one of the
| main reasons I started this project.
|
| I was working in creating immersive audio, and I just found
| that none of the plugins allowed the level of placement that I
| wanted. It was all fiddly, and you couldn't easily move the
| listening position around, while also moving sound sources.
|
| I came from an audio engineering background, not programming,
| so it took me a while to even learn how to make software! But
| now I made the tool I always needed.
| bitdivision wrote:
| Very cool project! And well done on doing it without being a
| software eng!
|
| How long have you been working on it out of interest?
| FpUser wrote:
| Lots of cool software was created by domain experts who
| taught themselves how to program. And quality is more than
| often is excellent.
| noahfk wrote:
| Thanks! Yeah it was a steep learning curve at the start but
| I'm very confident now.
|
| I started it in 2020 - here's a blog post about the history
| https://www.audiocube.app/blog/audiocube-history
|
| I've recently got it working on an iPad which was a
| frustrating grind but I think the concept is a lot of fun
| on touchscreen!
| gareth_untether wrote:
| That mic looks fun to play with. On their site there's a link
| to a bunch of recordings that can be downloaded:
|
| https://library.soundfield.com/
| hyperific wrote:
| YouTuber Benn Jordan would probably get a kick out of this. He's
| a major audiophile and did a series of ambisonic ambience.
| noahfk wrote:
| Thanks, I think so too by the looks of his channel, I'll get in
| touch!
| roddylindsay wrote:
| Nice work! Can you export for multichannel playback or is it
| binaural / stereo?
| noahfk wrote:
| I'm working on multichannel playback, I've had some
| experimental success, and plan on adding it into the next
| update. it just needs more testing on a range of audio
| interfaces etc.
|
| Currently it only exports/plays back in binaural stereo
| PaulHoule wrote:
| Outputting 5.1 as a DTS-CD (or something I can convert to a
| DTS-CD) would give me a reason to buy it.
| noahfk wrote:
| I'm researching the possibility of other export formats
| that can be encoded with more spatial information - it's
| highly requested
| radicality wrote:
| Ah. I think I misunderstood the purpose of the app. When
| viewing the main page, I had the impression that the output
| would be targeting large Atmos multi-speaker installations.
| Or that it would export to a format that can play spatially
| on AirPods.
|
| So right now, is it possible to make something in Audiocube
| and export it to native spatial AirPod sound ?
| noahfk wrote:
| I am working on multichannel output but it's a bit of a
| headache with all the formats and interfaces etc - I want a
| simple solution, so that needs a bit of research.
|
| Currently I believe apple spatial uses some kind of 5.1 or
| 7.1 format audio files mixed with a binauralizer to
| distribute on headphones. I'm building a method to export
| in surround formats but it needs a bit more research.
| Currently it exports as a binaural wav that sounds very 3d,
| but I'll look more into apple Spatial Audio as I haven't
| researched it much
| seltzered_ wrote:
| Not spatial audio, but reminded me of audioGL (2012, but a newer
| video posted in 2024): https://m.youtube.com/@AudioGL/videos
|
| https://news.ycombinator.com/item?id=3579543
| noahfk wrote:
| I believe the HRTF binauralization of object-based audio does
| qualify as spatial audio, but there are many different
| definitions floating around. AudioGL is cool though!
| rzzzt wrote:
| There was also a BeOS application with a 3D mixing interface:
| https://en.wikipedia.org/wiki/3dmiX
| beeburrt wrote:
| For those of us unfamiliar with the term DAW, I assume it's
| Digital Audio Workstation.
|
| At first glance I thought of: DEW
|
| https://en.m.wikipedia.org/wiki/Directed-energy_weapon
| beng-nl wrote:
| It's a digital audio workstation.
| ics wrote:
| Digital Audio Workstation:
| https://en.wikipedia.org/wiki/Digital_audio_workstation
| sitkack wrote:
| I assume this can do sound rendering, like simulating a
| conversation on a subway platform while a subway passes by?
|
| Or singing while walking through a tunnel?
|
| Since it has capabilities that would be hard to replicate, rather
| than show the tool on the landing page, I would show the output.
| Remove the clutter and force people to listen to what the tool
| can produce.
|
| The tool as it is now, is being marketed towards yourself, people
| that wanted to build that tool and know what it is. But everyone
| will know what it can do after listening to sample output.
| noahfk wrote:
| Yes it can do all of those things.
|
| You are right about the audio demos, I'll update the frontpage
| with this soon. Thanks!
| poeticfolly wrote:
| Very interesting! Did you write your own acoustic simulation
| engine for this?
| noahfk wrote:
| Yes - while the app itself is built in unity, it doesn't use
| Unity's native audio simulation. I implemented a custom
| spatializer and acoustic engine that can simulate reflections,
| occlusion, and other stuff that wasn't native in unity
| ggerules wrote:
| Will there be a Linux version anytime soon!?
| noahfk wrote:
| I don't have a Linux machine or much experience with it, but I
| can easily build to it. If you want to help test a Linux build
| email noah@audiocube.app
| ggerules wrote:
| Great! Look for a private message.
| dekhn wrote:
| IIUC, I'd be able to take individual closely miked recording on
| multiple different instruments and mix them into a soundspace,
| such that when I listen on stereo headphones, I'd be able to
| "locate" the sounds on a virtual stage?
|
| (asking because I listen to a lot of live jam music in stereo and
| noticed that they use a stereo mix with a virtual image)
| mystifyingpoi wrote:
| That's how normal panning works, yes (or using a stereo pair of
| ambient mics, same thing but baked into the recording). But
| that's left-right only, I assume that these Atmos tools can do
| it in 3D space.
| dekhn wrote:
| Does that work if you only have a pair of stereo headphones?
|
| I've had quad speakers before (with a "delay" unit for the
| rear speakers) but I think it would be hard to do 3D in
| stereo (I know about
| https://en.wikipedia.org/wiki/3D_audio_effect and HRTFs but I
| have never noticed any 3d-ness to the sound I'm listening to
| in games).
| noahfk wrote:
| Yes, on standard stereo headphones, a binauralized sound
| can give the impression of sound coming from all around
| your head. It mimics the filtering acoustic effects applied
| by your head and ears
| noahfk wrote:
| Yes exactly - this was something I wanted to achieve.
|
| For example, you could have an entire orchestra with each
| instrument close miked - and then arranged in audiocube in a
| vritual concert hall.
|
| The user could even move around the space for full placement
| control - or you could simply render it to a binauralized
| stereo file for more immersive listening on headphones.
|
| I'd like to develop a mobile "listening" app, which isn't for
| making projects, but listening to them and placing you in the
| virtual environment with control
| crazygringo wrote:
| This seems intriguing but I'm genuinely confused.
|
| It seems like "bakes in" spatial audio to binaural stereo?
|
| But who is the market for that?
|
| I love spatial audio on my AirPods but a big feature is that it
| moves with my head and can even be customized for my ears.
|
| And I certainly _don 't_ want it applied when downsizing to a
| mono Bluetooth speaker.
|
| It seems like you'd need to export your final product to
| surround/Atmos for the way people want to, and currently do,
| consume spatial audio? I assume the target here is Apple Music,
| short films, etc.?
|
| I mean I think the concept of the 3D DAW is great. I just want to
| make sure there's a product-market fit here, so you can succeed.
| Or is there a market I'm overlooking?
| noahfk wrote:
| Thanks for the intrigue, and yes these are all valid questions
| and things I'm trying to work out.
|
| I think I need to narrow down the market a bit. Currently I'm
| aiming more at audio and sound design experimenters rather than
| the deep technical stuff - although that is a goal later.
|
| I'd like to create a 'listener app' so people can have full
| immersion, control, and exploration of audio scenes.
|
| The binaural process is more of a high powered, more flexible
| and controllable version of a binarual plugin, so this would be
| for making binaural mixes rather than slapping on everything.
|
| I personally think the binauralizarion is one of the best
| sounding out there, it sounds great on headphones and has
| excellent sound localisation.
|
| But you are completely right. I'm still working out product
| market fit, and I think it's why I'm finding a bit of friction
| reaching the right audience.
|
| Thanks for your thoughts! Any feedback and input is very
| appreciated
| photonthug wrote:
| Just guessing, but even setting aside VR and whatever kind of
| stuff they are doing for theaters like imax, there does seem to
| be an interest in new kinds of immersive 3d audio/video. The
| Las Vegas sphere aimed to do this iirc, and the money behind it
| aimed to develop the IP and open similar venues in different
| places around the world. Supposedly media that's under
| development actually uses a smaller copy of the target venue,
| so if you can do that with a software suite instead it's
| probably a good thing.
| noahfk wrote:
| I've even heard of them using this kind of tech in theatre
| performances, so there is definitely a bunch of use cases!
| TheRealPomax wrote:
| > As a standalone app, Audiocube offers tools, workflows, and
| processes that go beyond the capabilities of VST plugins.
|
| But does it _support_ VST /AU in order to load instruments rather
| than "samples"?
| noahfk wrote:
| There is currently no VST/plugin support, although this is
| something I would like to include later on.
|
| It's just too much development work at this stage - not only
| the VST integration itself, but building a full MIDI sequencer
| timeline, editor, etc to support instruments.
|
| Other DAWs do this perfectly well so I'm focusing on the 3D
| virtual environment moreso than tradtional DAW functions.
| Audiocube isn't a tool to compete with the likes of Ableton and
| Logic, but something to run on the side.
|
| Although, if I had a big enough team I would love to give it
| all of these features. I'm just doing it solo at the moment, so
| I need to have a narrower focus.
| PaulDavisThe1st wrote:
| "Why You Should Not Write a DAW" (ADC 2024)
|
| https://www.youtube.com/watch?v=GMlnh6_9aTc
| noahfk wrote:
| Yeah this is an interesting talk. A full DAW is maybe over
| ambitious for one developer, but I don't think vst plugins
| would be able to achieve what I wanted. For example vsts
| can easily spatialise individual sounds, but there is no
| way to then easily move the listening position through the
| scene, without a disgusting amount of automation - hence
| building a virtual 3d environment daw with more placement
| flexibility
| PaulDavisThe1st wrote:
| ... or start with an open source DAW that already has all
| the rest, and add the placement flexibility you want ...
|
| But I'm speaking out of self-interest here, and anyway
| for now your output is limited to binaural stereo which
| is not really a viable target for production workflows at
| this point.
| S0y wrote:
| It's funny to see this now because I've been for a couple weeks
| looking into audio spacialization. After a lot of research and
| even trying to write my own spatializer plugin, I found that Game
| Engines have probably the most complete toolset to do this task.
| (Specifically I'm using Godot with
| https://valvesoftware.github.io/steam-audio/).
|
| Steam audio is pretty awesome in that regards because it supports
| HRTF and all the physical based goodies like occlusion/reflection
| and sound propagation. So you can get really really immersive
| spatial audio.
|
| The only downside with this solution is that you can't do offline
| rendering. So my question is:
|
| can Audiocube do offline rendering? seems like it would be one
| killer feature for my use case.
| noahfk wrote:
| It sounds like you're in a similar field to me so it would be
| great to connect! Totally agree, yeah Audiocube is built in
| unity as game engines made the most sense for what I wanted to
| achieve. Steam audio is super cool and was a lot of use to look
| at for writing my own stuff.
|
| Offline rendering is at the top of my research list currently.
| It's not in the app yet. It's going to take a lot of work to
| build given that the physics engine calculations are tied to
| frame rate etc, so there is a lot of work to do - but I'm sure
| it's possible.
|
| Thanks!
| meta-meta wrote:
| This looks great! How small of an audio buffer have you been able
| to get down to? Any plans for an API?
|
| I've been developing a VR spatial sound and music app for a few
| years with the Unity game engine, bypassing the game engine's
| audio and instead remote controlling Ambisonic VSTs in REAPER. I
| can achieve low latency with that approach but it's a bit limited
| because all the tracks and routing need to be setup beforehand.
| There's probably a way to script it on REAPER but that sounds
| like an uphill battle. It would be a lot more natural to
| interface with an audio backend that is organized in terms of
| audio objects in space.
|
| What I'd like is more flexibility to create and destroy objects
| on the fly. The VSTs I'm working with don't have any sort of
| occlusion either. That would be really nice to play with. Meta
| has released a baked audio raytracing solution for Quest, and
| that's fun for some situations but the latency is a bit too much
| for a satisfying virtual instrument.
|
| Here's my project for context: https://musicality.computer/vr
| noahfk wrote:
| Hey, I've got down to under 20ms audio buffer, but it depends
| on the complexity of the scene.
|
| What you're working on sounds really cool, I'll have a look at
| it!
|
| It sounds like Audiocube offers the kind of features that you
| need, although it doesn't have realtime audio input (yet, I'm
| working on it and have it partially working).
| meta-meta wrote:
| Hmm 20ms is quite a lot. I'm targeting <3ms, ideally.
| noahfk wrote:
| Looking at your project it looks like it would be great to
| integrate the two somehow - Audiocube would be awesome in vr
| but I have no vr dev experience
| ganonm wrote:
| There's a library Valve made for spatial audio for games
| (inc. VR). I've played around with it a bit, it's incredible.
| I'm surprised more games haven't adopted it.
|
| https://valvesoftware.github.io/steam-audio/
| meta-meta wrote:
| Oh yeah, I forgot about this. I think it was a bit less
| developed last I looked. Not sure if there was another
| reason I wasn't considering it. Thanks!
|
| edit: ah, it's because it is a plugin for Unity's audio
| system which I've already determined adds more latency than
| I want.
| meta-meta wrote:
| Thanks! I joined your Discord... And looks like I "liked"
| your Audiocube post on FB about 4 years ago :-)
| thot_experiment wrote:
| Really cool! I've been working on a side project that utilizes
| spatial audio and I've been pleasantly surprised by the quality
| I'm getting just using the WebAudioAPI HRTF spatialization. I'm
| sure this is leagues ahead but it was really nice to find that I
| didn't really need to do anything to get decent spatial audio
| other than set the panner node to HRTF mode.
| noahfk wrote:
| Thanks! Yeah this is definitely a lot more complex and deep
| than the WebAudio API but that's still a great tool for some
| situations.
| itishappy wrote:
| You using Blender's viewport? (Great idea if so, nearly perfect
| recreation if no!)
| noahfk wrote:
| Aha, yeah blender was a huge UI inspiration, it's all handmade
| from scratch though!
| itishappy wrote:
| Awesome work, you nailed it!
| ge96 wrote:
| Neat interface the grouped ones with matching neighbor contours
| noahfk wrote:
| thanks!
| bdahz wrote:
| The free download requires account registration, which will
| discourage most people from trying it. Even third-party login
| might make it easier, but I didn't find that option.
___________________________________________________________________
(page generated 2025-01-30 23:00 UTC)