[HN Gopher] Ask HN: What's the current best local/open speech-to...
       ___________________________________________________________________
        
       Ask HN: What's the current best local/open speech-to-speech setup?
        
       I'm trying to do the "voice assistant" thing fully locally: mic -
       model - speaker, low latency, ideally streaming + interruptible
       (barge-in).  Qwen3 Omni looks perfect on paper ("real-time",
       speech-to-speech, etc). But I've been poking around and I can't
       find a single reproducible "here's how I got the open weights doing
       real speech-to-speech locally" writeup. Lots of "speech in - text
       out" or "audio out after the model finishes", but not a usable
       realtime voice loop. Feels like either (a) the tooling isn't there
       yet, or (b) I'm missing the secret sauce.  What are people actually
       using in 2026 if they want open + local voice?  Is anyone doing
       true end-to-end speech models locally (streaming audio out), or is
       the SOTA still "streaming ASR + LLM + streaming TTS" glued
       together?  If you did get Qwen3 Omni speech-to-speech working: what
       stack (transformers / vLLM-omni / something else), what hardware,
       and is it actually realtime?  What's the most "works today" combo
       on a single GPU?  Bonus: rough numbers people see for mic - first
       audio back  Would love pointers to repos, configs, or "this is the
       one that finally worked for me" war stories.
        
       Author : dsrtslnd23
       Score  : 17 points
       Date   : 2026-01-23 11:04 UTC (11 hours ago)
        
       | jauntywundrkind wrote:
       | It was a little annoying getting old qt5 tools installed but I
       | really enjoyed using dsnote / Speech Note. Huge model selection
       | for my amd gpu. Good tool. I haven't done enough specific
       | studying yet to give you suggestions for which model to go with.
       | WhisperFlow is very popular.
       | 
       | Kyutai some very interesting work always. Their delayed streams
       | work is bleeding edge & sounds very promising especially for low
       | latency. Not sure why I have not yet tried it tbh.
       | https://github.com/kyutai-labs/delayed-streams-modeling
       | 
       | There's also a really nice elegant simple app Handy. Only
       | supports Whisper and Parakeet V3 but nice app & those are amazing
       | models. https://github.com/cjpais/Handy
        
       | mpaepper wrote:
       | You should look into the new Nvidia model:
       | https://research.nvidia.com/labs/adlr/personaplex/
       | 
       | It has dual channel input / output and a very permissible license
        
         | dsrtslnd23 wrote:
         | oh - very interesting indeed! thanks
        
       ___________________________________________________________________
       (page generated 2026-01-23 23:00 UTC)