[HN Gopher] AiOla open-sources ultra-fast 'multi-head' speech re...
       ___________________________________________________________________
        
       AiOla open-sources ultra-fast 'multi-head' speech recognition model
        
       Author : cheptsov
       Score  : 67 points
       Date   : 2024-08-03 08:30 UTC (14 hours ago)
        
 (HTM) web link (aiola.com)
 (TXT) w3m dump (aiola.com)
        
       | gronky_ wrote:
       | GH repo: https://github.com/aiola-lab/whisper-medusa
        
       | Doohickey-d wrote:
       | I'm curious which of the Whisper derivatives is actually the
       | fastest ?
       | 
       | Since faster-whisper claims 4x speedup over base Whisper, and
       | I've found WhisperX to be faster still (for longer audio where it
       | can do batch inference), at least on consumer GPUs.
       | 
       | So with AiOla saying "50% speedup", is that actually noteworthy?
        
         | gronky_ wrote:
         | From my understanding faster-whisper optimizes the inference
         | without changing the model itself. Here they seem to be
         | changing the model architecture but not applying other
         | optimizations.
         | 
         | 50% on its own doesn't make this the current best choice for
         | production. But I imagine this could become the new base model
         | that all of the inference optimizations are applied to.
         | 
         | Wonder if it's plug and play or if faster-whisper and others
         | would need to reimplement from scratch?
        
           | dloss wrote:
           | Is this even faster? https://github.com/Vaibhavs10/insanely-
           | fast-whisper
           | 
           | If so, is the quality still acceptable?
        
         | tomp wrote:
         | Depends what you mean by "fast".
         | 
         | I've tested WhisperLive, it's basically real-time (i.e. low
         | _latency_ ).
        
           | gunalx wrote:
           | Dosent whisperlive just use faster-whisper under the hood.
           | Witch can be way faster than real time.
        
       | qwertox wrote:
       | Nothing of interest here, it's an ad.
       | 
       | If you're interested, you might as well check out Gladia, at
       | least they have a pricing section and allow you to use it as a
       | developer, unlike just asking you to "Request a Demo".
       | 
       | And while a sibling comment links to the GitHub repository, their
       | entire website does not contain such a link.
       | 
       | ---
       | 
       | Edit: My bad, for some reason I first checked the website instead
       | of the blog post. Looks much more interesting now.
        
         | cheptsov wrote:
         | They have shared the link to GitHub [1], HuggingFace repo [2],
         | and the paper [3]:
         | 
         | 1. https://github.com/aiola-lab/whisper-medusa
         | 
         | 2. https://huggingface.co/aiola/whisper-medusa-v1
         | 
         | 3. https://paperswithcode.com/method/multi-head-attention
        
           | nmstoker wrote:
           | Looks like they left out all training code, presumably for
           | commercial reasons (but it only just came out so it's
           | conceivable they are just cleaning up that side of the code
           | but I doubt it). Totally their call, given they've put the
           | effort in, just a shame.
        
       | BetterWhisper wrote:
       | Does it do speaker recognition/ diarization? Can't see it from
       | the repo readme
        
       | phkahler wrote:
       | IIRC Whisper works on wave files. Can this do real time low
       | latency continuous ASR?
        
       ___________________________________________________________________
       (page generated 2024-08-03 23:01 UTC)