[HN Gopher] OpenAI releases Whisper v3, new generation open sour...
___________________________________________________________________
OpenAI releases Whisper v3, new generation open source ASR model
Author : crakenzak
Score : 44 points
Date : 2023-11-06 18:50 UTC (2 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| nshm wrote:
| Good improvements for many languages, numbers here
|
| https://github.com/openai/whisper/blob/main/language-breakdo...
| WXLCKNO wrote:
| Curious as to how dutch has the lowest error rate
| crucialfelix wrote:
| They enunciate.
| tekacs wrote:
| From the WER numbers alone it looks like a very small
| difference for English itself, but I've found WER to be a
| misleading assessment mechanism.
|
| Having extensively tested Whisper v2 large against other 'lower
| WER' models and found them wanting (because of differences in
| their methodology for generating output), I'm super curious to
| get a feel for how v3 holistically behaves.
|
| Will probably test it right now. :)
| Void_ wrote:
| I don't understand how a pop. 10M country - Czech Republic is
| among the best.
|
| And I can confirm - my app Whisper Memos
| (https://whispermemos.com) is very popular in Czech Republic.
|
| It makes perfect sense. Whisper is almost as good as
| transcribing Czech as English!
| GaggiX wrote:
| I don't know Czech, but Italian is extremely consistent in
| the way it's written, so it's at the top of the list with
| about one or two orders of magnitude less data.
| godelski wrote:
| I'm more impressed about Korean! I didn't even realize it was
| that good in V2. But I've just seen a lot of systems perform
| really poorly (judged by my Korean gf not me) and Korea is
| only a country of 52M (between Spain and Italy).
|
| A funny note, if Siri is set in Korean mode and reads your
| texts that come in as English, they sound like a racist
| imitation of a Korean accent. It is absolutely hilarious.
| ComputerGuru wrote:
| It looks like it's basically whisper-2 with extra training
| against datasets for specific languages that brought incidental
| improvements to the rest. Support for some of the languages is
| still really bad (from real-world experience).
| tomrod wrote:
| Word from my GenAI contact is that this (or similar announcement)
| replaces the need for RAG.
| danielbln wrote:
| An ASR model replaces RAG?!
| tekacs wrote:
| Pretty sure they're confusing it with the Retrieval feature
| added to the Assistants API: https://platform.openai.com/docs
| /assistants/tools/knowledge-...
| alex_young wrote:
| Still doesn't look like it can do real-time unfortunately.
|
| Edit: I understand that you can use small samples and approximate
| something like streaming, but the limitation here is you wind up
| without context for the samples, increasing WER. It would be nice
| if there was some streaming option.
| thot_experiment wrote:
| Hm? I've never had trouble getting faster than realtime perf
| out of the old Whisper? What sort of hardware are you running?
| manmal wrote:
| Isn't the problem that you can't stream live audio data in?
| tekacs wrote:
| So... you have to operate in chunks, 30s at a time
| generally, although you get reduced time-per-chunk if there
| are zeroes in the chunk and there are variants.
|
| The zeroes + faster model is how Gladia (mentioned in my
| other comment here) achieves live transcription by simply
| transcribing really short chunks one after the other, I
| believe.
|
| For more advanced stuff you kinda have to get your hands
| dirty, which I've done for my own product (not linked).
| modeless wrote:
| Interesting teaser. I thought there must be some way to
| better optimize the model for real time, but haven't dug
| in because it's decently fast as is and there's so much
| other stuff to work on. So many models, so little time!
| tekacs wrote:
| A few providers have done a variant of live transcription
| that's similar to how old-school providers do it, where they
| transcribe a short window (i.e. XXXms at a time) and this is
| definitely the easiest path. One such provider is Gladia:
| https://www.gladia.io/
|
| There are other ways too with different trade-offs, can e-mail
| me at the link in my profile if you'd like to talk about how.
| modeless wrote:
| I have Whisper working in real time here, with TTS too for a
| real time voice AI that's _much_ faster than ChatGPT 's voice
| mode: https://www.microsoft.com/store/apps/9NC624PBFGB7
|
| It's inefficient, but even older gaming GPUs are fast enough
| for real time performance, and accuracy is good. If you were
| going to train a model from scratch for real time you could do
| something more efficient, but it works as is.
|
| Edit: I'm not sure what you mean by "you wind up without
| context for the samples". You can supply context to Whisper.
| dang wrote:
| Related ongoing threads:
|
| _New models and developer products_ -
| https://news.ycombinator.com/item?id=38166420
|
| _OpenAI DevDay, Opening Keynote Livestream [video]_ -
| https://news.ycombinator.com/item?id=38165090
| ComputerGuru wrote:
| They say whisper-3 will be available via the api soon. Does
| anyone know why only whisper-1 was ever made available via the
| api (no whisper-2)?
| Topfi wrote:
| whisper-large-v2 was/is available via API from OpenAI, simply
| wasn't as much of a focus in their recent communications.
| Void_ wrote:
| > Developers can now use our open-source Whisper large-v2 model
| in the API with much faster and cost-effective results.
|
| https://openai.com/blog/introducing-chatgpt-and-whisper-apis
| bakkoting wrote:
| The endpoint was just confusingly named:
|
| > The Whisper v2-large model is currently available through our
| API with the whisper-1 model name.
|
| https://platform.openai.com/docs/models/whisper
| singularity2001 wrote:
| did they break the api?
|
| from openai import OpenAI
|
| Traceback (most recent call last): File "<stdin>", line 1, in
| <module> ImportError: cannot import name 'OpenAI' from 'openai'
|
| If so where is the current documentation?
| jsight wrote:
| This seems like the best free voice recognition in general.
|
| Is there a model that is the best at wake word detection? The
| last that I looked, it seemed like this was fairly lacking.
| Nitrolo wrote:
| Does anyone know of a nice UI wrapper for something like
| whisper.cpp?
|
| I need to write a lot of long texts for work and some good
| dictation software would be great. I know there's Dragon, but
| somehow I have not been able to find something that fits my need
| and is free.
| conradev wrote:
| for macOS: https://goodsnooze.gumroad.com/l/macwhisper
___________________________________________________________________
(page generated 2023-11-06 21:01 UTC)