[HN Gopher] Voxtral Transcribe 2
___________________________________________________________________
Voxtral Transcribe 2
Author : meetpateltech
Score : 620 points
Date : 2026-02-04 15:08 UTC (7 hours ago)
(HTM) web link (mistral.ai)
(TXT) w3m dump (mistral.ai)
| observationist wrote:
| Native diarization, this looks exciting. edit: or not, no
| diarization in real-time.
|
| https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-26...
|
| ~9GB model.
| coder543 wrote:
| The diarization is on Voxtral Mini Transcribe V2, not Voxtral
| Mini 4B.
| observationist wrote:
| Ahh, yeah, and it's explicitly not working for realtime
| streams. Good catch!
| sbrother wrote:
| Do you have experience with that model for diarization? Does
| it feel accurate, and what's its realtime factor on a typical
| GPU? Diarization has been the biggest thorn in my side for a
| long time..
| coder543 wrote:
| > Do you have experience with that model
|
| No, I just heard about it this morning.
| ashenke wrote:
| You can test it yourself for free on
| https://console.mistral.ai/build/audio/speech-to-text I
| tried it on an english-speaking podcast episode, and apart
| from identying one host as two different speakers (but only
| once for a few sentences at the start), the rest was
| flawless from what I could see
| sbrother wrote:
| Amazing. Thank you.
| serf wrote:
| things I hate:
|
| "Click me to try now!" banners that lead to a warning screen that
| says "Oh, only paying members, whoops!"
|
| So, you don't mean 'try this out', you mean 'buy this product'.
|
| Let's not act like it's a free sampler.
|
| I can't comment on the model : i'm not giving them money.
| ReadEvalPost wrote:
| You can try it on HF:
| https://huggingface.co/spaces/mistralai/Voxtral-Mini-Realtim...
| boobsbr wrote:
| I'm impressed.
| mdrzn wrote:
| There's no comparison to Whisper Large v3 or other Whisper
| models..
|
| Is it better? Worse? Why do they only compare to gpt4o mini
| transcribe?
| GaggiX wrote:
| Gpt4o mini transcribe is better and actually realtime. Whisper
| is trained to encode the entire audio (or at least 30s chunks)
| and then decode it.
| emmettm wrote:
| The linked article claims the average word error rate for
| Voxtral mini v2 is lower than GPT-4o mini transcribe
| GaggiX wrote:
| Gpt4o mini transcribe is better than whisper, the context
| is the parent comment.
| mdrzn wrote:
| So "gpt4o mini transcribe" is not just whisper v3 under the
| hood? Btw it's $0.006 / minute
|
| For Whisper API online (with v3 large) I've found "$0.00125
| per compute second" which is the cheapest absolute I've ever
| found.
| GaggiX wrote:
| >So it's not just whisper v3 under the hood?
|
| Why it should be Whisper v3? They even released an open
| model: https://huggingface.co/mistralai/Voxtral-
| Mini-4B-Realtime-26...
| breisa wrote:
| Deepinfra offers Whisper V3 at 0.00045$ / minute of
| transcribed audio.
| tekacs wrote:
| WER is slightly misleading, but Whisper Large v3 WER is
| classically around 10%, I think, and 12% with Turbo.
|
| The thing that makes it particularly misleading is that models
| that do transcription to lowercase and then use inverse text
| normalization to restore structure and grammar end up making a
| very different class of mistakes than Whisper, which goes
| directly to final form text including punctuation and quotes
| and tone.
|
| But nonetheless, they're claiming such a lower error rate than
| Whisper that it's almost not in the same bucket.
| tekacs wrote:
| On the topic of things being misleading, GPT-4o transcriber
| is a very _different_ transcriber to Whisper. I would say not
| better or worse, despite characterizations such. So it is a
| little difficult to compare on just the numbers.
|
| There's a reason that quite a lot of good transcribers still
| use V2, not V3.
| satvikpendem wrote:
| Different how?
| dmix wrote:
| > At approximately 4% word error rate on FLEURS and $0.003/min
|
| Amazons transcription service is $0.024 per minute, pretty big
| difference https://aws.amazon.com/transcribe/pricing/
| mdrzn wrote:
| Is it 0.003 per minute of audio uploaded, or "compute minute"?
|
| For example fal.ai has a Whisper API endpoint priced at
| "$0.00125 per compute second" which (at 10-25x realtime) is
| EXTREMELY cheaper than all the competitors.
| Oras wrote:
| I think the point is having it for real-time; this is for
| conversations rather than transcribing audio files.
| jamilton wrote:
| That quote was for the non-realtime model.
| Archelaos wrote:
| As a rule of thumb for software that I use regularly, it is very
| useful to consider the costs over a 10-year period in order to
| compare it with software that I purchase for lifetime to install
| at home. So that means 1,798.80 $ for the Pro version.
|
| What estimates do others use?
| antirez wrote:
| Italian represents, I believe, the most phonetically advanced
| human language. It has the right compromise among information
| density, understandability, and ability to speech much faster to
| compensate the redundancy. It's like if it had error correction
| built-in. Note that it's not just that it has the lower error
| rate, but is also underrepresented in most datasets.
| Archelaos wrote:
| This is largely due to the fact that modern Italian is a
| systematised language that emerged from a literary movement
| (whose most prominent representative is Alessandro Manzoni) to
| establish a uniform language for the Italian people. At the
| time of Italian unification in 1861, only about 2.5% of the
| population could speak this language.
| gbalduzzi wrote:
| The language itself was not invented for the purpose: it was
| the language spoken in Florence, than adopted by the literary
| movement and than selected as the national language.
|
| It seems like the best tradeoff between information density
| and understandability actually comes from the deep latin
| roots of the language
| gbalduzzi wrote:
| I was honestly surprised to find it in the first place, because
| I assumed English to be at first place given the simpler
| grammar and the huge dataset available.
|
| I agree with your belief, other languages have either lower
| density (e.g. German) or lower understandability (e.g. English)
| riffraff wrote:
| English has a ton of homophones, way more sounds that differ
| slightly (long/short vowels), and major pronunciation
| differences across major "official" languages (think
| Australia/US/Canada/UK).
|
| Italian has one official italian (two, if you count IT_ch,
| but difference is minor), doesn't pay much attention to
| stress and vowel length, and only has a few "confusable"
| sounds (gl/l, gn/n, double consonants, stuff you get wrong in
| primary school). Italian dialects would be a disaster tho :)
| NewsaHackO wrote:
| The only knowledge I have about how difficult Italian is comes
| from Inglourious Basterds.
| mmooss wrote:
| At least some relatively well-known research finds that all
| languages have similar information density in terms of
| bits/second (~39 bits/second based on a quick search).
| Languages do it with different amounts of phonetic sound /
| syllables / words per bit and per second, but the bps comes out
| the same.
|
| I don't know how widely accepted that conclusion is, what
| exceptions there may be, etc.
| hackyhacky wrote:
| > the most phonetically advanced human language
|
| That's interesting. As a linguist, I have to say that Haskell
| is the most computationally advanced programming language,
| having the best balance of clear syntax and expressiveness. I
| am qualified to say this because I once used Haskell to make a
| web site, and I also tried C++ but I kept on getting errors.
|
| /s obviously.
|
| Tldr: computer scientists feel unjustifiably entitled to make
| scientific-sounding but meaningless pronouncements on topics
| outside their field of expertise.
| nindalf wrote:
| I love seeing people from other countries share their own folk
| tales about what makes their countries special and unique. I've
| seen it up close in my country and I always cringed when I
| heard my fellow countrymen came up with these stories. In my
| adulthood I'm reassured that it happens everywhere and I find
| it endearing.
|
| On the information density of languages: it is true that some
| languages have a more information dense _textual_
| representation. But all spoken languages convey about the same
| information in the same time. Which is not all that surprising,
| it just means that human brains have an optimal range at which
| they process information.
|
| Further reading: Coupe, Christophe, et al. "Different
| Languages, Similar Encoding Efficiency: Comparable Information
| Rates across the Human Communicative Niche." Science Advances.
| https://doi.org/10.1126/sciadv.aaw2594
| antirez wrote:
| Different representations at the _same_ bitrate may have
| features that make one a lot more resilient to errors. This
| thing about Italian, you fill find in any benchmark of vastly
| different AI transcribing models. You can find similar
| results also on the way LLMs mostly trained on English
| generalize usually very well with Italian. All this despite
| Italian accounting for marginal percentage of the training
| set. How do you explain that? I always cringe when people
| refute evidence.
| testdelacc1 wrote:
| Where is this evidence you've cited for your claims?
| mr_tox wrote:
| in the end (our) italian language wasn't optimized by
| engineers, it was refactored by poets
| simonw wrote:
| This demo is _really_ impressive:
| https://huggingface.co/spaces/mistralai/Voxtral-Mini-Realtim...
|
| Don't be confused if it says "no microphone", the moment you
| click the record button it will request browser permission and
| then start working.
|
| I spoke fast and dropped in some jargon and it got it all right -
| I said this and it transcribed it exactly right, WebAssembly
| spelling included:
|
| > Can you tell me about RSS and Atom and the role of CSP headers
| in browser security, especially if you're using WebAssembly?
| Oras wrote:
| Thank you for the link! Their playground in Mistral does not
| have a microphone. it just uploads files, which does not
| demonstrate the speed and accuracy, but the link you shared
| does.
|
| I tried speaking in 2 languages at once, and it picked it up
| correctly. Truly impressive for real-time.
| druskacik wrote:
| According to the announcement blog Le Chat is powered by the
| new model as well: https://chat.mistral.ai/chat
| tekacs wrote:
| Having built with and tried every voice model over the last
| three years, real time and non-real time... this is off the
| charts compared to anything I've seen before.
|
| And open weight too! So grateful for this.
| daemonologist wrote:
| 404 on https://mistralai-voxtral-mini-
| realtime.hf.space/gradio_api/... for me (which shows up in the
| UI as a little red error in the top right).
| jaggederest wrote:
| It can transcribe Eminem's Rap God fast sequence, really,
| really impressive.
| rafram wrote:
| That's almost certainly in the training data, to be fair.
| keeganpoppen wrote:
| what a great test hahah
| pyprism wrote:
| Wow, that's weird. I tried Bengali, but the text transcribed
| into Hindi!I know there are some similar words in these
| languages, but I used pure Bengali that is not similar to
| Hindi.
| derefr wrote:
| Well, on the linked page, it mentions "strong transcription
| performance in 13 languages, including [...] Hindi" but with
| no mention of Bengali. It probably doesn't know a lick of
| Bengali, and is just trying to snap your words into the
| closest language it does know.
| keeganpoppen wrote:
| it must have _some_ exposure to bengali-- just not enough
| for them to advertise it. otherwise it would have a damn
| hard time.
| rafram wrote:
| Not terrible. It missed or mixed up a lot of words when I was
| speaking quickly (and not enunciating very well), but it does
| well with normal-paced speech.
| timhh wrote:
| Yeah it messed up a bit for me too when I didn't enunciate
| well. If I speak clearly it seems to work very well even with
| background noise. Remember Dragon Naturally Speaking? Imagine
| having this back then!
| sheepscreek wrote:
| I've been using AquaVoice for real-time transcription for a
| while now, and it has become a core part of my workflow. It
| gets everything, jargon, capitalization, everything. Now I'm
| looking forward to doing that with 100% local inference!
| carbocation wrote:
| This model was able to transcribe Bad Bunny lyrics over the
| sound of the background music, played casually from my
| speakers. Impressive, to me.
| skykooler wrote:
| Doesn't seem to work for me - tried in both Firefox and
| Chromium and I can see the waveform when I talk but the
| transcription just shows "Awaiting audio input".
| codethief wrote:
| Same here. In Chromium I don't even see the waveform.
| fragmede wrote:
| I had to turn off ad-block to get it to work.
| starkgoose wrote:
| Try disabling CSP for the page
| Barbing wrote:
| Doesn't seem to work in Safari on iOS 26.2, iPhone 17 Pro, just
| about anything extra disabled.
| darkwater wrote:
| It's really nice although I've got a sentence in French when I
| was speaking Italian but I corrected myself in the middle of a
| word.
|
| But I'm definitely going to keep an eye on this for local-only
| TTS for Home Assistant.
| mentalgear wrote:
| Here European Multilingual-Intelligence truly shines!
| colordrops wrote:
| is this demo running fully in the browser?
| simonw wrote:
| No, it's server-side.
|
| Model is around 7.5 GB - once they get above 4 GB running
| them in a browser gets quite difficult I believe.
| satvikpendem wrote:
| Looks like this model doesn't do realtime diarization, what model
| should I use if I want that? So far I've only seen paid models do
| diarization well. I heard about Nvidia NeMo but haven't tried
| that or even where to try it out.
| breisa wrote:
| Not sure if its "realtime" but the recently released VibeVoice-
| ASR from Microsoft does do diarization.
| https://huggingface.co/microsoft/VibeVoice-ASR
| aavci wrote:
| What's the cheapest device specs that this could realistically
| run on?
| kamranjon wrote:
| I haven't quite figured out if the open weights they released
| on huggingface amount to being able to run the (realtime) model
| locally - i hope so though! For the larger model with
| diarization I don't think they open sourced anything.
| IanCal wrote:
| The HF page suggests yes, with vllm.
|
| > We've worked hand-in-hand with the vLLM team to have
| production-grade support for Voxtral Mini 4B Realtime 2602
| with vLLM. Special thanks goes out to Joshua Deng, Yu Luo,
| Chen Zhang, Nick Hill, Nicolo Lucchesi, Roger Wang, and Cyrus
| Leung for the amazing work and help on building a production-
| ready audio streaming and realtime system in vLLM.
|
| https://huggingface.co/mistralai/Voxtral-
| Mini-4B-Realtime-26...
|
| https://docs.vllm.ai/en/latest/serving/openai_compatible_ser.
| ..
| pietz wrote:
| Do we know if this is better than Nvidia Parakeet V3? That has
| been my go-to model locally and it's hard to imagine there's
| something even better.
| tylergetsay wrote:
| I've been using Parakeet V3 locally and totally ancedotaly this
| feels more accurate but slightly slower
| czottmann wrote:
| I liked Parakeet v3 a lot until it started to drop whole
| sentences, willy-nilly.
| cypherpunks01 wrote:
| Yeah, I think the multilingual improvements in V3 caused some
| kind of regression for English - I've noticed large blocks
| occasionally dropped as well, so reverted to v2 for my usage.
| Specifically nvidia/parakeet-tdt-0.6b-v2 vs nvidia/parakeet-
| tdt-0.6b-v3
| whinvik wrote:
| Came here to ask the same question!
| m1el wrote:
| I've been using nemotron ASR with my own ported inference, and
| happy about it:
|
| https://huggingface.co/nvidia/nemotron-speech-streaming-en-0...
|
| https://github.com/m1el/nemotron-asr.cpp
| https://huggingface.co/m1el/nemotron-speech-streaming-0.6B-g...
| Multicomp wrote:
| I'm so amazed to find out just how close we are to the start
| trek voice computer.
|
| I used to use Dragon Dictation to draft my first novel, had
| to learn a 'language' to tell the rudimentary engine how to
| recognize my speech.
|
| And then I discovered [1] and have been using it for some
| basic speech recognition, amazed at what a local model can
| do.
|
| But it can't transcribe any text until I finish recording a
| file, and then it starts work, so very slow batches in terms
| of feedback latency cycles.
|
| And now you've posted this cool solution which streams audio
| chunks to a model in infinite small pieces, amazing, just
| amazing.
|
| Now if only I can figure out how to contribute to Handy or
| similar to do that Speech To Text in a streaming mode, STT
| locally will be a solved problem for me.
|
| [1] https://github.com/cjpais/Handy
| m1el wrote:
| you should check out
|
| https://github.com/pipecat-ai/nemotron-january-2026/
|
| discovered through this twitter post:
|
| https://x.com/kwindla/status/2008601717987045382
| kwindla wrote:
| Happy to answer questions about this (or work with people
| on further optimizing the open source inference code
| here). NVIDIA has more inference tooling coming, but it's
| also fun to hack on the PyTorch/etc stuff they've
| released so far.
| moffkalast wrote:
| Parakeet is really good imo too, and it's just 0.6B so it can
| actually run on edge devices. 4B is massive, I don't see
| Voxtral running realtime on an Orin or fitting on a Hailo. An
| Orin Nano probably can't even load it at BF16.
| boringg wrote:
| Pseudo related -- am I the only one uncomfortable using my voice
| with AI for the concern that once it is in the training model it
| is forever reproducible? As a non-public person it seems like a
| risk vector (albeit small),
| ffsm8 wrote:
| It's a real issue, but why do you only see it in ai? It's true
| for any case where you're speaking into a microphone
|
| Depending on the permissions granted to apps on your mobile
| device, it can even be passively exfiltrated without you ever
| noticing - and that's ignoring the video clips people take and
| put online. Like your grandma uploading to Facebook a short
| moment from a Christmas meet or similar
|
| There have already been successful scams - eg calls from
| "relatives" (AI) calling family members needing money urgently
| and convincing them to send the money...
| dumpstate wrote:
| I'm on voxtral-mini-latest and that's why I started seeing 500s
| today lol
| janalsncm wrote:
| I noticed that this model is multilingual and understands 14
| languages. For many use cases, we probably only need a single
| language, and the extra 13 are simply adding extra latency. I
| believe there will be a trend in the coming years of trimming the
| fat off of these jack of all trades models.
|
| https://aclanthology.org/2025.findings-acl.87/
| decide1000 wrote:
| I think this model proves it's very efficient and accurate.
| ethmarks wrote:
| But it could potentially be even _more_ efficient if it was
| single-language.
| popalchemist wrote:
| It doesn't make sense to have a language-restricted
| transcription model because of code switching. People aren't
| machines, we don't stick to our native languages without
| failure. Even monolingual people move in and out of their
| native language when using "borrowed" words/phrases. A single-
| language model will often fail to deal with that.
| javier123454321 wrote:
| yeah, one example I run into is getting my perplexity phone
| assistant to play a song in spanish. I cannot for the life of
| me get a model to translate: "Play senorita a mi me gusta su
| style on spotify" correctly
| keeganpoppen wrote:
| uhhh i cast doubt on multi-language support as affecting
| latency. model size, maybe, but what is the mechanism for
| making latency worse? i think of model latency as O(log(model
| size))... but i am open to being wrong / that being a not-good
| mental model / educated guess.
| make3 wrote:
| model size directly affects latency
| kergonath wrote:
| Even model size, it's modest. There is a lot of machinery
| that is going to be common for all languages. You don't
| multiply model size by 2 when you double the number of
| supported languages.
| ethmarks wrote:
| If encoding more learned languages and grammars and
| dictionaries makes the model size bigger, it will also
| increase latency. Try running a 1B model locally and then try
| to run a 500B model on the same hardware. You'll notice that
| latency has rather a lot to do with model size.
| raincole wrote:
| Imagine if ChatGPT started like this and thought they should
| trim coding abilities from their language model because most
| people don't code.
| ethmarks wrote:
| They've already done the inverse and trimmed non-coding
| abilities from their language model:
| https://openai.com/index/introducing-gpt-5-2-codex/. There's
| already precedent for creating domain-specific models.
|
| I think it's nice to have specialized models for specific
| tasks that don't try to be generalists. Voxtral Transcript 2
| is already extremely impressive, so imagine how much better
| it could be if it specialized in specific languages rather
| than cramming 14 languages into one model.
|
| That said, generalist models definitely have their uses. I
| _do_ want multilingual transcribing models to exist, I just
| also think that monolingual models could potentially achieve
| even better results for that specific language.
| depr wrote:
| STT services that have been around for longer, like Azure,
| Google and Amazon, generally require you to request a specific
| language, and their quality is a lot higher than models that
| advertise themselves as LLMs (even though I believe the clouds
| are also using the same types of models now).
| idiotsecant wrote:
| The hilarious part of this comment is all the comments around
| it complaining about not supporting _enough_ languages
| m463 wrote:
| I don't know. What about words inherited from other languages?
| I think a cross-language model could improve lots of things.
|
| For example, "here it is, voila!" "turn left on el camino real"
| XCSme wrote:
| Is it me or error rate of 3% is really high?
|
| If you transcribe a minute of conversation, you'll have like 5
| words transcribed wrongly. In an hour podcast, that is 300
| wrongly transcribed words.
| cootsnuck wrote:
| The error rate for human transcription can be as high as 5%.
| XCSme wrote:
| Oh wow, I thought humans are like 0.1% error rate, if they
| are native speakers and aware of the subject being discussed.
| zipy124 wrote:
| I was skepitcal upon hearing the figure but various sources
| do indeed back it up and [0] is a pretty interesting paper
| (old but still relevant human transcibers haven't changed
| in accuracy).
|
| [0] https://www.microsoft.com/en-us/research/wp-
| content/uploads/...
| XCSme wrote:
| I think it's actually hard to verify how correct a
| transcription is, at scale. Curious where those error
| rate numbers come from, because they should test it on
| people actually doing their job.
| rhdunn wrote:
| It can depend a lot on different factors like:
|
| - familiarity with the accent and/or speaker;
|
| - speed and style/cadence of the speech;
|
| - any other audio that is happening that can muffle or
| distort the audio;
|
| - etc.
|
| It can also take multiple passes to get a decent
| transcription.
| Nimitz14 wrote:
| Most of these errors will not be meaningful. Real speech is
| full of ambiguities. 3% is low
| siddbudd wrote:
| Wired advertises this as "Ultra-Fast Translation"[^1]. A bit
| weird coming from a tech magazine. I hope it's just a "typo".
|
| [^1]: https://www.wired.com/story/mistral-voxtral-real-time-ai-
| tra...
| bigyabai wrote:
| It might be capable of translation; OpenAI Whisper was a
| transcription model that could do it.
| ewuhic wrote:
| Can it translate in real time?
| unstatusthequo wrote:
| Also curious about this. Just need real time German to English.
| What does this?
| yewenjie wrote:
| One week ago I was on the hunt for an open source model that can
| do diatization and I had to literally give up because I could not
| find any easy to use setup.
| vojto11 wrote:
| WhisperX ?
| ashenke wrote:
| I don't know if that will change, but right now only the
| Voxtral Mini Transcribe V2 supports diarization and it's not
| open-weight. The Voxtral Realtime model doesn't support
| diarization, but is open-weight.
| derac wrote:
| Any chance Voxtral Mini Transcribe 2 will ever be an open model?
| jszymborski wrote:
| I'm guessing I won't be able to finetune this until they come out
| with a HF tranformers model, right?
| gwerbret wrote:
| I really wish those offering speech-to-text models provided
| transcription benchmarks specific to particular fields of
| endeavor. I imagine performance would vary wildly when using
| jargon peculiar to software development, medicine, physics, and
| law, as compared to everyday speech. Considering that
| "enterprise" use is often specialized or sub-specialized, it
| seems like they're leaving money on Dragon's table by not
| catering to any of those needs.
| iagooar wrote:
| In English it is pretty good. But talk to it in Polish, and
| suddenly it thinks you speak Russian? Ukranian? Belarus? I would
| understand if an American company launched this, but for a
| company being so proud about their European roots, I think it
| should have better support for major European languages.
|
| I tried English + Polish:
|
| > All right, I'm not really sure if transcribing this makes a lot
| of sense. Maybe not. A ts'omu nie mowisz po polsku. A ts'omu nie
| mowisz po polsku, nie po ukransku.
| mystifyingpoi wrote:
| TBH ChatGPT does the same, when I mix Polish and English.
| Generally getting some cyrillic characters and it gets super
| confused.
| tdb7893 wrote:
| Yeah, it's too bad. Apparently it only performs well in certain
| languages: "The model is natively multilingual, achieving
| strong transcription performance in 13 languages, including
| English, Chinese, Hindi, Spanish, Arabic, French, Portuguese,
| Russian, German, Japanese, Korean, Italian, and Dutch"
| ricardonunez wrote:
| It did great English and Spanish, it didn't switch to
| Portuguese, french nor German, maybe struggle with my accent.
| scotty79 wrote:
| Try to warn it you are going to switch language to
| Portugese. Worked for me.
| yko wrote:
| That's a mix of Polish and Ukrainian in the transcript. Now, if
| I try speaking Ukrainian, I'm getting transcript in Russian
| every time. That's upsetting.
| overfeed wrote:
| Oh no! The model won't translate to an unsupported language,
| and incorrectly reverts to one that it was explicitly trained
| on.
|
| The base likely was pretrained on days that included Polish
| and Ukrainian. You shouldn't be surprised to learn it doesn't
| perform great on languages it wasn't trained on, or perhaps
| had the highest share of training data.
| scotty79 wrote:
| Tell it you are going to speak Polish now. It helps.
| lm28469 wrote:
| > The model is natively multilingual, achieving strong
| transcription performance in 13 languages, including English,
| Chinese, Hindi, Spanish, Arabic, French, Portuguese, Russian,
| German, Japanese, Korean, Italian, and Dutch.
|
| Try sticking to the supported languages
| loire280 wrote:
| They don't claim to support Polish, but they do support
| Russian.
|
| > The model is natively multilingual, achieving strong
| transcription performance in 13 languages, including English,
| Chinese, Hindi, Spanish, Arabic, French, Portuguese, Russian,
| German, Japanese, Korean, Italian, and Dutch. With a 4B
| parameter footprint, it runs efficiently on edge devices,
| ensuring privacy and security for sensitive deployments.
|
| I wonder how much having languages with the same roots (e.g.
| the romance languages in the list above or multiple Slavic
| languages) affects the parameter count and the training set. Do
| you need more training data to differentiate between multiple
| similar languages? How would swapping, for example, Hindi
| (fairly distinct from the other 12 supported languages) for
| Ukrainian and Polish (both share some roots with Russian)
| affect the parameter count?
| MarcelOlsz wrote:
| Nobody ever supports Polish. It's the worst. They'll support
| like, Swahili, but not Polish.
| londons_explore wrote:
| 200 million people speak Swahili.
|
| 39 million people speak Polish, and most of those also
| speak English or another more common language.
| timhh wrote:
| You could say the same about Dutch to be fair. 90-95%
| speak English - I bet that's way higher than in Poland.
| chickenimprint wrote:
| Swahili is subcontinental lingua franca spoken by 200M
| people and growing quickly. Polish is spoken by a shrinking
| population in one country where English is understood
| anyways.
| moffkalast wrote:
| I'm not sure why but their multilingual performance in general
| has usually been below average. For a French company, their
| models are not even close to being best in French, even outdone
| by the likes of Qwen. I don't think they're focusing on
| anything but English, the rest is just marketing.
| jiehong wrote:
| It's nice, but the previous version wasn't actually that great
| compared to Parakeet for example.
|
| We need better independent comparison to see how it performs
| against the latest Qwen3-ASR, and so on.
|
| I can no longer take at face value the cherry picked comparisons
| of the companies showing off their new models.
|
| For now, NVIDIA Parakeet v3 is the best for my use case, and runs
| very fast on my laptop or my phone.
| nodja wrote:
| There is https://huggingface.co/spaces/hf-
| audio/open_asr_leaderboard but it hasn't been updated for half
| a year.
| archb wrote:
| I like Parakeet as well and use it via Handy on Mac. What app
| are you using on your phone?
| jiehong wrote:
| Spokenly has it on Mac and iOS, in both cases for free when
| using parakeet
| fph wrote:
| Is there an open source Android keyboard that would support it?
| Everything I find is based on Whisper, which is from 2022. Ages
| ago given how fast AI is evolving.
| antirez wrote:
| I wish I had a Google Keyboard that could easily run on Whisper
| Medium. This is already great. But unfortunately would be too
| much inference cost, incredibly slow. The problem with Whisper
| is not the inference quality: medium and large are incredible.
| Is that the base model is not enough, and the only one with
| fast inference in mobile devices.
| blobinabottle wrote:
| Impressive results, tested on crappy audio files (in french and
| english)...
| yko wrote:
| Played with the demo a bit. It's really good at English, and
| detects language change on the fly. Impressive.
|
| But whatever I tried, it could not recognise my Ukrainian and
| would default to Russian in absolutely ridiculous transcription.
| Other STT models recognise Ukrainian consistently, so I assume
| there is a lot of Russian in training material, and zero
| Ukrainian. Made me really sad.
| breisa wrote:
| Thats just the result of the model only supporting russian (and
| 12 other languages) and not urkainian. It maps to the closest
| words from training data.
| numbers wrote:
| does anyone know if there's any desktop tools I can use this
| transcription model with? e.g. something where like Wisper
| Flow/WillowVoice but with custom model selection
| tietjens wrote:
| There is Handy, an open source project meant to be a desktop
| tool, but I haven't installed it yet to see how you pick your
| model.
|
| Handy - Free open source speech-to-text app
| https://github.com/cjpais/Handy
| tallesborges92 wrote:
| I added it to my bot agent,let's see how it performs
| atentaten wrote:
| Nice. Can this be ran on a mobile device?
| ccleve wrote:
| This looks great, but it's not clear to me how to use it for a
| practical task. I need to transcribe about 10 years worth of
| monthly meetings. These are government hearings with a variety of
| speakers. All the videos are on YouTube. What's the most
| practical and cost-effective way to get reasonably accurate
| transcripts?
| jimmy76615 wrote:
| If they are on Youtube, try Gemini 3 Flash first. Use AI
| studio, it lets you insert YouTube videos into context.
| IanCal wrote:
| If you use something like youtube-dlp you can download the
| audio from the meetings, and you could try things out in
| mistrals ai studio.
|
| You could use their api (they have this snippet):
|
| ```curl -X POST
| "https://api.mistral.ai/v1/audio/transcriptions" \ -H
| "Authorization: Bearer $MISTRAL_API_KEY" \ -F model="voxtral-
| mini-latest" \ -F file=@"your-file.m4a" \ -F diarize=true \ -F
| timestamp_granularities="segment"```
|
| In the api it took 18s to do a 20m audio file I had lying
| around where someone is reviewing a product.
|
| There will, I'm sure, be ways of running this locally up and
| available soon (if they aren't in huggingface right now) but
| the API is $0.003/min. If it's something like 120 meetings (10
| years of monthly ones) then it's roughly $20 if the meetings
| are 1hr each. Depending on whether they're 1 or 10 hours (or if
| they're weekly or monthly but 10 parallel sessions or
| something) then this might be a price you're willing to pay if
| you get the results back in an afternoon.
|
| edit - their realtime model can be run with vllm, the batch
| model is not open
| isoprophlex wrote:
| - get an API key for this service
|
| - make sure you have a list of all these YouTube meeting URLs
| somewhere
|
| - ask your preferred coding assistant to write you up a script
| that downloads the audio for these videos with yt-dlp & calls
| Mixtrals' API
|
| - ????
|
| - profit
| scotty79 wrote:
| Do you know anything better for Polish language, low quality
| audio than Whisper large-v3 through WhisperX?
|
| This combo has almost unbeatable accuracy and it rejects noises
| in the background really well. It can even reject people talking
| in the background.
|
| The only better thing I've seen is Ursa model from Speechmatics.
| Not open weights unfortunately.
| sgt wrote:
| What's the best way to train this further on a specific dialect
| or accent or even terminology?
| mnbbrown wrote:
| Incroyable! Competitive (if not better) than deepgram nova-3, and
| much better than assembly and elevenlabs in basically all cases
| on our internal streaming benchmarking.
|
| The dataset is ~100 8kHz call recordings with gnarly UK accents
| (which I consider to be the final boss of english language ASR).
| It seems like it's SOTA.
|
| Where it does fall down seems to be the latency distribution but
| I'm testing against the API. Running it locally will no doubt
| improve that?
| antirez wrote:
| Disappointing how this lacks a clear reference implementation, if
| not mixed at almost yet unreleased VLLM (nightly version) stuff.
| I'm ok with Open Weights being a form of OSS in the case of
| models, because frankly I don't believe that, for large LLMs, it
| is feasible to release the training data, all the orchestration
| stuff, and so forth. But it can't be: here are the weights, we
| partnered with VLLM for inference. Come on. Open Weights _must_
| mean that you put me in a situation to write an implementation
| easily for any hardware.
|
| p.s. even the demo uses a remote server via websocket.
| sbinnee wrote:
| 3 hours for a single request sounds nice to me. Although the
| graph suggests that it's not going to perform as good as openai
| model I have been using, it is open source and surely I will give
| it a try.
| maxdo wrote:
| https://www.tavus.io/post/sparrow-1-human-level-conversation...
|
| how does it compare to sparrow-1?
___________________________________________________________________
(page generated 2026-02-04 23:00 UTC)