[HN Gopher] Opus 1.5 released: Opus gets a machine learning upgrade
___________________________________________________________________
Opus 1.5 released: Opus gets a machine learning upgrade
Author : summm
Score : 228 points
Date : 2024-03-04 17:36 UTC (5 hours ago)
(HTM) web link (opus-codec.org)
(TXT) w3m dump (opus-codec.org)
| travisporter wrote:
| Very cool. seems like they addressed the problem of
| hallucination. would be interesting to see an example of it
| hallucinating without redundancy and corrected with redundancy
| CharlesW wrote:
| Isn't packet loss concealment (PLC) a form of hallucination?
| Not saying it's bad, just that it's still Making Shit Up(tm) in
| a statistically-credible way.
| derf_ wrote:
| The PLC intentionally fades off after around 100 ms so as not
| to cause misleading hallucinations. It is really just about
| filling small gaps.
| skybrian wrote:
| In a broader context, though, this happens all the time.
| You'd be surprised what people mishear in noisy conditions.
| (Or if they're hard of hearing.) The only thing for it is to
| ask them to repeat back what they heard, when it matters.
|
| It might be an interesting test to compare what people
| mishear with and without this kind of compensation.
| jmvalin wrote:
| As part of the packet loss challenge, there was an ASR word
| accuracy evaluation to see how PLC impacted
| intelligibility. See https://www.microsoft.com/en-
| us/research/academic-program/au...
|
| The good news is that we were able to improve
| intelligibility slightly compared with filling with zeros
| (it's also a lot less annoying to listen to). The bad news
| is that you can only do so much with PLC, which is why we
| then pursued the Deep Redundancy (DRED) idea.
| tialaramex wrote:
| Right, this is why the Proper radio calls for a lot of
| systems have mandatory read back steps, so that we're sure
| two humans have achieved a shared understanding regardless
| of how sure they are of what they heard. It not only
| matters whether you heard correctly, it also matters
| whether you _understood_ correctly.
|
| e.g. train driver asks for an "Up Fast" block. His train is
| sat on Down Fast, the Up Fast is adjacent, so then he can
| walk on the (now safe) railway track and inspect his train
| at track level, which is exactly what he, knowing the fault
| he's investigating, was taught to do.
|
| Signaller hears "Up Fast" but thinks duh, stupid train
| driver forgot he's on Down Fast. He doesn't need a block,
| the signalling system knows the train is in the way and
| won't let the signaller route trains on that section. So
| the Up Fast line isn't made safe.
|
| If they leave the call here, both think they've achieved
| understanding but actually there is no shared understanding
| and that's a safety critical mistake.
|
| If they follow a read-back procedure they discover the
| mistake. "So I have my Up Fast block?" "You're stopped on
| Down Fast, you don't need an Up Fast block". "I know that,
| I need Up Fast. I want to walk along the track!" "Oh! I see
| now, I am filling out the paperwork for you to take Up
| Fast". Both humans now understand what's going on
| correctly.
| a_wild_dandan wrote:
| To borrow from Joscha Bach: if you _like_ the output, it 's
| called creativity. If you don't, it's called a hallucination.
| CharlesW wrote:
| I love that, what's it from? (My Google-fu failed.)
| Unexpected responses are often a joy when using AI in a
| creative context. https://www.cell.com/trends/neurosciences
| /abstract/S0166-223...
| Aachen wrote:
| That sounds funny, but is it true? Certainly there's a bias
| that goes towards what you're quoting, but would you
| otherwise genuinely call the computer _creative_? Is that a
| positive aspect of a speech codec or of an information
| source?
|
| Creative is when you ask a neural net to create a poem, or
| something else from "scratch" (meant to be unique).
| Hallucination is when you _didn 't_ ask it to make its
| answer up but to recite or rephrase things it has directly
| observed
|
| That's my layman's understanding anyway, let me know if you
| agree
| skybrian wrote:
| That's almost the same. You could say it's being creative
| by not following directions.
|
| Creativity isn't well-defined. If you generate things at
| random, they are all unique. If you then filter them to
| remove all the bad output, the result could be just as
| "creative" as anything someone could write. (In
| principle. In practice, it's not that easy.)
|
| And that's how evolution works. Many organisms have very
| "creative" designs. Filtering at scale, over a long
| enough period of time, is very powerful.
|
| Generative models are sort of like that in that they
| often use a random number generator as input, and they
| could generate thousands of possible outputs. So it's not
| clear why this couldn't be just as creative as anything
| else, in principle.
|
| The filtering step is often not that good, though.
| Sometimes it's done manually, and we call that cherry-
| picking.
| jmvalin wrote:
| Well, there's different ways to make things up. We decided
| against using a pure generative model to avoid making up
| phoneme or words. Instead, we predict the expected acoustic
| features (using a regression loss), which means that model is
| able to continue a vowel. If unsure it'll just pick the
| "middle point", which won't be something recognizable as a
| new word. That's in line with how traditional PLCs work. It
| just sounds better. The only generative part is the vocoder
| that reconstructs the waveform, but it's constrained to match
| the predicted spectrum so it can't hallucinate either.
| CharlesW wrote:
| That's really cool. Congratulations on the release!
| stevage wrote:
| Any demos of this to listen to? It sounds potentially
| really good.
| GaggiX wrote:
| There is a demo in the link shared by OP.
| out_of_protocol wrote:
| Why the hell opus still not in Bluetooth? Well i know - sweet
| sweet license fees
|
| (aKKtually, there IS opus codec, supported by pixel phones -
| google made it for VR/AR stuff. No one uses it, there are about
| ~1 headphone with opus support )
| lxgr wrote:
| As you already mention, it's already possible to use it. As for
| why hardware manufacturers don't actually use it, you can thank
| beautiful initiatives such as this: https://www.opuspool.com/
| (previous HN discussion:
| https://news.ycombinator.com/item?id=33158475).
| giantrobot wrote:
| The BT SIG moves kind of slow and there's a really long tail of
| devices. Until there's a chip with native Opus support (that's
| as cheap as ones with AAC etc) you wouldn't get Opus support
| even if it was in the spec.
|
| Realistically for most headphones people actually buy AAC (LC
| and HE) is more than good enough encoding quality for the audio
| the headphones can produce. Even if Opus was in the spec and
| Opus-supporting chips were common there would still be a
| hojillion Bluetooth devices in the wild that wouldn't support
| it.
|
| It would be cool to have Opus in A2DP but it would take a BT
| SIG member that was really in love with it to get it in the
| profile.
| out_of_protocol wrote:
| They chose to make totally new inferior LC3 codec though.
|
| Also, on my system (Android phone + BTR5/BTR15 Bluetooth DAC
| + Sennheiser H600) all options sound realy crappy compared to
| plain old usb, everything else is the same. LDAC 990kbps is
| less crappy, by sheer brute force. I suspect it's not only
| codec but other co-factors as well (like mandatory DSP on
| phone side)
| giantrobot wrote:
| I've got AirPods and a Beats headset so they both support
| AAC and to my ear sound great. Keep in mind I went to a lot
| of concerts in my 20s without earplugs so my hearing isn't
| necessarily the greatest anymore.
|
| AFAIK Android's AAC quality isn't that great so aptX and
| LDAC are the only real high quality options for Android and
| headphones. It's a shame as a lot of streaming is actually
| AAC bitstreams and _can_ be passed directly through to
| headphones with no intermediate lossy re-encode.
|
| Like I said though, to get Opus support in A2DP a BT SIG
| member would really have to be in love with it. Qualcomm
| and Sony have put forward aptX and LDAC respectively in
| order to get licensing money on decoders. Since no one is
| going to get Opus royalties there's not much incentive for
| anyone to push for its inclusion in A2DP.
| dogma1138 wrote:
| Opus isn't patent free, and what's worse it's not particularly
| clear who owns what. The biggest patent pool is currently
| OpusPool but it's not the only one.
|
| https://www.opuspool.com/
| pgeorgi wrote:
| No codec (or any other technical development, really - edit:
| except for 20+ years old stuff, and only if you don't add
| any, even "obvious" improvements) is known patent free, or
| clear on "who owns what."
|
| Folks set up pools all the time, but somehow they never offer
| indemnification for completeness of the pool - because they
| can't.
|
| See https://en.kangxin.com/html/2/218/219/220/11565.html for
| a few examples how the patent pool extortion scheme already
| went wrong in the past.
| dogma1138 wrote:
| No one said that Opus is the only one suffering from
| licensing ambiguity, but comparing it to say AptX and its
| variants which do have a clear one stop shop for licensing
| (Qualcomm) it's a much riskier venture especially when it
| comes to hardware.
| tux3 wrote:
| Making a patent troll is just a matter of putting up a
| press release and a web page.
|
| I could claim to have a long list of patents against
| AptX. Anyone could.
|
| Of course I'm not willing to disclose the list of patents
| at this time, but customers looking to be extorted may
| contact me privately.
| pgeorgi wrote:
| A drive-by patent owner can show up on anything, and if
| they don't want to license to you, your entire product is
| bust.
|
| Even if it's AptX and Qualcomm issues you a license in
| exchange for money. I wouldn't even bet on being able to
| claw back these license costs after being ordered to
| destroy your AptX-equipped product after it ran into
| somebody else's patent.
|
| The risk that this happens is _exactly_ the same for Opus
| or AptX.
| xoa wrote:
| FWIW, submarine patents are long dead, so it is possible to
| feel assured that old enough stuff is patent free. Of
| course that denies a lot of important improvements, but due
| to diminishing returns and the ramp of tech development
| it's still ever more significant. A lot of key stuff is
| going to lose monopoly lock this decade.
| pgeorgi wrote:
| You're right. I could still amend the post, so I added
| the 20+ years caveat. Thanks!
| rockdoe wrote:
| _Opus isn't patent free_
|
| The existence of a patent pool does not mean there are valid
| patent claims against it. But yes, you may be technically
| correct by saying "patent free" rather than "not infringing
| on any _valid_ patents ". That said historically Opus has had
| claims against it by patents that looked valid but upon
| closer investigation didn't actually cover what the codec
| does.
|
| Just looks like FUD to me. Meanwhile, the patent pools of
| competing technologies definitely still don't offer
| indemnification they cover all patents, but have no problem
| paying a bunch of people to spew exactly this kind of FUD -
| they're the ones who tried to set up this "patent pool" to
| begin with!
| mikae1 wrote:
| They'll have my upvote just for writing ML instead AI. Seriously,
| this is very exciting developments for audio compression.
| claudiojulio wrote:
| Machine Learning is Artificial Intelligence. Just look at
| Wikipedia:
| https://en.wikipedia.org/wiki/Artificial_intelligence
| declaredapple wrote:
| Many people are annoyed by the recent influx of calling
| everything "AI".
|
| Machine learning, statistical models, procedural generation,
| literally an usage of heuristics are all being called "AI"
| nowadays which obfuscates the "boring" nature in favor of
| "exciting buzzword"
|
| Selecting the quality of a video based on your download
| speed? That's "AI" now.
| sitzkrieg wrote:
| im quite tired of this. every snake oil shop now calls any
| algorithm "a i" to sound hip and sophisticated
| mikae1 wrote:
| _> Many people are annoyed by the recent influx of calling
| everything "AI"._
|
| Yes, that was the reason for my comment. :)
| mook wrote:
| On the other hand, it means that you can assume anything
| mentioning AI is overhyped and probably isn't as great as
| they claim. That can be slightly useful at times.
| wilg wrote:
| This is something you really shouldn't spend any cycles
| worrying about.
| sergiotapia wrote:
| I'd just like to interject for a moment. What you're
| referring to as AI, is in fact, Machine Learning, or as I've
| recently taken to calling it, Machine Learning plus
| Traditional AI methods.
| wilg wrote:
| My point is very clearly that you should not spend any time
| or energy thinking about about the terminology.
| sergiotapia wrote:
| I know lol this a famous quote by ganoo loonix enthusiast
| Richard Stallman.
| behnamoh wrote:
| Isn't it a strange coincidence that this shows up on HN while
| Claude Opus is also announced today and is on HN front page? I
| mean, what are the odds of seeing the word "Opus" twice in a day
| on one internet page?
| declaredapple wrote:
| Well it was released today
|
| Very likely a coincidence.
|
| https://opus-codec.org/release/stable/2024/03/04/libopus-1_5...
| mattnewton wrote:
| Not that strange when you consider what "opus" means- product
| of work, with the connotation of being large and artistically
| important. It's Latin, so it's friendly phonemes to speakers of
| Romance languages and very scientific-and-important-sounding to
| English speaking ears. Basically the most generic name you can
| give your fine "work" in the western world.
| behnamoh wrote:
| Thanks for the definition. I like the word! I just haven't
| come across it in a long time, and seeing it twice on HN
| frontpage is bizarre!
| stevage wrote:
| It's funny, I was expecting the article to be about the
| Opus music font and was trying to figure out how ML could
| be involved.
| p1esk wrote:
| Two inrelated "Opus" releases today, and both use ML. The other
| one is a new model from Anthropic.
| spacechild1 wrote:
| I'm using Opus as one of the main codecs in my peer-to-peer audio
| streaming library (https://git.iem.at/cm/aoo/ - still alpha), so
| this is very exciting news!
|
| I'll definitely play around with these new ML features!
| luplex wrote:
| I wonder: did they address common ML ethics questions?
| Specifically: Are the ML algorithms better/worse on male than on
| female speech? How about different languages or dialects? Are
| they specifically tuned for speech at all, or do they also work
| well for music or birdsong?
|
| That said, the examples are impressive and I can't wait for this
| level of understandability to become standard in my calls.
| radarsat1 wrote:
| This is an important question. However, I'd like to point out
| that similar biases can easily exist for non-ML, hand-tuned
| algorithms. Even in the latter case test sets and often even
| "training" and "validation" sets are used for finding good
| parameters. Any of these can be a source of bias, as can the
| ears of evaluators making these decisions.
|
| It's true that bias questions often come up in ML context
| because fundamentally these algorithms do not work without
| data, but _all_ algorithms are designed by people, and _many_
| can involve data in setting their parameters. Both of which can
| be sources of bias. ML is more known for it, I believe, because
| the _inductive_ biases are less than in traditional algorithms,
| and therefore are more keen to adopt biases present in the
| dataset.
| thomastjeffery wrote:
| Usually regular algorithms aren't _generating_ data that
| pretends to be raw data. That 's the significant difference
| here.
| jmvalin wrote:
| Quoting from our paper, training was done using "205 hours of
| 16-kHz speech from a combination of TTS datasets including more
| than 900 speakers in 34 languages and dialects". Mostly tested
| with English, but part of the idea of releasing early (none of
| that is standardized) is for people to try it out and report
| any issues.
|
| There's about equal male and female speakers, though codecs
| always have slight perceptual quality biases (in either
| direction) that depend on the pitch. Oh, and everything here is
| speech only.
| unixhero wrote:
| Why is the ethics question important? It is a new feature for
| an audio codec, not a new material to teach in your kids
| curriculum.
| The_Colonel wrote:
| Imagine you release a codec which optimizes for cis white
| male voice, every other kind of voice has perceptibly lower
| fidelity (at low bitrates). That would not go well...
| shrubble wrote:
| Are you able to determine between gay and straight men just
| by listening to them? I was not aware of that...
| p1esk wrote:
| An ML model might be able to, even if you can't.
| GaggiX wrote:
| Achieving the gaydar, do not give this technology to
| Saudi Arabia.
| Edman274 wrote:
| What people have historically called a "gay lisp" is
| actually a hyper-articulation of /s/ or /z/, and as you
| might expect, /s/ and /z/ have a lot of high frequency
| sounds in them. Weird as it sounds there is a possible
| scenario where an audio codec does a worse job
| reproducing the audio content of gay male speech compared
| to straight male speech.
| stevage wrote:
| https://en.wikipedia.org/wiki/Gay_male_speech?wprov=sfla1
|
| HTH.
| panzi wrote:
| Yeah, imagine a low bitrate situation where only English
| speaking men can still communicate. That would create quite
| a power imbalance.
| overstay8930 wrote:
| Meanwhile G.711 makes all dudes sound like disgruntled
| middle aged union workers
| unethical_ban wrote:
| I get your point, but the questioner wasn't being rude or
| angry, only curious. I think it's a valid question, too.
| While it isn't as important to be neutral in this instance
| as, say, a crime prediction model or a hiring model, it
| should be boilerplate to consider ML inputs for identity
| neutrality.
| gcr wrote:
| This is a great question! Here's a related failure case that
| I think illustrates the issue.
|
| In my country, public restroom facilities replaced all the
| buttons and levers on faucets, towel dispensers, etc. with
| sensors that detect your hand under the faucet. Black people
| tell me they aren't able to easily use these restrooms. I was
| surprised when I heard this, but if you google this, it's
| apparently a thing.
|
| Why does this happen? After all, the companies that made
| these products aren't obviously biased against black people
| (outwardly, anyway). So this sort of mistake must be easy to
| fall into, even for smart teams in good companies.
|
| The answer ultimately boils down to ignorance. When we make
| hand detector sensors for faucets, we typically calibrate
| them with white people in mind. Of course different skin
| tones have different albedo and different reflectance
| properties, so sensors are less likely to fire. Some black
| folks have a workaround where they hold a (white) napkin in
| their hand to get the faucet to work.
|
| How do we prevent this particular case from happening in the
| products we build? One approach is to ensure that the
| development teams for skin sensors have a wide variety of
| skin types. If the product development team had a black guy
| for example, he could say "hey, this doesn't work with my
| skin, we need to tune the threshold." Another approach is to
| ensure that different skin types are reflected in the data
| used to fit the skin statistical models we use. Today's push
| for "ethics in ML" is borne out of this second path as a
| direct desire to avoid these sorts of problems.
|
| I like this handwashing example because it's immediately
| apparent to everyone. You don't have to "prioritize DEI
| programs" to understand the importance of making sure your
| skin detector works for all skin types. But, teams that
| _already_ prioritize accessibility, user diversity, etc. are
| less likely to fall into these traps when conducting their
| ordinary business.
|
| For this audio codec, I could imagine that voices outside the
| "standard English dialect" (e.g. thick accents, different
| voices) might take more bytes to encode the same signal. That
| would raise bandwidth requirements, worsen latency, and
| increase data costs for these users. If the codec is designed
| for a standard American audience, that's less of an issue,
| but codecs work best when they fit reasonably well for all
| kinds of human physiology.
| cma wrote:
| What if it is a pareto improvement: better improvement for
| some dialects but no worse than the earlier version for
| anyone. Should it be shelved or tuned down so all
| improvement for each dialect see gains by an exactly equal
| percentage?
| yalok wrote:
| The main limitation for such codecs is CPU/battery life - and I
| like how they sparsely applied ML in it here and there, combining
| it with classic approach (non-ML algos) to achieve better
| tradeoff of CPU vs quality. E.g. for better low bitrate
| support/LACE - "we went for a different approach: start with the
| tried-and-true postfilter idea and sprinkle just enough DNN magic
| on top of it." The key was not to feed raw audio samples to the
| NN - "The audio itself never goes through the DNN. The result is
| a small and very-low-complexity model (by DNN standards) that can
| run even on older phones."
|
| Looks like the right direction for embedded algos and it seems to
| be a pretty unexplored one, as compared to the current fashion to
| do ML E2E.
| rhdunn wrote:
| I find the interplay between audio codecs, speech synthesis, and
| speech recognition fascinating. Advancements in one usually
| results in advancements in the others.
| brcmthrowaway wrote:
| This is game changing. When will H265 get a DL upgrade?
| h4x0rr wrote:
| Does this new Opus version close the gap to xHE-AAC, which is
| (was?) superior at lower bitrates?
| brnt wrote:
| What if there was a profiler or setting that helps to reencode
| existing lossy formats without introducing too many more
| artifacts? An sizeable collection runs into the issue, if the
| don't have (easily accessible) lossless masters.
|
| I'd be very interested if I could move a variety of mp3s, aacs
| and vorbis to Opus if I knew additional quality loss was minimal.
| Dwedit wrote:
| I just want to mention that getting such good speech quality at
| 9kbps by using NoLACE is absolutely insane.
___________________________________________________________________
(page generated 2024-03-04 23:00 UTC)