[HN Gopher] AI could end foreign-language subtitles
___________________________________________________________________
AI could end foreign-language subtitles
Author : inetsee
Score : 22 points
Date : 2022-01-19 22:11 UTC (51 minutes ago)
(HTM) web link (www.axios.com)
(TXT) w3m dump (www.axios.com)
| dmitriid wrote:
| This will not happen for many, many years to come.
|
| That is, companies will try it, there will be some bad and
| ridiculous results, there will be pusheback and the tech will
| largely be ignored for a decade or so.
|
| Main problems will be intonation and inflection. What is a
| question to an English speaker is a regular sentence to a
| Russian. And so on. "Petabytes of data from primarily English-
| speaking libraries" mean jack shit if you wnt to actually do
| voice translation into/from foreign languages
| nostrademons wrote:
| I'd encourage you to listen to some cutting edge speech
| synthesis like Google Duplex:
|
| https://www.youtube.com/watch?v=ijwHj2HaOT0
|
| This is live in production today - business owners are getting
| calls from robots, and you can trigger it yourself with Google
| Assistant.
|
| [Disclaimer: I work(ed) on Google Assistant - I'm in the
| process of transferring out - and have listened to some
| presentations about how Duplex works. It's pretty crazy
| technology.]
| nefitty wrote:
| An adjacent concern: Ads bad, yeah, but I appreciate the growing
| trend of subtitling them. I'm surprised there isn't some existing
| regulation on that, in the same way that streaming services are
| required to include them. The lower the cost of accessibility
| features, the more inclusive the media we consume can become.
| Rodeoclash wrote:
| I think it's a great option but their is an art to translations.
| This applies especially to books (I'm reading an English
| translation of https://en.wikipedia.org/wiki/Zorba_the_Greek
| which is very well done) and while you do have less dialog with
| movies, I wonder if some subtleties will be missed, particularly
| around humour.
| donkarma wrote:
| Yeah I forgot the only use of subtitles is for hearing people
| kingkawn wrote:
| I want to watch the original language with text, not listen to
| english adaptation whether synthesized or acted.
| aaron695 wrote:
| iratewizard wrote:
| About as likely as I am to sit through a blockbuster with my
| friends and family deepfaked onto the cast.
| echelon wrote:
| That sounds like a lot of fun. Especially if you can
| change/correct character position and posture.
| kashunstva wrote:
| I feel the same way about this as I do about reading poetry in
| translation, or listening to the (fortunately rare) opera sung in
| translation. There's something integral about the native
| qualities of the original language that I can't imagine would
| survive the AI process.
| 6gvONxR4sf7o wrote:
| > More Netflix subscribers watched dubbed versions of "Squid
| Game" than subtitled versions.
|
| The default of you load it is dubbed, right? That's what it
| started for me. So this stat probably has more to do with the
| default than anything else. They're making a weak case that this
| problem even needs solving.
| nkozyra wrote:
| > The default of you load it is dubbed, right?
|
| I don't think so, at least not for me.
| aeturnum wrote:
| This would be a lovely option to have and I find it hilarious
| that anyone imagines that subtitles are some kind of problem we
| are trying to get out of.
|
| Adding text to movies has a huge array of advantages: it
| preserves the original performances, it makes the dialog
| accessible to people with hearing or auditory processing
| disabilities, they can be multiplexed in a way that audio cannot,
| etc. Obviously it would be wonderful to have an 'autodub' option!
| But the times where one would use it seem quite narrow.
| Bilal_io wrote:
| I can confirm as an anime fan, when watching One Piece I prefer
| to watch it in Japanese as the dubbed version sounds odd.
|
| it was an advantage growing up watching Hollywood movies in
| English with Arabic subtitles, it helped me learn the language.
| moffkalast wrote:
| It also helps when what you're watching has atrocious audio
| mixing (looking at you Nolan), or you have shit speakers, or
| actors mumble or whisper, in which case it's helpful to have
| them as an aid when something's not clear.
| hepaminondas wrote:
| Yes. It also forces people to read and allows them to be
| exposed to foreign languages.
| Swizec wrote:
| Coming from a non english country I can assure you that dubbing
| is the worst. When we were kids we would actively avoid
| watching anything that was dubbed as soon as we could read.
|
| It's actually funny seeing how frustrated with subtitles my
| American girlfriend gets when we watch anything foreign. "You
| mean this basic skill every 7 year old has is hard for you, an
| educated super smart 30 year old? Bwahahaha"
|
| I still prefer watching everything with subtitles. Actors
| mumble a lot.
| samtheDamned wrote:
| > I still prefer watching everything with subtitles. Actors
| mumble a lot.
|
| Yep I have no problems hearing but newer movies and shows
| have uneven mixing where it can get really hard to hear over
| the background noise and subtitles come in clutch constantly
| for that
| stormbrew wrote:
| I have complicated feelings about this. I also watch most
| things with subtitles these days, because either (or probably
| both) my hearing is going a bit or sound design/editing in
| films and tv shows has gotten worse, but there is a non-
| trivial difference in how I perceive things with or without.
|
| With subtitles my eyes are drawn more often to a part of the
| screen where nothing visual is happening. They aren't glued
| there, but it does create some bias. I have occasionally
| rewatched a scene without subtitles and noticed a lot of
| subtlety in the performance I had missed the first time
| though.
|
| That said, a bad dub is at _least_ as distracting. Probably
| much more so, if anything. I would rather watch a bad sub
| than a bad dub, though I have occasionally enjoyed dubbed
| things as well. I don 't think it's an all or nothing thing,
| but "never watch dubs" is the hard line heuristic that makes
| sense to me if you're gonna go that way.
| openknot wrote:
| I share the same experience of preferring subtitles over
| dubs, coming from an English country. This is most popular
| with Japanese media like anime (like movies like 'Your
| Name'/'Kimi no wa na' or Attack on Titan). It also fits for
| foreign language films (e.g. preferring the original German
| for The Lives of Others/Wings of Desire).
|
| Crucially, outside of preference, subtitles are occasionally
| necessary for translating text on the screen (e.g. on
| building signs, newspaper front pages, posters, chalkboard
| writing), as information is conveyed solely through writing
| without spoken dialogue.
|
| For example, consider the chalkboard scene in "Komi san can't
| communicate": https://www.youtube.com/watch?v=JoH8I-NC5tM .
| It's key to the story that a character has trouble
| communicating verbally, so subtitles are essential to
| understand the written communication on the chalkboard. (You
| can then compare to a more extensively subtitled version that
| translates all the text on the board; good subtitles make a
| difference to storytelling: https://old.reddit.com/r/anime/co
| mments/qdjgba/the_blackboar...).
| pessimizer wrote:
| As an english first language speaker, I even find english
| subtitles on english language media very distracting. I
| consume a huge amount of subtitled films/tv because most good
| things aren't in english, but subtitles keep my eyes focused
| on the bottom of the screen.
|
| I don't like dubbing because it's generally much lower
| quality than the original performance, but I get why people
| who don't have a problem reading could prefer it.
| Kuinox wrote:
| It depends of the country.
|
| French have great dubs and most of the movie released are
| watched dubbed.
| commandlinefan wrote:
| > subtitles are some kind of problem
|
| Heck, they're not even limited to foreign languages. I put them
| on in English because I'm starting to lose my hearing now in my
| late 40's and if I turn up the volume loud enough that I can
| hear the dialogue ok, the sound effects would be so loud they'd
| set off the neighbor's car alarm.
| nkozyra wrote:
| > I find it hilarious that anyone imagines that subtitles are
| some kind of problem we are trying to get out of.
|
| I think there's a lot of gatekeeping about this. People -
| perhaps rightfully so - believe subtitled versions to be more
| ... "pure" than dubbed versions.
|
| I don't watch any language dubbed movies, but I wouldn't chide
| anyone who did. Not everyone wants to spend the whole two hours
| glancing up and down across the screen, nor are they
| necessarily wanting to learn a language passively, etc.
|
| So I think there's absolutely a perceived problem, it just
| doesn't apply to all viewers.
| jmckib wrote:
| I love subtitles, but for me they tend to spoil the timing of
| the lines being delivered, especially with comedy.
| Unfortunately subtitles are necessary even when I'm able to
| play audio as loud as I want, because it's almost always hard
| to understand at least 10-20% of the words that actors are
| saying.
|
| Maybe it would be nice to have subtitles that appear a word at
| a time as they are spoken, but I imagine that's not very
| practical for various reasons.
| nackerhewz wrote:
| Like those hard to follow auto translated generated captions on
| YouTube, but spoken...
|
| It's a good idea but I'm not sure how well that would work.
| black_puppydog wrote:
| Completely besides whether this is possible (I think it is; it's
| an offline learning problem with huge resources/economic
| potential and it's getting attention) I actually prefer
| subtitles...
|
| Coming from Germany, this wasn't always the case, but nowadays
| watching dubbed content feels so weird and impoverished, I don't
| even want to imagine what it'd be like knowing that the voice is
| fully synthetic.
| throw_away wrote:
| I would much rather we use AI to improve subtitling. Better auto
| speech recognition, per-word millisecond synchronization,
| positioning based on speaker/screen content-- all these could
| improve the subtitled experience and would be way less creepy
| than deepfake voices.
| xiii1408 wrote:
| Yes! My perennial problem is that most Chinese TV content has
| excellent subtitles, but they're burned in to the videos. So if
| I want to watch a Chinese show with a friend who doesn't read
| Chinese, there's no option even for auto-translated subtitles.
| I've often thought of writing a script to generate subtitle
| files using text recognition, but haven't gotten around to it.
| xiii1408 wrote:
| There's also the problem of content which doesn't have
| subtitles to begin with---YouTube's automatic subtitle
| generation is great, but could be improved and expanded to
| languages other than English.
| Jach wrote:
| It is expanded, at least to Japanese. What's also fun is
| you can then have it auto translate the auto generated
| subs. The results are... not typically great, but the
| potential!
| moffkalast wrote:
| > YouTubes auto mathy subtitle gen ration is grate
|
| FTFY
| testermelon wrote:
| The notion that some people prefer dub over sub is not new, but
| it baffles me that it is to the level of urgent business concern.
| From what I see in other comments, is it an American thing?
|
| We should invest in making people's ear accustomed/used to hear
| other languages. It has great intercultural benefits.
| misiti3780 wrote:
| I hope not, subtitles are a great way to learn a new language.
| friendlydog wrote:
| AI silent movie subtitles or AI generated comic book style
| captions with cell shaded actors would be awesome.
| bsenftner wrote:
| Oh come on everybody, this is a PR pseudo-advertisement for
| Veritone. The piece neglects to acknowledge that auto-dubbing is
| already a thing. No longer research has highly accurate voice
| synthesis of any specific voice with very limited source
| material. Realistic voice synthesis is very actively researched,
| and is already well into auto deep fake territory.
| fgdelcueto wrote:
| I grew up consuming movies and TV shows with subtitles so it has
| never bothered me. My theory is that if you're not used to them,
| then they are really annoying. Most of my American friends do not
| like them.
|
| There is something about watching a movie/show in its original
| language that you cannot understand. I think its awesome to be
| immersed in a different world, getting the sounds, inflections,
| etc from a different language of yours. I will always prefer
| subtitles to dubbed versions if possible.
|
| I understand some people don't, but I think it's just a matter of
| being exposed to it. And probably the younger, the better.
| thinkloop wrote:
| Translation barely works with text, it's still too literal, doubt
| this can be human-equivalent for a long time.
| aaronsdevera wrote:
| heck, auto-generated English captions on YouTube still struggle
| to keep up most the time...
|
| https://knowyourmeme.com/memes/youtube-automatic-caption-fai...
| echelon wrote:
| Absolutely coming.
|
| Goupon founder Andrew Mason's new startup Descript does a
| fantastic job editing podcasts from text. This same sort of
| tooling is absolutely going to make it into film.
|
| This sort of stuff is already trivially possible with zero
| effort:
|
| https://fakeyou.com/w2l/result/WR:t5m9xayqcwkjf9wng3nb4sxgzh...
|
| In ten years, we'll be able to reposition actors and fix flubbed
| takes. We won't have to visit set, use expensive lenses, or do
| many of the things that make Hollywood the domain of the rich and
| well funded.
|
| In thirty years, we'll just tell the machines what we want to
| watch.
| endisneigh wrote:
| What's really surprising to me is that synching subtitles isn't a
| solved problem.
|
| Why can't someone just loosely transcribe without time stamps and
| sync it to the video?
| echelon wrote:
| Have you turned on Google Meet subtitles recently?
| sp332 wrote:
| I think you can do this with YouTube.
| cardosof wrote:
| My wife is a translator and based on our conversations and her
| experience, subtitles are needed because people want to have the
| original audio + text, foreign subtitles are needed for obvious
| reasons, and human translators are needed for the "tricky" parts
| like new words and expressions, cultural context expressions,
| stuff that can't be directly translated, among other things.
| That's why in many countries a translated piece counts as a
| new/original piece, and the translator has the rights to claim
| some of the books IP.
| ahmed_ds wrote:
| On a tangential note: As far as training data for text and speech
| driven machine learning models go, isn't srt files truly
| remarkable?
|
| They are time stamped hence annotated precisely, hash and file
| matching makes media content to subtitle matching exact and
| overall there are language variations.
| cfcosta wrote:
| They are very artisanal, though. I used to do timing for anime
| fansubs when I was a teenager, and there's a lot of subtlety
| going on, specially when you don't speak the language (you're
| guided mostly by sound and phonetics, as the translations come
| more or less untimed).
|
| Timing is much more automatized nowadays, but still doesn't
| feel as high quality as handicrafted timing does.
___________________________________________________________________
(page generated 2022-01-19 23:03 UTC)