[HN Gopher] Neural Network Enabled Filmmaking
___________________________________________________________________
Neural Network Enabled Filmmaking
Author : doener
Score : 39 points
Date : 2021-05-23 19:11 UTC (3 hours ago)
(HTM) web link (www.flawlessai.com)
(TXT) w3m dump (www.flawlessai.com)
| fxtentacle wrote:
| TLDR: something like FaceGAN to force artificial lip sync for a
| dub into the video frames
|
| That way, you can dub videos however you like without having to
| rewrite your dub text to match the visible mouth movements. I
| like the idea, but in my opinion finding good dub speakers is the
| challenge, not figuring out what you want them to say.
| dalbasal wrote:
| I wonder...
|
| The problem with dubbing is that it inherently sucks. That's a
| poor starting point for making something good. Any kind of
| connoisseur would watch the original anyway. Not many actors
| would be enthusiastic about being the voice of Renee Zellweger
| for the unwashed masses.
|
| Animated films, OTOH, tend to have great dubbing. They hire
| famous actors, put in the effort... just like they do for the
| original language.
|
| If dubbing didn't suck, it might start a whole chain reaction
| towards good.
| scyzoryk_xyz wrote:
| Nevertheless, in some places in the world it is a mandatory
| standard to dub and it is a costly process distributors must
| handle to enter the market. I imagine VOD's could also be
| interested in this as they have large libraries. Optimizing
| dubbing for them can really shift the proportion of localized
| content which can be useful when seeking regulatory approval
| and what not. I There are also probably many other
| applications for something like this, albeit I'm with you
| that dubbing is super lame.
| [deleted]
| swframe2 wrote:
| I like this direction. There is still more work needed. They
| should post-process the frames similar to the restyle-encoder:
| https://github.com/yuval-alaluf/restyle-encoder
| jtthe13 wrote:
| FWIW the french bits are unimpressive at best. I'd stick with the
| subtitles
| simlevesque wrote:
| Yeah, it's worse than a good dub for french. The lips move kind
| of at the right moment, but that's it. The lips don't move the
| way they should to produce the sound.
|
| In short, it's not even uncanny valley level.
| Uehreka wrote:
| "Neural Network Lip Sync For Translations" feels like a more
| honest tagline than "Neural Network Enabled Filmmaking". I get
| that you're probably aiming for filmmakers, and they can
| definitely make good use of this tech, but I've gotta imagine the
| "whales" in this market are companies with hundreds of hours of
| corporate training videos that need to be localized.
|
| But no matter the audience, cutting to the chase about what the
| product does is super important.
| TaylorAlexander wrote:
| Honest request: re-do Short Circuit so the "Indian" character
| played by a white guy just looks and speaks more like a white
| guy. I tried watching that movie for fun but the blackface or
| whatever you would call it really tanked it for me.
|
| Also in your promo video you show "traditional dubbing" to
| English and I really expected that to be followed by your system
| on the same clip but instead you moved to other languages, which
| I can't really grasp.
|
| Edit: I see I've gotten a downvote. You can hear from the actor
| himself how he feels about playing that role:
| https://www.dailywire.com/news/fisher-stevens-regrets-playin...
| bredren wrote:
| It would be interesting to see it fully redubbed with the
| actor's own unaccented voice.
| [deleted]
| kleiba wrote:
| Sorry to say but the quality is everything but flawless. Looks
| very unnatural by itself IMO, and does not seem to match the
| audio very well either.
| 0-_-0 wrote:
| You're right. It doesn't look great _yet_ , but in a year or
| two...
| marstall wrote:
| dubbing, yech
| bredren wrote:
| Looks cool to me. I don't speak these other languages, but I like
| the way the actors' facial expressions are maintained through the
| dub.
|
| I wonder how longer or shorter phrasing would affect clips or cut
| length and how backgrounds might also have to accommodate script
| line length.
|
| This last year's best picture went to Parasite, a "foreign" film
| with English subtitles.
|
| While film people would probably always prefer the original, I
| bet most Americans would have watched Parasite dubbed with this
| technology and not even realized the actors didn't use English in
| the creation.
|
| Hollywood does butts-in-seats and I think this would put more
| butts in seats.
| scyzoryk_xyz wrote:
| I was expecting something to do with actual filmmaking - like
| writing, production optimization, etc. This is... dubbing?
| pedalpete wrote:
| I wasn't expecting this to be lip syncing, and so many negative
| comments here about quality and "I'd rather watch dubs".
|
| To me, this is "good enough" to watch. I don't like dubs because
| the voice is out of sync with the character, and removes you from
| the story. I don't like subtitles because I can read faster than
| they talk, and then I'm left with words blocking part of the
| scene, and I feel like I know what the person has said, yet their
| still talking.
|
| This is a great start, and they'll continue to improve in the
| future.
| ginko wrote:
| As someone who grew up with dubs I think this looks terrible and
| distracting.
| chairmanwow1 wrote:
| The over the top marketing on the landing page is so hilarious to
| me.
| bredren wrote:
| Having looked at some Hollywood-focused products and services
| recently, this type video is similar to a "reel," and from what
| I can tell this kind of presentation is typical of the biz.
| azinman2 wrote:
| Firmly in the uncanny valley. I'd rather watch the original with
| subtitles (and that's almost always the case, not only in terms
| of preserving acting, but to be exposed to other languages and
| cultures). But is a cool idea for the future. I imagine in 10
| years this could be pretty amazing.
___________________________________________________________________
(page generated 2021-05-23 23:01 UTC)