[HN Gopher] Realistic one-shot mesh-based head avatars
___________________________________________________________________
Realistic one-shot mesh-based head avatars
Author : lnyan
Score : 135 points
Date : 2022-06-17 04:01 UTC (1 days ago)
(HTM) web link (samsunglabs.github.io)
(TXT) w3m dump (samsunglabs.github.io)
| nr2x wrote:
| Such a shame that Putin took a country with such amazing
| technological potential down a dead end path of hydrocarbons,
| political repression, violence, and war.
|
| But hey, more world class engineers for the rest of us.
| mensetmanusman wrote:
| Good catch. Samsung just saved tens of millions of dollars
| hiring such talent.
| nr2x wrote:
| Yeah, Russia has the brains and education to have made their
| own Samsung...but too bad, an aging psychopath wants to
| silence anybody with half a brain and dig in the mud for
| "wealth".
| jokethrowaway wrote:
| Impressive work! The main cons of this seems to be unbaking
| natural light from the texture
| isuckatcoding wrote:
| What is the real use case for this?
| tristor wrote:
| This would be useful for anyone that has specific company
| requirements for their avatar/image as long as it can be
| combined with something that detects the face and eliminates
| the background
| zhyder wrote:
| I think it could be used to create a videoconferencing
| experience akin to sitting around a table with multiple people,
| while still selectively facing or making eye contact with
| individuals.
| [deleted]
| acd wrote:
| Video conferences. You can create a 3d world projecting 3d
| avatars made from images from peoples web cams. Creating a 3d
| sense of being in the same room.
|
| Acting in 3d recreation of your favorite movies. If you can
| scan faces you could play in rendered 3d recreations of movies.
|
| Video games you could from a web cam image play a 3d avatar of
| yourself in a game.
| jimmySixDOF wrote:
| It's a stepping stone towards the integration between NeRF
| derived and DALL-E 2 CLIP diffusion type systems to create user
| defined on demand real-time volumetric 3D digital spaces for
| spatial computing.
| sangnoir wrote:
| Is "user defined on demand real-time volumetric 3D digital
| spaces for spatial computing" in this instance a euphemism
| for fanfic porn?
| jokethrowaway wrote:
| I would make a model of myself create a nice background, then
| detect face expressions, apply them to the model and stream the
| model while I work shirtless from a dirty basement.
| swah wrote:
| Making avatars without the 3D camera thing that phones do.
| Could also make avatars of famous people, deceased loved
| ones...
| jayd16 wrote:
| User avatars for games or video conferences.
|
| Post fx tooling for movies. Cheaply touch up or add virtual
| extras to a scene.
| totalview wrote:
| Easier rigged characters in video games that can be
| personalized to a player
| scoopertrooper wrote:
| I think this could be used as a great basis for next generation
| video conferencing. Rather than compressing raster images of
| people speaking, we could send a model as an initial payload
| and subsequent instructions on how to manipulate it as the
| conversation progresses.
|
| Most current networks can handle video conferencing
| (relatively) well. However, if 8k video or something like Light
| Field displays ever catch on, then we might end up needing
| orders of magnitude more bandwidth to drive video conferencing,
| so this might be an enabling technology.
| ricardobeat wrote:
| Might not be that simple. In a recent demo from Unreal Engine
| they said that the facial animation data is the main
| bottleneck to shipping titles with more realistic avatars.
| Their minute-long demo took many TBs to store the motion data
| alone.
| cheschire wrote:
| My layperson understanding makes this baffling to me. Isn't
| motion inherently analog, and haven't we've developed high
| performing analog compression already? Motion seems
| significantly more amenable to lossy compression as well,
| and I could imagine a use-case for ML-based decompression
| to fill in between the keyframes.
| dmwallin wrote:
| You also have to consider that faces are one of the
| things that we have the strongest perception of, with
| lots of our neurons dedicated to the task, so when you
| get things wrong it's far more noticeable than many other
| bodily animations would be.
| jon_richards wrote:
| One of my favorite "weird" conjectures is that the
| existence of the uncanny valley implies that at some
| point it was evolutionarily advantageous for us to
| recognize something that looked human, but wasn't... and
| to be afraid of it.
| rjeli wrote:
| neanderthals?
| giantrobot wrote:
| Think also about face-like things we perceive because of
| pareidolia but might in fact be dangerous. One part of
| our brain is opportunistic about finding patterns while
| another is a sanity check on those patterns telling us
| not to just trust them outright.
| [deleted]
| HWR_14 wrote:
| Facial animation is a real issue. This is facial mapping,
| which is an easier to solve problem, in that throwing
| artists at the problem is a solution.
| abeppu wrote:
| I guess, would watching an animated head avatar in 8k or a
| Light Field display be a better experience than watching
| lower resolution video?
|
| I wonder whether the experience would be too undermined by
| artifacts from e.g. your coworker's cat, for which the system
| doesn't have a model, wanders into frame partway through a
| call, and suddenly the system must dynamically either (a)
| 'downgrade' to just video (b) be able to create a new model
| for an arbitrary new thing on the fly (c) create some hybrid
| of partial video data with part viewer-rendered animation or
| (d) your coworker looks crazy interacting with something the
| system hides from you entirely.
| ekianjo wrote:
| > Rather than compressing raster images of people speaking,
| we could send a model as an initial payload and subsequent
| instructions on how to manipulate it as the conversation
| progresses.
|
| What about latency though? Would that improve?
| fortran77 wrote:
| Or, at least, the faces that aren't actively speaking could
| be replaced by avatars. It will also help keep a clean image
| in case stuff is going on the the background, etc.
| itomato wrote:
| Very interesting to see the cross-reenactment renders.
|
| In that case, I think FOMM's interpretation is more recognizable
| but Bi-Layer David Spade, is particularly entertaining.
| jmpman wrote:
| Why is Samsung employing people in Russia?
| tartsky wrote:
| Why shouldn't they?
| jokethrowaway wrote:
| 0des wrote:
| Maybe a group of people did a thing and Samsung wanted it
| postsantum wrote:
| Sir, this is not Twitter
| [deleted]
| kgeist wrote:
| Strong math and programming school? OpenCV was largely
| developed by Intel Russia, for example.
___________________________________________________________________
(page generated 2022-06-18 23:01 UTC)