[HN Gopher] StyleNeRF: A Style-Based 3D-Aware Generator for High...
___________________________________________________________________
StyleNeRF: A Style-Based 3D-Aware Generator for High-Resolution
Image Synthesis
Author : lnyan
Score : 93 points
Date : 2022-03-11 07:52 UTC (15 hours ago)
(HTM) web link (jiataogu.me)
(TXT) w3m dump (jiataogu.me)
| birdyrooster wrote:
| Interesting that the reflections in each eye vary so much between
| each other.
| Mandelmus wrote:
| This might fix that: https://dorverbin.github.io/refnerf/
| petesergeant wrote:
| subtitle: an amazing trip through the uncanny valley
| harry-wood wrote:
| The twinkling children's TV style background music in that video
| is not making it any less creepy.
| motohagiography wrote:
| While watching it, I wondered whether the "this x does not
| exist," GAN sites also implies a subset of "this x _must_ exist "
| and if looking at it that way could yield discoveries in other
| areas. e.g. This X must exist, where x was a solar system,
| protein, wave function, chemical substance, cell, transcendental
| number, etc. Naive view, but when you're extapolating from a
| field of arbitrary 2D data, it seems there are a lot of ladders
| you could climb with that.
| Allstar wrote:
| The fact that this was in part developed with the help of
| Facebook AI Research is kind of unsettling.
|
| That said, the results are actually very impressive.
| throwoutway wrote:
| Didn't Facebook just get sued for illegally harvesting faces &
| tags of people? If derivatives of that data make it into
| research like this, where does that stand from a legal
| perspective?
| axg11 wrote:
| It's only unsettling if you are completely unfamiliar with the
| machine learning field. FAIR publishes research in many ML/AI
| fields. They're responsible for plenty of advances and
| contributions.
| macawfish wrote:
| Kinda puts a spin on the "face" in "facebook"
| thedorkknight wrote:
| It literally puts a spin on it, actually!
| kingcharles wrote:
| GANs are getting creepier by the day.
| nicoco wrote:
| Do you mean their potential (mis)use is creepy, or are you
| genuinely scared by their "abilities"?
| nlehuen wrote:
| This whole research domain is confusing to me. I do get that
| it's a hard and cool problem to solve and I do see why
| researcher could work on it on those grounds.
|
| But I'm concerned about what solving this is useful for,
| especially with the focus on generating human faces.
| Obligatory Jurassic Park quote: "Your scientists were so
| preoccupied with whether they could, they didn't stop to
| think if they should.".
|
| I can imagine this technology to be used in the entertainment
| industry (movies, video games) as a way to save money on art.
| But it's quite obvious that this technology can be weaponized
| and used in the propaganda industry (if it isn't already).
|
| So, this could save money in fun applications on one end, and
| provide means for mass opinion manipulation weapon on the
| other end. I am certainly biased but I'm not sure the
| recreative use case is worth the risk of the weaponized one
| (a bit like using tactical nukes for a firework show).
|
| What am I missing then? Are there any other use cases that
| would make this technology actually desirable for the greater
| good or humanity, or is it just a weapon with some potential
| recreational use cases?
| bsenftner wrote:
| Consider the case of how to create a very strong facial
| recognition system, without relying on photos of actual
| people. This system provides a data generation capability
| suitable to produce a training set with billions of
| "people", none of whom have had their privacy wounded by
| having their specific face in the training set.
| feanaro wrote:
| That's not exactly a happy use case. Why would we even
| want very strong facial recognition systems?
| bsenftner wrote:
| So short sighted. A strong facial recognition system can
| locate lost people in a wilderness from drone footage. A
| strong facial recognition system can discriminate between
| twins, near age siblings, and ethnically similar but
| unrelated individuals. A strong facial recognition system
| retains reliability when test imagery is poor quality. A
| strong facial recognition system can identify kidnapped
| children being trafficked, even after they have aged. A
| strong facial recognition system compensates for the
| complete lack of quality operator training within the
| facial recognition industry.
| uhhuhh wrote:
| Yea I mean the most eminent applications for this stuff
| is, putting unconsenting peoples likeness into porn,
| black mail/framing people, facial recognition, deceptive
| propaganda(because it's not deceptive enough already),
| weakening criminal evidence, etc. Then comes stuff like
| cgi for movies/games. Seeing a lot more of the former use
| cases not much of the latter. Pretty sure most people
| doing modern cv don't really realize where it's going,
| and people aren't publishing on the hazards so a lot of
| the criminality is and will remain novel until someone
| works it out..
| FeepingCreature wrote:
| I don't think of this as "we really wanted the ability to
| synthesize faces, this was very important to us as a
| civilization."
|
| Rather, I think of it as "if we create an AGI, it will have
| required the ability to imagine things. Also, we have huge
| datasets of faces that all sorta point the right way and
| have a good medium of feature complexity, and we're
| extremely specialized in spotting visual errors in faces."
|
| Consider the similar if slightly smaller research in
| creating photos of living rooms. (Or OpenAI's famous "comfy
| chair in the shape of an avocado.") Same reasoning. It's
| not about faces, faces just have beneficial properties.
| nicoco wrote:
| So you're not terminator-worried, just jurrasic park-
| worried then. (that was what my question was about)
|
| >I'm concerned about what solving this is useful for
|
| As for all research, it is pretty hard to predict what and
| what for something will be useful (despite researchers
| being pushed to say stuff like "my research is going to
| cure cancer" to get funding). But about neural networks for
| image processing, they certainly start to be useful for
| radiology.
| antegamisou wrote:
| > I do get that it's a hard and cool problem to solve and I
| do see why researcher could work on it on those grounds.
|
| There are quite a lot of harder and potentially even more
| useful if solved problems that receive significantly less
| attention and funding in this domain. Vast majority of GAN
| papers nowadays are just different applications of them,
| without significant contributions to underlying theoretical
| foundations.
| s0teri0s wrote:
| I guess the non-creepy application I'm seeing for the
| future of this research is the seamless conversion of a 2D
| image into 3D for when 3D projectors become more
| commonplace and perhaps overtake the ubiquity of
| flatscreens on our devices. And I am not sure, but it might
| also be applicable to the video-cam issue where we want to
| make it look as though the person we're having a video
| conference with is looking at us instead of the screen
| image of us and vice-versa.
| zdkl wrote:
| If it's "just code" someone somewhere is going to do it
| eventually, restrictions notwithstanding. Might as well do
| it in the open and publish about it so the problem space
| and its implications are (at least potentially) the object
| of further research. Oversimplification: Jurassic Park is
| about the lack of robust peer-review before going to prod!
| fxtentacle wrote:
| Cool idea :)
|
| 1. They generate a low-resolution position map using NeRF => Good
| camera and 3D control
|
| 2. They then upsample that with StyleGAN2 => Good features in the
| final 2D image.
|
| So in effect, this is a StyleGAN2 network where the latent space
| has the 3D-related components split off into a different
| generator network.
| axg11 wrote:
| I see there are a lot of questions/doubts in the comments about
| the usefulness of such research. This type of research is the
| precursor to gaining the ability to generative photorealistic 3D
| worlds. What is that useful for? Anywhere we currently generate
| 3D environments, including movies, games, VR/AR.
| egnehots wrote:
| that's weird that they mostly ignore the comparison to stylegan3:
| https://nvlabs.github.io/stylegan3/
|
| it's not a nerf based solution but since the goal is high
| resolution image syntesis..
___________________________________________________________________
(page generated 2022-03-11 23:02 UTC)