http://jiataogu.me/style_nerf/

StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image
Synthesis

Arxiv 2021

---------------------------------------------------------------------

Jiatao Gu^ 1   Lingjie Liu^ 2   Peng Wang^ 3   Christian Theobalt^
2  

^1Facebook AI Research   ^2MPI Informatics, Saarland Informatics
Campus   ^3University of Hong Kong  

Paper PDF

arXiv

Code & Data (Comming Soon)

Abstract

---------------------------------------------------------------------

We present StyleNeRF, a 3D-aware generative model that can synthesize
high-resolution images with high multi-view consistency.

[pipeline_i]

We propose StyleNeRF, a 3D-aware generative model for photo-realistic
high-resolution image synthesis with high multi-view consistency,
which can be trained on unstructured 2D images. Existing approaches
either cannot synthesize high-resolution images with fine details or
yield noticeable 3D-inconsistent artifacts. In addition, many of them
lack control over style attributes and explicit 3D camera poses.
StyleNeRF integrates the neural radiance field (NeRF) into a
style-based generator to tackle the aforementioned challenges, i.e.,
improving rendering efficiency and 3D consistency for high-resolution
image generation. We perform volume rendering only to produce a
low-resolution feature map and progressively apply upsampling in 2D
to address the first issue. To mitigate the inconsistencies caused by
2D upsampling, we propose multiple designs, including a better
upsampler and a new regularization loss. With these designs,
StyleNeRF can synthesize high-resolution images at interactive rates
while preserving 3D consistency at high quality. StyleNeRF also
enables control of camera poses and different levels of styles, which
can generalize to unseen views. It also supports challenging tasks,
including zoom-in and-out, style mixing, inversion, and semantic
editing.


Overview video

---------------------------------------------------------------------


Real-time demo video

---------------------------------------------------------------------


Citation

---------------------------------------------------------------------


  @misc{gu2021stylenerf,
    title={StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis}, 
    author={Jiatao Gu and Lingjie Liu and Peng Wang and Christian Theobalt},
    year={2021},
    eprint={2110.08985},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

---------------------------------------------------------------------
[fair_logo] [Logo_MPII] [Logo_gvv] [hku_logo]

Thanks to Lior Yariv for the website template.