https://make-a-video3d.github.io/ Text-To-4D Dynamic Scene Generation Uriel Singer* Shelly Sheynin* Adam Polyak* Oron Ashual Iurii Makarov Filippos Kokkinos Naman Goyal Andrea Vedaldi Devi Parikh Justin Johnson Yaniv Taigman ^*Equal Contribution Meta AI [Paper] Abstract We present MAV3D (Make-A-Video3D), a method for generating three-dimensional dynamic scenes from text descriptions. Our approach uses a 4D dynamic Neural Radiance Field (NeRF), which is optimized for scene appearance, density, and motion consistency by querying a Text-to-Video (T2V) diffusion-based model. The dynamic video output generated from the provided text can be viewed from any camera location and angle, and can be composited into any 3D environment. MAV3D does not require any 3D or 4D data and the T2V model is trained only on Text-Image pairs and unlabeled videos. We demonstrate the effectiveness of our approach using comprehensive quantitative and qualitative experiments and show an improvement over previously established internal baselines. To the best of our knowledge, our method is the first to generate 3D dynamic scenes given a text description. Text-to-4D A corgi playing with a ball. A panda dancing. A space shuttle launching. Clown fish swimming through the coral reef. An emoji of a baby panda reading a book. A dog riding a skateboard. 3D rendering of a fox playing videogame. A squirrel riding a motorcycle. A crocodile playing a drum set. A panda playing on a swing set. A goat drinking beer. A dog wearing a Superhero outfit with red cape flying through the sky. A silver humanoid robot flipping a coin. An octopus is underwater. A squirrel playing on a swing set. An alien playing the piano. A squirrel playing the saxophone. A humanoid robot playing the violin. A kangaroo cooking a meal. A baby panda eating ice cream. A yorkie dog eating a donut. A human skeleton drinking wine. Chihuahua running on the grass. A bear driving a car. A cat singing. A squirrel Djing. Shark swimming in the desert. A monkey eating a candy bar. Image-to-4D Input Image Generate Video Input Image Generate Video [a] - [a] - [a] - [a] - Citation @article{singer2023text4d, author = {Singer, Uriel and Sheynin, Shelly and Polyak, Adam and Ashual, Oron and Makarov, Iurii and Kokkinos, Filippos and Goyal, Naman and Vedaldi, Andrea and Parikh, Devi and Johnson, Justin and Taigman, Yaniv}, title = {Text-To-4D Dynamic Scene Generation}, journal = {arXiv:2301.11280}, year = {2023}, } For optimal viewing of dynamic 3D videos, please access our website from a desktop using Chrome.