https://ai.meta.com/research/publications/emu-enhancing-image-generation-models-using-photogenic-needles-in-a-haystack/ Refresh (0 sec) /research/publications/ emu-enhancing-image-generation-models-using-photogenic-needles-in-a-haystack /?_fb_noscript=1 [382155218_][382493653_][382493653_] Research Blog Resources About [83271035_2][83271035_2][83062140_6] Research Blog Resources About COMPUTER VISION Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystack September 27, 2023 Abstract Training text-to-image models with web scale image-text pairs enables the generation of a wide range of visual concepts from text. However, these pre-trained models often face challenges when it comes to generating highly aesthetic images. This creates the need for aesthetic alignment post pre-training. In this paper, we propose quality-tuning to effectively guide a pre-trained model to exclusively generate highly visually appealing images, while maintaining generality across visual concepts. Our key insight is that supervised fine-tuning with a set of surprisingly small but extremely visually appealing images can significantly improve the generation quality. We pre-train a latent diffusion model on 1.1 billion image-text pairs and fine-tune it with only a few thousand carefully selected high-quality images. The resulting model, Emu, achieves a win rate of 82.9% compared with its pre-trained only counterpart. Compared to the state-of-the-art SDXLv1.0, Emu is preferred 68.4% and 71.3% of the time on visual appeal on the standard PartiPrompts and our Open User Input benchmark based on the real-world usage of text-to-image models. In addition, we show that quality-tuning is a generic approach that is also effective for other architectures, including pixel diffusion and masked generative transformer models. Download the Paper AUTHORS Written by Xiaoliang Dai Ji Hou Kevin Chih-Yao Ma Sam Tsai Jialiang Wang Rui Wang Peizhao Zhang Simon Vandenhende Xiaofang Wang Abhimanyu Dubey Matthew Yu Abhishek Kadian Filip Radenovic Dhruv Mahajan Kunpeng Li Yue (R) Zhao Vladan Petrovic Mitesh Kumar Singh Simran Motwani Yiwen Song Yi Wen Roshan Sumbaly Vignesh Ramanathan Zijian He Peter Vajda Devi Parikh Publisher Meta Research Topics Computer Vision Related Publications August 31, 2023 COMPUTER VISION FACET: Fairness in Computer Vision Evaluation Benchmark Laura Gustafson, Chloe Rolland, Nikhila Ravi, Quentin Duval, Aaron Adcock, Cheng-Yang Fu, Melissa Hall, Candace Ross August 31, 2023 Read the Paper July 14, 2023 NLP COMPUTER VISION Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning Lili Yu, Bowen Shi, Ram Pasunuru, Benjamin Miller, Olga Golovneva, Tianlu Wang, Arun Babu, Binh Tang, Brian Karrer, Shelly Sheynin, Candace Ross, Adam Polyak, Russ Howes, Vasu Sharma, Jacob Xu, Uriel Singer, Daniel Li (FAIR), Gargi Ghosh, Yaniv Taigman, Maryam Fazel-Zarandi, Asli Celikyilmaz, Luke Zettlemoyer, Armen Aghajanyan July 14, 2023 Read the Paper June 20, 2023 COMPUTER VISION Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wild Garrick Brazil, Abhinav Kumar, Julian Straub, Nikhila Ravi, Justin Johnson, Georgia Gkioxari June 20, 2023 Read the Paper June 18, 2023 ROBOTICS REINFORCEMENT LEARNING Galactic: Scaling End-to-End Reinforcement Learning for Rearrangement at 100k Steps-Per-Second Vincent-Pierre Berges, Andrew Szot, Devendra Singh Chaplot, Aaron Gokaslan, Dhruv Batra, Eric Undersander June 18, 2023 Read the Paper See All Papers [90971213_2] Help Us Pioneer The Future of AI We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment. Join our Team Who We Are About People Careers Events Latest Work Research Infrastructure Blog Resources Our Actions Responsibilities Newsletter Sign Up [87524316_2] [335682312_] [335682312_] [336009607_] [336009607_] [336289415_] [336289415_] [335648731_] [335648731_] Who We Are Who We AreAboutPeopleCareersEvents Latest Work Latest WorkResearchInfrastructureBlogResources Our Actions Our ActionsResponsibilities Newsletter NewsletterSign Up [335682312_] [335682312_] [336009607_] [336009607_] [336289415_] [336289415_] [335648731_] [335648731_] Privacy Policy Terms Cookies Meta (c) 2023 [335682312_] [335682312_] [336009607_] [336009607_] [336289415_] [336289415_] [335648731_] [335648731_]