https://github.com/varunshenoy/opendream Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Customer Stories + White papers, Ebooks, Webinars + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. {{ message }} varunshenoy / opendream Public * Notifications * Fork 11 * Star 386 An extensible, easy-to-use, and portable diffusion web UI License MIT license 386 stars 11 forks Activity Star Notifications * Code * Issues 5 * Pull requests 1 * Actions * Projects 0 * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Security * Insights varunshenoy/opendream This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 1 branch 0 tags Code * Local * Codespaces * Clone HTTPS GitHub CLI [https://github.com/v] Use Git or checkout with SVN using the web URL. [gh repo clone varuns] Work fast with our official CLI. Learn more about the CLI. * Open with GitHub Desktop * Download ZIP Sign In Required Please sign in to use Codespaces. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @varunshenoy varunshenoy Clarify requirements for extensions ... 8ae90ed Aug 15, 2023 Clarify requirements for extensions 8ae90ed Git stats * 118 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time images edited June 23, 2023 14:00 opendream make controlnets clearer July 2, 2023 15:21 webapp/opendream-ui added textarea for prompt July 2, 2023 11:36 .gitignore remove workflows June 19, 2023 23:54 LICENSE done June 20, 2023 19:42 README.md Clarify requirements for extensions August 15, 2023 12:58 requirements.txt Update requirements.txt August 15, 2023 12:57 run_opendream.sh use venv in script August 15, 2023 12:10 View code [ ] Opendream: A Web UI For the Rest of Us Getting started Features Layering and Non-destructive Editing Save and Share Workflows Support Simple to Write, Easy to Install Extensions Extensions Loading an Existing Extension Writing Your Own Extension Contributions and Licensing README.md Opendream: A Web UI For the Rest of Us Opendream brings much needed and familiar features, such as layering, non-destructive editing, portability, and easy-to-write extensions, to your Stable Diffusion workflows. Check out our demo video. hero Getting started 1. Clone this repository. 2. Navigate to this project within your terminal and run sh ./ run_opendream.sh. After ~30 seconds, both the frontend and backend of the Opendream system should be up and running. Features Diffusion models have emerged as powerful tools in the world of image generation and manipulation. While they offer significant benefits, these models are often considered black boxes due to their inherent complexity. The current diffusion image generation ecosystem is defined by tools that allow one-off image manipulation tasks to control these models - text2img, in-painting, pix2pix, among others. For example, popular interfaces like Automatic1111, Midjourney, and Stability.AI's DreamStudio only support destructive editing: each edit "consumes" the previous image. This means users cannot easily build off of previous images or run multiple experiments on the same image, limiting their options for creative exploration. Layering and Non-destructive Editing Non-destructive editing is a method of image manipulation that preserves the original image data while allowing users to make adjustments and modifications without overwriting previous work. This approach facilitates experimentation and provides more control over the editing process by using layers and masks. When you delete a layer, all layers after it also get deleted. This guarantees that all layers currently on the canvas are a product of other existing layers. This also allows one to deterministically "replay" a workflow. Like Photoshop, Opendream supports non-destructive editing out of the box. Learn more about the principles of non-destructive editing in Photoshop here. layers Save and Share Workflows Users can also save their current workflows into a portable file format that can be opened up at a later time or shared with collaborators. In this context, a "state" is just a JSON file describing all of the current layers and how they were created. workflow Support Simple to Write, Easy to Install Extensions As the open-source ecosystem flourishes around these models and tools, extensibility has also become a major concern. While Automatic1111 does offer extensions, they are often difficult to program, use, and install. It is far from being as full-featured as an application like Adobe Photoshop. As new features for Stable Diffusion, like ControlNet, are released, users should be able to seamlessly integrate them into their artistic workflows with minimal overload and time. Opendream makes writing and using new diffusion features as simple as writing a Python function. Keep reading to learn how. Extensions From the get-go, Opendream supports two key primitive operations baked into the core system: dream and mask_and_inpaint. In this repository, extensions for instruct_pix2pix, controlnet_canny, controlnet_openpose, and sam (Segment Anything) are provided. Any image manipulation logic can be easily written as an extension. With extensions, you can also decide how certain operations work. For example, you can override the dream operation to use OpenAI's DALL-E instead or call a serverless endpoint on a service like AWS or Replicate. Here's an example using Baseten. Loading an Existing Extension There are two ways to load extensions. 1. Install a pre-written one through the Web UI. 2. (Manual) Download a valid extension file (or write one yourself!) and add it to the opendream/extensions folder. Instructions for writing your own extension are below. Here is a sampling of currently supported extensions. You can use the links to install any given extension through the Web UI. Extension Link OpenAI's DALL-E File Serverless Stable Diffusion File Instruct Pix2Pix File ControlNet Canny File ControlNet Openpose File Segment Anything File PhotoshopGPT Gist Note that extensions may have their own requirements you would need to include in the requirements.txt file. For example, you would need to add openai if you want to use the DALL-E extension. Feel free to make a PR if you create a useful extension! Writing Your Own Extension Users can write their own extensions as follows: 1. Create a new Python file in the opendream/extensions folder. 2. Write a method with type hints and a @opendream.define_op decorator. This decorator registers this method with the Opendream backend. The method has a few requirements: * Parameters must have type hints. These enable the backend to generate a schema for the input which is parsed into form components on the frontend. Valid types include: str, int, float, Layer, MaskLayer, or ImageLayer. * The only valid return types are a Layer or a list of Layer objects. Contributions and Licensing Opendream was built by Varun Shenoy, Eric Lou, Shashank Rammoorthy, and Rahul Shiv as a part of Stanford's CS 348K. Feel free to provide any contibutions you deem necessary or useful. This project is licensed under the MIT License. About An extensible, easy-to-use, and portable diffusion web UI Topics ai image-generation diffusion stable-diffusion automatic-1111 Resources Readme License MIT license Activity Stars 386 stars Watchers 4 watching Forks 11 forks Report repository Releases No releases published Packages 0 No packages published Contributors 5 * @varunshenoy * @EricZLou * @shashank2000 * @nazihkalo * @rshiv2 Languages * JavaScript 58.3% * Python 37.1% * HTML 2.7% * CSS 1.5% * Shell 0.4% Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time.