https://github.com/cabralpinto/modular-diffusion Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Customer Stories + White papers, Ebooks, Webinars + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} cabralpinto / modular-diffusion Public * Notifications * Fork 1 * Star 57 Python library for designing and training your own Diffusion Models with PyTorch. cabralpinto.github.io/modular-diffusion/ License MIT license 57 stars 1 fork Activity Star Notifications * Code * Issues 8 * Pull requests 0 * Actions * Projects 0 * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Security * Insights cabralpinto/modular-diffusion This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 1 branch 1 tag Code * Local * Codespaces * Clone HTTPS GitHub CLI [https://github.com/c] Use Git or checkout with SVN using the web URL. [gh repo clone cabral] Work fast with our official CLI. Learn more about the CLI. * Open with GitHub Desktop * Download ZIP Sign In Required Please sign in to use Codespaces. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @cabralpinto cabralpinto Add CITATION.cff ... 4f891cd Sep 2, 2023 Add CITATION.cff 4f891cd Git stats * 31 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .github/workflows Replace docs deployment GitHub action August 26, 2023 15:09 diffusion Allow str device August 28, 2023 16:36 docs Docs: Remove unnecessary h1 tags, add description August 29, 2023 16:29 examples Allow str device August 28, 2023 16:36 .gitignore Update metadata to v0.0.2 August 28, 2023 18:40 .pre-commit-config.yaml First commit June 22, 2023 11:32 .style.yapf First commit June 22, 2023 11:32 CITATION.cff Add CITATION.cff September 2, 2023 19:24 LICENSE First commit June 22, 2023 11:32 README.md Update README.md August 28, 2023 19:15 pyproject.toml Update requirements August 28, 2023 19:33 pyrightconfig.json Add initial code and docs August 26, 2023 13:24 requirements.txt Update requirements August 28, 2023 19:33 View code Modular Diffusion Features Installation Usage Contributing License README.md Modular Diffusion PyPI version Documentation MIT license Modular Diffusion provides an easy-to-use modular API to design and train custom Diffusion Models with PyTorch. Whether you're an enthusiast exploring Diffusion Models or a hardcore ML researcher, this framework is for you. Features * [?][?] Highly Modular Design: Effortlessly swap different components of the diffusion process, including noise type, schedule type, denoising network, and loss function. * Growing Library of Pre-built Modules: Get started right away with our comprehensive selection of pre-built modules. * Custom Module Creation Made Easy: Craft your own original modules by inheriting from a base class and implementing the required methods. * Integration with PyTorch: Built on top of PyTorch, Modular Diffusion enables you to develop custom modules using a familiar syntax. * Broad Range of Applications: From generating high-quality images to implementing non-autoregressive text synthesis pipelines, the possiblities are endless. Installation Modular Diffusion officially supports Python 3.10+ and is available on PyPI: pip install modular-diffusion You also need to install the correct PyTorch distribution for your system. Note: Although Modular Diffusion works with later Python versions, we currently recommend using Python 3.10. This is because torch.compile, which significantly improves the speed of the models, is not currently available for versions above Python 3.10. Usage With Modular Diffusion, you can build and train a custom Diffusion Model in just a few lines. First, load and normalize your dataset. We are using the dog pictures from AFHQ. x, _ = zip(*ImageFolder("afhq", ToTensor())) x = resize(x, [h, w], antialias=False) x = torch.stack(x) * 2 - 1 Next, build your custom model using either Modular Diffusion's prebuilt modules or your custom modules. model = diffusion.Model( data=Identity(x, batch=128, shuffle=True), schedule=Cosine(steps=1000), noise=Gaussian(parameter="epsilon", variance="fixed"), net=UNet(channels=(1, 64, 128, 256)), loss=Simple(parameter="epsilon"), ) Now, train and sample from the model. losses = [*model.train(epochs=400)] z = model.sample(batch=10) z = z[torch.linspace(0, z.shape[0] - 1, 10).long()] z = rearrange(z, "t b c h w -> c (b h) (t w)") save_image((z + 1) / 2, "output.png") Finally, marvel at the results. Modular Diffusion teaser Check out the Getting Started Guide to learn more and find more examples here. Contributing We appreciate your support and welcome your contributions! Please fell free to submit pull requests if you found a bug or typo you want to fix. If you want to contribute a new prebuilt module or feature, please start by opening an issue and discussing it with us. If you don't know where to begin, take a look at the open issues. License This project is licensed under the MIT License. About Python library for designing and training your own Diffusion Models with PyTorch. cabralpinto.github.io/modular-diffusion/ Topics python machine-learning deep-learning text-generation pytorch transformer image-generation modular-design u-net diffusion-models audio-generation Resources Readme License MIT license Activity Stars 57 stars Watchers 2 watching Forks 1 fork Report repository Releases 1 v0.0.3 Latest Aug 29, 2023 Packages 0 No packages published Languages * Python 100.0% Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time.