https://github.com/muxamilian/duralava Skip to content Sign up * Why GitHub? + Features + Mobile + Actions + Codespaces + Packages + Security + Code review + Issues + Integrations + GitHub Sponsors + Customer stories * Team * Enterprise * Explore + Explore GitHub + Learn and contribute + Topics + Collections + Trending + Learning Lab + Open source guides + Connect with others + The ReadME Project + Events + Community forum + GitHub Education + GitHub Stars program * Marketplace * Pricing + Plans + Compare plans + Contact Sales + Education [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this user All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} muxamilian / duralava Public * Notifications * Fork 0 * Star 26 * duralava is a neural network which can simulate a lava lamp in an infinite loop. 26 stars 0 forks Star Notifications * Code * Issues 0 * Pull requests 0 * Actions * Projects 0 * Wiki * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Wiki * Security * Insights This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags 1 branch 0 tags Code Latest commit @muxamilian muxamilian Update README.md ... 074e337 Feb 5, 2022 Update README.md 074e337 Git stats * 49 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time frames Add dataset Jan 9, 2022 logs/20220104-213105 Add weights Jan 15, 2022 .gitattributes Add movie Jan 9, 2022 README.md Update README.md Feb 5, 2022 lavalamp.mov Add lavalamp video Feb 5, 2022 learn.py Add comments Feb 5, 2022 requirements.txt Add live mode Feb 5, 2022 View code [ ] duralava Example videos Novelty How it works Learning Live mode Generating an output video Low-hanging fruit Dataset README.md duralava duralava is a neural network which can simulate a lava lamp in an infinite loop. Example videos This is not a real lava lamp but a "fake" one generated by duralava. (Might take some time to load.) out_180, duralava neural network deep learning lava lamp sample 1 out_170, duralava neural network deep learning lava lamp sample 2 out_160, duralava neural network deep learning lava lamp sample 3 Novelty duralava can * learn a physical process (a lava lamp). * generate an arbitarily long sequence of output, without diverging even after hours (outputting tens of thousands of frames). How it works Generative Adversarial Networks (GANs) can learn to generate new samples of data. For example, a GAN can be trained to output images of a lava lamp which look as real as possible. To accomplish this, the GAN gets an input vector with normally distributed noise. For duralava this vector is of length 64. Based on this random noise vector it generates a lava lamp image. The random vector thus encodes the state of the lava lamp. For training, the GAN is presented a real image of a lava lamp and also one of the fake lava lamp and then it learns to make the fake ones look as real as possible. For a lava lamp, a sequence of images has to be created. This sequence should in fact be infinite since a lava lamp can run forever. Thus the GAN should learn to output an arbitrarily long sequence of lava lamp images as a video. This is achieved by using a recurrent neural network (RNN). The RNN gets the 64 element noise vector of time step t and outputs the 64 element noise vector for time stemp t+1. The tricky part is to make sure that the state of the lava lamp (the 64 element random noise vector) remains stable. It could for example happen that over time the distribution of noise in the vector diverges from a normal distribution and that the mean becomes 10 and the standard deviation 52. In this case, the output images of the lava lamps wouldn't be correct anymore as the GAN was trained to expect the input vector to be normally distributed. To solve this problem, I make sure that in training the output of the RNN stays normally distributed. This is accomplished by adding penalization terms in the training which discourage the noise to diverge from the normal distribution. Learning To learn a new model run python learn.py Around 10 GB of combined CPU and GPU memory are required. I used python 3.9.7 and the pip requirements listed in requirements.txt. For creating APNGs from a trained neural network, you need to have ffmpeg installed. Live mode To generate an output video use some of the trained weights like this: python learn.py --weights logs/20220104-213105/weights.180 --mode live Generating an output video To generate an output video use some of the trained weights like this: python learn.py --weights logs/20220104-213105/weights.180 --mode video An APNG named out.png will be created in the current directory. Low-hanging fruit I trained on a MacBook Air with an M1 SoC with 16 GB of shared memory for CPU and GPU. Thus, memory was the limiting factor in my experiments. With more memory, one could * Increase the resolution (currently 64x64 pixels) * Increase the training sequence length (currently 20) * Increase the batch size (currently 32) * Increase the size of the recurrent neural networks, which model the evolution of the lava lamp over time Dataset lavalamp.mov contains more than 1 hour of footage of a lava lamp at 30 fps and can be freely used for any purpose. In the frames directory there are the individual frames of the video scaled to 64x64 pixels, which I used for training the model. About duralava is a neural network which can simulate a lava lamp in an infinite loop. Resources Readme Stars 26 stars Watchers 2 watching Forks 0 forks Releases No releases published Packages 0 No packages published * (c) 2022 GitHub, Inc. * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.