https://github.com/ElesionKyrie/Extreme-Video-Compression-With-Prediction-Using-Pre-trainded-Diffusion-Models- Skip to content Toggle navigation Sign in * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} ElesionKyrie / Extreme-Video-Compression-With-Prediction-Using-Pre-trainded-Diffusion-Models- Public * Notifications * Fork 4 * Star 18 * 18 stars 4 forks Branches Tags Activity Star Notifications * Code * Issues 0 * Pull requests 1 * Actions * Projects 0 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Actions * Projects * Security * Insights ElesionKyrie/ Extreme-Video-Compression-With-Prediction-Using-Pre-trainded-Diffusion-Models- This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main BranchesTags Go to file Code Folders and files Name Name Last commit Last commit message date Latest commit History 58 Commits .idea .idea ELICUtilis/layers ELICUtilis/ layers __pycache__ __pycache__ bench_npy bench_npy benchmark benchmark checkpoints checkpoints configs configs data_npy data_npy evaluation evaluation example_scripts example_scripts fvd_utils fvd_utils losses losses models models result_img result_img ret ret weights weights Inference.py Inference.py Network.py Network.py README.md README.md city_sender.py city_sender.py function.py function.py requirements.txt requirements.txt result_plot.py result_plot.py View all files Repository files navigation * README Extreme Video Compression With Prediction Using Pre-trainded Diffusion Models Usage Prerequisites Python 3.8 and conda, get Conda CUDA if want to use GPU Environment conda create -n $YOUR_PY38_ENV_NAME python=3.8 conda activate $YOUR_PY38_ENV_NAME pip install -r requirements.txt Compress For our project, the input is in the form of an array with a shape of (B, T, C, H, W), where each frame in the array has a fixed size of 128x128. The number of frames in each video is 30, resulting in a shape of (B, 30, 3,128, 128). Before using this project, you may need to preprocess your video data accordingly.In the code, we provide an example array "city_bonn.npy" with a shape of (46, 30, 3, 128, 128). This array contains 46 videos from the city of Bonn in the Cityscape dataset. Below is an example command. You can control which videos to process by choosing the values for start_idx and end_idx. Ensure that the selected range does not exceed the value of B (the number of videos in your dataset). python city_sender.py --data_npy "data_npy/city_bonn.npy" --output_path "your path" --start_idx 0 --end_idx 1 Benchmark In the Benchmark section, we provide code for computing compression metrics for H.264 and H.265. The input for this code should be 30 frames of 128x128 image frames, preferably named in the format "frame%d." the folder structure of dataset is like /your path/ - frame0.png - frame1.png - ... - frame29.png For project_str, this is simply a string used to distinguish your data.Here we are using "uvg." python bench.py --dataset "your path" --output_path "your path" --project_str uvg Checkpoint Regarding the checkpoints, we utilize two sets of them. One set includes "checkpoint_900000.pt," which is used for the video generation part. The other set contains six groups of checkpoints, and these checkpoints will be used for the image compression part, corresponding to six different compression qualities. checkpoints of image compression models The six weights need to be moved to the "checkpoints/neural network" folder. lambda quality 0.45 q5 0.15 q4 0.032 q3 0.015 q2 0.008 q1 0.004 q0 checkpoints of video generation models This individual weight needs to be moved to the "checkpoints/sender" folder. checkpoint of diffusion model checkpoint of diffusion model Model performance chart The following images compare the compression performance of our model with the traditional video compression standards, H.264 and H.265. It can be observed that our model outperforms them at low bitrates (bpp). These data were computed on the first 24 videos from city_bonn.npy. PSNR LPIPS FVD About No description, website, or topics provided. Resources Readme Activity Stars 18 stars Watchers 2 watching Forks 4 forks Report repository Releases No releases published Packages 0 No packages published Languages * Python 90.4% * Shell 7.0% * Cuda 2.3% * C++ 0.3% Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.