https://github.com/DachunKai/EvTexture Skip to content Navigation Menu Toggle navigation Sign in * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + GitHub Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions By size + Enterprise + Teams + Startups By industry + Healthcare + Financial services + Manufacturing By use case + CI/CD & Automation + DevOps + DevSecOps * Resources Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Enterprise + Enterprise platform AI-powered developer platform Available add-ons + Advanced Security Enterprise-grade security features + GitHub Copilot Enterprise-grade AI features + Premium Support Enterprise-grade 24/7 support * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} DachunKai / EvTexture Public * Notifications You must be signed in to change notification settings * Fork 32 * Star 619 [ICML 2024] EvTexture: Event-driven Texture Enhancement for Video Super-Resolution dachunkai.github.io/evtexture.github.io/ License Apache-2.0 license 619 stars 32 forks Branches Tags Activity Star Notifications You must be signed in to change notification settings * Code * Issues 9 * Pull requests 0 * Actions * Projects 0 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Actions * Projects * Security * Insights DachunKai/EvTexture This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main BranchesTags Go to file Code Folders and files Name Name Last commit Last commit message date Latest commit History 36 Commits basicsr basicsr datasets datasets docker docker options options scripts scripts .gitignore .gitignore LICENSE LICENSE README.md README.md VERSION VERSION requirements.txt requirements.txt setup.cfg setup.cfg setup.py setup.py View all files Repository files navigation * README * Apache-2.0 license EvTexture (ICML 2024) Official Pytorch implementation for the "EvTexture: Event-driven Texture Enhancement for Video Super-Resolution" paper (ICML 2024). Project | Paper | [?] Poster Authors: Dachun Kai^[?], Jiayao Lu, Yueyi Zhang^[?], Xiaoyan Sun, University of Science and Technology of China Feel free to ask questions. If our work helps, please don't hesitate to give us a ! News * [ ] Release training code * [*] 2024/06/28: Release details to prepare datasets * [*] 2024/06/08: Publish docker image * [*] 2024/06/08: Release pretrained models and test sets for quick testing * [*] 2024/06/07: Video demos released * [*] 2024/05/25: Initialize the repository * [*] 2024/05/02: Our paper was accepted in ICML'2024 Table of Content 1. Video Demos 2. Code 3. Citation 4. Contact 5. License and Acknowledgement Video Demos A $4\times$ upsampling results on the Vid4 and REDS4 test sets. Vid4_City.mp4 Vid4_Foliage.mp4 REDS_000.mp4 REDS_011.mp4 Code Installation * Dependencies: Miniconda, CUDA Toolkit 11.1.1, torch 1.10.2+cu111, and torchvision 0.11.3+cu111. * Run in Conda conda create -y -n evtexture python=3.7 conda activate evtexture pip install torch-1.10.2+cu111-cp37-cp37m-linux_x86_64.whl pip install torchvision-0.11.3+cu111-cp37-cp37m-linux_x86_64.whl git clone https://github.com/DachunKai/EvTexture.git cd EvTexture && pip install -r requirements.txt && python setup.py develop * Run in Docker Note: before running the Docker image, make sure to install nvidia-docker by following the official instructions. [Option 1] Directly pull the published Docker image we have provided from Alibaba Cloud. docker pull registry.cn-hangzhou.aliyuncs.com/dachunkai/evtexture:latest [Option 2] We also provide a Dockerfile that you can use to build the image yourself. cd EvTexture && docker build -t evtexture ./docker The pulled or self-built Docker image containes a complete conda environment named evtexture. After running the image, you can mount your data and operate within this environment. source activate evtexture && cd EvTexture && python setup.py develop Test 1. Download the pretrained models from (Releases / Onedrive / Google Drive / Baidu Cloud(n8hg)) and place them to experiments/ pretrained_models/EvTexture/. The network architecture code is in evtexture_arch.py. + EvTexture_REDS_BIx4.pth: trained on REDS dataset with BI degradation for $4\times$ SR scale. + EvTexture_Vimeo90K_BIx4.pth: trained on Vimeo-90K dataset with BI degradation for $4\times$ SR scale. 2. Download the preprocessed test sets (including events) for REDS4 and Vid4 from (Releases / Onedrive / Google Drive / Baidu Cloud (n8hg)), and place them to datasets/. + Vid4_h5: HDF5 files containing preprocessed test datasets for Vid4. + REDS4_h5: HDF5 files containing preprocessed test datasets for REDS4. 3. Run the following command: + Test on Vid4 for 4x VSR: ./scripts/dist_test.sh [num_gpus] options/test/EvTexture/test_EvTexture_Vid4_BIx4.yml + Test on REDS4 for 4x VSR: ./scripts/dist_test.sh [num_gpus] options/test/EvTexture/test_EvTexture_REDS4_BIx4.yml This will generate the inference results in results/. The output results on REDS4 and Vid4 can be downloaded from ( Releases / Onedrive / Google Drive / Baidu Cloud(n8hg)). Data Preparation * Both video and event data are required as input, as shown in the snippet. We package each video and its event data into an HDF5 file. * Example: The structure of calendar.h5 file from the Vid4 dataset is shown below. calendar.h5 +-- images | +-- 000000 # frame, ndarray, [H, W, C] | +-- ... +-- voxels_f | +-- 000000 # forward event voxel, ndarray, [Bins, H, W] | +-- ... +-- voxels_b | +-- 000000 # backward event voxel, ndarray, [Bins, H, W] | +-- ... * To simulate and generate the event voxels, refer to the dataset preparation details in DataPreparation.md. Inference on your own video [?] Seeking Collaboration: For issues #6 and #7, our method can indeed perform inference on videos without event data. The solution is to use an event camera simulator, such as vid2e, to generate event data from the video, and then input both the video data and the generated event data into our model. This part, however, may require extensive engineering work to package everything into a script, as detailed in DataPreparation.md. We currently do not have enough time to undertake this task, so we are looking for collaborators to join us in this effort! Citation If you find the code and pre-trained models useful for your research, please consider citing our paper. @inproceedings{kai2024evtexture, title={Ev{T}exture: {E}vent-driven {T}exture {E}nhancement for {V}ideo {S}uper-{R}esolution}, author={Kai, Dachun and Lu, Jiayao and Zhang, Yueyi and Sun, Xiaoyan}, booktitle={International Conference on Machine Learning}, year={2024}, organization={PMLR} } Contact If you meet any problems, please describe them in issues or contact: * Dachun Kai: dachunkai@mail.ustc.edu.cn License and Acknowledgement This project is released under the Apache-2.0 license. Our work is built upon BasicSR, which is an open source toolbox for image/video restoration tasks. Thanks to the inspirations and codes from RAFT and event_utils. About [ICML 2024] EvTexture: Event-driven Texture Enhancement for Video Super-Resolution dachunkai.github.io/evtexture.github.io/ Topics video-super-resolution video-restoration event-camera Resources Readme License Apache-2.0 license Activity Stars 619 stars Watchers 11 watching Forks 32 forks Report repository Releases 1 Pretrained models, test sets and visual results Latest Jun 19, 2024 Packages 0 No packages published Contributors 2 * * Languages * Python 76.6% * Cuda 13.5% * C++ 9.1% * Other 0.8% Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.