https://github.com/cyrusvahidi/jtfs-gpu Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Case Studies + Customer Stories + Resources * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this user All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} cyrusvahidi / jtfs-gpu Public * Notifications * Fork 2 * Star 41 code for the paper "Differentiable Time-Frequency Scattering on GPU" cyrusvahidi.github.io/jtfs-gpu/ License MIT license 41 stars 2 forks Star Notifications * Code * Issues 1 * Pull requests 0 * Actions * Projects 0 * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Security * Insights cyrusvahidi/jtfs-gpu This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 3 branches 0 tags Code * Local * Codespaces * Clone HTTPS GitHub CLI [https://github.com/c] Use Git or checkout with SVN using the web URL. [gh repo clone cyrusv] Work fast with our official CLI. Learn more about the CLI. * Open with GitHub Desktop * Download ZIP Sign In Required Please sign in to use Codespaces. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @cyrusvahidi cyrusvahidi cleanup ... 32e2b5d Jan 27, 2023 cleanup 32e2b5d Git stats * 395 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time docs ... September 8, 2022 19:02 kymatio kymatio repo September 8, 2022 09:06 kymjtfs cleanup January 27, 2023 17:59 notebooks cleanup January 27, 2023 17:58 scripts cleanup January 27, 2023 17:58 strf-like-model doce March 24, 2022 09:16 .gitignore added ignor September 8, 2022 12:01 LICENSE.md Create LICENSE.md June 2, 2022 16:50 README.md Update README.md December 26, 2022 12:07 requirements.txt added ignor September 8, 2022 12:01 setup.py setup.py December 31, 2022 13:31 View code [ ] Differentiable Time-Frequency Scattering on GPU source code by How to run Installation ConvNet Classifier Extract Scattering Features Configure gin Run training Isomap Visualizations & K-NN Regression Scale-Rate Visualizations and Resynthesis Notebook Guide Scale-Rate Visualisations Synthetic amplitude-modulated chirp dataset Manifold Embedding of the Nearest Neighbour Graph K-NN regression of synthesizer parameters 2-D CNN classifier Differentiable Resynthesis Citation README.md Differentiable Time-Frequency Scattering on GPU source code by Cyrus Vahidi^2 Changhong Wang^1, Han Han^1 Vincent Lostanlen^1 John Muradeli LS2N/CNRS Centrale Nantes^1 Queen Mary University of London^2 Paper Accompanying webpage Kymatio: open-source wavelet scattering in Python Many thanks to all open-source contributors to Kymatio and its dependencies. This repository contains code to replicate the paper "Differentiable Time-Frequency Scattering on GPU" (published at DAFx 2022, best paper award). Time-frequency scattering is available in Kymatio in beta and will be released in v0.4. To use this implementation, install Kymatio from source. To replicate the results in this paper, follow the installation instructions below. We assess Time-Frequency Scattering in Kymatio for 3 machine listening research applications: * unsupervised manifold learning of spectrotemporal modulations * hybrid jtfs + convnet supervised musical instrument classification * texture resynthesis How to run First, install dependencies Installation # clone project git clone https://github.com/cyrusvahidi/jtfs-gpu # install project cd jtfs-gpu pip install -e . pip install -r requirements.txt # install kymatio from source cd kymatio python setup.py develop * The JTFS algorithm source code to replicate the paper can be found here * The latest version of JTFS can be installed directly from the Kymatio source ConvNet Classifier Download Medley-solos-DB Extract Scattering Features python scripts/process_msdb_features.py --data_dir --feature Configure gin In /scripts/gin/config.gin set MSDB_DATA_DIR and MSDB_CSV according to the absolute path of your MSDB download. Run training python scripts/train_cnn.py Isomap Visualizations & K-NN Regression python scripts/isomap.py see the output in /img Scale-Rate Visualizations and Resynthesis cd notebooks jupyter notebook See Scale-Rate Visualization.ipynb and Resynthesis results.ipynb Notebook Guide Scale-Rate Visualisations H E L L O Synthetic amplitude-modulated chirp dataset * Factors of variation: * $f_c$ carrier frequency * $f_m$ amplitude modulation frequency * $\gamma$ chirp rate Manifold Embedding of the Nearest Neighbour Graph * MFCCs * Time Scattering * Time-Frequency Scattering * Open-L3 * Spectrotemporal Receptive Field (STRF) K-NN regression of synthesizer parameters 2-D CNN classifier Differentiable Resynthesis Citation @article{muradeli2022differentiable, title={Differentiable Time-Frequency Scattering in Kymatio}, author={John Muradeli, Cyrus Vahidi, Changhong Wang, Han Han, Vincent Lostanlen, Mathieu Lagrange, George Fazekas}, journal={arXiv preprint arXiv:2204.08269}, year={2022} } About code for the paper "Differentiable Time-Frequency Scattering on GPU" cyrusvahidi.github.io/jtfs-gpu/ Resources Readme License MIT license Stars 41 stars Watchers 4 watching Forks 2 forks Report repository Releases No releases published Packages 0 No packages published Contributors 4 * @cyrusvahidi cyrusvahidi Cyrus Vahidi * @OverLordGoldDragon OverLordGoldDragon John Muradeli * @lylyhan lylyhan Han Han * @lostanlen lostanlen Vincent Lostanlen Languages * Jupyter Notebook 95.9% * Python 4.1% Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.