https://github.com/joennlae/halutmatmul Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} joennlae / halutmatmul Public * Notifications * Fork 7 * Star 123 Hashed Lookup Table based Matrix Multiplication (halutmatmul) - Stella Nera accelerator License MIT license 123 stars 7 forks Activity Star Notifications * Code * Issues 0 * Pull requests 0 * Actions * Projects 0 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Actions * Projects * Security * Insights joennlae/halutmatmul This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. master Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 8 branches 3 tags Code * Local * Codespaces * Clone HTTPS GitHub CLI [https://github.com/j] Use Git or checkout with SVN using the web URL. [gh repo clone joennl] Work fast with our official CLI. Learn more about the CLI. * Open with GitHub Desktop * Download ZIP Sign In Required Please sign in to use Codespaces. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @joennlae joennlae docs: update citation ... 1228106 Nov 20, 2023 docs: update citation 1228106 Git stats * 775 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .github chore: update python linting November 15, 2023 11:19 .vscode refactor: update vscode config November 15, 2023 11:19 docs docs: update readme November 15, 2023 11:19 hardware refactor: remove unused files November 15, 2023 11:19 src/python feat(resnet9): update accuracy to 92.6% November 15, 2023 11:19 .editorconfig ci(mypy): add python typechecking March 31, 2022 02:24 .gitignore chore: update .gitignore November 15, 2023 11:19 .gitlint style: add commit lint March 6, 2022 17:08 .gitmodules feat(openroad): add OpenROAD as submodule again September 5, 2023 10:33 .pre-commit-config.yaml refactor: further maddness cleanup September 20, 2023 13:14 CHANGELOG.md docs: add changelog September 5, 2023 10:33 CITATION.cff docs: update citation November 20, 2023 11:05 Dockerfile chore: fix typo in dockerfile September 5, 2023 10:33 LICENSE.txt docs: add bolt license + MIT license March 8, 2022 19:37 README.md docs: update abstract November 20, 2023 10:34 environment_cpu.yml chore(env): add tensorboard to env October 11, 2023 09:00 environment_gpu.yml chore: update env and fix pytorch version to 2.0 November 15, 2023 11:19 halut refactor: further maddness cleanup September 20, 2023 13:14 halut.env chore: add iis internal tooling solution September 5, 2023 10:33 pyproject.toml refactor: further maddness cleanup September 20, 2023 13:14 View code [ ] Stella Nera - Halutmatmul Algorithmic CI ML CI Hardware CI Paper Abstract Algorithmic - Maddness Differentiable Maddness ResNet-9 LUTs, Thresholds, Dims Halutmatmul example Hardware - OpenROAD flow results from CI Full design (halutmatmul) Full Design Encoder Decoder Install References Hackernews mention (comments only) and discussion README.md Stella Nera - Halutmatmul Algorithmic CI PyTorch Layer Test | PyTest Python Linting Mypy - Typechecking ML CI ResNet9 - 92%+ accuracy Hardware CI HW Synth + PAR OpenROAD RTL Linting HW Design Verification Paper * Stella Nera: Achieving 161 TOp/s/W with Multiplier-free DNN Acceleration based on Approximate Matrix Multiplication Abstract The recent Maddness method approximates Matrix Multiplication (MatMul) without the need for multiplication by using a hash-based version of product quantization (PQ). The hash function is a decision tree, allowing for efficient hardware implementation, as multiply-accumulate operations are replaced by decision tree passes and LUT lookups. Stella Nera is the first Maddness accelerator achieving 15x higher area efficiency (GMAC/s/mm^2) and 25x higher energy efficiency (TMAC/s/W) than direct MatMul accelerators in the same technology. In a commercial 14 nm technology and scaled to 3 nm, we achieve an energy efficiency of 161 TOp/s/W@0.55V with a Top-1 accuracy on CIFAR-10 of over 92.5% using ResNet9. Algorithmic - Maddness Maddness Animation Differentiable Maddness Differentiable Maddness ResNet-9 LUTs, Thresholds, Dims * Download 92%+ Model Halutmatmul example * example.py import numpy as np from halutmatmul.halutmatmul import HalutMatmul A = np.random.random((10000, 512)) A_train = A[:8000] A_test = A[8000:] B = np.random.random((512, 10)) C = np.matmul(A_test, B) hm = HalutMatmul(C=32, K=16) hm.learn_offline(A_train, B) C_halut = hm.matmul_online(A_test) mse = np.square(C_halut - C).mean() print(mse) Hardware - OpenROAD flow results from CI All NanGate45 results are NOT OPTIMIZED! The results are only for reference and to show the flow works. All Designs NanGate45 All Report All History History Full design (halutmatmul) Run locally with: git submodule update --init --recursive cd hardware ACC_TYPE=INT DATA_WIDTH=8 NUM_M=8 NUM_DECODER_UNITS=4 NUM_C=16 make halut-open-synth-and-pnr-halut_matmul Full Design halut_matmul NanGate45 Area [mm^2] 128816 Freq [Mhz] 166.7 GE 161.423 kGE Std Cell [#] 65496 Voltage [V] 1.1 Util [%] 50.4 TNS 0 Clock Net Clock_net Routing Routing GDS GDS Download Encoder halut_encoder_4 NanGate45 Area [mm^2] 46782 Freq [Mhz] 166.7 GE 58.624 kGE Std Cell [#] 23130 Voltage [V] 1.1 Util [%] 48.7 TNS 0 Clock Net Clock_net Routing Routing GDS GDS Download Decoder halut_decoder NanGate45 Area [mm^2] 24667.5 Freq [Mhz] 166.7 GE 30.911 kGE Std Cell [#] 12256 Voltage [V] 1.1 Util [%] 52.1 TNS 0 Clock Net Clock_net Routing Routing GDS GDS Download Install # install conda environment & activate # mamba is recommended for faster install conda env create -f environment_gpu.yml conda activate halutmatmul # IIS prefixed env conda env create -f environment_gpu.yml --prefix /scratch/janniss/conda/halutmatmul_gpu References * arXiv Maddness paper * Based on MADDness/Bolt. Hackernews mention (comments only) and discussion * HN: Bolt: Faster matrix and vector operations that run on compressed data About Hashed Lookup Table based Matrix Multiplication (halutmatmul) - Stella Nera accelerator Topics machine-learning hardware pytorch hardware-acceleration approximate-inference maddness Resources Readme License MIT license Activity Stars 123 stars Watchers 8 watching Forks 7 forks Report repository Releases 3 tags Packages 3 Languages * Python 85.7% * SystemVerilog 10.0% * Makefile 2.5% * Other 1.8% Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time.