https://github.com/facebookresearch/shumai Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code + Explore + All features + Documentation + GitHub Skills + Blog * Solutions + By Plan + Enterprise + Teams + Compare all + By Solution + CI/CD & Automation + DevOps + DevSecOps + Case Studies + Customer Stories + Resources * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles + Repositories + Topics + Trending + Collections * Pricing [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this organization All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} facebookresearch / shumai Public * Notifications * Fork 10 * Star 450 Fast Differentiable Tensor Library in JavaScript and TypeScript with Bun + Flashlight facebookresearch.github.io/shumai License MIT license 450 stars 10 forks Star Notifications * Code * Issues 18 * Pull requests 3 * Actions * Projects 0 * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Security * Insights facebookresearch/shumai This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 5 branches 0 tags Code * Clone HTTPS GitHub CLI [https://github.com/f] Use Git or checkout with SVN using the web URL. [gh repo clone facebo] Work fast with our official CLI. Learn more. * Open with GitHub Desktop * Download ZIP Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @bwasti bwasti add better exception handling for arbitrary exceptions ... 0476fba Oct 17, 2022 add better exception handling for arbitrary exceptions 0476fba Git stats * 183 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .circleci Build Flashlight bindings inline in CI (#35) Sep 28, 2022 .github enable arm64 Oct 5, 2022 docs fix up types (tsc) and improve docs readme Sep 23, 2022 examples add conv2d Sep 30, 2022 scripts add better exception handling for arbitrary exceptions Oct 17, 2022 shumai add better exception handling for arbitrary exceptions Oct 17, 2022 test Implemented TensorVector bindings and sm.concatenate (#71) Oct 17, 2022 .clang-format add clang format file and reformat C++ code Sep 18, 2022 .editorconfig.json Switch to ESLint/Prettier; Update gen_binding.py to Accurately Type T... Aug 29, 2022 .eslintignore ignore docs/ Sep 15, 2022 .eslintrc.cjs improve serve semantics and fix lint errors Sep 13, 2022 .gitignore Create Tensor w dtype matching instance of TypedArray passed to c... Oct 4, 2022 .prettierrc.cjs Add first basic unit test for Tensor Operations (#3) Aug 30, 2022 CMakeLists.txt fix CI after directory change Sep 9, 2022 CODE_OF_CONDUCT.md This commit is the first release of shumai! Aug 27, 2022 CONTRIBUTING.md Switch to ESLint/Prettier; Update gen_binding.py to Accurately Type T... Aug 29, 2022 LICENSE This commit is the first release of shumai! Aug 27, 2022 README.md Windows support (#67) Oct 12, 2022 docs.md Clarify usage of gradients before/after optimizer Oct 13, 2022 package.json 0.0.6 Oct 5, 2022 setup.sh Remove FAIR Cluster-specific paths Aug 27, 2022 tsconfig.json rename index_assign to indexedAssign Sep 16, 2022 View code [ ] Quickstart Why build this? Usage Install Windows Support Building Native Libraries from Source Building on macOS from Source Profiling Building on Linux from Source Contributing License README.md shumai_logo_light shumai_logo_dark A fast differentiable tensor library for research in TypeScript and JavaScript. Built with bun + flashlight. shumai_big [?][?] This is experimental software! [?][?] docs build tests npm Discord GitHub commit activity GitHub --------------------------------------------------------------------- * Usage * Install * Build from source * Contributing * Supported operations Quickstart Install Bun and ArrayFire For MacOS users: You can use Homebrew to install ArrayFire: curl https://bun.sh/install | bash brew install arrayfire For Linux users: If you're running Ubuntu with x86-64, you can use the official distribution: curl https://bun.sh/install | bash sudo apt install -y gnupg2 ca-certificates sudo apt-key adv --fetch-key https://repo.arrayfire.com/GPG-PUB-KEY-ARRAYFIRE-2020.PUB echo "deb https://repo.arrayfire.com/debian all main" | sudo tee /etc/apt/sources.list.d/arrayfire.list sudo apt update sudo apt install -y arrayfire-cpu3-dev arrayfire-cpu3-openblas If you're running Ubuntu with ARMv8, you'll need to build from source: curl https://bun.sh/install | bash sudo apt remove libarrayfire-dev libarrayfire-cpu3 libarrayfire-cpu-dev sudo apt install -y libblas-dev liblapack-dev liblapacke-dev libfftw3-dev libboost-all-dev cmake make g++ cd /tmp sudo rm -rf arrayfire git clone https://github.com/arrayfire/arrayfire.git cd arrayfire cmake -Bbuild -DAF_BUILD_EXAMPLES=OFF -DCMAKE_BUILD_TYPE=Release -DAF_BUILD_UNIFIED=OFF -DAF_TEST_WITH_MTX_FILES=OFF -DBUILD_TESTING=OFF make -j4 -Cbuild sudo make install -Cbuild Otherwise, see the official ArrayFire installation guide. then run: bun install @shumai/shumai Only macOS and Linux are supported. Linux installs default to GPU computation with CUDA, and macOS to CPU. Detailed install instructions below. Install is work in progress: please file an issue if you run into problems. Why build this? With Shumai, we hope to make * Creating datasets + JavaScript, with native typed arrays and a JIT compiler, is perfect for twiddling with data before it can be made into big, flat GPU-compatible arrays. * Training small models + FFI bindings in Bun are crazy fast (~3ns), so JS gets out of the way when training small models * Advanced/fine-grained training/inference logic + Bun uses the JSC JIT compiler, meaning you can confidently write complex training logic without needing a native C++ implementation * Building applications + JavaScript has a [DEL:large:DEL] HUGE ecosystem, which facilitates better application development Usage shumai will always attempt to use an attached GPU or accelerator; although CPU computation will use the ArrayFire CPU backend, which is not well-optimized. We hope to support the ArrayFire OpenCL backend and other non-ArrayFire tensor backends soon. If shumai seems unusually slow, please file an issue! Standard array utilities: import * as sm from "@shumai/shumai" // create a 1024 by 1024 tensor, randomly filled with normal distribution let X = sm.randn([1024, 1024]) let W = sm.identity(1024) let Y = X.matmul(W) console.log(Y.shape) Conversion to and from JavaScript native arrays: const data : Float32Array = new Float32Array(128) for (let i = 0; i < 128; ++i) { data[i] = Math.random() } const X : Tensor = sm.tensor(data) const pi = sm.scalar(3.14) const Y = X.mul(pi) // tensors can be converted back to native JavaScript const Y_data = Y.toFloat32Array() // scalar tensors can be converted to JavaScript numbers const total : number = X.sum().toFloat32() Gradients: const W = sm.randn([128, 128]) W.requires_grad = true const X = sm.randn([128, 128]) const diff = X.sub(W) const mse = diff.mul(diff).sum() mse.backward() W.grad // this gradient is now populated // copy W without allowing gradient updates const Y = W.detach() Y.sum().backward() // nothing changes Some more examples can be found here. Supported operators can be found here. Install The install procedure is a work in progress! If you have any problems building or installing, we would greatly appreciate filed issues. Please tell us about your platform/OS when you do. Prerequisites: * Ensure you have bun installed (https://bun.sh). * Install ArrayFire. macOS users should install ArrayFire's CPU backend; Linux users should install the CUDA backend^. + macOS --- ArrayFire can easily be installed with Homebrew: brew install arrayfire * Linux --- instructions can be found here. On Ubuntu, ArrayFire can be installed via package managers (e.g. apt). Once bun and ArrayFire are installed, install the package and backing libs with bun: bun install @shumai/shumai ^Linux users can use the CPU backend by swapping the required package.json dependency from @shumai/linux_x64_shumai_flashlight to @shumai/linux_x64_shumai_flashlight_cpu, i.e. running: sed -i "s/linux_x64_shumai_flashlight/linux_x64_shumai_flashlight_cpu/g" package.json Windows Support While not officially supported, Windows users have been successful leveraging Docker + WSL2 + Linux. Including CUDA support. Building Native Libraries from Source Note: not required when developing TypeScript/Javascript library components locally. From source build instructions for: * macOS * Linux This process will build the dependent ffi libraries (libflashlight and libflashlight_binding) and pack them using npm pack to generate a @shumai/shumai_*.tgz package. You can then use npm install $PATH_TO_SOURCE/@shumai/shumai-*.tgz to install the package where you'd like. Building on macOS from Source First, install ArrayFire CPU with brew install arrayfire. Build and install Flashlight: mkdir -p $HOME/usr/ # installing flashlight here git clone --recursive --depth 1 https://github.com/flashlight/flashlight.git cd flashlight mkdir -p build cd build cmake .. \ -DCMAKE_BUILD_TYPE=RelWithDebInfo \ # or as specified -DFL_ARRAYFIRE_USE_CPU=ON \ -DFL_ARRAYFIRE_USE_CUDA=OFF \ -DFL_BUILD_DISTRIBUTED=OFF \ -DFL_USE_ONEDNN=OFF \ -DFL_BUILD_TESTS=OFF \ -DFL_BUILD_EXAMPLES=OFF \ -DFL_BUILD_SCRIPTS=OFF \ -DCMAKE_INSTALL_PREFIX=$HOME/usr/ make -j$(nproc) make install Build Flashlight bindings for Shumai: cd shumai mkdir -p build cd build cmake .. -Dflashlight_DIR=$HOME/usr/share/flashlight/cmake/ make -j$(nproc) Profiling On macOS, you can record perf with xcrun xctrace record --template "Time Profiler" --launch $(which bun) train.js. Building on Linux from Source First install ArrayFire. The Linux build for shumai uses the CUDA backend, but from source, you can build the CPU backend as well (OpenCL support coming soon). Build and install Flashlight: mkdir -p $HOME/usr/ # installing flashlight here git clone --recursive --depth 1 https://github.com/flashlight/flashlight.git cd flashlight mkdir -p build cd build cmake .. \ -DCMAKE_BUILD_TYPE=RelWithDebInfo \ # or as specified -DFL_ARRAYFIRE_USE_CPU=OFF \ \ # swap with the above to build for CPU -DFL_ARRAYFIRE_USE_CUDA=ON \ -DFL_BUILD_DISTRIBUTED=OFF \ -DFL_USE_ONEDNN=OFF \ -DFL_BUILD_TESTS=OFF \ -DFL_BUILD_EXAMPLES=OFF \ -DFL_BUILD_SCRIPTS=OFF \ -DCMAKE_INSTALL_PREFIX=$HOME/usr/ make -j$(nproc) make install Build bindings for shumai: mkdir -p build && cd build cmake .. \ -DBUILD_SHARED_LIBS=ON \ -DCMAKE_BUILD_TYPE=RelWithDebInfo \ # or as specified -Dflashlight_DIR=${FLASHLIGHT_INSTALL_PREFIX}/share/flashlight/cmake \ -DArrayFire_DIR=${ARRAYFIRE_INSTALL_PREFIX}/share/ArrayFire/cmake # if built from source, else not needed make -j$(nproc) Contributing If you'd like to make changes to the core bindings or ffi, first build from source. All files ending in *.inl or *_gen.ts are generated. These can be modified by editing scripts/gen_binding.py and running ./scripts/ gen_all_binding.sh. See the CONTRIBUTING file for style guidance and more info on how to help out. License shumai is MIT licensed, as found in the LICENSE file. About Fast Differentiable Tensor Library in JavaScript and TypeScript with Bun + Flashlight facebookresearch.github.io/shumai Resources Readme License MIT license Code of conduct Code of conduct Stars 450 stars Watchers 112 watching Forks 10 forks Releases No releases published Packages 0 No packages published Used by 1 * @raphaelrk @raphaelrk / matrix-mul-test Contributors 7 * @bwasti * @cryptodeal * @jacobkahn * @asilvas * @yushiyangk * @zhiqwang * @Hydrophobefireman Languages * TypeScript 81.4% * C++ 12.9% * Python 4.8% * Other 0.9% Footer (c) 2022 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.