https://github.com/HigherOrderCO/Bend Skip to content Navigation Menu Toggle navigation Sign in * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} HigherOrderCO / Bend Public * Notifications * Fork 65 * Star 2.9k * A massively parallel, high-level programming language higherorderco.com License Apache-2.0 license 2.9k stars 65 forks Branches Tags Activity Star Notifications * Code * Issues 6 * Pull requests 4 * Actions * Projects 0 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Actions * Projects * Security * Insights HigherOrderCO/Bend This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main BranchesTags Go to file Code Folders and files Name Name Last commit Last commit message date Latest commit History 1,550 Commits .github .github docs docs examples examples src src tests tests .clippy.toml .clippy.toml .gitignore .gitignore .rustfmt.toml .rustfmt.toml Cargo.lock Cargo.lock Cargo.toml Cargo.toml FEATURES.md FEATURES.md GUIDE.md GUIDE.md LICENSE-APACHE LICENSE-APACHE README.md README.md cspell.json cspell.json justfile justfile rust-toolchain.toml rust-toolchain.toml View all files Repository files navigation * README * Apache-2.0 license Bend Bend is a massively parallel, high-level programming language. Unlike low-level alternatives like CUDA and Metal, Bend has the feeling and features of expressive languages like Python and Haskell, including fast object allocations, higher-order functions with full closure support, unrestricted recursion, even continuations. Yet, it runs on massively parallel hardware like GPUs, with near-linear speedup based on core count, and zero explicit parallel annotations: no thread spawning, no locks, mutexes, atomics. Bend is powered by the HVM2 runtime. A Quick Demo Bend live demo Using Bend Currently not working on Windows, please use WSL2 as a workaround. First, install Rust nightly. Then, install both HVM2 and Bend with: cargo +nightly install hvm cargo +nightly install bend-lang Finally, write some Bend file, and run it with one of these commands: bend run # uses the Rust interpreter (sequential) bend run-c # uses the C interpreter (parallel) bend run-cu # uses the CUDA interpreter (massively parallel) You can also compile Bend to standalone C/CUDA files with gen-c and gen-cu, for maximum performance. But keep in mind our code gen is still on its infancy, and is nowhere as mature as SOTA compilers like GCC and GHC. Parallel Programming in Bend To write parallel programs in Bend, all you have to do is... nothing. Other than not making it inherently sequential! For example, the expression: (((1 + 2) + 3) + 4) Can not run in parallel, because +4 depends on +3 which depends on (1+2). But the following expression: ((1 + 2) + (3 + 4)) Can run in parallel, because (1+2) and (3+4) are independent; and it will, per Bend's fundamental pledge: Everything that can run in parallel, will run in parallel. For a more complete example, consider: # Sorting Network = just rotate trees! def sort(d, s, tree): switch d: case 0: return tree case _: (x,y) = tree lft = sort(d-1, 0, x) rgt = sort(d-1, 1, y) return rots(d, s, lft, rgt) # Rotates sub-trees (Blue/Green Box) def rots(d, s, tree): switch d: case 0: return tree case _: (x,y) = tree return down(d, s, warp(d-1, s, x, y)) (...) This file implements a bitonic sorter with just tree rotations. It is not the kind of algorithm you'd expect to run fast on GPUs. Yet, since it uses a divide-and-conquer approach, which is inherently parallel, Bend will run it multi-threaded. Some benchmarks: * CPU, Apple M3 Max, 1 thread: 12.15 seconds * CPU, Apple M3 Max, 16 threads: 0.96 seconds * GPU, NVIDIA RTX 4090, 16k threads: 0.21 seconds That's a 57x speedup by doing nothing. No thread spawning, no explicit management of locks, mutexes. We just asked bend to run our program on RTX, and it did. Simple as that. Bend isn't limited to a specific paradigm, like tensors or matrices. Any concurrent system, from shaders to Erlang-like actor models can be emulated on Bend. For example, to render images in real time, we could simply allocate an immutable tree on each frame: # given a shader, returns a square image def render(depth, shader): bend d = 0, i = 0: when d < depth: color = (fork(d+1, i*2+0), fork(d+1, i*2+1)) else: width = depth / 2 color = shader(i % width, i / width) return color # given a position, returns a color # for this demo, it just busy loops def demo_shader(x, y): bend i = 0: when i < 5000: color = fork(i + 1) else: color = 0x000001 return color # renders a 256x256 image using demo_shader def main: return render(16, demo_shader) And it would actually work. Even involved algorithms, such as a Bitonic Sort using tree rotations, parallelize well on Bend. Long-distance communication is performed by global beta-reduction (as per the Interaction Calculus), and synchronized correctly and efficiently by HVM2's atomic linker. * To jump straight into action, check Bend's GUIDE.md. * For an extensive list of features, check FEATURES.md. * To understand the tech behind Bend, check HVM2's paper. * Bend is developed by HigherOrderCO.com - join our Discord! About A massively parallel, high-level programming language higherorderco.com Resources Readme License Apache-2.0 license Activity Custom properties Stars 2.9k stars Watchers 34 watching Forks 65 forks Report repository Releases 5 tags Packages 0 No packages published Contributors 17 * @developedby * @imaqtkatt * @LunaAmora * @VictorTaelin * @tjjfvi * @FranchuFranchu * @kings177 * @KPCOFGS * @Sipher * @Janiczek * @eduhenke * @TheMhv * @edusporto * @davisuga + 3 contributors Languages * Rust 99.8% * Just 0.2% Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.