https://github.com/evanwashere/mitata Skip to content Navigation Menu Toggle navigation Sign in * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + GitHub Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions By size + Enterprise + Teams + Startups By industry + Healthcare + Financial services + Manufacturing By use case + CI/CD & Automation + DevOps + DevSecOps * Resources Topics + AI + DevOps + Security + Software Development + View all Explore + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Enterprise + Enterprise platform AI-powered developer platform Available add-ons + Advanced Security Enterprise-grade security features + GitHub Copilot Enterprise-grade AI features + Premium Support Enterprise-grade 24/7 support * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up Reseting focus You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} evanwashere / mitata Public * Notifications You must be signed in to change notification settings * Fork 16 * Star 586 benchmark tooling that loves you [?] License MIT license 586 stars 16 forks Branches Tags Activity Star Notifications You must be signed in to change notification settings * Code * Issues 0 * Pull requests 0 * Actions * Security * Insights Additional navigation options * Code * Issues * Pull requests * Actions * Security * Insights evanwashere/mitata This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. master BranchesTags Go to file Code Folders and files Name Name Last commit message Last commit date Latest commit History 43 Commits examples examples src src tools tools .clangd .clangd .gitignore .gitignore LICENSE LICENSE package.json package.json readme.md readme.md View all files Repository files navigation * README * MIT license mitata benchmark tooling that loves you [?] [68747470733a2f2f63646e2e6576616e2e6c6f6c2f6d6] Install bun add mitata npm install mitata Recommendations * use dedicated hardware for running benchmarks * run with garbage collection enabled (e.g. node --expose-gc ...) * make sure your runtime has high-resolution timers and other relevant options/permissions enabled Quick Start javascript c++ single header import { run, bench, boxplot } from 'mitata'; #include "src/mitata.hpp" function fibonacci(n) { int fibonacci(int n) { if (n <= 1) return n; if (n <= 1) return n; return fibonacci(n - 1) + fibonacci(n - 2); return fibonacci(n - 1) + fibonacci(n - 2); } } bench('fibonacci(40)', () => fibonacci(40)); int main() { mitata::runner runner; boxplot(() => { runner.bench("noop", []() { }); bench('new Array($size)', function* (state) { const size = state.get('size'); runner.summary([&]() { yield () => Array.from({ length: size }); runner.bench("empty fn", []() { }); }).range('size', 1, 1024); runner.bench("fibonacci", []() { fibonacci(20); }); }); }); await run(); auto stats = runner.run(); } configure your experience import { run } from 'mitata'; await run({ format: 'mitata', colors: false }); // default format await run({ filter: /new Array.*/ }) // only run benchmarks that match regex filter await run({ throw: true }); // will immediately throw instead of handling error quietly // c++ auto stats = runner.run({ .colors = true, .format = "json", .filter = std::regex(".*") }); automatic garbage collection On runtimes that expose gc (e.g. bun, node --expose-gc ...), mitata will automatically run garbage collection before each benchmark. universal compatibility Out of box mitata can detect engine/runtime it's running on and fall back to using alternative non-standard I/O functions. If your engine or runtime is missing support, open an issue or pr requesting for support. how to use mitata with engine CLIs like d8, jsc, graaljs, spidermonkey $ xs bench.mjs $ quickjs bench.mjs $ d8 --expose-gc bench.mjs $ spidermonkey -m bench.mjs $ graaljs --js.timer-resolution=1 bench.mjs $ /System/Library/Frameworks/JavaScriptCore.framework/Versions/Current/Helpers/jsc bench.mjs // bench.mjs import { print } from './src/lib.mjs'; import { run, bench } from './src/main.mjs'; // git clone import { run, bench } from './node_modules/mitata/src/main.mjs'; // npm install print('hello world'); // works on every engine argumentizing your benchmarks has never been so easy With other benchmarking libraries, often it's quite hard to easily make benchmarks that go over a range or run the same function with different arguments without writing spaghetti code, but now with mitata converting your benchmark to use arguments is just a function call away. import { bench } from 'mitata'; bench(function* look_mom_no_spaghetti(state) { const len = state.get('len'); const len2 = state.get('len2'); yield () => new Array(len * len2); }) .args('len', [1, 2, 3]) .range('len', 1, 1024) // 1, 8, 64, 512... .dense_range('len', 1, 100) // 1, 2, 3 ... 99, 100 .args({ len: [1, 2, 3], len2: ['4', '5', '6'] }) // every possible combination helpful warnings For those who love doing micro-benchmarks, mitata can automatically detect and inform you about optimization passes like dead code elimination without requiring any special engine flags. -------------------------------------- ------------------------------- 1 + 1 318.63 ps/iter 325.37 ps # # ! (267.92 ps ... 14.28 ns) 382.81 ps ##################### empty function 319.36 ps/iter 325.37 ps # # ! (248.62 ps ... 46.61 ns) 382.81 ps ##################### ! = benchmark was likely optimized out (dead code elimination) powerful visualizations right in your terminal with mitata's ascii rendering capabilities, now you can easily visualize samples in barplots, boxplots, lineplots, histograms, and get clear summaries without any additional tools or dependencies. -------------------------------------- ------------------------------- 1 + 1 318.11 ps/iter 325.37 ps # # ! (267.92 ps ... 11.14 ns) 363.97 ps ##################### Date.now() 27.69 ns/iter 27.48 ns # (27.17 ns ... 44.10 ns) 32.74 ns ##################### + + 1 + 1 +# 318.11 ps Date.now() +################################## 27.69 ns + + -------------------------------------- ------------------------------- Bubble Sort 2.11 ms/iter 2.26 ms # (1.78 ms ... 6.93 ms) 4.77 ms ##################### Quick Sort 159.60 us/iter 154.50 us # (133.13 us ... 792.21 us) 573.00 us ##################### Native Sort 97.20 us/iter 97.46 us ## (90.88 us ... 688.92 us) 105.00 us ##################### + + |+-+-+ | Bubble Sort ++ | +-----------------------+ |+-+-+ | + | Quick Sort |---+ + | + Native Sort | + + + 90.88 us 2.43 ms 4.77 ms -------------------------------------- ------------------------------- new Array(1) 3.57 ns/iter 3.20 ns 6.64 ns ########## new Array(8) 5.21 ns/iter 4.31 ns 8.85 ns ########## new Array(64) 17.94 ns/iter 13.40 ns 171.89 ns ########## new Array(512) 188.05 ns/iter 246.88 ns 441.81 ns ########## new Array(1024) 364.93 ns/iter 466.91 ns 600.34 ns ########## Array.from(1) 29.73 ns/iter 29.24 ns 36.88 ns ########## Array.from(8) 33.96 ns/iter 32.99 ns 42.45 ns ########## Array.from(64) 146.52 ns/iter 143.82 ns 310.93 ns ########## Array.from(512) 1.11 us/iter 1.18 us 1.34 us ########## Array.from(1024) 1.98 us/iter 2.09 us 2.40 us ########## summary new Array($len) 5.42...8.33x faster than Array.from($len) + + Array.from($size) [d68]i new Array($size) [d8]9a [d67]b [d8]s [d357]a [d67]i [d8]> [d67]b [d357]a [d8]s [d67]b [d8]> [d68]i [d78][d78]--3 [d567]a [d78][d67]-93ic [d78][d78][d78]-> [d78][d67]-3ic [d3678][d3678][d3678][d3678][d3678][d3678][d3678][d3678][d3678][d3678][d3678][d3678][d3578][d2578][d2578][d2478][d1478]x-----3ic + + give your own code power of mitata In case you don't need all the fluff that comes with mitata or just need raw results, mitata exports its fundamental building blocks to allow you to easily build your own tooling and wrappers without losing any core benefits of using mitata. #include "src/mitata.hpp" int main() { auto stats = mitata::lib::fn([]() { /***/ }) } import { B, measure } from 'mitata'; // lowest level for power users const stats = await measure(function* (state) { const size = state.get('x'); yield () => new Array(size); }, { args: { x: 1 }, batch_samples: 5 * 1024, min_cpu_time: 1000 * 1e6, }); // explore how magic happens console.log(stats.debug) // -> jit optimized source code of benchmark // higher level api that includes mitata's argument and range features const b = new B('new Array($x)', state => { const size = state.get('x'); for (const _ of state) new Array(size); }).args('x', [1, 5, 10]); const trial = await b.run(); accuracy down to picoseconds mitata pushes the limits of javascript with jit-generated zero-overhead measurement loops to provide high-resolution timings. This allows providing features like cpu clock frequency and dead code detection without requiring access outside the js sandbox. clk: ~3.13 GHz cpu: Apple M2 Pro runtime: node 22.8.0 (arm64-darwin) benchmark avg (min ... max) p75 p99 (min ... top 1%) -------------------------------------- ------------------------------- noop 93.09 ps/iter 91.55 ps # ! (61.04 ps ... 20.30 ns) 101.81 ps ##################### ! = benchmark was likely optimized out (dead code elimination) // vs other libraries 16041.00 ns/iter - node:perf_hooks (performance.timerify) 5.30 ns/iter - https://npmjs.com/benchmark noop x 188,640,251 ops/sec +-5.71% (73 runs sampled) 36.62 ns/iter - https://npmjs.com/tinybench +---------+-----------+--------------+-------------------+----------+----------+ | (index) | Task Name | ops/sec | Average Time (ns) | Margin | Samples | +---------+-----------+--------------+-------------------+----------+----------+ | 0 | 'noop' | '27,308,739' | 36.61831406333669 | '+-0.14%' | 13654370 | +---------+-----------+--------------+-------------------+----------+----------+ 156.5685 ns/iter - https://npmjs.com/cronometro +--------------+---------+-------------------+-----------+ | Slower tests | Samples | Result | Tolerance | +--------------+---------+-------------------+-----------+ | Fastest test | Samples | Result | Tolerance | +--------------+---------+-------------------+-----------+ | noop | 10000 | 6386980.78 op/sec | +- 1.85 % | +--------------+---------+-------------------+-----------+ same test with v8 jit compiler disabled: clk: ~0.06 GHz cpu: Apple M2 Pro runtime: node 22.8.0 (arm64-darwin) benchmark avg (min ... max) p75 p99 (min ... top 1%) -------------------------------------- ------------------------------- noop 14.69 ns/iter 15.09 ns ######### ! (13.33 ns ... 19.69 ns) 16.24 ns ##################### ! = benchmark was likely optimized out (dead code elimination) // vs other libraries 17500.00 ns/iter - node:perf_hooks (performance.timerify) 23.28 ns/iter - https://npmjs.com/benchmark noop x 42,952,144 ops/sec +-0.87% (98 runs sampled) 184.92 ns/iter - https://npmjs.com/tinybench +---------+-----------+-------------+--------------------+----------+---------+ | (index) | Task Name | ops/sec | Average Time (ns) | Margin | Samples | +---------+-----------+-------------+--------------------+----------+---------+ | 0 | 'noop' | '5,407,742' | 184.92003948393378 | '+-0.02%' | 2703872 | +---------+-----------+-------------+--------------------+----------+---------+ 659.9353333333333 ns/iter - https://npmjs.com/cronometro +--------------+---------+-------------------+-----------+ | Slower tests | Samples | Result | Tolerance | +--------------+---------+-------------------+-----------+ | Fastest test | Samples | Result | Tolerance | +--------------+---------+-------------------+-----------+ | noop | 1500 | 1515299.98 op/sec | +- 0.72 % | +--------------+---------+-------------------+-----------+ License MIT (c) Evan About benchmark tooling that loves you [?] Topics nodejs javascript benchmark library performance node v8 cpp spidermonkey bun single-header jsc deno graaljs microbenchmark Resources Readme License MIT license Activity Stars 586 stars Watchers 3 watching Forks 16 forks Report repository Releases 1 mitata v1 - stable release Latest Sep 28, 2024 Used by 416 * @kerrickchan * @tmg0 * @mapljs * @flo-bit * @lobomfz * @utftu * @hironow * @kyscott18 + 408 Contributors 4 * * * * Languages * JavaScript 57.2% * C++ 35.0% * TypeScript 7.8% Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.