https://blog.janestreet.com/how-we-accidentally-built-a-better-build-system-for-ocaml-index/ Jane Street logo * Home * All Posts * Authors Index * Jane Street * Subscribe to email updates Subscribe to Email Updates [ ] [ ] [Subscribe] * Featured Reads + [applying-t] ------------------------------------------------------------- Read more ------------------------------------------------------------- Posts about interviewing at Jane Street and our internship program + [scientist_] ------------------------------------------------------------- Read more ------------------------------------------------------------- Using ASCII waveforms to test hardware designs + [memory-lea] ------------------------------------------------------------- Read more ------------------------------------------------------------- Finding memory leaks with Memtrace * Tags + async + book + c + camlp4 + code-review + comments + compiler + core + hackerschool + hg + incremental + internship + interviewing + ocaml + parallel-programming + performance + ppx + real-world-ocaml + registers + speed + ui * Get the RSS feed * Jane Street Open Source * Join Our Team Jane Street logo [ ] How we accidentally built a better build system for OCaml Jan 24, 2025 | 6 min read * Share on Facebook * Share on Twitter * Share on LinkedIn dune-jenga.png amokhov avatar By: Andrey Mokhov A "build system" is one of the most important tools in a developer's toolbox. Roughly, it figures out how to create runnable programs from a bunch of different source files by calling out to the compiler, setting up and executing test suites, and so on. Because you interact with it daily, above all it has to be fast - but it also has to be flexible. Around 2012 we were growing dissatisfied with OMake, then one of OCaml's standard build systems, and decided to build our own; we called this new system Jenga. It worked quite well for us, and we thought the broader community might find it useful. So we decided to release Jenga. We hoped that when other people tried it, they'd like it, and maybe even contribute back to it. Releasing it would also make it easier for us to open source our code. Ha! What actually happened is that nobody really wanted to use Jenga. For one thing it didn't work on Windows. But also, to adopt Jenga was in effect to adopt the whole "Jane Street way" of building OCaml. The adoption of Jenga by those we hoped to embrace it was weak enough that we actually decided to un-open source it. And so we were back to the same place as before. By 2016 we had had enough of this, and decided to make a simple cross-platform tool, called Jbuilder, that would allow external users to build our code without having to adopt Jenga in full, and would release us from the obligation of rewriting our builds in OCamlbuild, then an emerging standard for building OCaml projects. Jbuilder understood the jbuild files that Jenga used for build configuration and simply executed all required compilation commands in a topological order. It wasn't a build system in the usual sense: it would simply re-execute all commands every time (instead of only re-executing commands whose inputs have changed). Jbuilder gets popular and becomes "Dune" Then something strange happened. People loved Jbuilder. They started using it to build not just our packages but their own, too. At first we didn't really understand this. Jbuilder wasn't a real build system, after all. It was just meant to be a little compatibility shim. What we realized, eventually, was that the compelling feature was speed. It turned out that Jbuilder was really a lot faster than the other options, compiling our projects something like 5x faster than OCamlbuild. That, plus the system being portable and easy to hack on, were the things that mattered for early adopters and contributors. So, in collaboration with OCaml Labs (and today, Tarides), we started working on making Jbuilder into more of a real build system, adding more of the features that would be required for it to be a useful tool for the broader open-source world. And then, we ran into another problem. The name. It turned out there was already a Borland Java IDE called "JBuilder." The system was long defunct, and we even went to the trouble of finding the current owners of the copyright and asking them if they'd mind us using the name. But, no dice. So we decided to pick a new name. We did a bit of community outreach, and "Dune" emerged as the winning name. In the meantime, Dune's popularity had exploded. People really began using it in earnest, and we found ourselves in a somewhat ridiculous, self-inflicted situation: we now had two full build systems to maintain and support. Jenga vs. Dune It became clear to us that Dune was the better system--a rethought design, faster for most people's builds, with wider adoption, and a better API and user experience--which brought up the question "when will we migrate Jane Street onto Dune?" Considering the provenance of the tool this felt like an absurd question, but oh well, that's where we'd ended up. The answer was inevitably "next year," since the build systems team had plenty to work on simply keeping up with our growing codebase. (In 2016, when Dune started, we had 4M lines of OCaml code; today we have 65M, plus 5M lines of Python.) A migration to Dune was daunting enough that we never quite fully embarked on the mission. But it wasn't so daunting to keep us from estimating, aspirationally, that it might well happen in the next six to twelve months. It was only last year that we decided to finally rip the band-aid off. We'd grown the build systems team to five full-time engineers, so we felt that we finally had the strength to tackle this monster we'd been afraid of for so long. Dune subsumes Jenga within Jane Street One large chunk of work we didn't really appreciate in the beginning was just making Dune scale to our huge codebase. Dune was quite fast externally--but that's in part because most users built relatively tiny things compared to Jane Street's 70M-line repository. And Jenga hadn't stood still, either. Over a decade, we had improved the implementation to deal with the growth of our codebase. That produced a system that was pretty well-optimized for scale, and carefully tailored to our monorepo's requirements. Now, much of that good optimization work had to be translated to Dune. There were more mundane problems. The build system is called by a variety of different workflows, notably, from three different editors--Vim, Emacs, and VSCode. Each one sadly had its own custom integrations with Jenga that had to be migrated to use Dune one by one. But after more than a year of focused work we're finally done: our codebase is now built by Dune. At the time of the switch, Dune's performance was across the board as good or better than Jenga's, and much better in some cases. In particular, builds where most of the build work is already in the cache (which is a surprisingly common case!) have gotten 2-3x faster. A lot of what we've done to improve Dune's performance will get open sourced, and some already has. We're keen not to end up with two build systems again--a Jane Street fork of Dune and the external Dune--so we're putting a lot of thought and energy into upstreaming our changes where we can. Dune is a very good foundation for doing new things. Some of that is because Dune's codebase is simpler and easier to work with; some because we can just focus on one system. But, features like distributed builds, shallow builds (also known as "builds without the bytes"), and cached loading of the build graph itself are all closer in reach than ever. The exciting thing going forward is that we have a single system and our velocity in improving it is going way up. The team has also grown to 12 full-time engineers, in New York, London and Singapore, which means we're now working on Dune 24 hours a day. It's been a long and sometimes meandering path, and certainly hasn't unfolded the way we might have planned. But we think that the end result is good for OCaml's build-system story, both within and beyond Jane Street's walls. Andrey joined Jane Street London in 2019, and in 2025 moved to the Singapore office to grow the Tools and Compilers team in Asia. Before Jane Street, Andrey was an academic at Newcastle University, UK, where he studied functional programming and asynchronous circuits. previous post Subscribe to Email Updates [ ] [ ] [Subscribe] Signals & Threads Podcast [signals-an] Listen to the latest episode Featured Tech Talk The Algorithm for Precision Medicine [youtube] Jane Street YouTube Channel Featured Reads * [applying-t] ----------------------------------------------------------------- Read more ----------------------------------------------------------------- Posts about interviewing at Jane Street and our internship program * [scientist_] ----------------------------------------------------------------- Read more ----------------------------------------------------------------- Using ASCII waveforms to test hardware designs * [memory-lea] ----------------------------------------------------------------- Read more ----------------------------------------------------------------- Finding memory leaks with Memtrace Tags * async * book * c * camlp4 * code-review * comments * compiler * core * hackerschool * hg * incremental * internship * interviewing * ocaml * parallel-programming * performance * ppx * real-world-ocaml * registers * speed * ui RSS Get the RSS feed Jane Street Open Source From committing patches to the Linux kernel to releasing our own projects, we're always looking for ways to participate in the open source community. Join Our Team Jane Street Where FP meets the real world. If you're interested in working at a place where functional programming meets the real world, then apply for a job at Jane Street. Learn more * Who We Are * What We Do * Client Offering * The Latest * Culture * Join Jane Street * Contact Us Disclosures & Policies (c) Copyright 2015-2025 Jane Street Group, LLC. All rights reserved. Services are provided in the U.S. by Jane Street Capital, LLC and Jane Street Execution Services, LLC, each of which is a SEC-registered broker dealer and member of FINRA (www.finra.org). Regulated activities are undertaken in Europe by Jane Street Financial Limited, an investment firm authorized and regulated by the U.K. Financial Conduct Authority, and Jane Street Netherlands B.V., an investment firm authorized and regulated by the Netherlands Authority for the Financial Markets (Autoriteit Financiele Markten), and in Hong Kong by Jane Street Hong Kong Limited, a regulated entity under the Hong Kong Securities and Futures Commission (CE No. BAL548). Each of these entities is a wholly owned subsidiary of Jane Street Group, LLC. This material is provided for informational purposes only and does not constitute an offer or solicitation for the purchase or sale of any security or other financial instrument. | Jane Street and the concentric circle mark are registered trademarks of Jane Street. Privacy Cookies This site uses cookies to help us learn and improve. You are welcome to opt out if you would like. Learn more in our Ad and Cookie Policy. Accept All Reject All