https://engineering.fb.com/2023/06/27/developer-tools/meta-developer-tools-open-source/ Skip to content Engineering at Meta Search this site [ ] # * Open Source + Open Source + Meta Open Source * Platforms + Android + iOS + Web * Infrastructure Systems + Core Data + Data Infrastructure + DevInfra + Production Engineering + Security + Research Publications * Physical Infrastructure + Connectivity + Data Center Engineering + Networking & Traffic + Research Publications * Video Engineering & AR/VR + Video Engineering + Virtual Reality + Research Publications * Artificial Intelligence + ML Applications + AI Research + Research Publications * Watch Videos [ ] POSTED ON JUNE 27, 2023 TO DevInfra, Open Source Meta developer tools: Working at scale Meta's developer workflow By Neil Mitchell Every day, thousands of developers at Meta are working in repositories with millions of files. Those developers need tools that help them at every stage of the workflow while working at extreme scale. In this article we'll go through a few of the tools in the development process. And, as an added bonus, those we talk about below are open source so you can try them yourself. Sapling: Scaling version control Sapling is a version control system that can scale to huge sizes, but also emphasizes usability. There are three main components to Sapling - a server, a client, and a virtual file system. The server stores all the data and is a careful mix of clever storage formats, wire protocols, and algorithms, mostly implemented in Rust and architected to scale. The client then talks to that server, providing all the familiar operations (check out, rebase, commit, amend, etc). In addition, the client is also capable of talking to a git server, meaning that open source GitHub repositories can be worked on using our open-source Sapling release. The final component is the virtual file system. When checking out a repository of our scale, just the disk I/O of writing all the files can take a significant amount of time. One solution is sparse checkouts, where a developer declares what subset of the repo they wish to see in advance. A more ergonomic alternative is EdenFS, which checks out everything in a few seconds but then only actually downloads the files from the server when they are accessed. Buck2: Build system After making changes, many developers at Meta use Buck2 to compile the results and test out their changes. Buck2 is designed to work at large scale, supporting remote caching and execution, so that developers can share each other's compilations and a single developer can have access to thousands of machines to run compilations in parallel. Buck2 is also designed to support multiple programming languages simultaneously - so if you want your OCaml program to depend on a Rust library that uses a C++ library whose source code was generated by Erlang, that will work just fine. Buck2 works well without Sapling, but has specific design considerations to enable Sapling and EdenFS. Buck2 uses Watchman to find out which files have changed, and Watchman supports EdenFS so that it can integrate smoothly with files that aren't on the disk. Buck2 can also use specific EdenFS operations to access the file without going via the disk, optimizing performance on systems where virtual file systems can be slower. Infer, RacerD, and Jest: Testing and static analysis Handwritten tests and static analysis play an important role in making sure all of our code functions as intended. Working with the quantity of code we do at Meta means that we need tools that provide a high-quality signal, and do so very quickly. For general static analysis, we use a platform called Infer, which is interprocedural and supports multiple languages, including Java and C++. We also have more tailor-made analysis tools such as RacerD, which detects Java concurrency bugs. RacerD played a big role in our project to move Facebook's News Feed on Android from single-threaded to multi-threaded. We also have language-specific testing frameworks. For example, Jest is our Javascript testing framework. In 2022 we officially transferred Jest to the OpenJS Foundation to further support its growth within the wider industry. Finally, there are tools that sit between static analysis and manual test cases. Our Sapienz tool, for example, automatically tests mobile apps by allowing developers to simulate the user experience to seek out crashes and other potential issues. Learn more about Meta's developer workflow In addition to our open-source tools, our developers also use a number of proprietary tools in their day-to-day workflows as well. For example, Phabricator (Phab for short), our CI and reviewing tool, helps our developers review and submit stacks of diffs. You can find more about these tools (along with the ones covered above) in the article on Meta developer's workflow. Read More in DevInfra View All Buck2 APR 6, 2023 Build faster with Buck2: Our open source build system [Eng-Blog-Self-Serve-Hero-Images-ANALYTICS-402-Teale] FEB 6, 2023 The evolution of Facebook's iOS app architecture [Data-Center-hero] JAN 26, 2023 Tulip: Modernizing Meta's data platform [Eng-Blog-Self-Serve-Hero-Images-ANALYTICS-401-Orange] NOV 30, 2022 Enabling static analysis of SQL queries at Meta [Debugging-Hero] NOV 22, 2022 Retrofitting null-safety onto Java at Meta [Eng-Blog-Self-Serve-Hero-Images-DEBUGGING-203-] NOV 16, 2022 Move faster, wait less: Improving code review time at Meta Related Posts --------------------------------------------------------------------- [Buck2-Hero] Apr 06, 2023 Build faster with Buck2: Our open source build system [Sapling-Hero-smaller] Nov 15, 2022 Sapling: Source control that's user-friendly and scalable [new_sapienz_hero1] May 02, 2018 Sapienz: Intelligent automated software testing at scale Related Positions --------------------------------------------------------------------- * Software Engineer, Product - Generative AI MENLO PARK, US * Software Engineer, Product - Generative AI SEATTLE, US * Software Engineer, Product - Generative AI NEW YORK, US * Software Engineer, Product - Generative AI REMOTE, US * Software Engineer (Technical Leadership), Machine Learning - Generative AI MENLO PARK, US See All Jobs Available Positions --------------------------------------------------------------------- * Software Engineer, Product - Generative AI MENLO PARK, US * Software Engineer, Product - Generative AI SEATTLE, US * Software Engineer, Product - Generative AI NEW YORK, US * Software Engineer, Product - Generative AI REMOTE, US * Software Engineer (Technical Leadership), Machine Learning - Generative AI MENLO PARK, US See All Jobs Stay Connected * footer-fb-engineering Engineering at Meta Like * footer-fbopensource Meta Open Source Follow * footer-research Meta Research Like * footer-developers Meta for Developers Like * footer-rss RSS Subscribe Open Source Meta believes in building community through open source technology. Explore our latest projects in Artificial Intelligence, Data Infrastructure, Development Tools, Front End, Languages, Platforms, Security, Virtual Reality, and more. * android ANDROID * ios iOS * web WEB * backend BACKEND * hardware HARDWARE Learn More Meta Engineering at Meta is a technical news resource for engineers interested in how we solve large-scale technical challenges at Meta. * Home * Company Info * Careers (c) 2023 Meta * Terms * Privacy * Cookies * Help To help personalize content, tailor and measure ads and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookie Policy Accept *