https://webapp.io/blog/github-actions-10x-faster-with-firecracker/ * Community * Docs * Blog * Contact * Already a user? Log in Sign Up CI How to run end-to-end tests 10x faster with firecracker Firecracker is the technology which powers Serverless, but it can be also be repurposed to run end-to-end tests quickly and cheaply. * Colin Chartier Colin Chartier Colin is co-founder & CEO at webapp.io. He previously worked as CTO at ParseHub and as a software design lecturer at the University of Toronto. You can reach him at [email protected] More posts by Colin Chartier. Colin Chartier Colin Chartier 17 Apr 2022 * 4 min read How to run end-to-end tests 10x faster with firecracker End-to-end tests and CI When you make changes to code in a repository, you usually want to make sure that you haven't broken core user flows like the sign up button. To automatically check nothing is broken, people usually start a copy of their app within their CI pipeline and then run test browsers to click around. That way you can make sure your app still builds, starts, and mostly works after every new change. For a typical full-stack webapp, these are the sorts of steps that run after every new change: Explanation of CI pipeline: A build step where "docker-compose build" runs, then a step where "docker-compose up" runs, then a final step where browser tests run This pipeline works fine, but it's significantly more expensive and slow than it could be: * Images have to be built from scratch, or constantly pushed/pulled from a registry somewhere * The database has to be re-migrated and re-seeded every time * It's hard to parallelize things, because the setup needs to re-run for every independent copy of the environment. Faster and cheaper e2e tests using base images Since the build steps, database, and deployment definitions rarely change between code changes, it'd be nice if we could have a "base image" of a machine which already had everything started. This base image would have: * A migrated and seeded database already running * Build caches populated with files from an earlier build * Images and large files pre-downloaded * The repository itself pre-loaded If we had such a base image, we could pull in our new change and build/start things in seconds. For a long time this was technically infeasible, but there's an exciting new technology that facilitates this exact use-case. Firecracker Firecracker is the framework that powers AWS Lambda. It allows you to make "snapshots" of a running VM, which contain the entire disk/ memory of the vm. When you restore one, you also restore all of the VM's files and processes. Diagram of a VM turning into a file, then back into a VMFirecracker turns VMs into files and back into VMs. We could create a script that created a firecracker VM & set it up to be an ideal base image. Then we could copy the file that contained the VM's files and processes, Getting started with firecracker It's actually relatively simple to use firecracker because of their quickstart guide and other amazing resources. Using firecracker on Linux can be as simple as a few commands: # Step 1. download firecracker curl -fSsL https://github.com/firecracker-microvm/firecracker/releases/download/v1.0.0/firecracker-v1.0.0-x86_64.tgz | tar -xz mv firecracker-* firecracker # Step 2. Download files required to start the VM image_bucket_url="https://s3.amazonaws.com/spec.ccfc.min/img/quickstart_guide/$arch" wget "${image_bucket_url}/kernels/vmlinux.bin" wget "${image_bucket_url}/rootfs/bionic.rootfs.ext4" # Step 3. Create a config file cat < vmconfig.json { "boot-source": { "kernel_image_path": "vmlinux.bin", "boot_args": "console=ttyS0 reboot=k panic=1 pci=off" }, "drives": [ { "drive_id": "rootfs", "path_on_host": "bionic.rootfs.ext4", "is_root_device": true, "is_read_only": false } ] } EOF # Step 4. Start the VM ./firecracker --unix-socket /tmp/firecracker.socket --config-file vmconfig.json Great, we have a VM, now what? We wanted to use firecracker to create a "zygote snapshot" which contained all of our dependencies. If you're following along, use Julia Evans' tutorial to set up networking, and you should be able to ssh directly into your VM! From there, run your setup commands as necessary: $ apt-get install docker postgresql $ docker pull ubuntu:18.04 $ wget http://example.com/my-large-file.txt ~/my-large-file.txt Some example commands that set up a VM with the dependencies for running brower tests Finally, exit the VM and take a snapshot! (see Firecracker docs) curl --unix-socket /tmp/firecracker.socket -i \ -X PATCH 'http://localhost/vm' -H 'Accept: application/json' \ -H 'Content-Type: application/json' \ -d '{"state": "Paused"}' curl --unix-socket /tmp/firecracker.socket -i \ -X PUT 'http://localhost/snapshot/create' \ -H 'Accept: application/json' \ -H 'Content-Type: application/json' \ -d '{ "snapshot_type": "Full", "snapshot_path": "./snapshot_file", "mem_file_path": "./mem_file", "version": "0.23.0" }' How it all fits together Now that we have our disk and memory snapshot files, we can just run a simple script to prepare the VM to run our CI pipeline: 1. Copy the zygote files: bionic.rootfs.ext4, snapshot_file, and mem_file to another directory 2. Restore the snapshot files using the firecracker API 3. Run the build for your pipeline within the restored VM [Frame-115-] Benchmark Enough about theory, let's look at a benchmark! This benchmark uses an open source Slack clone, which uses Docker Compose, Go, PostgreSQL, and React. Firecracker-based CI is 10.5x faster [Benchmark-]build / start / test for a full-stack webapp This benchmark shows a drastic 10x speedup due to having a warm cache. The firecracker VM came up almost instantly, and ran the build steps in just a few seconds due to the reused caches. This meant that the whole pipeline could run in only 30s. In comparison, the "traditional CI" VMs took over a minute just to boot and clone the code, and then had to re-build the images from scratch, before ever being able to run the tests. See these links for methodology: * GitHub actions build * Firecracker build (notice the "skipped" steps) * Commit which triggered the builds Want to try firecracker-based CI yourself? Webapp.io offers firecracker-based VMs for CI even on our free plan. If you'd like to try out the approach mentioned in this post without setting it up yourself, you can create a webapp.io account and try a sample in just a few minutes at https://webapp.io. Discussions: * Hacker news * Reddit Want to join a Slack community of 100+ webapp developers? Join the webapp developers community More in CI * GitHub + webapp.io Pull Request Automation Demo 21 May 2021 - 1 min read * What is Code Coverage? 27 Apr 2021 - 4 min read * Crypto miners are killing free CI 25 Apr 2021 - 3 min read See all 7 posts - Q1/22 Launch Week Day 5: Webapp.io Academy Q1/22 Launch Week Day 5: Webapp.io Academy On the final day of launch, we're excited to share the launch of Academy, a hub for software developers to learn about best practices on DevOps, software development, and more * Zoya Feza Zoya Feza Zoya Feza 8 Apr 2022 * 1 min read webapp.io Reference What is CI? Academy Docs API Examples Company About Us Press Careers Security Contact Us JellyBear Community Pricing Developer Slack Community Academy Case studies bxblue - Fintech Sheertex - E-commerce Pulley - Fintech Fellow - SaaS Welcome - SaaS Vitau - Healthtech BioBox - Biotech Mentum - SaaS Status Status Dashboard Incidents +1 (800) 575-2971 [email protected] (c) Layer Devops Inc. 2019-2022 Terms of Service Privacy Policy