https://github.com/containers/composefs Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code + Explore + All features + Documentation + GitHub Skills + Blog * Solutions + For + Enterprise + Teams + Startups + Education + By Solution + CI/CD & Automation + DevOps + DevSecOps + Case Studies + Customer Stories + Resources * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles + Repositories + Topics + Trending + Collections * Pricing [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this organization All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} containers / composefs Public * Notifications * Fork 9 * Star 151 a file system for mounting container images 151 stars 9 forks Star Notifications * Code * Issues 2 * Pull requests 0 * Actions * Projects 0 * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Security * Insights containers/composefs This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 7 branches 0 tags Code * Local * Codespaces * Clone HTTPS GitHub CLI [https://github.com/c] Use Git or checkout with SVN using the web URL. [gh repo clone contai] Work fast with our official CLI. Learn more. * Open with GitHub Desktop * Download ZIP Sign In Required Please sign in to use Codespaces. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @giuseppe giuseppe Merge pull request #99 from alexlarsson/fix-binay-search ... 26cfb8d Jan 24, 2023 Merge pull request #99 from alexlarsson/fix-binay-search kernel: Fix binary search bounds check 26cfb8d Git stats * 374 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .github/workflows Drop the fuzzer Sep 20, 2022 hacking/clang-format clang-format: Change PenaltyExcessCharacter to 3 Jan 10, 2023 kernel Merge pull request #99 from alexlarsson/fix-binay-search Jan 24, 2023 libcomposefs Fix on-disk struct sizes Jan 19, 2023 tests Add tests/ directory with tools to test composefs kernel build using ... Jan 13, 2023 tools dump: Fix payload printing and times on 32bit arches Jan 18, 2023 .gitignore Build and use a separate libcomposefs library Jun 8, 2022 Makefile.am Build and use a separate libcomposefs library Jun 8, 2022 README.md Fix documentation typos Dec 3, 2022 autogen.sh Build and use a separate libcomposefs library Jun 8, 2022 composefs.cil Fix selinux module extension Jun 7, 2022 configure.ac Makefile: Use names for crypto deps that are easier to use as git sub... Jun 22, 2022 View code composefs Usecase: container images Usecase: OSTree user space tools kernel module SELinux issues README.md composefs Composefs is a native Linux file system designed to help sharing filesystem contents, as well as ensuring said content is not modified. The initial target usecase are container images and ostree commits. The basic idea is to have a single binary file that contains all the metadata of the filesystem, including the filenames, the permissions, the timestamps, etc. However, it doesn't contain the actual contents, but rather filenames to the real files that contain the contents. This is somewhat similar to overlayfs, which also doesn't store the file. You pass the filename of the blob as well as the base directory for the content files when you mount the filesystem like this: # mount /path/to/blob -t composefs -o basedir=/path/to/content /mnt This by itself doesn't seem very useful. You could use a single squashfs image, or regular directory with the files instead. However, the advantage comes if you want to store many such images. By storing the files content-addressed (e.g. using the hash of the content to name the file) shared files need only be stored once, yet can appear in multiple mounts. Since these are normal files they will also only be stored once in the page cache, meaning that the duplication is avoided both on disk and in ram. Composefs also supports fs-verity validation of the content files. When using this, the digest of the content files is stored in the image, and composefs will validate that the content file it uses has a matching enabled fs-verity digest. This means that the backing content cannot be changed in any way (by mistake or by malice) without this being detected when the file is used. You can also use fs-verity on the image file itself, and pass the expected fs-verity digest as a mount option, which composefs will validate. In this case we have full trust of both data and metadata of the mounted file. This solves a weakness that fs-verity has when used on on its own, in that it can only verify file data, not metadata. Usecase: container images When pulling a container image to the local storage we normally just untar each layer by itself. Instead we can store the file content in a content-addressed fashion, and then generate a composefs file for the layer (or perhaps the combined layers). This allows sharing of content files between images, even if the metadata (like the timestamps or file ownership) vary between images. Together with something like zstd:chunked this will speed up pulling container images and make them available for usage, without the need to even create these files if already present! Usecase: OSTree OSTree already uses a content-address object store. However, normally this has to be checked out into a regular directory (using hardlinks into the object store for regular files). This directory is then bind-mounted as the rootfs when the system boots. OSTree already supports enabling fs-verity on the files in the store, but nothing can protect against changes to the checkout directories. A malicious user can add, remove or replace files there. We want to use composefs to avoid this. Instead of checking out to a directory we generate a composefs image pointing into the object store and mount that as the root fs. We can then enable fs-verity of the composefs image and embed the digest of that in the kernel commandline which specifies the rootfs. Since composefs generation is reproducible, we can even verify that the composefs image we generated is correct by comparing its digest to one in the ostree metadata that was generated when the ostree image was built. user space tools The directory tools/ contains some user space tools to create the binary blob to pass to the client. They are all experimental and lack documentation. * mkcomposefs: Creates a composefs image given a directory pathname. Can also compute digests and create a content store directory. * writer-json: convert from a CRFS metadata file to the binary blob. * dump: prints the content of the binary blob. * ostree-convert-commit.py: converts an OSTree commit into a CRFS config file that writer-json can use. kernel module How to build: # make -C $KERNEL_SOURCE modules M=$PWD && make -C $KERNEL_SOURCE modules_install M=$PWD # insmod /lib/modules/$(uname -r)/extra/composefs.ko Once it is loaded, it can be used as: # mount /path/to/blob -t composefs -o basedir=$BASE_DIR /mnt Mount options: * basedir: is the directory to use as a base when resolving relative content paths. * verity_check=0,1,2: When to verify backing file fs-verity: 0 == never, 1 == if specified in image, 2 == always and require it in image. * digest: A fs-verity sha256 digest that the image file must match. If set, verity_check defaults to 2. SELinux issues Composefs support xattrs natively, and selinux normally uses xattrs to store selinux file contexts. However, this only works if the local policy allows a particular filesystem type to use xattrs for selinux, and the default is to not allow it. So, until the default selinux contexts supports composefs, you need to manually install a local policy for this. To enable composefs selinux support, run: # semodule -i composefs.cil And, to later revert it, run: # semodule -r composefs About a file system for mounting container images Topics linux containers filesystem oci oci-image Resources Readme Stars 151 stars Watchers 10 watching Forks 9 forks Releases No releases published Contributors 6 * * * * * * Languages * C 95.3% * Python 1.7% * M4 1.6% * Other 1.4% Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.