https://github.com/awslabs/mountpoint-s3 Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code + Explore + All features + Documentation + GitHub Skills + Blog * Solutions + For + Enterprise + Teams + Startups + Education + By Solution + CI/CD & Automation + DevOps + DevSecOps + Case Studies + Customer Stories + Resources * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles + Repositories + Topics + Trending + Collections * Pricing [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this organization All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} awslabs / mountpoint-s3 Public * Notifications * Fork 7 * Star 426 A simple, high-throughput file client for mounting an Amazon S3 bucket as a local file system. License Apache-2.0 license 426 stars 7 forks Star Notifications * Code * Issues 25 * Pull requests 3 * Actions * Projects 1 * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Security * Insights awslabs/mountpoint-s3 This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 16 branches 0 tags Code * Local * Codespaces * Clone HTTPS GitHub CLI [https://github.com/a] Use Git or checkout with SVN using the web URL. [gh repo clone awslab] Work fast with our official CLI. Learn more. * Open with GitHub Desktop * Download ZIP Sign In Required Please sign in to use Codespaces. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @rbowen rbowen A few typos (#142) ... b8363a4 Mar 14, 2023 A few typos (#142) Signed-off-by: Rich Bowen b8363a4 Git stats * 249 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .github Use proper events to trigger benchmark CI (#138) March 14, 2023 13:46 doc Various documentation updates (#129) March 13, 2023 16:44 mountpoint-s3-client Bump crate versions to 0.2.0 (#140) March 14, 2023 16:04 mountpoint-s3-crt-sys Bump crate versions to 0.2.0 (#140) March 14, 2023 16:04 mountpoint-s3-crt A few typos (#142) March 14, 2023 13:34 mountpoint-s3 Bump crate versions to 0.2.0 (#140) March 14, 2023 16:04 vendor/fuser Update vendored fuser to 9c11ad3 March 13, 2023 10:48 .gitignore Use tokio for async ListObjectsv2 September 3, 2022 00:39 .gitmodules Rename "s3-file-connector" to "mountpoint-s3" (#125) March 8, 2023 11:44 Cargo.lock Bump crate versions to 0.2.0 (#140) March 14, 2023 16:04 Cargo.toml Rename "s3-file-connector" to "mountpoint-s3" (#125) March 8, 2023 11:44 LICENSE Initial commit August 22, 2022 14:19 Makefile Rename "s3-file-connector" to "mountpoint-s3" (#125) March 8, 2023 11:44 NOTICE Initial commit August 22, 2022 14:19 README.md Add tech blog to readme (#141) March 14, 2023 18:04 deny.toml Add Zlib License to allowed licenses list February 8, 2023 11:51 lsan-suppressions.txt Add address sanitizer to CI October 2, 2022 21:05 rustfmt.toml Move rustfmt and clippy into Makefile targets September 8, 2022 22:49 vendor-fuser.sh Switch to cleaned up non-async fuser fork March 13, 2023 10:48 View code [ ] Mountpoint for Amazon S3 Getting started Configuration Current status Contributing Security Code of conduct License README.md Mountpoint for Amazon S3 CI Status Mountpoint for Amazon S3 is a simple, high-throughput file client for mounting an Amazon S3 bucket as a local file system. With Mountpoint for Amazon S3, your applications can access objects stored in Amazon S3 through file operations like open and read. Mountpoint for Amazon S3 automatically translates these operations into S3 object API calls, giving your applications access to the elastic storage and throughput of Amazon S3 through a file interface. Mountpoint for Amazon S3 is optimized for read-heavy workloads that need high throughput. It intentionally does not implement the full POSIX specification for file systems. See SEMANTICS.md for a detailed description of the client's behavior and POSIX support. This is an alpha release and not yet ready for production use. We're especially interested in early feedback on features, performance, and compatibility. Please send feedback by opening a GitHub issue. See Current status for more limitations. Getting started The alpha release of Mountpoint for Amazon S3 is only available by building from source, and only supports Linux. These instructions are for Amazon Linux 2, but other Linux distributions should be similar. First, install dependencies, including the Rust compiler via rustup: sudo yum install fuse fuse-devel cmake3 clang-devel curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh Now clone this repository and its submodules: git clone --recurse-submodules https://github.com/awslabs/mountpoint-s3.git Finally, compile the client: cd mountpoint-s3 cargo build --release The final binary will be at target/release/mount-s3. To use the client, first ensure you have access to valid AWS credentials. For example, you could create a new IAM user and add it to the ~/.aws/credentials file, or configure an EC2 instance with an IAM role. Then run the client, specifying the directory in which your S3 bucket should be mounted (here, ~/mnt): mkdir ~/mnt mount-s3 my-s3-bucket-name ~/mnt The client will run in the background by default, and the ~/mnt directory now gives access to the objects in your S3 bucket. Configuration For the alpha release, some additional configuration is required to use Mountpoint for Amazon S3 in some circumstances: * High-bandwidth EC2 instances: manually specify the available network bandwidth in Gbps with the --throughput-target-gbps command-line argument. Defaults to 10 Gbps if not configured. This configuration will be automated in the future (#3). * AWS GovCloud, China (Beijing), and China (Ningxia) regions: manually specify the S3 endpoint with the --endpoint-url command-line argument and the region name with the --region argument. For example, for the AWS GovCloud (US-West) region, specify --endpoint-url https://s3.us-gov-west-1.amazonaws.com --region us-gov-west-1. See the S3 endpoints documentation for more details. These regions will be supported via only the --region command-line argument in the future (#4). * Access points: use the bucket-style access point alias for the access point as the bucket name to mount. * FIPS, dualstack, and transfer acceleration endpoints: manually specify the S3 endpoint with the --endpoint-url command-line argument. See the S3 endpoints documentation for more details. These endpoints will be configurable in the future (#4). Current status Mountpoint for Amazon S3 is currently an alpha release and should not be used in production. We're tracking its production readiness and future features on the Mountpoint for Amazon S3 public roadmap. We welcome suggestions for new features or use cases! Our thinking around features is guided by our tenets for Mountpoint for Amazon S3. We don't intend to implement (and likely won't accept contributions of) features that cannot be efficiently executed against S3's object APIs. This rules out Mountpoint for Amazon S3 supporting some POSIX file system functionality like ACLs or file locking. For workloads that need these features, we recommend AWS fully managed file system services like Amazon FSx for Lustre and its support for linking S3 buckets. Contributing We welcome contributions to Mountpoint for Amazon S3! Please see CONTRIBUTING.md for more information on how to report bugs or submit pull requests, and LOGGING.md for details on how to capture logging data for bug reports. Mountpoint for Amazon S3 is designed for high-performance access to the Amazon S3 service. While it may be functional against other storage services that use S3-like APIs, we aren't able to provide support for those use cases, and they may inadvertently break when we make changes to better support Amazon S3. We welcome contributions of minor compatibility fixes or performance improvements for these services if the changes can be tested against Amazon S3. Security If you discover a potential security issue in this project we ask that you notify AWS Security via our vulnerability reporting page. Please do not create a public GitHub issue. Code of conduct This project has adopted the Amazon Open Source Code of Conduct. See CODE_OF_CONDUCT.md for more details. License This project is licensed under the Apache-2.0 License. It builds on a number of other awesome projects with open source licenses, primarily: * The fuser bindings for FUSE (MIT license) * The AWS Common Runtime for interacting with S3 (Apache-2.0 license) See deny.toml for a list of licenses used by our dependencies. About A simple, high-throughput file client for mounting an Amazon S3 bucket as a local file system. Topics aws fuse filesystem s3 Resources Readme License Apache-2.0 license Code of conduct Code of conduct Security policy Security policy Stars 426 stars Watchers 18 watching Forks 7 forks Used by 0 * @awslabs @awslabs / mountpoint-s3 Contributors 9 * * * * * * * * * Languages * Rust 98.6% * Other 1.4% Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.