https://info.varnish-software.com/blog/how-memory-maps-mmap-deliver-25x-faster-file-access-in-go Varnish Software Talk to an Expert Menu * en-US + en-US de-DE es-ES fr-FR * Contact us * Blog * Developer Portal * Products < Products varnish-how-it-works-diagram-v2-02 + Varnish Enterprise Edge caching and content delivery software that unlocks fast, reliable digital experiences for users and systems. + Varnish Pro Scalable, secure, and flexible HTTP acceleration for modern cloud environments. + Varnish Controller Robust, UI-based management of Varnish instances and VCL for easy deployments and monitoring. + Varnish Cache Free, open source software that enables super fast delivery of HTTP. + Varnish Traffic Router Flexible HTTP request routing that ensures clients fetch content from the optimal cache, for excellent user experience at scale. + How it works * Solutions < Solutions nav-images-393px-x-355px-06 By use case: + Web & API Performance o Web Application Acceleration o API Acceleration o Web App Security o Origin Security o eCommerce & CMS Acceleration + Content Delivery o Origin Shield o Private CDN o Streaming o Edge Compute + Data & AI Acceleration o Data Pipeline Accelerator o DevOps Artifact & Software Distribution o S3, Object Storage and Cloud Optimization * Pricing * Resources < Resources nav-images-393px-x-355px-04 + Learn o Blog o Varnish Book o Webinars o Case Studies o White Papers o Videos and Demos + Documentation o Developer Portal o Glossary o Reports o Books + Training * Support < Support + Professional Services + Varnish Managed Services + Developer Portal + Community * About us < About us + Company + Events + Press + Careers + Partners * Get Varnish Talk to an Expert Return to Blog September 30, 2025 4 min read time How Memory Maps (mmap) Deliver 25x Faster File Access in Go Per Buer Per Buer Performance Varnish Massive Storage Engine memory map [How] One of the slowest things you can do in an application is making system calls. They're slow because you do have to enter the kernel, which is quite expensive. What should you do when you need to do a lot of disk I/O but you care about performance? One solution is to use memory maps. Memory maps are a modern Unix mechanism where you can take a file and make it part of the virtual memory. In Unix context, modern means that it was introduced in the 1980s or later. You have a file, containing data, you mmap it and you'll get a pointer to where this resides. Now, instead of seeking and reading, you just read from this pointer, adjusting the offset to get to the right data. Performance To show what kind of performance you can get using memory maps, I've written a little Go library that allows you to read from a file using a memory map or a ReaderAt. ReaderAt will do a pread(), which is a seek/read combo, while mmap will just read from the memory map. Random lookup (ReaderAt): 416.4 ns/op Random lookup (mmap): 3.3 ns/op --- Iteration (ReaderAt): 333.3 ns/op Iteration (mmap): 1.3 ns/op This almost feels like magic. Initially, when we launched Varnish Cache back in 2006, this was one of the features that made Varnish Cache very fast when delivering content. Varnish Cache would use memory maps to deliver content at blistering speeds. Also, since you can operate with pointers into memory that is allocated by the memory map, you'll reduce memory pressure as well as raw latency. The Downside of Memory Maps The downside of memory maps is that you really can't write to the memory map. The reason is due to the way virtual memory works. When you're writing to a part of virtual memory that isn't mapped into physical memory, the CPU will generate a page fault. On a modern computer, the CPU is responsible for tracking what virtual memory pages are mapped onto what physical memory. Since you're writing to a page that isn't mapped, the CPU needs help. So, when the page fault occurs, the OS will 1) allocate a new memory page, 2) read the contents of the file at the correct offset, 3) write this to the new memory page. Then control is returned to the application. The application will now overwrite the virtual memory page with new data. Can we stop and appreciate how extremely inefficient this is? I think it is fairly safe to say that writing through a memory map is never a good idea when considering performance. At least if there is any risk, the file isn't mapped up in physical memory. Let me illustrate this with a few more benchmarks. Mmap writes, unmapped pages: 1870 ns/op Mmap writes, mapped in memory pages: 79 ns/op WriterAt: 303 ns/op As you can see, whether or not the pages are in cache is crucial for performance. WriterAt, which uses the pwrite call, is a much more predictable bet. Still, writing through memory maps, was what Varnish Cache did initially. It somehow got away with it, but mostly because the competition was pretty bad. This is why Varnish Cache got the malloc backend and why Varnish Enterprise got the various Massive Storage Engines. The malloc backend resolved the problem by just allocating system memory through the malloc system call, and the Massive Storage Engine uses io_uring, which is so new that support for it is still somewhat limited. Using Memory Maps to Solve Real-world Performance Problems The last couple of weeks I've been working on an HTTP-backed filesystem. This is part of our AI Storage Acceleration solution, geared towards high performance computing environments. In this filesystem we needed a way to transfer folder data over HTTP. A folder is really just a listing of files, symbolic links and directories. The naive approach would be just to use JSON encoding, but JSON is notorious for being slow. Our priority is performance. We made a benchmarking suite, comparing various databases with each other. CDB was overall the fastest. Looking at the numbers, we'd still see that CDB would spend something like 1200ns on a database lookup that was entirely in the page cache. This seems very slow to me. After all, everything should be in memory and spending 1200ns reading memory sounds at least 100x too slow. I started looking into the CDB implementation I was using. It was the above ReaderAt implementation. So, most of the time is likely spent waiting for the operating system. Some hours later, I was able to replace the seek/read with a memory map. This resulted in a 25x improvement in performance. Again, it feels like magic. Unlike the original file stevedore in Varnish Cache, this performance improvement has no downside. Benchmarks: https://github.com/perbu/mmaps-in-go CDB64 files with memory maps: https://github.com/perbu/cdb Related Posts You may also like this [Varnish] March 7, 2024 4 min read time DevOps Artifact Caching: How Varnish Enterprise MSE Speeds Up CI/CD & Cuts Costs DevOps bottlenecks are frustrating. Long artifact retrieval times slow down CI/CD pipelines,... SUBSCRIBE TO OUR BLOG SEARCH OUR BLOG [ ] Search Explore articles from Varnish experts on web performance, advanced caching techniques, CDN optimization and more, plus all the latest tips and insights for enhancing your content delivery operations. Contact us Stockholm +46 8 410 909 30 Paris +33 1 70 75 27 81 Singapore: +65 8434 8028 Contact us Products & Services Varnish Enterprise Varnish Controller Varnish Traffic Router Varnish Cache Support Professional Services Varnish as a Managed Service Solutions Web Acceleration API Acceleration Web App Protection Artifact Caching Private CDN Hybrid CDN Video Delivery Origin Shield Telco Edge Edge Compute On-Premise Edge Application Delivery Resources Documentation The Varnish Book Getting started with Varnish Case studies White papers Webinars Videos & demos Company About us Blog Careers Partners Events Customer guide Community Privacy Policy Cookie Policy Trademark Compliance Let's Connect facebook twitter linkedin github youtube [en-US] Sign up for our newsletter! * Cookie Settings * Press * Branding * Copyright Varnish Software AB (r)Varnish Software, Wallingatan 12, 111 60 Stockholm, Organization nr. 556805-6203