https://dl.acm.org/doi/10.1145/3132747.3132763 ACM Digital Library home ACM home * Advanced Search * Browse * About * + Sign in + Register * * Advanced Search * Journals * Magazines * Proceedings * Books * SIGs * Conferences * People * * More * Search ACM Digital Library[ ] SearchSearch Advanced Search 10.1145/3132747.3132763acmconferencesArticle/Chapter ViewAbstract Publication PagessospConference Proceedingsconference-collections sosp * Conference * Proceedings * Upcoming Events * Authors * Affiliations * Award Winners * More HomeConferencesSOSPProceedingsSOSP '17My VM is Lighter (and Safer) than your Container research-article Open Access Share on My VM is Lighter (and Safer) than your Container * Authors: * #Filipe Manco NEC Laboratories Europe NEC Laboratories Europe View Profile , * #Costin Lupu Univ. Politehnica of Bucharest Univ. Politehnica of Bucharest View Profile , * #Florian Schmidt NEC Laboratories Europe NEC Laboratories Europe View Profile , * #Jose Mendes NEC Laboratories Europe NEC Laboratories Europe View Profile , * #Simon Kuenzer NEC Laboratories Europe NEC Laboratories Europe View Profile , * #Sumit Sati NEC Laboratories Europe NEC Laboratories Europe View Profile , * #Kenichi Yasukata NEC Laboratories Europe NEC Laboratories Europe View Profile , * #Costin Raiciu Univ. Politehnica of Bucharest Univ. Politehnica of Bucharest View Profile , * #Felipe Huici NEC Laboratories Europe NEC Laboratories Europe View Profile Authors Info & Claims SOSP '17: Proceedings of the 26th Symposium on Operating Systems PrinciplesOctober 2017 Pages 218-233https://doi.org/10.1145/ 3132747.3132763 Online:14 October 2017Publication History * 133citation * 12,238 * Downloads Metrics Total Citations133 Total Downloads12,238 Last 12 Months1,508 Last 6 weeks71 * * Get Citation Alerts New Citation Alert added! This alert has been successfully added and will be sent to: You will be notified whenever a record that you have chosen has been cited. To manage your alert preferences, click on the button below. Manage my Alerts New Citation Alert! Please log in to your account * Save to Binder Save to Binder [loader] Create a New Binder Name [ ] + Cancel + Create * Export Citation * Publisher Site * * eReader * PDF SOSP '17: Proceedings of the 26th Symposium on Operating Systems Principles My VM is Lighter (and Safer) than your Container Pages 218-233 PreviousChapterNextChapter ACM Digital Library ABSTRACT Containers are in great demand because they are lightweight when compared to virtual machines. On the downside, containers offer weaker isolation than VMs, to the point where people run containers in virtual machines to achieve proper isolation. In this paper, we examine whether there is indeed a strict tradeoff between isolation (VMs) and efficiency (containers). We find that VMs can be as nimble as containers, as long as they are small and the toolstack is fast enough. We achieve lightweight VMs by using unikernels for specialized applications and with Tinyx, a tool that enables creating tailor-made, trimmed-down Linux virtual machines. By themselves, lightweight virtual machines are not enough to ensure good performance since the virtualization control plane (the toolstack) becomes the performance bottleneck. We present LightVM, a new virtualization solution based on Xen that is optimized to offer fast boot-times regardless of the number of active VMs. LightVM features a complete redesign of Xen's control plane, transforming its centralized operation to a distributed one where interactions with the hypervisor are reduced to a minimum. LightVM can boot a VM in 2.3ms, comparable to fork/exec on Linux (1ms), and two orders of magnitude faster than Docker. LightVM can pack thousands of LightVM guests on modest hardware with memory and CPU usage comparable to that of processes. Supplemental Material my_vm_lighter.mp4 mp4 2.2 GB Play streamDownload References 1. Amazon Web Services {n. d.}. Amazon EC2 Container Service. https: //aws.amazon.com/ecs/. ({n. d.}).Google ScholarGoogle Scholar 2. Amazon Web Services {n. d.}. AWS Lambda - Serverless Compute. https://aws.amazon.com/lambda. ({n. d.}).Google ScholarGoogle Scholar 3. Paul Barham, Boris Dragovic, Keir Fraser, Steven Hand, Tim Harris, Alex Ho, Rolf Neugebauer, Ian Pratt, and Andrew Warfield. 2003. Xen and the Art of Virtualization. SIGOPS Open Syst. Rev. 37, 5 (Oct. 2003), 164--177. Google ScholarGoogle ScholarDigital LibraryDigital Library 4. J. Clark. {n. d.}. Google: "EVERYTHING at Google runs in a container". http//:www.theregister.co.uk/2014/05/23/ google_containerizationtwobillion/. ({n. d.}).Google Scholar Google Scholar 5. Patrick Colp, Mihir Nanavati, Jun Zhu, William Aiello, George Coker, Tim Deegan, Peter Loscocco, and Andrew Warfield. 2011. Breaking Up is Hard to Do: Security and Functionality in a Commodity Hypervisor. In Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles (SOSP '11). ACM, New York, NY, USA, 189--202. Google ScholarGoogle ScholarDigital LibraryDigital Library 6. Docker {n. d.}. The Docker Containerization Platform. https:// www.docker.com/. ({n. d.}).Google ScholarGoogle Scholar 7. John R. Douceur, Jeremy Elson, Jon Howell, and Jacob R. Lorch. 2008. Leveraging Legacy Code to Deploy Desktop Applications on the Web. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation (OSDI'08). USENIX Association, Berkeley, CA, USA, 339--354. http://dl.acm.org/citation.cfm?id= 1855741.1855765 Google ScholarGoogle ScholarDigital Library Digital Library 8. D. R. Engler, M. F. Kaashoek, and J. O'Toole, Jr. 1995. Exokernel: An Operating System Architecture for Application-level Resource Management. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (SOSP '95). ACM, New York, NY, USA, 251--266. Google ScholarGoogle ScholarDigital LibraryDigital Library 9. Erlang on Xen 2012. Erlang on Xen. http://erlangonxen.org/. (July 2012).Google ScholarGoogle Scholar 10. Google Cloud Platform {n. d.}. The Google Cloud Platform Container Engine. https://cloud.google.com/container-engine. ({n. d.}).Google ScholarGoogle Scholar 11. A. Grattafiori. {n. d.}. Understanding and Hardening Linux Containers. https://www.nccgroup.trust/us/our-research/ understanding-and-hardening-linux-containers/. ({n. d.}).Google ScholarGoogle Scholar 12. Cameron Hamilton-Rich. {n. d.}. axTLS Embedded SSL. http:// axtls.sourceforge.net. ({n. d.}).Google ScholarGoogle Scholar 13. Poul henning Kamp and Robert N. M. Watson. 2000. Jails: Confining the omnipotent root. In In Proc. 2nd Intl. SANE Conference.Google ScholarGoogle Scholar 14. J. Hertz. {n. d.}. Abusing Privileged and Unprivileged Linux Containers. https://www.nccgroup.tmst/uk/our-research/ abusing-privileged-and-unprivileged-linux-containers/, ({n. d.}). Google ScholarGoogle Scholar 15. Jon Howell, Bryan Parno, and John R. Douceur. 2013. Embassies: Radically Refactoring the Web. In Presented as part of the 10th USENTX Symposium on Networked Systems Design and Implementation (NSDI13). USENIX, Lombard, IL, 529--545. https://www.usenix.org/ conference/nsdil3/technical-sessions/presentation/howell Google ScholarGoogle ScholarDigital LibraryDigital Library 16. Yun Chao Hu, Milan Patel, Dario Sabella, Nurit Sprecher, and Valerie Young. 2015. Mobile Edge Computing - A key technology towards 5G. ETSI White Paper No. 11, First edition (2015).Google ScholarGoogle Scholar 17. IBM. {n. d.}. Docker at insane scale on IBM Power Systems. https: //www.ibm.com/blogs/bluemix/2015/ll/ docker-insane-scale-on-ibm-power-systems. ({n. d.}).Google ScholarGoogle Scholar 18. IBM developerWorks Open {n. d.}. Solo5 Unikernel. https:// developer.ibm.com/open/openprojects/solo5-unikernel/. ({n. d.}). Google ScholarGoogle Scholar 19. Intel. {n. d.}. Intel Clear Containers: A Breakthrough Combination of Speed and Workload Isolation. https:// clearlinux.org/sites/default/files/vmscontainers_wp_v5.pdf. ({n. d.}).Google ScholarGoogle Scholar 20. Avi Kivity, Yaniv Kamay, Dor Laor, Uri Lublin, and Anthony Liguori. 2007. KVM: the Linux Virtual Machine Monitor. In In Proc. 2007 Ottawa Linux Symposium (OLS '07).Google ScholarGoogle Scholar 21. Avi Kivity, Dor Laor, Glauber Costa, Pekka Enberg, Nadav Har'El, Don Marti, and Vlad Zolotarov. 2014. OSv---Optimizing the Operating System for Virtual Machines. In Proceedings of the 2014 USENTX Annual Technical Conference (USENIX ATC '14). USENIX Association, Philadelphia, PA, 61--72. https://www.usenix.org/ conference/atcl4/technical-sessions/presentation/kivity Google ScholarGoogle ScholarDigital LibraryDigital Library 22. E. Kovacs. {n. d.}. Docker Fixes Vulnerabilities, Shares Plans For Making Platform Safer. http//:www.securityweek.com/ docker-fixes-vulnerabilities-shares-plans-making-platform-safer. ({n. d.}).Google ScholarGoogle Scholar 23. Simon Kuenzer, Anton Ivanov, Filipe Manco, Jose Mendes, Yuri Volchkov, Florian Schmidt, Kenichi Yasukata, Michio Honda, and Felipe Huici. 2017. Unikernels Everywhere: The Case for Elastic CDNs. In Proceedings of the 13th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE '17). ACM, New York, NY, USA, 15--29. Google ScholarGoogle ScholarDigital LibraryDigital Library 24. Horacio Andres Lagar-Cavilla, Joseph Andrew Whitney, Adin Matthew Scannell, Philip Patchin, Stephen M. Rumble, Eyal de Lara, Michael Brudno, and Mahadev Satyanarayanan. 2009. SnowFlock: Rapid Virtual Machine Cloning for Cloud Computing. In Proceedings of the 4th ACM European Conference on Computer Systems (EuroSys '09). ACM, New York, NY, USA, 1--12. Google ScholarGoogle Scholar Digital LibraryDigital Library 25. LinuxContainers.org {n. d.}. LinuxContainers.org. https:// linuxcontainers.org. ({n. d.}).Google ScholarGoogle Scholar 26. Anil Madhavapeddy, Thomas Leonard, Magnus Skjegstad, Thomas Gazagnaire, David Sheets, Dave Scott, Richard Mortier, Amir Chaudhry, Balraj Singh, Jon Ludlam, Jon Crowcroft, and Ian Leslie. 2015. Jitsu: Just-In-Time Summoning of Unikernels. In 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI '15). USENIX Association, Oakland, CA, 559--573. https://www.usenix.org/conference/nsdil5/ technical-sessions/presentation/madhavapeddy Google ScholarGoogle ScholarDigital LibraryDigital Library 27. Anil Madhavapeddy and David J. Scott. 2013. Unikernels: Rise of the Virtual Library Operating System. Queue 11, 11, Article 30 (Dec. 2013), 15 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library 28. Y. Mao, J. Zhang, and K. B. Letaief. 2016. Dynamic Computation Offloading for Mobile-Edge Computing With Energy Harvesting Devices. IEEE Journal on Selected Areas in Communications 34, 12 (Dec 2016), 3590--3605. Google ScholarGoogle ScholarDigital LibraryDigital Library 29. Joao Martins, Mohamed Ahmed, Costin Raiciu, Vladimir Olteanu, Michio Honda, Roberto Bifulco, and Felipe Huici. 2014. ClickOS and the Art of Network Function Virtualization. In 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI '14). USENIX Association, Seattle, WA, 459--473. https:// www.usenix.org/conference/nsdil4/technical-sessions/presentation/ martins Google ScholarGoogle ScholarDigital LibraryDigital Library 30. McAffee. 2016. Mobile Threat Report. https://www.mcafee.com/us/ resources/reports/rp-mobile-threat-report-2016.pdf. (2016).Google ScholarGoogle Scholar 31. MicroPython {n. d.}. MicroPython. https://micropython.org/. ({n. d.}).Google ScholarGoogle Scholar 32. Microsoft. {n. d.}. Azure Container Service. https:// azure.microsoft.com/en-us/services/container-service/. ({n. d.}). Google ScholarGoogle Scholar 33. Microsoft Research. {n. d.}. Drawbridge. https:// www.microsoft.com/en-us/research/project/drawbridge/. ({n. d.}). Google ScholarGoogle Scholar 34. minios {n. d.}. Mini-OS. https://wiki.xenproject.org/wiki/ Mini-OS. ({n. d.}).Google ScholarGoogle Scholar 35. A. Mourat. {n. d.}. 5 security concerns when using Docker. https: //www.oreilly.com/ideas/five-security-concerns-when-using-docker. ({n. d.}).Google ScholarGoogle Scholar 36. Vlad Nitu, Pierre Olivier, Alain Tchana, Daniel Chiba, Antonio Barbalace, Daniel Hagimont, and Binoy Ravindran. 2017. Swift Birth and Quick Death: Enabling Fast Parallel Guest Boot and Destruction in the Xen Hypervisor. In Proceedings of the 13th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE '17). ACM, New York, NY, USA, 1--14. Google ScholarGoogle ScholarDigital LibraryDigital Library 37. MAN page. {n. d.}. Linux system calls list. http://man7.org/linux /man-pages/man2/syscalls.2.html. ({n. d.}).Google ScholarGoogle Scholar 38. Rumpkernel.org {n. d.}. Rump Kernels. http://rumpkernel.org/. ({n. d.}).Google ScholarGoogle Scholar 39. Sandvine. {n. d.}. Internet traffic encryption. https:// www.sandvine.com/trends/encryption.html. ({n. d.}).Google Scholar Google Scholar 40. Mahadev Satyanarayanan, Paramvir Bahl, Ramon Caceres, and Nigel Davies. 2009. The Case for VM-Based Cloudlets in Mobile Computing. IEEE Pervasive Computing 8, 4 (Oct. 2009), 14--23. Google ScholarGoogle ScholarDigital LibraryDigital Library 41. Justine Sherry, Shaddi Hasan, Colin Scott, Arvind Krishnamurthy, Sylvia Ratnasamy, and Vyas Sekar. 2012. Making Middleboxes Someone Else's Problem: Network Processing As a Cloud Service. In Proceedings of the ACM SIGCOMM 2012 Conference on Computer Communication (SIGCOMM '12). ACM, New York, NY, USA, 13--24. Google ScholarGoogle ScholarDigital LibraryDigital Library 42. Stephen Soltesz, Herbert Potzl, Marc E. Fiuczynski, Andy Bavier, and Larry Peterson. 2007. Container-based Operating System Virtualization: A Scalable, High-performance Alternative to Hypervisors. SIGOPS Oper. Syst. Rev. 41, 3 (March 2007), 275--287. Google ScholarGoogle ScholarDigital LibraryDigital Library 43. S. Stabellini. {n. d.}. Xen on ARM. http//:www.slideshare.net/ xen_com_mgr/alsf13-stabellini. ({n. d.}).Google ScholarGoogle Scholar 44. Udo Steinberg and Bernhard Kauer. 2010. NOVA: A Microhypervisorbased Secure Virtualization Architecture. In Proceedings of the 5th European Conference on Computer Systems (EuroSys '10). ACM, New York, NY, USA, 209--222. Google Scholar Google ScholarDigital LibraryDigital Library 45. A. van de Ven. {n. d.}. An introduction to Clear Containers. https://lwn.net/Articles/644675/. ({n. d.}).Google ScholarGoogle Scholar 46. Akshat Verma, Gargi Dasgupta, Tapan Kumar Nayak, Pradipta De, and Ravi Kothari. 2009. Server Workload Analysis for Power Minimization Using Consolidation. In Proceedings of the 2009 USENIX Annual Technical Conference (USENIX ATC '09). USENIX Association, Berkeley, CA, USA, 28--28. http://dl.acm.org/ citation.cfm?id=1855807.1855835 Google ScholarGoogle Scholar Digital LibraryDigital Library 47. VMWare. {n. d.}. vSphere ESXi Bare-Metal Hypervisor. http/ /:www.vmware.com/products/esxi-and-esx.html. ({n. d.}).Google ScholarGoogle Scholar 48. Michael Vrable, Justin Ma, Jay Chen, David Moore, Erik Vandekieft, Alex C. Snoeren, Geoffrey M. Voelker, and Stefan Savage. 2005. Scalability, Fidelity, and Containment in the Potemkin Virtual Honey-farm. SIGOPS Oper. Syst. Rev. 39, 5 (Oct. 2005), 148--162. Google ScholarGoogle ScholarDigital Library Digital Library 49. Andrew Whitaker, Marianne Shaw, and Steven D. Gribble. 2002. Scale and Performance in the Denali Isolation Kernel. SIGOPS Oper. Syst. Rev. 36, SI (Dec. 2002), 195--209. Google Scholar Google ScholarDigital LibraryDigital Library 50. Dan Williams and Ricardo Koller. 2016. Unikernel Monitors: Extending Minimalism Outside of the Box. In 8th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud '16). USENIX Association, Denver, CO. https://www.usenix.org/conference/ hotcloud16/workshop-program/presentation/williams Google Scholar Google ScholarDigital LibraryDigital Library 51. Wei Zhang, Jinho Hwang, Shriram Rajagopalan, K.K. Ramakrishnan, and Timothy Wood. 2016. Flurries: Countless Fine-Grained NFs for Flexible Per-Flow Customization. In Proceedings of the 12th International on Conference on Emerging Networking EXperiments and Technologies (CoNEXT '16). ACM, New York, NY, USA, 3--17. Google ScholarGoogle ScholarDigital LibraryDigital Library Index Terms 1. My VM is Lighter (and Safer) than your Container 1. Software and its engineering 1. Software organization and properties 1. Contextual software domains 1. Operating systems 2. Software infrastructure 1. Virtual machines Comments Please enable JavaScript to view thecomments powered by Disqus. Login options Check if you have access through your login credentials or your institution to get full access on this article. Sign in Full Access Get this Publication * Information * Contributors * Published in ACM Conferences cover image SOSP '17: Proceedings of the 26th Symposium on Operating Systems Principles October 2017 677 pages ISBN:9781450350853 DOI:10.1145/3132747 Copyright (c) 2017 Owner/Author Sponsors In-Cooperation Publisher Association for Computing Machinery New York, NY, United States Publication History + Online: 14 October 2017 Permissions Request permissions about this article. Request Permissions Author Tags + operating systems + virtual machine + hypervisor + containers + specialization + Virtualization + unikernels + Xen Qualifiers + research-article + Research + Refereed limited Conference Acceptance Rates Overall Acceptance Rate 58 of 303 submissions, 19% Funding Sources * [loader] Other Metrics View Article Metrics * Bibliometrics * Citations133 * Article Metrics + 133 Total Citations View Citations + 12,238 Total Downloads + Downloads (Last 12 months)1,508 + Downloads (Last 6 weeks)71 Other Metrics View Author Metrics * Cited By View all PDF Format View or Download as a PDF file. PDF eReader View online with eReader. eReader Digital Edition View this article in digital edition. View Digital Edition * Figures * Other * * Share this Publication link https://dl.acm.org/doi/10.1145/3132747.3132763 Copy Link Share on Social Media Share on * * * * 0References * * * Close Figure Viewer Browse AllReturnChange zoom level[ ] Caption View Table of Contents Export Citations Select Citation format[BibTeX ] * Preview is not available. By clicking download,a new tab will open to start the export process. The process may takea few minutes but once it finishes a file will be downloaded on your browser soplease do not close the new tab. Download + Download citation + Copy citation Categories * Journals * Magazines * Books * Proceedings * SIGs * Conferences * Collections * People About * About ACM Digital Library * ACM Digital Library Board * Subscription Information * Author Guidelines * Using ACM Digital Library * All Holdings within the ACM Digital Library * ACM Computing Classification System * Digital Library Accessibility Join * Join ACM * Join SIGs * Subscribe to Publications * Institutions and Libraries Connect * Contact * Facebook * Twitter * Linkedin * Feedback * Bug Report The ACM Digital Library is published by the Association for Computing Machinery. Copyright (c) 2022 ACM, Inc. * Terms of Usage * Privacy Policy * Code of Ethics ACM Digital Library home ACM home About Cookies On This Site We use cookies to ensure that we give you the best experience on our website. Learn more Got it!