[HN Gopher] Microsandbox: Virtual Machines that feel and perform...
___________________________________________________________________
Microsandbox: Virtual Machines that feel and perform like
containers
Author : makeboss
Score : 218 points
Date : 2025-05-30 13:20 UTC (9 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| appcypher wrote:
| Thanks for sharing!
|
| I'm the creator of microsandbox. If there is anything you need to
| know about the project, let me know.
|
| This project is meant to make creating microvms from your machine
| as easy as using Docker containers.
|
| Ask me anything.
| esafak wrote:
| Looks neat. If I understand correctly, I can use it to spin up
| backends on the fly? You have an ambitious list of languages to
| support:
| https://github.com/microsandbox/microsandbox/tree/main/sdk
|
| edit: A fleshed out contributors guide to add support for a new
| language would help.
| https://github.com/microsandbox/microsandbox/blob/main/CONTR...
| appcypher wrote:
| Yes. Self-hosting and using it on your own backend infra is
| the main use-case. And JVM support should just work since it
| is a Linux machine.
| 0cf8612b2e1e wrote:
| Only did a quick skim of the readme, but a few questions which
| I would like some elaboration.
|
| How is it so fast? Is it making any trade offs vs a traditional
| VM? Is there potential the VM isolation is compromised?
|
| Can I run a GUI inside of it?
|
| Do you think of this as a new Vagrant?
|
| How do I get data in/out?
| appcypher wrote:
| > How is it so fast? Is it making any trade offs vs a
| traditional VM? Is there potential the VM isolation is
| compromised?
|
| It is a lighweight VM and uses the same technology as
| Firecracker
|
| > Can I run a GUI inside of it?
|
| It is planned but not yet implemented. But it is absolutely
| possible.
|
| > Do you think of this as a new Vagrant?
|
| I would consider Docker for VMs instead. In a similar way, it
| focuses on dev ops type use case like deplying apps, etc.
|
| > How do I get data in/out?
|
| There is an SDK and server that help does that and file
| streaming is planned. But right now, you can execute commands
| in the VM and get the result back via the server
| westurner wrote:
| > _I would consider Docker for VMs instead._
|
| Native Containers would probably solve here, too.
|
| From https://news.ycombinator.com/item?id=43553198 :
|
| >>> _ostree native containers are bootable host images that
| can also be built and signed with a SLSA provenance
| attestation;https://coreos.github.io/rpm-ostree/container/
| _
|
| And also from that thread:
|
| > _How should a microkernel run (WASI) WASM runtimes?_
|
| What is the most minimal microvm for WASM / WASI, and what
| are the advantages to running WASM workloads with
| firecracker or microsandbox?
| appcypher wrote:
| > What is the most minimal microvm for WASM / WASI,
|
| By setting up an image with wasmtime for example.
|
| > and what are the advantages to running WASM workloads
| with firecracker or microsandbox?
|
| I can think of stronger isolation or when you have legacy
| stuff you need to run alongside.
| hugs wrote:
| Looks great! This might be extremely useful for a
| distributed/decentralized software testing network I'm building
| (called Valet Network)...
|
| Question: How does networking work? Can I restrict/limit
| microvms so that they can only access public IP addresses? (or
| in other words... making sure the microvms can't access any
| local network IP addresses)
| appcypher wrote:
| Yes! With the `scope` property.
|
| https://github.com/microsandbox/microsandbox/blob/0c13fc27ab.
| ..
| hugs wrote:
| thanks! have an example on how to use that in a
| sandboxfile?
|
| (also, this project is really cool. great work!)
| appcypher wrote:
| Yeah. I need to fix that in the docs!
| hugs wrote:
| no prob!
| simonw wrote:
| What's the story for macOS support?
| appcypher wrote:
| It uses libkrun which uses Hypervisor.framework on macOS.
| wolfhumble wrote:
| Can you use Microsandbox for everything you can use Docker for,
| or are there cases where containers make more sense?
|
| Congratulations on the launch!
| appcypher wrote:
| We want microsandbox to be usable for everything you can with
| Docker.
|
| That said, hosting microVMs require dedicated hardware or VMs
| with nested virt support. Containers don't have that problem.
| nqzero wrote:
| i'm on a mid-level laptop, at times with slow or expensive
| internet, running ubuntu. i want to be able to run nominally-
| isolated "copies" of my laptop at near-native speed
|
| 1. each one should have it's own network config, eg so i can
| use wireguard or a vpn
|
| 2. gui pass-through to the host, eg wayland, for trusted tools,
| eg firefox, zoom or citrix
|
| 3. needs to be lightweight. eg gnome-boxes is dead simple to
| setup and run and it works, but the resource usage was
| noticeably higher than native
|
| 4. optional - more security is better (ie, i might run semi-
| untrusted software in one of them, eg from a github repo or
| npm), but i'm not expecting miracles and accept that escape is
| possible
|
| 5. optional - sharing disk with the host via COW would be nice,
| so i'd only need to install the env-specific packages, not the
| full OS
|
| i'm currently working on a podman solution, and i believe that
| it will work (but rebuilding seems to hammer the network - i'm
| hoping i can tweak the layers to reduce this). does
| microsandbox offer any advantages for this use case ?
| appcypher wrote:
| > 1. each one should have it's own network config, eg so i
| can use wireguard or a vpn
|
| This is possible right now but the networking is not where I
| want it to be yet. It uses libkrun's default TSI impl;
| performant and simplifies setup but can be inflexible. I plan
| to implement an alternative user-space networking stack soon.
|
| > 2. gui pass-through to the host, eg wayland, for trusted
| tools, eg firefox, zoom or citrix
|
| We don't have GUI passthrough. VNC?
|
| > 3. needs to be lightweight. eg gnome-boxes is dead simple
| to setup and run and it works, but the resource usage was
| noticeably higher than native
|
| It is lightweight in the sense that it is not a full vm
|
| > 4. optional - more security is better (ie, i might run
| semi-untrusted software in one of them, eg from a github repo
| or npm), but i'm not expecting miracles and accept that
| escape is possible
|
| The security guarantees are similar to what typical VMs
| support. It is hardware-virtualized so I would say you should
| be fine.
|
| > 5. optional - sharing disk with the host via COW would be
| nice, so i'd only need to install the env-specific packages,
| not the full OS
|
| Yeah. It uses virtio-fs and has overlayfs on top of that for
| COW.
| simonw wrote:
| I'm trying this out now and it's very promising. One problem
| I'm running into with the Python library is that I'd like to
| keep that sandbox running for several minutes while I do things
| like set variables in one call and then use them for stuff
| several calls later. I keep seeing this error intermittently:
| Error: Sandbox is not started. Call start() first
|
| Is there a suggested way of keeping a sandbox around for
| longer?
|
| The documented code pattern is this: async
| def main(): async with
| PythonSandbox.create(name="my-sandbox") as sb:
| exec = await sb.run("print('Hello, World!')")
| print(await exec.output())
|
| Due to the way my code works I want to instantiate the sandbox
| once for a specific class and then have multiple calls to it by
| class methods, which isn't a clean fit for that "async with"
| pattern.
|
| Any recommendations?
| appcypher wrote:
| Right. You can skip the `with` context manager and call start
| and stop yourself.
|
| There is an example of that here:
|
| https://github.com/microsandbox/microsandbox/blob/0c13fc27ab.
| ..
| gcharbonnier wrote:
| async with is just syntactic sugar. You could very well call
| __aenter__ and __aexit__ manually. You could also use an
| AsyncExitStack, call __aenter__ manually, then
| enter_async_context, and call aclose when you're done. Since
| aclose method exists I guess this is not an anti-pattern.
|
| https://docs.python.org/3/library/contextlib.html#contextlib.
| ..
| codethief wrote:
| Hi appcypher, very cool project! Does the underlying MicroVM
| feature provide an OCI runtime interface, so that it could be
| used as a replacement for runc/crun in Docker/Podman?
| Tsarp wrote:
| Wow. This looks awesome.
|
| Can we build our own python sandbox using the sandboxfile spec?
| This is if I want to add my own packages. Would this be just
| having my own requirements file here -
| https://github.com/microsandbox/microsandbox/blob/main/MSB_V...
| appcypher wrote:
| Thank you!
|
| > Can we build our own python sandbox using the sandboxfile
| spec?
|
| Yes and I plan to make that work with the SDK.
|
| PS: Multi-stage build is WIP.
| Tsarp wrote:
| Great will join the discord. Is this embeddable? Will it work
| with a cross platform desktop app(Tauri)?
| jauntywundrkind wrote:
| Why not some of the existing microvm efforts?
|
| Cloud Hypervisor and Firecracker both have an excellent
| reputation for ultra lightweight VM's. Both are usable in the
| very popular Kata Containers project (as well as other upstart
| VM's Dragonball, & StratoVirt). In us by for example the CNCF
| Confidential Containers https://github.com/kata-containers/kata-
| containers/blob/main... https://confidentialcontainers.org/
|
| There's also smaller efforts such as firecracker-containerd or
| Virtink, both which bring OCI powered microvms into a Docker like
| position (easy to slot into Kubernetes), via Firecracker and
| Cloud Hypervisor respectively.
| https://github.com/smartxworks/virtink
| https://github.com/firecracker-microvm/firecracker-container...
|
| Poking around under the hood, microsandbox appears to use krun.
| There is krunvm for OCI support (includes MacOS/arm64 support!).
| https://github.com/containers/krunvm https://github.com/slp/krun
|
| The orientation as a safe sandbox for AI / MCP tools is a very
| nicely packaged looking experience, and very well marketred.
| Congratulations! I'm still not sure why this warrants being it's
| own project.
| simonw wrote:
| If we get enough of these sandboxes, maybe we will finally get
| one that's easy for me to run on my own machines.
| mike_hearn wrote:
| Which platforms do you use?
| simonw wrote:
| macOS on my laptop, anything that runs in a container for
| when I deploy things.
| tough wrote:
| I had luck using ALVM which users Apple Hypervisor
| framework while exploring linux micro-vm's in macos fwiw
| https://github.com/mathetake/alvm
| simonw wrote:
| That looks _really cool_ , but it's missing the one
| feature I want most from anything that runs a sandbox (or
| any security-related software): I need something which a
| billion dollar company with a professional security team
| is running in production on a daily basis.
|
| So much of the solutions to this stuff I see come from a
| GitHub repo with a few dozen commits and often a README
| that says "do not rely on this software yet".
|
| Definitely going to play with it a bit though, I love the
| idea of hooking into Apple's Hypervisor.framework (which
| absolutely fits my billion-dollar-company requirement.)
| ericb wrote:
| Working gVisor Mac install instructions here.
|
| https://dev.to/rimelek/using-gvisors-container-runtime-
| in-do...
|
| After this is done, it is:
|
| docker run --rm --runtime=runsc hello-world
| tough wrote:
| would you be OK with a -hardened- with default profiles
| docker containers one?
| appcypher wrote:
| I don't understand what you mean? Can you clarify?
| tough wrote:
| sorry i meant to ask simon directly if they require a
| non-docker solution
|
| im working on a wrapper that lets you swap runtimes and
| my first implementation is mostly a wrapper around docker
| containers
|
| planning to add firecracker next
|
| will explore adding microsandbox too cool stuff!
| simonw wrote:
| My ideal solution is non-Docker purely because I build
| software for other people to use. I don't want to have to
| tell my users "step 1: install Docker" if I can avoid it.
| tough wrote:
| that does make sense, sadly firecracker seems to be
| mostly relegated to linux for now so there's no good
| multi-arch story i'm aware of
| appcypher wrote:
| That's the plan lol. There is too much friction setting up
| existing solutions.
| hobofan wrote:
| Exactly my thoughts when I read the headline, after having
| read a similar one every few months.
|
| However, by looking at it and playing with a few simple
| examples, I think this is the one that looks the closest so
| far.
|
| Definitely interested to see the FS support, and also some
| instruction on how to customize the images to e.g. pre-
| install common Python packages or Rust crates. As an example,
| I tried to use the MCP with some very typical use-cases for
| code-execution that OpenAI/Anthropic models would generate
| for data analysis, and they almost always include using numpy
| or a excel library, so you very quicly hit a wall here
| without the ability to include libraries.
| appcypher wrote:
| Because those have different directions than microsandbox and
| you've already mentioned one. I want easy secure sandboxes for
| AI builders. IMHO, microsandbox is easier to get started with.
|
| That said I don't think either KataContainer or Cloud
| Hypervisor has first-class support for macOS.
| dataflow wrote:
| Tangential question: why does it normally take so long to start
| traditional VMs in the first place? At least on Windows, if you
| start a traditional VM, it takes several seconds for it to start
| running _anything_.
|
| Edit: when I say _anything_ , I'm not talking user programs. I
| mean as in, before even the first instruction of the firmware --
| before even the virtual disk file is zeroed out, in cases where
| it needs to be. You literally can't pause the VM during this
| interval because the window hasn't even popped up yet, and even
| when it has, you still can't for a while because it literally
| hasn't started running _anything_. So the kernel and even
| firmware initialization slowness are entirely irrelevant to my
| question.
|
| Why is that?
| diggan wrote:
| I mean it is basically booting a computer from scratch, kind of
| makes sense. You have to allocate memory, start virtual CPUs,
| initialize devices, run BIOS/UEFI checks, perform hardware
| enumeration, all that jazz while emulating all of it, which
| tends to be slower than "real" implementations. I guess there
| is a bunch of processes for security as well, like wiping like
| zeroing pages and similar things that takes additional time.
|
| If I let a VM use most of my hardware, it takes a few seconds
| from start to login prompt, which is the same time it takes for
| my Arch desktop to boot from pressing the button to seeing the
| login prompt.
| dataflow wrote:
| > You have to allocate memory, start virtual CPUs, initialize
| devices, run BIOS/UEFI checks, perform hardware enumeration,
| all that jazz while emulating all of it, which tends to be
| slower than "real" implementations.
|
| That's not what I'm asking.
|
| I'm saying it takes a long time for it to even execute a
| single instruction, in the BIOS itself. Even for the window
| to pop up, before you can even pause the VM (because it
| hasn't even started yet). What you're describing comes after
| all that, which I already understand and am not asking about.
| drewg123 wrote:
| Without any context in terms of what the VM is doing or
| what VMM software you use, my best guess is that the OS/VMM
| are pre-allocating memory for the VM. This might involve
| paging out other processes' memory, which could take some
| time.
|
| I think task manager would tell you if there is a blip of
| memory usage and paging activity at the time. And I'm sure
| windows itself has profilers that can tell you what is
| happening when the VM is started..
| dataflow wrote:
| VirtualBox on Windows, primarily. Though I feel like
| haven't seen other VMs in the past start up a whole ton
| faster (maybe a somewhat) (ignoring WSL2). Page files are
| already disabled, there's plenty of free RAM, and it
| makes no difference how little RAM the guest is allocated
| (even if it's 256MB). So no, those are not the issues.
| VirtualBox itself seems to be doing something slow during
| that time and I don't know what that is.
| mynameisvlad wrote:
| So the issue is pretty clearly with VirtualBox itself,
| but you are making it sound like it's an issue with VMs
| on Windows or in general.
| gopher_space wrote:
| I remembered something about VirtualBox not playing
| nicely with Hyper-V on Windows, and dug up a possibly
| relevant post[0] on their forums. IIRC we ended up moving
| a few build systems to Docker and dropping VirtualBox
| because of hyper-v related issues, but it's been a few
| years.
|
| [0] https://forums.virtualbox.org/viewtopic.php?t=112113
| dataflow wrote:
| That's the unrelated green-turtle issue. It's only
| relevant after the guest has actually started running
| instructions. I'm talking about before that point.
| gopher_space wrote:
| I'm not aware of any turtles, that was just the first
| thing I found when trying to see if VirtualBox and
| Hyper-V were still a problematic combo.
|
| Again, it was a few years ago, but we didn't solve the
| problem or identify an actual root cause. We stopped
| banging our heads against that particular wall and
| switched technologies.
| drewg123 wrote:
| For some reason I can't reply to your reply. I'd strongly
| suggest that you profile virtual box. It beats
| speculation..
| bityard wrote:
| In defense of the replies, your initial question was very
| vague and left people to assume you meant the obvious
| thing.
| dataflow wrote:
| Sure, that's why I clarified.
| jeroenhd wrote:
| You can optimize a lot to start a Linux kernel in under a
| second, but if you're using a standard kernel, there are all
| manners of timeouts and poll attempts that make the kernel
| waste time booting. There's also a non-trivial amount of time
| the VM spends in the UEFI/CSM system preparing the virtual
| hardware and initializing the system environment for your
| bootloader. I'm pretty sure WSL2 uses a special kernel to avoid
| the unnecessary overhead.
|
| You also need to start OS services, configure filesystems,
| prepare caches, configure networking, and so on. If you're not
| booting UKIs or similar tools, you'll also be loading a
| bootloader, then loading an initramfs into memory, then loading
| the main OS and starting the services you actually need, with
| eachsstep requiring certain daemons and hardware probes to work
| correctly.
|
| There are tools to fix this problem. Amazon's Firecracker can
| start a Linux VM in a time similar to that of a container
| (milliseconds) by basically storing the initialized state of
| the VM and loading that into memory instead of actually
| performing a real boot. https://firecracker-microvm.github.io/
|
| On Windows, I think it depends on the hypervisor you use. Hyper
| V has a pretty slow UEFI environment, its hard disk access
| always seems rather slow to me, and most Linux distro don't
| seem to package dedicated minimal kernels for it.
| dataflow wrote:
| That's not what I'm asking about.
|
| I'm saying it takes a long time for it to even execute a
| single instruction, in the BIOS itself. Even for the window
| to pop up, before you can even pause the VM (because it
| hasn't even started yet). What you're describing comes after
| all that, which I already understand and am not asking about.
| hnuser123456 wrote:
| probably the intel ME setting up for virtualization in a
| way that it can infiltrate
| LoganDark wrote:
| Ah yes, the source of all slowness in the CPU: hostile
| backdoors taking their time to compromise the work.
| Classic...
| orev wrote:
| I think you need to provide more details on what VM software
| you're using. On VirtualBox what you describe is very
| noticeable, and it didn't have that delay in older versions. So
| it could be just an issue with that VM software and not a
| general "traditional VMs" issue.
| dataflow wrote:
| Yup I'm asking about VirtualBox mainly, I just don't
| understand what the heck it's doing during that time that
| takes so long. Although I don't recall other VMs (like say,
| Hyper-V) being dramatically different either (ignoring WSL2
| here).
| _factor wrote:
| Try disabling Windows Defender and trying again.
| dataflow wrote:
| Are you just guessing or have you actually seen the delay
| I'm talking about disappear as a result of this (or as a
| result of anything else for that matter)? Because I've
| already done this (yes, entirely, even the kernel mode
| drivers) and it's definitely not the issue.
| hinkley wrote:
| There was a release of subversion back in the day that
| reduced the number of files that were opened during a
| repo action like pull, and the number of times any one
| file got opened. On Linux it ran about 2-3x faster. Very
| nice change.
|
| On windows it was almost 10x faster. On the project where
| this change was released, my morning ritual was to come
| in, log on, run an svn pull command, lock my screen and
| go get coffee. I had at least ten minutes to kill after I
| got coffee, if the pot wasn't empty when I got there.
|
| Windows is hot garbage about fopen particularly when
| virus scanning is on.
| icedchai wrote:
| Linux KVM/qemu VMs start pretty fast.
| speed_spread wrote:
| Creating the VM itself is fast. It depends on what you run in
| it. Unikernel VMs can start in a few milliseconds. For example,
| checkout OSv.
| dataflow wrote:
| You're saying this is true on a _Windows_ host?
| akdev1l wrote:
| Yes. The delay you're complaining about happens because you
| are looking at general hypervisors which also come with
| virtualized hardware and need to mimic a bunch of stuff so
| that most software will work as usual.
|
| For example: your VM starts up with the CPU in 16 bit mode
| because that's just how things work in x86 and then it
| waits for the guest OS to set the CPU into 64 bit mode.
|
| This is completely unnecessary if you just want to run
| x86-64 code in a virtualized environment and you control
| the guest kernel and can just assume things are in 64bit
| mode because it's not the 70s or whatever
|
| The guest OS would also need to probe few ports to get a
| bootable disk. If you control the kernel then you can just
| not do that and boot directly.
|
| There's a ton of stuff that isn't needed
| dataflow wrote:
| The 16 bit mode stuff and the guest OS probes are after
| what I'm asking, not before.
| akdev1l wrote:
| No it is not. The "first instruction in the BIOS" is 16
| bit mode code when dealing with an x86 VM.
|
| A virtual environment doesn't even really need any BIOS
| or anything like that.
|
| You can feel free to test with qemu direct kernel booting
| to see this skips a lot of delay without even having to
| use a specialized hypervisor like firecracker
| dist-epoch wrote:
| Sounds like a VirtualBox problem.
|
| I'm using Hyper-V and I can connect through XRDP to a GUI
| Ubuntu 22 in 10 seconds and I can SSH into a Ubuntu 22 server
| in 3 seconds after start.
| akdev1l wrote:
| The answer is that it doesn't have to be like that.
|
| In practice virtual machines are trying to emulate a lot of
| stuff that isn't really needed but they're doing it for
| compatibility.
|
| If one builds a hypervisor which is optimized for startup speed
| and doesn't need to support generalized legacy software then
| you can:
|
| > Unlike traditional VMs that might take several seconds to
| start, Firecracker VMs can boot up in as little as 125ms.
| jiggawatts wrote:
| Try Windows Server Core on an SSD. I've seen VMs launch in low
| single-digit seconds. You can strip it down even further by
| removing non-64-bit support, Defender, etc...
| Jayakumark wrote:
| Windows support ? and can we VNC in to the sandbox and stream it
| ?
| appcypher wrote:
| Windows support is a work in progress. I haven't tested using
| VNC yet but it should be possible.
| h1fra wrote:
| Can't wait to test, if it's really what's advertised it would be
| much easier to use than workerd or firecracker
| McAlpine5892 wrote:
| This looks awesome. The amount of super lightweight and almost-
| disposable VM options in recent years is crazy. I remember when
| VMs were slow, clunky, and generally painful.
|
| I wonder how this compares to Orbstack's [0] tech stack on macOS,
| specifically the "Linux machines" [1] feature. Seems like Orb
| might reuse a single VM?
|
| ---
|
| [0] https://orbstack.dev
|
| [1] https://docs.orbstack.dev/machines/
| jbverschoor wrote:
| Related, https://github.com/jrz/container-shell which uses docker
| to create adhoc shells / chroots in the current directory.
| manveru wrote:
| Are the SDKs AI generated? I looked at the Crystal, Ruby, and Zig
| ones and all they contain is a hello world example with some docs
| that have little to do with the code. Sorry if this comment seems
| rude, just curious.
| appcypher wrote:
| The other SDKs are generated hello-worlds at the moment. I will
| get to them one by one, but I welcome and appreciate any
| contributions to them.
| jmehman wrote:
| I've been looking for something I could host for this kind of
| thing - for LLM agents. Ended up on https://www.daytona.io/ as I
| couldn't find anything suitable to self host and realised it was
| a complex thing to manage. It seems Daytona is open source,
| including the server platform, but there is no documentation for
| the server element. Azure also seem to offer a service for this,
| it's a space that is growing rapidly.
| appcypher wrote:
| Microsandbox is for people that would like to maintain their
| own infra. I'm not going to stop trying to make it better to
| self-host.
| jmehman wrote:
| Yeah, it looks great, makes me reconsider the self hosted
| route
| patrick4urcloud wrote:
| very nice ! i will definetly try
| ATechGuy wrote:
| Congrats on launching! Booting VMs in milliseconds is certainly
| important, but it can also be achieved with
| CloudHypervisor/Firecracker. Where Containers beat VMs is runtime
| perf. The overhead in case of VMs stems from emulation of IO
| devices. I believe the overhead will become noticeable for AI
| agentic use cases. Any plans to address perf issues?
| appcypher wrote:
| You are right. We leverage libkrun. Libkrun uses virtio-mmio
| transport for block, vsock and virtio-fs to keep overhead
| minimal so we basically depend on any perf improvement made
| upstream.
|
| Firecracker is no different btw and E2B uses that for agentic
| AI workloads. Anyway, I don't have any major plan except fix
| some issues with the filesystem rn.
| SwiftyBug wrote:
| Kind of almost off-topic: I'm working on a project where I must
| run possibly untrusted JavaScript code. I want to run it in an
| isolated environment. This looks like a very nice solution as I
| could spin up a microsandbox and securely run the code. I could
| even have a pool os live sandboxes so I wouldn't even experience
| the 200ms starts. Because this is OCI-compatible, I could even
| provide a whole sandboxed environment on which to run that code.
| Would that be a good use case for this? Are there better
| alternatives?
| appcypher wrote:
| > Would that be a good use case for this?
|
| That is an ideal use case
|
| > Are there better alternatives?
|
| Created microsandbox because I didn't find any
| SwiftyBug wrote:
| Awesome. This is really good timing. I'm going to give it a
| try.
| ericb wrote:
| runsc / gVisor is interesting also as the runsc engine can be
| run from within Docker/Docker Desktop.
|
| gVisor has performance problems, though. Their data shows 1/3rd
| the throughput vs. docker runtime for concurrent network calls
| --if that's an issue for your use-case.
| hinkley wrote:
| How's performance? What's the overhead versus docker? Terraform
| or Pulumi integration on the horizon?
| elwebmaster wrote:
| One topic I am not finding anything about is networking. Can
| these microsandbox instances listen on ports? How is the port
| forwarding configured? Can they access the internet or any
| resources on the host?
| appcypher wrote:
| They can. I need to improve the doc. Working on that right now
| zackmorris wrote:
| This is great!
|
| I'd like to see a formal container security grade that works
| like: 1) Curate a list of all known (container)
| exploits 2) Run each exploit in environments of increasing
| security like permissions-based, jail, Docker and emulator
| 3) The percentage of prevented exploits would be the score from
| 0-100%
|
| Under this scheme, I'd expect naive attempts at containerization
| with permissions and jails to score around 0%, while Docker might
| be above 50% and Microsandbox could potentially reach 100%.
|
| This might satisfy some of our intuition around questions like
| "why not just use a jail?". Also the containers could run on a
| site on the open web as honeypots with cash or crypto prizes for
| pwning them to "prove" which containers achieve 100%.
|
| We might also need to redefine what "secure" means, since
| exploits like Rowhammer and Spectre may make nearly all
| conventional and cloud computing insecure. Or maybe it's a moving
| target, like how 64 bit encryption might have once been
| considered secure but now we need 128 bit or higher.
|
| Edit: the motivation behind this would be to find a container
| that's 100% secure without emulation, for performance and cost-
| savings benefits, as well as gaining insights into how to secure
| operating systems by containerizing their various services.
| bjackman wrote:
| You cannot build a secure container runtime (against malicious
| containers) because underlying it is the Linux kernel.
|
| The only way to make Linux containers a meaningful sandbox is
| to drastically restrict the syscall API surface available to
| the sandboxee, which quickly reduces its value. It's no longer
| a "generic platform that you can throw any workload onto" but
| instead a bespoke thing that needs to be tuned and reconfigured
| for every usecase.
|
| This is why you need virtualization. Until we have a properly
| hardened and memory safe OS, it's the only way. And if we do
| build such an OS it's unclear to me whether it will be faster
| than running MicroVMs on a Linux host.
| Veserv wrote:
| You cannot build a secure virtualization runtime because
| underlying it is the VMM. Until you have a secure VMM you are
| subject to precisely the same class of problems plaguing
| container runtimes.
|
| The only meaningful difference is that Linux containers
| target partitioning Linux kernel services which is a shared-
| by-default/default-allow environment that was never designed
| for and has never achieved meaningful security. The number of
| vulnerabilities resulting from, "whoopsie, we forgot to
| partition shared service 123" would be hilarious if it were
| not a complete lapse of security engineering in a product
| people are convinced is adequate for security-critical
| applications.
|
| Present a vulnerability assessment demonstrating a team of 10
| with 3 years time (~10-30 M$, comparable to many
| commercially-motivated single-victim attacks these days) can
| find no vulnerabilities in your deployment or a formal proof
| of security and correctness otherwise we should stick with
| the default assumption that software if easily hacked instead
| of the extraordinary claim that demands extraordinary
| evidence.
| akdev1l wrote:
| One can definitely build a container runtime that uses
| virtualization to protect the host
|
| For example there is Kata containers
|
| https://katacontainers.io/
|
| This can be used with regular `podman` by just changing the
| container runtime so there's no even need for any extra
| tooling
|
| In theory you could shove the container runtime into
| something like k8s
| ignoramous wrote:
| > _... drastically restrict the syscall API surface available
| to the sandboxee, which quickly reduces its value ..._
|
| Depends I guess as Android has had quite a bit of success
| with seccomp-bpf & Android-specific flavour of SELinux [0]
|
| > _Until we have a properly hardened and memory safe OS ...
| faster than running MicroVMs on a Linux host._
|
| Andy Tanenbaum might say, Micro Kernels would do just as
| well.
|
| [0] https://youtu.be/WxbOq8IGEiE
| carlhjerpe wrote:
| You also have gVisor, which runs all syscall through some
| Go history that's supposedly safe enough for Google.
| tptacek wrote:
| The issue, at least with multitenant workloads, isn't
| "container vulnerabilities" as such; it's that standard
| containers are premised on sharing a kernel, which makes every
| kernel LPE a potential container escape --- there's a long
| history of those bugs, and they're only rarely flagged as
| "container escapes"; it's just sort of understood that a kernel
| LPE is going to break containers.
| Etheryte wrote:
| In a way, containers already run as honeypots with cash or
| crypto prizes, it's called production code and plenty of people
| are looking for holes day and night. While this setup sounds
| like a nice idea conceptually, the monetary incentives it could
| offer would surely be miniscule compared to real targets.
| godelski wrote:
| Importantly I'd like to see the configurations of the machines.
| There's a lot you can do to docker or systemd spawns that
| greatly vary the security levels. This would really help show
| what needs to be done and what configurations lead to what
| risks.
|
| Basically I'd love to see a giant ablation
| rbitar wrote:
| Looks great and excited to try this out. We've also had success
| using CodeSandbox SDK and E2B, can you share some thoughts on how
| you compare or future direction? Do you also use Firecracker
| under the hood?
| pkkkzip wrote:
| I can't tell if it uses firecracker but thats my main question
| too. I'm curious as to whether microsandbox will be maintained
| and proper auditing will be done.
|
| I welcome alternatives. It's been tough wrestling with
| Firecracker and OCI images. Kata container is also tough.
| appcypher wrote:
| It will be maintained as I will be using it for some other
| product. And it will be audited in the future but it still
| early days.
| appcypher wrote:
| > can you share some thoughts on how you compare or future
| direction?
|
| Microsandbox does not offer a cloud solution. It is self-
| hosted, designed to do what E2B does, to make it easier working
| with microVM-based sandboxes on your local machine whether that
| is Linux, macOS or Windows (planned) and to seamlessly
| transition to prod.
|
| > Do you also use Firecracker under the hood?
|
| It uses libkrun.
| rbitar wrote:
| Self-hosting is definitely something we are keen to explore
| as most of the cloud solutions have resource constrains (ie,
| total active MicroVMs and/or specs per VM) and managing
| billing gets complicated even with hibernation features.
| Great project and we'll definitely take it for a spin
| sureglymop wrote:
| Always interested when things like this come up.
|
| What like about containers is how quickly I can run something,
| e.g. `docker run --rm ...` without having to specify disk size,
| amount of cpu cores, etc. I can then diff the state of the
| container with the image (and other things) to see what some
| program did while it ran.
|
| So I basically want the same but instead with small vms to have
| better sandboxing. Sometimes I also use bwrap but it's not really
| intended to be used on the command line like that.
| srmatto wrote:
| It has a YAML config format to declare all of that so you could
| just do that once, or template it, generate it on the fly,
| fetch it from remote, or many other methods.
| eamann wrote:
| > Ever needed to run code you don't fully trust?
|
| Then the installation instructions include piping a remote script
| directly to Bash ... Oh irony ...
|
| That said, the concept itself is intriguing.
| appcypher wrote:
| Your statement initially went over my head. Sorry lol. You can
| always download the installer script and audit yourself. I will
| set up proper distribution later.
| amelius wrote:
| For my taste, container technology is pushing the OS too far. By
| typing: mount
|
| you immediately see what I mean. Stuff that should be hidden is
| now in plain sight, and destroys the usefulness of simple system
| commands. And worse, the user can fiddle with the data
| structures. It's like giving the user peek and poke commands.
|
| The idea of containers is nice, but they are a hack until kernels
| are re-architected.
| sbassi wrote:
| There are python and node environment for this, so they are not
| VMs in the sense that I can host a OS and arbitrary executables?
| appcypher wrote:
| They are Linux VMs and you can host any executable that can
| work on that. The python/node environment you see is part of
| what makes the SDK work. Really, it's very similar to Docker in
| use.
| sbassi wrote:
| thank you. Is there any "docker host" or centralized repo
| where I can pull VMs from?
| appcypher wrote:
| We support just Docker hub for now. Let me know if you want
| any other OCI-compatible registry.
|
| PS: microsandbox will likely have its own OCI registry in
| the future
___________________________________________________________________
(page generated 2025-05-30 23:00 UTC)