[HN Gopher] GPU memory snapshots: sub-second startup (2025)
       ___________________________________________________________________
        
       GPU memory snapshots: sub-second startup (2025)
        
       Author : jxmorris12
       Score  : 16 points
       Date   : 2026-01-08 17:10 UTC (2 days ago)
        
 (HTM) web link (modal.com)
 (TXT) w3m dump (modal.com)
        
       | erwaen98 wrote:
       | Looks great
        
       | erichocean wrote:
       | Tried it out, first curl after deploy gave me a 303, but second
       | attempt worked.
        
       | Imustaskforhelp wrote:
       | Is modal running every single service inside gvisor?
       | 
       | I have heard that gvisor isn't recommended to run every single
       | production but rather only some front facing or some other
       | activities but it has some serious performance degradation which
       | is why most end up using firecracker
       | 
       | This is really cool though, does this mean that we could probably
       | have AI models that are snapshotted?
       | 
       | Are the states of checkpoint/recovery encrypted by default or how
       | would that even work? Like what are the privacy aspects of it. I
       | don't think even using something like modal would be the private
       | llm that many people sometimes want on subreddits like localllama
       | but the people dont have gpu. of course nothing beats privacy if
       | you have your own gpu's but I'd be curious to know what people's
       | thoughts are
        
         | markasoftware wrote:
         | the thing is modal is running untrusted containers, so there's
         | not really a concept of "some front facing" containers. Any
         | container running an untrusted workload is at high risk / is
         | "front facing".
         | 
         | If Modal's customers' workloads are mainly GPU-bound, then the
         | performance hit of gvisor isn't as big as it might be for other
         | workloads. GPU activity does have to go through the fairly
         | heavyweight nvproxy to be executed on the host, but most gpu
         | activity is longer-lived async calls like running kernels so a
         | bit of overhead in starting / retrieving the results from those
         | calls can be tolerated.
        
           | Imustaskforhelp wrote:
           | Well if someone is gonna use Modal exactly for GPU purposes
           | then I guess its okay but anything compute related just feels
           | like it would have some issues performance wise
           | 
           | So I can agree that perhaps Modal might make sense for LLM's
           | but they position themselves as sandbox including something
           | like running python code etc. and some of this may be more
           | intensive in workflows than others so I just wanted to point
           | it out
           | 
           | Fly.io uses firecracker so I kinda like firecracker related
           | applications (I tried to run firecracker myself its way too
           | hard to build your own firecracker based provider or
           | anything) and they recently released https://sprites.dev/
           | 
           | E2B is another well known solution out there. I have talked
           | to their developers once and they mentioned that they run it
           | on top of gcp
           | 
           | I am really interested in kata containers as well because I
           | think kata runs on top of firecracker and can hook with
           | docker rather quickly.
        
       ___________________________________________________________________
       (page generated 2026-01-10 23:00 UTC)