[HN Gopher] Show HN: Software for Remote GPU-over-IP
       ___________________________________________________________________
        
       Show HN: Software for Remote GPU-over-IP
        
       We built installable software for Windows & Linux that makes any
       remote Nvidia GPU accessible to, and shareable across, any number
       of remote clients running local applications, all over standard
       networking.
        
       Author : stevegolik
       Score  : 44 points
       Date   : 2022-12-14 16:10 UTC (6 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | allanrbo wrote:
       | It surprises me that this works well enough to be useful. I would
       | have thought that network latency, being orders of magnitude
       | higher than memory latency, would be a huge problem. Latency
       | Numbers Everyone Should Know:
       | https://static.googleusercontent.com/media/sre.google/en//st...
        
         | capableweb wrote:
         | For gaming, this is obviously a no-go. But for bunch of AI/ML
         | related workloads, it might make perfect sense.
        
           | bob1029 wrote:
           | Not so sure about no-go. The amount of GPU latency in modern
           | AAA titles already approaches 20+ms in the most egregious
           | cases.
           | 
           | Unless there is a need to evict all gpu memory on every
           | frame, I think it is feasible to game on GPUs that live
           | across a very fast LAN.
        
           | Melatonic wrote:
           | Fast ethernet is getting cheaper than ever - you can easily
           | get 10gb on consumer gear or even 20 and used hardware on I
           | believe 40 or maybe 100 is getting pretty affordable.
        
         | delijati wrote:
         | PCI-Express 16x 4.0 has 31,5 GByte/s. The fastest fiber ETH has
         | 50 GB/s. So it "could" be useful if you have datacenter grade
         | equipment ;)
        
           | AnIdiotOnTheNet wrote:
           | Those aren't latency numbers though, they're throughput.
        
           | [deleted]
        
         | cobertos wrote:
         | I'd be surprised if this works for anything latency sensitive
         | over anything more than a LAN.
         | 
         | Even just the time it takes speed of light between NY and LA (4
         | _10^6m /3_10^8m/s=1/75s) is roughly how long a 60 fps frame is
         | (1/60s). Add OS serializing the frame from the GPU onto the
         | network card, network switching of those packets, and you're
         | starting to really feel that latency.
        
           | denkmoon wrote:
           | There are people out there gaming at 30fps with their TV set
           | to Super Duper Image Processing Mode 500ms Latency Edition.
           | Though I suppose these are realistically already served by
           | the cloud gaming offerings.
        
           | johanvts wrote:
           | The datacenter is probably not thousands, but hundreds of
           | kilometers away so there is room to deliver 60fps. I was
           | surprised how well GeForce Now works.
        
       | xrd wrote:
       | I see lots of comments in various ML repositores about trouble
       | running on multiple GPUs. This seems like a great way to run
       | across multiple low VRAM GPUs instead of buying a huge expensive
       | single card. It feels reminiscent of how Google built their
       | clusters on commodity hardware where they would just throw away a
       | failed device rather than trying to fix it. This is really cool.
        
       | fock wrote:
       | Didn't we have those things already? Virtual-GL and Co. say hi.
       | 
       | Also for most real GPU applications, you need to get the data in
       | and out. I don't think splitting compute across a (insert any
       | non-Infiniband-link) solves this
        
         | Melatonic wrote:
         | 100gbe is pretty similar to infiniband no? Or does infiniband
         | still kill it on latency?
        
       | zamadatix wrote:
       | That's really awesome. I'm not sure what I'd use it for but just
       | being able to makes me want to find an excuse! What's impressive
       | is this seems to have more capabilities than most "local"
       | software vGPU solutions for e.g. VMs.
        
       | nimitt wrote:
       | Do you have any numbers on the viability of using this for ML/AI
       | workloads? seems like once a model is ingested into a gpu vram
       | theoretically the transactional new inputs / outputs would be
       | trivial.
        
       | Mo3 wrote:
       | Damn, this is cool. Nice work.
        
       | dezmou wrote:
       | does it really feel like the GPU I use is one on my machine ? or
       | do I have lot of boilerplate to make it work client side ?
        
         | neuronexmachina wrote:
         | I haven't tried it yet, but based on their doc it seems like
         | after setting the host in the juice.cfg, you basically just
         | need to run `juicify [application path]`:
         | https://github.com/Juice-Labs/Juice-Labs/wiki/Juice-for-Wind...
        
       | yangikan wrote:
       | Very nice.
        
       ___________________________________________________________________
       (page generated 2022-12-14 23:01 UTC)