[HN Gopher] HPC at Autodesk
       ___________________________________________________________________
        
       HPC at Autodesk
        
       Author : savin-goyal
       Score  : 25 points
       Date   : 2023-10-02 05:57 UTC (1 days ago)
        
 (HTM) web link (forums.autodesk.com)
 (TXT) w3m dump (forums.autodesk.com)
        
       | JamesCoyne wrote:
       | Can anyone speak to the general appetite at Autodesk for open-
       | source projects?
        
         | mrcwinn wrote:
         | [Former employee of several years.] Generally speaking, I would
         | say if Autodesk believes open source can advance a business
         | goal, it will support that open source project. But I never got
         | a sense of Autodesk sort of culturally or in a deeply embedded
         | way caring about open source. High marks for how they treat
         | employees and as a place to work, but not an OS leader.
        
           | JamesCoyne wrote:
           | [Current Autodesk CAD user] Thanks! Nice to hear their
           | employees are well treated; atleast some good is coming from
           | the yearly subscriptions.
        
       | kristianp wrote:
       | Actual title: Autodesk and Outerbounds Partner to Open Source Ray
       | and HPC Integration in Metaflow
        
       | riedel wrote:
       | I do not quite get this. How does this enable someone to run ray
       | or metaflow on a typical batch scheduled HPC system (slurm or
       | alike)? Inter node communication is done via the lustre file
       | system, right?
        
         | linksnapzz wrote:
         | I think it said that data access is via Lustre, and
         | communication is by Nvidia MLNX NCCL, which seems to be some
         | kind of nvidia gpu-specific MPI type library; it would seem to
         | be doing RDMA from GPU to GPU via fabric interconnects, so far
         | as I can tell...
        
       | mgaunard wrote:
       | In my experience ray in AWS is a good way to badly utilize
       | resources and waste a lot of money (as is generally anything
       | cloud or anything python; when you do both it multiplies).
       | 
       | I'd rather have a real HPC cluster.
        
         | davnn wrote:
         | Python as a glue language (as it's mostly used in data
         | intensive applications) for something like MPI should not add
         | too much overhead?
        
         | dekhn wrote:
         | You can build a SLURM cluster out of UltraCluster nodes in AWS.
         | Money comparisons can be misleading because many people ignore
         | ancillary expenses in running an HPC facility.
        
           | mgaunard wrote:
           | Virtual machines perform extremely poorly, so you must take
           | metal instances. These will cost you the same as buying the
           | hardware outright after 3 months of usage.
           | 
           | And you're still stuck on a non-deterministic high-latency
           | network you can't get rid of, and with very limited hardware
           | configurations.
           | 
           | It's more like a grid than a HPC cluster.
           | 
           | There are only two possible advantages:
           | 
           | - you want a lot of hardware very quickly rather than wait
           | for it to be delivered.
           | 
           | - you don't have the desire/capability to be/hire a network
           | engineer.
        
       ___________________________________________________________________
       (page generated 2023-10-03 23:01 UTC)