[HN Gopher] Building unorthodox deep learning GPU machines
       ___________________________________________________________________
        
       Building unorthodox deep learning GPU machines
        
       Author : simonpure
       Score  : 79 points
       Date   : 2024-02-28 01:48 UTC (21 hours ago)
        
 (HTM) web link (www.kyleboddy.com)
 (TXT) w3m dump (www.kyleboddy.com)
        
       | gnabgib wrote:
       | Unorthodox machines, and possibly not a load-bearing site, but
       | luckily it's in an archive: https://archive.is/fARR4
        
         | fbdab103 wrote:
         | Hopefully not hosted on Netlify.
        
       | verditelabs wrote:
       | I think I recognize the author of this from /r/localllama, where
       | plenty of other people are building similar frankenstein rigs.
       | This post only mentions Intel setups, but AMD Epyc Milan and Rome
       | based rigs are also very viable alternatives. They're a bit more
       | expensive, but much better perf/watt, and the incremental price
       | increase after factoring in a lot of GPUs is fairly slim. With 7
       | PCIE bifurcators on a motherboard such as the AsRock ROMED8-2T
       | and a 14 risers, you can get up to 14 GPUs at PCIE 4.0x8.
        
         | 3abiton wrote:
         | A bit sad hobbyist have to resort to such measures to get
         | tinkering, not to mention the initial capital needed. We're all
         | slaves to Nvidia's VRAM monopoly until AMD or Intel steps in
         | and release a competitive alternative with beefy vrams.
        
           | loudmax wrote:
           | Beefy VRAM is a start, but Nvidia's real moat is CUDA. If
           | PyTorch runs on AMD's ROCm, or the Intel equivalent, as well
           | as it runs on CUDA, then we'll see some real competition
           | here.
           | 
           | Chris Lattner's Mojo programming language may present an
           | alternative here, but it's still closed source.
        
       | kookamamie wrote:
       | Do note that, due to NVidia's absurd EULAs, you cannot run RTX
       | 3090 in a data-center for compute purposes (https://www.nvidia.co
       | m/content/DriverDownloads/licence.php?l...).
       | 
       | Should you care about the EULA? Not really, until you're a
       | business of any significant scale.
        
         | echelon wrote:
         | If you update your drivers. You could use an older license.
        
           | hkgirjenk wrote:
           | Nvidia is known to retaliate (delaying future shipments) if
           | you do stuff like this.
        
             | PrayagBhakar wrote:
             | I run a basement compute server[^1], what's Nvidia gonna
             | do? Not let me buy their hella expensive H100s? At least
             | now I get to learn ML skills without my failed experiments
             | exponentially scaling on the cloud.
             | 
             | [^1]: https://prayag.bhakar.org/apollo-ai-compute-cluster-
             | for-the-...
        
       | brucethemoose2 wrote:
       | Its insane that a _2020_ gaming GPU is such a niche for deep
       | learning. Thats ancient history in the world of graphics, and its
       | only that way because the market is straight up anticompetitive
       | (and AMD is too dense to break out of it with a 48GB consumer
       | card).
        
         | hackerlight wrote:
         | 48GB consumer card won't change much if people are already
         | passing over the much cheaper AMD 7900XTX that has 24GB. Nvidia
         | is winning because of the perceived CUDA to ROCm gap, that's
         | all it comes down to. (and they're winning in the consumer
         | space because of the DLSS and FSR gap)
        
         | HPsquared wrote:
         | And that it was considered highly overpriced at the time (for
         | gaming) but now seen as a bargain choice.
        
         | shikon7 wrote:
         | The RTX 3090 is still one of the best gaming/consumer GPUs,
         | beat only by the 4080 and 4090. And the 5000 series might still
         | be almost a year away.
        
       | johnklos wrote:
       | Maybe they should build an unorthodox web server, which these
       | days seems to be one that's self hosted, instead of trying to use
       | a weak Digitalocean instance.
       | 
       | I bet the machine mentioned in the article could host this site,
       | even with Xeons, with two orders of magnitude more traffic before
       | being able to register the load.
        
       | nmfisher wrote:
       | Website just gives me "Error establishing a database connection".
       | 
       | (Did he achieve AGI internally?)
        
       | siwakotisaurav wrote:
       | HN hug of death
        
       | Weryj wrote:
       | My unorthodox contribution is an Intel Optane drive as a swap
       | drive to give my CPU only box 96GB of ram to work with.
        
       ___________________________________________________________________
       (page generated 2024-02-28 23:02 UTC)