[HN Gopher] Cloudflare Gen 12 Server: Bigger, Better, Cooler in ...
       ___________________________________________________________________
        
       Cloudflare Gen 12 Server: Bigger, Better, Cooler in a 2U1N Form
       Factor
        
       Author : todsacerdoti
       Score  : 70 points
       Date   : 2023-12-01 18:46 UTC (4 hours ago)
        
 (HTM) web link (blog.cloudflare.com)
 (TXT) w3m dump (blog.cloudflare.com)
        
       | fasteo wrote:
       | >>> A single fan running at full speed consumes about 40W
       | 
       | I thought fans consumed like 10 times less
        
         | temp0826 wrote:
         | A normal fan in your home PC yes. Not the 40mm jet engines you
         | only find in data centers.
        
           | buildbot wrote:
           | Yeah some of these fans are capable of lifting off helicopter
           | style...
        
           | amluto wrote:
           | I'm a bit surprised the big datacenter / OpenCompute buyers
           | haven't moved toward a form factor that's more square. Big,
           | high-power, quiet desktop systems have been using monster
           | fans for many years, and ISTM one could do better with
           | systems that are maybe 4U high but only half width, enabling
           | a proper fan in back. Or maybe even a form factor that's
           | packs a lot of individual machines in front of a full width
           | truly immense fan (19" or more!).
        
             | jeffrallen wrote:
             | In 1999, Tellme was using square servers from Dialogic, a
             | vendor of PCI Telco cards.
        
             | Voultapher wrote:
             | Oxide Computer has for their systems. Could be a trend.
        
             | wmf wrote:
             | Facebook/Meta has been using 2OU (which is slightly taller
             | than 2U) for years. Oxide also uses 2U.
        
       | buildbot wrote:
       | I guess it must be, but does the performance per watt make sense
       | going from 350 to 500W? That's a 42% increase in power! I suppose
       | the # of cores per watt per u makes this solution make sense,
       | especially with the 1u cooling overhead, but it seems like it
       | might be closer to a wash than one might think initially.
        
         | mastax wrote:
         | The 500W is an estimate for the next generation CPUs so we have
         | to speculate, but yes the performance per watt should make
         | sense unless those products are awful. Thus far every EPYC
         | generation has been substantially better performance/watt but
         | the top-end power draw has gone up because the number of cores
         | has gone up. EPYC 3 (Milan) maxed out at 64 cores but EPYC 4
         | maxes out at 96 (Genoa Zen4) or 128 (Bergamo Zen4c). So you get
         | twice as many cores, faster cores, and much less than twice the
         | max power draw (~240W > ~360W). The highest core count CPUs
         | tend to be the most efficient as well because you amortize the
         | relatively static power draw of the IO-die (and the other parts
         | on the motherboard) and because they tend to have lower more
         | efficient clock frequencies.
         | 
         | It's pretty safe to assume the situation will be the same with
         | next generation EPYC 5 CPUs with maybe 192 or 256 cores, higher
         | power draw, but more efficiency.
        
           | buildbot wrote:
           | Makes a lot of sense, thank you!
        
       | soggybread wrote:
       | I'm surprised. They mention that cooling is only going to get
       | worse and that fans consume a lot of power, should liquid cooling
       | not be something to look at? Maybe they did and found it either
       | unnecessary or improbable to do at a colo depending on how many
       | servers they have at the data center it might not be worthwhile
       | but if they have 10 or 20 servers surely it would be worth while
       | no?
        
         | yaxe1 wrote:
         | My guess is liquid cooling isn't reliable enough. Something is
         | going to leak eventually.
        
         | bluedino wrote:
         | You need tanks, plumbing, pumps, coolant...if the datacenter
         | doesn't have it available it's not a small undertaking.
         | 
         | Example: https://www.supermicro.com/en/solutions/liquid-
         | cooling#solut...
        
           | soggybread wrote:
           | You're right and while it might take up valuable rack space I
           | would imagine there are some rack-based solutions that have
           | most of the stuff included so that you don't have to rely on
           | the dc providing it, pretty sure ServeTheHome has done a
           | couple reports on them
        
           | amluto wrote:
           | There are at least a couple vendors with self-contained
           | liquid cooling systems, kind of like the "all-in-one" systems
           | available for desktop and gaming use. I don't know how
           | effective they are.
        
         | aftbit wrote:
         | AIUI this is just starting to take off. Look at Supermicro's
         | 2U4N and 1U2N offerings, which need to use liquid cooling to
         | reach that density. I know Alibaba has been doing vertical
         | mounted servers in a non-conductive oil bath for a while. I bet
         | their generation 13 servers have liquid cooling.
        
         | eastdakota wrote:
         | If you're building your own facility, there's a lot of things
         | you can do with design. In our case, these servers need to go
         | into other people's facilities (e.g., ISPs around the world) so
         | we need to live within their constraints. Things like liquid
         | cooling either wouldn't be possible or would add significant
         | complexity to install and maintain.
        
       | alberth wrote:
       | I might have missed it but ...
       | 
       | What CPU did they end up selecting?
       | 
       | Cores/RAM/Disk ... even GPU since they mention the importance.
       | 
       | TL:DR - newer CPUs TDP is so high, they needed to change from 1U
       | to 2U and that changes economics at a macro level per rack (since
       | you're density now halves).
        
         | wmf wrote:
         | They didn't say what CPU they're using. Sapphire/Emerald Rapids
         | sucks so it's either Genoa, Bergamo, or Ampere One.
        
       | jeffrallen wrote:
       | Anyone who has been following Oxide knows the physics of why big
       | fans rock.
        
         | alberth wrote:
         | It's simple math of pr^2
         | 
         | 40mm fan can push ~5,000 sqmm of air
         | 
         | 80mm fan can push ~20,000 sqmm of air
         | 
         | Doubling the fan size results in 4x more airflow.
        
       | aftbit wrote:
       | It's very interesting to see this. I wonder what Amazon,
       | Facebook, Google, and Microsoft are going to do in the next few
       | generations. Will they be moving to 2U1N as well? Or do they have
       | liquid cooling in mind? Or maybe they'll be keeping lower power
       | budgets? Or building data centers that run cooler and support
       | higher power density (probably not great for the environment).
       | 
       | I knew fan power usage could be a substantial amount of a
       | server's power draw, but I didn't realize they would need 320W to
       | move enough air to cool 500W of dissapation.
        
         | nagisa wrote:
         | The major cloud vendors have their own data centers & do not
         | need to deal with space or power limits at a rack unit. They
         | have the entire data center worth of space and power supply to
         | work with.
        
           | jonatron wrote:
           | If the data centre is owned by digital reality, equinix,
           | telehouse, etc, is it really theirs?
           | https://aws.amazon.com/directconnect/locations/
        
             | arrakeenrevived wrote:
             | Direct Connect locations are not the locations where the
             | main region data centers are. AWS has a network link from
             | their DC to Equinix, etc to make it easier for customers
             | who are in Equinix's colo to make a direct connect to the
             | AWS network.
        
           | toast0 wrote:
           | All data centers have space and power/cooling limits. If
           | you're in someone else's DC, they are usually going to quote
           | you as per-rack power, but if you really want you can
           | probably convince them to let you use 2 racks worth of power
           | in one rack and hold a second rack empty. In owned and
           | operated DCs, it's kind of similar, if the DC was built a
           | while ago, as server power density has increased, a
           | significant portion of the floor space may be unusable for
           | racks.
           | 
           | 10-20 years ago, 1U and blades made a lot of sense because
           | space was a major constraint; now 2U or larger can make sense
           | because 1U has a lot of tradeoffs for size, and power is the
           | bigger constraint.
           | 
           | If space density is still needed, 2 nodes in 2u or similar
           | narrow but tall configurations provide a more square profile
           | that allows generating airflow with larger fans which is more
           | efficient than the 40mm fans needed in 1U systems.
        
         | folmar wrote:
         | OVH is doing liquid cooling to some extent since 2003.
         | https://blog.ovhcloud.com/water-cooling-from-innovation-to-d...
        
         | infogulch wrote:
         | Yes it's insane to waste 70% of the CPU power again on cooling
         | the CPU (read: cutting into power density).
         | 
         | Oxide's rack uses 80mm fans that run at less than half the
         | stock minimum speed to reduce this waste and run super quiet:
         | https://oxide.computer/blog/the-cloud-computer
        
         | mcronce wrote:
         | It's nuts how much energy server fans can use.
         | 
         | I recently picked up an R730xd. By default, a non-Dell-branded
         | PCI-E device will force the fans to ~16000RPM, but there's an
         | IPMI command to disable that and go back to regular,
         | temperature-driven, fan control. Total server power came down
         | by 70W after issuing that command.
        
       | jjtheblunt wrote:
       | They don't seem to mention why EPYC and not Ampere's new
       | architecture.
       | 
       | Is the power and performance / core count comparable, so it's
       | more momentum preserving to stay with x86_64?
        
         | robin_reala wrote:
         | The post is specifically talking about their x86 servers:
         | 
         |  _It 's nearly time for another refresh to our x86
         | infrastructure, with deployment planned for 2024._
        
         | eastdakota wrote:
         | We use both. This post was about the x86 version of our edge
         | servers.
        
       | nagisa wrote:
       | Would love one of those decommissioned 7713s to upgrade my own
       | server with!
        
       | yardie wrote:
       | I bought my last 1U server a few years ago. The space savings I
       | thought I was gaining was quickly diminished by the fact that we
       | weren't going to be able to fill a cabinet simply due to running
       | over our power envelope. It was going to require a bigger
       | circuit, an electrician to install it, a change in our cool
       | billing, etc.
       | 
       | So, these newer CPUs while being a lot more powerful also draw a
       | lot more power as well. In the end the squeezing 1000s of CFM
       | through a 1.75" aperture wasn't worth the headache.
       | 
       | I'm glad to see cloud providers are still using datacenter
       | standard 1U chassis. I assumed everyone had moved to Open Compute
       | rack dimensions 537x44mm.
       | 
       | The neighboring cabinet to ours has a 3x Walmart box fans. I
       | thought that was pretty innovative. Just use big silent, slow,
       | fans instead of these tiny, noisy airhorns.
        
       | NicoJuicy wrote:
       | Nice to see some of the hardware details behind cloudflare!
       | 
       | One that I found nice too, Cloudflare is able to quickly install
       | GPU's everywhere, because for over 6 years, they didn't utilize
       | 1/6th of the slots for that purpose.
       | 
       | https://www.fool.com/investing/2023/11/22/cloudflare-started...
       | 
       | Would have loved to see that information from their blog, instead
       | of their earning call :p
        
       | antisthenes wrote:
       | 1U is essentially obsolete due to the power/heating envelope of
       | modern server CPUs that push 32-64 cores easily.
       | 
       | I just don't see a rack full of 1U being anything but a giant
       | liability in terms of trying to cool it and the following
       | reliability issues.
        
       | Am4TIfIsER0ppos wrote:
       | What does "1N" mean?
        
         | wiml wrote:
         | It means there's one node in that chassis, eg here's a 2U4N
         | product: https://www.servethehome.com/dell-emc-
         | poweredge-c6525-review...
        
           | Dylan16807 wrote:
           | Seems to be right, but as far as I can tell almost nobody
           | uses this terminology and it's pretty rude of the article not
           | to explain.
           | 
           | Like, this article is one of the top results for 2U1N, plus
           | some dell pages and random jank.
        
         | tekacs wrote:
         | I mean this very positively, not condescendingly and I'm very
         | much not trying to preach or push, but I've found that this
         | sort of question is exactly the sort that I benefit a lot from
         | handing to GPT to answer for me:
         | 
         | https://chat.openai.com/share/37774e90-a271-4f6c-a5d3-3937e9...
         | 
         | I was curious too because I hadn't heard '1N' in a long time so
         | I did the above and you might like the answer it gave me too!
        
           | Am4TIfIsER0ppos wrote:
           | Based on your sibling's comment chatgpt has it wrong.
        
           | nemothekid wrote:
           | > _N: This refers to the redundancy of the power and cooling
           | systems. "N" stands for "Need" or "Normal." In data center
           | design, "N" is used to denote the capacity required to
           | support the IT load. "1N" implies a single path for power and
           | cooling, with no built-in redundancy. In other words, each
           | component necessary to support the IT equipment is present,
           | but there is no backup if one of these components fails._
           | 
           | I'm pretty sure this is wrong. N means "node", so a 2U4N is a
           | 2U chassis that houses 4 servers. I'm pretty ambivalent for
           | this reason for using chatgpt for research in things I cannot
           | easily verify. I will admit this is a scarily convincing
           | hallucination.
        
           | flumpcakes wrote:
           | I think this is completely wrong from ChatGPT...
        
       | solardev wrote:
       | Cloudflare's blog posts are always so comforting. It's like
       | listening to Grandpa tell war stories next to the fireplace.
        
       ___________________________________________________________________
       (page generated 2023-12-01 23:01 UTC)