[HN Gopher] Hetzner GPU Server
___________________________________________________________________
Hetzner GPU Server
Author : matteocontrini
Score : 220 points
Date : 2024-02-20 12:39 UTC (10 hours ago)
(HTM) web link (robot.hetzner.com)
(TXT) w3m dump (robot.hetzner.com)
| ecmascript wrote:
| Not really related to their GPU offering but Hetzner is such a
| great company. I have a dedicated server I rented where I host
| all of my side projects. For the same price as one shitty VPS I
| get 8 cores, 64GB ram and a disk of 240GB or something like that.
|
| Just incredible, response times are incredibly low and there is
| no cap on data. The few times I have contacted support they've
| been great. The only times I have experienced downtime is when I
| have stopped or rebooted the servers.
|
| Can't recommend enough, especially if you are inside the EU and
| care about data privacy. The fact that they're a european company
| is of great value to me.
|
| If you are new on server provisioning and want to learn how to
| setup a new server, I wrote a small guide for how to do it for
| node-projects (works for pretty much any web app if you replace
| the node.js stuff): https://deployjs.com/ (hosted on that server)
| gokhan wrote:
| Had a server with them for more than 10 years. Had they
| eliminated the setup fee, they would get more of my money for
| sure for small projects.
| ecmascript wrote:
| The setup fee is a one-time fee though, so if you have a
| server for many years that won't hurt so much. I pay around
| EUR30/month for the server I am renting but I know it will
| last a long time even if I get spikes on any on my many
| projects. For a 8 core, 16GB droplet/vps at DigitalOcean it
| will cost you ~$96/month. The performance is better at
| Hetzner for about 1/3 the cost.
| gokhan wrote:
| Yes. I'm talking about occasional needs for a small server
| for a month or two.
| ecmascript wrote:
| They have VPS cloud offerings similar to DigitalOcean as
| well, with no setup fees.
| gowthamgts12 wrote:
| Why not go with hertzner cloud? No setup fees and you'll
| be billed by the usage. Still cheaper than other cloud
| providers.
| throwaway11460 wrote:
| The VM performance tax is huge. Also, no GPU and
| limited/paid traffic (large limit though).
| e12e wrote:
| Use the server auction? No setup fee then.
| sshagent wrote:
| They often have a couple of servers with no setup.
| Citizen_Lame wrote:
| Ah yes, they are great if you messing about. Not so great if
| you have crucial business server with them.
| ecmascript wrote:
| why not?
| rapsey wrote:
| We've been running stuff on their platform for years with
| very little problems. Why not?
| withinboredom wrote:
| I haven't run into a single issue for years, what do you mean
| by that?
| elric wrote:
| Care to elaborate on that?
|
| I've been using their services for some 20 years at various
| scales, and the only complaints I've had are that their
| shared mailservers occasionally get blacklisted.
|
| That, and the UI of their KonsoleH could do with a bit of a
| refresh.
| happylion0801 wrote:
| Signed up from an Asian country and during signup they asked me
| to upload a passport copy page.
|
| I thought that was weird (I mean no other cloud provider asked
| for it but okay). I uploaded anyway because I thought it's in
| EU so they at least have better privacy laws so they may end up
| deleting it later from their systems but I immediately got
| rejected. Not sure why no reason specified. So now they have my
| passport copy page without me having an account.
|
| Oh well, seems like either they have an overly aggressive spam
| filter that gets false positives or they don't really like
| accepting signups from Asia
| lakomen wrote:
| Hetzner is pretty hostile towards foreigners.
|
| And don't you dare suggest in their forum that foreign tech
| is better than German, instaban.
|
| Fuck Hetzner
| EVa5I7bHFq9mnYK wrote:
| What German tech - AMD, Intel or Nvidia?
| callalex wrote:
| Please don't feed the trolls.
| lukan wrote:
| I can see a possible scenario, in where rather your general
| attitude might have been the problem.
| Shorel wrote:
| I'm from South America, got a server with them, no issues
| so far.
| deltaknight wrote:
| I had a similar experience very recently (albeit within
| Europe). Did you try their manual verification process? It
| was available to me after two failed automatic attempts, and
| it basically had me send an email to Hetzner support asking
| if they needed any more information. I got approved manually
| the very next morning.
|
| Maybe not the most customer-friendly experience, but I figure
| half an hour of signup weirdness is okay for a solid after-
| signup service in the long term, especially as that half an
| hour is likely to deter a decent chunk of fraud.
| lofties wrote:
| Had the same experience. Contacted support for manual
| verification, was online the next day with Hetzner. Machine
| has been running without interruption for a year now.
| EVa5I7bHFq9mnYK wrote:
| Same story from the US. I think they mostly worry that you
| are trying to avoid paying VAT.
| pmontra wrote:
| Which server do you have on Hetzner that I missed? Dedicated
| servers start at 37.30 Euro per month now there, the AX-41
| couch is pretty good and has no setup fee. On the other side
| I'm paying OVH about 12 Euro per month for two small VPSes and
| they are even too much for me, performance wise.
| e12e wrote:
| Have a look at the server auction? Tbh, I would limit my
| dedicated server rentals to server/ddr ram setups (which is a
| little more expensive).
|
| But if you just need some cpu (to transcode some video, crack
| some passwords, what-have-you, run a game/videochat server) -
| consumer cpu models give you unmetered gigabit or 30tb/month
| 10gbps. So can definitely be worth it vs a VPS.
| thatwasunusual wrote:
| Ditto.
|
| Been using them since 2012 or something, and have never had a
| problem (that wasn't my own fault).
| V__ wrote:
| Note: For now for existing Hetzner customers only.
| greatNespresso wrote:
| Is Nvidia RTX 4000 solid for training? Alternatively this card
| seems listed for around $1800 so at $180/month on Hetzner, it may
| be even cheaper to buy it, depending on the expected usage.
| Ayesh wrote:
| It's with ample RAM and the latest Intel CPU. Assuming the
| whole thing draws 100W, that's also including 72kWh of
| electricity.
| bluedino wrote:
| I'm not sure what the exact 'rule' is here, but you wouldn't
| use a contraction in this sentence.
|
| It is, with ample RAM and the latest CPU.
| sjsdaiuasgdia wrote:
| From https://www.yourdictionary.com/articles/contractions-
| correct... -
|
| Contractions can be used in any position in a sentence;
| however, homophone contractions such as "it's" and
| "they're" sound better when followed by another word or
| phrase. The reason is that the sounds of "its" and "it's"
| and "they're" and "their" are so similar that they can be
| confusing unless they are used with the context of an
| additional word.
| zimpenfish wrote:
| To be fair, the contracted version doesn't read nearly as
| nicely. Q. Is it good for training?
| A. It's with ample RAM and the latest Intel CPU.
|
| vs Q. Is it good for training?
| A. It is with ample RAM and the latest Intel CPU.
| bluedino wrote:
| I think they are agreeing with it not reading as nicely.
| But you still need the comma. Otherwise it sounds like:
|
| "It is with great pleasure..."
| 0xcde4c3db wrote:
| I think it's less to do with homophone confusion and more
| to do with usage patterns relating the contracted part to
| a phrase rather than a single word [1] [2]. By the logic
| of the homophone explanation, the following exchange
| should be much more acceptable than a sentence ending
| with "it's" or "they're", but to me it seems equally
| strange:
|
| "Are you waiting for someone?"
|
| "Yes, I'm."
|
| [1] https://www.youtube.com/watch?v=CkZyZFa5qO0
|
| [2] https://en.wikipedia.org/wiki/Clitic
| Aerroon wrote:
| Why does this GPU cost so much? It seems to be like an RTX 3060
| Ti/3070 with more VRAM and less power draw. Does that really
| justify the price tag?
| kookamamie wrote:
| Because it's "enterprise".
| HPsquared wrote:
| Like Apple, they use RAM for market segmentation.
| petercooper wrote:
| NVIDIA's licensing. Note that the 3060/3070 is a "GeForce"
| whereas a 4000 is a "Quadro". Then look at the GeForce driver
| EULA: https://www.nvidia.com/content/DriverDownloads/licence.
| php?l...
|
| > No Datacenter Deployment. The SOFTWARE is not licensed for
| datacenter deployment, except that blockchain processing in a
| datacenter is permitted.
|
| I could never figure out why they allowed blockchain
| processing, given how wasteful much of it was and how it
| impacted card supply for years, but there you go.
| pmontra wrote:
| Because they made a lot of money selling to miners at
| prices that maybe they couldn't ask to gamers?
| 8organicbits wrote:
| Wouldn't the GeForce license then say "no block chain"
| and a different NVIDIA brand (BlockForce?) would allow
| it, letting NVDA charge more for large scale mining?
| Otherwise it looks like gamers and miners are paying the
| same prices.
| bbatsell wrote:
| They do. But since most miners would just ignore the
| license, they implemented LHR (low hashrate) beginning on
| GeForce 3000-series that detects the types of ops used
| for mining and artificially slows the GPU to about 25% of
| its actual speed.
| pmontra wrote:
| They did that but with mixed success. One random link
| from a Google search: https://old.reddit.com/r/EtherMinin
| g/comments/t4mwxj/workaro...
| latchkey wrote:
| Their source code got leaked and it was reverse
| engineered to defeat it. That said, it was all kind of
| late in the game, GPU mining died not too long
| afterwards.
| sodality2 wrote:
| I bet that now the AI demand spike will be the new crypto
| demand spike that NVDA latches onto.
| Aerroon wrote:
| This seems insane to me. It makes me even more confused
| that AMD/Intel aren't trying their hardest to go after AI
| compute. Nvidia is basically leaving the door open.
|
| But I suppose that the idea could be to protect their
| gaming market in a way. If datacenters were buying up
| gaming GPUs then that could cause shortages for regular
| customers.
| latchkey wrote:
| AMD is trying their hardest.
| stefan_ wrote:
| Datacenter customers don't have options, gamers do.
| rmbyrro wrote:
| You have to discount the price of a similar server without the
| GPU, tough.
|
| But even so, it's still quite expensive. I have the impression
| Nvidia charges more from data center customers. So, for cards
| available to retail consumers, it's not a fair cost comparison.
|
| With only 20GB, RTX 4000 has limited use... I was expecting
| they'd be offering something with at least 60GB at this point.
| kookamamie wrote:
| No, not a great GPU for training. Consider something like
| A5000, A6000 or their Ada variants - if you want to go full
| "data-center", A40, L40, L40S are solid picks for small-scale
| training.
| brucethemoose2 wrote:
| Are you going to be training constantly?
|
| TBH renting a big GPU (A100 or better) and getting the run over
| with quick is usually best. VRAM is everything.
| christkv wrote:
| How is the gpu for inference or tuning ?
| SushiHippie wrote:
| FWIW, on the server auction, you could get GPU servers since a
| while:
|
| https://www.hetzner.com/sb/#additional=GPU
|
| Looks like currently there are only GTX 1080s on the server
| auction, I don't know if they normally have other GPUs available
| as well.
| alberth wrote:
| Nvidia RTX 4000 SFF Ada
|
| From the OP link, the GPU-Server GEX44 server line includes RTX
| 4000.
|
| And the server only cost EUR184/month.
| SushiHippie wrote:
| I meant if they have other GPUs normally available in the
| server auction. As far as I understand the servers in the
| server auction are mostly custom built ones for customers and
| once they stop renting them they land on the server auction
| for others to rent.
|
| The one with the RTX 4000 seems to be a standardized line of
| products.
| dizhn wrote:
| They briefly had GPU servers before. I believe they stopped
| when crypto messed up the gpu prices and supply. That's why
| you're seeing them in the auction now. But they usually go
| quick. This might change now since they have a brand new
| offering.
|
| The old GPUs would have very litle memory by the way. Not
| ideal for AI.
| albert180 wrote:
| They stopped because they had a high percentage of abuse
| with Morons mining crypto
| aborsy wrote:
| The price of a brand new Dell precision PC with a core i9 13900
| CPU (24 cores), 64 GB RAM and 2X2TB NVMe is probably around
| 2.5k$. It costs tens of Euros per month in electricity to run
| this PC. The hardware is good to go for some 3 years, before
| running into various issues (newer CPUs becoming much more power
| efficient, firmware updates getting less and less frequent etc).
| Consider also the cost of components failure, time wasted in
| dealing with hardware, etc.
|
| On hetzner it all costs less than 100$ per month. Doesn't this
| imply that it's better to rent than to buy?
| throwaway11460 wrote:
| Yes, the volume discounts and economies of scale they can get
| (while you can't) are insane.
| singhrac wrote:
| One of the simplest is that data center energy pricing is
| just simply different from residential energy, because they
| often get wholesale (plus small markup) rather than
| residential rates.
| formerly_proven wrote:
| The majority of residential rates (about 40 ct/kWh) are
| taxes and levies, not the actual price of the electricity.
| That's why industrial rates can be less than half of
| residential rates.
| throwaway11460 wrote:
| Can you be more specific? What kind of taxes? VAT is paid
| by the end consumer so you're not getting out of that
| one. Are there special consumption taxes on electricity
| in Germany?
|
| In my experience the largest difference is distribution
| cost. The mandated distribution monopoly charges a lot
| (regulated price) to small customers.
| Propelloni wrote:
| You need to compare prices again. Residential electricity
| costs are significantly below your quoted rate for quite
| some time now. I've just checked and the lowest rate I
| saw was 21 ct/kWh. Most vendors are somewhere in the 25
| to 30 ct/kWh bracket.
| lelandbatey wrote:
| Average USA energy prices are (according the Bureau of
| Labor Statistics) ~$0.18/kWh at the moment, with some
| folks at $0.40+ (San Francisco, San Diego, Hawaii) while
| others are as low as $0.13/kWh (Seattle, Saint Lewis)
|
| https://www.bls.gov/regions/midwest/data/averageenergypri
| ces...
| formerly_proven wrote:
| > The average electricity price for households fell by
| almost 8 percent at the beginning of 2024 compared to the
| annual average for 2023 and now amounts to an average of
| 42.22 ct/kWh (2023: 45.73 ct/kWh; base price included pro
| rata for a consumption of 3,500 kWh/a ).
|
| https://www.bdew.de/service/daten-und-grafiken/bdew-
| strompre...
| noAnswer wrote:
| I'm at a small local energy provider. I have their most
| expensive 100% renewable tariff. Even after the invasion
| the price only went up to 28,36 EURct/kWh. Yet, my
| parents pay above 45 to RWE. (That's what cheap nuclear
| power stations get you I guess.) My father finally
| changed providers but thanks to the super fair contract
| has to wait now over a year for it to happen. And thanks
| to the "Strompreisbremse" I have to subsidise those ass
| companies.
|
| sorry for being of topic
| callalex wrote:
| There's no point in having this discussion, it is
| extremely specific to different locations.
| EVa5I7bHFq9mnYK wrote:
| Large part of their operation is in Helsinki, where
| electricity prices are a fraction of Germany prices.
| singhrac wrote:
| I think "taxes and levies" is a bit true, but also you
| pay for distribution. If your power line goes down in
| your neighborhood, you basically just grumble but it gets
| fixed. In wholesale connections you pay for the
| distribution power line (+ maintenance) basically up
| front, but it doesn't get put in the final wholesale
| bill.
|
| This isn't to say your local utility isn't wasteful with
| its ratepayer money; it totally can be. There isn't
| enough pressure to lower rates. It's just worth saying
| that these specific "taxes" do have an intended
| destination, not just general govt.
| xhkkffbf wrote:
| In the winter time, I get free heat from my server.
|
| In the summer, I have to pay to run an AC to get it out of
| the envelope.
| teaearlgraycold wrote:
| Gotta run some ducting for that server!
| schroeding wrote:
| If you pay European (German) energy prices, this very well may
| be the case, in my experience.
|
| For some Hetzner servers (especially from their "Server
| Auctions"), for domestic customers the cost of electricity
| alone, disregarding the hardware cost, would sometimes be
| higher than the rent Hetzner wants.
| elric wrote:
| They build their own servers using some custom components.
| Economies of scale probably don't really factor into the
| pricing of their GPU stuff (unless I'm seriously
| underestimating their GPU customer base), but they do factor
| into the peripheral stuff like PSUs, fans, hard drives, etc.
| jsheard wrote:
| There's some videos out there showing off their custom
| motherboards, it's an odd mixture of standard-ish consumer
| ATX but stripped down to the absolute bare essentials and
| with some components rearranged for better airflow in racks
| chassis they also build themselves:
| https://youtube.com/watch?v=V2P8mjWRqpk
|
| Another video showing the custom racks and other
| infrastructure: https://www.youtube.com/watch?v=5eo8nz_niiM
| wongarsu wrote:
| Hetzner is not only able to get better volume discounts, they
| also spend a lot of engineering time bringing down costs, and
| have decades of experience doing so.
|
| They offer some Dell Servers for those that really want them,
| but most of their servers have a custom mix of consumer
| hardware, server hardware and in-house hardware (for example
| they use their own racking system), optimized to minimize
| lifetime cost in a datacenter. For example most servers use
| datacenter SSDs, but consumer CPUs.
| dist-epoch wrote:
| Hetzner is buying ASUS consumer AM5 motherboards, but with
| the whole CPU + memory block rotated so that the memory
| sticks are horizontal instead of vertical for better airflow:
|
| https://www.youtube.com/watch?v=V2P8mjWRqpk
| justinclift wrote:
| They also buy (at least) ASRock Rack motherboards as well.
|
| Saying that from looking at the dmidecode output from some
| of our dedicated server currently with them:
| Handle 0x0002, DMI type 2, 15 bytes Base Board
| Information Manufacturer: ASRockRack
| Product Name: B565D4-V1L
| cobertos wrote:
| Do you have more details on their own racking system? I
| couldn't find anything on this.
| wongarsu wrote:
| The best resource I know is a tour of one of their data
| centers. Around minute 6 you get some nice views on their
| current-gen racks, and later in the video you see the back
| side, followed by the assembly of the servers.
|
| https://www.youtube.com/watch?v=5eo8nz_niiM
| YetAnotherNick wrote:
| It's definitely weird that cloud costs which is supposed to be
| commoditized is differing by 2 orders of magnitude. Hetzner/OVH
| seemed to have solved the hard part of making the service
| cheaper but just can't solve the easier task of making the
| platform usable. Why do I need to upload documents for
| verification. Why can't I try server for 2 hours right after
| signing up for the service.
| mike_hearn wrote:
| Those are part of making it cheaper. ID verification is an
| anti-fraud measure and ensures that bans stick, which in turn
| means they don't have to invest so heavily in heuristic anti-
| fraud and "spam filtering" type work. A delay on provisioning
| means they are provisioning hardware JIT in some cases,
| meaning they need less idle float capacity, which in turn
| drives down costs.
| Scotrix wrote:
| Have been in deep contact with data centre and rental server
| offerings in the past, there is a crazy large amount of fraud
| going on, e.g. 2 hours for free means 2 hours free resource
| to DDOS.
| YetAnotherNick wrote:
| I don't want it free. I want to put $2 on the account. Try
| the server and decide after using the server.
| Scotrix wrote:
| same problem, credit card fraud/chargebacks happening
| anyway, that's why you have an intense verification
| process to reduce the fraud going on.
| albert180 wrote:
| And they want a customer who will rent the dedicated box
| for a long time and not wipe it after every 2$/2h
| Trialcustomer
| throwaway81523 wrote:
| You can do that with Hetzner virtual (cloud) servers,
| just not with dedicated servers.
| delfinom wrote:
| You can thank crypto for ruining the internet for us.
| EVa5I7bHFq9mnYK wrote:
| Sure you can. I rented a server and after two weeks decided I
| don't like it, and got refunded all the money, including the
| setup fee.
| brudgers wrote:
| _Doesn't this imply that it's better to rent than to buy?_
|
| TANSTAAFL
|
| 1. Physical hardware is a useful abstraction.
|
| 2. At the end of a three year cycle, you can sell something you
| own. (This has associated costs of course).
|
| 3. Rents can go up. Terms and Conditions can change. Credit
| cards can expire or be cancelled. In other words, renting
| introduces a significant dependency.
|
| 4. You are your own most important customer. Statistically, you
| are not The Clouds's most important customer. When The Clouds
| has a fault, it gets resolved based on its business model.
|
| Engineering decisions are specific to a specific problem and
| all of them come with tradeoffs.
| Dalewyn wrote:
| >2. At the end of a three year cycle, you can sell something
| you own. (This has associated costs of course).
|
| This is huge and often overlooked or undervalued. Renting
| means you have nothing when the contract ends, buying means
| you have something for your money spent.
| cduzz wrote:
| A 3 year depreciation cycle may be reasonable for some
| spaces but not others.
|
| I'm not going to lose sleep over not getting firmware
| updates; when they've got all the bugs on my stuff ironed
| out I don't want more firmware updates.
|
| I haven't done the spreadsheet work to identify the cross-
| over point for where old systems become uneconomical but
| when I've done it in the past for my heavily clustered
| workloads it's typically better to buy an off-lease server
| that's 3 years old and run it for another 8 years, than it
| is to forklift all my infrastructure every 3 years chasing
| the newest generation thing.
|
| Certainly the cloud's got some advantages, but running
| equipment until it's really old is also pretty good too.
| I'd bet that AWS doesn't throw older systems out _just_
| because they 're old.
| CapeTheory wrote:
| Do you happen to have a preferred source for these 3-year
| old off-lease servers?
| Slartie wrote:
| > A 3 year depreciation cycle may be reasonable for some
| spaces but not others.
|
| Exactly.
|
| I am running a server at Hetzner which has a CPU that was
| discontinued about 7 years ago. I don't know the exact
| age of the machine because it was already used when I got
| it 4 years ago, but based on the CPU availability it's at
| least 7 or 8 years old, potentially even older.
|
| Nothing on that machine has failed in the last 4 years
| except for HDDs (the spinning platter type), which are
| immediately swapped when broken, RAID rebuilds,
| everything's fine.
|
| 3 years is no time for hardware nowadays. It can live
| much longer, especially if storage is solid-state. And
| the performance improvements often aren't substantial
| enough to warrant a swap within anything shorter than 5
| or 6 years.
| k8sToGo wrote:
| You forgot that Hetzner also has spare parts.
| brudgers wrote:
| And Dells have warranties.
|
| And both offer service agreements.
|
| And Hetzner is a straight line expense while a Dell might
| be a depreciating asset.
|
| And either both or neither may offer discounts off retail.
|
| And so on.
|
| Because that's the nature of actual decisions versus online
| arguments.
| chrisweekly wrote:
| > TANSTAAFL
|
| "There ain't no such thing as a free lunch"
|
| (I had to look it up, figure I'll save someone else yhe
| trouble)
| nortonham wrote:
| thanks, I had no idea what that meant either
| echelon wrote:
| It used to be moderately common acronym back in the 90's
| and 00's. It felt like a blast from the past seeing it crop
| up again.
| tiffanyh wrote:
| Yes.
|
| This is what you expect prices to be when a vendor is buying in
| bulk.
|
| Cloud vendors _should_ be like "Costco" (buy servers in bulk
| and can pass along those savings & this is what Hetzner does)
|
| The trick AWS (and other cloud vendors have done), is charge
| "Uber Eats" pricing for Costco items.
| fragmede wrote:
| That's a really clever analogy, but is it really a trick?
| UberEats delivers the food to my door but if I go to Costco I
| have to make my dinner myself. Sometimes life happens and
| UberEats isn't just a convenience thing, it's the only way
| I'll manage to feed me and my crew. I could run a pubsub
| queuing system and a database myself, but if I don't have to
| do that, it frees me up to focus on the tasks I'm really
| trying to accomplish instead.
| whizzter wrote:
| It seems a bit like car leasing, providers gets a bit of
| discounts for their volume purchases and customers pays a
| premium (that is still affordable) compared to just taking the
| entire deprecation cost themselves. The provider then still has
| something worth a fair bit of cash that can still generate
| income (through resale).
|
| So if you look at a 2 year horizon and want to keep cutting
| edge (or prefer operating expenditures to capital expenditures)
| then it's cheaper whereas looking at a 5 year horizon the
| capital expenditure will pay off.
|
| Hetzner is very bare-bones compared to regular clouds (a friend
| who was after performance but wasn't prepared for this ran into
| some issues when a disk died), so you need some procedures for
| backup/replication in place but if your procedures are in place
| then you can save big compared to the regular clouds.
| chpatrick wrote:
| There's definitely a tradeoff when it comes to maintenance
| effort. You need to make sure your RAID and monitoring works
| and if a disk fails you need to message support to replace
| it. In general I've had a great experience with them.
| treffer wrote:
| Last time I checked the GCP prices vs. hetzner servers there
| was a factor of 10 in monthly cost. So I cant say this is a
| general rule of thumb. It is true though that hetzner can
| provide superb performance per $. Potentially below what you
| can do (unless you buy racks of hardware).
|
| Also keep in mind that hetzner is mostly a beowulf cluster. See
| e.g. this press picture from their newsroom page:
| https://cdn.hetzner.com/assets/Uploads/IMG-0546-91.jpg
|
| They do have dell servers, too. But don't expect to run on one
| of these nodes unless the machine type has Dell / PowerEdge in
| their name. This means the management capabilities are minimal
| (but usually enough). A fully licensed management card can do
| way more.
|
| Their renting pricing is ridiculously cheap though. Hardware
| failure is more common than on cloud providers (see cluster
| hardware), support has been helpful in the past. Overall IMHO a
| good choice if you are price sensitive. But even then consider
| development vs. deployment vs. running (server) costs. Servers
| might make up less of the total cost of a service than you
| expect.
| rthnbgrredf wrote:
| To be fair, companies that merely launch VMs on GCP
| constitute a small fraction. GCP truly excels when you
| leverage its object storage, BigQuery, managed postgres
| database (which starts at 7$/month), and serverless solutions
| for cloud-native applications. Our company operates around
| 500 services, with billing per second, and a significant
| number of them are scaled down to zero when not in use. If
| you need a GPU for a batch processing task involving 10,000
| images, you can simply activate a VM equipped with a high-
| performance GPU for an hour, pay for that duration, and then
| shut it down. At Hetzner, you're required to pay for a whole
| month upfront, regardless of whether you need the GPU for
| just an hour each day.
|
| Therefore, I'd argue that if you require continuous, raw
| computing power, Hetzner is indeed cost-effective. However,
| the cost-effectiveness elsewhere really hinges on your usage
| patterns.
| petercooper wrote:
| I've just ordered one to ditz around with. We're early in the
| process of deploying our own internal server for Mixtral work,
| but it'll be interesting to see how this performs (in raw terms,
| almost certainly worse, since we can roll out a 4090 without
| getting into trouble - but Hetzner has a better connection and
| handles the maintenance, so..) Order has been accepted but not
| deployed yet. I'll update when I have some initial inference
| numbers.
| petercooper wrote:
| I got it after a few hours. You need to install drivers/CUDA
| yourself, but all very straightforward. Unfortunately due to
| having 20GB of VRAM, I'm limited to
| mixtral:8x7b-instruct-v0.1-q2_K but it runs fine, generating at
| about 40 tokens/s (65 tok/s for eval). As per official specs,
| it's running maxed out at 70W (being an SFF card).
|
| (I've now tried running the Q4 mixtral which is 26GB. 18GB is
| on GPU, 8GB through CPU. Gets about 11 tok/s.)
| gregoriol wrote:
| It's a good price at ~0.30$/hour comparing to Paperspace's RTX
| 4000 with 30GB RAM at ~0.56$/hour.
|
| Are there many other providers for such environments?
| pacohernandezg wrote:
| vast.ai, runpod.io, tensordock.com, ... I have just ordered a
| GEX44 from Hetzner. :-)
| binarymax wrote:
| The GPU product page: https://www.nvidia.com/en-us/design-
| visualization/rtx-4000-s...
|
| 20GB RAM, can fit 13B param models, and maybe some quantized
| larger models.
| ramesh31 wrote:
| Does every European tech company have some kind of committe that
| ensures their customer facing website looks like a decrepit ASP
| page from 2004?
|
| I mean what explains this? It's a billion dollar corporation and
| I am having serious reservations about whether I would even enter
| my credit card info on this site.
| lakomen wrote:
| It's true. And it breaks on mobile when you use the menu.
| delfinom wrote:
| Hetzner's site looks fine to me. Highly readable and to the
| point unlike modern web shit where you have to scroll through 5
| miles of marketing animatinos.
| rsynnott wrote:
| I mean, see also AWS; for a very long time their console looked
| like something from the dark ages. I think they have largely
| spruced it up now, though I'm not sure it helped _usability_
| too much, and may have actually made it worse.
| system2 wrote:
| I see the same thing on HN posts. 1999 vibes everywhere.
| hipadev23 wrote:
| Just make sure any workloads you place on Hetzner are treated as
| spot instances and outputs are immediately sent off network. The
| reliability of their systems and customer support is very low.
| Hetzner is a 5-star or 1-star company. Are you willing to risk
| your business on the flip of a coin? My own experience [2] was
| definitely a 1-star.
|
| [1] https://www.trustpilot.com/review/hetzner.com
|
| [2] https://news.ycombinator.com/item?id=39075608
| tomschwiha wrote:
| I've been with Hetzner for around 10 years and didn't had a
| single larger incident so far. Support also answered promptly -
| didn't had the need to request a lot support so far. Running
| 2-3 servers and a few cloud instances.
| gizmo wrote:
| Reliability and customer support are unbelievably good. 99.995%
| uptime for the past 10 years. Hetzner will happily customize
| your servers however you like, and at their low prices you can
| have hot standbys for everything and still save money.
| whizzter wrote:
| My friend went onto Hetzner for the performance/price,
| unfortunately the machines drive died within 2 months iirc.
| Seems like if you're buying on auction you'll get a wiped
| drive and if you're unlucky it's been written to a lot
| already, don't remember the specifics but he really wasn't a
| happy customer afterwards.
|
| I think the last sentence you have there is pertient though,
| maybe don't go for Hetzner unless you can also afford to have
| a hot-spare (but for the performance/price you probably can
| compared to any public cloud). In my friends case he was
| boot-strapping but needed the CPU so he was really penny
| pinching.
| dr_faustus wrote:
| Trustpilot is a scam which encourages bad reviews to force
| companies to buy their paid plan to "manage" those bad reviews
| and to feed it good reviews.
| hipadev23 wrote:
| Hetzner is also a fraudulent operation with zero support in
| my personal experience. But as you can see in this thread
| alone, any negative feedback is magically silenced. It sucks.
| Maybe we go back to webhostingtalk?
| dmaa wrote:
| It looks like a good value for money, but immediately after
| registering, I get this reply
|
| "After reviewing your updated customer information, we have
| decided to deactivate your account because of some concerns we
| have regarding this information. Therefore, we have cancelled all
| your existing products and orders with us."
|
| Someone has a similar experience? I've tried it twice, filled
| everything truthfuly, valid credit cards etc. and no still the
| same.
| stanislavb wrote:
| I haven't faced this issue myself, however, lots of people from
| India and some other countries have this experience.
|
| I'd assume it's something to do with collecting recurring CC
| payments from Indian cards. That's inherently difficult.
| dmaa wrote:
| That would also be my guess, but I live in western Europe.
| breakingcups wrote:
| There's a manual verification process if you contact support.
| Obnoxious that it's needed, but works for many people.
| belk wrote:
| I got this when I first registered, contacted support and it
| was lifted in a day or two.
| nubinetwork wrote:
| After the exchange rate, that's $270 CAD a month... after a few
| months, you might as well just buy a raptorlake and run it
| locally.
| zymhan wrote:
| Welcome to Cloud Computing.
| throwaway81523 wrote:
| It's a GPU system, not just raptor lake.
| pinetroey wrote:
| A few months ago I tried to register an account, but I was
| denied.
|
| I don't know why, never got a response...
|
| I went with ovh.
| ThinkBeat wrote:
| I need some help:
|
| I have no idea where the NVIDIA RTX 4000 SFF belongs in the
| hierarchy of speed and suitability for "AI" computing.
|
| How does this compare to a NVIDIA A800 40G? (if that is still
| considered good, I can't keep up)
|
| To me the RTX part of the name sounds more on the consumer side
| but I am probably wrong about that as well.
| renewiltord wrote:
| They had a strange setup process. I had to scan my passport and
| stuff. However, I did have a VPS with them that had uptime
| greater than a decade and that I killed only because the
| underlying hardware was going away. That was some 7 years ago or
| so.
| kristianp wrote:
| It's running an RTX 4000 SFF Ada, 20GB GDDR6. Interesting that
| they've gone with a single-slot, low power card. Doesn't pull
| enough power to mess with their cooling/power design, I guess.
| Not a lot of graphics RAM to do much with LLMs.
|
| That card is about $1500: https://www.amazon.com/dp/B0C2KMXQYG
| yread wrote:
| It's a pity you can't commit to 3 year rental for a discount
___________________________________________________________________
(page generated 2024-02-20 23:01 UTC)