[HN Gopher] Alternative clouds are booming as companies seek che...
___________________________________________________________________
Alternative clouds are booming as companies seek cheaper access to
GPUs
Author : belter
Score : 209 points
Date : 2024-05-06 12:05 UTC (10 hours ago)
(HTM) web link (techcrunch.com)
(TXT) w3m dump (techcrunch.com)
| freeqaz wrote:
| How are these companies even getting these GPUs? I would imagine
| NVIDIA would give them all to Microsoft and Google if they are in
| short supply.
| m3kw9 wrote:
| Nvidia needs these alt cloud so when big co builds their own
| GPUs they have them to suck up supply. If they are all dead,
| Nvidia may have no back path
| rvba wrote:
| Why couldnt NVIDIA auction some to get best price?
|
| I mean locking customers long term is probably better, but a
| certain % of product still vould be sold for a spot price? This
| made me wonder what is the optimum split on that - obviously
| depends on your market placement, but isnt nvidia basically a
| monopoly now?
| zer00eyz wrote:
| > Why couldnt NVIDIA auction some to get best price?
|
| Have you seen their pricing? They are robbing everyone and
| selling out quickly. I don't think "auctions" are going to
| add much to thee bottom line.
| autoexecbat wrote:
| At some point, nvidia does have to worry about amd getting
| their act together once they've annoyed all the current
| customers
| nothercastle wrote:
| Amd will never figure out drivers and software stuff.
| That's always been their weakness
| Palmik wrote:
| MI250 and MI300 are competitive for inference and
| training. The problem is that there's no cloud with
| simple registration that offers these for reasonable on
| demand price. This means that basically no one in the
| open source AI community can use them, so most
| development and tooling is being tested and built for
| NVIDIA.
| latchkey wrote:
| I am working on solving that issue with MI300x.
| mym1990 wrote:
| I think one of Jensen Huang's big interests is to enable a
| diverse number of markets that can utilize the technology in a
| short feedback loop. I don't believe that can be achieved if
| all of the hardware goes to Microsoft/Google.
| SllX wrote:
| It's better to have a diversified customer-base rather than
| being completely beholden to a small number of big customers
| that might decide they don't need you down the line.
| yardie wrote:
| Nvidia is a shovel maker. If you only have 1-2 shovel buyers,
| buyers who've started on making their own shovels in-house,
| then you run the risk of being trapped in a monopsony. You lose
| 1 shovel buying client and then you might be insolvent before
| too long.
| jedberg wrote:
| I head up Product at Lambda. We are an NVIDIA preferred partner
| (and keep winning the preferred partner award). I don't know
| what the allocation numbers are for other companies, but we get
| a lot.
|
| It's in NVIDIAs best interest to spread the love around and
| make sure all the GPUs don't go to the hyperscalers.
| leetharris wrote:
| I head engineering at Rev and every time we talk to Nvidia
| about GPUs they tell us to go to you guys. Every time. They
| really like Lambda!
| jedberg wrote:
| Very happy to hear this!
| tpurves wrote:
| I anyone buying AMD MI300? I am curious if anyone is finding
| AMD, as an alternate GPU vendor, available or useful yet?
| latchkey wrote:
| My company, Hot Aisle, is.
| Palmik wrote:
| Are there any plans to support your on demand cloud in the
| future? In the past few months there are basically no GPUs
| available.
|
| My understanding is that your reserved cloud starts at 64
| GPUs (8 pods) and with quite a long reservation time frame.
| jedberg wrote:
| We have new product coming in just a couple weeks to get
| clusters as small as 16 GPUs per cluster and only a two
| week minimum reservation, with very short lead times (the
| first people to reserve will have almost no lead time!).
|
| Also we expect to have a bunch of capacity come online of
| the on-demand cloud this year. We're getting the GPUs as
| quickly as we can and racking them as quickly as possible,
| but we have to wait just like everyone else for the GPUs to
| roll off the fab. :)
| hinkley wrote:
| I was at a Christmas party a couple jobs ago and they did a
| presentation that was supposed to be all good news and
| instead basically ruined the party for me.
|
| The bit that freaked me out was 2 customers were 60% of
| revenue.
|
| Those two customers can change your policies in ways you
| don't want to, because you can't afford to piss them off. And
| if something happens to one of them you're fucked.
| Workaccount2 wrote:
| Google doesn't use nVidia chips, at least for AI. They have
| their own in house solution.
| FrankPetrilli wrote:
| https://cloud.google.com/compute/docs/gpus#nvidia_gpus_for_c.
| ..
| mym1990 wrote:
| Isn't the mid/late game of all of these alternative clouds going
| to be the same as the big cloud players...get customer lock in
| and then prices slowly creep up until its kind of a wash?
| lelanthran wrote:
| > Isn't the mid/late game of all of these alternative clouds
| going to be the same as the big cloud players...get customer
| lock in and then prices slowly creep up until its kind of a
| wash?
|
| That might be the plan, but I doubt it's going to work for GPUs
| like it did for AWS.
|
| There's no moat here, other than the hardware. There's no
| value-add that the provider can add for free and lock you in.
| swores wrote:
| > _There 's no value-add that the provider can add for free
| and lock you in._
|
| Couldn't you have said that about cloud services like AWS
| before they existed, that they can rent servers to people but
| there won't be any lock in?
|
| And equally, just because that's the case with GPU providers
| right now, is there a reason one or more of them wouldn't be
| able to develop software which runs on the GPUs to do some of
| the things people are currently renting GPUs for, or middle-
| ground software which make it easier to do other stuff on
| them than just renting plain GPU access, and turning it into
| optional services (with lock in) on top of the raw hardware
| rental?
|
| As an example idea: it's good that NVIDIA decided to let as
| many people use CUDA as possible (and I suspect if they
| hadn't then they wouldn't have seen nearly as much success),
| but if they or anyone else releases an equivalent to "CUDA
| v2" tomorrow, but instead of allowing anyone to download it
| instead put it behind a billing page with AWS-style pricing
| that covers both software and GPU, would it not succeed if
| the software did make thinks easier for people just like AWS
| does (in some ways)?
|
| edit: I just realised I ignored the "for free" bit of your
| comment - but it wasn't free for Amazon or Google to build
| their cloud software either.
| Gormo wrote:
| That doesn't seem to be the case for long-tail VPS providers.
|
| It doesn't seem to really be the case for the big cloud
| providers, either, as most of what they offer is fairly
| commoditized -- I've been using GCP for years to host
| Kubernetes clusters, Postgres databases, Redis instances,
| standard Linux VMs, etc., and while there'd be a cost
| associated with migrating out of GCP, I don't feel locked into
| it in any way, as the tools I'm using there are the same tools
| I'd be using regardless of where they were hosted.
|
| The market for IaaS and for SaaS are very different from each
| other, and the walled-garden approach doesn't seem to have
| significant traction in the IaaS world, thankfully.
| tw04 wrote:
| Not every vendor needs Amazon's eye watering margins to justify
| their existence. Plenty of people more than happy to make their
| 12% and call it a day. The difficulty and cost of owning a data
| center has been grossly exaggerated. It's just that bean
| counters would rather lease because it's easier to massage the
| numbers than dealing with a depreciating asset.
| more_corn wrote:
| Amazon has also spent decades and tons of money building out
| the ecosystem of AWS: Ebs, alb, rds, etc. You don't need any
| of that stuff for training. You just need gpus. If you don't
| need it don't pay for it.
| 1123581321 wrote:
| Not without some distinctive advantage to the provider's
| infrastructure beyond raw GPU access. Plus, as more options
| exist, more SDKs will facilitate switching or distributing
| workloads to multiple providers as de facto connection
| standards emerge (like what's happened to S3-compatible
| storage, as a simple example.)
| TheGRS wrote:
| I would think the late game would be getting bought out by
| AWS/G Cloud/Azure.
| 015a wrote:
| We're absolutely seeing the same thing even in non-GPU
| traditional servers. Here is an estimated monthly cost breakdown
| for a new GCP instance type, the n4-standard-2, in us-central1:
| 2 vCPU + 8 GB memory: $69.18 10 GB Hyperdisk Balanced:
| $0.80 3060 provisioned IOPS: $15.30 155 MB/s
| provisioned throughput: $6.20 Total: $91.48
|
| Like, you can tear apart that $69/mo for 2vCPU + 8gb of memory,
| no problem. That's utterly insane. Its Emerald Rapids, so you're
| paying a premium for new chips, whatever. You can also tear apart
| the network egress, obviously.
|
| But just look at the SSD pricing. That's a ten gigabyte SSD
| provisioned for 155MB/s, for $22/month. You can go just outright
| buy a 256gb NVME at significantly higher bandwidth on Amazon for
| like $25, flat. The n4 tier instances removed the ability to use
| their cheaper general-purpose SSDs; you have to use hyperdisks.
|
| I'd be surprised if we don't see the big cloud providers start
| struggling over the next ten years. I think they engineered
| planetary-scale systems that are just way too expensive and
| complex to justify the cost they're charging; ZIRP phenomena.
| carbocation wrote:
| Also on GCP, the vCPUs are usually hyperthreads (except for
| t2d- instance types, and perhaps a few others). So that machine
| you've described has 1 CPU core.
| williamstein wrote:
| Reference: https://cloud.google.com/compute/docs/machine-
| resource#recom...
|
| The t2d, t2a and h3 instance types have vCPU = core, and all
| other instance types have vCPU = thread.
| ikiris wrote:
| if you don't need hyperdisk, why are you on that type of
| server?
| 015a wrote:
| Its the only instance class that's on Emerald Rapids. So if
| you want the best that Intel has to offer, you need to adopt
| hyperdisks.
|
| But, to be clear: We're not.
| acchow wrote:
| Curious what the use case is for targeting a specific CPU.
| These are virtual CPUs anyways, so what benefit does using
| the latest intel chip offer?
| evilduck wrote:
| Sometimes the instruction sets change (like the
| relatively recently added AVX10 extensions) and you have
| a workload that specifically needs those? I'm just
| guessing though.
| williamstein wrote:
| On Google Cloud hyperdisks can be used on h3, c3, c3d, m3, n4
| instance types and are required for n4. I.e., you are not
| allowed to use the n4 instance type without using a
| hyperdisk.
| candiddevmike wrote:
| IMO, the big thing holding back customers (especially smaller
| ones) from going on-premise these days is networking. Getting a
| fat pipe similar to what you get with an instance on a cloud
| provider can be prohibitively expensive (internet service,
| gear, staff, etc), especially when you want it to be highly
| available.
| tempest_ wrote:
| It is really the second one. The minute you want a second
| site or even HA at a single site the complexity and costs
| start to explode.
| cpill wrote:
| k3s is very simple to setup and and nodes to. if the
| machines are on the same LAN or the internet then it's not
| such a complex job albeit you need to know the basics of
| kubernetes.
| fwip wrote:
| Kubernetes is only the software layer, a lot of the cost
| is in the hardware/infrastructure, and in the salary for
| those experts.
| tempest_ wrote:
| Yeah, k3s doesn't buy 2 routers, 2 switches, 2 PDUs, 2
| firewalls, 2 proxies to sit in front of k3s, 2 internet
| connections (if those are offered) etc etc the list goes
| on. Not to mention that HA things like to come in 3s.
|
| Then if you are going to have remember that cloud
| networking is pretty beefy and if you want k3s to do
| distributed storage you will need some pretty beefy
| network hardware.
|
| There are a lot of things hidden in the cloud costs that
| people forget about.
|
| The one thing running your own stuff does allow you to do
| is make choices and trade offs. If this switch goes down
| and we have 6 hours of downtime to replace it what is
| that worth etc.
| fffrantz wrote:
| Agreed 100 percent. Software is the easy part. Getting
| HVAC, power and network up to the levels of cloud
| providers is difficult to get right and prohibitively
| expensive.
|
| For instance, the cost for a pair of redundant symmetric
| gigabit fiber is in the thousands a month and may require
| tens of thousands of construction costs. These quickly
| add up, and the upfront costs can quickly reach six
| figures.
| saltminer wrote:
| I remember seeing a quote for 500/500 metro E from
| Comcast several years ago. $12k to install, $1.2k/mo. And
| that only involved laying a few miles of fiber, no
| redundancy. Dedicated lines are no joke. If you're AWS or
| GCP, you can be your own ISP and mitigate this to some
| extent, but that's just the physical connection they save
| on.
|
| You can always save by going on-prem, assuming you have
| no uptime requirements. But the moment you sign an SLA,
| those savings go out the window.
| bradstewart wrote:
| Not to mention security compliance. If you can afford all
| of that, seems pretty likely you'll also have SOC2/etc
| needs. Being able to "ignore" the whole physical security
| aspect of that stuff is a huge benefit of the cloud.
| amluto wrote:
| There's a huge middle ground between on-prem and GCP/AWS.
| You can rent space and connectivity from in very
| competent datacenter without any of these big fixed
| costs.
| hot_gril wrote:
| Can rent the space, but you still have to buy the
| hardware. Maybe there's money to be made running some
| low-availability cloud service offering newer hardware.
| amluto wrote:
| Have you checked the price for a system capable of using
| two redundant 10Gbps links lately? It's _cheap_. You
| could put gear like this in your closet at home and not
| feel particularly silly about it, especially if you are
| willing to buy still-current used enterprise gear.
|
| For that matter, have you checked the price, in qty 1, of
| a server that will absolutely destroy anything reasonable
| from a major cloud vendor in terms of IOPS to stick
| behind that switch or router? Even if you believe the
| numbers on the website of a major server vendor and
| forget to ask for a discount, it's still quite reasonable
| in comparison to a major cloud.
| hot_gril wrote:
| Yeah, tends to be. But it's more efficient for multiple
| customers who don't need the hardware full-time to share
| it. Someone could set that up without all the expensive
| HA guarantees and other stuff a regular cloud provides.
| hot_gril wrote:
| Starting with a basic web backend, you probably have a
| database that you can't simply run replicas of.
| wmf wrote:
| Are you talking about colo or an office? Because carrier-
| neutral colos are pre-wired with plenty of bandwidth that's
| 10x-100x cheaper than public clouds. Yes, you need routers
| but the savings elsewhere should pay for them.
| phh wrote:
| You feel like it's new, I don't. I've evaluated migrating from
| full blown servers to cloud at various scales for various
| usages: OTA server for 1M active devices, stat server for 1M
| active devices, build server for Android (both pure Android
| that parallelizes and checks out super nearly and vendor
| Android where they broke all of that). In all those cases the
| cost of cloud was like an order of magnitude costlier. You're
| mentioning SSD, the various clouds I tried (I couldn't remember
| which I tried sorry...) had bad storage performance (it was
| especially bad when building Android). Also not all those
| servers were maintained by my employer, when we pay top end
| 100Gbps server we can pay someone to maintain it for us as well
| for much lower money than the "cloud tax".
|
| I have no doubt there are great use case for cloud, and that at
| the proper scale you can negotiate, and I understand that
| startups might be faster moving with the cloud. But I feel like
| the highest value cloud provides is 1. replacing capex with
| opex, and 2. making scaling easier with the direction: cloud is
| pay first, get questions from direction later. "on-prem" is
| negotiate with the direction until the service degrades and
| then scramble to integrate the new server under the pressure.
| hot_gril wrote:
| Clouds have historically been designed for high-availability
| workloads, which are very hard to handle yourself. It doesn't
| always make sense for experimentation or AI training, though
| they might be trying to optimize more for that now. At past
| startups, we were fine just buying machines to run on-prem.
| wmf wrote:
| Hyperdisk is a SAN; it's not comparable to local storage.
| Unfortunately Google's local SSDs are also overpriced.
| 015a wrote:
| Sure, but you literally cannot use any other kind of SSD with
| the n4-class instances, and n4 are the only instances they
| offer on Emerald Rapids; they're advertised as general
| purpose, flexible, and high performance, basically their
| workhorses. If you want to use a local SSD you have to use
| older generation chips.
| jiggawatts wrote:
| > start struggling over the next
|
| They're already struggling!
|
| In the past, the three big clouds would deploy cutting-edge
| CPUs at scale ahead of general availability for ordinary
| rackmount servers.
|
| Now?
|
| The AMD EPYC 9004 series processors were announced over a year
| ago in March 2023, but are still trickling out as "preview" in
| selected regions in Azure. Similarly, Intel Xeon fourth-gen
| CPUs haven't even been announced by Azure, but Intel is already
| shipping fifth-generation CPUs!
|
| I suspect that up until a couple of years ago, the usage of
| public cloud was increasing at such a pace that the providers
| were buying a truckload of CPUs every six months, so they were
| keeping up with the latest tech.
|
| They must have had new signups dry up as soon as interest rates
| went up, and they're now milking their existing kit instead of
| expanding with new generation servers.
| mgdev wrote:
| This will be a boon for Oracle.
| Sarkie wrote:
| Aren't they killing it in this space?
|
| Nvidia love OCI I thought I read
| htrp wrote:
| OCI loves Nvidia.... the feeling is less than mutual
| apitman wrote:
| Over the weekend a coworker was helping someone in our lab
| prepare data for an important conference talk.
|
| We accidentally ran up a rather large bill because while the EFS
| storage pricing was simple enough, the usage pricing bit us.
|
| It seems like AWS' entire business model is making the pricing so
| confusing that you don't know what it will cost until after
| you've used it. It feels weirdly similar to the US
| healthcare/insurance situation.
|
| More competition in this space can only be a good thing.
| matsz wrote:
| This is precisely why I stick to dedicated servers for my own
| personal projects.
|
| $40/month for a machine, doesn't get any more predictable than
| this.
| baobabKoodaa wrote:
| If you rent a dedicated server from AWS, you will be hit for
| various additional fees which will likely dwarf that
| $40/month.
|
| So the issue here is not really shared vs dedicated
| instances. The issue here is that a particular cloud provider
| (namely AWS) has set up an opaque fee structure.
| drdaeman wrote:
| > If you rent a dedicated server from AWS
|
| Who, in their right mind, goes for a bare metal to AWS,
| when there are so many decent and time-tested options out
| there?
| jedberg wrote:
| Data Gravity. If you already have all your data in AWS,
| and your app that is generating new data is in AWS, it
| makes a lot of sense to get bare metal in AWS to do batch
| workloads on that data, so there is no egress fees.
| smabie wrote:
| crypto hft
| mywittyname wrote:
| Anyone who has experience with AWS or is looking to hire
| candidates with known skills. AWS is the industry
| standard. A lot of quality candidates know it and use it,
| because it pays the best and has the most job
| opportunities. And most companies use it because they
| were first to market and its easy to find candidates.
|
| I'd argue that startups should have a good reason for not
| using AWS. The costs for their basic services is not that
| much compared to the cost of development.
| ElevenLathe wrote:
| In my experience, when people say "dedicated server" they
| typically mean something like OVH rather than a real
| "cloud" provider (AWS, GCP, Azure, etc.). In other words,
| something morally equivalent to colo but without having to
| ship servers around.
| baobabKoodaa wrote:
| Yes, that is likely the case here as well. But if someone
| is not well versed in cloud providers and reads these
| comments, they might be misinformed without clarifying
| that this issue is AWS vs other providers, as opposed to
| dedicated vs shared resources.
| matsz wrote:
| Oh no, I would never go AWS for dedicated. Should've
| specified that in my comment so people don't get
| disappointed if they try to get that from AWS.
| oooyay wrote:
| I suspect they're not getting a dedicated server from
| Amazon. Anyone doing that is either foolish _or_ is so
| locked into AWS infrastructure that it cost-wise makes more
| sense than to expand a network to another provider. Fun
| times, indeed.
|
| What I started doing is just running my own "cloud" out of
| my house for personal projects. I have all of the things I
| need. There's some overhead in terms of maintenance and up-
| front setup cost in terms of time and equipment, but after
| that it's pretty smooth sailing.
| whiplash451 wrote:
| You still have network performance limitations + the
| security aspects to run by yourself (as in: not getting
| your data drive encrypted by a ransomware)
| whiplash451 wrote:
| You can get much cheaper dedicated servers with alternative
| providers such as OVH.
| zeroxfe wrote:
| There's no way you're getting a machine for $40/month. :-)
|
| Did you mean a dedicated VM or VPS?
|
| (I have a bunch of actual dedicated machines with different
| providers, and this would save me a lot of money.)
|
| (Edit: holy moly those prices are fantastic!)
| distantsounds wrote:
| OVH and Hetzner have offerings at that price range. you can
| use a tool like serverhunter.com to find all sorts of
| economically-priced servers.
| matsz wrote:
| Yep, I'm renting two servers from Hetzner; have one in
| Germany and another in Finland. Both cheap (EUR40/month)
| and over the few years I've been using their services, I
| have nothing to complain about.
| javchz wrote:
| OVH's so you start and server4you offer dedicated machines
| for around that price. For personal projects they work
| great
| dinvlad wrote:
| Yeah, Hetzner has 20 vCPU 64GB RAM i5-13500 servers for
| $40/mo:
|
| https://www.hetzner.com/dedicated-rootserver/matrix-ex/
|
| This is ~10x cheaper than the closest AWS option, and
| without the extra fees.
|
| Also of note that Hetzner is profitable, which means AWS
| has been operating at an insane markup.
| thorncorona wrote:
| > Also of note that Hetzner is profitable, which means
| AWS has been operating at an insane markup.
|
| You can also evidence of this on their 10-Q :^)
| ikiris wrote:
| If you think offering a cloud service is at all the same
| as offering someone a box in a rack, I invite you to
| compete in the space.
| dinvlad wrote:
| Hetzner does have a cloud VPS option, which is still very
| affordable: https://www.hetzner.com/cloud
|
| This is pretty heads-to-heads with EC2, in terms of how
| it works behind the scenes.
| ikiris wrote:
| So go with hetzner?
| dinvlad wrote:
| Already do :-)
| TacticalCoder wrote:
| It's more than a box in a rack. These providers do
| actively monitor and fix these boxen. They can all be
| rebooted remotely as if you physically hit the button,
| you've got interfaces to access the machine as if you
| were logging in physically with a
| DB-9/RS232/ethernet/whatever console, etc.
|
| It's not just "space in a rack and you deal with the
| servers yourself and you come to fix them if they break".
|
| These companies know what they're doing.
| ikiris wrote:
| You're missing the point. There's a massive difference
| between getting a box service, and getting a highly
| available regionally distributed service with a semblance
| of a SLA of bandwidth to anywhere on the planet. To quote
| a former manager, its not even apples and oranges, is
| apples and pumpkins. They simply aren't in any way the
| same scope.
| notarealllama wrote:
| Dang, that is a fantastic deal. The EUR100 / month is
| even better - DDR5 RAM, 2TB NVME raid 1, and it's all
| customizable too. Just have to wait for Ubuntu 24.04 to
| be available and might have to make this switch.
| dinvlad wrote:
| Yeah, they're all pretty fantastic. Can even upgrade
| existing servers with more RAM/disk, or store ~250 TB of
| data for $400/mo etc.
|
| Not to mention unlimited free 1 Gbit egress/server
| yashg wrote:
| Just setup two Ubuntu 24.04 servers in last couple of
| days. One at Hetzner and another on AWS.
| margorczynski wrote:
| And this is just for the "dumb" EC2 instance, the markup
| on their "smarter" stuff is probably much higher. In
| general I understand why one would want to start off in
| the cloud but staying there for 10+ years is quite absurd
| given the costs.
| toast0 wrote:
| I've got a dedicated machine for $30/month. It's ancient, a
| xeon L5640 with 16 GB ram, and 1 TB spinning disk, but it's
| dedicated and it works great. Well actually, the first one
| stopped working well, and I got a replacement with double
| those specs for the same price; and the second one is
| working great. I also run with full disk encryption,
| because I don't trust their opsec on wiping drives, so
| that's a bit of a hassle to reboots, I have to get a
| console with IPMI and put in the disk password, although I
| saw something [1] last night that inspires me to consider
| the possibility of automation.
|
| I recommend shopping at https://lowendbox.com/ and
| https://lowendtalk.com/categories/offers
|
| [1] https://github.com/emtiu/freebsd-outerbase
| eddd-ddde wrote:
| Also a dedicated VM is /literally/ a dedicated machine.
| It's in the name!
| jameshart wrote:
| A dedicated virtual machine is a _kind_ of dedicated
| machine, sure. Like a private community swimming pool is
| a kind of private swimming pool.
| TacticalCoder wrote:
| > There's no way you're getting a machine for $40/month.
| :-)
|
| Oh you can!
|
| I've got several dedicated servers at OVH. My absolute
| cheapest one is an "ECO" / Kimsufi (Kimsufi is a company
| which spun out of OVH then, a few years later, back into
| OVH) which I pay... 5 EUR / month. 6 EUR / month with VAT
| (so 6.5 USD per month).
|
| Sure, it's not beefy at that price: an Atom N2800 with 4 GB
| or RAM but it _is_ a dedicated server with its own IPv4 IP
| (yup, there can be uses for that).
|
| I mostly use it as a jump host / reverse-ssh-with-a-known-
| fixed-IP thinggy.
|
| They've got great dedicated servers at very good price and
| they're not the only ones in that space.
|
| These can be rebooted/reinstalled remotely and they're
| monitored: OVH shall deal with hardware failure, if any,
| for you (never had any so far).
| buildbot wrote:
| I'm renting a box from Hetzner - 4x 22tb HD, 2x 1.5TB NVME
| SSD, Ryzen 3600, 64gb ecc ram... for 100$ per month! It's
| nice :)
| rco8786 wrote:
| > AWS' entire business model is making the pricing so confusing
|
| My go to line here is that Cloud was a ZIRP. Like the whole
| entire thing. Took us ~10 years to wind up the cloud, and it
| will take years to unwind it, but the mass migration away is
| already happening.
|
| To be clear, I don't mean like AWS is going out of business or
| anything. Just that companies are a) realizing how insanely
| expensive it is, b) realizing how wildly volatile the pricing
| is, and c) starting to reach for services with transparent,
| fixed pricing
| brigadier132 wrote:
| I don't think people understand how absurd egress costs are.
| I was talking with some people in game dev because im making
| a multiplayer game and i wanted to understand what my future
| costs would be. Bandwidth was #1 over compute.
| apitman wrote:
| I've done a decent amount of research on the cheapest VPS
| providers for network-heavy applications. Take a look at
| Hetzner and OVH if you haven't already. Also here's a
| useful comparison:
|
| https://getdeploying.com/reference/data-egress
| bauruine wrote:
| Yeah it's more than absurd. I have a rented server that
| does 230TB egress a month. That's 4710$ in AWS egress fees
| (0.02$ per GB). I pay 40 Euros or about 0.0002 per GB and
| that includes an AMD Ryzen 5 3600 with 64GB RAM.
| apitman wrote:
| May I ask who your provider is?
| bauruine wrote:
| That's https://mevspace.com/ but hetzner, scaleway, ovh,
| psychz.net, reliablesite.net etc. have similar pricing.
| If you increase your budget to 100 there are many, many
| more in Europe and the US.
| apitman wrote:
| Thanks!
| hn_throwaway_99 wrote:
| > My go to line here is that Cloud was a ZIRP. Like the whole
| entire thing. Took us ~10 years to wind up the cloud, and it
| will take years to unwind it, but the mass migration away is
| already happening.
|
| This is Hacker News echo chamber stuff. There is certainly no
| mass migration away from the cloud. Yes, I personally saw
| companies in the 2010s say "we're moving everything to the
| cloud!" without adequate planning or cost analysis and then
| saying "OK, everyone off the cloud" once they got an insane
| cloud bill.
|
| But cloud costs can be managed, and for many, many companies
| the cost of hiring people to manage all this infrastructure
| and services is usually _way_ more than a well-managed cloud
| project. Also, the canonical rationale I see on HN for moving
| away from the cloud is "I can just rent a box for $X/month".
| If all you're using the cloud for is a dumb, static set of
| compute, I agree that you can probably do it cheaper on your
| own. I know hardly any companies (from small startups to
| large enterprises) who use the cloud that way.
| rco8786 wrote:
| > for many, many companies the cost of hiring people to
| manage all this infrastructure and services is usually way
| more than a well-managed cloud project.
|
| Yea, this is the line that everyone uses. I've worked at
| these companies, and the reality just doesn't line up with
| that. You end up still needing your whole Ops team, they're
| just building cloud tooling instead of on-prem tooling.
|
| As to whether or not the migration away is
| happening...major cloud providers are already seeing people
| leave, and profits starting to contract. _It 's very
| early_. Like I said in the original post, it took us a long
| time to wind up the cloud and it's going to take a long
| time to come back to reality.
|
| Something roughly cloud-shaped will probably always
| remain..there are some legit use cases especially for
| companies that have spiky load profiles. I don't mean we're
| literally going back to running servers out of our IT
| closets. I just mean that as a whole we're going to be
| moving back to simpler deployments, simpler architectures,
| and most importantly, fixed/predictable costs.
| hn_throwaway_99 wrote:
| > I just mean that as a whole we're going to be moving
| back to simpler deployments, simpler architectures, and
| most importantly, fixed/predictable costs.
|
| That I can definitely agree with, I just believe that's
| fully possible to do with cloud (for the most part,
| though there are certainly some head turners like the
| recent news that AWS was charging for forbidden attempts
| on _private_ S3 buckets, which is bonkers) and cloud cost
| management tools.
| rco8786 wrote:
| Yea that's totally fair. I think we're using slightly
| different definitions of "cloud" here. I'm mainly focused
| on the managed services/abstractions, things that
| abstract away the server itself and are usage based
| billed. But firing up a few ec2 instances is still
| technically "the cloud", you are right.
| npalli wrote:
| No, not HN echo chamber. 83% of CIOs want to move workloads
| back to on-prem or private cloud.
|
| https://x.com/michaeldell/status/1780672823167742135?s=46&t
| =...
| hn_throwaway_99 wrote:
| I would take that with a giant tub of salt:
|
| 1. As the saying goes, talk is cheap. It's one thing to
| ask "What do you plan to do?" vs. what you actually do.
| Look at revenue graphs for AWS, Azure and GCP over the
| past 5-10 years, right up until the end of 2023. They are
| definitely not shrinking.
|
| 2. I'd be more than a bit skeptical of the messenger,
| given that Dell obviously has a vested interest in
| telling people they need to buy more servers.
|
| 3. Even if you take what the surveyed CIOs say at face
| value, asking "Are you planning to move some workloads
| back to private cloud/on-prem from public cloud" is
| totally consistent with what I said. There was rush of
| "just put everything on the cloud" without thinking
| through it strategically. But just because you're pulling
| back on a few ill-thought-out cloud projects doesn't mean
| that overall industry-wide public cloud investment isn't
| still skyrocketing.
| ryandrake wrote:
| > It seems like AWS' entire business model is making the
| pricing so confusing that you don't know what it will cost
| until after you've used it. It feels weirdly similar to the US
| healthcare/insurance situation.
|
| Not just the healthcare situation, but everything. There's
| nothing much more stereotypically American than "not knowing
| what you're going to pay for something until you're billed."
| Dozens of little fees on your cable or ISP bill, resort fees in
| hotels, service charges on your restaurant bill, fees on car
| rentals, fees from your bank when you so much as breathe on
| your account, and of course sales taxes which for some reason
| are never listed on the tag in any store.
| doctorpangloss wrote:
| People complain about this relentlessly, but never change.
|
| Additionally, every single business owner I know complains
| that whatever it is they're selling, (1) it isn't worth the
| brain damage to sell to customers looking for the lowest
| price, (2) as long as people comparison shop in a harebrained
| way, hook pricing (aka up front pricing that looks low and
| turns out high) is only rational. It's not like they're
| providing a bad service for the cost.
| pixl97 wrote:
| I mean this is why a lot of first world countries have
| consumer protections that demands the retailer/service
| publishes prices up front. Course in the US there is quite
| a counter lobbyist group that prevents just that from
| happening.
| thfuran wrote:
| >hook pricing (aka up front pricing that looks low and
| turns out high) is only rational.
|
| Yes, many deceptive business practices are rational acts on
| the part of the businesses. That doesn't mean they should
| be tolerated.
| thaumaturgy wrote:
| I have begun to think of this as a "large population
| error", and I'm noticing it in a lot of different
| markets.
|
| At small scales, annoying your customers is bad business.
| You only need to lose a few before it begins to hurt.
| Customer complaints are more likely to be a consideration
| in business decisions.
|
| At larger scales, a business can begin to preferentially
| adopt practices intended to drive away some customers.
| Perhaps you don't want the "pathological" customers, to
| borrow one of Patrick McKenzie's terms. You can make more
| money with less effort by being more selective about your
| customers.
|
| At _extremely large_ scales, you largely stop thinking
| about groups of customers altogether. All of your
| decisions are driven by aggregates -- did _all sales_ go
| up this quarter, or down? The effort required to do a
| deep dive into the behaviors and preferences of any
| individual market segment may not make sense anymore on a
| quarterly basis. At this scale, you might be able to
| afford to annoy tens of thousands of customers and still
| have a very nice graph next quarter.
|
| So, when someone says a business practice shouldn't be
| "tolerated", that's a perfectly reasonable position,
| except it doesn't actually work for businesses operating
| at extremely large scales. It's too difficult for
| customers to organize a protest in a way that will
| influence that business's decision-making.
|
| So much business has moved online in the last 20 years,
| while the US has leveled off at 80% urbanization over the
| same time period, along with more and more businesses
| congealing into BigCos, combined with the recent
| domination of private equity: lots and lots of things are
| now operating at a scale where customer concerns just
| aren't a part of the business model anymore.
|
| Coffee shops, fast food, big-box retail, online retail,
| SaaS, PaaS: all of these can thrive while running on
| exorbitant pricing and abusive customer policies, because
| their volume of customers is so large that it's nearly
| impossible to be so bad that you'll piss off enough
| customers to impact your decision-making. (Unless you're
| Sony.)
| staunton wrote:
| > It's too difficult for customers to organize a protest
| in a way that will influence that business's decision-
| making.
|
| It's actually very easy (in theory). You (vote for
| someone who will) _ban_ fraudulent and anti-competitive
| behavior, _sue_ the offenders and have them pay _huge_
| fines. The fact that this very rarely works out is a
| failure of the political system.
| s1artibartfast wrote:
| The intuitive answer is competition, where customers move
| from user-hostile companies to new ones.
|
| The problem in my mind is that the largest companies have
| too much efficiencies of scale to compete with on price.
|
| When competition can't undercut on price, it is hard to
| argue that customers aren't being served by monopoly mega
| corps.
| wongarsu wrote:
| Which is exactly why they should be banned to keep the
| market overall healthy. Preventing deceptive practices in
| a market economy this is a prime example of useful
| government "intervention", just like supporting contract
| enforcement or preventing theft
| AnthonyMouse wrote:
| They can always justify it as not knowing the price.
|
| The sales tax rate is different in different places. It
| costs a different amount to ship to Florida than Alaska
| and picking it up at the factory is free (even if nobody
| does). The advertised price is if you have your own
| modem, renting one from the cable company is more.
|
| None of these are inherently wrong. You _should_ be
| paying more if you 're having it shipped to a remote
| location with high shipping costs, and the cost of that
| shouldn't be dumped on every other customer. But it's
| kind of a loophole if you want the advertised price to be
| lower than what people are actually going to be paying in
| practice.
| wongarsu wrote:
| There are some cases where not knowing the price is
| reasonable. As you say, shipping can often only be
| calculated if you know all items and their destination.
| If you advertise on national TV you can't name a price
| that includes sales price (unless the company eats the
| difference). But these could be treated as tightly
| regulated necessary evils, not as a justification that
| showing final prices is always impossible.
|
| There is no reason you can't show final price including
| tax on the label in a physical shop. There is no reason
| why a restaurant should be able to charge a 20% service
| charge instead of increasing regular prices by 20%. If
| you are buying a concert ticket or airline ticket the
| displayed price should include all mandatory fees. They
| can upsell you on additional services, but they can't
| suddenly notice in the last checkout step that your price
| is higher because the website you are using is charging a
| fee; that fee was known to them at the beginning of the
| transaction and should have to be disclosed at that point
| in time at the latest. If you want to go even further you
| can also dictate that shipping and handling fees are only
| allowed to include reasonable costs of actual shipping
| and handling.
|
| All of these are normal common-sense regulations in most
| first-world countries.
| AnthonyMouse wrote:
| > There is no reason you can't show final price including
| tax on the label in a physical shop.
|
| Sure there is. When the price label is affixed by the
| factory/warehouse then you would have to track where
| everything is going and be unable to share inventory.
| Also, if the sales tax rate changes then all the labels
| become wrong. These would ultimately increase costs for
| consumers.
|
| Adding sales tax is also not at all misleading because
| the customer is not going to be surprised by it and there
| isn't going to be a competing merchant across the street
| who can avoid charging it.
|
| There are also business customers with their own sales
| tax ID and they can buy things without paying sales tax
| when they're being incorporated into a product where they
| collect the sales tax themselves.
|
| > There is no reason why a restaurant should be able to
| charge a 20% service charge instead of increasing regular
| prices by 20%.
|
| This is actually true. A mandatory undisclosed fee is BS.
| But it doesn't help much, because if they want to do it
| then they just make it "optional" where the way to avoid
| it is more of an inconvenience than paying the fee.
|
| > They can upsell you on additional services, but they
| can't suddenly notice in the last checkout step that your
| price is higher because the website you are using is
| charging a fee; that fee was known to them at the
| beginning of the transaction and should have to be
| disclosed at that point in time at the latest.
|
| The last step is where you disclose your address. Before
| that they may not even know which _country_ you 're in,
| much less the city/state, and there are a thousand
| legitimate reasons to have different prices or fees in
| different jurisdictions.
|
| > If you want to go even further you can also dictate
| that shipping and handling fees are only allowed to
| include reasonable costs of actual shipping and handling.
|
| That doesn't really help, they're typically charging the
| actual cost. They just don't include it in the advertised
| price because it makes you inclined to make the purchase
| online instead of saving $10 by picking it up for the
| same price but no shipping charge the next time you go to
| the competing local store.
|
| There's a reason Amazon's major competitive advantage is
| free two day shipping, derived from having the scale to
| achieve low shipping costs themselves.
| cogman10 wrote:
| > When the price label is affixed by the
| factory/warehouse then you would have to track where
| everything is going and be unable to share inventory.
|
| Price tags are pretty much universally handled at the
| stores. There are some goods where the price tag is
| attached, but that's more the exception and not the rule.
|
| This is a solvable problem. So much so that if you've
| traveled in most nations you'll see that all prices
| include tax. Not including tax is a particularly weird
| aspect of US culture that simply doesn't exist in other
| nations, even those with a large amount of imported
| goods.
| satvikpendem wrote:
| This argument reminds me of pre-ordering games. People
| are simply too dumb to stop doing so, even if it's in
| their best interest. But those that do are often rewarded
| for not pre-ordering in the form of lower prices and more
| content, hen the game eventually goes on sale. It doesn't
| mean that pre-ordering should be banned, however.
| duxup wrote:
| > it isn't worth the brain damage to sell to customers
| looking for the lowest price
|
| The cheapest customers are always the most expensive to
| work with. It's a sad reality.
|
| I worked at several companies who for some foolish reason
| saw the cheapo customers as some untapped market and when
| they raced to the bottom they lost every time.
| antisthenes wrote:
| > It's not like they're providing a bad service for the
| cost.
|
| When it comes to healthcare, they absolutely are. The US
| spends more than double for the same outcomes as other
| developed countries.
|
| And no, this isn't because of the cost to develop new and
| novel drugs (which aren't used in 99.9% of routine health
| care)
| jonahhorowitz wrote:
| California gets a lot of flack for having too much
| regulation, but this change is very welcome for consumers.
|
| No more junk fees in CA.
|
| - https://www.sfgate.com/food/article/sf-restaurants-junk-
| fees...
| autoexecbat wrote:
| Is this just restaurants?
| krisoft wrote:
| > People complain about this relentlessly, but never change
|
| What do you mean by this? How would "people change" to get
| out of resort fees or confusing pricing systems?
| joshstrange wrote:
| I believe they are saying that people fall for the hook
| pricing and won't go with an alternative that is upfront
| about their price (because it looks higher). If companies
| aren't rewarded for doing the "right" thing then why
| would they do it?
| vanviegen wrote:
| Vote for politicians that haven't been bought by big corp
| (yet).
| jimjimjim wrote:
| "people" as a group need to stop shopping solely on
| price. Btw, You may look at other factors but as a
| general rule "people" look at price.
| tpm wrote:
| Ban them. Here in the EU such behavior is mostly illegal.
| s1artibartfast wrote:
| people could stop buying from sellers with confusing
| pricing systems.
|
| People are capable of change.
| diob wrote:
| This is why other countries just have laws for this, it
| evens the playing field and removes the bs.
| alex_lav wrote:
| Hiding blatant fraud behind the label of "convenience". Every
| industry. Pay for Amazon prime to get 2 day shipping "for
| free". Item arrives in six days. Prime does not get refunded.
|
| Pay +10% for "priority" rides in Lyft, supposed to arrive in
| 1-5 minutes, whereas "regular" is 7-15. Car shows up in 18
| minutes. Priority payment does not get refunded.
|
| Honestly what's even the point in caring anymore. Living in
| America is about getting grifted until you can hopefully
| figure out your own grift. The irony of posting this thought
| on this website is intentional.
| lapphi wrote:
| Yes. Frank reynolds in the popular television show it's
| always sunny in Philadelphia lays it out in plain terms on
| one of the episodes. "In America you're either the duper or
| the dupee". PT Barnum also knew this essential truth about
| our nation.
| xkcd-sucks wrote:
| Less advanced economies have discovered this pricing
| innovation, but are generally not able to scale it beyond
| hostess bar scams and the like :)
| wongarsu wrote:
| Which is especially weird for the country that fought the
| cold war to show the supremacy of capitalism; a system that
| is based on market actors knowing the price and value of
| every product and service offered and making rational
| purchase decisions based on that.
| heavyset_go wrote:
| Capitalism can't work without information asymmetry when it
| comes to pricing.
| TeMPOraL wrote:
| Maybe that's why credit cards are so extremely popular in the
| US? I suppose if you never know how much you're going to pay
| for anything, it helps to have a buffer between the vendor
| and your bank account, allowing you to review the charges
| after the fact, and should you dispute some, be on a somewhat
| even playing field with the vendor.
| silverquiet wrote:
| I have to say that my response to "not knowing what you're
| going to pay for something until you're billed" is basically,
| "well I guess we'll see if I can pay it then". You sort of
| start asking yourself, "what if I just don't pay?". I have a
| (deadbeat) buddy who's entire medical plan is essentially to
| give a fake name to the ER, and if you're poor in America,
| what else can you do?
| pjlegato wrote:
| > give a fake name to the ER, and if you're poor in
| America, what else can you do?
|
| The American government spends something approaching a
| trillion dollars annually on Medicaid, "a government
| program that provides health insurance for adults and
| children with limited income and resources."[1]
|
| Seperately, Obamacare[2] created a private health insurance
| market where low income people can obtain free or heavily
| discounted private health insurance coverage, according to
| their income.
|
| [1] https://en.wikipedia.org/wiki/Medicaid [2]
| https://en.wikipedia.org/wiki/Affordable_Care_Act
| notaustinpowers wrote:
| Obamacare is (largely) a failure due to insurance
| meddling and laws preventing government insurance
| programs from negotiating prices with drug companies.
|
| I'm a healthy individual, not even 30 yet, never smoked,
| never broken a bone, and never even had stitches in my
| life. And when I last checked their prices, I was making
| only $45k/yr. For coverage with Obamacare, I would be
| paying $350/month for the "catastrophic" plan. Which
| includes no prescription copay, no dental, no vision, and
| only kicks in after I've spent $100,000 in one year, and
| it takes 6 months to take effect after signing up. It's
| only there for serious issues like losing a leg or
| cancer.
| dangrossman wrote:
| Obamacare is not a plan, it's a law that set minimum
| standards for health insurance plans, created
| marketplaces to cross-shop plans across providers on a
| single website, and created a system where you can
| receive government subsidies towards their cost if your
| income is low enough. At $45K, you would receive
| subsidies towards your insurance premiums.
|
| I've bought my insurance on the ACA marketplace since it
| opened ten years ago. There's no difference I can see
| between the plans offered on the health insurance
| marketplace and those offered directly from the websites
| of the same insurers that offer coverage in my state
| (Blue Cross, CVS Aetna, United, Ambetter, etc).
|
| The very highest deductible "catastrophic" plan offered
| on the marketplace in California for a 29 year old has a
| $9450/year deductible, which is also the maximum out-of-
| pocket expense for the year if you have this plan. A
| $100K deductible plan does not exist, and when you enroll
| during the annual enrollment period or after a qualifying
| life event, plans take effect the day you make your first
| payment, not months later.
|
| I'm 10 years your senior and pay less than $350 per month
| with a lower deductible than the plan quoted above, with
| no government subsidies.
| notaustinpowers wrote:
| I use Obamacare in the rural southern way, referencing
| ACA and healthcare.gov, lol.
|
| I'm also in Georgia which may have different regulations
| regarding the deductibles. I remember looking at the
| plans around 2 years ago and realizing that there was no
| way I could afford the premium, let alone the yearly
| deductible.
|
| I just took another quick look at a non-Healthcare.gov
| site. Insurance for me would be $313/month with a $9,100
| deductible. But it does not cover doctor visits, generic
| drugs, or specialist visits until after I pay the full
| $9,100.
|
| Why would I want to pay $313/month for essentially no
| coverage until I spend 20% of my income towards a
| deductible before I see any benefits?
| dangrossman wrote:
| You get an annual doctor visit for free with any
| insurance plan. There is no deductible or copay. There
| are other categories that are covered with no deductible.
| My wife had a $13,000 IUD insertion under anesthesia at a
| hospital due to complications, and this cost her $0 with
| insurance, without having met her deductible, since
| reproductive health is covered with no out of pocket cost
| under all ACA plans.
|
| I'm on healthcare.gov looking at Georgia's plans this
| year for someone with $45K of income. You have options
| starting at $129/month. Many of these sub-$200 plans get
| you doctors visits for $40-60, prescription drugs for
| under $25 each, mental health treatment for under $60 per
| visit. This is all without hitting your deductible at
| all, they're day 1 prices.
|
| If you paid the cash prices for many of these doctors,
| specialists, therapists, they'd be many times higher than
| the insurance negotiated costs. Look under the "covered
| costs" estimates for things like mental health treatment,
| diabetes maintenance, broken bone treatment, etc and
| you'll see that the estimated annual cost for the insured
| is often half or less the plan's deductible -- which
| tells you that hitting the deductible is not when the
| savings start. I don't think my wife or I have ever hit
| our out-of-pocket maximums in a year, yet the insurance
| has saved us more than it's cost in most years.
|
| You're going to start interacting with the healthcare
| system a lot more than you have in your 20s once you're
| in your 30s. We all do at that age. And if you have even
| the worst ACA plan, you'll start to understand what it's
| doing for you regardless of the deductible.
| phonon wrote:
| This is the cheapest plan I could find in GA. It said it
| would subsidize the $357/m cost down to $113 for $45k 40
| year old Male non-smoker, single. It seems quite a bit
| better than what you are suggesting. (I used
| https://www.healthsherpa.com/ to more easily check out
| available plans.) It covers preventative care, and after
| the $9100 deductible, it seems to cover pretty much
| everything with $0 copay, as long as its in-network.
|
| (You also get their negotiated rates when you go to the
| doctor, I assume.)
|
| https://d3ul0st9g52g6o.cloudfront.net/2024/GA/sbc/2024_58
| 081...
| softsound wrote:
| It's still expensive, I worked as a contractor full time
| for a tech company for a year which basically means I pay
| for my own insurance through Obamacare (though they did
| suggest insurance at work but they didn't pay anything
| towards it so it is just is in case you want a 4th party
| to help you help yourself). Still long before that I
| basically just said I didn't have insurance and would get
| the price knocked down considerably. Half the time it's
| cheaper not to even have insurance... But now I'm
| "responsible" in the hopes it might bring down the price
| for other things in the future. For planned events I
| recommend health insurance, despite how often it's not
| all that worth it. Honestly though my current insurance
| copay cost as much as being uninsured in other countries
| so I kinda laugh at this crazy idea. I pay about $100 to
| visit any specialist doctor and it only cost $80 to be
| uninsured in some other countries on top of the $350 a
| month I already pay.
| silverquiet wrote:
| We live in Texas.
| pjlegato wrote:
| Sounds like your issue is with Texas, then, not with
| America.
|
| Why perpetuate crude and inaccurate stereotypes that
| smear and disparage America in general? America is
| spending a huge amount of money every year precisely on
| providing healthcare for the poor.
|
| Besides which, it looks like Medicaid does indeed operate
| in Texas, which is discoverable in less than 10 seconds
| of Internet searching:
| https://www.hhs.texas.gov/services/health/medicaid-
| chip/abou...
|
| And, seperately, Obamacare's marketplace works
| nationally, including in Texas: "More Texans than ever
| before enrolled in ACA health plans in 2024"[1]
|
| [1] https://www.texastribune.org/2024/01/24/texas-aca-
| health-ins...
| silverquiet wrote:
| Have you considered perhaps that because I live in Texas,
| I know a bit more about those programs than you who have
| just googled and linked the top results? And that perhaps
| because I live in Texas, I indeed have issues with Texas
| (still a part of America in spite of the wishes of some
| on the right wing)?
|
| From your own article (I knew this stat would be in
| there):
|
| > Currently, Texas leads the nation in the number of
| uninsured residents with nearly 5 million people living
| here without health insurance coverage, nearly double the
| national average.
|
| Why do you think so many are uninsured if it is so simple
| for them to get health coverage?
| pjlegato wrote:
| You said giving a fake name at the ER was the only option
| for poor AMERICANS to obtain healthcare.
|
| That is categorically false, and testifes to some sort of
| deep seated bias against our own country profoundly
| embedded in your worldview -- one which extends all the
| way to glib counterfactual promotion of verifiably false
| information about the country and its supposed moral
| shortcomings.
|
| 2) Texas has a much larger population than 49 of the
| other 50 states -- about 30 million, higher than any
| state but California. Of course the number of uninsured
| residents will be higher than the national average. You
| didn't control for uninsured per capita.
|
| Why are so many uninsured if it's possible (I didn't say
| easy) for them to get coverage? Probably, a large part of
| the reason is people who go around promoting the
| (completely false) common trope that "poor Americans just
| can't get health coverage, the government does nothing
| for them."
| silverquiet wrote:
| Pretty sure my buddy is an American - I've known him
| almost all my life. I'm quite sure he's poor - he asks me
| for money a bit more than I'd like. The fake name thing
| was just a strategy he developed after going to the ER
| due to a blow to the head which caused him to forget his
| name; I'm sure there are many creative ways to get
| healthcare, but most of the of poor Texans (whom I'm also
| pretty sure are Americans considering I'm related to a
| few of them) I've known just go to the ER and ignore the
| bills. It's probably not the best system, but it's the
| one we've got I guess.
|
| As to point 2) I thought about including the info in my
| previous comment, but it wasn't in your own source
| material and I'm a bit lazy. So here, I'll do it in this
| one.
|
| > Texas is still the state with the highest percentage of
| uninsured residents, at nearly 17 percent, according to
| the most recent U.S. Census Bureau survey released
| Thursday. [0]
|
| [0]https://www.texastribune.org/2023/09/14/census-bureau-
| texas-...
|
| Speaking as a Texan (also pretty sure I'm American) who
| lives with some disability, I can tell you that I've
| looked at the systems and it's a real fear of mine that
| I'll end up as one of these statistics.
| vel0city wrote:
| > Why do you think so many are uninsured if it is so
| simple for them to get health coverage?
|
| High numbers of people who have illegally immigrated and
| are worried about getting into programs subsidized by the
| government which might result in questions about
| residency status leading to deportation?
|
| High rates of misinformation surrounding the costs of
| insurance and availability of welfare programs in the
| state?
|
| I just looked up getting a plan in Texas. 40 year old
| male non-smoker in Texas earning $45k/yr can get
| insurance for $128/mo. They're eligible for an HSA, so
| they can put tax-free savings that roll over every year
| into an investment account to help cover the $7,400
| deductible. PCP and preventative care visits are free.
| Generic drugs are $10. Urgent care out of pocket is $160.
| Other plans have slightly higher premiums but much lower
| deductibles, some have different co-pays.
| silverquiet wrote:
| Yes - my plan if ever separated from employer insurance
| is to manipulate my income and expenses (I have
| significant investments I can draw upon and access to a
| family real estate portfolio that allows me to live rent-
| free) in order to get the full subsidy. Most people are
| not like me however; my buddy has several times had his
| car impounded for not paying car insurance. I don't think
| health insurance will make the cut in his budget.
| heavyset_go wrote:
| Medicaid wasn't expanded in several states with tens of
| millions of people living in them[1]. For example, in
| Texas, nobody is eligible for Medicaid based solely on
| their income alone.
|
| Where Medicaid is expanded, the income requirements for
| Medicaid are far from sane. If you make over 100% to 138%
| of the federal poverty line, which is $15,060/year for an
| individual, you are not eligible for coverage. For
| example, someone who makes $16k to $21k a year, depending
| on where they live, is ineligible for coverage despite
| making poverty wages.
|
| [1] https://www.kff.org/affordable-care-act/issue-
| brief/status-o...
| rmah wrote:
| Most Americans don't realize just how huge and extensive
| medicare/medicaid is. They cover 65mil and 88mil people
| respectively, including 40mil children. That's 153mil
| people out of 330mil population -- that's 46% of the
| USA's population. Total spending is $1.9T for both. A
| cost of $12,400/person covered. $1.9T is 7.5% of the
| total US GDP of $25.44T. Medicare/medicaid is a massive
| program. Bigger than social security at $1.4T. It's crazy
| huge.
| silverquiet wrote:
| In that case, why not just go all the way to Medicare for
| All?
| jonfromsf wrote:
| Because with the current medical cost, it would be
| absolutely ruinous. The root problem is American health
| care is far too expensive. The entire system is massively
| wasteful and complex.
| silverquiet wrote:
| Seems like unifying it under a single, already-extant
| federal system would greatly reduce administrative
| overhead though, don't you think?
|
| What's funny is I'm probably one of the people whose
| paycheck is dependent on me not understanding that to
| paraphrase Upton Sinclair. When I started at my current
| employer, one of the devs straight up said the government
| should be doing what we do and we shouldn't have a
| business.
| danielvaughn wrote:
| I worked in a couple of emergency rooms for a total of 6
| years, and you aren't wrong. Assuming the care you need can
| be entirely performed within the ER, then that's a viable
| strategy so long as you don't intend to re-visit the
| hospital.
|
| Hospitals are obligated to provide care regardless of your
| ability to pay in the moment, and of course they often take
| care of indigent and foreign patients, with the
| understanding that they won't be able to recoup their
| costs.
|
| If you don't have your identification on you, then they
| simply give you a number to call and they usually ask to
| respond within a few business days because after that, it's
| harder to get insurance approval.
|
| This all changes if you're admitted to the hospital and
| need surgery, because I think there's pre-approval required
| from your insurance carrier. Though since I worked in the
| ER, my memory is hazy on that.
|
| If they believe you were deceptive and simply refused to be
| identified, then that is technically illegal, so they will
| put up your picture and will alert the police if they see
| you again. But if you never intend to visit the hospital
| again, it doesn't really matter. No one is going to hunt
| you down unless you're doing it on such a large scale that
| it can't be ignored.
|
| This all said, I want to say that I still think it's an
| unethical thing to do.
| explaininjs wrote:
| I have a friend that incurred massive medical debts in
| college when their (college-provided) insurance refused
| to cover services rendered at the (college) hospital. The
| college then made all sorts of threats like "you won't be
| able to graduate until this $$,$$$ bill is paid!!!". Keep
| in mind my friend was only able to attend college on a
| full scholarship as a result of coming from a poor ESL
| family.
|
| They ignored all the threats, the department in charge of
| threatening seemingly didn't talk to the department in
| charge of graduation, and to this day (5 years later)
| they still receive near-daily letters in the mail
| requesting payment. I can't say I find my friend's
| actions unethical in the slightest.
|
| Ah! The UC system...
| simonbarker87 wrote:
| As the old saying goes "If I owe you $100 it's my
| problem, if I owe you $100,000 ... it's your problem"
| silverquiet wrote:
| > alert the police if they see you again
|
| Aren't inmates technically supposed to get healthcare?
| Could be a last-ditch option I suppose if you can't get
| something covered any other way. Personally I'm trying to
| stay employed to keep my coverage, but sometimes you
| can't outrun the layoffs.
| godelski wrote:
| Don't get me started on dark patterns! I think the fact that
| we create them should be something we discuss.
|
| But I think we need to up our game in this cat and mouse game
| a bit. For example, in aggregators -- like Expedia or Google
| Flights, etc -- why not try to capture some of these fees in
| the price? I can search for hotels with parking but what
| about sorting hotels by price and including the parking
| price? It's hard to compare when I see a $120 hotel that has
| a $50 valet vs a $150 hotel that includes parking. But that's
| the thing I'm actually after a lot of times. Or similar with
| flights and baggage fees. We should be able to collect a lot
| of this type of information and properly present it to the
| users and try to make these types of dark patterns
| ineffective (still will be cat and mouse and this is only a
| specific type of pattern, but still, I think there are things
| we can do)
| mikepurvis wrote:
| Probably an opportunity there to gather and expose that
| data, same as how flight aggregators like orbitz twenty
| years ago started showing prices inclusive of taxes and
| fees.
| jameshart wrote:
| Then pay for it with a credit card that gives you a random
| amount of cash back, and pay your credit card bill using your
| tax refund that came as a nice surprise at the end of the
| year, because you over withheld all year so you don't
| actually even know what your real take home pay is supposed
| to be.
|
| No wonder Americans have no idea how the economy is doing.
| TechDebtDevin wrote:
| I'm always surprised by people who expect and are excited
| for their refund. Half the country is pitching about
| politicians taxation plans when all the while they're
| volunteering to overpay out of every paycheck.
| jkingsman wrote:
| When you lack the self-control to save, over-withholding
| + refund becomes an ersatz year-long savings account.
| It's terrible, lacking any interest at all, but when it's
| all you've got, breaking open the piggy bank to get your
| money back feels good even if it doesn't make financial
| sense.
| tatpacc wrote:
| > people who expect and are excited for their refund.
|
| and then pay % of your refund as a fee to CPA, so they
| help you maximize your refund.
| gadflyinyoureye wrote:
| I know how many guns I have. I know the Feds have more.
| Therefore give them their pound of flesh before they show
| up with the myriad of guns.
| _factor wrote:
| Even paying cash. Unless the establishment offers a cash
| discount, you're paying inflated credit card induced
| compensatory pricing.
|
| It's a real shame banks received 2-5% of most transaction
| for what costs them pennies. Sure, there are benefits, but
| their ask isn't covering it.
| 127 wrote:
| It's strange from a foreigners perspective because all of
| this looks exactly like a dark pattern made to distract and
| confuse gambling addicts. Why does American government allow
| direct predation on its own citizens?
| bee_rider wrote:
| Did they manage to get AWS to provide some mechanism to get
| an up-front price in your country? That's pretty neat if
| so.
|
| I think lots of people go along with these as-you-go
| services because they'd rather deal with an unexpected
| bill, than having their servers shut off.
| dheera wrote:
| We really need laws that allow customers to set a legally-
| binding max spend per month on any service, after which it's
| upto the company to suspend services.
|
| This should apply to everything -- cloud compute, healthcare,
| phone bills, internet services, everything.
| balls187 wrote:
| You sort of have that already in the US for health, its tied
| to your insurance and its called max out of pocket. Of course
| it only applies to covered services, and insurance companies
| and medical providers don't necessarily make it simple to
| know what is and is not covered (though it IS improving).
|
| If you plan on seeking medical services without going through
| insurance, the ACA requires providers to provide you with a
| good faith estimate upfront.
| dheera wrote:
| > its tied to your insurance and its called max out of
| pocket
|
| F that, if I paid for insurance I shouldn't have to pay
| _anything_ out of pocket
|
| > If you plan on seeking medical services without going
| through insurance, the ACA requires providers to provide
| you with a good faith estimate upfront.
|
| F that, if there is any out of pocket payments at all, I
| should be entitled to estimates even WITH insurance and
| they should be legally mandated to be within 5% of the
| actual cost.
| balls187 wrote:
| > F that, if I paid for insurance I shouldn't have to pay
| anything out of pocket
|
| That's not how insurance works, bud. For one, you likely
| don't pay for insurance, its heavily subsidized by your
| employer, and they will determine which policies to offer
| you. For two, you share the risk with everyone else who
| has a policy with your provider.
|
| I could see an argument for insurance companies being
| legislated to force them to cover more previously
| uncovered services, as the ACA did.
|
| > F that, if there is any out of pocket payments at all,
| I should be entitled to estimates even WITH insurance and
| they should be legally mandated to be within 5% of the
| actual cost.
|
| Sure, you are welcome to contact your insurance BEFORE
| you obtain services and find out what they will cover. In
| fact, its incumbent upon you to do that, and not expect
| HCP's to do that for you.
| superfrank wrote:
| > It seems like AWS' entire business model is making the
| pricing so confusing that you don't know what it will cost
| until after you've used it.
|
| Maybe I'm naive, but I don't think that's intentional. I think
| it's just a byproduct of trying to build something that works
| for everyone and every use case.
|
| As you make your target market bigger and bigger, you
| continually hit edge case after edge case that you try and
| solve with "just one more" rule or option. Eventually the
| system becomes so complex that no layman can understand it.
| tjoff wrote:
| That would make some sense if the edge-cases didnt have
| 10.000% margin.
| superfrank wrote:
| I've worked at companies that were spending 6 figures a
| month on AWS and we had dedicated AWS employees who helped
| us understand our bill and keep pricing in check. We also
| had GCP and Azure reps constantly reaching out to see if
| they could win us over by showing us how they could lower
| our bill. Overcharging enterprise customers for things
| they're not using is actually a risk to AWS as it gives
| customers a reason to jump ship.
|
| The whales on AWS aren't overspending because AWS uses the
| granularity of their billing to make sure they aren't.
| That's how they keep the whales happy.
|
| For the little guys, I'm not even sure it's worth AWS's
| time to nickel and dime them. AWS hands out thousands to
| ten of thousands of dollars in credits like candy. I'm
| pretty sure it's more important to them to lock in whales
| when they're still minnows than to bilk an extra $500 a
| year out of a 10 person start up.
| Hikikomori wrote:
| Idk about that, it becomes worse at larger scales. We're
| at 7 figures weekly with obscene amounts of waste. But at
| that scale it's like hundreds of Aws bills to understand.
| ikiris wrote:
| Was it "I didn't understand it and/or put in the effort to
| understand it first" ? Nah it must be "theres a conspiracy to
| make it confusing so we pay more"
|
| AWS has one of the nicer to comprehend billing systems.
| zb3 wrote:
| Is there a pre-paid option?
| cyberax wrote:
| Yes. AWS Advance Pay.
| zb3 wrote:
| I couldn't find a list of "eligible charges", also that
| doesn't seem to mean advance payment would be the only
| payment option so I couldn't be billed and the service
| would just stop.. did I miss anything?
| zb3 wrote:
| So you're saying that they can't implement a hard limit, yet
| are able to provide hard-limited trials / student credits?
| It's impossibly hard to believe they couldn't implement the
| pre-paid model
| thefourthchime wrote:
| The other way EFS will bite you is if you don't have enough
| provisioning and you push changes to prod that make it throttle
| to a point where it doesn't work and then you're making
| emergency changes from a hotel room on vacation.
|
| I don't think that's AWS explicit business model, but I think
| they're perfectly fine with it happening.
| balls187 wrote:
| > It seems like AWS' entire business model is making the
| pricing so confusing that you don't know what it will cost
| until after you've used it.
|
| Complicated cost calculations are only a part of the issue. You
| (or your team) also have fault in that you did not take the
| time to understand the costs associated with your decisions and
| not utilize AWS cost management capabilities (that is assuming
| you did get a surprise bill in lieu of an alert saying you hit
| a budget threshold).
|
| And that is in part due to the shift from having dedicated ops
| teams, to having programmers take on more infrastructure tasks.
| This isn't unique or novel--incorrectly configured buckets,
| committing access keys, poor IAM setup, etc happen so
| frequently due to devs who have no real practical experience
| managing production infrastructure having unchecked access to
| AWS.
| robertlagrant wrote:
| Well, all the infra people did this too, just earlier.
| Tollen wrote:
| I wish there were more of these for artists who specifically need
| GPU rendering for their own animations and heavy duty production.
| It seems like all of this new infrastructure isn't going towards
| "classic" GPU use cases.
| lowlevelprog wrote:
| I have switched to buying GPUs and I already see saved costs as
| compared to cloud renting. Also, networking is fun.
| latchkey wrote:
| > Also, networking is fun.
|
| Let me know how your 400G deployment goes with your vendors VRF
| implementation.
| tehlike wrote:
| Switching to hetzner was one of the good things I did last year!
| iotapi322 wrote:
| Crypto has this covered with 4090's available on clore and
| several other platforms.
| Gbox4 wrote:
| A funny comment from that article:
|
| "On CoreWeave, renting an Nvidia A100 40GB -- one popular choice
| for model training and inferencing -- costs $2.39 per hour, which
| works out to $1,200 per month. On Azure, the same GPU costs $3.40
| per hour, or $2,482 per month; on Google Cloud, it's $3.67 per
| hour, or $2,682 per month."
|
| Am I missing something? I am sure I'm a bit rusty in math, but I
| can still handle a calculator. ~720 hours in a month (roughly),
| and that means CoreWeave would cost $1,720.80 per month, Azure is
| $2,448 per month, and Google Cloud is $2,642.40 per month.
|
| Why are all of these numbers reported in the article off? Some
| slightly--Azure and Google Cloud are close, but CoreWeave is off
| by about 30%. I won't go further into the numbers as to why the
| author came up with these results, but I'm just wondering if this
| article was written by AI, which would explain why basic
| multiplication is incorrect.
| spacebanana7 wrote:
| Could it be a discount for purchasing an entire month's worth
| of capacity? Even if so, such costing plans should be explicit
| in the article
| fancyfredbot wrote:
| The whole thing is nuts. They have the wrong costs multiplied
| by the wrong time period to get the wrong answers.
| https://coreweave.com/gpu-cloud-pricing says an A100 40GB
| NVLink is $2.06 whereas the article says $2.39.
|
| That's $1483.20 a month, whereas the article says $1200 and
| should say $1720 if they'd got the maths right.
| programjames wrote:
| Maybe an LLM helped with the math?
| Onawa wrote:
| I think your guess of AI generation makes sense for the math
| discrepancy.
| Thrymr wrote:
| Wonderful that we have evolved large linear algebra models
| running on expensive computers to the point that they can no
| longer do basic arithmetic correctly.
| jmgao wrote:
| The CoreWeave number is completely wrong, but Azure and Google
| Cloud appear to be exactly correct at 730 hours per month,
| which happens to be the number of hours in 365 / 12 days.
| coffeebeqn wrote:
| Have you ever asked a LLM to calculate costs for you? This is
| it exactly what it looks like
| umeshunni wrote:
| My favorite quote about journalists goes something like "Never
| trust a journalist's math. If they could do math, they wouldn't
| have become journalists."
| zackangelo wrote:
| Would like to take a moment to recommend fly.io for GPU
| workloads.
|
| I've been building a prototype using them for the last couple of
| weeks and it's been great to use. I didn't have to jump through
| any hoops or apply for any quota adjustments to get started. And
| I especially appreciate how easy they make it to automatically
| scale your GPU instances to zero based on traffic.
| nextworddev wrote:
| Taking a step back, was it necessary to roll your own LLM (or
| whatever FM) API as opposed to using an off the shelf API
| indigodaddy wrote:
| Wow that is quite decent pricing actually. This makes me want
| to try to deploy something like llamafile/ollama or similar to
| fly.io+gpu for my personal on-demand llama/llm whims. (@simonw
| I'm looking at you-- I feel like if you haven't already done
| this on fly.io, that you're probably thinking about it :) )
| Seems private enough.. could throw some basic auth on top of it
| for me and trusted friends/family so I don't get crazy bills.
| But privacy-wise with fly.io I think it's good enough.
|
| Only problem might be everytime it spins up to download the
| large model might be wasteful as far as getting charged for
| network/bw usage-- wonder if it would be more cost-efficient to
| have persistent storage or just see how much time and bw it is
| to download on every cold start...
| Palmik wrote:
| It's horrible horrible pricing! Their on demand price for
| A100 is what gets you H100 sxm in other places.
| dangrairo wrote:
| I am curious what "other places" are you comparing it to.
| indigodaddy wrote:
| Also we'd have to do apples to apples right? One cannot
| complain about fly.io gpu pricing and then point to say
| vast.ai or lambda labs as "evidence" of that. They aren't
| at all the same type of service..
| benwaffle wrote:
| We've got a one-liner for spinning up your own ollama UI. See
| https://github.com/fly-apps/ollama-open-webui
| breakingcups wrote:
| Hn's opinion of Fly seems to fluctuate a lot. First it was an
| HN darling, especially with all the high-quality blog posts
| they were putting out, but on the latest threads there's been a
| lot of complaining about platform stability and features.
| paxys wrote:
| It's funny that "We managed to get our hands on a handful of
| H100s/A100s" is an actual (and profitable) tech business model
| right now.
| ptero wrote:
| It seems to be that a large part of the AWS business model is the
| revenue received as a fledgling startup tries to grow (and
| ideally succeeds) into a large company.
|
| As a small tech startup, an easy button for compute needs is
| perfectly sensible, as it allows focusing on the product. If the
| startup grows switching becomes more expensive, so AWS gets its
| money as long as the amounts are not seen as the main cost
| driver. That stage, I think (with no hard data), is the AWS
| sweetspot. The company is paying a lot for AWS, but does not yet
| want to do a full analysis, hire dedicated cloud cost
| optimization staff and deal with friction of switching.
|
| If the startup grows stable and profitable it will likely do a
| proper cost analysis and make AWS bills saner, maybe with a mix
| of on-prem, AWS and non-AWS cloud services. But that requires a
| stable period, both in time and in functionality, which is not
| something that an unprofitable startup has.
|
| I think with the end of ZIRP and tighter access to VC funds the
| number of startups that can afford losing a lot of money to go
| through an explosive growth period will shrink, and so will the
| AWS profits.
| ctocoder wrote:
| https://console.crusoecloud.com/request
|
| They are providing H100s, A100s, L40s for very cheap. They also
| do not charge for the network usage. I Highly recommend them as
| the price per flop is unbeatable anyplace, and they have over
| 4000 gpus to use at a time.
| apitman wrote:
| > Given hyperscaler dominance of the overall public cloud market,
| which demands vast investments in infrastructure and range of
| services that make little or no revenue, challengers like
| CoreWeave have an opportunity to succeed with a focus on premium
| AI services without the burden of hypercaler-level investments
| overall
|
| Interesting. GPU-only providers targeting the AI market only need
| to implement a fraction of the services that AWS does. They don't
| even need to be geographically distributed. What does it matter
| if your GPU cluster is on the other side of the planet?
| ReptileMan wrote:
| To have cheaper GPUs we need at least four companies to design
| them and four companies that are able to produce them. Before
| that you always have cartel like behavior.
| latchkey wrote:
| Maybe on the consumer low end, but on the enterprise level...
| it'll always be expensive.
| rootedbox wrote:
| @ Core-weave an A100 40GB NVLink is $2.06 ... the only way they
| are doing this is by burning investor money. At that price it
| running it 24/7 it will take way over a year to recoup hardware +
| electricity cost.
|
| So my suggestion.. dump all your work @ Core-weave.. It's cheaper
| than buying the hardware yourself let alone the cost of managing
| it.
| jerrygenser wrote:
| Does this assume retail price? Maybe they are getting a
| discount for buying in bulk
| latchkey wrote:
| > the only way they are doing this is by burning investor
| money... it will take way over a year to recoup hardware
|
| They've been running for years already. They can also offer
| these lower end gpus at that price cause the higher end ones
| offset things.
| nextworddev wrote:
| I have good reasons to believe that all this capex into GPUs will
| backfire. Metrics aren't looking good for AI adoption, and growth
| rate is slowing down while inference and training costs are
| plummeting.
| DrNosferatu wrote:
| Besides Alibaba Cloud and AWS, are there other FPGA cloud
| services?
| teitoklien wrote:
| VMAccel is popular for FPGAs like Xillinx High End FPGAs,
| Achronix, etc.
|
| https://www.vmaccel.com/solutions
| maxchehab wrote:
| Both Core Weave & Lambda Labs have fairly predatory pricing
| making it impossible to rent GPUs without a yearly contract.
|
| This doesn't make sense for training models, where a training run
| is on the scale of days & weeks.
|
| I wished that the techcrunch article mentioned other companies,
| like sfcompute, which offer hourly compute instead of yearly
| contracts.
| thundergolfer wrote:
| Lambda Labs has on-demand GPUs. Just put in a credit card and
| you're able to launch. I launched an 1x H100 server just 10
| minutes ago on Lambda Labs.
|
| The price is also $2.49/hr which does not seem predatory at
| all.
| matroid wrote:
| I have never seen 1x H100 available on Lambda Labs. Don't
| know why though.
| williamstein wrote:
| I've been checking about twice a week for the last 6
| months, and they are very rare, but it does happen. I
| caught one on video 2 weeks ago!
| https://youtu.be/NkNx6tx3nu0?t=744
| latchkey wrote:
| I don't think it is predatory, I think it just happened over
| time due to demand. It is a marketplace after all.
|
| https://news.ycombinator.com/item?id=40101377
| ec109685 wrote:
| That would make the TechCrunch article highly misleading since
| AWS and the other clouds offer big savings for reserved
| instances.
| dmyriel wrote:
| Hardware is a vendor lock-in. I can't imagine sinking millions
| into a massive project and getting stuck with a huge bill.
|
| Then, to migrate away you need all sorts of devops folks and the
| ability to deal with incompatibilities.
|
| Uncertainty about pricing and the hardware bottleneck is a real
| problem for our users.
|
| I just raised this point in our blog today.
|
| https://qdrant.tech/blog/are-you-vendor-locked/
| Palmik wrote:
| I really don't get this. Most of these low cost providers
| actually re-rent GPUs from Azure, AWS or GCP (*), yet they offer
| much better on-demand pricing (as low as $3.8/hr for H100 sxm and
| $2.5/hr for PCIe).
|
| And it's a fact that you can get even much better on-demand (not
| to mention reserved) pricing from the big clouds if you're a
| decent startup with connections.
|
| If one of these clouds offered fair pricing to SMBs, it could be
| a great bottoms up growth strategy.
|
| (*) Not LambdaLabs afaik, but they rarely have on demand capacity
| anyway, and you can only get reasonable price with 3 year
| reservation (which is, surprise surprise, more than the hardware
| cost).
| thundergolfer wrote:
| It's the opposite of what you say. Slim to none of the
| alternative cloud providers re-rent their GPUs.
|
| Coreweave, Fluidstack, Lambda Labs, Paperspace, Cudo Compute,
| Hydra, Datacrunch.io, Vultr, Crusoe Cloud, SF Compute.
|
| As far as I can tell none of these providers give you a GPU
| originating in Azure, AWS, or GCP.
| claytonjy wrote:
| The middlemen you're talking about do two things: buy lots of
| reserved compute on the hyperscalers, and then pit the
| hyperscalers against eachother to get better pricing.
|
| If you're reserving thousands of GPUs from the same
| hyperscaler, even if they're the only cloud you run on, you're
| not paying the price shown in the calculator. If you have other
| suppliers, you'll get an even better deal. Then you resell that
| reserved compute as on-demand compute, somewhere between your
| costs and what your customers would pay a hyperscaler directly.
| edgoode wrote:
| This is a trend we noticed early last year, so we started
| building a single console for all these clouds at
| https://shadeform.ai.
|
| It has been amazing to watch this industry explode, and we
| believe it is great for consumers. The same instances on Amazon
| versus these alternative providers are 3x more expensive.
|
| NVIDIA and many hardware providers are leaning into this trend.
| As clouds become more and more vertically integrated, AMD,
| NVIDIA, and others will benefit from spreading their hardware to
| more clouds.
|
| Knowing that these models will not be running in 3 easily
| controlled clouds may also benefit us in the long run as each
| provider will have different levels of comfort with models of
| varying capabilities.
| breakingcups wrote:
| How does Shadeform make money?
| ilaksh wrote:
| If you want a _really_ cheap alternative cloud, look at vast.ai.
| Not sure you can beat their prices, and they have 4090s and 3090s
| if you can use those. Something like RunPod might be second place
| for pricing.
| claytonjy wrote:
| What are folk's experiences with alternative cloud GPUs for
| _inference_?
|
| If you're doing a lot of model training, buying GPUs or long-term
| reservations of GPUs is a no-brainer. But when it comes to
| inference, latency matters and it gets trickier talking between
| e.g. your AWS infra and your GPUs somewhere else.
|
| It seems lots of providers can give you enough to get by doing
| inference in a company's earliest stages. But what if I need
| hundreds or thousands of A100s during peak usage? Is anyone doing
| this successfully with a non-hyperscaler?
| htrp wrote:
| Just go with an inference provider like
| fireworks/together/modal/baseten
| claytonjy wrote:
| An issue we've had when looking into some of these is that,
| they provide a layer of software abstraction we're not
| looking for. I don't want to use some providers bespoke
| library to wrap my model code; I just want to use NVIDIA
| Triton, either by providing an image or by providing a model
| repo. I only want the inference provider to handle the
| hardware.
|
| I understand that's exactly what those provider _don't_ want,
| because it means they can't lock us in. But particularly when
| comparing an inference provider to GCP, where we already run
| everything in Triton on GKE, I don't want to rewrite my code
| just to see what their hardware layer is like.
|
| Another complication is we often run multiple tightly-
| integrated models for a single application, where having them
| on the same GPU is critical. This is tricky or impossible in
| some inference-provider-frameworks.
|
| There's too many options for running the latest LLM, and far
| fewer for running a bespoke set of fine-tuned models on GPUs.
| bzmrgonz wrote:
| saw one touting itself to be the airbnb for gpu's
| latchkey wrote:
| That was just a marketplace to resell compute that others are
| running. A super common practice in the industry.
| la64710 wrote:
| How are people using these GPU clouds with their data residing in
| one of the big cloud providers like AWS or Azure? Are they paying
| for egress to get data onto the GPU clouds?
| htrp wrote:
| Yes.... you eat the egress cost.
|
| Pretty soon, these specialists will build object storage and
| all of the other "costs" that the legacy hyperscalers already
| incur.
| user3939382 wrote:
| I've been using Wasabi for S3-interface cloud storage. Way
| cheaper and works great.
___________________________________________________________________
(page generated 2024-05-06 23:01 UTC)