[HN Gopher] Cloud Egress Costs
___________________________________________________________________
Cloud Egress Costs
Author : amzans
Score : 68 points
Date : 2024-02-10 09:55 UTC (13 hours ago)
(HTM) web link (getdeploying.com)
(TXT) w3m dump (getdeploying.com)
| naiv wrote:
| I think the overview is missing a column of what data is actually
| allowed for 'free' egress.
|
| Iirc eg Cloudflare does not allow videos etc. as part of their
| free tier.
|
| edit: confused ingress and egress
| kkielhofner wrote:
| We're talking about egress - traffic leaving the network. Due
| to traffic patterns and other reasons (Hotel California)
| ingress is always free, egress is charged.
|
| There is some debate about the status of hosting video when
| using Cloudflare as a CDN.
|
| These comparisons are for hosts/object store.
|
| Cloudflare R2 has no restrictions on the type of content while
| not charging for egress.
| naiv wrote:
| sorry , I typed too fast and meant egress
| asmor wrote:
| Hetzner has seen two known cases of people being evicted for
| bandwidth use, and they were saturating the link 24/7 for
| several months by syncing an endless amount of footage to
| archive.
| dijit wrote:
| For the google cloud one, is that the premium networking option
| or the standard networking?
|
| Those are very different, the standard network is more akin to
| AWSs egress as GCPs Premium option (the default) uses local PoPs
| and dark fibre over Googles backbone to origin.
| bcaxis wrote:
| Nice chart
|
| Some vendors let you pool your egress that comes with the compute
| you buy. Others do not. That can impact costs quite a bit.
| rezonant wrote:
| Right, Linode works this way for one.
| jupp0r wrote:
| I'm confused by that website using $/mo as the unit to describe
| the cost of 1TB egress. How does the time component play into
| this? I would have expected the cost to be described in $/TB.
| mocamoca wrote:
| I got confused too at first.
|
| But it makes sense because the counters get reset each month.
|
| So if you consume "101TB" over a year, distributed as 90TB in
| January and then 1TB per month... You could get hit pretty hard
| in January and have free bandwidth for the rest of the year.
| amzans wrote:
| Thanks, that makes sense. I'll update this so it's clearer.
| mholt wrote:
| This website is missing Wasabi [0], which has free egress. (Not
| necessarily endorsing, just noticed.)
|
| [0]: https://wasabi.com
| asmor wrote:
| heavily recommending against wasabi, they increased prices
| recently and announced this via their newsletter, the one with
| an unsubscribe button, and then blamed me for unsubscribing
| (and never refunded the charge). highly unprofessional and a
| lack of understanding of the GDPR for someone who is supposed
| to hold all your data.
| ijhuygft776 wrote:
| Do a credit card charge back and unsubscribe from the
| service...
| naiv wrote:
| They have free egress up to your data storage usage.
|
| So if you use 1TB storage but 2TB egress , it will not be free
| anymore.
| mholt wrote:
| Right; Backblaze has similar pricing as you can see on the
| table.
| PreInternet01 wrote:
| The cost of egress traffic is a _very_ good reason for many
| organizations to not fully migrate to a cloud provider anytime
| soon. And since, unlike with storage costs, there doesn 't seem
| to be an actual _reason_ (other than: it makes migrating to
| competitors cost-prohibitive in a subset of cases), that seems
| kind of... weird?
|
| Small example: an actual company I do some work for is in the
| business of delivering creative assets to distributors. This
| results in an egress of around 180TB per month, which is, on
| average just, around 500Mb/s.
|
| So, this company currently operates 2 racks in commercial data
| centers, linked via 10Gb/s Ethernet-over-DWDM, with 2x512Mb/s and
| 1x1Gb/s Internet uplinks per DC. Each rack has 2 generic-OEM
| servers with ~64 AMD Zen cores, 1/2TB RAM, ~8TB NVMe and ~100TB
| SAS RAID6 storage per node.
|
| Just the cost-savings over egress on AWS is enough to justify
| that setup, including the cost of an engineer to keep it all up
| and running (even though the effort required for that turns out
| to be minimal).
|
| So, are cloud providers ignoring a significant market here, or is
| the markup on their current customers lucrative enough?
| jonatron wrote:
| The AWS Enterprise Discount Program apparently requires $1M per
| year spend. 180TB is about $13k on AWS so presumably not enough
| to be interesting to them. Hopefully someone who works at AWS
| can share some info.
| tky wrote:
| EDP can be great if you can meet their required year over
| year growth requirement and if your spend is high enough to
| make the discounts (which are variable and negotiated often
| at a product offering level, not blanket) offset the required
| top-shelf support contract. For smaller orgs even at the
| $1-2M/mo level, it can often be a risk not worth taking vs
| other cost savings mechanisms and locating high-egress use
| cases elsewhere.
|
| Egress bandwidth pricing has been the sacred cow money
| firehose forever, despite transit costs continuing to shrink.
| asmor wrote:
| It really depends on the quality of the peering you expect. It
| doesn't matter, until it does. Consumer ISPs sometimes do their
| utmost to not peer with open exchanges, and the entire thing
| gets even more complex when you go to places where bandwidth is
| more expensive (i.e. Oceania).
|
| There's a reason the favorite chart to exemplify value
| Cloudflare reps like to show is Argo Smart Routing, and why it
| costs about $100 per TB just like AWS and GCP.
| naiv wrote:
| We are using Argo for our xhr search traffic as it makes more
| sense than setting up different servers/vms in parts of the
| world. Each request is only 1kb max.
|
| But I would not use it for static assets. For this we use
| Bunny edge storage to provide faster response times at very
| reasonable prices.
| Fripplebubby wrote:
| I agree, and I would also put forward that most people don't
| understand what peering is or how it works. When people
| (usually developers who are not network engineers and have
| not worked at that level of the stack) talk about "egress",
| they mean delivering bits from your network (cloud or
| otherwise) to any other network on the internet. How can you
| put just one price on delivering a bit either to a host
| within the same datacenter or one on the opposite side of the
| planet? Physics still mean that one is more expensive than
| the other.
|
| The existence of the world wide web has tricked us into
| thinking that sending traffic anywhere is/should be the same,
| but of course it is not. So while the price you (a cloud
| customer) pay for egress pricing is (often) indiscriminate on
| where that traffic is going, using common sense, we can
| understand that some traffic is more expensive than others,
| and the price we pay is a blended price with that aspect
| "baked in" or "priced in".
| toast0 wrote:
| My feeling is that egress is easily measured, so it's where
| costs that are hard to assess get moved to.
|
| It doesn't feel great to be line item billed for stuff at 10x
| the rate of credible other offers.
|
| I think there is also some geo-specific pricing that gets
| hidden in a global price; bandwidth can be a lot more expensive
| in some locations than others and if you are charged 5x for
| egress in south america, nobody will use the south america
| locations and that's not good for business.
| raid2000 wrote:
| Right. Egress is an imperfect, but reasonable metric for
| overall utilization. If they started charging for CPU hertz
| above a certain threshold, that'd be a harder sell.
| amluto wrote:
| I don't believe this. Operating an internal cloud network is
| _expensive_ , but it's expensive because of internal traffic,
| and they don't charge for that internal traffic. Egress is
| just like traffic to any other system, and AWS doesn't charge
| for that.
|
| Also:
|
| > It doesn't feel great to be line item billed for stuff at
| 10x the rate of credible other offers.
|
| It's quite a bit worse than 10x
| thimp wrote:
| I saw a hilarious fuck up a few months ago. Company sets up an
| AWS hosted always on VPN solution. Connects 1000 staff through
| it. Celebrates how they saved $50k on the VPN solution. Gets
| $25k AWS bill for the just the first month of egress traffic.
| Turns out the data was leaving AWS egress three separate times.
| ailurooo wrote:
| how were they spending that 50k previously? a bespoke saas
| thing or self hosting?
| ozr wrote:
| Definitely lucrative enough. The use case you've described
| isn't particularly uncommon, but lots of companies just pay for
| the egress.
|
| The problem is that there are now multiple generations of
| software engineers that do not know how bandwidth is priced.
| They've only used managed providers that charge per unit of
| ingress/egress, at some fractional dollar per GB.
| api wrote:
| I've had people refuse to believe that bandwidth is actually
| very cheap and cloud markup is insane (hundreds or even
| thousands of times cost).
|
| I show them bare metal providers and colo that bills by size
| of pipe rather than transfer. They refuse to believe it or
| assume there must be a catch. There usually isn't, though
| sometimes the very cheapest skimp on things like rich peering
| and can be slightly slower or less reliable. But still
| cheapest is relative here. Expensive bare metal or colo
| bandwidth is still usually hundreds of times less than big
| three cloud egress.
|
| It's just nuts.
|
| It's a subset of a wider problem of multiple generations of
| developers being fully brainwashed by "cloud native" in lots
| of ways. What an amazing racket this all has been for
| providers...
| amluto wrote:
| > other than: it makes migrating to competitors cost-
| prohibitive in a subset of cases
|
| My theory: it forces third party services into the same cloud.
|
| Suppose you use AWS and you want to pay a third party SaaS
| provider for some service involving moderate-to-large amounts
| of data. Here's one of many examples:
|
| https://www.snowflake.com/en/data-cloud/pricing-options/
|
| And look at this remarkable choice: you get to pick AWS, Azure,
| or GCP! Snowflake is paying a _lot_ of money to host on those
| clouds, and they're passing those costs on to customers.
|
| Snowflake is big. They have lots of engineers. They are
| obviously cloud-agnostic: they already support three clouds. It
| would surely be _much_ cheaper to operate a physical facility,
| and they could plausibly offer better performance (because NVMe
| is amazing), and they could split the cost savings with
| customers. But they don't, and my theory is that egress from
| customers to Snowflake would negate any cost savings, and the
| _variable_ nature of the costs would scare away customers.
|
| So my theory is that the ways that customers _avoid_ egress
| fees makes the major clouds a lot of money. IMO regulators
| should take a very careful look at this, but it's an excellent
| business decision on the parts of the clouds.
| Eridrus wrote:
| Snowflake's margins are like 90%+ on top of the compute they
| sell, and they pass on all these costs including egress
| directly to customers.
| amluto wrote:
| Their list price for storage capacity is only on the order
| of 2x what S3 charges, and Snowflake and S3 likely both
| offer discounts. Comparing compute costs is harder.
|
| If I were running a service like Snowflake, I would
| certainly appreciate the effortless scaling that the major
| clouds offer. But I also know what I pay for actual servers
| and roughly what I would pay in a major cloud, and I much
| prefer the former.
| Eridrus wrote:
| Yeah, storage is much more competitive, but there have
| been leaks on what hardware they run on, their margins
| are incredible, which makes sense since they are selling
| software, not hardware!
|
| Anyway, there is an explicit egress charge:
| https://docs.snowflake.com/en/user-guide/cost-
| understanding-...
| dalyons wrote:
| tbf, snowflakes whole thing & what makes them "next gen"
| vs a traditional datawarehouse is that you dont may much
| for data at rest. You pay as little as possible for the
| "warehousing" part, and pay mostly/only for
| compute/querying.
| api wrote:
| "Let's get rid of IT and stop using on prem software.
| Outsource it all to the cloud and SaaS. We'll save so much
| money!" ... said the entire world pretty much ...
|
| This is hilarious.
|
| I told you so. So did loads of others. Oh well.
| kansi wrote:
| Are you able to share which provider do they use for their rack
| setup?
| xyst wrote:
| just 1 engineer? lol. Let's hope it's not business critical for
| the servers to be up and running at all times if you only need
| 1 engineer.
| amenghra wrote:
| _some cloud providers offer a certain amount of free egress each
| month, while others charge a flat fee for egress over a certain
| amount_ I parse this sentance as saying the same thing twice.
| mocamoca wrote:
| OVHCloud is free https://us.ovhcloud.com/public-cloud/faq/
|
| As a data company we really benefit from Scaleway, Hetzner and
| Ovh.
|
| However, Scale way has no plans to add an US infra
|
| And I don't know if Hetzner has US presence?
| reaperman wrote:
| Hetzner has a limited US presence for VM products but not
| baremetal/dedicated solutions.
| raid2000 wrote:
| European providers benefit from lower cross-connect fees in
| datacenters and more internet exchanges for easy peering. It's
| not surprising they offer more bandwidth at the same cost.
| teamspirit wrote:
| Isn't this what makes cloudflare's R2 a great proposition? No
| egress costs with very competitive storage and operation pricing.
|
| It's not as mature as the others, I don't believe you can use
| your own encryption key for example, but why would anyone
| starting out not choose R2? Hopefully they push the market in
| their direction.
| shdh wrote:
| R2 works great from my experience, no egress costs are a huge
| sell.
| l5870uoo9y wrote:
| I offloaded all heavy files (videos and GIFs) to R2 and serve
| them on my web app. I reckon I save at least $240/yearly
| (12*$20) compared to serving them through Vercel. I didn't
| expect mere static files being such a cost factor.
| rezonant wrote:
| R2 is an insane value proposition. We calculated the cost of
| moving HLS video delivery to it and unless you are delivering a
| huge amount of traffic, the costs are far lower. Highly
| recommend you consider it, assuming you are willing/able to
| handle appropriate transcoding yourself (if not, Cloudflare has
| it's Stream product)
| deskamess wrote:
| What is 'appropriate transcoding'?
| shdh wrote:
| CloudFlare is great
| ijhuygft776 wrote:
| not in this universe
| geor9e wrote:
| I had no idea it was this crazy expensive. I can host a 1 TB
| folder of movies on my Google Fiber internet Synology NAS
| website, and let my friend download it for free. But if I hosted
| my website on Google Cloud, they bill me $111.60? How are these
| cloud services getting away with this pricing?
| jeffbee wrote:
| Your little NAS is not directly connected to hundreds of metro
| areas around the globe. Even on Google Fiber, which is an
| excellent network, you only have a few dozen points of
| exchange, all in America. As a Google Cloud customer your
| egress traffic will be on Google's network all the way to the
| very edges of the world, at hundreds of interconnect points and
| exchanges. Your traffic will be on private networks until it is
| with a few miles of your end user, in all likelihood. This is a
| comparison between apples and diamonds.
| veeti wrote:
| Except they will charge you the same extortionate egress fees
| for outbound traffic from a single compute instance ("an
| apple"), no global content delivery network involved ("a
| diamond").
| wrs wrote:
| [delayed]
| dehrmann wrote:
| > let my friend download it for free
|
| Except you paid the cost for the fiber connection. You also get
| one more 9 of reliability from their DC.
| cj wrote:
| > Cloudflare -- Free
|
| Perhaps on self-serve plans with insignificant traffic. On
| enterprise plans they do set bandwidth caps. Our company exceeded
| our bandwidth cap and our account rep reached out to reprice our
| contract.
|
| I asked him if turning off Argo (a service billed by bandwidth)
| would remove the bandwidth overage and they said no.
|
| TlDR: Cloudflare bandwidth isn't free at the enterprise tier.
| rezonant wrote:
| Cloudflare's enterprise offerings are bizarre. We went through
| the process to get it priced out and it made no sense for us. I
| could see how some of the features would be worth it for some
| businesses, but not for us.
| naiv wrote:
| I assume they internally have different quality levels of
| traffic so Enterprise would get better (and more expensive)
| routing?
| tootie wrote:
| Their cap is very high though. 1TB/mo would cover a lot of
| small to mid-size businesses depending on their use case.
| thepaulmcbride wrote:
| Offt, not a good look for Railway, Render or Vercel!
| paulgb wrote:
| I'm a fan of all three, but my mindset is that I'm not just
| paying for raw infrastructure, I'm paying for the engineering
| work I don't have to do when I use them.
| charcircuit wrote:
| AWS's egress needs to decrease by 100x before it will make sense
| for me to use them.
| jintrosartro wrote:
| Isn't GCP egress also free by now?
| https://cloud.google.com/blog/products/networking/eliminatin...
| clhodapp wrote:
| That's probably how they wanted you to read it, but in fact
| it's only free if you are transferring it out and shutting down
| your account
| lolinder wrote:
| That seems to be specifically about the transfer fees that kick
| in when you're leaving GCP. It looks like there's an
| application process to get the free egress for moving off the
| cloud, so they haven't just done away with egress costs
| altogether.
|
| This article is about egress costs for just operating a service
| that talks to the outside world, not just when migrating data
| to a new service.
| web3-is-a-scam wrote:
| AWS egress is one of if not the biggest costs on our bill, it's
| downright criminal.
| njitbew wrote:
| Oracle Cloud only charges a fraction of want Google, Microsoft,
| and Amazon charge. Any idea how Oracle is able to keep the cost
| so low? Or are the others just inflating the price so customers
| don't move to the competitor? In that case Oracle deserves a
| shout out for not applying these vendor lock-in practices.
| andersa wrote:
| The large providers are overcharging between 10 and 100x on
| egress. Cloudflare has a blog on it somewhere.
| vlovich123 wrote:
| Egress prices have insane levels of markup.
| suyash wrote:
| Other are just charging what they are because people are too
| stupid to complain and will just pay.
| Scubabear68 wrote:
| The article says " Cloud providers charge for egress because it
| costs them money to send data out of their network. They have to
| pay for the infrastructure and bandwidth required to send data to
| users".
|
| The charge is not based on cost in the case of the big names.
| They charge an arm and a leg because they want to keep you and
| your data on their platform. When you move it you are breaking
| free.
|
| Hence the high costs to deter this behavior.
| zokier wrote:
| [delayed]
| cheema33 wrote:
| We have used OVH for years. We were on MS Azure before that.
| Getting nickled and dimed for every little thing drove us away.
| Zero regrets.
___________________________________________________________________
(page generated 2024-02-10 23:00 UTC)