[HN Gopher] Ahrefs Saved US$400M in 3 Years by Not Going to the ...
___________________________________________________________________
Ahrefs Saved US$400M in 3 Years by Not Going to the Cloud
Author : dgudkov
Score : 92 points
Date : 2023-03-11 15:07 UTC (7 hours ago)
(HTM) web link (tech.ahrefs.com)
(TXT) w3m dump (tech.ahrefs.com)
| margorczynski wrote:
| Once your operation is stable and you know (almost) exactly what
| you need then sticking to the Cloud makes less and less sense.
| The premium they charge is enormous.
|
| Not sure why e.g. Netflix doesn't exit AWS considering their
| needs flatlined. Or maybe they have some special agreement with
| much lower prices than market.
| yegle wrote:
| A 24/7 oncall rotation team needs 5 people to keep it <1
| rotation per month. This alone costs 1M/y.
|
| I think many companies simply get away by having unhealthy
| oncall rotations. E.g. ask people to oncall outside business
| hours constantly.
| cbrozefsky wrote:
| Or their stuff doesn't break such that being oncall is a
| formality.
| pgwhalen wrote:
| This is a great way to put it, and I don't see this reality
| acknowledged on HN very often.
| marcinzm wrote:
| What the cloud gives you is the removal of the inter-department
| conflict between IT and Product Engineering. This is especially
| bad since IT is a cost-center rather than a revenue generator.
| Having a single provider who in turn has a single client is a
| horrible dynamic. It can work for the short term but long term
| you get really odd dynamics. Sure the cloud is expensive but so
| is the aggregate opportunity cost of waiting 3 months and
| spending 20 hours in meetings for a single dev server.
| trollied wrote:
| This quite often comes down to CapEx vs OpEx.
| djfobbz wrote:
| Way to go, I approve! Been running bare metal for my SaaS since
| 2007.
| codingbot3000 wrote:
| It looks clear to me that given the hardware they need, and the
| scale they operate at, buying their own metal makes sense. Also
| them sticking to similar servers is a good idea imho.
|
| What I am missing though in the monthly cost is the personnel
| cost. I would assume that a decently qualified system and network
| administrator in Singapore, capable of operating and servicing
| this setup, will cost upward of $10,000 per month. And only
| having one is probably not enough, due to SPOF and other
| considerations. To be honest, even for the cloud you probably
| need someone to administer the setup, but chances are high that
| there is much less to do.
|
| Paying for, and finding qualified staff is from my experience the
| main driver for small and mid-size companies to go to the cloud.
| And for many this actually makes sense financially. Even though
| many create unnecessary complex setups or get trapped in vendor
| lock-ins.
| hdjjhhvvhga wrote:
| > What I am missing though in the monthly cost is the personnel
| cost.
|
| It's not like operating AWS doesn't involve any personnel cost,
| does it? Even though it was marketed as such, it has become
| clear that managing a public cloud such as AWS is at least as
| complex as managing a datacenter. Sure, you don't swap broken
| drives in RAID arrays but you have other complexities to deal
| with - and that complexity increases year by year.
| [deleted]
| pphysch wrote:
| A small-medium DC can be a lot of work to install, specifically
| in terms of backup power/cooling, but not much to maintain on a
| daily basis.
|
| Occasionally you need to replace expired components, and
| install new systems, but that's about it. There will be
| weeklong periods where no action is required.
|
| Finding qualified staff is definitely hard. Especially
| nowadays, there are a lot of DevOps astronauts that have never
| swapped a bad DIMM.
| indymike wrote:
| I've had great luck just leasing what I need from a company
| who already has the DC and is willing to manage the hardware
| as part of the lease.
| aldarisbm wrote:
| > So we decided to consolidate all the costs spent for this data
| center, divide them by the number of servers, and compare them to
| the cost of a similar installation in the Amazon Web Services
| (AWS) cloud (we use this as the leading IaaS example).
|
| so compare them to buying AWS VMs you don't need? one of the main
| attractions of the cloud is the fact that you don't have to pay
| for what you don't use. Elasticity.
|
| Every time I see one of these is: I bought 100 servers (which i
| use 40 of), I would've saved money if I was renting 100 servers
| of AWS. Yeah ... no shit. If you were renting only 40, and had
| the ability to go down as usage decreases, there would be
| savings, mind you there are such things as, RIs, savings plans,
| etc, etc. meh.
| bhahn wrote:
| I've only worked at companies running in the cloud. Is there a
| console (open-source or otherwise) and other tooling similar to
| AWS' for managing your resources in your own data center?
| ocdtrekkie wrote:
| Plenty of them, yes. I probably would decline to move to a job
| with a cloud-based setup. It's short-term thinking at it's
| finest.
| mdaniel wrote:
| The OpenStack console is pretty good, imho
|
| If you have an 8GB vm you can see it in action via
| https://docs.openstack.org/devstack/zed/guides/single-vm.htm...
| (takes about 45 minutes) or if you want the "full" experience
| fire up an education cluster from https://openmetal.io/use-
| cases/on-demand-openstack-cloud/ (it takes about 4 hours or so
| but it's also a "real" setup unlike the vm which for sure
| allows running instances via qemu but isn't going to allow
| exercising the complete suite as easily)
| Nextgrid wrote:
| Duplicate: https://news.ycombinator.com/item?id=35094407
| karmicthreat wrote:
| I can definitely see advantages for very specific use cases in
| favor of bare metal colo. But AWS does provide a good amount of
| value in flexibility and reliability. If you have a whole lot of
| bandwidth or storage needs you should probably look at a hybrid
| system. But you will probably pay for it with increased
| engineering and maintenance costs.
| Xeoncross wrote:
| Another interesting thing is that the farther you go from the CPU
| cache, the more copies of data and the more extra processing you
| have to do.
|
| Think of how many copies of the data you need (and how much
| processing it takes) to use:
|
| - a L3 CPU cache.
|
| - a 16GB RAM cache.
|
| - an embedded C/Rust/Go SQLite database.
|
| - a distributed, fault tolerant Cassandra/elastic cache cluster.
|
| You can literally solve the same typeahead/autocomplete/user
| lookup/session management/etc.. features using the last three of
| these - but the CPU cycle, memory usage and latency costs (plus
| complexity) explodes.
|
| I'm always impressed by people that handle 1 million connections
| on a single box while I'm watching teams trying to handle a
| thousand connections with a whole fleet of infrastructure and
| caching layers.
|
| Then again, part of the issue is the wrong language and wrong
| designs patterns. Highly optimized Rust and Go someone obsessed
| on all weekend is going to shred your team's unoptimized vanilla
| Ruby or Java install you didn't even have time to write tests for
| and don't honestly care about because you're too worried about
| being laid off.
| hot_gril wrote:
| For use cases with simple business logic and high throughput,
| yeah the optimized Rust, maybe on your own hardware, makes
| sense.
|
| Tons of cloud customers running NodeJS or Java stuff don't fit
| this. Maybe they have a few optimized pieces with high
| throughput and majority unoptimized where resource usage is low
| either way. Or they might be ok spending 10X compute resources
| even on something high-throughput because landing new
| features/products matters way more than optimizing a small part
| of their budget. Like, there are businesses renting big
| 8-socket servers with TBs of RAM to support some monstrous
| corporate DB that can only run on a single machine.
| Xeoncross wrote:
| Yeah its kind of sad that we still have to use node and java
| with better options out there. Don't get me wrong,
| Typescript, Deno, JVM, etc.. are all doing their best to
| right those ships, but it's would still be better to just
| scrap that stuff for a better foundation.
|
| That is what everyone knows,
|
| so that is what's taught,
|
| so that is what everyone knows,
|
| so that is what's taught.
|
| I'd gladly onboard projects to Erlang, Rust, Zig, Go, or
| anything modern. Would save our planet so much wasted energy.
| moomoo11 wrote:
| The problem is that our industry democratized to the point
| that people want to hire the cheapest resources who can
| ship something. A mediocre node or ruby developer costs
| peanuts to hire compared to one who is mediocre at Go or
| Erlang. Btw.. If you hire a mediocre go or Erlang developer
| you're fucked.
|
| For IO bound work there's not much difference between a
| service built with Go or Node. It's really when you need
| services that have soft/hard real-time requirements or
| continuous processing requirements that imo it's necessary
| to hire the right people for the right stuff.
|
| For anyone else who is making a web platform that doesn't
| have much complex backend processing, they can just use
| ruby or node.
|
| Personally I use Go for all backend tasks and node for any
| one off scripting work. Go spits out a binary I can run on
| any Linux based system. Node scripts are easy to write and
| I use them for fixing issues or for simple single
| responsibility tasks.
| invalidname wrote:
| JVM can do 1M concurrent on a single machine. Don't lump it
| with Ruby please. This will be even easier with Loom as we
| won't need asynchronous APIs.
| mdaniel wrote:
| Damn, how many times is this article going to be submitted?
| https://news.ycombinator.com/from?site=ahrefs.com
|
| https://news.ycombinator.com/item?id=35108429
|
| https://news.ycombinator.com/item?id=35100866
|
| https://news.ycombinator.com/item?id=35095604
|
| https://news.ycombinator.com/item?id=35094407
| elijaht wrote:
| Yea, I thought that was weird. Was wondering why there weren't
| many comments when it had been on the front page for so long
| Atlas22 wrote:
| Just gotta wait for the comment claiming admin/maintenance
| costs makes non-cloud more expensive than cloud then we can
| have yet another full repeat discussion.
| ramesh31 wrote:
| This fits Ahrefs because computation and data is their secret
| sauce, not software. For the average SaaS this isn't the case.
| twelve40 wrote:
| great point, i didn't even realize how much compute power they
| need (and how much money they make)
| PaywallBuster wrote:
| right!
|
| Most services are probably well served by AWS/Heroku and others
| and simply saving the engineering time on developing the
| product, or optimizing when it becomes a problem
|
| AWS can be 2x as expensive as DO for a simple VM, which you can
| easily work around by acquiring Reserve Instances or using
| "spot" instances
|
| but, the "AWS premium" becomes a problem: - 10x when dealing
| with bandwidth which you cannot do much except a very big
| enterprise package and getting some discount that even then it
| will be multiple times higher than the alternatives
|
| The way I see it, AWS is great but you need to consider your 1)
| key infra requirements 2) consider cost effectiveness of
| different solutions/services 3) consider alternatives i.e.
|
| - Cloudfront is fine, but if you doing 100 tb/month probably
| Cloudflare would be best! - Lambda is fine but if you doing
| 100M requests/month EC2/ECS would probably work better (or more
| cost effective)
|
| Consider 1) what you need to get started and ship 2) what you
| need to scale up 3) what's the most cost effective solution
| long term
|
| The requirements will change over time, be ready to adapt!
| marcopicentini wrote:
| Me too! Running a B2B SaaS using Digital Ocean and Cloud66.
|
| I am free to switch vendor every 12 months without pain.
| intev wrote:
| How has the experience been with Cloud66? Why them over using
| something like caprover?
___________________________________________________________________
(page generated 2023-03-11 23:01 UTC)