[HN Gopher] Switching to AWS Graviton slashed our infrastructure...
___________________________________________________________________
Switching to AWS Graviton slashed our infrastructure bill
Author : Chris86
Score : 96 points
Date : 2022-11-30 20:01 UTC (2 hours ago)
(HTM) web link (squeaky.ai)
(TXT) w3m dump (squeaky.ai)
| Agingcoder wrote:
| I'm not sure I understand the point of this article : in theory
| they don't depend on x86 only code, so they've switched to arm
| and it worked, as expected, and things are cheaper.
|
| I'm happy that they've shrunk their bill, but I somehow expected
| some kind of 'unfortunately, things went wrong because of bizarre
| memory model issues causing difficult concurrency bugs'.
|
| What am I missing?
| MuffinFlavored wrote:
| > in theory they don't depend on x86 only code, so they've
| switched to arm and it worked, as expected, and things are
| cheaper.
|
| What's Intel's response to this as a company? I know that isn't
| mentioned in the article but... just curious
|
| Does Intel have any ARM offering whatsoever?
|
| Does AMD have any ARM offering?
| sofixa wrote:
| > What's Intel's response to this as a company?
|
| They've lowered prices on their CPUs, and they have come out
| with so-called BIG.little designs (small eco low powered CPU
| cores for mundane stuff, and higher power ones for heavy
| lifting, in the same package with automatic switching -
| phones with ARM have had this for years, but it's new for x86
| desktops/laptops).
|
| Intel don't make ARM based chips, while AMD (who generally do
| more diverse things like special designs for consoles) have
| indicated they might. I'm not sure what competitive advantage
| they'd have, but Qualcomm need more ARM competition so anyone
| is welcome. The big blocker is TSMC capacity though.
| moloch-hai wrote:
| You can get those bugs when you are doing your own atomics, and
| your code relies on x86's relaxed memory semantics. It looks
| like their code is JS and Go, which buries that stuff. Services
| they use were already proven out on ARM. (Or, maybe, are not on
| ARM?)
|
| Relaxed memory bus semantics imposes a pretty substantial
| performance cost. Depending on how they are billed, this
| _might_ account for a big chunk of their lower cost. But
| probably not.
|
| Their real problem is that they are firmly entrenched in
| proprietary Amazon services, so switching to another cloud
| would be very difficult. Amazon can raise prices 35% anytime,
| and what can they do?
| Terretta wrote:
| The marriage of an all arm64 dev env and a Graviton op env is a
| match made in heaven.
|
| Everyone please do this so we collectively fix all the things to
| work with this. :-)
| StillBored wrote:
| I upvoted to compensate for the downvotes, but I really curious
| why you think its a match made in heaven vs just using a random
| x86 laptop and an x86 cloud instance?
|
| My take, is that you want to cheer Arm on in this space, not
| because there is some huge technical advantage, or the arch
| meets your fancy or whatever, but because it is adding another
| competitor to the space. One that brings its own baggage, but
| having three+ competitors competing to be the best
| (intel/amd/various arm vendors) is a good thing for the
| industry.
|
| But the blank fanboyism is just harmful, the mid 2010's with
| all the Intel fanboys talking up Intel, while they screw
| everyone with low clocked server processors limited to 2 cores
| copy pasted in their laptops when you could buy a freeking
| phone with 8 cores, is what you get when one company gets to
| much marketshare or too far ahead of everyone else. The same
| thing is going to happen if gravaton becomes the dominate
| platform, except its going to be a case that you won't be able
| to buy competitive onprem hardware, or any number of other
| "shortcommings". Or like the Mac a piece of hardware which
| isn't technically locked down, but also will likely always have
| subpar support running any operating system not shipped by
| apple, and could be locked down tomorrow without affecting
| their business one bit.
|
| So, careful what you wish for. You want competitors that show
| the giant monopoly that maybe designing a processor for an
| actual laptop is avantagious over shoveling whatever leftovers
| from the hyperscalers happen to exist. You also want
| competitors that show up and pack 2x the cores at 1/2 the
| price. Or competitors that show up with huge power hungry
| processors that are pushing the limits of single threaded, high
| Ghz processors, or 500W GPUs because that is what some people
| need/want.
| _joel wrote:
| It is unless you're the first poor soul to embark on the
| journey with lots of x86_64 buildup. Having said that, it's
| been fun though so far. Managed to migrate our dev local k8s
| toolchain and been using buildx to make multiarch images and
| manifests for our internal stuff.
| fred_is_fred wrote:
| We've moved everything to graviton except EKS (MSK, RDS,
| etc). Did you have any major issues? Would you do it again?
| _joel wrote:
| We've only just started! Hence me hitting all the fun
| issues with our 'organically' grown dev toolchain :)
|
| It's been fairly simple so far with a few niggles. We won't
| be able to go full arm due to having to support 'stuff' but
| for the management planes and prometheus etc it's all hunky
| dory.
| fxtentacle wrote:
| FYI https://squeaky.ai/legal/gdpr I believe using Amazon AWS
| already disqualifies you from being fully GDPR-compliant. Same
| issue as Shopify has with using US CDNs: https://lsww.de/shopify-
| illegal/
|
| Since your core sales feature is "privacy friendly" which will
| surely be appreciated in the EU, it might make sense to offer
| local hosting or self-hosting.
| mbesto wrote:
| > I believe using Amazon AWS already disqualifies you from
| being fully GDPR-compliant.
|
| AFRIK - There is nowhere in GDPR that says your data ought to
| reside in EU server per GDPR.
|
| However, if I understand the Shopify legality complaint it's
| saying "because your data is hosted by a US entity and
| theoretically could be accessed by the US authorities it means
| the US authorities are now part of the data custody and you
| can't guarantee that they also have that data". That's a legal
| grey area with a lot of political ramifications.
|
| According to Shopify this doesn't make it illegal:
| https://www.shopify.com/de/blog/shopify-dsgvo-konform-deutsc...
| fxtentacle wrote:
| Yeah, according to Shopify.
|
| According to a German court, a US parent company being able
| to access your data - which is the case both for Shopify and
| here - automatically disqualifies you from being GDPR-
| compliant: https://gdprhub.eu/index.php?title=VK_Baden-W%C3%B
| Crttemberg...
| mbesto wrote:
| > which is the case both for Shopify and here -
| automatically disqualifies you from being GDPR-compliant:
|
| It doesn't _automatically_ disqualify you. No reason to
| spread this FUD. From your article:
|
| > _The Chamber found that, contrary to what Company A
| stated in their offer, it did disclose customer data to a
| third party. More specifically, it disclosed customer data
| to a third party in a third country (its parent company in
| the U.S.). Therefore, a transfer pursuant to Article 44
| GDPR would take place. The Chamber explained that a
| transfer in this context must also be assumed when data can
| be accessed from a third country, regardless of whether
| this actually takes place. The fact that the physical
| location of the server that provided such access was
| located in the EU was irrelevant._
|
| This has to do with the transfer of data from the EEA
| region to the US, which AWS covers:
| https://aws.amazon.com/compliance/gdpr-center/#GDPR_FAQs
|
| So, no, from a blanket perspective using AWS doesn't
| automatically disqualify you from GDPR, but it may have
| implications based on how you transfer the data.
|
| EDIT: To add, as part of Article 44 of GDPR:
|
| > _Under Article 44 GDPR, the transfer (or the onward
| transfer) shall only take place "subject to the other
| provisions of this Regulation". As a result, data
| controllers or processors exporting personal data to third
| countries or international organisations must ensure the
| GDPR compliance of the overall processing activity._
|
| So, if AWS follows GDPR compliance in the US (which as a
| default AWS US does) and you transfer from EU to US, then
| you can still achieve GDPR compliance. The reason this was
| thrown out: https://gdprhub.eu/index.php?title=VK_Baden-W%C
| 3%BCrttemberg... was because the company said "that it
| would not disclose customer data to any third party", but
| when they reviewed the case they found out that because a
| parent company. So they are not GDPR compliant because they
| failed to disclose that data would transfer to AWS US, NOT
| that they are using AWS. This is the discrepancy. Lesson
| learned here - GPDR is more about process control than it
| is technology.
|
| FYI - that case appears to be a public trading company that
| is owned by the public serving the government. It seems
| clear to me that they wanted to send a message of "hey just
| don't use any US-based company for your services to German
| government services, here's how we're legally going to
| penalize you for it". This would be like having an
| "American first" policy for gov't procurement and making
| sure TenCent's US based wholly owned entity can't do
| business with the US gov't.
| fuzzyengineer wrote:
| We have a very similar story at my org. We run around 100 RDS
| aurora clusters and switched to graviton. I'm surprised to see
| 35% gains here, we saw more like 10-15%. But since amazon
| natively supports mysql on aurora we didn't have to worry about
| compatibility. Our main highlight was the way we wrote our infra
| as code where we made switching instances types or service we use
| fairly simple task, so we have switched instance types a couple
| of times in past and could easily make dev use t3s. Getting on
| cloud is a trap and not the usual we deploy on the servers and we
| live situation. Give weight to write some good code to manage
| your infra and able to adopt optimizations as they occur. It will
| ramp up in expense soon otherwise.
| matt-p wrote:
| Imagine how much they'd save by not using AWS in the first place
| makestuff wrote:
| I have only ever used AWS, what is the go to cloud provider
| these days? I know GCP and Azure are catching up, but are
| people just going back to renting some boxes in a data center
| and just hosting their stuff on there?
| andrewxdiamond wrote:
| Nothing, because they would have bought legacy servers and
| would be stuck with Intel for another decade
| matt-p wrote:
| Aws is not a sensible place for general compute even if arm
| where 10x as performant as the intel equivalent and you were
| using reserved instances.
| fooyc wrote:
| There is a world between using AWS and buying your own
| servers
| ecliptik wrote:
| I'm interested in hearing more about their switching to Graviton
| with Clickhouse.
|
| We've been testing Clickhouse on Graviton and the performance
| isn't there due to a variety of reasons, most notably it seems
| because Clickhouse for arm64 is cross-complied and JIT isn't
| enabled like it is for amd64[1].
|
| 1. https://fosstodon.org/@manish/109397948927679076
| [deleted]
| c7DJTLrn wrote:
| Cloud cost optimisation is underrated. In the companies I've
| worked in nobody has really given a shit (at least not under
| normal economic circumstances). In the industry there's a strong
| avoidance of ARM compute instances for no good reason. If I were
| building from scratch today I would definitely go with Graviton.
| adrr wrote:
| Switching to Gravitron isn't an automatic cost savings.
| Everything is optimized for x86. It maybe cheaper, but
| significantly slower. We've been trying to migrate for the last
| year for both cost saving but also we switched to ARM based
| laptops.
| hdjjhhvvhga wrote:
| If I started to build today, I'd definitely go for Hetzner
| Cloud. There is zero possibility that I get surprised by a
| large bill.
| jack_pp wrote:
| I feel it's currently in beta, I've tried it and apparently I
| can't create more than a few instances because my account is
| "too new", without a clear way to remove that limit so you're
| right, can't have a large bill if you can't even create 10
| instances.
| capableweb wrote:
| Did you try writing them and asking them to increase the
| limit?
|
| No cloud provider will give you the option to create as
| many instances as there are available ones, they all have
| limits from the get-go. Usually you have to write them/fill
| out some form if you want to go above the standard limit,
| Hetzner Cloud as well.
| jack_pp wrote:
| I haven't, mainly because of this warning on their Limits
| page:
|
| > Your account is too new to request a limit increase.
| Please note that we generally do not answer questions
| regarding limit increase on the telephone.
| Reitet00 wrote:
| Too bad Hetzner doesn't have cheaper Arm servers available.
| Their Ampere pricing is not really convincing.
| jiggawatts wrote:
| At $dayjob I found an unused box in the cloud running an
| expensive database engine. It was idle for months, created to
| be used by a consultant on a project that had wound up. The
| consultant had quit his consultancy on top of this.
|
| I was told under no uncertain terms not to even think of
| touching this VM because "the budget has been approved".
|
| I was shocked at the flagrant waste of money and assumed it was
| a one-off aberration.
|
| Nope, for months afterwards I kept hearing the same refrain
| from manager after manager, from product owners and dev team
| leads.
|
| "Don't touch! We fought hard for this budget! You'll take it
| from our cold dead hands!"
|
| Eventually I soured on the whole idea of cloud cost
| optimisation a service for unmotivated third parties and gave
| up on the whole notion.
| yazaddaruvala wrote:
| FWIW, in these situations you're better off proposing:
|
| "I'm going to reuse this VM, to help our ... fleet scale
| better."
|
| That way your management continues to use their allocated
| budget, and your real prod systems work slightly better (also
| will eventually require less additional $ to scale up -
| helping the company i.e. shareholders).
|
| The thing to remember:
|
| You would assume all middle management really manages are a
| top line and a bottom line. Numbers related to their
| KPIs/OKRs are roughly a top line, and numbers related to
| their resources (humans and cloud infra budget) are roughly
| their bottom line.
|
| The reality: Middle management's resources (humans and cloud
| infra budget) are not their bottom line. Middle management
| gets rewarded (promoted) when they have "enough scope", scope
| has roughly always been defined by number of people (it now
| also includes things like cloud cost budget). As such middle
| management has to say "we need to do more with less", but
| they are promoted based on these numbers going up!
|
| Is this reward structure in the best interest of companies
| (i.e. customers and/or shareholders)? No, neither. Is there a
| better system? Not yet. Is the reward structure created by
| middle management for middle management? Likely.
|
| So in the meanwhile, if you don't want to become unmotivated,
| might as well work within the current reward structures.
| Thaxll wrote:
| Because ARM perf was far far from being on part with Intel /
| AMD, also you need to be able to compile on that arch.
| nodesocket wrote:
| Agree, I've gone to Graviton instances by default for RDS and
| ElastiCache (run and own a DevOps consulting company). The big
| problem that I continue to deal with is native arm64 Docker
| containers (if you a cool kid running containers / Kubernetes).
| For example, the very popular Bitnami charts don't support arm
| builds even though the community has been screaming for
| support.
| andrewstuart wrote:
| If I started to build today I'd build and host my own servers,
| or go with servers from ionos. Cloud is very expensive.
| gryf wrote:
| I just worked on a massive "optimised" cloud migration like
| you've never seen. We moved from multiple DCs to AWS and the
| costs are approximately 8x what the pre-migration costs are.
| We were realistically expecting 2x which gave us some
| regional agility and was expected but the unconstrained
| growth and misunderstanding of the cost model was terrible.
| It's designed to be so convoluted that you can't possibly
| estimate costs until you get the first bill at which point
| you are committed on a multi-month or year project. On top of
| that the assumption at the time of development is the cost is
| someone else's so the sprawl since the migration is dangerous
| which means we cannot leave ever now we've embraced the PaaS
| options.
|
| _The whole proposition relies on the idea of a sunk cost
| being accepted._
|
| So yes, back to servers please. IaaS should be the maximal
| offering that is accepted by a business from a risk
| perspective unless the tool or technology is disposable in a
| 6 month window. There is space there for gains. PaaS hell no.
|
| Edit: worth mentioning that AWS support is somewhere near
| dire. We've had issues with multiple services and despite
| being a VERY high roller with enterprise support we can't get
| anything fixed in any reasonable time. It's just someone
| else's crap you're using and they aren't any better at it
| than you are, just adding lead time to any issues. In some
| cases I've had to actually call out complete bad
| implementations that break function guarantees provided by
| open source projects (I can't logically warn people away from
| services as it's pretty obvious who I am if I do). One rule
| I've developed is that if it's not a core project: S3, EC2,
| EBS, ALB etc then it's probably a commercial liability in
| some way. There are no people working or with any knowledge
| on some major bits of AWS infra.
| tester756 wrote:
| >We moved from multiple DCs to AWS and the costs are
| approximately 8x what the pre-migration costs are.
|
| 8x?? that's crazy, what where you doing wrong then?
| gryf wrote:
| Everything, all at once.
|
| SMEs can't reliably manage that transition with any
| skillset and still deliver a product at the same time.
| jiggawatts wrote:
| I'm very curious to hear more details!
|
| Did you use reservations to reduce costs?
|
| Was it a lift and shift with VM configs staying as-is?
| (I've seen a lot of empty 1TB "app" drives burning money
| in the cloud!)
|
| You complain about PaaS services, but I can't imagine 8
| data centres worth of stuff being converted to PaaS in
| hurry!
| gryf wrote:
| Cost savings are mostly consolidation, scaling down stuff
| we don't need (we have peak hours) and migrating stuff to
| kubernetes and packing it tight.
| jiggawatts wrote:
| Despite that you were paying 8x the previous amount!?
|
| How is that possible?
|
| E.g.: With Kubernetes and AWS you ought to be able to use
| clusters with a base of "reserved" capacity plus spot
| pricing for peak hours on top, right?
|
| From what I've seen (in my limited experience), that
| _should_ reduce costs for most orgs, not increase them!
| gryf wrote:
| I have identified the limits of human incompetence.
|
| Please someone hire me so I don't have to live through
| this nightmare any longer.
| tester756 wrote:
| it feels like the "refactor" was the way to improve
| gryf wrote:
| I love reading people saying this. Try a couple of
| thousand DB tables evolved organically. Nope not
| happening.
| tester756 wrote:
| I meant that what you did was "refactor" - rethinking the
| whole mess
|
| So it wasn't purely cloud/renting racks issue
| Spivak wrote:
| Rent servers: yes. Host your own: maybe. You can run a whole-
| ass company on 2 $70/mo servers from Hetzner (and some B2 for
| durable storage) while you figure out whether you have a
| market or not.
|
| Like there's just no point in coloing when you're small
| because either all non-server bits will cost you for no
| reason or you're using something managed which is just cloud
| but more annoying.
| rjh29 wrote:
| We are building everything for arm and I've surprised if other
| large companies aren't optimising for it.
| tester756 wrote:
| Let's go even further - "Cloud cost is underrated"
| gryf wrote:
| This. We have three people entirely dedicated to reducing
| costs.
|
| As for avoiding ARM, we do only x86-64 because corporate
| security policy demands that we have Windows laptops so that
| some box ticking overlord can fill out a security policy
| compliance form. That means we're stuck limping along with
| docker and WSL2. Every single engineer in the org has an arm64
| machine at home already and wants a proper computer at work,
| which can ironically work in the same policy framework if
| anyone gave enough of a shit to deal with it.
|
| So that's why we don't use Graviton; corporate security
| policies. Our customers will just have to eat the price hikes.
| pclmulqdq wrote:
| I think the main reason is "I want to run the same binaries
| locally that I run in the cloud," and it's a pretty valid one.
| However, it's also an expensive one sometimes.
| jacobwg wrote:
| Anecdotally, this is starting to shift with M1 MacBooks,
| Graviton is looking more attractive for precisely that
| architecture parity reason for teams using majority M1
| devices.
| aidos wrote:
| That's the transition we went through. Our dependencies are
| / were pretty weird so the transition took a bit of effort
| - I suspect more complicated than many people would have to
| go through.
|
| We all use Macs at work so we knew it was a matter of time
| before we were on ARM. I'm glad we made the transition. M1
| airs are a delight to work with and Graviton machines are
| great bang for buck.
| gryf wrote:
| Yeah if only. Our ops people are too uneducated to be able
| to deploy anything Apple. Literally there are armies of
| factory pressed Windows monkeys but nothing in the Apple
| space.
|
| Note to apple: please start concentrating on the enterprise
| sector. We're dying over here. My Dell weighs 3x my
| personal M1 MBP, has a shitty keyboard with keys designed
| for Borrowers, the battery lasts 8 minutes and it reduces
| my sperm count if I put it on my lap. It feels like I have
| a ball and chain around my ankle 24/7. My only escape is
| WSL2 which is broken as fuck as well (can't run services,
| cron jobs, X problems etc) and we can't install a simple
| non-WSL VM on the node because Device Guard requires
| hyper-v to be enabled excluding sensible and pure VM
| options like VirtualBox. Docker for windows is a comedy of
| errors too.
| sbarre wrote:
| For what it's worth, I work in a large 30,000 employee
| company. Everyone gets a Windows machine by default.
|
| 6 years ago our department of 200 people "went rogue" and
| started provisioning Macs, because it was the only way we
| could hire developers and provide a good developer
| experience for the work we were doing.
|
| This took some convincing, but it was possible. We agreed
| to be unsupported by the in-house help desk, but we had 2
| people in IT that supported us for provisioning and
| fixing machines, and sorting out a small amount of
| required enterprise software like a Cisco VPN client and
| some fleet management background agents.
|
| Otherwise, we self-organized support over Slack and in-
| office and also made use of Apple's business support
| directly.
|
| As of earlier this year, our department is now over 600
| people, and we've given our internal IT enough incentive
| to officially support Macs, which they now do, alongside
| Windows.
|
| They use some kind of MDM software to manage and update
| and monitor our Macs the same as they do Windows.
|
| There are also now additional much larger teams in the
| organization exploring Mac adoption where it makes sense
| for their developers too, and we could soon have
| thousands of Macs in use.
|
| So it's definitely possible, even if you have to start
| small.
| lostmsu wrote:
| Are your developers forced to use laptops?
| gryf wrote:
| Unfortunately yes. I would rather a desktop but they
| don't know how to pay half as much for the same
| specification. The desks in the office are all equipped
| with docks and expensive WiFi mesh driven by COVID
| mentality so that is the status quo.
|
| Just send me a fucking workstation. Nope too hard.
| scarface74 wrote:
| Who wants to be in an office in 2022 unless necessary
| (specialized hardware, etc.)
| gryf wrote:
| Exactly that. I don't and I won't do it again.
| scarface74 wrote:
| Do you propose they carry desktops back and forth from
| their office and to conference rooms?
| lostmsu wrote:
| Every conference room could have a device to facilitate
| presentation.
| gryf wrote:
| Unfortunately we're in a regulated industry so getting a
| PO signed off for one Mac is nigh on impossible without
| involving the entire corporate machinery.
| dserodio wrote:
| Never user Docker for Windows, but Docker for Mac has not
| been great lately either.
|
| Granted, macOS changes a lot between each release but our
| company is paying for Docker Desktop licenses and the
| experience has really been disappointing.
| scarface74 wrote:
| > Note to apple: please start concentrating on the
| enterprise sector. We're dying over here.
|
| Enterprise sales are where the customer is not the user.
| Apple does best when the user is the purchaser.
|
| Also, I know for a fact that Macs are well supported at
| scale by many large tech companies including my own.
| vetinari wrote:
| FYI, with WSL2 1.0, you can finally enable systemd, so
| you can run services and cron jobs.
| gryf wrote:
| Thanks for the tip off. I will look into this tomorrow.
| This is why I'm here. The distributed consciousness of HN
| is a wonderful problem solving engine :)
| throwaway821909 wrote:
| Can you use a plain Hyper-V VM? i.e. with Hyper-V Manager
| gryf wrote:
| Tried that but unfortunately there are some painful
| addressing and routing issues when you are subjected to
| when dealign with a corporate always on VPN. Ergo you
| can't actually contact clusters which you have to admin
| via kubectl.
| arecurrence wrote:
| Completely agree... the only exception I've run into is that
| for small operations build tooling often doesn't work well with
| arm64.
|
| EG: GitHub actions can build a container in a few minutes in
| x64 or 35 minutes in arm64... likewise aws-cdk literally could
| not run an arm64 fargate ecs deployment for months after
| support was added (They simply did not support the required
| attribute in the container definition).
|
| I would love to see this change as I've had nothing but great
| experiences with graviton for virtually anything arm supported.
| _joel wrote:
| Are you building on arm64 natively or via qemu. A few mins vs
| 35 for the same roughish spec of CPU, seems a bit off, even
| with optimisation considerations.
|
| I've found arm64 builds on amd64 take longer when using one
| build context/arch (but doing multiple platforms), but that's
| as it's being emulated.
|
| It's the oppostite on my M1, the buildx amd64 takes longer.
| sofixa wrote:
| > GitHub actions can build a container in a few minutes in
| x64 or 35 minutes in arm64
|
| What type of container, and on what runner? That has not been
| my experience at all, a cross-compiling buildx build with
| Python and a bunch of libraries takes only slightly longer
| for arm64 than it did for x86.
| waych wrote:
| My favorite way to watch this slow down is to introduce
| some node workloads into the build workflow.
| forty wrote:
| First graviton is not magic. We switched our main service,
| which is a nodejs monolith, and did not get any cost
| improvement (we had to add more instances to handle the same
| workload, which ended up being equivalent cost wise). There are
| certainly use cases when it's better, but it doesn't seem to be
| the only and obvious choice for all use cases.
|
| Second our laptops and our CI are amd64 machines, and being
| able to run the same docker images in prod and locally is nice,
| and not having to build the image with qemu on the CI is also
| good.
|
| I don't mind cloud-ARM, but there definitely are good reasons
| not to use it (which of course don't apply to everyone)
| mk89 wrote:
| I agree. Until it becomes an issue, where everyone runs
| screaming like chickens, literally nobody gives a shit.
| phamilton wrote:
| For AWS managed resources definitely use Graviton. But for spot
| instances in EC2 we've found better pricing and greater
| availability by staying on x86. (We run 100% of our web services
| and background workers on spot instances).
| kaustubhvp wrote:
| The link throws a 503 rn. I wonder what would be the reason
| lemonJS wrote:
| Sorry, autoscaling took a second to ramp up
| beaviskhan wrote:
| The biggest downside I've found with Graviton is that it's gotten
| popular enough that availability of capacity is a problem in some
| regions/AZs - particularly if you're using larger EC2 instance
| types.
|
| Also, Fargate Spot on Graviton is still not available, so if you
| want to run Spot in non-production environments, you're facing
| with running different architectures in prod vs. non-prod, which
| I don't like at all. Do the math on whether it's cheaper for your
| use case to go x86 spot/non-spot vs. Graviton non-spot.
| glogla wrote:
| I have two comments.
|
| AWS Graviton is interesting because it is pretty different
| machine to their AMD and Intel offerings. A "16 vCPU" machine
| from AWS is 8 cores/16 threads, not 16 cores - except for
| Graviton, which actually has 16 cores, although much weaker ones.
| So for problems where you the cores can actually work in
| parallel, Graviton can keep up with AMD and Intel, while being
| somewhat cheaper. In single-threaded workload you get about half
| the performance.
|
| Second thing I curious about is this very AWS heavy approach.
| ECS, CodeDeploy, ElastiCache. If I was their architect, I would
| probably go EKS, GitHub/Lab, Redis on EKS, just for the peace of
| mind.
| dserodio wrote:
| ECS is so much simpler to use and understand than Kubernetes,
| even on EKS.
|
| But as for CodeDeploy... IMHO the only reason to use it is "I
| don't want to deal with another vendor" due to
| procurement/compliance hell in large companies.
| andrewstuart wrote:
| I found graviton to be a mixed bag. It was certainly extremely
| fast when using the very high end instances and I tested it
| successfully using a Rust based message queue system I was
| writing and it got some ridiculously fast number like 8 million
| messages a second, from memory, using the fastest possible
| graviton instance (this was about 18 months ago).
|
| I did try to switch some of my database servers to it a couple of
| years ago and after random hangs, I gave up and went back to
| intel. I tried again further down the track and same thing -
| random hangs. I assume this sort of thing comes with a new
| architecture but I'd be hesitant to move any production
| infrastructure to it without extensive long term testing.
|
| In the case of graviton based GPU instances I found that the GPU
| enabled software I wanted to use didn't work.
|
| If you are comparing performance, I'd suggest buying a fast AMD
| machine and run it locally and compare performance - local
| servers tend to be much faster and cheaper than cloud. And if
| your application uses GPUs then if you possibly can then its very
| much in your interests to run local servers.
| axiak wrote:
| Arm has a much looser memory model than x86 [1 for a
| comparison]. It's possible that the random hangs are due to a
| race condition in PG that doesn't show up in x86 because memory
| visibility doesn't require as much synchronization.
|
| 1: https://www.nickwilcox.com/blog/arm_vs_x86_memory_model/
| neodypsis wrote:
| Which database are you using?
| andrewstuart wrote:
| pg
| neodypsis wrote:
| Was this a while ago or was it a recent experience? I'm
| asking because I'm planning on using a serverless instance
| of PG and was interested in trying the ARM64 version.
| andrewstuart wrote:
| About 18 months ago.
|
| Try it - it might work fine for you.
| no_wizard wrote:
| I'd argue just find a different cloud provider.
|
| GCP, Azure, Supabase, Cloudflare etc if you want managed
| services.
|
| If you want a mix of managed services and raw compute, look
| more at Fly.io, Linode, Digital Ocean perhaps?
|
| I have found AWS being the "cheapest" or even "reasonable" in
| the cost department to be slimmer every year.
| MuffinFlavored wrote:
| Cloudflare doesn't let you host Docker containers or offer
| managed Postgres do they?
| no_wizard wrote:
| Its all about how they may fit in your stack. Most
| definitely fly.io does. I think Cloudflare as far as I'm
| aware is they're looking at supporting Docker.
|
| I just listed managed services (not all of them may fit I
| imagined)
| glogla wrote:
| There are huge differences in the machine generations. We found
| that for our workload Graviton3 (c7g) is the best, followed by
| AMD (m6a), followed by Intel (m6i) with Graviton2 (m6g)
| somewhat lagging. We can't use Graviton3 however because of
| memory limitations, so we're using AMD. The difference to the
| old machine types (m5) is staggering, the m6a is basically
| twice the performance of m5, while being cheaper.
|
| However, I've seen a lot of benchmarks telling a different
| story, so it is important to actually measure your workloads.
| whalesalad wrote:
| I've been enjoying them here and there but I've also found that
| for some of my workloads a high clock Intel node is required.
| Even the Epyc nodes couldn't keep up. I don't completely know
| why, never dug too far into it.
| judge2020 wrote:
| > local servers tend to be much faster and cheaper than cloud.
|
| Of course, running a server in your house is not going to
| achieve five or even three 9's of reliability, and even
| colocating a single rack in a single location might be more
| expensive than putting that infra in AWS (depending on how
| data-heavy your use case is, given AWS' exorbitant data
| transfer costs).
| lantry wrote:
| you can hit three nines even if you're down for 1.5 minutes
| every day, or ten minutes a week. It's really not as hard to
| hit as it sounds. For a compute heavy process that isn't end
| user facing (e.g. batch processing) it's perfectly viable.
|
| https://uptime.is/
|
| Also, most cloud providers don't guarantee five nines anyway.
| GCE SLA is 99.5 on a single instance, 99.99 on a region
|
| https://cloud.google.com/compute/sla
| Dowwie wrote:
| I'm curious about that Rust-based message queue system
| andrewstuart wrote:
| What do you want to know? It was a prototype. I was trying to
| learn Rust (didn't succeed), but I did manage to hack
| together a message queue that used HTTP for client
| interaction.
|
| I'd previously written a SQL database message queue in Python
| which worked with Postgres/MySQL and SQL server. This worked
| well but it was not fast enough for my liking. My goal was to
| build the fastest and simplest message queue server that
| exists, with zero configuration (I hate configuration).
|
| I used Rust with Actix and I tried two strategies - one
| strategy was to use the plain old file system as a storage
| back end, with each message in a single file. This was so
| fast that it easily maxed out the capability of the disk way
| before the CPU capabilities were maxed out. The advantage of
| using plain old file system as a storage back end is it
| requires no configuration at all. So I moved on to a RAM only
| strategy in which the message queue was entirely ephemeral,
| leaving the responsibility for message
| persistence/storage/reliability to the client. This was the
| configuration that got about 8 million messages a second.
|
| As far as I could tell my prototype left almost all message
| queue servers in the dust. This is because message queue
| servers seem to almost all integrate "reliable" message
| storage - that makes the entire solution much, much more
| complex and slow. My thinking was to separate the concerns of
| storage/reliability/delivery and focus my message queue only
| on message delivery, and push status information back to the
| client, which could then decide what to do about storage and
| retries.
|
| I gave up because I didn't see the point in the end because
| it wasn't going to make me any money, and I was finding Rust
| frustratingly hard to learn and I had other things to do.
| moloch-hai wrote:
| It seems very diplomatic of you to say you found Rust hard
| to learn, rather than that it was hard to make Rust do what
| you wanted. You seem very clear on what you wanted to do.
___________________________________________________________________
(page generated 2022-11-30 23:01 UTC)