[HN Gopher] A single developer dropped AWS costs by 90%, then di...
___________________________________________________________________
A single developer dropped AWS costs by 90%, then disappeared
Author : develatio
Score : 95 points
Date : 2023-01-09 20:26 UTC (2 hours ago)
(HTM) web link (medium.com)
(TXT) w3m dump (medium.com)
| cloudking wrote:
| Reminds of a developer I hired through Fiverr that claimed he
| could achieve 99/100 Google pagespeed score for any Shopify
| website. He implemented his changes, and sure enough it scored
| 99/100. However, the site still seemed to be running slowly when
| I tested it. I dug into what code he had actually changed, and
| there was only a few lines of JavaScript injected into the site.
| The JS was obfuscated, after de-obfuscating it I realized all his
| code did was prevent any other JavaScript from being loaded if it
| detected the user-agent was coming from Google pagespeed :)
|
| After some back and forth with Fiverr I was able to get refunded,
| but I'm sure the thousands of other customers that gave him 5
| star reviews have no idea.
| selimthegrim wrote:
| Original Twitter thread in Russian is here:
| https://mobile.twitter.com/xanf_ua/status/160812131107877683...
| mc32 wrote:
| If something seems unbelievable, it probably is. How can anyone
| not question how a 90% reduction was possible without proof?
|
| That said it looks like the company did save money for a few
| months so the 50 grand was worth it although hard to maintain
| indefinitely and of course against ToSes and morally
| questionable.
| wstuartcl wrote:
| "probably" but not always. I have worked on a few projects that
| saved 70-90% of costs for AWS -- but these all were starting
| from a position of auto scaling out a ton of ec2 (on demand
| full cost instances) to batch process huge amounts of data. In
| both scenarios by optimizing the batching to align tasks to
| pack as much as possible into (just short of hour long) hour
| long windows and converting to a (wide list of ec2 types to
| avoid contention lockout) spot instances there was easy fodder
| for the improvement.
|
| But yeah -- no one should see their bill change by that much
| without understanding exactly what was done to trigger it. Even
| in the best cases, there are always tradeoffs that must be
| known and accepted.
| poulsbohemian wrote:
| Agreed with both you and parent commenter - the story comes
| across as almost apocryphal; a warning both that if it's too
| good to seem true, it probably is, and that one must always
| understand the work done and the tradeoffs involved (IE:
| don't fall asleep at the wheel)
| prottog wrote:
| To be fair, a lot of cloud deployments are not optimized for
| cost, and it's easy to rack up big bills without really needing
| to.
| imron wrote:
| To be fair, as an IT director, when you see this sort of
| reduction you should dig deeper and understand where the cost
| savings are coming from.
| CrimsonRain wrote:
| Reducing cloud costs, especially if the dev team does not have
| anyone experienced enough to focus on this, is often possible
| on massive scales. More so if you have free reign to
| optimize/rewrite stuff as you see fit. While not 90%, I've
| reduced cost by 83% in one case while increasing the ability to
| serve during peak times also.
| daniel-cussen wrote:
| [dead]
| deltree7 wrote:
| Reducing 90% cloud bill is not out of the ordinary.
| dikaio wrote:
| Wonder if stealing a companies slogan ""We Shall Build Good Ships
| Here; At A Profit If We Can, At A Loss If We Must, But Always
| Good Ships."" -- Collis Potter Huntington is considered the same
| thing?
| uejfiweun wrote:
| This is why you need strict code reviews, and leaders who
| actually understand the technology. Any competent organization
| would have caught this.
| 0xbadcafebee wrote:
| * * *
| joshstrange wrote:
| I'm almost annoyed the SaaS company in question profited from
| this for those 6 months before their costs started going back up
| again. Any IT Director who bought into this snake oil, allowed
| the person access to their codebase, and accepted their changes
| without review should also be fired.
| worldsavior wrote:
| But it's illegal. It's abusing the system.
| hsbauauvhabzb wrote:
| It's against the terms of service, and a bad idea because
| you'll get banned and lose all the data in the free accounts,
| but is it illegal?
| scrollaway wrote:
| Not a lawyer. It's likely fraud. (Doesn't matter because the
| story is fictional anyway)
| nottorp wrote:
| This ticks me off every time i see it. Neither Amazon, nor
| Google or any other tech company are allowed to make laws. It's
| a breach of the terms of service maybe, and maybe illegal, but
| whatever the terms of service say is not a law and said terms
| of service could be themselves illegal.
|
| They have enough power, please stop attributing them even more
| by your choice of terms.
| paxys wrote:
| It is most definitely illegal to buy and use stolen
| credentials from darknet markets (which is what the author
| hypothesizes happened in this case). It is also illegal to
| misrepresent the service you are providing to a customer.
| advisedwang wrote:
| It's likely wire fraud
| imwillofficial wrote:
| Being against the terms of service is not illegal.
|
| They are different things.
|
| Google might have a case for fraud, but that would be unlikely
| to move forward given the circumstances.
| al2o3cr wrote:
| Dupe of https://news.ycombinator.com/item?id=34281802
| reidjs wrote:
| Pretty clever scam. He had a huge list of accounts, then set up a
| script that took advantage of the free introductory compute AWS
| gives to new users. When the script ran out of the accounts to
| cycle through, it fell back to the old system.
| 411111111111111 wrote:
| How can you consider it a scam though? They paid 50k and saved
| hundreds of thousands according to this story (90+k reduced by
| 90% for 6 month.)
|
| Google is the only one that got scammed as far as I can tell.
| paxys wrote:
| It's a scam because (1) they did not promise a cost reduction
| for just a few months, and (2) what they did was illegal
| and/or against the terms of service of the company's cloud
| provider. Just like how selling you a perfect working car for
| a great price - which just happens to be stolen - would be a
| scam.
| comprev wrote:
| A good story, and likely untrue although still possible.
|
| I've helped clients save huge chunks of cash over the years from
| AWS because the previous developer/team paid little attention to
| costs.
|
| Reserved instances, alternative routing of data, cheaper S3
| tiers, etc. soon add up significant savings.
| jsnell wrote:
| Sorry, why are we believing this story? The technical details
| make little sense; e.g. moving a ml processing pipeline to a
| different cloud just by "editing a Chrome extension" doesn't seem
| very plausible.
|
| It isn't even internally consistent, the hallmark of a sloppy
| fabrication. At first the accounts were sourced from the darknet,
| then they were created by the Chrome extension.
|
| Creating a million Google accounts is pretty hard. And _buying_
| so many accounts from the markets would probably cost his whole
| $50k price.
|
| Also, it was a million accounts and the savings were about 500k.
| Each account spent 50 cents worth of compute out of the $300 free
| trial? I assume you need to give a credit card number for the
| free trial. I wonder how many of those were needed for this to
| work...
| advisedwang wrote:
| Your skepticism is warranted, but just to note:
|
| > At first the accounts were sourced from the darknet, then
| they were created by the Chrome extension
|
| The article says they bought Google Accounts on darknet and
| _created GCP accounts_ from the chrome extension. Presumably
| "GCP Account" here means a new GCP project/billing account with
| free trial or using the always free usage. So it's vague and
| difficult to believe but not actually contradictory.
| generalizations wrote:
| > A few months later costs started to rise again. Each time a GCP
| free-tier account did not work, data was sent back to the old AWS
| system.
|
| I like this part. He may have run a total scam, but he still
| wrote good code, dammit.
| bostonsre wrote:
| The company also still saved money in the long run, just not as
| much as they had hoped initially.
| paxys wrote:
| The money the company saved would have paled in comparison to
| legal costs had Google or some government authority decided
| to take action against them for buying and using stolen
| accounts. And no "this random developer (who we can't find)
| did it" isn't a valid excuse. The company is still fully
| liable for their code.
| [deleted]
| kichik wrote:
| And still saved them about half a million dollars (6*$90k -
| $50k).
| xwdv wrote:
| IMO this is what allows him to escape guilt free. Although
| his solution didn't last, it lasted long enough that the
| money saved was far greater than if they had not paid him
| $50k. This could be a good hack for a startup that needs to
| temporarily reduce its cost for some short period of time.
|
| Do this as quickly as possible for a a dozen or so clients
| and you could build a decent nest egg in a short period of
| time.
___________________________________________________________________
(page generated 2023-01-09 23:01 UTC)