[HN Gopher] Problems with AWS Amplify and its use of DynamoDB
___________________________________________________________________
Problems with AWS Amplify and its use of DynamoDB
Author : kevinslin
Score : 129 points
Date : 2023-04-10 16:15 UTC (6 hours ago)
(HTM) web link (samthor.au)
(TXT) w3m dump (samthor.au)
| inssein wrote:
| AWS Amplify is the worst AWS service I have used, by far. The
| console was constantly broken in chrome, their reverse proxy is
| slow, the pre-baked auth components with Cognito were pretty
| useless, and so much more..
|
| It feels like the team is trying to do way too much, all at once,
| and none of it well.
| outworlder wrote:
| > Consider using just a normal database--personally for me that
| would be Firestore (Google-hosted) or MongoDB (basically self-
| hotsed), because I strongly prefer scalable NoSQL databases.
|
| And then goes on to say:
|
| > But for most people, it's probably going to literally just be
| Postgres. You don't need more than Postgres.
|
| Those statements are basically contradictory. Maybe the author
| only works on applications where NoSQL is a good fit. Otherwise,
| it's not clear how the author is different from "most people".
|
| This idea that SQL databases are not "scalable" has become a meme
| and is difficult to eradicate. Sure, sharding/replicating
| Postgres (if you need to) can be more difficult than just
| increasing a field in some auto-scaling group. But a well-tuned
| Postgres can service a ridiculous number of requests. If you are
| doing microservices, then it's even better, since you'll have
| multiple smaller databases - some of these can even be NoSQL if
| it fits the application.
|
| What 'scalable' NoSQL databases usually means is that you can
| just throw money at the problem and increase the number of
| instances, without spending time doing silly things like
| 'optimizing' queries.
|
| Of course, for some applications, NoSQL is a perfect fit. But far
| too often I see people reaching out to NoSQL based on some future
| (usually unspecified) "scale" needs. Heck, I've even seen that
| mindset in internal, corporate network only apps.
| dmattia wrote:
| I use Amplify in a lot of my personal projects because it's
| backed by mostly serverless components and I can run my personal
| websites on it essentially for free. While in my job, I often
| lean for other infra for the services I manage. So I overall
| agree with the thesis, but I just couldn't get myself to agree
| with much of the reasoning in this article :/
|
| Parts I liked:
|
| - Callouts of the subscription model are good. I have used
| Firebase rtdb and firestore quite a bit, and agree it's much
| easier with those tools to just get an up-to-date view. With
| amplify, I often avoid subscriptions and just re-call list
| methods after updates (supported well in Apollo, but still
| unideal)
|
| - The discussions on Amplify/NoSQL/Dynamo being difficult to
| pivot match my experience, especially when I was less experienced
| with NoSQL patterns
|
| > Amplify's approach in using DynamoDB is literally called out as
| something you should avoid by AWS
|
| Alex DeBrie is a wonderful resource, and I love his books, but I
| wouldn't go as far as to say that all of AWS calls out using
| multiple tables. There is lots of active debate on the single-
| table vs multi-table designs
|
| > the fundamental mistake that's made here is that AWS Amplify
| puts your data into DynamoDB, which is not a general-purpose
| database
|
| Citation needed. It's a fairly standard NoSQL database and can be
| used generally.
|
| > But DynamoDB Scales?
|
| I was a bit confused on this article between the frequent
| mentions of DynamoDB only being worth it at a large scale ("If
| you're storing that much plain text data, good for you--maybe
| DynamoDB can help you!", "But you should only use it when a
| traditional database won't cut it", etc.) while also claiming
| more broadly that Amplify will not be useful for large apps
| because it doesn't scale. It seems to contradict itself there?
|
| I do think this section brings up a great point on the
| inflexibility of Amplify and trying to pivot a data model, which
| can absolutely be tricky! But the core message of this section
| doesn't seem to support the thesis.
|
| > If I was AWS, I would... provide Postgres or similar as a
| database choice
|
| I use RDS with Aurora/postgres with Amplify by using lambda
| resolvers and it works well. You can't use VTL, but in my mind
| that's a feature, as VTL is not my favorite for a few reasons
| like being hard to test.
|
| In general, this was one of my larger frustrations with this
| article: it doesn't mention the flexibility of Amplify. Amplify
| comes with highly opinionated built-ins, but you don't have to
| use almost any of it, and it's quite easy to plug-and-play other
| tools if you want. Lambda resolvers are a good example because
| they're extremely flexible, you can use many different
| languages/tools/even docker to run your resolvers. But also, I
| opt to not use Amplify hosting and it's fine. And you can also
| use your own auth without Cognito if you'd like (especially if
| you're already using custom resolvers).
|
| It sounds like the author inherited some parts of the codebase in
| question, so if some of these things were already setup I could
| see it being time consuming to pivot.
|
| > rethink whether GraphQL is actually fit for purpose: It's just
| not very good
|
| Again, while I'm not a huge GraphQL fan, this is very much an
| opinion and a highly debatable one. For many use cases, GraphQL
| can work well, and I think from a technical perspective it is at
| minimum arguable that something like GraphQL makes sense for a
| tool like Amplify where one of its core use cases is multi-
| platform usage across web/mobile.
| cameroncf wrote:
| The biggest problem with Amplify is not any of it's underlying
| technologies. These are all great services. DynamoDB is one of
| the best things in the entire AWS service portfolio.
|
| The biggest problem with Amplify is that it hides complexity that
| the developer really needs to have a firm grasp of when designing
| their application. You really need to think a lot about your
| access patterns before using DynamoDB. DynamoDB is OLTP and not
| OLAP. You aren't going to have a ton of ability to do ad-hoc
| queries with DynamoDB. that's just not a use case it's well
| suited for.
|
| HOWEVER - AWS Amplify is marketed without that warning - people
| use it without really understanding that. Then they get stuck
| when they start trying to treat it like a traditional RDBMS
| datastore, which it is not.
|
| Amplify "dumbs down" something that really should not be dumbed
| down. That's it's biggest problem and the reason I also stopped
| using Amplify some time ago.
| qaq wrote:
| DynamoDB is a sad hack that heavy marketing had elevated in
| absence of better offerings on AWS at the time. It's not
| particularly good as KV store let alone something more serious.
| time0ut wrote:
| DynamoDB is an absolute beast in the right hands. It is just
| difficult to use and far less forgiving than a basic SQL
| database - which should have been the go to for something
| like Amplify.
| finikytou wrote:
| maybe you could argument and suggest as to why it is not good
| as a KVS which is the number 1 use case for dynamoDB and
| everything in dynamo was optimized to be a KVS which a lot of
| multipurpose nosql database can't say.
| taeric wrote:
| I'm curious if you can give a long form critique? I've never
| heard any complaints of it from folks using it "in anger."
|
| I have heard a lot of complaints on it not doing as much. But
| I swear every time I've taken a dive on those complaints, it
| has been by folks that don't pay attention to the costs of
| what they are wanting. :(
| simplotek wrote:
| > It's not particularly good as KV store let alone something
| more serious.
|
| Nonsense. DynamoDB is a pretty good KV store, and a killer
| feature as part of AWS' free tier. I dare you to present a
| cheaper and more performance KV store.
| nslog wrote:
| This is spot on. Parts of Amplify are aimed at people just
| getting started with Cloud and Programming, which is good and
| bad. You can shoot yourself in the foot if you drive the car
| too fast. I think a lot of more advanced people use Amplify
| services along with the Amplify client SDKs and then do the
| provisioning using CDK rather than the Amplify CLI.
| twawaaay wrote:
| I would add a bit of my own experience.
|
| Products like this are great. But if you already know the
| technology, know what is happening and you are using it to
| basically eliminate a bunch of coding or setup tasks that you
| would have to do.
|
| The problem this is only great for a certain type of
| knowledgeable developer who is also setting up the system.
|
| The next person that comes or a junior developer who does not
| have grasp of the underlying tech will just see a bunch of
| magic. But he will be required to be productive with it.
|
| So if the original devs leave and don't get people with grasp
| of the underlying tech, the new people will start accumulating
| technical debt at an ever faster rate without ever being able
| to pay it back. Because with so little knowledge and
| expectation of performance, whenever something small happens
| that requires you to learn stuff you will skip learning and
| just move stuff around hoping some random change will fix it.
|
| So I will amend my original statement:
|
| "Products like this are great." But only if you are working on
| the project alone and don't expect anybody else to join it in
| the future.
| _fat_santa wrote:
| > You really need to think a lot about your access patterns
| before using DynamoDB. DynamoDB is OLTP and not OLAP. You
| aren't going to have a ton of ability to do ad-hoc queries with
| DynamoDB. that's just not a use case it's well suited for.
|
| This was a trap that I fell into. While I didn't go with
| Amplify, I went with DynamoDB assuming that "it's a high
| performance database, surely I'll be able to bend it do what I
| need it too, right?". I feel like AWS should be more upfront
| about the limitations of DDB, it's a great database but they
| sell it like the solution to everything, and it simply isn't.
| ramblinrose wrote:
| I use Amplify for a mobile app that I created and maintain just
| because I'm not familiar with backend technologies and it seemed
| like the quickest and easiest option for me, a mobile developer.
| The app isn't super complex, but I wouldn't describe it as a
| "toy" application either. I haven't run into any major headaches
| with DynamoDB or GraphQL limitations... yet.
|
| But this article has me thinking about exploring other options.
| To try to get ahead of the seemingly inevitable wave of tech
| debt. Does anyone have advice or can point to any resources for
| embarking on that particular journey? Are there similar
| technologies that do it better than Amplify? Anything in between
| Amplify and learning all the disparate AWS tools from scratch?
| spmurrayzzz wrote:
| > Anything in between Amplify and learning all the disparate
| AWS tools from scratch?
|
| It depends how much of the backend you ultimately would like to
| avoid (and related to that, how complex your service ecosystem
| will be for your app).
|
| Without knowing much more about your app, I can throw some
| darts. You could use RDS to get you postgres/maria/etc and then
| just run an app server on EC2 for your backend REST/SOAP/etc
| API in front of that. If the latter body of work isn't
| comfortable for you, there is a swath of low code tooling out
| there you could use as well to put an API in front of your SQL
| for you (nocodb comes to mind, or just postgrest if all you
| need is CRUD ops).
| camhart wrote:
| The title of the article isn't a fair representation of what the
| author actually says. The author dislikes specific technologies
| within the amplify umbrella, but admits other ones are
| acceptable.
|
| From experience, AWS Amplify auth for interacting with aws
| cognito is a perfectly acceptable high value (low cost) solution
| for many use cases. I can't speak to the other technologies.
| MuffinFlavored wrote:
| > AWS Amplify auth for interacting with aws cognito is a
| perfectly acceptable high value (low cost) solution for many
| use cases.
|
| https://www.passportjs.org/ is this an alternative or not
| really because you still have to "build it yourself"?
| camhart wrote:
| Passport is nodejs specific it looks like (no personal
| experience with it). AWS Amplify auth with cognito gives you
| libs for most major platforms (android, ios, web, etc) in
| most frameworks (vue, angular, react, native, etc). Plus aws
| cognito integrates well with AWS.
| thedougd wrote:
| I believe Amazon folded the Cognito SDKs into the Amplify
| SDK. You can use the SDK for Cognito and not use Amplify,
| the service, at all.
| MuffinFlavored wrote:
| > AWS Amplify is a complete solution that lets frontend web and
| mobile developers easily build, ship, and host full-stack
| applications on AWS, with the flexibility to leverage the breadth
| of AWS services as use cases evolve. No cloud expertise needed.
|
| How does this compare to say... a $10/mo DigitalOcean Linux VPS
| that you deploy stuff to with like... Ansible?
| dilyevsky wrote:
| One is a complete solution that lets frontend web and mobile
| developers easily build, ship, and host full-stack applications
| and the other is just old-school VPS in a cool wrap that does
| none of those things
| yamtaddle wrote:
| DO's network performance gets... _iffy_ under serious load. Or
| sometimes just has consistently-terrible throughput and /or
| latency to certain addresses, for unclear reasons (I have a
| suspicion their peering agreements and interconnects are simply
| awful for a lot of routes--probably as a cost-saving measure).
| You may not notice any of this on hobbyist or small-time
| commercial projects, but it's a real problem.
|
| So that's one difference. AWS is expensive, and its performance
| tiers for stuff like networking can be frustratingly opaque,
| but when you pay enough, you do get more-or-less what you paid
| for. And fighting DO's networking can get expensive in a hurry,
| in lost business or technician hours.
|
| As for Amplify vs. Ansible, yeah, dunno.
|
| [EDIT] Incidentally, this isn't the first time I've seen
| hosting of this general sort fall apart at the networking level
| --I ran into very similar problems with RackSpace over a decade
| ago, there was simply _no_ way to come anywhere near taxing the
| hardware we were paying for before the pipe got saturated,
| like, there was an order-of-magnitude mismatch between the two,
| requests timing out _en masse_ while the hardware ( "what a
| good bargain this price is!") was practically idle.
|
| [EDIT EDIT] It's probably no coincidence that it's easy to list
| and understand hardware specs, but difficult to communicate or
| evaluate real-world Internet-connected network performance
| without trying it--so it's probably tempting as hell to scrimp
| on networking while selling based on bang-for-the-buck in
| hardware allocation.
| walls wrote:
| I've run IRC bots on several VPS services, and connectivity
| on DO was by far the most unreliable. It's crazy how often
| they would randomly drop out, even under no load.
| yamtaddle wrote:
| Yeah, for some addresses that have no trouble whatsoever
| with AWS we've seen crazy-high dropped packet rates from
| DO. Sometimes, but not always, it's the reason for
| perceived extreme slowness from them, though other times
| they're just routing through some connection, somewhere
| along the way, that's about as wide as a straw, for
| whatever reason. Non-North-American addresses tend to get
| it the worst, but we've seen it with US and Canadian ones,
| too. AWS? Smooth sailing for all, from datacenters
| physically located near the DO ones we were using.
|
| With DO we'll have some clients seeing great performance,
| and others who may as well be on a spotty dial-up
| connection at the exact same time. Frustrating, and there's
| not really anything you can do about it except switch to
| another host. I'd love to know what the story is with 'em.
| cheapsteak wrote:
| Curious what you've found to be reliable / would recommend?
| nslog wrote:
| Stopped reading when this guy started talking about Single Table
| DDB being best practice and linking to AWS when he clearly did
| zero research on AWS recommendations with GraphQL including from
| the AWS Hero Alex Debrie he mentions:
| https://www.alexdebrie.com/posts/dynamodb-graphql/
| https://aws.amazon.com/blogs/mobile/single-table-vs-multi-ta...
| https://aws.amazon.com/blogs/mobile/enhancing-dynamodb-singl...
|
| Also he mentions the race condition around clients connecting to
| the backend between queries and mutations, then in the same
| breath says the DataStore client handles this but also it's
| mistake is being reliant on Subscriptions? This contradiction
| doesn't make sense.
|
| There are some legit bugs in this article though that I've seen,
| like that Analytics one. It's really a Pinpoint issue though when
| you dig under the covers that I've been waiting for them to solve
| for years.
| QuinnyPig wrote:
| It also charges a healthy fee on top of the already high AWS data
| egress charges...
| JCharante wrote:
| While Amplify offers a lot, using it for deployments only is
| really pain-free. If you have a node.js app that you want to
| deploy with support for feature branches and automatic webhooks
| listening to your git repo then it's a great serverless
| deployment choice compared to choosing some serverless frameworks
| where you still have to deal with CLIs and shell scripts in the
| repo.
| emptysea wrote:
| Sounds similar to Meteor, works fine for a prototype but
| collapses under complexity and perf issues once you want more
| features
|
| Mongo has a lot of performance gotchas with things like pipelines
| / aggregations that don't scale the same way they do in Postgres
| & similar
|
| Took a while but we eventually migrated to a plain rest api with
| a react front end. Still used mongo
| indymike wrote:
| I've used both Meteor and Amplify. Amplify has been fantastic
| for SPAs that don't need a backend database and require zero
| SSR. You get nice deployment tooling (especially for jr. devs)
| and CDN for very little money. When we started on our app, you
| could call a page generator during build, but doing dynamic SSR
| was not possible... yet... so we kind of dodged the bullet.
|
| Meteor was fantastic for building highly reactive apps with a
| small team. I think it got a lot of bad run for scaling up
| because it was dependent (at least when I used it) on tailing
| MongoDB's op log for reactivity. That lead to having to make
| very different architecture choices to make it work well, and I
| seem to recall each server instance getting bogged when they
| had 70-100 concurrent users , so you ended up using lots of
| containers for meteor application servers. Not sure where it
| all ended because I moved on to a different platform three yeas
| ago.
| kristianpaul wrote:
| Amplify as other similar software stack choices are very good for
| a MVP or POC. If your project is thriving start early and migrate
| your strategy to something that scale better and its easy to
| change.
| willio58 wrote:
| Speaking of limitations with scan in DynamoDb, I've recently been
| working with DynamoDB and found that our implementation was
| solely using scan operations. I wrote a couple lines of code to
| generate a secondary index, used a query operation instead for
| our endpoint serving page responses and that's going to save our
| company ~$15k this year (more as we scale) _and_ decreased our
| page load times by 10x (3s to .3s) for all of our users. In total
| that 's going to save over a year of page load time for all of
| our users over the next year.
|
| It's just odd to me that I had to make this change in the first
| place. I wonder how many millions of dollars AWS is raking in
| just because devs are unaware of things like that. Also makes me
| wonder how much energy is being wasted on inefficient DB
| operations..
| lightbendover wrote:
| Years ago after joining a new team, I was shocked that I just
| had a GSI to solve a performance problem that was somehow
| eluding a dozen engineers making around 200k each.
| geodel wrote:
| Well what are you doing is ideally lot many engineers should
| do. But in my experience devs, IT departments, companies are
| drunk on Cloud Mezcal. So any effort which need effort is "oops
| thats not our business lets increase instances in Cloud to
| improve performance" And I am not making this up, A service we
| are running on ten datacenter VMs, an equivalent service is
| running on 150 instances of same size in AWS.
|
| The cloud service is crappy architected, developed Java
| application but instead of fixing it , solution is to
| double/triple or 10 times the instance using Kubernetes
| Autoscaler. One thing they have going is ever larger budget
| since it is next gen cloud service, whereas my serivce being
| plain old efficient Java app is _legacy_ to be deprecated and
| replaced with new cloud version.
| re-thc wrote:
| This has less to do with the cloud and more to do with the
| common mantra of "premature optimization".
|
| To a lot of engineers there's always bigger fish to fry.
| silisili wrote:
| I just learned DynamoDB this year, after I was forced into
| it...not a NoSQL fan.
|
| In any event, while I find the structure and docs pretty
| confounding overall, they do make that behavior pretty clear.
| The libraries even separate out scans from queries, with the
| latter taking only a key.
|
| I'd put the blame on the implementer in this case, not AWS.
| willio58 wrote:
| I actually agree. This was probably only an issue because
| some dev I work with didn't read the docs or didn't think it
| mattered at the time because our userbase was so small the
| difference between a scan and a query was negligible at the
| time.
|
| Part of me does wonder if there would be a benefit in
| DynamoDB providing a function that under the hood creates
| secondary indices and uses queries where that would be the
| most efficient choice automatically. But maybe that would be
| a better fit for an ORM of some sort.
| gregplaysguitar wrote:
| That sounds like a query planner... but it's fundamentally
| not really what dynamodb is about. Query planners are
| amazing but the tradeoff is unpredictable performance -
| dependent on traffic, data size etc. Dynamodb offers
| "predictable performance at any scale" precisely because it
| doesn't offer this sort of abstraction.
| silisili wrote:
| That would have been a killer feature for me. Instead of
| reading about GSIs, LSIs, and even going so far as having
| to create a dummy field to do sorting properly[1], it would
| be pretty nice if you could somehow input the queries you
| want to do, and have Dynamo make suggestions.
|
| 1 - In a collection, I could find no way to easily do 'give
| me the latest record.', which is super easy in rdb. I ended
| up creating a key something static like 'x', with date as
| the range key to accomplish this. If anyone has a cleaner
| way, please share.
| vanjajaja1 wrote:
| It's funny to read something like this contrast against all the
| statements about how leetcode interviews do not help and ds/a
| is something you can learn on the job
| lozenge wrote:
| That's funny because reading SQL doesn't tell you whether the
| DB will scan or use indexes. With Dynamo DB it's a separate api
| request, it couldn't get more explicit.
| simplotek wrote:
| > It's just odd to me that I had to make this change in the
| first place. I wonder how many millions of dollars AWS is
| raking in just because devs are unaware of things like that.
|
| What "things like that"? Do you mean the basics of querying
| data with DynamoDB?
|
| https://docs.aws.amazon.com/amazondynamodb/latest/developerg...
|
| This is literally the second topic in any intro to DynamoDB
| course, that comes right after creating your first DynamoDB
| table. Even that covers explicitly the basic requirements of
| running queries.
| camhart wrote:
| Their documentation is very clear about scans being expensive.
| Amazon isn't to blame for the poor engineering decision.
| lofatdairy wrote:
| That was my initial impression reading the OP as well, if
| your implementation is slow because you're using scans, well,
| it's because you're using scans. That said, I honestly don't
| know enough about the details regarding OP's implementation
| and why they couldn't refactor to properly Query the PK, but
| it sounds similar to how you'd authenticate reads on Firebase
| so maybe it's habits from a different platform kicking in.
| That said, Amplify allowing users to just get into GraphQL
| whether or not they actually need it might be a legitimate
| noob trap.
| nslog wrote:
| Eh their documentation is pretty clear about all this stuff
| and all the new directives released a couple years back
| (like the PK and index stuff) clearly guide people towards
| using indexed queries and not scans:
| https://docs.amplify.aws/cli/graphql/data-modeling/
|
| A lot of this is on the author IMO
| wnolens wrote:
| "I wonder how many devs don't know that hash tables are bad for
| range scans".
|
| Fixed that for you.
| yamtaddle wrote:
| Our entire industry is plagued by people rushing forward while
| having no idea how the tools at their disposal work. It's
| immensely wasteful.
|
| I can't even claim to be immune from this, though I do _try_ to
| know what 's available and how to use it. I'm damn near 40 but
| it's not unusual for me to still, today, stumble on some tool
| older than me that does exactly what some New Shiny crap does--
| but, not infrequently, better, at least for some common use-
| cases, plus it'll have all the kinks ironed out, or at least
| documented. Good luck promoting any of that in the workplace,
| though. For all we go on and on about the importance of re-use
| and avoiding NIH and all that, we sure are quick to disregard
| solutions that we can't npm-install.
|
| No clue what the fix for this is, or if it even makes sense,
| all things considered, to try to fix it.
| Swizec wrote:
| Symptom of a rapidly growing industry. With N(developers)
| doubling every 5 years or so, chasing newness at all times
| and not knowing about good old things is the norm.
|
| I'd rather have that than an industry where any tech newer
| than 10 years is radical and scary.
|
| Us old farts can remind the kids that old solutions exist
| when appropriate. It is much harder to try introducing
| something new into an org that doesn't want it, but needs it.
| sleepybrett wrote:
| Yeah I liken it to someone who picks up a blacksmiths hammer
| and then goes and drives some nails with it. It's like 'sure
| I guess it works, but man there are way better options that
| will work better'. Happens with all our industry tools, but
| perhaps there is no better example than database choice.
|
| Engineers often see a database simply as a place to store
| things, it's just a box.. they work on a product for awhile
| that's using mongo, they are more likely to choose it later
| when they are developing a solution because they've used it
| before and know it 'works' and more importantly they know
| something about it. Databases as a topic of study are
| complicated, there are tons of tradeoffs happening all over
| the place and until you hit any given pothole you may not
| really understand those tradeoffs.
|
| I guess what I'm saying is that in my life I'm lucky enough
| to have an old friend that is REALLY into databses. You
| should find this friend or become this friend.
|
| Honestly at the end of the day this is all human nature. The
| hammer worked for you before why do the research to find out
| that there is a better hammer for you to use that would get
| your job done quicker and easier. You've gotta get these
| boards nailed down RIGHT FUCKING NOW or your bosses are going
| to be pissed.
| yamtaddle wrote:
| > Engineers often see a database simply as a place to store
| things, it's just a box..
|
| Oh man, I had an edit right after posting where I _almost_
| attached an addendum about this specifically, but then
| decided against it to avoid distracting from my core point.
| There 's a totally bizarre allergy to actually using
| database features, for fear of "lock-in" or something, yet
| most programs are more likely to be re-written on top of
| the same database than to have the database swapped out
| from under it, unless someone made a really god-awful
| choice of DB, or load scales tremendously (which falls
| under "good problem to have, and you can afford the
| migration"). The result is that we as an industry waste an
| awful lot of time and money creating often-buggy
| application-layer solutions to things that could have been
| solved by leveraging features of, for example, PostgreSQL,
| with results that'd be cheaper, less-buggy, and better-
| performing. Drives me absolutely nuts.
|
| Or, like, we have nginx in our stack and it could do
| something we need done, with a config change or a little
| Lua... but no, we'll spend 10-100x as long to make a worse-
| performing custom solution to this problem that _a thing we
| 're already using_ can trivially solve for us, but we'll do
| it in Javascript or Ruby or Python or whatever, as part of
| our "app". Ugh.
|
| And don't get me started on system-level tools and daemons.
| Like, we're already using Linux or FreeBSD or whatever,
| which come with a _stupid-large_ set of great, proven
| tools, so how about we _actually use it_ rather than just
| using it as a fancy DOS and paying for SaaS to do things
| that our OS or distro can already do, probably more
| reliably? But no, we don 't even bother to configure a
| recipient for root alert emails more often than not, and we
| just treat the whole thing as a dumb application runner
| that needs some complex external support system to keep it
| working right.
| ctvo wrote:
| > It's just odd to me that I had to make this change in the
| first place. I wonder how many millions of dollars AWS is
| raking in just because devs are unaware of things like that.
|
| AWS documentation very clearly outlines the behavior of a query
| and a scan. The internet is full of guides [1][2] on how to
| model your data so you utilize queries vs. scans.
|
| If the developers at your company misuse a product, and aren't
| aware of these features when does personal responsibility come
| in? How is AWS as a platform suppose to discern that the trade-
| off with a GSI is what you want, and how is this scenario not
| generalized to every product? People misuse SQL databases _all_
| the time, for example.
|
| 1 - https://www.alexdebrie.com/posts/dynamodb-single-table/
|
| 2 - https://www.sensedeep.com/blog/posts/2021/dynamodb-
| singletab...
| 015a wrote:
| I think its hard to draw a fine line on where this
| responsibility lies.
|
| For example: for years, MongoDB took a lot of shit because by
| default it spins up unauthenticated. Hypothetically; devs
| take on the responsibility of adding a password.
| Realistically, shodan was able to index hundreds, maybe
| thousands, of databases across the internet with no
| authentication, some containing highly sensitive data.
|
| Firestore is a great analogue to DynamoDB. Not perfect, but
| they solve a lot of the same problems in a lot of the same
| ways. Firestore automatically indexes every field in every
| document in every collection. Flat out. Everything is on an
| index. That single decision has reverberated to save tons of
| hours in compute time across all of their customers.
|
| Is this expensive to do? I have no doubt; and its reflected
| in the cost of the two services. But believe it or not:
| Firebase isn't _that_ much more expensive than DynamoDB
| ($0.25 /million reads, versus $0.60/million).
| badrequest wrote:
| There are two sides to this arrangment: devs who mostly don't
| have a choice over the tools they use vs. the trillion-dollar
| company that is building and documenting them. The notion
| that the devs have just as much culpability in how things end
| up as AWS is absurd.
| ctvo wrote:
| > devs who mostly don't have a choice over the tools they
| use
|
| I'm not going to assume that DynamoDB was forced upon them.
| I don't know. What I'm more sure about:
|
| Developers have a choice in if they want to invest the time
| in understanding the tools they use, and learning about the
| tool's capabilities. In this specific case, we're not
| talking about a lot of time. 2-3 hours to read the
| documentation for a new database you're going to be
| building your business on top of.
|
| What is your argument exactly?
| CharlieDigital wrote:
| I use Firestore and it won't let your perform multi-field
| filter and order operations without an explicit index.
|
| When you try to do so, it spits out an error that tells you
| this operation requires an index and then provides a link to
| create the specific index.
|
| I know it is a different database paradigm, but I think this
| is great "UX" design.
|
| Firebase has a bunch of additional great features in this
| vein. It allows definition of min/max instances for Functions
| and if you set the min instances in the code, it will force a
| confirmation of price changes to prevent any unexpected
| charges. (GitHub Actions will fail, for example, until the
| price increase is confirmed via CLI.)
|
| So i get OP. Yes, it is the responsibility of the engineer to
| know these things and make decisions, but the vendor can
| provide much better UX around these constraints.
| JimmyAustin wrote:
| The equivalent in a relational database would be writing a
| query without a where statement, and then implementing the
| where in application code. AWS has a lot of unimpressive bits,
| but DynamoDB is not one of them, you just have to know how to
| use it properly (like any database!), and design your schema
| for the database (which isn't always easy). In general, using
| scan means you aren't using Dynamo correctly.
|
| Caveat: I used to work at Amazon (not in AWS), but don't
| anymore.
| nkozyra wrote:
| > The equivalent in a relational database would be writing a
| query without a where statement,
|
| Sort of, though I'd say scan operations are typically more
| expensive in any RDMS.
|
| The probably becomes when you want/need something more than
| the secondary index, then you gotta get more creative.
| [deleted]
| jwsteigerwalt wrote:
| You can pick and choose what you use amplify for. I use it for
| all the front-end CI/CD tasks and API Gateway/Lambda/etc. however
| I want for the backend. Use it for what it's good for without
| getting locked in.
| othiagocruz wrote:
| I'm curious what was the company doing with Cognito that required
| anything more than DynamoDB, perhaps storing user profile
| information or relationships? This is the wrong approach, you use
| Cognito/Amplify for session management, not data relationship.
| ursuscamp wrote:
| One of the things I always thought was funny was that Amazon
| always tells you to put everything in one DynamoDB table, but the
| Amplify splits everything across multiple tables.
| oliverrice wrote:
| disclaimer: author of supabase/pg_graphql
|
| > What should AWS do -> provide Postgres or similar as a database
| choice
|
| If you're interested a postgres option you could check out
| Supabase GraphQL[1] which is based on pg_graphql [2], a native
| PostgreSQL extension.
|
| You define your schema in SQL (including any indexes you'd like)
| and it reflects a full GraphQL API. With that stack, the example
| you gave about filtering for security would use be solved using a
| Row Level Security [3] policy where the work all occurs on the
| DB.
|
| While not a part of pg_graphql, using Supabase as a backend also
| handles Auth [4] & Storage [5] (both OSS) which covers most of
| the Amplify bases
|
| [1] https://supabase.com/docs/guides/api#graphql-api-overview [2]
| https://github.com/supabase/pg_graphql [3]
| https://www.postgresql.org/docs/current/sql-createpolicy.htm...
| [5] https://supabase.com/docs/guides/auth [5]
| https://supabase.com/docs/guides/storage
| smotched wrote:
| Supabase will never be a realistic option for any company thats
| heard of mobile.
|
| Even if the startup is starting web only, they're kneecapping
| their future so its not an option
|
| Amplify has amazing mobile sdks (esp ios).
| simlevesque wrote:
| > Supabase will never be a realistic option
|
| What if they make a mobile SDK too ? Seems like your comment
| is an hyperbole.
| trilobyte wrote:
| This is a pretty good write up on the actual implementation
| problems of the solution vs. just a rant. The company I'm at now
| built their own custom graph database years ago and is plagued by
| problems that would be cousins of some of those listed here, so I
| see the pain the author is dealing with because I'm hearing about
| similar problems from some team or customer support at least
| 1x/month.
| harrisonjackson wrote:
| I use amplify in lots of projects. It is a nice wrapper for the
| auth and registration stuff. We also use it for some frontend
| ci/cd. Nothing else though.
|
| Custom authorizer in front of aws lambda with serverless
| framework works great with amplify's frontend api client.
|
| We also use dynamodb serverless but not with app sync and no
| graphQL. Would I rather have a good and easy to use serverless
| Postgres db that played well with aws lambda. Yes. Am I going to
| switch away because I have to plan my access patterns and indices
| out carefully... no.
| freedomben wrote:
| We went all in with Dynamo years back for an app because it was
| exciting. It went fine so we did it again, and again. Over time
| though (and in some cases not much time) we quickly realized that
| we were writing a lot of code to do sanity/safety checks, that
| was basically just re-implementing relational database features.
| As we'd hit bugs, we'd add more checking to our app, and
| eventually we just had a ton of relational DB features at the app
| level. I've just used postgres ever since.
| lightbendover wrote:
| Dynamo is almost never something you want to use as a
| "solution" to an open-ended I-don't-quite-know-my-requirements
| type of problem (which is almost all holistic problems).
| Sometimes I feel like it's the only tool anyone knows how to
| reach for, but it comes with dozens of ways to shoot yourself
| in the foot one way or another if you're trying to use it
| generally.
| ngc248 wrote:
| DynamoDB is amazing when you know all the access patterns for
| your data and the access patterns are not liable to change
| much. The performance it gives in such a case is superlative to
| just about anything else. But if you have changing access
| patterns and require flexibility in querying, then DynamoDB is
| not for you.
| solatic wrote:
| > provide Postgres or similar as a database choice
|
| Amplify's core architectural presumption is to only use
| serverless options to try to keep costs low. Serverless SQL
| databases are currently a big hole in AWS's offering. Yes,
| there's Aurora Serverless, but you need to run it in a VPC, you
| can't scale to zero (so, for some definitions, not really
| serverless), v2 threw out their Data API, and v1's Data API had
| severe scaling issues anyway. v2 is so expensive that you have to
| very seriously think through the numbers on the small RDS
| databases and ask what value you're actually going to get out of
| v2. So what OP proposes can't really be done by the Amplify team.
|
| That said, the fact that Planetscale and Neon (which also have
| branching!) continue without serious competition from AWS is a
| serious headscratcher.
| Scubabear68 wrote:
| V2 Aurora costs really took me by surprise at my last gig they
| used it. It seems to be extremely expensive compared to just
| about anything else.
| nkozyra wrote:
| That said, nothing prevents you from spinning up
| services/endpoints that lean on whatever datasource you care
| about. Assuming it's accessible within the VPC you can shove
| all your data in there and call it a day.
|
| You have another potential problem in your stack, but it can be
| done.
___________________________________________________________________
(page generated 2023-04-10 23:02 UTC)