[HN Gopher] Amazon Aurora Supports PostgreSQL 14
___________________________________________________________________
Amazon Aurora Supports PostgreSQL 14
Author : schranz
Score : 88 points
Date : 2022-07-10 18:10 UTC (4 hours ago)
(HTM) web link (www.infoq.com)
(TXT) w3m dump (www.infoq.com)
| torvald wrote:
| Google recently also their variant of this product, called
| AlloyDB.
|
| https://cloud.google.com/blog/products/databases/introducing...
| nikita wrote:
| Also there is an open source alternative:
| https://github.com/neondatabase
|
| (I'm ceo of neon)
| garblegarble wrote:
| Somewhat OT and forgive me if I'm missing something on your
| github, I don't see how to set up a cluster locally - am I
| thinking about Neon wrong and it only runs on one node at a
| time with shared storage if you want to migrate which node
| runs a given database at a given moment?
| [deleted]
| brightball wrote:
| I'm really interested to try this one because I'm fairly sure
| it's a better option for analytic queries than Aurora.
| godelmachine wrote:
| I work in an enterprise software ecosystem and on current version
| (something we call the _converged platform_ ), we are getting our
| SaaS customers upgraded to Aurora from previous SQL DB. It's just
| what our DevOps calls a "life and shift upgrade" (whatever the
| heck that means coz the issues that entails it gives us a hard
| time)
| FBISurveillance wrote:
| Lift and shift probably - means to move as is.
| Spivak wrote:
| Lift and Shift -- move an application from one environment to
| another with minimal architecture changes. The difficulty comes
| when you have built-in assumptions that can't be replicated to
| the new environment. Some good examples would be clusters that
| rely on floating ips, ARP, interface bonding, SCSI, or
| multicast.
|
| Over the years cloud providers have added these features but
| they weren't always available and definitely not something you
| would reach for in a cloud native setup.
| johnthuss wrote:
| I'm really glad to see this. Historically Aurora has been VERY
| slow to support new versions of Postgres (despite promising this
| would not be the case). But lately they have really stepped up
| their pace to the point where I would consider migrating some
| databases to Aurora. I'ma big fan of the feature that allows you
| to clone a large database quickly (in 30 minutes maybe). It's
| extremely useful for troubleshooting issues where you need
| production data.
| ianschmitz wrote:
| We have a database with a few TB of data. We can create an
| aurora clone in around 10 mins give or take, but what we've
| found is that this the clone loads data in lazily as needed.
| Meaning the first time you hit a given table in a query it can
| be very slow. Afterwards it works well though
| everfrustrated wrote:
| My understanding is the earlier versions of postgres required
| more expensive patching to support their proprietary storage
| layer. Some of the common hooks have since been upstreamed
| (whether aws gets any credit or if it was part of other work I
| couldn't say), but in theory aws doesn't have to do as much
| work to support new versions as they used to.
| gcbirzan wrote:
| Back when we were on non aurora PostgreSQL, cloning from a
| snapshot was similarly fast. Maybe you're talking about getting
| a read replica?
| syspec wrote:
| N00b question: is Amazon aurora a reimplementation of Postgres,
| or just something that allows you to use Postgres?
| everfrustrated wrote:
| There's some deep dives on the aws YouTube channel that talk
| about their storage layer if you want some detail. AWS happily
| share quite a lot more under NDA as well, if that's an option
| for you. The short answer is the proprietary changes are very
| well done in a way that provides no meaningful vendor lock-in.
| whirlwin wrote:
| Aurora is a managed Postgres offering, similar to Amazon RDS,
| but with clustering capabilities.
|
| We use Aurora for all apps. It costs more, but needs little to
| no maintenance from our side
| wahnfrieden wrote:
| it's way better than vanilla RDS even without any clustering.
| it is far more stable under load / near capacity. the storage
| layer is amazing. people here who are underselling it
| relative to vanilla RDS simply lack operational experience
| under load and have a checklist-level understanding
| leesalminen wrote:
| +1 Aurora was the secret sauce for us to scale a db-heavy
| SaaS very quickly with no dedicated ops team. Got us an
| extra 9 in uptime, too. The cost difference from vanilla
| RDS was negligible for our revenue model. YMMV, of course
| on that.
| gcbirzan wrote:
| Except... That's not always true. We were told the same,
| but we found the performance to be underwhelming, at best.
| Concurrent writes to the same row are orders of magnitude
| worse, certain read patterns are similarly slow, and you
| are severely limited in how many transactions you can have
| before the storage daemon gives up and you end up with
| double digit write latency for any write.
|
| Yeah, for the most part it's better, but you lose the page
| cache buffer to stop you from running oom if you
| miscalculated your shared buffers, your cpu usage will go
| up significantly and sometimes things will be slower. While
| on paper the max IOPS are way higher, you do get more IOPS
| from migrating and then you run into other issues.
|
| If you are or close to the 80k IOPS limit, or 65TB storage
| limit, there's no alternative, but the "aurora is always
| faster" line that AWS tries to sell you is bullshit.
| gurjeet wrote:
| Aurora Postgres is a fork of Postgres code, with modifications
| that allow it to leverage AWS' Aurora storage, and a few other
| things (replicas, etc.).
|
| The goal is to remain 100% Postgres compatible. So the network
| protocol, SQL queries, and tools, all can be used without any
| change. There are (were?) some (read: very few)
| caveats/restrictions, though, because of the heritage of, and
| need to be compatible with, Amazon RDS Postgres.
|
| Disclosure: Member of the founding-team of Aurora Postgres;
| left the team quite a while back, so my knowledge is quite
| outdated.
|
| Edit: There's a lot of literature, and public information
| (articles, conference talks, etc.) published by the Aurora team
| over the years, which all allude to these facts, and you can
| use that material to build your confidence that it is in fact
| Postgres compatible.
| gedy wrote:
| Is this the same situation that came up with Elasticsearch?
| dhd415 wrote:
| It's a long story, but not exactly. Elasticsearch was
| developed independently by Shay Banon in 2010 and a couple
| others who are the founders of Elastic
| (http://www.elastic.co) under the Apache2 open source
| license. In March of 2015, Elastic acquired Found, a
| company offering cloud-hosted managed Elasticsearch
| clusters which became their "Elastic Cloud" product. In
| October of 2015, AWS launched their own managed
| Elasticsearch service called "AWS Elasticsearch Service".
| Werner Vogels announced it in a now-deleted tweet saying
| that it was a "great partnership between @elastic and #AWS"
| when, in fact, there was no partnership at all (thing https
| ://www.theregister.com/2021/01/21/aws_not_ok_says_elast...)
| . In January of 2021, Elastic changed the license under
| which Elasticsearch was developed from Apache2 to the
| "Elastic" license which allowed essentially all the same
| things as Apache2 (free use, modification, etc) so long as
| it was not used to provide a managed offering of
| Elasticsearch (https://www.elastic.co/blog/licensing-
| change). This was done because Elastic viewed itself as
| developing the Elasticsearch product that AWS immediately
| took and used for its own "AWS Elasticsearch Service" in
| competition against Elastic's own Elastic Cloud service
| which was a significant source of revenue for Elastic.
| Additionally, Elastic sued AWS for trademark infringement
| for the use of the "Elasticsearch" trademark in the "AWS
| Elasticsearch Service" name. In response, AWS forked the
| last Apache2 release of Elasticsearch and named their fork
| "OpenSearch". They also changed the name of their managed
| offering to the "AWS OpenSearch Service".
|
| During the same timeframe, many other companies have seen
| the same risk of AWS taking their open source products and
| providing competing managed offerings and have chosen
| various methods of protecting themselves from AWS
| competition. Many of them (including MongoDB, MariaDB,
| Confluent, CockroachDB, Sentry.io, Apollo, Graylog,
| Couchbase) have adopted licenses such as the SSPL, BSL, and
| even the Elastic license that prevent or strongly
| discourage AWS from using their products in a managed
| offering. Others such as Grafana Labs have partnered with
| AWS (under undisclosed terms) for a managed offering. It's
| an ongoing tension between companies that want to offer an
| open source product with a managed offering that monetizes
| it.
| metadat wrote:
| Interesting, thanks for sharing this tidbit.
|
| For no particularly good reason, I was under the impression
| Postgres Aurora was just a Postgres compatibility interface
| layer on top of a common substrate shared with the MySQL
| Aurora implementation.
| js4ever wrote:
| It's not open source so we don't really know. But from what I
| read online it's a fork of postgres with a custom storage layer
| melony wrote:
| Dynamo underneath amirite?
| nkozyra wrote:
| Pardon my ignorance but isn't dynamo just a document store
| itself? Most storage engines are designed and optimized for
| RDBs, so I'd expect inefficiency here unless there's a
| layer between them.
| randtrain34 wrote:
| I think GP was being humorous
| smalltalks wrote:
| From one their talk the CTO hinted it was regular EBS.
|
| But for many other database such as "Amazon DocumentDB" ,
| "Time Series" humour suggest it's Dynamo.
| why-el wrote:
| What about RDS Proxy? I believe it's still only supports 13 which
| means if you use both the Proxy and Aurora you will have to wait?
| RDS Proxy does still mention only support for v13[1].
|
| [1]
| https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide...
| topher200 wrote:
| RDS Proxy is very specific with which versions it supports. If
| it says it doesn't support it, it doesn't support it.
|
| Postgres 14 is supposed to have built-in connection pooling. If
| you're on it, do you still feel the need for RDS Proxy? We're
| using pgbouncer and can't decide if we should switch to RDS
| Proxy or upgrade to Postgres 14 and drop the external
| connection pooler.
| sudhirj wrote:
| > Postgres 14 is supposed to have built-in connection
| pooling.
|
| Any references on this? I see a note somewhere about better
| connection handling, but nothing that would remove the need
| for connection pooling.
| mattashii wrote:
| > Postgres 14 is supposed to have built-in connection
| pooling.
|
| This is false. No PostgreSQL version has had a built-in
| connection pooler. There was a patch some years ago[0], but
| it didn't make it into PostgreSQL.
|
| [0] https://www.postgresql.org/message-
| id/flat/ac873432-31cf-d5e...
| [deleted]
___________________________________________________________________
(page generated 2022-07-10 23:01 UTC)