[HN Gopher] Quicker serverless Postgres connections
___________________________________________________________________
Quicker serverless Postgres connections
Author : nikita
Score : 127 points
Date : 2023-03-28 15:17 UTC (7 hours ago)
(HTM) web link (neon.tech)
(TXT) w3m dump (neon.tech)
| boundlessdreamz wrote:
| I didn't know that neon supports using S3 as cold storage. That's
| seriously amazing. How does that work with indexes and queries?
| It's there a single index which will always grow and stored on
| disk or is the index split?
| nikita wrote:
| The storage works at the page level. Pageservers provide low
| latency reads and safekeepers low latency writes. Then we move
| cold data to S3. Think about safekeepers + S3 being the system
| of record for most recent + historical data. And pageservers is
| a scratch space for updating data and service reads. Pages are
| also on compute nodes.
|
| The problem that S3 doesn't love small files. So we organize
| pages in LSM trees on the pageservers and offload layers to S3.
| jrochkind1 wrote:
| > The problem that S3 doesn't love small files.
|
| I hadn't heard it before and am not finding references
| googling.
|
| I'm curious what problems you have with small files on S3; or
| if someone wants to feel free to point me to a link to
| discussion of this apparently known fact!
| nikita wrote:
| Small as in 8kb page. A sizable Postgres database would
| have millions of them.
|
| Lots of stuff on the web. Quick googling found:
| https://www.upsolver.com/blog/small-file-problem-s3
| zinclozenge wrote:
| Do you guys do compaction on the levels that live in S3?
| nikita wrote:
| Compaction is performed only by pageserver and only for L0
| layers. But L0 layers can be offloaded to S3 and loaded
| back. So we actually perform compaction of S3 layers (but
| we download them to the pageserver).
| janpio wrote:
| That is some serious optimization. Nice!
|
| The article ends with the statement that you are pretty much done
| here for now. Would optimizing your TLS termination not maybe
| offer some more ways to speed this up? Or is that also already
| fully optimized?
|
| I did not realize before that your approach with Websockets
| actually meant that there was no application/client side pooling
| of connections. What made you choose this approach over an HTTP
| API (as for example PlanetScale did) anyway?
| kelvich wrote:
| > The article ends with the statement that you are pretty much
| done here for now. Would optimizing your TLS termination not
| maybe offer some more ways to speed this up? Or is that also
| already fully optimized?
|
| No, we don't do early termination yet, but it makes sense to
| try it out too. Here we mostly concentrated on how far we can
| get in terms of reducing number of round-trips.
|
| > I did not realize before that your approach with Websockets
| actually meant that there was no application/client side
| pooling of connections. What made you choose this approach over
| an HTTP API (as for example PlanetScale did) anyway?
|
| To keep compatibility with current code using postgres.js.
| janpio wrote:
| > To keep compatibility with current code using postgres.js.
|
| That makes a lot of sense - not needing an additional
| driver/client package is indeed a good point. Any plans to
| add a HTTP based API though anyway?
| nikita wrote:
| Potentially. We will follow what our users will ask us to
| do.
| 0xbadcafebee wrote:
| With a persistent OS-level network tunnel (ex. wireguard) that
| exists when the app starts, you wouldn't need HTTP, WebSockets,
| TLS, etc. The app could open a single TCP connection to a sql
| connection pool and send its request, and it would be tunneled
| securely from point to point, with no additional round trips or
| protocols.
|
| The idealist in me would rather this kind of thing become a
| standardized extension of address + transport protocols. But
| every person I talk to would rather stack 12 protocols on top of
| each other than work on improving the existing status quo. It
| feels like a truism of human societies; everyone wants
| improvement, but nobody wants fundamental reform.
| nikita wrote:
| I'm CEO of Neon. Happy to answer any questions you might have.
| codegeek wrote:
| How do you compare against planetscale (I understand that they
| are mysql vs you postgres but I am looking for serverless db
| right now so business/product/support comparison is important)
| manigandham wrote:
| Planetscale[1] is MySQL that is automatically sharded and
| proxied with a system called Vitess to scale horizontally.
| Because it's real MySQL, most functionality still works
| without the operations overhead.
|
| Neon[2] is Postgres that has a surgically enhanced data layer
| that reads from a distributed set of nodes and object
| storage, but is still a single Postgres instance that's
| started and stopped on-demand and not automatically sharded.
| Because it's real Postgres, all functionality works including
| extensions without dealing with issues from sharded datasets,
| but also without the horizontally scalability and instant
| start (so far). However because the data layer is improved,
| the actual compute node is much more efficient in both
| startup and processing so it works well in many scenarios.
|
| CockroachDB[3] is a proprietary distributed database built to
| Postgres wire/data protocol with natively separated storage
| and compute. Their serverless plan also starts and stops
| instances as traffic comes in but with fast startup because
| of their specific architecture. Because it's not real
| Postgres though, there's a good bit of missing functionality.
|
| There's also TiDB[4] from PingCap which is similar to a MySQL
| version of CockroachDB, and Yugabyte[5] which is somewhat
| like Neon but with both distributed compute and data layers
| also using real Postgres components - however neither has a
| serverless offering.
|
| 1. https://planetscale.com/ 2. https://neon.tech/ 3.
| https://www.cockroachlabs.com/ 4.
| https://www.pingcap.com/tidb/ 5. https://www.yugabyte.com/
| Normal_gaussian wrote:
| I have a non-tech business approaching limited release. I'm
| very interested in two things:
|
| - Data safety and privacy. As I'm UK based - can my data be
| restricted to the UK only (no magic round the world trips
| inside your backend)? do you comply with the GDPR?
|
| - Can I set up postgres (WAL) replication to a non-neon
| database?
| evangow wrote:
| Any plans to introduce a way to connect to a Neon database
| without making it publicly accessible via the internet?
| MuffinFlavored wrote:
| how would they do this? allow it to be added to a wireguard
| VPN or something?
| koolba wrote:
| Or within AWS world you can do VPC peering.
| nikita wrote:
| How about private link?
| nikita wrote:
| Yes. What's your use case? Security or egress costs (those
| are 0 if you are in the same region).
| evangow wrote:
| Security
| nikita wrote:
| We are looking at private link for this. Will this work
| for you?
| evangow wrote:
| I haven't used PrivateLink before, but from my brief
| look, it looks like it would work.
| irq-1 wrote:
| Why do web services not discuss the metrics that were common
| for cloud servers? Like average response time and distribution,
| throughput, error rates, etc... and what can you tell us about
| neon?
| r3trohack3r wrote:
| Neon looks like everything I've ever wanted out of a serverless
| database. I send a vendor pg compatible queries, vendor takes
| care of everything, charges me for usage - and connections work
| with 3rd party serverless runtimes.
|
| I have a number of workloads I'd migrate to you over the next
| few weeks if you're ready for it. What is the state of Neon?
|
| Is it ready for small-volume but business critical loads?
|
| I'm struggling to find support guarantees and SLAs on your
| website, do you have them yet?
|
| What is your off-site backup story? Can I export my backups to
| something like R2, S3, or B2?
|
| Many of my customers are running in Cloudflare Workers. Their
| volume is low, but it's business critical (DB downtime means
| business downtime - business will fail if we lose all the
| data).
| nikita wrote:
| > I have a number of workloads I'd migrate to you over the
| next few weeks if you're ready for it. What is the state of
| Neon? Please do we will support it! We have 22K hosted
| databases on the platform and are quite stable.
|
| > Is it ready for small-volume but business critical loads?
| Depends on business criticality. But generally yes. We are
| debating internally when we are going to announce it.
|
| > I'm struggling to find support guarantees and SLAs on your
| website, do you have them yet? Good feedback. It's in the
| works.
|
| > What is your off-site backup story? Can I export my backups
| to something like R2, S3, or B2? Since our storage is
| integrated with s3 and supports branching we can treat
| branches as backups. > Many of my customers are running in
| Cloudflare Workers. Their volume is low, but it's business
| critical (DB downtime means business downtime - business will
| fail if we lose all the data). We haven't lost data yet.
| There are a lot of redundancy in the system. Safekeepers, S3,
| branches (backups).
| sebk wrote:
| I'm glad to have an answer to the question I asked you 7 months
| ago: https://news.ycombinator.com/item?id=32472443 ( _What 's
| Neon's point of view about transient state in nodes? Is there a
| world where serverless client connections are stateless, or is
| the set up overhead not expected to be worth the cost?_).
|
| This is a good analysis of network time and optimization. I'd
| love to see a followup exploring the impact on the server side;
| Considering how Postgres was historically not very efficient at
| setting up new connections but has been steadily getting better
| with each major release. It would also be great to see what
| portion of the request is taken up by the setup versus the
| query itself, to have some idea of cost.
| amerine wrote:
| Is your geo-distributed read replicas going to lean on WAL
| replication?
| nikita wrote:
| Yes, we are working on this at the moment. Since we control
| storage we will be implementing this at the storage level.
|
| From there we are thinking to expose
| <project_id>.read.neon.tech and <project_id>.write.neon.tech
| and for read queries route traffic to the local replica.
|
| It's not set in stone yet, but that's the gist of it.
| amerine wrote:
| Excellent. That work's wonderfully for us too ;-)
| slig wrote:
| Due to latency, Neon is only for AWS/GCP/Azure datacenters or
| can we use it from smaller providers, such as DO/Linode/etc?
| nikita wrote:
| You can use in anywhere. We will keep working to drive GLOBAL
| latency down. As in connect to Neon from anywhere.
| carlsverre wrote:
| Pretty insightful observation on authentication. Using a password
| method that requires the client to compute a brute force
| resistant hash that takes ~100ms of cpu time is a non-starter for
| most serverless platforms.
|
| I'm curious if Cloudflare, or other serverless platforms offer
| the ability to generate short lived JWTs assigned to each worker.
| Combined with some configuration (pub key exchange, claim setup,
| etc) a platform like Neon could use these JWTs to establish
| identity. Sorta SSO for workers without the CPU overhead.
|
| Seems like a safer approach than basic auth with fixed pws.
| nikita wrote:
| We are working with the Cloudflare team and see if they can
| build it. This is a great idea.
| yamtaddle wrote:
| Since serverless is basically just the PHP model with more
| languages (so: also basically just CGI) I'm waiting for them to
| balance clients onto as few machines as possible in each DC and
| add a (revolutionary! State-of-the-art!) connection-pooling
| agent to each one.
|
| I reckon the current hype-cycle will be just about at its peak,
| when that happens.
|
| (FWIW I like serverless--but then, I liked CGI, so of course I
| do)
| kentonv wrote:
| In Cloudflare Workers we like to represent things like this
| with "bindings". A binding is like an environment variable,
| except instead of containing a string, it contains a live
| object with methods that access the remote resource.
| Authentication is baked into this object, so the Worker code
| itself never has to see an API key at all. For example, if you
| have a Worker configured to use a KV namespace, you can write
| code like: let value = await
| env.MY_KV.get(keyName);
|
| No setup needed, at least in code. You create the binding
| either through the configuration UI or API.
|
| So far we've mostly used this technique to connect Workers to
| other Cloudflare-provided services like Workers KV, but I'm
| super-interested in the idea of third-party bindings.
| Hopefully, you'd be able to configure them through an OAuth-
| like flow, where the Cloudflare dashboard redirects you to the
| third-party service, that service prompts you for permission,
| then redirects back to Cloudflare, and you never have to
| copy/paste a single secret.
|
| Of course, under the hood this would all be backed up by strong
| authentication, but it's high time we stop making application
| developers waste time thinking about this stuff.
|
| (I'm the tech lead for Workers.)
| carlsverre wrote:
| Heck yes - this is the way. Make this available and you will
| unlock an entirely new ecosystem of CF worker plugins. A++
| grrdotcloud wrote:
| My new hero, please update the documentation on the workers,
| bindings, and examples.
|
| I can't cite examples while I'm being ushered by a distracted
| Uber driver but this explanation goes beyond the documents.
|
| Also, please extend Cloud Workers to be written as NanoVMs.
| tyingq wrote:
| Postgres also supports auth via SSL client certs, so that's
| perhaps another option. I don't know how well that maps to apis
| in Cloudflare and the server side of Neon.
| carlsverre wrote:
| This is a good idea... how much compute does SSL client cert
| auth require on the client side? I would suppose it has to
| produce a signature of some kind, perhaps signing a challenge
| from the server?
| [deleted]
| chatmasta wrote:
| This is basically how we do authentication in the Splitgraph
| DDN [0], which is kind of like a multi-tenant serverless
| Postgres.
|
| We implement the Postgres frontend with a forked version of
| PgBouncer, and we changed the authentication method such that
| when the user authenticates, we issue them a JWT which we store
| as a session variable. That session variable has the same
| security properties as a cookie in a web browser (the user can
| change/manipulate it, but if it's signed by us we can trust its
| claims).
|
| That's the simple explanation that skips over the multi-tenant
| part. I don't want to derail from the thread - Neon is very
| cool, and we are actually experimenting with it right now, for
| storing the Seafowl [1] catalog [2] when deploying to "scale to
| zero" services like Google Cloud Run or AWS Lambda, which don't
| have persistent storage.
|
| [0] https://www.splitgraph.com/connect/query
|
| [1] https://seafowl.io
|
| [2] https://seafowl.io/docs/learning/architecture
| armatav wrote:
| So how do you do development with this?
|
| Is the recommendation to have a per-dev branch off the main DB?
| kelvich wrote:
| Main idea was one branch per pull request to better test
| migrations on a fresh data. But one branch per dev or branch
| per dev per pr will also work.
| kccqzy wrote:
| > A key feature of these environments is that state is not
| generally persisted from one request to the next. That means we
| can't use standard client-side database connection pooling.
|
| So we introduced so many optimizations just because we can't
| persist state. I can't help but think this is being penny wise
| pound foolish; the problems this article is solving wouldn't have
| been problems in the first place if you choose a boring
| architecture.
| nikita wrote:
| Serverless environments are a matter of fact. They exist, they
| are super convenient, and people want to use Postgres in them.
| This is a bridge for such people. Yes, it requires some
| sophisticated engineering, but it also enables to have less
| boilerplate code for a lot of people.
| stickfigure wrote:
| I've been doing "serverless" on app engine for more than a
| decade and it handles postgres connections fine without any
| heroics.
|
| There's always instances with state somewhere; it's just a
| question of how smart your serverless infrastructure is about
| pooling the instances.
| Cthulhu_ wrote:
| I think if this particular logic is hidden away behind a
| server, it doesn't really matter - people want convenience,
| and for someone else to hide the complexity.
|
| Serverless postgres-as-a-service isn't a stretch at all.
|
| This thread does remind me of a previous thread, I believe it
| was something like optimizing a basic string operation
| (concatenation?) by going down to assembly and SIMD. Few
| people have a need to know the details of that particular
| implementation, and mere mortals cannot comprehend the
| wizardry going on there either, but that's just fine because
| if it gets incorporated into standard libraries or the
| underlying implementations that said standard libraries use,
| everybody benefits without ever having to understand the
| inner workings.
| manigandham wrote:
| Also known as layers of abstractions. The entirety of
| (software) technology is built on that concept.
| interstice wrote:
| Unless it breaks one day and no one knows why
| jrsj wrote:
| Things are getting more serverlesss too. With Deno Deploy,
| Cloudflare Workers, etc I see myself eventually wanting to
| run _most_ of my application on platforms like this, and a
| serverless & distributed database makes that _much_ easier
| and I expect it'll probably perform better in many cases too.
| nikita wrote:
| 100%
| chmod600 wrote:
| Boring architectures already exist. Why reimplement what's
| already done?
| insanitybit wrote:
| As if boring architectures don't have their own problems.
| Cthulhu_ wrote:
| Such as? Besides of course not being able to find people that
| want to work on it.
| endisneigh wrote:
| Boring architectures generally don't scale well
| yamtaddle wrote:
| I usually find that "interesting" architectures scale
| even worse, at least for the 99% of projects and
| organizations that'd never actually outgrow the "boring"
| solution. All the pain of the big fancy solution, but
| without the actual need that makes it truly better than
| the alternative.
|
| ... of course, the best-paying jobs are at the places
| that _really do_ need that scale, so of course we all
| want an excuse to play with those tools, even when it 's
| not the right business choice. And it's not as if
| software folks are the only ones with a principal-agent
| problem, in business. Oh well.
| manigandham wrote:
| Being boring has nothing to do with scale. Some of the
| biggest applications run on incredibly simple
| architectures, and often are able to scale so well
| because of that (boring) simplicity.
| endisneigh wrote:
| What are the biggest apps on the simplest architectures
| and what are the architectures?
| fabian2k wrote:
| Stack Overflow is probably one of the more public
| examples. That is mostly a C# monolith with a relational
| database behind it.
| manigandham wrote:
| Monolithic apps running on big servers talking to
| databases running on big servers. Very little
| orchestration overhead or complexity with serverless and
| the rest.
|
| Everything from StackOverflow to ad serving systems to
| high-frequency trading exchanges are run this way.
| endisneigh wrote:
| Stack overflow is unique in that it's most read only.
| Link to high frequency trading "boring" architecture?
| manigandham wrote:
| What is your definition of non-boring architecture? I'm
| curious as to what you consider the exciting
| architectures necessary for scaling to make sure we're
| talking about the same thing.
|
| Quora, Pinterest, Twitter, etc are all just big app
| instances talking to DB instances, with separate systems
| for background processing, queues, and caching. Are you
| suggesting that they would scale better with serverless
| functions instead?
|
| Here's a list of architectures:
| http://highscalability.com/all-time-favorites/
| leetbulb wrote:
| Ironic that this page is down? (also not https? _heh_ )
| SebJansen wrote:
| Cassandra is probably in the realm 'boring and scalable'
| manigandham wrote:
| Definitely not. I'd much rather scale Postgres manually
| through layers of sharding and proxies than handle
| Cassandra.
|
| There is ScyllaDB for a much better reimplementation of
| Cassandra/Dynamo, but wide-column databases are still
| best for niche scenarios, especially as RDBMS are rapidly
| evolving into natively distributed architectures.
| spullara wrote:
| This isn't just useful for serverless. Any process-per-request
| based architecture can also benefit (PHP, Ruby, etc).
| spapas82 wrote:
| A simple optimization in these cases is to keep the connection
| open (see https://docs.djangoproject.com/en/4.1/ref/databases/#
| persist...)
| throwaway290 wrote:
| To spare someone a click, yes there is still a postgres server.
|
| Can't stop being mad at the misuse of "serverless". Even worse
| than using crypto for cryptocurrencies. Words have meaning...
| halayli wrote:
| And what's your definition of serverless?
| chungy wrote:
| At least there are cryptographic functions backing
| cryptocurrency. It's not _as_ inaccurate as "serverless".
| manigandham wrote:
| It is a terrible term but in this case it's referring to the
| client application, not the database instance.
___________________________________________________________________
(page generated 2023-03-28 23:01 UTC)