[HN Gopher] HTTP vs. WebSockets: Which one is the fastest for Po...
___________________________________________________________________
HTTP vs. WebSockets: Which one is the fastest for Postgres queries
at the edge?
Author : nikita
Score : 86 points
Date : 2023-07-11 15:06 UTC (7 hours ago)
(HTM) web link (neon.tech)
(TXT) w3m dump (neon.tech)
| l5870uoo9y wrote:
| Here is the actual package used [1]. I thought it would have been
| similar to serverless-pg[2] with initiation outside the handler
| as with the node implementation. Any insights into how they
| achieve connection caching (similar to pooling?) on the edge?
|
| [1]: https://github.com/neondatabase/serverless [2]:
| https://github.com/MatteoGioioso/serverless-pg
| raoufchebri wrote:
| Connection caching is achieved on proxy side. Edge <->
| Proxy<->Compute(Postgres)
| raoufchebri wrote:
| Thanks, I didn't know about serverless-pg. I'll have a look.
| bluefishinit wrote:
| It seems like if speed was a concern you wouldn't make the extra
| hop from a serverless function to their websocket server, then on
| to the db. Instead you could serve your api directly off the app
| server which would then connect to the db without the
| intermediary step.
| gmac wrote:
| Every connection to a Neon database (traditional TCP,
| WebSockets, https) now goes via the same custom (open-source)
| proxy server that's close to the database compute node.
|
| So there's technically an additional hop irrespective of the
| transport type, but not one that should add meaningfully to the
| latency, or that would make WebSockets slower than the others.
| [deleted]
| bob1029 wrote:
| The diagram is confusing me too. We are using Az Functions to
| serve webapps via HTTP triggers and they just take a direct
| connection to SQL. Our average trip to the database is ~5
| milliseconds for the trivial views. No web sockets, middleware,
| etc. What value is added with this extra
| middleware/ceremony/etc?
| gmac wrote:
| Right. If TCP is available, any common-or-garden Postgres
| driver will do (for Neon or any other Postgres DB). But not
| all serverless compute environments support TCP connections.
| raoufchebri wrote:
| We use a proxy that redirects the request to the compute node.
| The proxy uses WebSockets in this case.
| https://neon.tech/blog/serverless-driver-for-postgres
| PaulWaldman wrote:
| Is there anything like this to expose Postgres over gRPC?
| tristan957 wrote:
| How popular is gRPC in a serverless world?
| raoufchebri wrote:
| Hey @PaulWaldman, can you tell us more about your use case with
| gRPC and Postgres?
| PaulWaldman wrote:
| Performant streaming of structured telemetry data from Edge
| networks. Edge in this case meaning on-prem installations,
| not necessarily from a remote DC. These networks have limited
| network bandwidth and potentially intermittent connectivity,
| minimizing the message size and overhead as much as possible
| is ideal.
| rubenfiszel wrote:
| What I would absolutely love is to have a proxy server that would
| act as postgres, and use this new driver to connect to a remote
| postgres. This would avoid us having to rewrite our remote agents
| (which require a postgres connection to the main Database) to use
| an http api. Instead, we could just keep them as it is, using
| rust's sqlx queries, point them to the local proxy. Essentially
| pgBouncer but through http.
|
| Postgres <> Neon HTTP Proxy <---HTTP---> Local Fake Postgres
| Server <--unix socket--> Agents
|
| We need it for the open-source project windmill.dev and we will
| build this if no one else does it.
| gmac wrote:
| This sounds cool (but it's not something we've built at Neon).
|
| In case it's useful, I can clarify:
|
| * Every Neon database is accessible over an ordinary TCP
| Postgres connection (as well as WebSockets and http).
|
| * Our JavaScript serverless driver is based on node-postgres
| and is a drop-in replacement for it. So for JS, no rewriting is
| needed.
|
| Assuming that doesn't help, do you specifically need http
| (rather than WebSockets), and a proxy that speaks the pg
| protocol?
|
| If not, the simplest thing might be to (1) use Neon, or run a
| WebSocket proxy next to your own database (e.g.
| https://github.com/neondatabase/wsproxy); and (2) find/write a
| similar WebSocket proxy, but in reverse -- i.e. one that
| listens out for TCP/socket connections and tunnels them over
| WebSockets -- to run next to the agents.
| rubenfiszel wrote:
| Windmill is written in Rust so cannot use the javascript
| serverless driver. I love sqlx and sqlx would not be easy to
| replace so short of making it an extension for sqlx, the only
| route is to make believe the sqlx connection pool that it's
| talking to a pg server, when it's actually just talking to
| the local proxy.
|
| If we were using javascript/typescript, then yes we would
| probably just be able to use that library.
|
| > (2) find/write a similar WebSocket proxy, but in reverse --
| i.e. one that listens out for TCP/socket connections and
| tunnels them over WebSockets -- to run next to the agents.
|
| It's that part that I would love to see. My assumption is
| that this proxy next to my agents would need to speak pg
| protocol and then transcribe it to wss. If there is a simpler
| way, let me know :)
| habibur wrote:
| First check if that can be done with a port forwarder or socks
| proxy.
| KingMob wrote:
| Cool article, thanks for sharing.
|
| You don't mention which version of HTTP you were using, which
| makes some of my comments here conjecture, but if I had to guess,
| I'd say you were using HTTP/1.1. Is that right?
|
| It would really help if you could break down where the time is
| being spent. I'd love to know what fraction of the time is spent
| in processing, transit, querying the db, etc.
|
| Did you consider HTTP/2 or 3? I would expect them to have lower
| setup overhead, and be more competitive with Websockets.
|
| In particular, if you were using HTTP/2+, did you look into
| CONNECT proxying? It could replace Websockets while having lower
| connection setup overhead, albeit at the cost of a little payload
| unwrapping overhead in the proxy. But theoretically, this could
| offer both the faster startup of HTTP/2+, combined with the lower
| processing overhead for the db of a TCP stream. Comparing the
| graphs of 10 queries, it _appears_ that processing HTTP imposes a
| ~15 ms overhead over raw bytes. (Theoretically, the CONNECT proxy
| has to obey the host and port in the :authority pseudoheader, but
| in your case, the proxy would just ignore it, and hook up the
| edge function to whatever compute node it pleased.)
| gmac wrote:
| Yes, we're currently on HTTP/1.1, but I'm sure we'll be looking
| at later versions. Unfortunately the Rust web server library
| we're using doesn't currently support HTTP/3, but we could
| potentially e.g. put an nginx proxy in front of it for this
| purpose.
|
| A key point of our WebSocket and http transports is to support
| environments where TCP connections aren't available. Are there
| environments that would let us send and receive binary data
| following an http CONNECT that wouldn't simply allow an
| ordinary TCP connection?
| toomim wrote:
| Try HTTP/2. You don't need /3.
| [deleted]
| raoufchebri wrote:
| > I'd love to know what fraction of the time is spent in
| processing, transit, querying the db, etc.
|
| We experimented with SELECT 1, so most of the latency is
| transit
| kelvich wrote:
| Yes, exploring http/3 is on the roadmap. One of the learnings
| for us was the fact that v8 is quite good with caching open TCP
| connection; For consecutive queries the same TCP+TLS session is
| being reused. Even when queries are sent from unrelated V8
| isolates.
| VWWHFSfQ wrote:
| Are you saying that V8 is reusing TCP connections between
| different browser tabs and websites? Seems like a
| timing/side-channel attack could be done with that.
| gmac wrote:
| No, this is in a serverless environment running on V8
| isolates (specifically, Vercel Edge Functions).
| VWWHFSfQ wrote:
| Oh I see. Not Chrome v8
| raoufchebri wrote:
| Blog post author here. Let me know if you have any questions!
| com2kid wrote:
| The axis on all your graphs are unlabeled and have the same
| scale in both the X and the Y, accordingly I am not sure
| exactly what is being shown on the graph.
| gmac wrote:
| Fair point! We'll fix this. In the meantime, the graphs show
| latency distributions, so the X axis is time (in
| milliseconds) and the Y axis is a simple count.
| raoufchebri wrote:
| Thank you. We fixed it now.
| FlyingSnake wrote:
| Pardon my ignorance as I'm not aware of these recent
| developments, but what is a valid use case for this? Why would
| I put my database into an ephemeral serverless DB[1]?
|
| 1: https://news.ycombinator.com/item?id=36684121
| gmac wrote:
| Well, the data is absolutely not ephemeral: it's safely
| tucked away in S3.
|
| But to crib from https://neon.tech, you might choose Neon
| because it's scalable, cost-effective, and easy to deploy and
| manage. Or you might choose it for its copy-on-write
| branching and point-in-time recovery, all open-source, and
| all on top of the solid, performant, feature-rich foundation
| that is Postgres. :)
| tristan957 wrote:
| To follow up on Raouf's comment, here is a blog post[0] that
| describes Neon's architecture in more detail.
|
| [0]: https://neon.tech/blog/architecture-decisions-in-neon
| raoufchebri wrote:
| Neon separates storage and compute. The compute part is
| serverless and can scale up and down on demand. The storage
| has multiple layers: Safekeepers for durability and
| Pageservers to reconstruct pages from WAL.
|
| The data is also offloaded to S3. Here is a diagram that
| describes the architecture:
| https://neon.tech/_next/image?url=https%3A%2F%2Fneon-
| hwp.dre...
| lfmunoz4 wrote:
| [dead]
| Lichtso wrote:
| I assume by "http" you mean http1.1 and http2, not http3? Would
| be nice to differentiate these in the testing as well.
|
| WebSockets still seem to be stuck with TCP as rfc9220 [0] is
| not implemented I think.
|
| [0]: https://datatracker.ietf.org/doc/rfc9220/
| raoufchebri wrote:
| Yeah! these are just normal websockets. We're definitely
| interested in supporting and testing more protocols and ways
| of accessing postgres and neon as we can. We're also super
| excited to see the development and adoption of new protocols!
| redm wrote:
| @raoufchebri & @nikita, What about comparisons to CloudFlare
| Workers using Workers Sock API to connect to PG? [1] I know
| CloudFlare workers support Neon directly through API so how does
| that compare to the alternative PG implementations?
|
| [1]
| https://developers.cloudflare.com/workers/learning/integrati...
| raoufchebri wrote:
| We have only compared on Vercel Edge and Serverless
| environments. I think it makes a lot of sense to compare our
| driver to pg on CF with connect.
| CuriouslyC wrote:
| The biggest selling point of websockets over http is that you can
| stream the results back easily and quickly, which enables stuff
| like canceling slow queries easily, a transfer progress bar, etc.
| Dhruva23 wrote:
| [dead]
| nikita wrote:
| CEO of Neon here. We have done a lot of latency work. Feel free
| to ask questions.
| dathinab wrote:
| But why?
|
| The edge is in general not the users browser.
|
| So you can do a TLS/postgres connection instead of adding
| additional layers.
|
| Sure you want to probably run a patched pg client/server which
| has various optimizations to reduce the number of round trips
| when opening a connection, but that also applies to any
| HTTP/WebSocket wrapper.
| gmac wrote:
| > So you can do a TLS/postgres connection instead of adding
| additional layers.
|
| Sure, sometimes you can. But a key use-case for a serverless
| database is connecting from a serverless compute environment,
| and not all serverless compute environments support TCP
| connections.
|
| That's why Neon offers this serverless driver, which supports
| WebSockets and now also http.
___________________________________________________________________
(page generated 2023-07-11 23:02 UTC)