[HN Gopher] The Journey from WebSockets to HTTP Streams
___________________________________________________________________
The Journey from WebSockets to HTTP Streams
Author : owulveryck
Score : 40 points
Date : 2023-12-10 07:16 UTC (1 days ago)
(HTM) web link (blog.owulveryck.info)
(TXT) w3m dump (blog.owulveryck.info)
| freedomben wrote:
| This is a really great write-up!
|
| On a side note, the Remarkable Tablet is such a cool and hackable
| device. It surprises me that the company doesn't open source more
| of their client software to get help from the community, which I
| bet would do a large chunk of the work for them for free as it's
| mutually beneficial. Particularly as the market they are in is
| highly competitive, that could be a great way to get a great
| featureset, lower costs, while also providing something of
| immense value to the open source community. They could even offer
| a third-party app store of some sort and become the de-facto
| standard platform for e-ink devices! The rest are so locked down
| that this would be a huge competitive advantage.
| azeirah wrote:
| ReMarkable recently got a new CEO, so who knows what's next?
| https://goodereader.com/blog/remarkable-news/remarkable-is-g...
| stonogo wrote:
| I guess things didn't work out at Signifier?
| https://signifiermedical.com/2020/09/02/signifier-medical-
| te...
| MuffinFlavored wrote:
| Does "HTTP streams" mean "Server-Sent Events" here:
| https://developer.mozilla.org/en-US/docs/Web/API/Server-sent... ?
|
| I don't see it spelled out in the article.
|
| Edit: yes I do.
|
| > Internet and ChatGPT gave it a name: Server Sent Events
|
| Interesting callout: websockets can be "two way" (server ->
| client, client -> server). SSE, you just get one way (server ->
| client) and you have to do your typical POST to achieve the other
| way (client -> server)
| voidwtf wrote:
| I would advise anyone finding inspiration in this post against
| using a similar solution in the real world. While this may work
| in a local only environment, once you encounter the real world
| consisting of Cloudflare, DPI/MITM proxies, load balancers,
| zscaler, etc... it would most likely fail to work in this manner.
|
| Standards are at least somewhat supported among the former, often
| not even correctly. Any solution you choose I'd suggest
| implementing a fall-back to the most basic of http methods
| (standard get/post and polling for instance). You'll find all
| types of problems once you start encountering middleware that
| just absolutely thrashes your expectations of a well functioning
| internet.
| lantastic wrote:
| Interesting. Can you expand a bit on the kinds of problems that
| websockets would encounter with such middleware? I've seen a
| fair share of surprises on just TCP from DPI/MitM and high
| bandwidth-delay products, so I'm curious about ws analogues.
| voidwtf wrote:
| This article was not about using web sockets, it was about
| using a standard HTTP stream in a somewhat unorthodox way by
| never completing the request. Instead just beginning the
| response and occasionally "flush"ing data in the buffer to
| the client. The problem is that you'll often run into
| middleware that will do one of the following.
|
| I've personally encountered WAF proxies that will wait for
| the request to complete before sending any data to the
| client, that means that the client would appear to receive no
| data until the request timed out or the server completed the
| response.
|
| I've also encountered MITM/DPI and load-balancing proxies
| that will do the same, waiting until a buffer is filled
| and/or the response is completed.
| Matthias247 wrote:
| HTTP Streams (and Server Sent Events - which is a predefined
| body format that is understood among web browsers) are
| supported just fine by most infrastructure - probably even
| better than websockets.
|
| The advantage of them is that "they are just HTTP requests", so
| as long as your proxy actually can forward request and response
| bodies in a streaming fashion it will work. There's no need to
| understand the contents. It will not work if the proxy is
| implemented by waiting for the whole HTTP response to complete
| before forwarding the body. But that wouldn't work very well
| for a whole lot of other use-cases like file transfers, where
| buffering the whole body is not practical.
|
| A caveat is that most proxies won't allow for indefinite body
| body streaming, since they want to avoid all the TCP
| connections being tied up, so it's likely that the streaming
| would be interrupted at some time. In that case a reconnect
| logic would be necessary. But same applies to websockets.
| paulgb wrote:
| > In that case a reconnect logic would be necessary. But same
| applies to websockets.
|
| Arguably you're even better off in the SSE case, because it
| specifies a reconnect mechanism that allows clients to
| reconnect without receiving duplicate data. If you're working
| with a client library that understands this (which includes
| any to-spec browser-based client), you just need to handle
| the reconnect header on the server and you get reconnects for
| "free".
| voidwtf wrote:
| He is not using SSE, he is writing to the stream of an
| open/incomplete http response. The caveat you name is exactly
| why I said this solution is one I'd avoid. You're also not
| taking into account several other pieces of middleware that
| expect a whole response before forwarding it on.
| pyrolistical wrote:
| When using server sent events, the only issue I worry about is
| short tcp timeouts. If either of end terminates the tcp
| connection, you have to restart the event stream. If the
| timeout is low, this will basically be inefficient polling.
|
| But with that said something that forcefully terminates your
| tcp connection with a really short timeout is making the
| internet generally unusable, so it's something very unlikely to
| happen.
| voidwtf wrote:
| He did not use SSE as I read it, but a home grown solution
| writing to the response stream of standard request.
___________________________________________________________________
(page generated 2023-12-11 23:01 UTC)