[HN Gopher] Tcpls: Modern Transport Services with TCP and TLS
___________________________________________________________________
Tcpls: Modern Transport Services with TCP and TLS
Author : enz
Score : 77 points
Date : 2022-06-05 13:27 UTC (9 hours ago)
(HTM) web link (blog.apnic.net)
(TXT) w3m dump (blog.apnic.net)
| a-dub wrote:
| interesting debate whether encryption should occur at the network
| vs. transport layer. on the one hand, encryption tends to have
| session state, so the transport layer makes sense but on the
| other i often feel i'd like to see a new version of IP where
| encryption is baked in as a primary design principle thereby
| eliminating all notions of leaks.
| zamadatix wrote:
| IPsec tunnel mode delivers that function
| https://datatracker.ietf.org/doc/html/rfc4303
|
| This contrasts to IPsec transport mode which only encrypts the
| payload not the IP info
| https://upload.wikimedia.org/wikipedia/commons/6/64/Ipsec-es...
|
| Generally there is encryption available at each layer in the
| network stack but middleware (operating systems, firewalls,
| NAT) tend to get in the way at lower layers. E.g. IPsec is
| often now run with NAT traversal which means the ESP header is
| stuck under a UDP header so it can get through to end devices
| even though IPsec itself doesn't require this. This, and a few
| other things, have left IPsec as a thing most people only know
| for corporate VPNs.
| badrabbit wrote:
| TCP is terrible at long distance data transfer. I am sure some
| people will still prefer it since it can manage congestion but
| shouldn't newer things use UDP and handle delivery guarantees at
| the application layer?
| gsliepen wrote:
| There is nothing inherently wrong with modern TCP for long
| distance data transfer. You do need to use the right congestion
| algorithm and some extensions enabled to handle large window
| sizes to get high speeds at long distances. But UDP does not
| magically solve anything; if you want reliable data transfer
| over UDP you have to basically implement almost everything TCP
| does on top of it. The main benefit is that you are no longer
| relying on the operating system's implementation of TCP, the
| drawback is that you won't get any help from the OS or even
| hardware.
| badrabbit wrote:
| I thought you can't keep increasing window sizes, it is a
| physics limit? The mere fact that you need to wait for an ACK
| is the problem?
|
| https://en.m.wikipedia.org/wiki/Bandwidth-delay_product
| zinekeller wrote:
| As GP said:
|
| > if you want reliable data transfer over UDP you have to
| basically implement almost everything TCP does on top of
| it.
|
| Even QUIC has an ACK-like signal so you will never avoid it
| as otherwise you will deal with the two generals' problem
| (https://en.wikipedia.org/wiki/Two_Generals%27_Problem).
| The reason QUIC is more efficient than TCP is simply due to
| the fact that TCP can't evolve further due to middleboxes
| (called ossification,
| https://en.wikipedia.org/wiki/Protocol_ossification), not
| that TCP is inherently bad.
| dilyevsky wrote:
| With scaling tcp can use window up to 1GB. Which is enough
| for 1s rtt which is ridiculously high @ 8gbps which is also
| ridiculous for long haul on a single flow. In practice you
| probably almost always just parallelize into multiple flows
| Matthias247 wrote:
| The congestion control algorithms that are applied in TCP and
| QUIC are actually even the same ones (e.g. Cubic, BBR, etc),
| so the technologies behave not too different over higher
| RTTs. Both will ramp up a congestion control window, which
| can get pretty big.
|
| There are differences in behavior regarding what happens if
| loss is encountered and what exactly is retransmitted, but
| those still don't make QUIC radically different in terms of
| behavior and performance.
| scarmig wrote:
| https://dl.acm.org/doi/10.1145/3485983.3494865 seems like a more
| understandable source.
| metadat wrote:
| This link was much easier for me to read and parse, thank you
| scarming.
| simmervigor wrote:
| Some related discussion in this Twitter thread:
|
| https://twitter.com/alagoutte/status/1532013841718120449?t=W...
| Matthias247 wrote:
| Some thoughts from someone working on QUIC for a couple of years:
|
| There's some good ideas in this one! The idea of just keeping a
| TCP stream per Tcpls stream while sharing some crypto state means
| it can be really efficient, since all optimizations that exist
| for TCP (including hardware offloads) will continue to work. That
| is also visible in the efficiency benchmarks [1] on the site.
|
| One might even argue that with such a design where the most
| expensive parts of TLS (handshakes) become no-ops an introduce no
| additional latency, techniques like HTTP/2 become unnecessary -
| one could keep just having a single TCP stream per request and
| enjoy better flow control, fairness and observability than with
| anything application managed.
|
| However there are some challenges that need to be resolved for
| running protocols on "real-world infrastructure and datacenters",
| which change the picture a bit. One is that a single IP address
| announced by large-scale webservices doesn't necessarily point to
| a single server, but to e.g. a rack of them or even bigger units.
| This means that a second TCP connection might (or even should be)
| established to a different server - which would not be aware
| about the TLS session state. If the state is kept on a different
| server, the original one would need to forward all packets either
| via IP-in-IP encapsulation or by creating a separate upstream TCP
| connection and proxying all data bidirectionally. All those
| approaches would be fairly expensive - every time a TLS session
| would be joined the amount of packets handled by a server would
| be double of the ideal version.
|
| While a similar challenge exists for QUIC, it's actually cheaper
| on average: Since multiple streams use the same QUIC connection
| and usually also the same source/destination IP and port pairs,
| traffic would usually be directed to the same host by common
| infrastructure and require no special handling. It would only
| require effort to "fix" misdirected QUIC connections in the event
| of connection migrations - which are rather rare. If packet
| forwarding is required, it can be achieved rather cheap and
| easily by interpreting the QUIC connection ID fields in all UDP
| packets and doing raw UDP packet forwarding.
|
| Therefore as far as I understand the proposals so far, it might
| not be necessarily clear-cut on whether running QUIC or TCPLS
| would be cheaper and an ingress or load-balancing layer - which
| are the layers where both protocols have the biggest benefits due
| to the highest amount of concurrent connections and higher
| client-facing RTT.
|
| But maybe the challenge for TCPLS could be somehow solved by
| having a shared place for the TLS state between hosts and making
| sure streams are operating fully independently - but it requires
| a bit more thoughts.
|
| I'm wondering how close such a proposal is then to the existing
| TLS session resumption mechanisms - just extended by allowing
| more than 1 follow-up session.
|
| [1] Note that the diagrams are really about efficiency and not
| performance. They tell nothing about what throughput might be
| available for end-users that are e.g. 100ms away from the server.
| For local tests the delay and packet introduced by large-scale
| internet infrastructure doesn't play a role.
| gsliepen wrote:
| This is quite interesting. The only issue I see is when you want
| to stream real-time audio and video over TCPLS. That is when you
| want packets that are dropped, late or reordered to not interfere
| with the delivery of other packets. Using TCP as the underlying
| transport makes that problematic, and even having multiple TCP
| connections cannot help much with that.
| throwaway787544 wrote:
| I would appreciate it if the industry would stop trying to shove
| everything into user space. Imagine it's 40 years in the future,
| and we still have to kludge new protocols over an 80 year old
| TCP/IP stack because nobody wants to add a new protocol to the
| standard. Every application ships with its own unique protocol
| stack implementation. Duplicate code abounds. Bugs are fixed
| haphazardly. Quality and stability varies by application. RCEs
| are more common because an increase in implementations increases
| the probability of security bugs.
|
| Compare that to finalizing a new protocol, putting it in the
| kernel, and letting all applications just _connect()_ and
| _send()_ and _recv()_. We lose the ability to constantly mutate
| the protocol whenever FAANG feels like it, but we also gain the
| ability for every application to use the new protocol just by
| changing a flag passed to a syscall.
| boardwaalk wrote:
| And it would be great if SCTP support was common across many
| platforms and supported over networks so we could have
| something (in ways) better than TCP or UDP.
|
| Unfortunately it's over 20 years old and people just shove it
| in user space and use it over UDP (WebRTC).
|
| Your sentiment is appreciated but it wouldn't be a way to get
| anything actually done :/.
| xxpor wrote:
| Pushing it to the kernel means you have to upgrade your kernel
| to use it. Instead, you can just include a library that exposes
| a send() and recv() style API to the main app code, while the
| library just throws everything into a UDP socket. Seems better
| to me, IMO.
|
| The real issue is that we can't get away from UDP/TCP because
| of middleboxes (firewalls, load balancers, etc) and the fact
| that switches only know how to hash on 5 tuples of protocols
| they know about. If you're lucky, your router will be able to
| look inside a GRE header or something, but that's about it.
|
| When was the last time you used SCTP, for example? (If you've
| used it recently, I bet it was tunneled over UDP)
| kazen44 wrote:
| technically speaking, switches do not hash TCP information
| but hash based on data in the Ethernet frame. (this is mainly
| used for correctly balancing load across links in an lacp
| bundle).
|
| High end routers are able to do pretty conplex lookups into
| their asics, but this usually only extends to specific use
| cases. (as far as I am aware for instance, an juniper MX
| series is able to do a lookup of both vlan tags in a single
| asic pipeline step when using q-in-q. the same also applies
| to mpls stacked lable lookups on provider edge routers).
| xxpor wrote:
| The separation between a switch and a router is getting
| blurrier by the day, but any datacenter grade switch today
| will support L4 hashing for LACP. Even my el cheapo
| Mikrotik does.
| ThePhysicist wrote:
| QUIC's connection migration feature is really amazing for
| applications where you switch IPs, e.g. when your phone goes from
| mobile data to Wifi. Another nice thing is that you can cache
| sessions and resume them later, so connections can even survive
| reboots and server failures, or you can move them between
| different machines.
|
| TCPLS seems like a good proposal but I won't hold my breath for
| it as the big players like Cloudflare, Google, Akamai, Fastly are
| already committed to QUIC. There's still significant churn in the
| proposals and some features like MASQUE are still being
| finalized, but overall QUIC works really well.
| stormbrew wrote:
| I'm not a huge fan of how spdy and then quic were kinda
| ramrodded into the HTTP standards process, but realistically I
| think QUIC is basically the future of connection transport
| wherever tcp or udp alone are failing at this point. I recently
| discovered the nQuick proposal to do noise-based connection
| initiation for peer-to-peer connections and I love it[1]. It's
| an incredibly versatile protocol.
|
| It seems likely that a lot of the issues raised in the OP will
| be somewhat forced to be resolved due to the aforementioned
| ramrodding of QUIC into the HTTP3 path, but maybe in the long
| run the two ideas will co-evolve to the same solution.
|
| [1] https://eprint.iacr.org/2019/028.pdf
| unmole wrote:
| > quic were kinda ramrodded into the HTTP standards process
|
| QUIC took 4 years and 34 drafts to get standardized. The
| final IETF version is significantly different from Google
| initially came up with. So, no it most definitely was not
| ramrodded through.
| stormbrew wrote:
| I didn't say quic itself was -- if quic had been introduced
| completely independently of http then sure, that's fine.
| But it was introduced intermingled with making yet another
| drastic change to a backbone protocol of the internet.
|
| Considering HTTP/1.1 stood for over 15 years before http/2
| came along (and was a pretty modest improvement over
| http/1.0 compared to what we're talking about now, so I'm
| being generous by not saying over 20 years). http/2 had
| only been promoted to a proposed standard for 3 before
| http/3 was proposed. That, to me, is a very aggressive
| process. But again, it's the relationship to http that
| bothers me, not the standardization process for quic
| itself.
| unmole wrote:
| What's the point of having QUIC if there isn't a protocol
| that actually uses it? And what would be a better
| protocol than HTTP? HTTP/2 was a marginal improvement but
| HTTP/3 and QUIC significantly improve user experience.
| Especially if the users have jittery networks which is
| the case for the majority of the world. I don't
| understand why it would have been a better idea to sit on
| these improvements for a while before getting them into
| the hands of users.
| stormbrew wrote:
| I don't really see "is it beneficial in the long run" and
| "is it an aggressive standardization campaign by one
| interested party" as being the same question.
|
| At any rate I also didn't say they shouldn't experiment
| with using it as an accelerated path for http. I didn't
| think they shouldn't with spdy either. But historically
| RFCs were built on practice, not the other way around.
|
| If I were the Internet Dictator, I'd be a big fan of
| standardizing the semantics of http separate from the
| physical transport and then defining best practices for
| using alternative transports in the wild, and then
| eventually standardizing what comes out of that process.
|
| To me it's an indictment of how all this shook out that
| we're so quickly looking to replace http/2. I think
| moving to quic is a good move but maybe we didn't need an
| official http/2 standard in the meantime.
| unmole wrote:
| > is it an aggressive standardization campaign by one
| interested party
|
| Everyone from FB to Microsoft to even Huawei wanted QUIC.
| It's was not something that only Google was interested
| in. Hell, the primary author left Google well before the
| WG was done.
|
| > But historically RFCs were built on practice, not the
| other way around.
|
| Right from the early drafts, there have been multiple
| independent QUIC implementations constantly being tested.
| There was plenty of working code before there was rough
| consensus.
|
| > standardizing the semantics of http separate from the
| physical transport
|
| What actual semantics of HTTP changed with QUIC?
| [deleted]
| simmervigor wrote:
| > If I were the Internet Dictator, I'd be a big fan of
| standardizing the semantics of http separate from the
| physical transport and then defining best practices for
| using alternative transports in the wild, and then
| eventually standardizing what comes out of that process.
|
| This is what the IETF has been doing for several years
| already as part of the core HTTP specification rewrite.
| Namely, see the HTTP Semantics document
| https://www.ietf.org/archive/id/draft-ietf-httpbis-
| semantics...
|
| Semantics are common, how those map to specific transport
| - how it looks on the wire - is then the job of each
| individual HTTP version.
|
| It reads like you're overlooking the act of
| standardization in improving the definition of a
| protocol. Experimentation is good but often only focuses
| on a limited set of use cases, deployments, or
| implementations. Standardization is great for
| interoperability across a more diverse set of all those
| three. This typically yields protocols that are more
| generally applicable for a wider population of the
| Internet.
|
| Google QUIC was focused on an HTTP use case and thatbran
| through the entire design. Through the IETF process, the
| design evolved to be a far more generic transport with
| rich extensibility and modularity.
| aseipp wrote:
| You've got it backwards. The final problem with HTTP/2
| couldn't be solved by the application layer protocol
| alone, so "intermingling" is a "not-even-wrong" complaint
| -- if you transported a large JPEG and an HTML page over
| the network in an HTTP/2 stream, a lost packet in the
| jpeg file would cause the html stream to potentially
| stall until it was corrected and finished, even though
| you can load both concurrently. This effectively causes
| head-of-line blocking on what should be a concurrent
| system. So a primary attraction of HTTP/2 is effectively
| defeated and possibly even made worse; it is a behavior
| instituted by TCP, there's nothing you can solve it with.
|
| QUIC was "intermingled" because it exactly solved this
| problem at the transport layer, the only place it _can_
| be solved. They couldn 't just introduce any random
| thing, and no other protocol existed, so they ratified
| QUIC independently as its own protocol and applied it to
| HTTP/3 at the same time. They _had_ to introduce a new
| standardized transport if the hope was to solve the HOL
| blocking issue. If you "dis-entangled" it you'd just
| have HTTP/2. The new transport protocol _is_ the defining
| characteristic.
|
| Furthermore it all would have been pointless to define
| QUIC as a standard without actually _applying it in
| practice_ , in anger, to real workloads. Which is exactly
| what multiple stakeholders in the process did throughout
| the multi-year ratification process. What are they going
| to do, define QUIC, sit on their thumbs for 5 years and
| cross their fingers hoping someone else tries it?
|
| There are other factors like most of the involved
| stakeholders had previous experience watching with other
| attempted standards in similar veins like TCP Fast Open
| die on the vine. That also influenced decisions like
| using UDP for QUIC's transport (and the ever present
| discussions about middleboxes.) Actual experience
| indicates most low-level protocols have ossified badly or
| are handled very poorly, including TCP and UDP except
| most things can handle those, so moving the transport
| layer into the application stack is seen as providing
| actual agility needed to respond to issues like the HOL
| blocking issue. That's how they revised the protocol
| several dozen times over the years; applications are
| actually fairly agile. Now you could argue these people
| should have discovered that issue when designing HTTP/2,
| but "Why didn't this thing get discovered and found out?"
| is always a weird complaint. Why _would_ you expect
| everything to be found out early? I just can 't answer
| that complaint honestly. Weirder things have happened
| than "Some engineers missed something."
|
| Many people seem to hold some unwritten belief that if a
| protocol is defined as, say, an application layer
| protocol at one point, then it can never ever redefine
| its parameters to be anything beyond that. It can never
| become a transport protocol, for example, or have the two
| be mandated together, despite decades of experience
| suggesting that might be a fruitful path for many
| technical reasons. But there's no random rules of the
| universe defining what a network protocol looks like. It
| can be whatever we want it to be. Humans define this,
| there isn't like a book of 10 commandments out there
| written by God.
|
| There is also no necessary indication that HTTP 1.1
| sticking around for so long versus HTTP/2 necessarily
| meant it was the best among all options, by the way. We
| abused it a lot for many things. Being around for so long
| could also mean we were just bad at enacting change;
| certainly browser monocultures have made change easier
| for better and worse, on that note. But the whole HTTP/2
| and HTTP/3 process has been about 13 years on its own if
| you count it as starting with SPDY. It's not like there's
| 200 years of precedent for how long it _should_ take.
| stormbrew wrote:
| Most of this is just retreading the other thread so I'm
| largely going to just say you can read my replies there,
| but to explicitly reiterate:
|
| > They had to introduce a new standardized transport if
| the hope was to solve the HOL blocking issue.
|
| I take no issue with quic being standardized and I
| applaud that it was.
|
| > Furthermore it all would have been pointless to define
| QUIC as a standard without actually applying it in
| practice, in anger, to real workloads.
|
| Protocols are deployed on real workloads in anger in
| practice every day without even having an ietf rfc, let
| alone being blessed as the next official version of one
| of the most important protocols on the internet, and it
| works out fine. I never said, meant, or in any way
| advocated for anyone sitting on their ass.
| aseipp wrote:
| So based on reading your other responses your complaint
| is that "I think RFCs are built on practice than the
| other way around" which doesn't mean much, considering
| that's what is happening! They are building on practice.
| Most if not all RFCs anywhere, including the IETF,
| actually see lots of revisions, and QUIC is hardly very
| different, and there is no hard rule about how standards
| are formed. This is how it works.
|
| EDIT: I was too rude/curt and re-edited this. Sorry.
| [deleted]
| bediger4000 wrote:
| Lots of dupes:
|
| https://news.ycombinator.com/item?id=31589575
|
| https://news.ycombinator.com/item?id=31569066
| [deleted]
| pvg wrote:
| They need to have some discussion to count as HN-dupes and
| those don't.
___________________________________________________________________
(page generated 2022-06-05 23:01 UTC)