[HN Gopher] Multipath TCP for Linux (2022)
___________________________________________________________________
Multipath TCP for Linux (2022)
Author : pantalaimon
Score : 117 points
Date : 2024-04-19 17:28 UTC (5 hours ago)
(HTM) web link (www.mptcp.dev)
(TXT) w3m dump (www.mptcp.dev)
| vlovich123 wrote:
| Why does this require explicit opt in by applications if there's
| transparent fallback? Wouldn't it make most sense for the kernel
| to do it transparently for every TCP connection so that it can
| make more global decisions about path aggregation / link
| preference?
| chmod775 wrote:
| Using this implies that there are multiple IPs per endpoint
| associated with a single TCP connection. That is going to need
| explicit support/awarness by the application in many cases.
| brenns10 wrote:
| My understanding is that it was basically a condition enforced
| by the maintainers of the Linux TCP / networking subsystems. If
| you look at the initial upstreaming discussions[1], this was
| setup as a ground rule.
|
| If you look at the older multipath TCP implementation, prior to
| the upstreaming, it was intended to be fully transparent to the
| application, which I think makes more sense for the intent of
| the protocol. Sure, in many cases MPTCP may be better with
| application-guided logic, but having a standard system approach
| (e.g. establish sub-flows on an LTE connection for automatic
| failover, but don't send any data along those sub-flows) would
| have worked for 95% of cases.
|
| [1]
| https://lore.kernel.org/all/alpine.OSX.2.21.1707181728570.11...
| londons_explore wrote:
| I can imagine new security holes being opened up by allowing
| multiple IP's to talk over the same TCP connection...
|
| Imagine you have an application which checks the clients IP
| (eg. against a whitelist) at the time of connection and then
| assumes it doesn't change...
| apitman wrote:
| Is this something that middle boxes tend to support in practice?
| flounder3 wrote:
| We found that most proxies/firewalls (90%+ ? I forget) didn't
| tamper with it. The largest hurdle was working with load
| balancer vendors to implement it.
| iknowstuff wrote:
| Apple supports it too and uses it for Siri
| https://developer.apple.com/documentation/foundation/urlsess...
| Arcanum-XIII wrote:
| It's quite easy to use in other app - it's there in the basics.
| We were quite amazed to find our voip app to be quite robust in
| 2011 :D
| ohnoesjmr wrote:
| I've heard about MPTCP back in 2013.
|
| It made so much sense back then, when mobile apps were not that
| robust to networks changing, I assumed it's going to get adopted
| in no time due to how much of a ux improvement it would have been
| back in the day.
|
| It's incredibly depressing that this gained barely any traction
| in the last 10 years, and kernel options are appearing just
| recently, after everyone has wrapped they http calls in multiple
| retry handlers, and mobile operating systems have abstracted
| network connectivity to the point where it feels more like you
| are using zeromq rather than tcp.
| zoobab wrote:
| ZeroMQ yeah!
| quectophoton wrote:
| > I've heard about MPTCP back in 2013.
|
| > I assumed it's going to get adopted in no time due to how
| much of a ux improvement it would have been back in the day.
|
| You might also be interested in SCTP[1] from the year 2000,
| which also hasn't gotten any traction so far.
|
| [1]:
| https://en.wikipedia.org/wiki/Stream_Control_Transmission_Pr...
| gamegod wrote:
| WebRTC data channels use SCTP, which ain't nothing!
| https://datatracker.ietf.org/doc/html/rfc8831
|
| (SCTP over DTLS, that is...)
| fanf2 wrote:
| SCTP is used a lot inside telco networks for carrying
| switching control metadata for voice connections.
| https://en.wikipedia.org/wiki/SIGTRAN
| throw0101b wrote:
| > _You might also be interested in SCTP[1] from the year
| 2000, which also hasn 't gotten any traction so far._
|
| Probably partly because middleware boxes (e.g., firewalls)
| either didn't/don't support it and/or rules were written to
| only support "TCP" (as opposed to 'stream') or "UDP" (as
| opposed to 'dgram'; see also "DCCP").
| fragmede wrote:
| The protocol is specified by a byte in the IP packet; how
| many middleware boxes block everything except for ICMP,
| TCP, and UDP? What is the probability that a packet with
| that byte set to something unexpected actually gets from
| source to destination?
| znpy wrote:
| The "funny" thing is that http3 really really looks like
| a transport protocol encapsulated into... uso. Exactly
| because many middle boxes block anything that's not a
| very well known protocol
| Hikikomori wrote:
| How does sctp work with NAT that your typical home box
| uses?
| toast0 wrote:
| I wanted to like it, and Apple included it in iOS, but
| supporting it on real servers was going to be too hard...
|
| When I was deployed on FreeBSD with no load balancers, there
| weren't recent patches. And even if there were, I'd need to do
| some serious work to avoid advertising the private network ips
| as alternates...
|
| When I was on Linux behind a load balancer, it's too complex to
| get the streams to the right place. And the load balancer
| doesn't want to do it anyway.
|
| Processing two streams together involves a lot of complexity in
| a high throughput code path. It's a lot of risk, and you've got
| to reboot for changes.
|
| And then you do all that work and it only benefits iOS users,
| who tend to be on better networks anyway.
| bbss wrote:
| Apple also contributed[1] MPTCP support to Envoy Proxy.
|
| [1]https://github.com/envoyproxy/envoy/pull/18780
| ajb wrote:
| I suspect that a lot of innovation energy moved to QUIC,
| because with TCP your nice new variant can be randomly nobbled
| by middleboxes. For example, see
| https://blog.apnic.net/2021/12/08/efficient-multipath-transp...
| zoobab wrote:
| Too bad there is no link to projects using mptcp, such as Openwrt
| derivatives.
|
| PS: I mentored a student for 2 years at GSOC to patch Openwrt
| with mptcp:
|
| https://blog.freifunk.net/2017/05/29/gsoc-2017-add-mptcp-sup...
| atarvaneitor wrote:
| one example on openwrt http://www.openmptcprouter.com/
| londons_explore wrote:
| Whats the benefit of an openwrt router supporting mptcp?
|
| I'd think support on web servers and mobile devices would be
| most important...
| ranger_danger wrote:
| Link aggregation and redundancy
| DarrisMackelroy wrote:
| I got fiber run to my neighborhood, and for a while, had a
| 1gb coax connection and a 1gb fiber connection. I used
| openmptcprouter to aggregate my connections through a droplet
| and I effectively had a 2 gigabit internet connection. I
| would have stuck with it, but having a datacenter IP for your
| home network really doesn't work.
| londons_explore wrote:
| Except TCP is just a bad protocol to start with for
| tunnelling, because packetized data has to be delivered in-
| order, and head of line blocking messes up congestion
| control algorithms in the tunnelled data.
| jburgess777 wrote:
| It was recently enabled in the Home Assistant 'HAOS' kernel.
| https://github.com/home-assistant/operating-system/pull/3248
| vesinisa wrote:
| > If any middlebox in between does not support it, the returned
| SYN+ACK packet will not contain MPTCP options in the TCP option
| field.
|
| That sounds .. quite restrictive. Is the only requirement on a
| middlebox to just forward the MPTCP options as-is?
| extraduder_ire wrote:
| The examples given on the page seem to focus on multipath to get
| to a device over the internet, but I can see this being more
| likely to work properly without needing to fallback on home
| networks.
| txutxu wrote:
| At home/lan we use LACP, VRRP... I mean link aggregation and HA
| needs are solved time ago.
|
| With multiple ISPs, or on a complex enough LAN, we can use
| multiple routing tables + weights too.
|
| Also, if the ISP at home can do 10Gbps, 1Gbps, 300 Mbps
| whatever... I want to be able to use them with a single path,
| so there is no gain using multiple paths. Eventually, when I
| have cable+wifi connected at the same time, I use to force one
| of both, cannot see a reason to prefer using both at the same
| time.
|
| Maybe the latency thing? Never had that issue at home, but
| could understand that usage case "just use the network segment
| with less latency to reach $thing".
| slt2021 wrote:
| This can help in security/privacy setting.
|
| for example Great Chinese firewall: if you can split your traffic
| across multiple uplink channels, the firewall will have a hard
| time to put them together for enforcement?
| Hikikomori wrote:
| They just block or rate limit it severely when they dont know
| what it is.
| EvanAnderson wrote:
| I don't know which makes me sadder-- IPv4 only having a 32-bit
| address space or TCP using the source and destination IP
| addresses in the connection tuple. That's one of those "if I had
| a time machine" of things-- I'd go back and have Cert and Kahn
| change both of those items.
| bobmcnamara wrote:
| How would you change TCP?
|
| Do you mean how we have to use 4 fields to track a connection -
| IP address and port for both ends?
| EvanAnderson wrote:
| If TCP had a protocol specific identifier for connections (a
| couple of 32-bit values, for example-- a client nonce and
| server nonce) rather than using the source/destination IP
| addresses multi-homed hosts and seamless transition between
| different networks would become native features of the
| protocol. A client could roam between two different IP
| networks and TCP connections would "survive", for example.
| (I'm oversimplifying nearly to the point of hyperbole, to be
| sure...)
|
| (Another fun future would have been one where SCTP got
| widespread adoption.)
| mauriciob wrote:
| How would routing be done without source/destination? When
| the device changes networks, how does the origin and all
| routers along the way know that this device is on a new
| network?
| zare_st wrote:
| Yeah it's a shortsighted plan. How would i set up rules
| on firewall if I don't know how to distingush the
| connections?
| mlyle wrote:
| There's two separate ideas here:
|
| * Where to send a frame to get to the other side of the
| connection
|
| * Whose connection this is.
|
| TCP combined the two, because we didn't have mobile
| clients or a lot of multihomed systems that would benefit
| from distinguishing them. Also, every octet in the header
| counted.
|
| In practice, this means we have to keep building a lot of
| infrastructure on top of TCP (or parallel to it, in
| datagram protocols) to handle retries and splitting flows
| well. In turn, these things are completely opaque to the
| network and it's difficult to write rules about them.
|
| Whereas if we had different packet fields for "where am I
| sending this packet right now" and "whose flow does this
| belong to"? we could write better firewall rules, have
| less infrastructure built on top of TCP, and have better
| typical application performance.
| mlyle wrote:
| > how does the origin and all routers along the way
|
| It's just the origin that needs to know what address(es)
| it should be using as the destination at layer 3.
|
| The big problems with this is that it depends upon things
| that weren't really feasible in the early 80's -- bigger
| packet headers, a bit more state on each side of the
| connection, potential need for cryptographic
| authentication.
| topspin wrote:
| > How would routing be done without source/destination?
|
| There is still a source/destination address. Routing
| still works. But those addresses are allowed to change
| without disrupting the connection because the connection
| isn't based on the values of these addresses.
|
| > When the device changes networks, how does the origin
| and all routers along the way know
|
| The routers don't need to "know" these things.
|
| MPQUIC does this. To the network it's just UDP packets
| moving around. Connection state is dealt with at higher
| levels and doesn't rely on IP addresses.
| zare_st wrote:
| But the stuff that carries TCP is IP. That's why TCP can
| work seamlessly, because it uses identification from a
| previous layer. Consider I bind a server to an ID, and not
| IP:port, the operating system running it must know how to
| communicate that via IP, so there will be a corellation map
| somewhere and that map needs to be synchronized between all
| peers that wish to host the roaming server.
|
| Otherwise you're just switching port (16-bit) value to
| arbitrary 32-bit identifier.
| mlyle wrote:
| If TCP didn't use L3 source and destination addresses to
| distinguish connections, it could be more easily taught
| to deal with:
|
| * Clients roaming between L3 addresses
|
| * Clients/servers with multiple L3 addresses
| EvanAnderson wrote:
| The protocol would have to handle binding the network to
| the transport. MPTCP and SCTP both handle that via
| registering and un-register network layer endpoints. This
| parallel universe TCP would be the same in that regard.
|
| (I did say I was oversimplifying...
| alerighi wrote:
| The problem is that the TCP/IP model stops at level 4, and
| if we consider TCP a protocol of transport, it shouldn't do
| that.
|
| In the OSI model what you talk about is level 5, that is
| session, but in TCP/IP there is no such level, thus it must
| be handled by the application (e.g. trough a session
| cookie, in HTTP).
| eptcyka wrote:
| How do you even firewall such a protocol?
| infogulch wrote:
| I found [1] which describes the architectural difference between
| MPTCP and QUIC, and also introduces the authors' proposed MPQUIC
| protocol:
|
| > QUIC multiplexes application streams on a single UDP flow,
| whereas MPTCP splits a single stream on multiple TCP subflows.
| MPQUIC combines both features by multiplex- ing application
| streams on multiple UDP subflows.
|
| [1]: "Multipath QUIC: A Deployable Multipath Transport Protocol"
| https://www.researchgate.net/publication/327122884_Multipath...
|
| Now I'm curious about how these protocols compare in production
| operation. Anybody have experience with both?
___________________________________________________________________
(page generated 2024-04-19 23:00 UTC)