[HN Gopher] A transport protocol's view of Starlink
       ___________________________________________________________________
        
       A transport protocol's view of Starlink
        
       Author : fanf2
       Score  : 168 points
       Date   : 2024-05-17 00:17 UTC (22 hours ago)
        
 (HTM) web link (www.potaroo.net)
 (TXT) w3m dump (www.potaroo.net)
        
       | CaliforniaKarl wrote:
       | One comment about the analysis of ping times in the section "Low
       | Earth Orbit Systems". Specifically, the analysis of ping times
       | "within each 15s satellite tracking interval".
       | 
       | Most routers do not put ping-processing in the "fast path". That
       | is, instead of having the ping be processed by an ASIC, the ping
       | gets processed by the router's CPU. And ping-processing is
       | typically a lower-priority task. Because of that, you can't
       | assume that the high variation in latency is because of Starlink.
        
         | simoncion wrote:
         | > That is, instead of having the ping be processed by an ASIC,
         | the ping gets processed by the router's CPU.
         | 
         | I wonder how true that is anymore, with ICMPv6 processing being
         | a mandatory part of IPv6. I could totally see ICMP processing
         | being a low-priority task, but am far less certain that it
         | would not be done by dedicated hardware these days.
         | 
         | On top of that, I've never, ever, ever noticed the behavior
         | he's observing with either my cable ISP connection, or the
         | terrestrial microwave link provided by my local WISP. I don't
         | have enough data to say that I'd _never_ see that behavior if I
         | happened to ping some router powered by a potato... but I 've
         | pinged a whole bunch of systems over the years, and have never
         | seen variation like what he describes.
        
           | lxgr wrote:
           | I've seen ping latencies of up to 30 seconds (yes, 30 000
           | milliseconds!) with a certain cable ISP, while at the same
           | time VoIP (RDP over regular Internet IP, not "cable voice" or
           | ISP-provided SIP, which often has its own QoS class) was
           | borderline usable.
           | 
           | Could have been traffic shaping or prioritization (their
           | network was in complete shambles after all), but ICMP was
           | definitely taking some lower priority queue or path.
        
             | kkielhofner wrote:
             | *RTP
        
           | toast0 wrote:
           | > I wonder how true that is anymore, with ICMPv6 processing
           | being a mandatory part of IPv6.
           | 
           | ICMPv6 may be mandatory according to specs, but you can still
           | drop most of it with no ill effects. You probably shouldn't
           | drop needs fragmentation packets, but everyone has adapted to
           | those being dropped sometimes, so...
           | 
           | If you ignore neighbor discovery, you'll have trouble
           | reaching your neighbors, though.
           | 
           | But neighbor discovery is low volume mostly, and can be
           | handled by the cpu, not the asic. Needs frag likely won't be
           | directed at the router ip, the asic can forward them just
           | fine though.
           | 
           | I've certainly seen much more variable ping times for routers
           | than for the hosts behind them. If the router's cpu is less
           | busy, ping times are usually a bit more than a host that's
           | right there, and as the router's cpu gets more busy, the rtt
           | increases or pings get dropped. It's not usually a factor of
           | how busy the links are either; routers tend to have a pretty
           | limited amount of cpu and if they're getting a lot of
           | traffic, it's easy to overload them.
           | 
           | I haven't seen it happen quite in the bands like in the
           | article, but sometimes it varies quite a bit. The results
           | tend to be much more consistent if you can get a rtt from
           | full host and not a router.
        
         | jiveturkey wrote:
         | Indeed. Surprising that such a technical article from a
         | supremely technical and knowledgeable person would not at least
         | have a disclaimer about it. I would also be interested to know
         | how much a Starlink satellite is an ASIC router vs pure CPU.
         | 
         | I believe a better technique would be to ping _through_ the
         | router to another endpoint under ones own control, on the same
         | network as the origin client, and where processing latency can
         | be controlled. Basically ping yourself but via starlink and
         | back.
         | 
         | That said, in practical terms he's quite right about the jitter
         | problem. I maintain 2 networks myself at my remote home. One my
         | microwave WISP for tight latency and jitter, and then Starlink
         | for much better bandwidth/throughput.
        
           | alfons_foobar wrote:
           | He does ping through the starlink terminal:
           | 
           | > ping across a Starlink connection from the customer side
           | terminal to the first IP end point behind the Starlink earth
           | station
        
             | jiveturkey wrote:
             | I interpreted that statement incorrectly as being an
             | endpoint on the satellite itself. I guess the "earth
             | station" is the round trip terrestrial endpoint via the
             | satellite. I was misreading it as being the local terminal.
        
         | alfons_foobar wrote:
         | That is true for ICMP packets directed at the router itself.
         | 
         | Pretty sure that is the reason why he used
         | 
         | > ping across a Starlink connection from the customer side
         | terminal to the first IP end point behind the Starlink earth
         | station
         | 
         | instead of pinging the terminal itself.
         | 
         | (ICMP packets passing _through_ a router are handled just like
         | any other traffic and are therefore OK for an analysis like
         | this)
        
           | maxmouchet wrote:
           | Additionally, I would not expect such "baseline" changes in
           | the minimum RTT to be due to variation in CPU processing
           | time.
           | 
           | These changes are more typical of a physical path change, as
           | suggested by the author.
           | 
           | CPU/soft processing latency would look more like additive
           | noise.
           | 
           | I give some examples of RTT patterns in this talk:
           | https://ripe77.ripe.net/archives/video/2250/
        
           | CaliforniaKarl wrote:
           | It's unclear to me. `traceroute` typically just returns
           | router IPs, so I'm assuming that "the first IP end point
           | behind the Starlink earth station" is the IP address of a
           | router of some sort.
        
             | alfons_foobar wrote:
             | Hm, I understood "ip endpoint" to mean something else than
             | a router (because I am very sure that Geoff is aware of the
             | "ICMP is not handled on the ASIC"-issue).
        
         | chgs wrote:
         | I ping routers around the world all the time and rarely see any
         | significant (>1ms) change
        
           | damsalor wrote:
           | Most ppl do not enjoy that privilege
        
             | alfons_foobar wrote:
             | What do you mean by "privilege" in this context?
        
               | piltdownman wrote:
               | 1st World Infrastructure and Routing, being based in a
               | densely populated and developed locale I'd imagine.
               | 
               | YMMV if you attempt the same traceroute from Bangalore or
               | Sydney to London/Paris/NYC.
        
               | bayindirh wrote:
               | I just tested something to a similar effect. It's much
               | more jumpy time-wise, yes.
        
               | chgs wrote:
               | Just pinged my isp router in Kathmandu for 30 seconds,
               | 172-174ms return. It's on AS4007.
               | 
               | From my router in Dhaka pining a router on airtel in
               | India, 36-38ms
               | 
               | Routers in the real would give very stable ping
               | responses, no matter where they are based.
        
       | justsomehnguy wrote:
       | > A "flooding" ping sends a new ping packet each time a packet is
       | received from the remote end
       | 
       | Uhm, no:                   -f            Flood ping. For every
       | ECHO_REQUEST sent a period "." is            printed, while for
       | every ECHO_REPLY received a backspace is            printed. This
       | provides a rapid display of how many packets            are being
       | dropped. If interval is not given, it sets interval            to
       | zero and outputs packets as fast as they come back or one
       | hundred times per second, whichever is more. Only the
       | super-user may use this option with zero interval.
        
         | londons_explore wrote:
         | > outputs packets as fast as they come back
         | 
         |  _If_ the latency is under 10 ms, then what op said is correct
         | .
        
       | metadat wrote:
       | > This orbital velocity at the surface of the earth is some
       | 40,320 km/sec.
       | 
       | Huh? Is this right? That's really fast, like 24,800 miles per
       | second, which is a significant fraction of c (the speed of light,
       | 186,000 miles/sec). >10% of c, is this a typo?
        
         | fulafel wrote:
         | They probably meant km/h - that's in the right ballpark for the
         | oft cited 7.5 km/sec for LEO.
        
         | nonameiguess wrote:
         | Earth's orbit has a circumference of about 940 million km.
         | Divide that by 365.25 days x 24 hrs/day x 3600 seconds/hr and
         | I'm getting in the arena of 29.78 km/sec.
        
         | markbnj wrote:
         | No, it's not. This immediately caught my eye too. According to
         | google the right number is 7.8 k/s so it was very far off the
         | mark.
        
       | pingpong250 wrote:
       | Is this scenario not what TCP Peach was created for?
        
         | njoubert wrote:
         | Can you say more?
        
       | iam-TJ wrote:
       | For those not aware of how Starlink operates: The customer
       | antenna is called the User Terminal (U.T.) a.k.a. "dish" although
       | all production models are rectangular - only the pre-production
       | beta model is round and dish-like.
       | 
       | The U.T. contains a phased array antenna that can electronically
       | 'steer' the bore-sight (aim) of the transmitted (and received)
       | signal at the current satellite that is in view. In ideal
       | circumstances the U.T. antenna has approximately 110 degrees
       | field of view (~ 35 degrees above each horizon).
       | 
       | The satellites pass from west to east and take approximately 15
       | seconds to pass through the field of view of the U.T. The
       | satellite forms a beam aimed at a fixed location on the ground -
       | this is called a 'cell'. All U.T. within that area share the
       | radio link that has a fixed bandwidth, so contention is managed
       | by the satellite.
       | 
       | The path length to a satellite directly overhead would be around
       | 550km (in most cases the satellite is slightly north, or slightly
       | south, of the U.T. but for round numbers sake assume 550km).
       | 
       | The path length to a satellite appearing 35 degrees above the
       | horizon (the slant range) is ~ 2568km.
       | 
       | Satellites relay the packets from the U.T. to the (nearest) Earth
       | ground station, so the path length and therefore travel-time will
       | vary enormously over just 15 seconds.
       | 
       | The round-trip for the minimum case is 4 x 550km = 2200km but for
       | the maximal case is 4 x 2568km = 10272km. These equate to a
       | travel time of between 1.8 and 3.6ms per leg, so that gives a
       | hard physical minimum of 4 x 1.8ms = 7.2ms to 4 x 3.6ms = 14.4ms
       | 
       | As more satellites are added to the constellation so the gap
       | between satellites decreases and the angle above horizon at which
       | a satellite is acquired can increase thus shortening the maximum
       | path and lowering the latency.
       | 
       | Starlink has a publicly stated goal of less than 20ms round trip
       | latency and published a report in March 2024 about the
       | engineering efforts to achieve this [0]. Much of the effort that
       | customers see focuses on two issues:                 1. reducing
       | latency between ground station and Internet connection point
       | 2. scheduling the radio links between satellite and all U.T.s in
       | its beam area
       | 
       | Starlink balances contention by sometimes restricting and
       | sometimes promoting activation of new U.T.s in each area - this
       | is why on occasion a fully subscribed cell will impose a waiting
       | list on new activations. At other times Starlink will, and does,
       | dynamically change the monthly subscription cost. Recently some
       | areas had their residential price reduce from US$120 to US$90
       | where others in congested areas had an increase from US$90 to
       | US$120 (in the USA).
       | 
       | [0] https://api.starlink.com/public-files/StarlinkLatency.pdf
        
         | numpad0 wrote:
         | fyi, User Terminal or UT is quite an established term. Oldest
         | random Googlable example comes to my mind is a thin client
         | server process named "SUNWut"(a ticker code + "ut") from early
         | 2000s. I'm pretty sure the usage traces back multiple decades
         | from that point in phone and software industries.
        
           | iam-TJ wrote:
           | General usage stemmed from the Tactical User Terminal's
           | deployed in the 1980s by the U.S. Army in Germany; part of
           | codename TENCAP ELINT programme - and were truck-sized!
        
           | bewaretheirs wrote:
           | Most of the references I can find to "SUNWut" are related to
           | the Sun Ray thin client. See
           | https://en.wikipedia.org/wiki/Sun_Ray
        
       | friseurtermin wrote:
       | What I miss in this article are some references to some of the
       | great research on Starlink performance and characteristics that
       | has come out in the past ~2 years. [1] and [2] come to mind. They
       | also go into much more detail on the 15s interval, quote [1]:
       | 
       | > Interestingly, we observe that the Starlink OWD (one-way delay)
       | often noticeably shifts at interval points that occur at 15 s
       | increments. Further investigation reveals the cause to be the
       | Starlink reconfiguration interval, which, as reported in FCC
       | filings [71], is the time-step at which the satellite paths are
       | reallocated to the users.
       | 
       | AFAIK it is not the dish itself that does the tracking but a
       | central orchestration.
       | 
       | [1]: https://arxiv.org/abs/2310.09242 [2]:
       | https://arxiv.org/abs/2306.07469
        
       | dogben wrote:
       | > This orbital velocity at the surface of the earth is some
       | 40,320 km/sec.
       | 
       | Is that a typo? Orbital velocity on the surface should be
       | 7.9km/s.
        
         | marssaxman wrote:
         | It seems like it must be - 40,320 km is comparable to the
         | circumference of the earth!
        
         | out_of_protocol wrote:
         | 40,320 km/sec is full earth rotation per second, a tiny bit too
         | fast . Should be /per day or so. Also, 40k km/s is ~13% of c
         | (speed of light)
        
       | NelsonMinar wrote:
       | BBR may not be perfect but it sure is an improvement for Starlink
       | over Cubic. I switched to it on a server I connect to frequently
       | from Starlink and got a 2x performance improvement on TCP
       | connections. Before, big file transfers would regularly bounce
       | between 5 Mbytes/s and 12. After it'd hold a steady 12. Which is
       | what you'd expect given Starlink's high packet loss rate (0.5%
       | for me) and how TCP congestion control traditionally works.
        
       ___________________________________________________________________
       (page generated 2024-05-17 23:01 UTC)