[HN Gopher] AWS inter-region latency chart
___________________________________________________________________
AWS inter-region latency chart
Author : mooreds
Score : 73 points
Date : 2021-04-30 17:27 UTC (5 hours ago)
(HTM) web link (www.cloudping.co)
(TXT) w3m dump (www.cloudping.co)
| nrmitchi wrote:
| Mhmm I would have expected the co-located source/destination
| numbers to be 1, slightly lower, and 2, more consistent between
| regions.
|
| Ie, I would not have expected eu-central-1/eu-central-1 to be
| 1.68ms, and us-east-2/us-east-2 is 8.26ms (almost 5x)
| mattashii wrote:
| AWS's availability zones are unlike Google Clouds AZs, as they
| are physically meaningfully distant.
|
| A geological or technical event near one AWS AZ is highly
| unlikely to also affect the others of that region, unless the
| event is regional (heh) in scale. For Google Cloud, the AZs are
| effectively co-located buildings (or at least, Eemshaven has 3
| GC AZ DCs at < 1km distance from each other), making even local
| disruptions reasonably likely to impact availability for the
| whole region.
| nrmitchi wrote:
| Yes, that being said, I would expect those same precautions
| to be taken in all regions, and thus not give a meaningful
| difference per region, between regions (if that makes sense).
| kbenson wrote:
| Different regions have different types of worries. Distance
| and precautions required to deal with redundancy for an
| earthquake may not be the same as required for a tornado,
| or hurricane, or flood, etc. In some cases, I imagine
| elsewhere in the same large city may suffice. In others,
| maybe it requires 50-100 miles.
| finnh wrote:
| The chart doesn't show AZ data, which seems like an important
| piece of this puzzle.
| itisit wrote:
| Perhaps OP can answer. The AZ ID[0], as opposed to the name,
| does identify a specific AZ location, so certainly possible
| to measure AZ-to-AZ latencies.
|
| [0] https://docs.aws.amazon.com/ram/latest/userguide/working-
| wit...
| AdamN wrote:
| 1) <10ms should be gray since those are effectively in-region
| numbers (and it would make the chart easier to understand
| quickly)
|
| 2) Times should be tagged with a theoretical minimum time and how
| far off the real number is from that.
|
| 3) (Bonus points) Times should also be tagged with a 'real'
| minimum time that is based on the lengths of the actual fiber
| lines and number of interchanges.
| yukinon wrote:
| This looks great, but I wish that this was more colorblind
| friendly.
|
| Note: For anyone else that is on a Windows 10 machine and is
| struggling to see the difference, there is a colorblind mode that
| you can toggle with Winkey + Ctrl + C.
| notyourwork wrote:
| Great information!
|
| My only suggestion would be that a column and row hover wold make
| the data easier to inspect and navigate.
| ape4 wrote:
| Might be neat to see a digraph of this. Would it look like the
| globe?
| m00dy wrote:
| I was wondering where this could be useful.
| mooreds wrote:
| Ah, I found this when we were discussing internally building
| out a multi region installation of one of our products. We were
| discussing the tradeoffs between a single master database with
| calls from other regions to that database, vs replicating a
| database between regions. The latter is more complicated to run
| but has far less latency.
|
| That's when I found this chart through the magic of google to
| bring some real numbers (tm) to the latency discussion.
| m00dy wrote:
| thank you
| ImJasonH wrote:
| Here's the equivalent for GCP, FYI:
| https://datastudio.google.com/u/0/reporting/fc733b10-9744-4a...
|
| Ironically less colorful. :)
| ignoramous wrote:
| I used to read ThousandEyes's annual free reports on networking
| capabilities and measurements across the Big 3 cloud providers,
| which were quite comprehensive in terms of covering various
| scenarios [0]. I don't think they publish those anymore.
|
| With cloudping.co, it isn't clear from the website, but it seems
| like the inter-region latency they measure is over the public
| Internet. The Big 3 run their own backbone across all their DCs
| around the globe, and so I reckon, those numbers would look
| vastly different if traffic was instead relayed through those
| uncongested backbones.
|
| With AWS, the cheapest way I know (in terms of development time
| and cost) to accomplish inter-region over their backbone is via
| Global Load Accelerator. The clients connect to GLA at the
| nearest anycast IP location advertised across 150+ AWS PoPs. You
| can then play with endpoint-groups, connection-affinities,
| source/port tuples to control routing traffic to different
| backends in various regions.
|
| We used this technique to prototype a VPN with exit nodes in
| multiple countries but entry nodes closest to clients at every
| AWS PoP. It worked quite nicely for a toy:
| https://news.ycombinator.com/item?id=21071593
|
| [0] https://www.thousandeyes.com/blog/top-takeaways-cloud-
| perfor... (2019)
| bradleydwyer wrote:
| This is quite useful. Would love to see it extended to different
| providers (not just the latency internal to each provider but
| between them).
|
| For example, I'd like to find a set of 3 locations across 3
| providers that have the lowest latencies between them.
| roomey wrote:
| I don't know how they are accounting for the various different
| data centers within a region (IE. Availability zones). It could
| explain self latency if a packet is leaving a DC
| dtech wrote:
| anecdotally the connections between AZs are so low latency you
| often can't tell whether a server is in the same or a different
| AZ
| jzoch wrote:
| Definitely not true. Cross AZ is incredibly high compared to
| within the same AZ. We run a bespoke multi-master database
| setup that must be colocated to the same AZ due to
| unacceptable latencies when run spread across 3 AZ's. A few
| ms difference
| dastbe wrote:
| (i work at aws)
|
| yes, while in my opinion most services won't care about the
| distinction between cross-az/same-az (you're definitely not
| most services :) ), you can definitely tell the difference
| between
|
| * in the same region vs not
|
| * in the same az vs not
|
| * in the same placement group or not
|
| which shouldn't be surprising. you should get a latency
| benefit from increased physical proximity!
| weird-eye-issue wrote:
| Some AZs have multiple DCs btw
| tuananh wrote:
| AZ usually have 2-5 DC if i recall correctly
| outworlder wrote:
| > various different data centers within a region (IE.
| Availability zones).
|
| A single availability zone is often (always?) composed of
| multiple datacenters which are spread relatively far apart from
| one another.
| judge2020 wrote:
| This might be a rare case that actually benefits from being an
| interactive map with lines colored differently/dotted differently
| to indicate interconnect speed (of course, keeping the table for
| non-js compatibility).
| skipwalker wrote:
| Is there supposed to be a way to hide regions? The "Enabled
| Regions" menu doesn't appear to do anything for me.
| VWWHFSfQ wrote:
| It would be interesting to also see what the lowest possible
| latency between the regions would be so you can see how much
| overhead the interchanges are adding.
|
| For example, the distance between us-east-1 and us-west-1 is
| approximately 2,500 miles. Light travels at 186,000 miles/second,
| so that's 13 milliseconds one-way. So the fastest possible TCP
| SYN/ACK/SYNACK is 39 milliseconds between the two DCs just to
| establish a connection.
| dheera wrote:
| I spent about 15 minutes trying to create a custom Distance
| function in Google Sheets: const DISTANCE =
| function(a, b) { [latA, lonA] = a.split(",");
| [latB, lonB] = b.split(","); latA=parseFloat(latA);
| lonA=parseFloat(lonA); latB=parseFloat(latB);
| lonB=parseFloat(lonB); return 2 * 6371000 * AS
| IN(SQRT((SIN((latB*(3.14159/180)-latA*(3.14159/180))/2))^2+COS(
| latB*(3.14159/180))*COS(latA*(3.14159/180))*SIN(((lonB*(3.14159
| /180)-lonA*(3.14159/180))/2))^2)) }
|
| So that I could just paste concatenated "lat,lon" coords in
| place of the AWS region names and just compute e.g.
| =DISTANCE(A2,B1)
|
| Into a second sheet, and then just divide the first sheet by
| the second sheet as a third sheet.
|
| but I kept getting some "#NAME" error and I don't have time to
| figure it out.
|
| Bad UX, Google. This should have been easy.
|
| Better yet, you're freaking Google, why isn't there a
| DISTANCE("Hong Kong", "Singapore", "straightline") function?
|
| I have to get back to work but if someone can figure the rest
| of this out please comment back.
| scottlamb wrote:
| Light travels at 186,000 miles/second _in a vacuum_. A quick
| websearch says fiber 's index of refraction is ~1.467. One
| round trip then takes 39 ms (coincidentally the same number you
| said for 3 legs), vs the 62.36 ms on this chart. In theory they
| could reduce latency by ~23 ms then with a truly direct path.
| To do better would require abandoning fiber.
| legulere wrote:
| Hollow fiber would almost reach full light speed:
|
| https://www.nojitter.com/enterprise-networking/hollow-
| fiber-...
|
| https://www.rp-photonics.com/hollow_core_fibers.html
| scottlamb wrote:
| Neat! I'd never heard of that. I look forward to the day
| when it's generally used rather than yet another cool
| technology wasted on stupid high-frequency trading.
___________________________________________________________________
(page generated 2021-04-30 23:02 UTC)