[HN Gopher] The journey of an internet packet: Exploring network...
___________________________________________________________________
The journey of an internet packet: Exploring networks with
traceroute
Author : marinesebastian
Score : 299 points
Date : 2024-08-23 09:10 UTC (5 days ago)
(HTM) web link (sebastianmarines.com)
(TXT) w3m dump (sebastianmarines.com)
| samstave wrote:
| Tracert is more powerful than you think:
|
| There are a lot of good talks on Tracert.
|
| This one is pretty good:
|
| https://www.youtube.com/watch?v=jGYAW5z6BJc
|
| The article OP links to is only talking from the perspective of
| an internal network. Tracrt in a 10. network is boring info.
|
| In the vid I posted, he gives you really good common advice about
| using tracert to show you actually the physical layout of the
| path:
|
| https://i.imgur.com/LxN9Mr4.png <-- Using the DNS name of the
| router is great, because us network nerds like to use naming
| conventions in a graph format: so you can tell that its edge
| router number N at location B in City X and using tracert - you
| get to see the national networks the packet hits.
|
| THen by seeing the carrier, you can also see where there is not
| just a change in carrier, but also that indicates that at that
| location is a datacenter....
|
| You can go onto DatacenterMaps and find out who/what/where a DC
| is....
|
| (There is a really exceptional tech talk on tracert thats quite
| long that goes into bitlevel detail of weaponized tracrt - but I
| cant find it)....
|
| ---
|
| WRT DataCenterMaps -- There was an HNer that posted about mapping
| nuclear facilities (active and decommissioned) - and by using his
| map, the UnderSeaCableMap and the DataCenterMap - then by looking
| at shipping supply-chains for components
| used_by/made_by/received_at companies that are either Data
| Centers, or NVIDIA - we could track where large scale AI
| componentry is being installed into what data centers, which are
| fed by which Nuclear Power Plants, who have to report on their
| Consumption Graph - and which Cable Infra is likely feeding each
| DC.
|
| We can see where AI traffic flows - and by using tracert at a
| deeper level - we can see exactly the AI's Physical NeuroNet' -
| and find a way to measure its power consumption and physical
| footprint.
|
| ---
|
| HNer @externedguy " _..built interactive map of active &
| decommissioned nuclear stations/reactors_"
|
| https://news.ycombinator.com/item?id=41189056
|
| (I correlated the Nuclear reactor locations with DataCenters,
| undersea cable endpoints (which will be near both nukes and
| datacenters)
|
| As they could be layers - and we track shipments and we can see
| where AI consumes:
|
| ---
|
| ...if we add the layers of the SubmarinCableMap [0] DataCenterMap
| [1] - and we begin to track shipments
|
| And
|
| https://i.imgur.com/zO0yz6J.png -- Left is nuke, top = cables,
| bottom = datacenters. I went to ImportYeti to look into the
| NVIDIA shipments: https://i.imgur.com/k9018EC.png
|
| And you look at the suppliers that are coming from Taiwan, such
| as the water-coolers and power cables to sus out where they may
| be shipping to, https://i.imgur.com/B5iWFQ1.png -- but instead,
| it would be better to find shipping lables for datacenters that
| are receiving containers from Taiwan, and the same suppliers as
| NVIDIA for things such as power cables. While the free data is
| out of date on ImportYeti - it gives a good supply line idea for
| NVIDIA... with the goal to find out which datacenters that are
| getting such shipments, you can begin to measure the footprint of
| AI as it grows, and which nuke plants they are likely powered
| from.
|
| Then, looking into whatever reporting one may access for the
| consumption/util of the nuke's capacity in various regions, we
| can estimate the power footprint of growing Global Compute.
|
| DataCenterNews and all sorts of datasets are available - and now
| the ability to create this crawler/tracker is likely full
| implementable
|
| https://i.imgur.com/gsM75dz.png https://i.imgur.com/a7nGGKh.png
|
| [0] https://www.submarinecablemap.com/
|
| [1] https://www.datacentermap.com/
|
| ----
|
| And 8 months back I posted:
|
| _In the increasingly interconnected global economy, the reliance
| on Cloud Services raises questions about the national security
| implications of data centers. As these critical economic
| infrastructure sites, often strategically located underground,
| underwater, or in remote-cold locales, play a pivotal role,
| considerations arise regarding the role of military forces in
| safeguarding their security. While physical security measures and
| location obscurity provide some protection, the integration of AI
| into various aspects of daily life and the pervasive influence of
| cloud-based technologies on devices, as evident in CES GPT-
| enabled products, further accentuates the importance of these
| infrastructure sites._ _Notably, instances such as the seizure of
| a college thesis mapping communication lines in the U.S.
| underscore the sensitivity of disclosing key communications
| infrastructure._
|
| _Companies like AWS, running data centers for the Department of
| Defense (DoD) and Intelligence Community (IC), demonstrate close
| collaboration between private entities and defense agencies. The
| question remains: are major cloud service providers actively
| involved in a national security strategy to protect the private
| internet infrastructure that underpins the global economy, or
| does the responsibility solely rest with individual companies?_
|
| (There was talk on this topic recently in the news)
|
| EDIT:
|
| If you like this sort of networking - then courses/material like
| this are great:
|
| https://www.youtube.com/watch?v=Ih3KgQnT6T0 <-- network recon,
| scanning, countermeasures - failryl vanilla, but concise.
|
| ---
|
| I still cant find the defcon-style talk that really dives into
| tracert sorcery....
| sulandor wrote:
| seems like clickbait
|
| traceroute uses udp to a high-port for discovery, not icmp-echo.
| gatnoodle wrote:
| traceroute supports multiple protocols, not just UDP and ICMP.
| sulandor wrote:
| true, the type of packet does not really matter.
|
| my apologies if this misconstrued my point about the lacking
| quality of the article.
| matja wrote:
| Windows tracert uses ICMP
|
| Source: https://support.microsoft.com/en-gb/topic/how-to-use-
| tracert...
|
| > The TRACERT diagnostic utility determines the route to a
| destination by sending Internet Control Message Protocol (ICMP)
| echo packets to the destination
| sulandor wrote:
| true, most computers don't run windows though.
|
| anyways, the type of packet does not really matter.
| bc569a80a344f9c wrote:
| I agree that this doesn't seem to have been written by someone
| that actually understands the topic.
|
| Even the network diagram at the beginning is not very good. Can
| you create network architectures where 10.0.0.1 and 10.0.0.2
| are not layer 2 adjacent? Yes, but they're fairly complex and
| would imply that a lot of other necessary information is
| missing from the diagram. And should you use such an
| architecture as an example to explain traceroute? Absolutely
| not. It's hard to imagine someone with even a CCNA level
| understanding of networking coming up with this.
| Hikikomori wrote:
| /31 linknets are common in the ISP world, it might even be
| /32 loopback of those devices, who knows.
|
| I do agree that its bad but not for the same reason. On the
| diagram it looks like router IPs are their loopback IPs
| rather than the link IP, as we know a traceroute response
| comes from the interface the packet came in and that IP is
| used in the response. Seems like the creator tried to
| simplify the diagram and excluded linknets but made it more
| confusing instead.
| xraystyle wrote:
| Glad I'm not the only one who was bothered by this. No subnet
| masks in the diagram, and I'm supposed to assume that
| 10.0.0.1 and 10.0.0.2 are not in the same broadcast domain.
| So my default gateway is what, 10.0.0.0, and that router has
| a route to 10.0.0.2/? with a next-hop of 10.1.0.5?
|
| Wackiest network I've seen in a while, and I've seen some
| real winners.
| TabTwo wrote:
| As a firewall-admin this comment really annoys me.
|
| Heres why: Lots of calls from people complaining their
| application is not working because the network is broken. Look!
| at! this! traceroute! supporting! my! complaint!!!
|
| Yeah, it is because udp from any to any is blocked on the
| firewall for good reasons while icmp traceroute is open. And be
| carefull because -I specifies the outgoint interface on some
| os, not the use of ICMP.
| sulandor wrote:
| most applications don't run over icmp so it is indeed
| sensible to not use it as a default.
| dopylitty wrote:
| On the other hand traceroute isn't real[0]
|
| 0: https://gekk.info/articles/traceroute.htm
| chgs wrote:
| > It is, generally speaking, not possible to call AT&T and say
| "Hey, when I try to ping one of your subscribers in California
| from a Level3 circuit in New York, I'm hitting a routing loop."
|
| I've reported similar things to my isp and they've changed
| their lock prefs to send the traffic via a different peer and
| bypass the problem.
| d1sxeyes wrote:
| There are more private subscribers that pretend they know
| what they're talking about than actually do. Probably makes
| sense to ignore a good proportion of the nonsense people
| throw at ISP L1 teams.
| yusyusyus wrote:
| this article is informative but misleading. i do the testing at
| a big ISP (your packets go over us for at least something). we
| test traceroute. we complain to vendors when traceroute doesn't
| work right. we do investigate weird traceroutes.
| lostemptations5 wrote:
| > your packets go over us for at least something
|
| I'm in South East Asia -- are you sure?
| AureliusMA wrote:
| I think he meant "us" as ISPs in general.
| dspillett wrote:
| _> Traceroute is a filthy hack_
|
| This is said like it is a bad thing...
| globular-toast wrote:
| Once you know how routing is supposed to work a nice homelab
| challenge is to set up a redundant route between your PC and the
| internet, like going through your NAS or something, and watch it
| failover when you unplug your network cable.
| chgs wrote:
| It always amazes me in-depth HN gets on pretty much any subject,
| yet the most basic cursory network page gets massive response
|
| Is there simply a total lack of understanding of how networks
| work in the tech community?
| mrbluecoat wrote:
| I'm often tasked with explaining technical concepts to people
| with entry-level knowledge so simple, clear documents like this
| are very helpful resources.
| chgs wrote:
| Would a "a for loop in node js" or "serving an webpage in
| Python" get so much attention?
| RALaBarge wrote:
| Make one, and lets find out
| immibis wrote:
| People who want to learn how ISP networks work can go and join
| DN42, a LARPing copy of the Internet.
| StrLght wrote:
| Networking is a black box for many developers.
|
| Why would it be any different if you're developing a CRUD
| application with a bunch of SREs / datacenter engineers on the
| line? They are very good at abstracting networking away from
| software engineers.
| js2 wrote:
| Yes. I've worked across a range of companies for small startups
| to Fortune 500s for nearly 30 years, interviewing for positions
| from system administrators to dev ops to programmers, beginners
| to architects. There's just not a lot of generalists out there.
| It always surprises me how far folks can get in their careers
| with a very narrow knowledge base.
|
| A couple years ago I was bringing a Java programmer up to speed
| on some C code just to learn they had no idea what a call stack
| was or how it worked. They were familiar with the stack as a
| data structure, but had no idea how a CPU worked or that it has
| a stack pointer register.
|
| As long as the abstractions don't leak, I guess everything is
| fine. I mean, I can't really tell you at a physics level
| exactly how semiconductors work. At best I can hand wave an
| explanation.
|
| That said, the lack of even hand waving knowledge about how the
| internet works among professionals who use it every day
| continues to surprise me.
| hunterbrooks wrote:
| Yes. Learning to "program" has never been easier, but nitty
| gritty stuff like networks and routing is still much more
| technical than the avg SWE
| ru552 wrote:
| Wait until DNS comes up again. I don't know how many DevOps,
| SRE, Senior Architects, etc. that I've worked with that just
| don't understand how names are resolved. There's a wide array
| of specialists that don't understand general fundamentals.
| icedchai wrote:
| I also see this. Getting an MX record or TLS cert deployed is
| often a source of confusion, even with "seniors.'
| HappMacDonald wrote:
| Personally, I understand the fundamentals until they all
| change and DNS has had a lot of outgrowths relating to
| encryption, Email spam prevention, IPv6, etc that have made
| things a lot muddier for me over time.
| toast0 wrote:
| As someone with a good deal of networkimg knowledge, yeah, most
| people don't have very much.
|
| You can do a lot without knowing anything about routing, and
| very little about protocols. And all those network people
| always seem grumpy, why learn about it if you don't want to be
| grumpy. ;p
|
| Edit to add: how networking works in reality and how networking
| works in books and coursework tends to be quite different as
| well. GeoDNS and Anycast IP are a big conceptual jump for
| people who only understood networks from coursework. There's
| also a common assumption that routing protocols favor least
| distance routing which is certainly not the case. I may be
| obsessed with path MTU problems, but that's a real hard concept
| to explain to experienced tech people with slim networking
| knowledge. Also, a lot of people seem to believe the OSI model
| is fact and not a model of the world; it's not unuseful, but
| there's lots of hardware and software that operates at more
| than one OSI layer simultaneously and people stuck on a rigid
| model can have an incorrect view of the world which impedes
| their problem solving.
| makestuff wrote:
| Yes, I started my career in big tech and everything was so
| abstracted away I never really knew how it all worked as a
| junior. When I went to a startup and it was not longer all
| abstracted away I learned a ton about how it works (we were
| working on something cybersecurity related).
|
| When I went back to big tech it really helped me out just
| knowing basic things about networking. For example, my manager
| joked that people should have to take an intro to DNS as part
| of their on-boarding to big tech.
|
| I don't fault people for not knowing either, it was just never
| really a priority and the point was you shouldn't have to care.
| You should become an expert in your business domain and spend
| time solving business problems. Whether or not that was the
| right approach is up for debate though.
| spelunker wrote:
| Absolutely. I'm a software dev and have been doing this for
| many years, but never had to think about the network much. I
| recently went down a BGP/AS rabbit hole and found it
| _fascinating_, and that's just the tip of the iceberg - go up
| or down the OSI stack and there's all kinds of stuff I didn't
| know about!
| gosub100 wrote:
| If the tools to set up a network actually did what I asked them
| to, this page would be simple. In another comment you compared
| it to a node.js script.
|
| The difference between networking and programming is when you
| call a function it will actually do what you say, every time.
| In a network you can waste hours and get absolutely nowhere.
|
| I've been in IT and software for 25 years at various levels
| both as a job and a hobby, and I can confidently say that
| networking is the biggest waste of time, most frustrating, and
| least rewarding aspect of working with computers. No matter
| what I try to set up, it never, I mean _never_ works the way I
| want it to. And I 've done plenty of projects to get a healthy
| sampling, some of them include NFS/CIFS, RDP, DHCPD+PXE, VPN,
| IPFW, BIND, iptables, SMTP/IMAP. Working with anything
| networking is absolute hell. Every single time. Guaranteed.
| elevation wrote:
| > NFS/CIFS, RDP, DHCPD+PXE, VPN, IPFW, BIND, iptables,
| SMTP/IMAP
|
| The hardest part of configuring the services you list is not
| the networking, but all the non-networking topics you need to
| be familiar with: zone file syntax; cipher suite selection;
| OS-specific security policy and permissions models; PKI
| administration; PXE ROM peculiarities, etc.
|
| Once you're confident with arp, addressing/subnetting,
| routing tables, and a few other tools like netstat and `dig`
| you can quickly eliminate the network as the cause of the
| issue and focus on grokking the application itself.
| gosub100 wrote:
| Then we have different definitions of the word. That's
| fine, the top comment at the time of my posting made it
| sound like "networking" was understanding the interactions
| of optical repeaters on the glass cable. To me "networking"
| means actually using network applications for my benefit.
| GaryNumanVevo wrote:
| Yes and that's why hiring an experienced network engineer is
| incredibly goated
| wil2095 wrote:
| If the destination machine has disabled ping, what response is
| recieved?
| dec0dedab0de wrote:
| nothing from that machine, but the way the ttls work, it
| doesn't affect the responses from routers along the way. Same
| if the destination doesn't exist at all.
| mannyv wrote:
| Totally untrue. Network admins will often disable traceroute
| responses because security.
|
| Edit: the less someone knows about your internal topology the
| better. Security through obscurity does work.
| ru552 wrote:
| ~~ Security through obscurity does work.
|
| As a layer.
| Hikikomori wrote:
| Windows disables ping by default for what I'm guessing is
| security.
| teddyh wrote:
| "ping" is ICMP ECHO_REQUEST and ICMP ECHO_RESPONSE. Traceroute
| uses ICMP TIME_EXCEEDED. So blocking only "ping" will not
| affect traceroute. And if you block _all_ ICMP, you break your
| own internet: <http://shouldiblockicmp.com/>
| newman314 wrote:
| The number of companies that I've encountered that wholesale
| block ICMP because "security" is painfully high.
| myself248 wrote:
| Traceroute doesn't see 90% of the machines your packet passes
| through.
|
| When your packet leaves a router at some-pop-some-port-wherever,
| that fiber isn't usually the same piece of glass that plugs into
| the next hop. There's a whole chain of amplifiers and possibly
| multiplexers that handle it between here and there.
|
| Some of those provide reliable transport service, giving you the
| illusion of a fiber that never breaks, despite backhoes doing
| what backhoes do. Some of those shift the wavelength of your
| signal, letting you use cheap optics without troubling in the
| nuances of DWDM that packs your signal alongside dozens of others
| onto the same long-haul fiber. Some of those just boost the
| signal, along with all those others on the same fiber.
|
| But what all those machines have in common, is that none of them
| speak IP. None of them touch the payload. None of them are
| capable of decrementing a hop count. They're "part of the wire"
| as far as the packet is concerned.
|
| In my experience, this leads to two types of network engineers,
| separated by their understanding of these underlying realities.
| zamfi wrote:
| In traceroute's defense, it is trace _route_ -- for sure it
| doesn't tell you anything about the devices that don't operate
| at the IP level. Those devices either don't affect the IP
| "route" abstraction (e.g., signal boosters) or do so in ways
| that end up plausibly visible in the next hop.
|
| There's a reason the network layer abstraction is so strong,
| and an analogy to CPU ISAs here that have a similar strength.
|
| TCP, similarly, doesn't tell you when packets are
| deduplicated/resent/reordered/etc. -- that's just not part of
| the presented abstraction. Want that? Use UDP.
| PeterCorless wrote:
| "It doesn't show me the local digital loop carrier!"
|
| "Is the digital loop carrier doing any IP-level routing?"
|
| "No, but..."
| latchkey wrote:
| > _In my experience, this leads to two types of network
| engineers, separated by their understanding of these underlying
| realities._
|
| What's wrong with that? Certainly, someone with a complete
| picture is "better", but it is effectively two different types
| of problems. Do they need to be combined?
| aftbit wrote:
| I think you're reading the connotation into this. In my
| opinion, there is nothing wrong with being a higher level
| engineer. There's plenty of work "above the fold" so to
| speak.
|
| For example, I think it behooves every software engineer to
| have a general grasp of how CPUs work, what speculative
| execution is, how CPU caching and invalidation works, etc,
| but the average webdev doesn't really need to know this, and
| might run into some abstraction breaking implications only a
| few times in their career, while debugging a tricky bug or
| performance regression.
|
| I imagine something similar is true for network engineers.
| Likely many can work for years at a time without worrying
| about fiber signal repeaters, other than that one weird
| packet loss issue that ends up getting traced back to a
| marginal optic in a cable vault somewhere.
|
| Of course, none of this applies to the compiler engineers or
| the people who build the physical network layer. They are in
| the "second type" of engineer that actually needs to
| understand this stuff in depth in order to do their jobs on a
| day to day basis.
| latchkey wrote:
| > _I think you 're reading the connotation into this._
|
| I am reading the connotation into this and asking about it.
| 4 paragraphs of talking only about tech and then it
| diverged into a personal statement at the end.
| myself248 wrote:
| When everything's happy and packets are flowing as they
| should, there's absolutely nothing wrong with living at
| whatever layer of abstraction you wish.
|
| But when something is broken, troubleshooting is a different
| animal. When people make assumptions based on their
| abstractions, they can waste a lot of time chasing a problem
| that can't exist, because the abstraction hides several more
| layers of complexity. Their mental model of the problem
| relies on a fantasy which they may not even realize is a
| fantasy.
|
| It's unlikely for someone to accurately diagnose a fault
| that's several layers below where they're operating.
| Understanding that their abstractions are abstractions, and
| knowing when to hand things off to another layer of engineer,
| is critically important.
|
| I mention it here because people aren't generally
| tracerouting things unless they suspect breakage somewhere.
| It's a troubleshooting tool. But people whose mental model of
| the network _only_ goes as low as the IP layer, are unlikely
| to do anything useful with traceroute unless the fault also
| happens to be at the IP layer.
| archgoon wrote:
| Sounds like fun :)
|
| You should write up a blog post where you had to debug some
| weird gnarly issues. A lot of us higher up the tech stack
| are pretty far removed from the signals level issues you're
| talking about but would love to hear about a low level
| debug session!
| codethief wrote:
| I would love to read such a blog post as well!
| latchkey wrote:
| Thanks for the clarity in your response. I think we are
| pretty well aligned.
|
| It is human stacking of the OSI model. The person in the
| ditch repairing the line probably doesn't care about
| traceroute, but the person with traceroute probably should
| care about what happens in the ditch.
| RandomThoughts3 wrote:
| The OSI model exists for a reason.
|
| You don't think about the life of the electrons going through
| your processors when you code.
|
| Traceroute is a view at a certain level of abstraction. It also
| doesn't tell you if your packet was delivered using ethernet,
| wifi or a token ring. It just doesn't matter.
| mecsred wrote:
| It just doesn't matter until it does. It's fine to work at a
| higher level of abstraction. But people who understand a
| lower level of abstraction can do things people will call
| "impossible" with fault injection exploits, rowhammer etc.
| EvanAnderson wrote:
| The more you know about how something works the better
| equipped you are to handle things breaking. It's a safe bet
| that semiconductor physics and the gate-level construction of
| CPUs isn't necessary to be a good programmer, but not much
| further up that stack are things like understanding
| superscalar processor architecture, how caches work, how CPU
| protection levels work, etc. Knowing about those things, for
| sufficiently performance or security-intensive applications,
| can make a ton of difference.
|
| There's an analogy to networking there, too. You don't
| necessarily need to know how wave-division multiplexing, BGP,
| or DNS work to communicate over the Internet. For some
| categories of problems, though, a little bit of knowledge
| allows you to punch just a bit above your level.
| treve wrote:
| The OSI model hasn't been accurate representation of ip
| networking since pretty much day 1. It was made specifically
| for a different protocol, but in the stack we use today some
| layers are better split up in 2, some protocols exists in
| multiple layers. It's a nice metaphor but I think it's time
| to drop it!
|
| https://computer.rip/2021-03-27-the-actual-osi-model.html
| w7 wrote:
| Something that I've noticed that somehow ends up lost when
| people learn "the model" is the encapsulation aspects.
|
| I don't know if it's missing in people's course work or
| what, but I've had to use
| http://www.tcpipguide.com/free/diagrams/ipencap.png many a
| times to explain how stuff like VPNs work, correct
| statements like "firewalls don't have a routing table,
| firewalling is layer 4", explain things like MTU and
| payload size, or why certain traffic doesn't go beyond a
| broadcast segment normally.
|
| Personally I think this is one of the better
| visualizations.
| juancb wrote:
| To clarify my previous post, asymmetric routing is strictly
| an L3 behavior, and ECMP routing can also be an L3 behavior
| where a router chooses one of many equal-cost next hops based
| purely on data in the IP headers. The exact behavior of
| course depends on the ECMP load-balancing algorithm in use,
| whether it's per packet, per destination, or using a hash.
| And furthermore whether it's strictly IP or if it looks
| deeper into the packet and uses L3+L4 headers in its decision
| making.
|
| Both asymmetric routing and ECMP routing are visible from L3.
| In the latter case, the routing decision can utilize some L4
| data, so some L4 frobbing to get useful data points in
| practice is necessary for useful real-world diagnosis.
|
| I agree with others that the OSI model is a good metaphor and
| a framework for reasoning about networking, but it is far
| from perfect, and the reality for those designing and
| operating network protocols and devices is messy.
|
| MPLS is admittedly invisible and there isn't a thing you can
| do about it in the same way that you can't expect traceroute
| to give you a view of the switch ports it went through on a
| LAN. Of course it is useful to understand and keep in mind
| the fact that there may be, sometimes huge, gaps in your
| traceroutes. A sudden huge jump in RTT from one hop to the
| next can be confusing when trying to understand and
| troubleshoot a network issue.
| chrisstanchak wrote:
| CCNA baby
| remram wrote:
| OSI was supposed to be a competitor to IP and Ethernet.
| That's the reason it exists.
| FuriouslyAdrift wrote:
| A lot of ISP internal networks is going to be running over MPLS
| or encapsulation. You're not going to see any of those hops.
| The packet will just look like it teleported from the CPE to
| the Internet.
| Hikikomori wrote:
| Not sure why this matters for anyone not working for the ISP
| that those packets go through, and people working there would
| know about it. And they're mostly completely passive devices
| like filters.
|
| Traceroute is what we have, and its mostly enough as you can
| say at which hop things went wrong, then its up the ISP to
| figure out what devices they need to look at.
| mcmcmc wrote:
| It definitely helps to understand it when you are trying to
| explain to the ISP that it's their problem. Knowing what
| you're talking about will get your issue resolved much more
| quickly.
| groby_b wrote:
| That depends who's doing the talking. You're a large
| business with a dedicated connection and your own network
| engineering team? Yeah, totally matters.
|
| You're a single consumer, or a small business? Your dialog
| is limited to "Yes, I switched the router on and off. Yes,
| my local network works. No, I can reach the website over my
| phone, this is your problem".
|
| Repeat three times, then get the inevitable reply of "it'll
| be up again in the next 24 hours".
|
| Unless you're a network engineer, this really doesn't
| matter.
| Hikikomori wrote:
| How does it help the ISP if you're just guessing what they
| might have between two routers?
| yusyusyus wrote:
| as a predominantly wave-purchaser than builder, it matters when
| it matters and it doesn't when it doesn't.
|
| we get by fairly well assuming l1 is working as designed when
| the light is on and not throwing errors.. at least on day-to-
| day ops. planning/sourcing is a bit of a different thing.
|
| one note to your point though, back when people were still
| regularly picking up OC3s or doing frame relay or whatever, it
| was certainly more day-to-day to understand these things, but
| cant really fault anyone since most of that junk has gone away
| and we're left with the happy ethernet handoff. especially in
| small scale DC/enterprise. cloud, too, made us care less.
| juancb wrote:
| And on the active networking component side of things it
| doesn't touch on MPLS which also doesn't modify the IP headers.
| You can enter a network in New York and get MPLS switched
| across the country via active network devices all the way to
| California and have it show up as a single hop on traceroute.
|
| The explanation is great for a toy network bu in today's
| Internet the vast majority of routes are going to be
| asymmetrical and that requires running traceroutes from both
| ends and interpreting the results to find the faulty hop.
|
| The author also doesn't cover equal cost multipath (ECMP) which
| is everywhere. With ECMP you have multiple ports that lead to
| the same. Next hop and packets are hashed based on some part of
| the fourtuple, sometimes five tuple including the input Port.
| In order to track down the faulty link, you need to pro each
| and every one of the ports which requires that you use a higher
| level protocol like UDP. Using icmp in this case will not show
| you an issue some percent of time, providing false negatives
| which makes it less useful.
| johannes1234321 wrote:
| Below the IP layer there is always the physical layer somewhere
| with all the complications.
|
| Traceroute also doesn't show VPNs or other transports.
|
| But it shows the TCP hops and the timing, thus gives me the
| tools to see who to call or which routing table or wore to
| analyse or fix. When doing that I got to lok at that
| connection.
| graycat wrote:
| Right, there is also the issue of BGP, Border Gateway Protocol
| for the _core_ of the Internet?
| icehawk wrote:
| The smart man knows all of these things exist. The wise man
| understands when these things matter.
|
| DWDM protected waves, MPLS, etc, are all out of scope when
| someone does a traceroute, finds their traffic is going to
| Europe asks me about it, and then I see a local upstream has
| picked up Cogent as a transit provider, and Cogent is
| preferring their customer routes over routes from their Tier 1
| settlement free peers.
|
| But this article is very much a primer, it's using the unix
| format of traceroute and talking about ICMP.
| begueradj wrote:
| Which command does go beyond the "traceroute" limitations you
| mentioned ?
| nullindividual wrote:
| This is a significantly better technical presentation on how
| traceroute works[0]; for example, unlike the illustrations in the
| linked article, traceroute does not necessarily take a
| symmetrical return path; the return path is hidden from the
| client -- the client only sees the forward path.
|
| [0]
| https://archive.nanog.org/sites/default/files/traceroute-201...
| wang_li wrote:
| Traceroute doesn't even show you a path. It shows you a bunch
| of devices that happened to have a packet when its TTL expired.
| Every item listed in traceroute's output is a different packet
| and can take a different path towards the destination.
|
| On a different subject, why are people writing blogs about
| topics that are in the "literature" already?
| FujiApple wrote:
| It's not guaranteed to be accurate, but tracing using the
| UDP/dublin strategy with a fixed dest port and varying src
| port per round can help to identify and visualize valid ECMP
| flows. I recently wrote some guidance [1] on using Trippy in
| this way.
|
| [1] https://github.com/fujiapple852/trippy?tab=readme-ov-
| file#ud...
| Hikikomori wrote:
| Once I used iperf3 with 100 different udp streams/srcports
| to troubleshoot an issue, a small % of connections had >90%
| packet loss and this caused the connection pool of this
| service to fill up until it had only failed connections
| waiting to time out or going extremely slowly. ISP told me
| it was a broken linecard in a router, so packets were being
| dropped/corrupted on the backplane between the linecards.
|
| Traceroute and mtr didn't use enough ports to show the
| issue clearly.
| ta1243 wrote:
| I used this to identify some intermittent loss in one
| direction from I think Kyrgyzstan to the UK, There was a
| claim that sometime flows were fine and sometimes they
| weren't. Problem was an ECMP in Pakistan or Iran which was
| using udp port numbers to balance the flow. Could
| consistently get 25% loss with certain numbers and not with
| others.
| nullindividual wrote:
| In English, route is synonymous with 'path'.
|
| Feel free to submit a PR if you disagree:
|
| https://man.openbsd.org/traceroute
| wang_li wrote:
| This isn't an English language question. The technology in
| question doesn't guarantee a singular route or a path to a
| destination. Subsequent packets from an originating host to
| a destination host can follow different paths. Even during
| an activity such as a traceroute. A packet that reached its
| TTL at what is presented as hop 12 didn't necessarily
| traverse previous hops displayed by traceroute.
| nullindividual wrote:
| You'll learn pedantry gets tiring when you get older. How
| traceroute is understood and displayed to the end user
| constitutes a route. If you have an issue with how the
| developers of traceroute framed it, that's on you.
| lostemptations5 wrote:
| OP was the pedant in this case, unfortunately (for you).
| rudasn wrote:
| Hey! What literature would you recommend for learning more
| about this stuff (routing, firewalls, etc)?
|
| I'm learning as I'm going and my goal is to come up with a
| way to build secure/beyondcorp/zerotrust-style networks using
| only existing tools and established tech, unless absolutely
| necessary.
|
| Tools I'm using include wireguard, iptables, nftables, nginx,
| ping, traceroute, and nmap.
|
| Which articles, books, authors, or repos must I read and
| which concepts must I understand?
|
| Trying to find real knowledge about this stuff online is a
| nightmare!
| kccqzy wrote:
| Nothing can be assumed to be take a symmetrical return path.
| Even if I send a normal packet to some server on the Internet,
| the server's reply may or may not come back using the same
| path. As a former Google employee, I remember vaguely that this
| is something Google occasionally does on purpose.
| foobiekr wrote:
| Transport networks are often mpls, SR, whatever, and this
| approach reveals nothing inside the SP ASes for the most part.
| a-dub wrote:
| building a visualization system for traceroute in the mid 90s was
| how i taught myself java.
|
| traceroute itself was wrapped by a perl rpc wrapper and then a
| perl www cgi script would take a list of hosts and then reach out
| to all of them to ask them to traceroute each other, then a java
| applet would render an interactive graph that you could rearrange
| as you saw fit.
|
| interesting learning: internet routing can be asymmetric!
| jerf wrote:
| I miss traceroute. It seems like more and more, the linked page
| is hypothetical, and what I get in practice is a couple of hops,
| an arbitrary number of "* * *" lines, and maybe the last host or
| two. It's so nice when it works. $ traceroute
| youtube.com traceroute to youtube.com (142.250.114.91),
| 30 hops max, 60 byte packets 1 10.202.10.88
| (10.202.10.88) 0.384 ms 0.356 ms 0.342 ms 2
| 10.202.35.103 (10.202.35.103) 36.386 ms 36.370 ms 36.325 ms
| 3 10.202.32.4 (10.202.32.4) 36.301 ms 10.202.32.3 (10.202.32.3)
| 36.301 ms 10.202.32.4 (10.202.32.4) 36.287 ms 4
| 10.202.32.2 (10.202.32.2) 0.764 ms 1.279 ms 1.250 ms
| 5 lo0-0.gw2.rin1.us.linode.com (45.79.12.102) 0.610 ms
| lo0-0.gw1.rin1.us.linode.com (45.79.12.101) 0.663 ms 0.650 ms
| 6 ae62.r22.dfw01.ien.netarch.akamai.com (23.203.147.40) 0.986
| ms 0.881 ms 0.837 ms 7 72.14.204.254 (72.14.204.254)
| 1.164 ms 72.14.198.98 (72.14.198.98) 1.153 ms 142.250.47.248
| (142.250.47.248) 2.926 ms 8 * * * 9
| 209.85.251.24 (209.85.251.24) 1.082 ms 142.251.71.114
| (142.251.71.114) 1.302 ms 0.950 ms 10 142.251.234.214
| (142.251.234.214) 1.267 ms 216.239.58.16 (216.239.58.16) 3.296
| ms 142.251.66.192 (142.251.66.192) 20.440 ms 11
| 108.170.228.86 (108.170.228.86) 1.161 ms 108.170.228.82
| (108.170.228.82) 4.548 ms 108.170.228.81 (108.170.228.81) 1.746
| ms 12 108.170.231.7 (108.170.231.7) 3.484 ms
| 108.170.229.87 (108.170.229.87) 3.071 ms 142.251.70.211
| (142.251.70.211) 2.938 ms 13 142.250.236.158
| (142.250.236.158) 2.298 ms 216.239.51.220 (216.239.51.220)
| 27.388 ms 108.170.233.60 (108.170.233.60) 2.001 ms 14
| 142.250.224.27 (142.250.224.27) 2.085 ms 142.250.224.25
| (142.250.224.25) 1.994 ms 142.250.224.23 (142.250.224.23) 2.558
| ms 15 * * * 16 * * * 17 * * *
| 18 * * * 19 * * * 20 * * * 21 * * *
| 22 * * * 23 * * * 24 rr-in-f91.1e100.net
| (142.250.114.91) 1.997 ms 1.981 ms 1.967 ms
|
| Just an example. There's still a lot of stuff that works, but it
| just seems to me more and more often that when I have an actual
| problem I'm going to see a lot of * * *. So many things turning
| the requisite packets off.
|
| (To save people excited to see if I've leaked something important
| some time, this comes from my Linode-hosted website's box, not my
| home connection.)
| spelunker wrote:
| traceroute noob - what do the * * * mean? A host is choosing
| not to participate and dropping data or something?
| mccoyc wrote:
| Traceroute works by observing Internet Control Message
| Protocol (ICMP) time to live/hop count exceeded messages and
| noting the source IP of those messages. There's plenty of
| resources you can search on to learn more about this. When
| you see * * *, it means that for that given TTL value, no
| corresponding ICMP messages were returned to the source.
|
| It could be because they were filtered at the boundary of the
| network. If source IPs are private (RFC-1918 or carrier grade
| NAT space), those ICMP messages should get dropped. This is
| Best Current Practice (BCP) 38, if you wanted to read more
| about it.
|
| Ever want to know why IPv6 is so important? This is another
| reason why. Troubleshooting networks without globally
| significant IP addresses on the intermediate hops is a real
| pain.
| spelunker wrote:
| Great info, ty!
| packetslave wrote:
| To be fair, that entire traceroute never touches "the
| Internet". Everything after hop 6 is inside Google's backbone
| network, and everything before hop 7 is inside Linode/Akamai.
| alexchamberlain wrote:
| So it hits the Internet once between Akamai and Google I
| guess?
| packetslave wrote:
| I mean, KIND OF, if you squint. From the trace, Akamai and
| Google have a direct peering link (hop 6 looks like an
| Akamai edge/peering router -- Akamai owns Linode so that
| makes sense -- and hop 7 is a Google peering router).
|
| For it to be hitting "the Internet" you'd expect to see
| hops on a Tier 1 carrier like AT&T, Lumen, etc. and/or an
| ISP like Comcast, Spectrum, etc.
| Hikikomori wrote:
| It's still the internet, tier 1 carriers aren't special,
| they're just big.
| justinsaccount wrote:
| > But then, the path is broken, and we can't reach the
| destination computer.
|
| No. This is not what this output means.
|
| People should not read more into what traceroute says than what
| it actually does.
|
| * * * means that traceroute did not get an ICMP time exceeded for
| that probe. that's it. It says nothing about if you can actually
| reach the destination.
| runjake wrote:
| These days, us network engineers are more often using mtr to
| explore networks.
|
| https://www.cloudflare.com/learning/network-layer/what-is-mt...
| cwilby wrote:
| Seeing the title reminded me of "Warriors of the Net".
|
| We were genuinely told, as a class, to watch this video to learn
| how the internet worked.
|
| https://www.youtube.com/watch?v=RhvKm0RdUY0&themeRefresh=1
|
| Good times!
| lbeckman314 wrote:
| Somewhat related is this cool project by hack club [0] (not
| affiliated just a big fan!):
|
| how-did-i-get-here: "A tool/website/article by @kognise about how
| routing on the Internet works."
|
| Site: https://how-did-i-get-here.net/
|
| Github: https://github.com/hackclub/how-did-i-get-here
|
| [0] https://github.com/hackclub
| dkga wrote:
| Back in the day, all I wanted is to be able to do like Natalya
| Simyonova in Goldeneye, in the train. I recall during the late
| 90s there was a (Windows?) programme I knew that did this, based
| on traceroute/tracert I would suppose. Just a small memory to
| share here, maybe others had it too :)
| leinelissen wrote:
| Trace route has such a nice promise of untangling the internet
| into its constituent parts. We used it last year in an
| installation where we physicalised the internet as marble run.
| You would create a packet for any website, and the visit all the
| hops one-by-one across their various locations[1].
|
| [1]: https://youtu.be/9uIs0sh4iYU
___________________________________________________________________
(page generated 2024-08-28 23:00 UTC)