[HN Gopher] How Raw sockets behave differently in macOS and Linux
___________________________________________________________________
How Raw sockets behave differently in macOS and Linux
Author : adityaathalye
Score : 104 points
Date : 2024-09-14 04:14 UTC (4 days ago)
(HTM) web link (swagnik.netlify.app)
(TXT) w3m dump (swagnik.netlify.app)
| jiehong wrote:
| Speaking of sockets on Mac, anyone knows if there is a
| documentation about how the network stack on MacOs works?
| (Routing, firewall, resolving, kernel extensions
| filtering/offloading, etc).
| donaldihunter wrote:
| This might be a good place to start:
|
| https://developer.apple.com/library/archive/documentation/Ne...
| egberts1 wrote:
| Thr stark difference between Linux and macos is that ONLY macos
| can filter inbound Ether packets by its destination process ID.
|
| Becasue BSD raw socket rules.
|
| That's how you get Sniffer to be performing waaaaay more
| intuitively on macos than at all on Linux.
| codetrotter wrote:
| Do you have a link to Sniffer?
| tankenmate wrote:
| You can do the same on Linux, just not easily from the command
| line; ebpf however can do all this and more. Once you have
| built it you can just create a program that you can use from
| the command line. There are guides out there to show you
| how.[0]
|
| [0] https://www.datadoghq.com/blog/ebpf-guide/
| carlmr wrote:
| I've been meaning to try out eBPF. I read a tutorial on
| writing a simple perf application in Rust, but I kind of lack
| the intuition to know what it's good for.
|
| Do you know a good overview of what can be done with eBPF?
| packetlost wrote:
| Think of it like small, isolated programs that run in a VM
| that is managed by the kernel and has access to many kernel
| APIs and datastructures without needing to do syscalls.
|
| I _believe_ they can both monitor and manipulate data as it
| flows through the OS primatives (sockets, file descriptors,
| etc.).
| Denvercoder9 wrote:
| nftables supports matching by cgroups, which arguably is a
| better idea than process IDs.
|
| https://wiki.nftables.org/wiki-nftables/index.php/Matching_p...
| donaldihunter wrote:
| It's fascinating watching someone write code for macOS while only
| referring to Linux and FreeBSD man pages.
|
| On macOS, 'man ip' gives all the necessary info about raw sockets
| and IP_HDRINCL:
|
| > Outgoing packets automatically have an IP header prepended to
| them (based on the destination address and the protocol number
| the socket is created with), unless the IP_HDRINCL option has
| been set.
|
| > Unlike previous BSD releases, the program must set all the
| fields of the IP header
|
| > Note that the ip_off and ip_len fields are in host byte order.
| chrsig wrote:
| In a lot of cases, the macos man pages leave a lot to be
| desired, or don't come up in a google search. I know I'm
| personally not in the habit of remembering apropros exists, so
| I can imagine that the first step to manpage discovery being a
| google search.
|
| And most of the time the docs are interchangable enough. Until
| they're not. At the raw socket layer, I'd still expect the
| macos docs to be consulted _eventually_.
| torginus wrote:
| This is why i f**ing hate macOS. Developers wax poetic about 'oh
| its a UNIX', but NOTHING works like Linux once you dig into it.
| Zsh is different from bash, subtle API differences, different
| filesystem, esp with stuff like /dev or /proc or /etc, no
| systemd, some cli utils have different flags, subtle differences
| like this.
|
| At least Windows is completely different and there's no
| expectation of compatibility, but with the mac you can start off
| thinking things would work, but they don't.
| tikkabhuna wrote:
| Its been a decade since I had a Mac, but it definitely gave me
| that uncanny valley feeling. So close that you start expecting
| certain behaviour.
|
| Windows and WSL feels better to me at least.
| freedomben wrote:
| Indeed, I've been Linux-only for a bit over 10 years now, and
| when I have to do some work on a Windows or Mac for work,
| Windows with WSL feels a lot better than modern macos. You'll
| encounter the occasional alien thing when Windows details
| leak through, but for the most part it just behaves like you
| expect. If I had to pick a mac or windows pc for daily
| driver, Windows would be an easy choice now (although as
| coming of age in the 90s when MS was evil, it's mind-blowing
| to hear me say this all!)
| simonh wrote:
| Having cut my teeth on Solaris (both flavours), AIX, HPUX and
| later Linux in the 90s, get off my lawn! ;)
| OskarS wrote:
| Linux != UNIX
| notepad0x90 wrote:
| > "no systemd"
|
| I'd call that an excellent feature and more true to Linux than
| Linux is to itself today.
|
| Even Linux is not a true UNIX,it is UNIX-like. You can't expect
| macOS to be like Linux, it should be quite the other way
| around.
| donaldihunter wrote:
| Hah, Linux deviates probably as much from UNIX as macOS does.
| scoodah wrote:
| Some of the things you point out here are things you cannot
| rely on in Linux consistently, either. You can't rely on
| Systemd existing in Linux. You can't rely on bash being the
| shell in Linux. Depending on the users system and distribution
| these tools may or may not exist.
|
| Also the difference in some of the tools CLI flags wind up
| boiling down to being the BSD versions of them as opposed to
| the GNU versions of them. Which, again, isn't a problem
| isolated to MacOS.
|
| If you're considering all Unixes everything you said basically
| still applies even if you remove MacOS from the equation. Unix
| isn't just Linux.
| PaulDavisThe1st wrote:
| EDIT: I'm stupid and a bad reader.
| scoodah wrote:
| What are you talking about? I'm responding directly to a
| comment that called those two things out.
|
| My point was that systemd and bash are _not_ part of a
| linux system, and you can't count on them being there.
| PaulDavisThe1st wrote:
| You're right, and I was not reading carefully (the grayed
| out GGP didn't help)
| scoodah wrote:
| No worries. Also definitely not something to call
| yourself stupid over in your edit :)
| tzs wrote:
| Tip: when a greyed out comment is difficult to read,
| click on its timestamp. That will open a view on just
| that comment and its replies and it will not be greyed
| out in that view.
| dmvdoug wrote:
| That is a great tip, I had no idea. Thanks!
| dgfitz wrote:
| I appreciate your candor, I wish it was more common.
| torginus wrote:
| The only practically relevant Linux environments you can't
| expect these things to exist are stripped down distros used
| for docker images, like Alpine (where you don't expect much
| from the system anyways) and Android (not sure about the
| latter, haven't used it in production in quite a while, but
| afair it's as non-standard as it gets).
| zbentley wrote:
| Not true at all.
|
| > Zsh is different from bash
|
| Ubuntu has had a non-Bash system shell (dash is /bin/sh)
| since 6.10.
|
| > no systemd
|
| Without getting into objected-on-principle distributions
| like Devuan, tons of stripped-down and embedded linux
| distros besides Android and Alpine don't have systemd, e.g.
| DD-WRT, OpenWRT, TinyCore. Heck, launchd is _more_ similar
| to systemd than SysV init was.
|
| > /dev or /proc or /etc
|
| Other than the parts specified by POSIX, the /dev
| filesystem is the wild west across tons of Linux and Unix
| systems, and expecting further standardization in behavior
| from it is a fool's errand (e.g. device name selection,
| device ID assignment order at boot, presence of metadata-
| symlinked directories like by-id, overlay/loopback devices
| ... all of these are highly variable even across modern,
| systemd-using Linux distros). I do miss /proc, though I
| don't miss the file descriptors it costs. /etc and XDG
| standards for configs are nice conventions to be sure, but
| a significant minority of Linux software breaks with those
| conventions (all-/opt install locations, anyone?).
|
| > different filesystem,
|
| Sure, APFS isn't ext3/4. Neither are XFS, ZFS, BTRFS, and
| so on. ext3/4 is a tenuous standard at best, and many
| businesses make a point of preferring other filesystems on
| Linux.
|
| ...like, I think you might have just been getting lucky and
| using a fairly similar set of Linux distros such that MacOS
| was a big switch. There are plenty of valid beefs with
| MacOS's divergences (don't get me started on an OS that
| claims certified POSIX compliance while providing a
| plethora of low-level system APIs documented to break in
| the presence of fork(2), or the not-quite-superset
| clusterfuck that is FSEvents vs. MacOS's not-quite-BSD-
| complete kqueue implementation, or the Sophie's choice of
| bizarre xattrs behavior vs. the historical accident that is
| aliases), but what you listed isn't even remotely standard
| Linux behavior, much less Unix behavior.
| mannyv wrote:
| Having used hp-ux, sunos, solaris, aix, plan 9, various flavors
| of linux and a bunch of embedded OSs, VMS, and various Windoze
| systems I formally welcome you to the Real World.
|
| The Real World - it's got more stuff than you can imagine.
| torginus wrote:
| This is false plurality - while some bank or legacy
| datacenter might have these OSes in production, 99% of the
| time, you'll only encounter Linux in a professional server
| environment (or the Real World as you call it).
| kelnos wrote:
| How does that matter, though? You're complaining that macOS
| is different from Linux. That's the salient point. And,
| duh: they're different OSes. macOS is just as different
| from Linux as any other OS out there is. And macOS is
| _closer_ to Linux than some (like Windows).
|
| You initially said, seemingly implying this was a "false
| cool thing":
|
| > _Developers wax poetic about 'oh its a UNIX'_
|
| I don't think they meant what you think they meant by that.
| When people said that (back in the early '00s), people were
| familiar with Linux, yes, but they also interacted with
| several other commercial UNIXes that were still in wide use
| at the time (like those mentioned by the GP), as well as
| the BSDs. Being able to treat macOS like a UNIX (or, more
| specifically, like a BSD) was a _big_ step up in
| convenience from Mac OS Classic.
|
| No one expected macOS to be like Linux, at least not in the
| ways you are talking about.
| severino wrote:
| You would probably feel the same if, today, you were to use
| Solaris. And you won't say it's not UNIX, right?
| icehawk wrote:
| How many non-Linux UNIX operating systems have you used? It's
| Linux that's the weird one.
| kelnos wrote:
| That's a weird objection. Linux isn't UNIX, technically. All of
| the current and former UNIXes had various incompatibilities.
| /proc was never standardized. /dev has some standard names, but
| not many. Not sure what you mean about /etc, that's up to
| whatever's installed in userland. systemd is more or less Linux
| only; it's weird to blame other OSes for not having it. (The
| funny thing is that systemd has functionality in part inspired
| by macOS's launchd.) The ext* family of filesystems (and
| certainly stuff like btrfs) is a Linux thing; other UNIXes used
| different file systems.
|
| CLI utils are an interesting thing: the BSDs have always(?) had
| their own, and don't use GNU coreutils (which is what Linux
| uses). There wasn't really much coordination over the decades,
| so they went in different directions. Most (all?) of them
| should be POSIX-compliant (possibly after setting an env var,
| at least in GNUs case), so if you restrict yourself to
| functionality specified in POSIX, you should be fine.
|
| I don't really like macOS, but your specific criticism of it
| doesn't really hold water. If you're going to hate on macOS for
| this stuff, then you also have to hate on all the BSDs, which
| seems a little silly and unwarranted.
|
| > _with the mac you can start off thinking things would work,
| but they don 't._
|
| If you're expecting things to work on macOS just like on Linux,
| then your expectations were way off in the first place. If you
| want cross-platform compat in these areas, you'll have to stick
| with POSIX. Which sucks, especially since no one really
| implements POSIX correctly, but that's just how it is, and has
| always been.
| eptcyka wrote:
| Why would anyone resort to reading FreeBSD documentation when you
| can read the source and manpages for macOS?
| swagnikdutta wrote:
| I guess I was tunnel visioned and lost clarity (and also,
| common sense). While searching for reasons why the code (from
| Graham King's blog) had worked on Linux and not on macOS, a
| stack overflow answer [1] from 2015, from a person who had also
| been on the same journey, directed me to that page [2] -- that
| talked about raw socket peculiarities in FreeBSD. So I dived
| straight into FreeBSD docs.
|
| [1]- https://stackoverflow.com/a/32599757/3728336
|
| [2] - https://cseweb.ucsd.edu/~braghava/notes/freebsd-
| sockets.txt
| ks2048 wrote:
| Apple has a relatively new (I think introduced in 2018)
| "Network.framework" Swift API [1] for working with raw sockets.
|
| Maybe this post is dealing with the layer below - I'm out of my
| comfort zone with networking - but I recently built a custom
| protocol using Network and it's working well for me so far.
|
| [1] https://developer.apple.com/documentation/network
| kuhsaft wrote:
| ICMP is lower-level than what Network.framework provides.
|
| CFSocket would be the Swift/Obj-C API.
|
| https://developer.apple.com/documentation/corefoundation/cfs...
| mzs wrote:
| There's no NWProtocolRAW class so no:
| https://developer.apple.com/documentation/network/building-a...
| mzs wrote:
| This post links to some good documentation about sniffer related
| options:
| https://wiki.wireshark.org/CaptureSetup/Offloading#checksum-...
|
| The sysctls are net.link.generic.system.hwcksum_tx and
| net.link.generic.system.hwcksum_rx on macos.
___________________________________________________________________
(page generated 2024-09-18 23:01 UTC)