[HN Gopher] Go 1.16 will make system calls through Libc on OpenBSD
       ___________________________________________________________________
        
       Go 1.16 will make system calls through Libc on OpenBSD
        
       Author : lladnar
       Score  : 286 points
       Date   : 2021-02-02 04:39 UTC (18 hours ago)
        
 (HTM) web link (utcc.utoronto.ca)
 (TXT) w3m dump (utcc.utoronto.ca)
        
       | simias wrote:
       | Whenever the topic of Go bypassing the libc was brought up before
       | the concerns were usually dismissed by the Go devs, saying that
       | it was technically impractical due to some of Go's requirements
       | not matching the libc's semantics.
       | 
       | Does this mean that the Go developers managed to work around this
       | problem or that it was just a flimsy post-hoc justification for
       | chronic NIH syndrome?
       | 
       | TFA itself links to another blog post discussing "Some reasons
       | for Go to not make system calls through the standard C
       | library"[1] but as far as I can tell it doesn't explain why it
       | suddenly stopped being a problem on OpenBSD.
       | 
       | [1]
       | https://utcc.utoronto.ca/~cks/space/blog/programming/GoCLibr...
        
         | killingtime74 wrote:
         | The NIH one, as always with Go
        
           | knorker wrote:
           | It's a clash of two NIH worlds: Go and OpenBSD. :-)
        
             | masklinn wrote:
             | I mean OpenBSD is hardly the first let alone only system
             | which mandates going through libc (or equivalent) to
             | interact with the OS.
        
               | knorker wrote:
               | Don't get me wrong, it makes perfect sense. And Go seems
               | to aim more towards pragmaticism than OpenBSD's elegance
               | and correctness.
               | 
               | OpenBSD is deciding to invent their own paintbrush
               | without even looking at what other people are painting
               | with, while Go took the closest bucket of paint and threw
               | it on the floor, not realizing they were standing in a
               | corner while doing so.
               | 
               | And I say that as a happy user of both.
        
         | AnIdiotOnTheNet wrote:
         | > Does this mean that the Go developers managed to work around
         | this problem or that it was just a flimsy post-hoc
         | justification for chronic NIH syndrome?
         | 
         | No, there is almost certainly a bunch of headache having to do
         | system calls through libc for Go. It didn't stop being a
         | problem, there just isn't another option in OpenBSD's case.
         | UNIXs are tricky for anyone who doesn't want to use libc since
         | they typically define that as the official interface to the
         | system. Linux is the outlier here by keeping its syscall ABI
         | very stable.
        
           | simias wrote:
           | I understand that but if they're going to go down that route
           | for OpenBSD why not bring it to all platforms? While I can
           | believe that going the raw syscall route can be easier than
           | dealing with libc idiosyncrasies, it seems to me that
           | maintaining both interfaces depending on the target OS would
           | be trickier than just cutting your losses and going the libc
           | route everywhere.
           | 
           | After all as far I can tell it's not just an OpenBSD problem
           | since famously they got breakage in MacOS as well.
           | 
           | I must admit that I haven't taken the time to analyze in
           | depth the pros and cons here, but Go's history of NIH coupled
           | with the fact that basically every other mainstream language
           | manages to work by binding the libc leaves me very perplex.
           | 
           | In particular some of the points raised in the blogpost I
           | linked seem fishy to me. For instance errno being a global:
           | this is in no way a Go-specific problem (multithreaded C
           | couldn't run concurrent syscalls if it was true). In practice
           | errno is thread-local instead of being a true global. It's
           | explicitly documented in the man page:
           | 
           | > errno is defined by the ISO C standard to be a modifiable
           | lvalue of type int, and must not be explicitly declared;
           | errno may be a macro. errno is thread-local; setting it in
           | one thread does not affect its value in any other thread.
           | 
           | I can believe that there are other issues I fail to consider,
           | but again it works for everybody else, what makes Go so
           | special here?
        
             | wbl wrote:
             | Go wants low overhead multithreading and static linkage.
             | Glibc is inherently dynamically linked, and sets up threads
             | in ways Go doesn't want.
             | 
             | Error doesn't work for everyone else. It's a hack, you
             | wouldn't do it if it wasn't for backwards compatability.
        
               | IgorPartola wrote:
               | Not every libc is a glibc.
        
               | wbl wrote:
               | True but errorno is still a hack.
        
               | Blikkentrekker wrote:
               | But why does it want to not dynamically link against a
               | platform's _libc_?
               | 
               | What is the advantage in that? including that into a
               | binary seems about as senseless to me as including the
               | entire kernel in it.
        
               | badsectoracula wrote:
               | Not all platforms have a "libc". Linux is an example:
               | there is no standard c library and while glibc is very
               | common, there are several distributions that use other C
               | libraries like uClibc and musl. For example a gaming
               | handheld i have which is running Linux uses uClibc
               | 
               | Another example is Windows, the platform API does not
               | provide a C library (even MSVC has its own). While there
               | is an MSVCRT.DLL it is not recommended to link against it
               | as it is there only because some other software relies on
               | it and its semantics are for around Visual C++ 4 (IIRC).
        
               | pjmlp wrote:
               | Windows used to be an example like any other non-POSIX
               | OS, however as of Windows 10, the C standard library is
               | part of the OS.
               | 
               | https://devblogs.microsoft.com/cppblog/introducing-the-
               | unive...
        
               | Blikkentrekker wrote:
               | But _Linux_ is not a platform in general.
               | 
               | It is already next to impossible to write software that
               | requires " _Linux_ " and nothing more with all the kernel
               | functionality that can be enabled or disabled.
               | 
               |  _Linux_ is a component of many different platforms,
               | which indeed do provide different _libc_ s, but also
               | different t.l.s. libraries, different c.p.u.
               | architectures, different _Linux_ configurations and
               | whatever else.
               | 
               | As far as I know with respect to _Windows_ , it only
               | provides stable interfaces _via_ _C_ libraries, and does
               | not have a stable binary interface to the kernel
               | directly.
        
               | zlynx wrote:
               | For containers Go static binaries are great. Used to
               | build Docker containers for Kubernetes using a 8 MB Go
               | binary. That was all you needed. No libraries. No
               | "minimum OS' Alpine or Ubuntu image.
        
               | Blikkentrekker wrote:
               | That seems to be a sensible use yes, but those aren't
               | really any targeted and supported platform.
        
             | masklinn wrote:
             | > I understand that but if they're going to go down that
             | route for OpenBSD why not bring it to all platforms?
             | 
             | Because they _really_ don't want to, so they will avoid
             | doing it until forced to, as previously happened with e.g.
             | macOS.
             | 
             | > I must admit that I haven't taken the time to analyze in
             | depth the pros and cons here, but Go's history of NIH
             | coupled with the fact that basically every other mainstream
             | language manages to work by binding the libc leaves me very
             | perplex.
             | 
             | One of the issues Go has is it uses its own non-C stack.
             | Libcs generally assume a C stack and don't understand
             | growable movable stacks so they can get very cross when
             | called with an unexpectedly small stack (iirc Go defaults
             | to 2k while the smallest C stack I know of is 64k on old
             | OpenBSD, then macOS's 512k for the non-main threads).
        
               | aduitsis wrote:
               | Obviously I'm missing something here, because I was under
               | the impression that, regardless of how big the stack is,
               | the pages are going to be mapped (in the data structure
               | pointed to by the CR3 register) when accessed and no
               | earlier. Unless the allocation of the stack makes sure
               | that it is mapped in its entirety upfront and stack
               | access cannot incur minor page faults, which may not be
               | so outlandish come to think of it. Can you please provide
               | some clarification here? Thanks!
        
               | masklinn wrote:
               | I'm not sure what clarification I could provide given I
               | don't know what you're missing.
               | 
               | You're correct (AFAIK anyway) that stack pages are mapped
               | lazily. That doesn't change the fact that C stacks are
               | large allocations, while Go will only allocate a very
               | small stack (2k last I checked) per goroutine.
        
               | aduitsis wrote:
               | Yes, sorry, let me be more clear, I don't understand why
               | Go should allocate a small 2k stack (you are correct, the
               | Go stack size has fluctuated through the years, nowadays
               | it's 2k) and if it grows go to the trouble of copying it
               | to a larger structure? Instead of just allocating a big
               | stack that will grow lazily anyway?
               | 
               | Obviously the folks that created Go aren't stupid, far
               | from it, so there must be a real and valid reason. But
               | can't easily imagine what it is.
        
               | JulianMorrison wrote:
               | Go is very, very profligate with threads.
               | 
               | When you have 100,000 threads, big stacks add up.
        
               | masklinn wrote:
               | It's only vmem though is the point. It doesn't really
               | matter that you have 100000 stacks of 8MB when all of
               | that is vmem and you've got a page committed on each.
        
               | aduitsis wrote:
               | My point exactly, thanks.
        
               | JulianMorrison wrote:
               | You aren't thinking with big numbers. Small amounts of
               | _admin work_ add up. Virtual pages aren 't free. Go has
               | had to think very hard about what to pare down to avoid
               | dragging to a halt with orders of magnitude fewer
               | threads.
        
               | labawi wrote:
               | Each allocation also needs an entry in the page table
               | (actually a virtual memory tree). On x86/amd64 the fanout
               | is about 1:512.
               | 
               | If you have 8MiB stacks, then a minimally allocated stack
               | uses a 4KiB data page, but also 2MiB of address range
               | uses up a full 4KiB bottom level page table, and 8MiB
               | range takes up 4/512 x 4KB of 2nd level page table and so
               | on. So you use about 8.03 KiB RAM if you never touch more
               | than 4KB and your 8MB reservations are mostly grouped.
               | Some architectures have bigger pages, increasing fanout
               | but also the minimum allocation.
               | 
               | Contrast to 2KiB stacks without reservations/overcommit -
               | you use about 2KiB of usable RAM + (1/2) x (1/512) of
               | 4KiB 1st level page table + .. , assuming allocation are
               | again mostly grouped. Hence, for up to 2KiB of stack
               | memory you need about1 2.005 KiB of RAM. Works the same
               | for 16 KiB and even 64 KiB page sizes.
               | 
               | 100000 * (8MiB reserved, 1..4KiB used stack) needs
               | ~784MiB RAM.
               | 
               | 100000 * (2KiB reserved, 1..2KiB used stack) needs
               | ~200MiB RAM.
               | 
               | Note that, if you actually touch your reserved stack,
               | even once, your allocation can balloon to possibly tens
               | or hundreds of GiB (100k * 8MiB = ~800GiB), unless you do
               | complicated cleanup, while a segmented stack can keep the
               | allocations within reasonable efficiency, freeing any
               | excessive stack allocations in userspace.
               | 
               | 1 ignoring bookkeeping overhead in both cases, to keep
               | the calculation clear. Hopefully it isn't more than a
               | dozen bytes or so.
        
               | masklinn wrote:
               | > 100000 * (8MiB reserved, 1..4KiB used stack) needs
               | ~784MiB RAM.
               | 
               | Yeah so nothing really relevant, that's 1/20th of a
               | relatively basic dev laptop's memory for 100k threads.
               | 
               | And of course that's an insane worst-case scenario of 8MB
               | stacks, which the linux devs picked because they wanted
               | to add a limit to the stack but didn't really care for
               | having one, Windows uses 1MB stacks and macOS uses 512k
               | off-main stacks, so you don't need anywhere near 8MB to
               | get C-compatible stack sizes.
        
               | labawi wrote:
               | > It's only vmem though is the point
               | 
               | With 8M/4M/2M/1M/512k stack size, 4K page size, you get
               | about 800M/800M/800M/600M/500M RAM usage, of which only
               | 400M is usable, rest is overhead. At 16K page size, it's
               | ~1600M. Compared to 200M if using 2K side-by-side stacks,
               | in any configuration.
               | 
               | Yes, probably not too excessive, even though it is
               | noticeable when you spawn theads for anything and
               | everything, and run more than a single application on a
               | non-SV-developer PC.
               | 
               | I think main problems start when you actually touch more
               | than the base allocation of the stack (or just use 16K
               | pages). Maybe segmented stacks grow from 200M to 300M,
               | 500M or whatever you actually use at a given time (with
               | say 20-60% efficiency), but your C-compatible stacks
               | might go from 500M to 3G1 if you on average touch just
               | 32K of stack per thread with some unlucky function, even
               | though at any given time only the same ~100M of memory
               | actually stores useful information.
               | 
               | 1 or more, no idea how high it typically goes, but that
               | in itself is a nasty gotcha, and a likely reason you
               | won't find "lightweight" threading in combination with
               | native per-thread stacks
        
               | [deleted]
        
               | tuxychandru wrote:
               | This will mean a virtual memory allocation of 800 GB.
               | Such large allocations can fail, even if virtual,
               | depending on over commit settings.
        
               | masklinn wrote:
               | > Such large allocations can fail, even if virtual,
               | depending on over commit settings.
               | 
               | Given how Go has no issue being prescriptive as hell on
               | other things, I don't see why they couldn't just go "set
               | vm. overcommit_memory to 1 and fuck off", that's exactly
               | what e.g. Redis tells you.
        
               | pjmlp wrote:
               | Because that is very OS specific.
        
               | masklinn wrote:
               | AFAIK BSDs don't have a concept of overcommit because you
               | can't even turn it off.
               | 
               | And again it would hardly be the first time Go makes OS-
               | specific decisions.
        
               | simias wrote:
               | Goroutines don't map directly to OS threads though, do
               | they? So in practice it only matters for "hard" threads.
               | I have no idea about what the Go scheduler looks like
               | though, so I can't say if there's a direct relation
               | between the two (coroutine stack vs. thread stack).
               | 
               | Also the default pthread stack size is (usually) 2MB
               | which is indeed non negligible if you have a ton of
               | threads, but with pthread_attr_setstacksize you can lower
               | it. I can't seem to find the actual minimal size at the
               | moment but I have a vague memory that you can reduce it
               | to 16kB portably.
               | 
               | I'm currently working on a multithreaded Rust program
               | with a bunch of threads and strong memory constraints and
               | I use the runtime to reduce the stack to 64kB, it seems
               | to work just fine.
               | 
               | And again, given that the language is garbage collected I
               | can't help but find it a bit amusing that they're being
               | stingy with a few MB of VMEM per thread. I guess that
               | frees virtual memory for use in the balasts!
        
             | Quekid5 wrote:
             | > > [errno]
             | 
             | > I can believe that there are other issues I fail to
             | consider, but again it works for everybody else, what makes
             | Go so special here?
             | 
             | errno being thread-local doesn't really help all that much
             | with a M:N threading model -- the runtime is going to have
             | to be _extremely_ careful to not stomp all over it.
             | (Whenever calling into anything which uses errno. Of course
             | system calls do that as well, but it 's a much smaller
             | surface area than "most of libc".)
        
         | hda111 wrote:
         | One reason not to go through libc is the possibility to create
         | a pure Go userland like it's done with the u-root project that
         | is apparently in use at Google and resembles Busybox based
         | systems with a memory safe language.
        
           | macksd wrote:
           | I've actually wondered if there would be demand for a
           | Busbybox / Toybox clone built in Go for this reason. I have
           | zero need for it personally, but I think it would be fun to
           | make.
        
           | simias wrote:
           | Unless you plan to never run anything but Go on your system
           | you're going to need a libc at one point or an other. If
           | anything this decision reinforces the NIH hypothesis for me.
           | Almost like a prideful "NO C ALLOWED" stance which is not
           | really pragmatic IMO.
           | 
           | The only counterargument I can think off would be to remove
           | the libc footprint for ultra-small, go-only distributions but
           | I highly doubt that it's significant on any modern system
           | (even an embedded one) and if you care so much about reducing
           | the runtime footprint why would you go with a GC language in
           | the first place?
        
             | macksd wrote:
             | >> Unless you plan to never run anything but Go on your
             | system
             | 
             | That's actually exactly the use case for a lot of
             | containerized applications written in Go.
             | 
             | edit: I've worked on multiple applications like this and
             | there's the odd service where you need some other
             | dependency, and then that service has to start running on
             | Alpine Linux or something. But if the majority of your many
             | "microservices" are pulling in bits of userland from a
             | distro, they stop being "micro" very quickly which is a big
             | deal when there's a lot of them. It's far preferable in
             | that architecture if the entirety of your userland is your
             | own binary.
        
               | acoard wrote:
               | >Alpine Linux
               | 
               | From their website:[0] >"Alpine Linux is built around
               | musl libc and busybox"
               | 
               | I could be off-base here, but it seems to me the better
               | analogy would be some sort of "Go-Linux" that's like a
               | Docker linux OS written entirely in Go.
               | 
               | [0]: https://alpinelinux.org/about/
        
               | macksd wrote:
               | I wasn't making an analogy - I literally use Alpine Linux
               | when I need some common *nix tools but I want something
               | lighter-weight than the Docker images from more
               | mainstream distros. musl libc and glibc aren't 2
               | fundamentally different things - the former is just a
               | more permissive license and a lighterweight
               | implementation.
        
             | throwaway894345 wrote:
             | In containers it's entirely reasonable and desirable that
             | you only ship your app with minimal dependencies. I suspect
             | that the people who are grumpy about upstarts doing away
             | with libc are also inclined to argue that containers are
             | the work of the devil or some such.
        
               | macksd wrote:
               | >> containers are the work of the devil or some such
               | 
               | And ironically it was the BSDs and Solarises of the world
               | that had more fully developed similar concepts long
               | before they were mainstream on Linux. I'm not terribly
               | surprised by OpenBSD's stance here, though, and even
               | though I'm more in the Go / Linux ecosystem I can't
               | really argue - it's an unfortunate collision of very
               | different philosophies.
        
         | quotemstr wrote:
         | > Go devs, saying that it was technically impractical due to
         | some of Go's requirements not matching the libc's semantics.
         | 
         | I've always considered claims like this to be bullshit. I've
         | worked a lot with both libc level system calls and (for crash
         | report generation) direct system calls bypassing libc. They're
         | the same interface. There's nothing you can do with a raw
         | system call interface that you can't do through raw libc ---
         | with rare exceptions for things like libc not having system
         | call wrappers for some new system call or legacy ill-advised
         | emulation like in fallocate. None of these exceptions is a
         | justification for bypassing libc for calling open(2) or
         | write(2).
         | 
         | As far as I can tell, the real reason Go eschewed libc system
         | call interfaces is that the Go developers want to think of
         | themselves as "not C".
        
           | ominous_prime wrote:
           | Go has a different calling convention and stack layout from
           | C, and wanted completely static binaries which you cannot
           | always get when linking to the standard libc implementations.
           | What is "bullshit" about these compatibility concerns? It's
           | not like _Ken Thompson_ was unaware of how to use libc.
        
             | quotemstr wrote:
             | Yet Go can call libc functions just fine on many OSes. Why
             | can't it on Linux? Every single other managed runtime ---
             | Mono, Java, Python --- can call through libc just fine.
             | There is zero technical case for Go not doing the same
             | thing.
             | 
             | The bullshit lies in these FUDlike insinuations that using
             | libc would limit Go in some way. These insinuations are
             | never backed up with technical specifics. I don't care that
             | famous names are involved with Go: the presence of these
             | people doesn't make Go's behavior correct or necessary.
             | 
             | There is zero technical case for Go doing what it does on
             | Linux. You can make a "fully static binary" (which is a
             | terrible idea anyway) with libc. Nobody should be making
             | static binaries.
        
             | int_19h wrote:
             | There are many languages and runtimes that use their own
             | calling conventions - not the least of which is C++ - and
             | even stack layouts. The problem with Go is specifically the
             | spaghetti stack that they need for coroutines. But that is
             | an inherent design issue - a systems programming language
             | is supposed to play well with the system, and that means
             | being able to use the standard OS APIs _properly_. Which on
             | basically all platforms other than Linux means going via
             | libc or equivalent (e.g. Win32). Go designers just
             | unilaterally decided that the rules don 't apply to them,
             | because reasons. And the end result was stuff like this:
             | 
             | https://github.com/golang/go/issues/16606
             | 
             | So now they're gradually backtracking on all platforms -
             | macOS switched to libc a while ago.
        
           | bborud wrote:
           | Those sound like assumptions. Can you back it up with
           | examples of what cases Go developers saw as problematic and
           | point out what they should have done instead (and that this
           | indeed does not represent a problem)?
        
         | masklinn wrote:
         | > as far as I can tell it doesn't explain why it suddenly
         | stopped being a problem on OpenBSD.
         | 
         | It doesn't. It's just that as previously with macOS, Illumos,
         | or Windows, the Go project ended up with its back against the
         | wall: in this case, ultimately only the OpenBSD libc will be
         | allowed to make syscalls so their choices are "use libc" and
         | "no Go on OpenBSD".
         | 
         | And in all honesty the followup
         | https://utcc.utoronto.ca/~cks/space/blog/unix/CLibraryAPIReq...
         | is _much_ more convincing as to why it 's a hassle to go
         | through libc.
        
         | tyingq wrote:
         | It's OpenBSD's plan to eventually shut down any other option
         | via system call origin verification. So golang has to bite the
         | bullet at some point.
         | 
         |  _" The eventual goal would be to disallow system calls from
         | anywhere but the region mapped for libc"_
         | 
         | https://lwn.net/Articles/806776/
        
         | hacknat wrote:
         | Normally I agree with anit-NIH sentiments, but why are we all
         | supposed to just have to be happy with a language ABI that's
         | been around since the 1970s?
        
           | simias wrote:
           | If it ain't broke...
           | 
           | Having a common entrypoint into the kernel we can hook into
           | is fairly valuable IMO, especially when said interface is for
           | the most part stable between all un _x (and to a somewhat
           | lesser extent even on Windows).
           | 
           | Having this lowest common denominator ABI makes interop
           | _relatively* painless. But clearly Go wants to eat the world
           | and doesn't seem to care very deeply about this. I suppose
           | it's a valid approach, even if I don't necessarily agree with
           | it.
        
             | convolvatron wrote:
             | If it ain't broke...
             | 
             | user supplied read addresses
             | 
             | getpwent() - actually really the whole notion of users
             | 
             | file permissions
             | 
             | aynch storage event completion (all asynchrony is pretty
             | broken)
             | 
             | system metadata control (sysctl, /proc, interface socket
             | messages...)
             | 
             | signals
             | 
             | ttys
             | 
             | errno
             | 
             | affinity interfaces ...
        
           | macksd wrote:
           | You would probably find yourself asking similar questions a
           | lot in OpenBSD land - they're very committed to stability and
           | security even if it means forfeiting many modern features. As
           | I say in another comment, wanting to run Go on OpenBSD the
           | way it does on Linux is a collision of very different
           | philosophies. Hard to say one is wrong, but if they want to
           | coexist one of them would eventually have to compromise, and
           | it was never going to be OpenBSD in the end.
        
       | tptacek wrote:
       | It was very relevant to me a couple days ago, so I think it's
       | _not_ the case that Go programs that look up names link by
       | default dynamically to libc, just for whatever that 's worth to
       | anyone.
       | 
       | (Our company, Fly.io, runs container images for customers on
       | Firecracker microVMs around the world, and I had to build a DNS-
       | dependent service, in Go, that runs directly from our (Rust) init
       | and can't assume a libc exists).
        
         | CameronNemo wrote:
         | Is your Rust init private? What does it do?
        
           | tptacek wrote:
           | Mostly just init stuff. We're transforming Docker containers
           | (mostly) into standalone VMs, so it's doing all the scut work
           | of taking a completely stripped-bare booted kernel and
           | getting it to a state where you can run an arbitrary Linux
           | program on it.
           | 
           | I wouldn't want to take the thread off on a huge tangent,
           | it's just funny that this was just recently super relevant to
           | me (it would have been problematic if DNS-dependent Go
           | programs depended on libc, because right now I can't assume
           | there's a libc binary to be dynamically linked to).
           | 
           | Bringing it back to Go and its (sometimes libc-dependent) DNS
           | libraries: it is very annoying how fiddly it is to get a Go
           | program to use an alternative DNS server.
        
             | vidarh wrote:
             | If you can can control which DNS servers it talks to,
             | implementing the DNS spec directly is a pretty trivial
             | exercise to get full control. (If you can't, the main
             | complexity is dealing with implementation quirks)
        
             | alaties wrote:
             | Had a similar problem a couple years ago where I needed to
             | use alternative DNS libraries to troubleshoot issues in a
             | company's infrastructure.
             | 
             | Golang's rules for what implementation to use are found
             | here: https://golang.org/pkg/net/#hdr-Name_Resolution
             | 
             | A really solid alternative DNS client implementation can be
             | found here: https://github.com/miekg/dns. Real easy to read
             | and vet compared to a few other libraries I ran into when
             | working on this problem.
        
             | JoshTriplett wrote:
             | I'm currently in the process, right now, of writing a
             | minimal init in Rust to do exactly the same thing. Is yours
             | something you'd consider sharing?
             | 
             | The sum total of what I want to do: bring up loopback,
             | bring up the one and only Ethernet interface, set up its IP
             | and basic routing, run another program, and do some basic
             | log reporting.
        
               | tptacek wrote:
               | I hack on our init, but I didn't write it. So it's not my
               | place to share it. And there's some us-specific stuff in
               | it that wouldn't be super helpful to you. But you could
               | definitely ask Jerome on our team; he's a super helpful
               | guy, even if he's super quiet here (he's like the anti-
               | me). One way or the other I'm sure we can help you get
               | where you're going!
        
               | JoshTriplett wrote:
               | I reached out to Jerome. Thanks!
        
               | tptacek wrote:
               | Feel free to follow up with Kurt, me or Michael if you
               | don't get a quick response; Jerome is in Montreal and he
               | may be buried under a mountain of snow and brown gravy.
        
         | X-Istence wrote:
         | The downside to Go shipping it's own implementation of DNS
         | resolution is that on systems that support far more rich set of
         | DNS options (such as split DNS based upon hostnames in macOS)
         | is that it doesn't work if the binary is built with a pure Go
         | implementation.
         | 
         | That leaves users annoyed, because sending all DNS into a VPN
         | tunnel is not always an option.
        
       | saagarjha wrote:
       | Guess they finally saw sense and realized that the when platforms
       | say the syscall ABI is unstable, it really _is_ unstable...
        
         | chungy wrote:
         | OpenBSD has never really cared about backwards compatibility
         | with binaries. It often works, but then suddenly they break.
         | The OpenBSD developers really prefer fixing bugs instead of
         | working around them.
        
           | segfaultbuserr wrote:
           | It's also the advantage you have when you officially maintain
           | a complete operating system, not just a kernel.
        
             | anthk wrote:
             | As long as you can (cross) compile go source easily, I
             | don't see it as a problem.
        
         | jart wrote:
         | There's no good reason for a Unix system to have an unstable
         | system call ABI. I'm sorry but companies like Apple (who broke
         | every Go binary a few years ago) don't get to copy syscall
         | definitions from Bell System V and then declare it their own
         | internal API.
         | 
         | Usually the only time the SYSCALL ABI breaks, it's because
         | kernel authors intentionally chose to do it for no apparent
         | reason. For example, OpenBSD at some point changed how mmap()
         | was defined so that it takes seven arguments:
         | void *sys_mmap(void *addr, size_t len, int prot,
         | int flags, int fd, long pad, off_t pos);
         | 
         | The sixth argument doesn't do anything. It just breaks binary
         | compatibility. It's also noncompliant with the System V ABI
         | specification, which says system calls have six arguments max.
         | 
         | I work on a project called Cosmopolitan Libc which lets you
         | create static binaries that just work on Linux + Mac + Windows
         | + FreeBSD + OpenBSD. It was only possible to do this because
         | Unix systems generally agree on definitions. I worked really
         | hard to support OpenBSD since I believe in the project. I just
         | hope they keep the ABI stable going forward.
         | 
         | For example, I'm really happy that this restriction only
         | applies to dynamic binaries. It seems perfectly reasonable that
         | they'd want to make the assumption that if a program chooses to
         | link OpenBSD's Libc that it intends to use it. That's fine just
         | so long as we continue having the ability to build static
         | binaries with an alternative cross-platform Libc like
         | Cosmopolitan.
         | 
         | Speaking of which, I think I might actually implement some of
         | OpenBSD's ideas in Cosmopolitan. I could probably track down
         | all the functions that need raw SYSCALL and use __section__ so
         | they're all linked to the same part of the binary and then call
         | the msyscall() function to limit it just to that page. That way
         | as a guest libc author I'm upholding the spirit of the intent.
         | When in Rome do as the Romans.
        
           | tedunangst wrote:
           | You should perhaps look into the history of the mmap syscall
           | arguments.
        
             | jart wrote:
             | 4.2BSD System Manual specifies it as mmap(addr, len, prot,
             | share, fd, pos). There are six arguments. My best guess is
             | something went horribly wrong with off_t on big endian
             | systems and that it somehow leaked into x86_64 system call
             | abi. Do you know?
        
               | tedunangst wrote:
               | off_t needs 64bit alignment so registers get copied to
               | the stack argument structure correctly on kernel entry.
               | This is ancient voodoo.
        
           | josefx wrote:
           | > The sixth argument doesn't do anything. It just breaks
           | binary compatibility.
           | 
           | As far as I can tell it might be used to align the memory
           | layout of the following 64 bit arguments to 64 bits. Or at
           | least ensure that there is no auto generated padding that
           | might contain random values.
           | 
           | Edit: The link provided by ainar-g 3 mentions that the way
           | gcc handled padding of the offset field changed between gcc 1
           | and 2.
        
           | ainar-g wrote:
           | > For example, OpenBSD at some point changed how mmap() was
           | defined so that it takes seven arguments:
           | 
           | It seems like that was the case from day one, when a copy of
           | the NetBSD code was imported to later become OpenBSD[1]. And
           | if my reasoning is correct, saying that OpenBSD "changed it
           | at some point" is not quite correct.
           | 
           | https://github.com/openbsd/src/blob/df930be708d50e9715f173ca.
           | ..
        
             | comfydragon wrote:
             | But, that link shows mmap taking 6 arguments?
        
               | ainar-g wrote:
               | The parent comment talked about the syscall, not the
               | library function.
               | return((caddr_t)(long)__syscall((quad_t)SYS_mmap, addr,
               | len, prot,            flags, fd, 0, offset));
               | 
               | The first argument is the syscall number, the rest are
               | the seven arguments in question.
        
           | saagarjha wrote:
           | You've brought this up before, but I still don't have insight
           | into why you say that copying system call definitions from
           | Bell (if there was ever such a thing, POSIX standardized
           | above that level for a reason...). macOS, Windows, the BSDs,
           | all declare their stable ABI to be above the syscall layer.
           | Why must it be defined lower?
        
             | jart wrote:
             | No they don't. Only Windows forbids developers from linking
             | static binaries. That's because they change the SYSCALL
             | ordinals every few months. Which means that in order to
             | load a program on Windows you have to give up control of
             | the virtual memory address space to the operating system so
             | that it can load DLLs at arbitrary addresses.
             | 
             | Unix systems have never had that restriction. Because if
             | you use the official libc and pass -static to gcc then that
             | means the "stable api" creates a binary that depends on the
             | kernel abi. If the kernel authors break the abi then it
             | means you need to recompile all your software in order to
             | upgrade.
        
               | masklinn wrote:
               | > No they don't.
               | 
               | Yes they do.
               | 
               | > Only Windows forbids developers from linking static
               | binaries. That's because they change the SYSCALL ordinals
               | every few months.
               | 
               | That Windows somewhat actively precludes raw syscalls
               | doesn't mean they are supported elsewhere. It's always
               | been bsd (and especially macOS) policy that syscalls are
               | an implementation detail.
               | 
               | > Because if you use the official libc and pass -static
               | to gcc then that means the "stable api" creates a binary
               | that depends on the kernel abi.
               | 
               | Try doing that on macOS, you'll find out that there is no
               | static version of libSystem, or crt0.
        
               | pjmlp wrote:
               | Plenty of OS allow for static linking, including Windows.
        
             | leeter wrote:
             | Windows does it because the vast majority of functionality
             | is not implemented in the kernel per se, but rather in user
             | space by preference for stability. Even things like display
             | drivers are often primarily implemented in userspace.
             | 
             | The main advantage of not having a stable kernel ABI is
             | that you have the freedom to change it for
             | security/performance reasons. This means the OS can change
             | how they implement things over time without breaking things
             | as easily. MS is notorious for this and for using this fact
             | to allow them to 'emulate' older ways of doing things even
             | when the real implementation has long since moved on.
             | 
             | The main advantage of having a stable kernel ABI is that
             | the userspace can be whatever the developer wants it to be
             | realistically. If they want it to be just a single GO
             | program and literally nothing else they can do that
             | (routers are a good example of devices that do this).
        
           | quotemstr wrote:
           | There's no reason for an operating system to have a stable
           | system call ABI. A stable ABI means that the kernel support
           | boundary grows forever and that userspace shims are
           | impossible in the general case. And what's the point? Calling
           | through libc or ntdll or whatever is appropriate on a given
           | system is no great burden. Go's libc avoidance is just
           | hubris.
        
             | jart wrote:
             | Even if you use the system libc, if you pass the -static
             | flag to gcc then you end up with a binary that depends on
             | the syscall abi. If the kernel interface breaks, then all
             | your programs need to be recompiled from scratch in order
             | for them to work again. Are you opposed to static linking?
        
               | quotemstr wrote:
               | Yes, I am opposed to fully static linking. What's the
               | point of static linking? Windows has no static linking
               | (all system calls go through ntdll) and it has a
               | compatibility story better than any Unix. Static linking
               | to libc is unnecessary for long term ABI support.
               | 
               | Dynamically link against libc and statically link the
               | rest for all I care, but there's no reason not to talk to
               | libc.
               | 
               | Also: the vDSO is also a form of dynamic linking. Are you
               | opposed to the vDSO?
        
               | Blikkentrekker wrote:
               | What, do you believe, are the advantages of static
               | linking against a _libc_?
        
               | a1369209993 wrote:
               | The same as static linking any other library: it stops
               | the library semantics from being changed out from under
               | you by a 'update'.
        
             | wahern wrote:
             | > Go's libc avoidance is just hubris.
             | 
             | Maybe my impression was wrong, but it was my understanding
             | that Go originally preferred direct syscalls because of
             | stack management headaches. You can't know how much stack
             | space a libc syscall wrapper requires, which even for
             | seemingly simple syscalls can be quite complex--e.g. glibc
             | has to emulate POSIX thread semantics. OTOH, treating such
             | libc wrappers like regular C FFI functions would obliterate
             | the design and implementation assumptions around goroutine
             | stack management. Considering that Linux was originally the
             | first (and, let's be honest, only _real_ target), it made
             | perfect sense to rely on Linux syscall ABI promises.
             | 
             | Fast-forward a few years: 1) Go has a more mature binary
             | format and dynamic linking capabilities, shrinking the gap
             | between Go's internal ABI and the native libc ABI. 2)
             | Goroutine stacks switched from split-stacks to movable
             | stacks, and the minimum stack size became larger. 3) Demand
             | and motivation for supporting libc wrappers (i.e. for
             | Windows) grows. Result: Go surmounts one of its original
             | simplifying design compromises. Though, I would assume that
             | libc wrapper support still incurs ongoing maintenance costs
             | on each platform; namely, managing the minimum stack
             | requirement for each particular call, which could change
             | overtime, while it's important not to be too pessimistic so
             | that a syscall doesn't force an unnecessary stack resize.
        
             | jeremyjh wrote:
             | For Linux it makes sense because there is no project
             | running the entire operating system. The kernel's system
             | call interface is stable because there _is_ no other
             | "Linux" system interface. For an OS that maintains a libc
             | they can make a different choice.
        
               | quotemstr wrote:
               | So we should endure technical mediocrity forever because
               | we couldn't get our act together socially?
               | 
               | There is a way out. I've previously proposed on LKML that
               | the Linux kernel team provide an official userspace
               | system call library that sits below libc and that all
               | libc implementations would share. We'd forbid new system
               | calls being called except through this library.
               | Optionally, we'd enforce this constraint on all system
               | calls.
               | 
               | This is how Fuchsia works, by the way: all Fuchsia system
               | calls must go through one giant vDSO.
        
               | jeremyjh wrote:
               | I agree this is a better architecture but this creates
               | more work for the kernel team but doesn't free them from
               | ABI stability constraints anywhere in the near-term. It
               | would take 10-15 years to pay off I think.
        
           | ChrisSD wrote:
           | > Apple (who broke every Go binary a few years ago)
           | 
           | That's a really weird way to phrase it. Apple says that
           | interface is unstable. Go uses it anyway. The unstable
           | interface turns out to be unstable.
           | 
           | It wasn't Apple that broke Go binaries. It was Go.
        
             | mhh__ wrote:
             | Just because they don't maintain it stably doesn't mean
             | they shouldn't do so.
        
               | mhh__ wrote:
               | I must say I'm amused that my own personal opinion on
               | software is this down worthy
        
               | ChrisSD wrote:
               | Wishing Apple did something differently does not make it
               | so. Either you accept the stable interface for what it is
               | or you accept your binaries may break. And you've only
               | yourself to "blame" in the latter case.
        
               | mhh__ wrote:
               | I don't have to blame anyone because I don't own any
               | Apple products, what I'm getting at is that Apple's
               | "Minimal Documentation, force through our blessed
               | channels" way of going about things is kind of remarkable
               | given that microsoft have been absolutely slammed for
               | similar actions in the past.
        
               | my123 wrote:
               | BSDs traditionally don't have a stable syscall ABI.
               | 
               | Windows never had a stable syscall ABI either.
               | 
               | Almost only Linux does it... because the kernel and libc
               | are maintained as separate projects in the Linux world.
        
               | seg_lol wrote:
               | Systemd should also be the libc provider.
        
               | mhh__ wrote:
               | I don't really do OS development so I was mostly using
               | this as a microcosm of the wider practice.
               | 
               | Practically as long as it's trivially callable from C I'm
               | not bothered.
        
               | trasz wrote:
               | FreeBSD does have a stable syscall ABI. But it's not
               | _the_ stable ABI people should be using; the advertised
               | stable ABI is the libc.
        
               | littlestymaar wrote:
               | > Almost only Linux does it... because the kernel and
               | libc are maintained as separate projects in the Linux
               | world.
               | 
               | That's because Linux is just that: a kernel. And people
               | are free to build their userspace on top of it.
        
               | MarkSweep wrote:
               | There has to be a line draw somewhere to demarcate where
               | the interface of a system is defined. Just as Linux does
               | not attempt to ensure that using /dev/mem to manipulate
               | kernel data structures works the same between versions,
               | many operating systems don't make promises about the
               | syscall interface.
        
       | bithavoc wrote:
       | Same for macOS and iOS since Go 1.11, it all goes through
       | libSystem now.
        
       | bitcharmer wrote:
       | I always thought GO makes syscalls via libc. If it doesn't am I
       | correct assuming no LD_PRELOAD magic will work with GO? As in no
       | custom memory allocators, no kernel bypass for TCP stack, etc.
        
         | rfoo wrote:
         | Yes, you are correct. That also means no proxychains.
        
       | lifeplusplus wrote:
       | does this have performance implication
        
         | IcePic wrote:
         | It will cost a cycle or three extra, but given the great cost
         | to make syscalls in the first place (especially if you have to
         | do flushes and mitigations for the 'recent' security bugs on
         | x86) it probably won't matter much. 1000 vs 1003 or 10000 vs
         | 10003 would be hard to measure in a real world program.
         | 
         | Also, many things libc does (which go programs need to do
         | themselves) is to wrap a lot of the calls to things like
         | malloc/calloc/realloc to use internal buckets and as seldom as
         | possible call out to the kernel to get one or ten pages of ram
         | in one call, then hand out suballocations from them for each
         | "obj = malloc(8);" so that you minimize the amount of syscalls
         | made, regardless of if you do them directly or via libc.
         | 
         | Since syscalls always were expensive (you need to save
         | registers, check userid permissions, flip to kernel mode, do
         | the work asked for with or without SMP locking protections,
         | give permissions to uid for the resource returned, perhaps
         | check if its time to deliver signals or switch to another
         | process and if not, flip out of kernel mode, restore registers
         | and return to the process again) a lot of the calls done by a C
         | program is kept in libc if possible.
         | 
         | For example, gettimeofday() springs to mind, where a lot of
         | trickery is done to give all programs a readonly page with the
         | current time mapped into your program space so the call doesn't
         | have to go via the kernel but instead becomes a memory read,
         | since programs tend to call this thousands of times.
        
           | saagarjha wrote:
           | Of course, on Linux the kernel API extends to the vDSO, so Go
           | can certainly rely on its existence. But reimplementing a
           | libc for no good reason is typically a fool's errand.
        
             | masklinn wrote:
             | > Of course, on Linux the kernel API extends to the vDSO
             | 
             | The kernel _ABI_ , however, does not. vDSO are shared
             | objects exposing a C ABI, and oddball compilation settings
             | have broken Go's vDSO calls in the past:
             | https://marcan.st/2017/12/debugging-an-evil-go-runtime-bug/
        
         | Hydraulix989 wrote:
         | OpenBSD often trades off performance for security.
        
       | Hydraulix989 wrote:
       | I agree with OpenBSD's philosophy of introducing breaking ABI
       | changes between versions, if security can be improved.
        
       | floatboth wrote:
       | Now do this on FreeBSD too.
        
         | EdSchouten wrote:
         | Why? As far as I know, FreeBSD's system call ABI is supposed to
         | be (relatively) stable. It's a requirement anyway if you want
         | to run jails of a different version.
        
           | floatboth wrote:
           | Personally: because it makes porting to new CPU architectures
           | _hell_. (The Go assembler is the worst!) I 've abandoned the
           | FreeBSD/arm64 port and two other people had to pick it up to
           | finish it.
           | 
           | But also: while it's stable, it's _not public_. IIRC, Go
           | developers were told about this, but decided to ignore.
           | 
           | And of course: stop breaking LD_PRELOAD hooks :P
        
       | tedunangst wrote:
       | The driving force for this was the syscall origin check, but
       | syscall ABIs change too. There's a speculative execution bug in
       | ARM CPUs which requires barriers. This also requires patching.
       | 
       | https://marc.info/?l=openbsd-ports-cvs&m=158083696719245&w=2
        
         | prepperdev wrote:
         | vDSO is a good way to mitigate issues like these. It's also a
         | better stable ABI than libc, and easier to maintain than pure
         | kernel ABI (like Linux does), because nobody forbids the kernel
         | to inject different vDSOs to different binaries, if a need
         | arises.
         | 
         | vDSO is also a language independent construct, so there's no
         | special treatment for any favorite language, be it C (OpenBSD),
         | C++ (Windows) or Oberon OS (Oberon).
         | 
         | vDSO: https://en.wikipedia.org/wiki/VDSO
        
           | remexre wrote:
           | vDSO functions are still allowed to use arbitrary amounts of
           | stack though, which means Go might still have problems with
           | them without having to allocate much larger stacks than it
           | normally would.
        
             | prepperdev wrote:
             | That would be up to a specific OS to decide which
             | guarantees on stack size limit would it like to provide.
             | But I agree that it's a valid concern, and not all possible
             | vDSO implementations are reasonable.
        
             | masklinn wrote:
             | In fact that is an issue Go hit in the past:
             | https://marcan.st/2017/12/debugging-an-evil-go-runtime-bug/
        
         | Flow wrote:
         | > All except some of the most recent arm64 processors have a
         | speculative execution flaw that occurs across a syscall
         | boundary, which cannot be mitigated in the kernel.
         | 
         | Big Ouch. I wonder how the Linux developers will approach this
         | bug since they don't enforce syscalls to be done from glibc.
        
           | the8472 wrote:
           | > will approach this bug
           | 
           | Note that the mail is from 2020. So it probably already is
           | fixed. It might be this one
           | 
           | https://patchwork.kernel.org/project/linux-arm-
           | kernel/patch/...
        
           | spijdar wrote:
           | It's not enforced, but I'd dare say by and large almost
           | everything will just use glibc. I'd assume if you're playing
           | with fire enough to be calling syscalls yourself, you can
           | mitigate the bugs yourself.
           | 
           | I don't think it'll be a bit problem, anyway. In my
           | experience not very much calls syscalls directly. Go is a big
           | exception, though...
        
             | [deleted]
        
             | arghwhat wrote:
             | > It's not enforced, but I'd dare say by and large almost
             | everything will just use glibc.
             | 
             | What about musl? uClibc?
             | 
             | Linux is well known for the fact that it guarantees its
             | syscall interface as primary contract to userspace.
        
               | spijdar wrote:
               | Okay, so I'm referring mainly to the typical "GNU/Linux"
               | desktop/server OS vs embedded or "container Linux" which
               | is where the majority of alternative libc use will
               | happen.
               | 
               | In either case though there is _a_ libc that most
               | software will use, and the mitigations can be applied
               | there. Even though direct use of syscalls is legal on
               | Linux, the fact that it 's stable is primarily relevant
               | and interesting to said libc developers.
               | 
               | The fact that syscalls aren't guaranteed on other systems
               | is usually of little consequence since the libc is
               | developed in tandem with the kernels of those systems.
               | Linux's situation as a fully decoupled kernel means it
               | does things differently in that sense. The developers are
               | fully separated, so there needs to be a strong "contract"
               | that syscalls will be stable.
               | 
               | Doesn't mean (IMO) it's a good idea for end-users e.g.
               | software developers to use syscalls except in exceptional
               | circumstances. Which is usually the case!
        
               | arghwhat wrote:
               | On Linux, libc developers are in the exact same group as
               | regular developers.
               | 
               | libc is not intended to be the official entry point in
               | any way or form, and kernel vulnerabilities and
               | workarounds are not meant to be handled by a libc
               | implementation.
               | 
               | That other OSs make libc their official interface is
               | primarily because it's the simplest thing to do when
               | kernel, libc and the rest of userspace is co-developed,
               | as it allows for breaking kernel changes and other fun
               | things that are not allowed under Linux ABI guarantees
               | anyways.
               | 
               | It is not because it is the most secure choice, or that
               | dealing with syscalls is hard (syscalls are easy and safe
               | to work with). It's just that stable ABIs are a lot of
               | work to develop, and this structure is just the simplest
               | for smaller OS communities to develop.
        
               | rwmj wrote:
               | _> libc is not intended to be the official entry point in
               | any way or form, and kernel vulnerabilities and
               | workarounds are not meant to be handled by a libc
               | implementation._
               | 
               | Depending on how you define "workarounds", glibc is full
               | of those. For example stat(2) is very much not the same
               | syscall now as it was back in the 1990s. In some cases
               | glibc will do a runtime test to see which syscall
               | variants are supported by the kernel and implement
               | workarounds (I even saw a case where this caused a bug in
               | some programs).
        
               | arghwhat wrote:
               | This is indicative of glibc's bad design more than
               | anything else.
               | 
               | stat(2) is not a single syscall. The changes are exposed
               | as new, isolated syscalls (sys_stat, sys_newstat,
               | sys_stat64), with glibc switching internally between them
               | as it sees fit, surprising developers in the process.
               | 
               | This makes stat a great example of the syscall being
               | easier to work with, more stable and more reliable than
               | the glibc wrapper.
        
               | quotemstr wrote:
               | Libc is absolutely the official entry point: it's libc
               | that implements POSIX interfaces, not the kernel. If
               | POSIX isn't official, what is?
               | 
               | Making libc the stable support boundary has all sorts of
               | advantages to an operating system and basically zero
               | downside. Only vanity argues for doing it the Linux way.
        
               | tsimionescu wrote:
               | There are several popular libc variants on Linux. Which
               | of them is official?
               | 
               | Also, Linux is not POSIX compliant and doesn't
               | necessarily care about being. There are several important
               | IO options on Linux that have nothing to do with POSIX.
        
               | Flow wrote:
               | Not sure what you are advocating here.
               | 
               | If someone successfully injects code that perform a
               | direct syscall they can successfully use this info leak
               | despite a safe and patched (g)libc.
        
               | TheDong wrote:
               | > If someone successfully injects code that perform a
               | direct syscall they can successfully use this info leak
               | despite a safe and patched (g)libc.
               | 
               | As Raymond Chen wrote, that's the other side of the
               | airtight hatch (https://devblogs.microsoft.com/oldnewthin
               | g/20060508-22/?p=31...).
               | 
               | If someone is capable of directly running their own code
               | that ignores libc (or other existing mitigations, such as
               | may exist in javascript runtimes / go compiler) then
               | cool, they can use spectre to perform a timing attack
               | against their own code that they're running. Or they
               | could just read their own memory.
               | 
               | Spectre's main risk was for reading other program's
               | memory or for doing so remotely with javascript. If you
               | can already make the process you're attacking run
               | arbitrary syscalls instead of use glibc, then you've
               | already won and no amount of protection will help.
        
               | lxgr wrote:
               | > If you can already make the process you're attacking
               | run arbitrary syscalls instead of use glibc, then you've
               | already won and no amount of protection will help
               | 
               | You're basically arguing that in a post-spectre world,
               | native processes can fundamentally never be a security
               | boundary again, right?
               | 
               | I'm wondering if this is necessarily true. For the
               | concrete example at hand, Linux could offer some opt-in
               | mechanism, e.g. an argument to exec(), that restricts
               | syscalls to glibc only. A sandboxing mechanism could then
               | require all executed processes to go through glibc and
               | instantiate them only using that option.
        
               | [deleted]
        
             | bregma wrote:
             | Android is not GNU/Linux. There are an awful lot of Android
             | computers out there. Embedded Linux is almost never
             | GNU/Linux. There's an awful lot of embedded Linux out
             | there.
             | 
             | Desktop Linux is hardly the majority of Linux
             | installations. How many containers in the cloud are using
             | something like Alpine Linux?
             | 
             | I'd dare say by and large glibc is in the minority.
        
         | lxgr wrote:
         | For anyone wondering what the syscall origin check is about or
         | how it's implemented: https://lwn.net/Articles/806776/
         | 
         | Seems like this is a good idea for multiple reasons. Another
         | benefit is that it would seem to make something like "Wine in
         | reverse" possible/easier to implement.
        
           | severino wrote:
           | Can you please elaborate on this "Wine in reverse" concept?
        
             | lxgr wrote:
             | I'm talking about a userspace-only Linux compatibility
             | layer for Windows.
             | 
             | As far as I understand, Wine (a userspace-only Win32
             | emulator) is possible because Windows applications always
             | go through the standard library for system calls, which
             | therefore can be hooked without kernel support.
             | 
             | The same is not generally true for Linux binaries - these
             | usually do go through glibc, but direct syscalls are
             | possible (through raising the appropriate interrupt or
             | instruction).
             | 
             | I don't think there's an easy way to trap these without
             | kernel support (which is what WSL 1 has been doing).
        
               | masklinn wrote:
               | > The same is not generally true for Linux binaries -
               | these usually do go through glibc, but direct syscalls
               | are possible (through raising the appropriate interrupt
               | or instruction).
               | 
               | OTOH Linux having a well defined set of syscalls you can
               | very convincingly fake being a Linux kernel. That's what
               | wsl1 did. That's also what SmartOS does, which allows it
               | to mix native and Linux (LX) zones.
        
       | ashishmax31 wrote:
       | So does that mean no statically linked binaries since glibc can't
       | be statically linked?
        
         | dilyevsky wrote:
         | Not glibc in mac os/openbsd case but yeah can't have fully
         | static go binaries on those.
        
         | pjmlp wrote:
         | glibc is Linux only.
        
           | sanxiyn wrote:
           | glibc also supports Hurd.
        
             | pjmlp wrote:
             | Yep, it is already at 1.0 finally?
        
             | Blikkentrekker wrote:
             | Did Debian not also port most of _glibc_ to support
             | _FreeBSD_ 's kernel?
        
               | sanxiyn wrote:
               | Yes they did. See http://www.nongnu.org/glibc-bsd/.
        
         | [deleted]
        
         | tephra wrote:
         | OpenBSD does not use glibc.
        
         | makapuf wrote:
         | But if even Go can't use static binaries, does that mean that
         | static binaries are not supported by those systems ?
        
           | fctorial wrote:
           | "static binaries" are dynamically linked to the kernel. Only
           | the OS ISOs are fully statically linked.
        
           | ashishmax31 wrote:
           | I would think so. Can anyone eli5 why statically linking
           | binaries is a big deal? Even light weight container oriented
           | linux distributions like alpine ship with musl. In which
           | scenario would it find it use cases?
        
             | saagarjha wrote:
             | Some have the viewpoint that static linking allows for what
             | is essentially dependency pinning of binaries.
        
             | arp242 wrote:
             | One advantage is that you need to worry less about
             | versions; if I build a dynamic library that uses a feature
             | from GNU libc 2.30, and someone tries to run it with GNU
             | libc 2.23 then they will get an error.
             | 
             | These versions are not chosen at random: I ran in to this
             | issue with people trying to run my binary on Ubuntu 16.04
             | (LTS release), which was solved by linking it statically.
             | 
             | Also, people may use musl libc, and while it has _some_
             | compatibility with GNU libc this is far from complete.
             | 
             | So in short, linking it statically means it will work for
             | the largest amount of people with a minimal of fuss for
             | both the person building the binaries, and the people
             | running them.
             | 
             | As people have mentioned, these issues are less present on
             | non-Linux systems.
        
               | account42 wrote:
               | > which was solved by linking it statically
               | 
               | You could also have linked against glibc 2.23 or 2.22 or
               | older instead of bloating your binary.
        
               | arp242 wrote:
               | But that's a lot more effort, and who knows if someone is
               | still using a CentOS or whatnot with an even older
               | version. And it still won't work for other libc
               | implementations.
               | 
               | Adding an extra megabyte or so is a reasonable trade-off,
               | with no real other downsides. It's not that large -
               | smaller than many websites.
        
             | q3k wrote:
             | Massively simplifies the build process, especially for
             | cross-compilation. You can just build the binary, copy it
             | over and run it, without having to ever touch containers,
             | or having to worry about sysroots.
             | 
             | You can go ahead and do the following on your Mac:
             | GOARCH=riscv GOOS=linux go build
             | 
             | to build a binary that just runs on Linux on RISC-V.
        
       | DominoTree wrote:
       | At one point, building Go and running its test suite on OpenBSD
       | would replace /dev/null with a standard flat file. After some
       | time, the disk would fill up because everything piped to
       | /dev/null was now being stored on disk. Not to mention the
       | additional I/O happening.
       | 
       | Go already uses libc by default on many platforms. But there are
       | issues - sometimes the libc behaves differently than Go's
       | documented APIs, but this is primarily a documentation issue.
       | Contrariwise, sometimes, Go's native APIs don't behave on systems
       | due to platform-specific implementation bugs.
       | 
       | I think this is a good move overall.
        
       | sivizius wrote:
       | https://lwn.net/Articles/806776/ says, system-call-origin
       | verification is for mitigation of ROP. ROP is (always?) a result
       | of stack buffer overflows, which are a result of bad programming
       | in asm/C/C++, but usually not a thing at all with modern
       | languages like Go, Rust, Swift, Haskell, etc. while
       | C/C++-programs usually use the libc already. This enforces
       | programs written in such languages to use another layer, written
       | in C, that is not really necessary but might introduce new
       | vulnerabilities. Do I miss something?
        
         | roblabla wrote:
         | The system call interface is usually written in an unsafe
         | language such as asm anyways. Going through the libc is very
         | unlikely to actually introduce vulnerabilities, especially if
         | going through the lowest level function that directly wrap the
         | syscall.
         | 
         | Not using the libc was always a risky proposition on BSDs
         | anyways. They don't have a stable kernel ABI the same way the
         | Linux kernel does. From OpenBSD's perspective, the stable ABI
         | is the libc, and anything using the kernel ABI directly is
         | liable for breakage with each update.
        
           | loosescrews wrote:
           | > Going through the libc is very unlikely to actually
           | introduce vulnerabilities
           | 
           | Citation needed. glibc has a long history of security bugs.
           | 
           | https://www.cvedetails.com/vulnerability-
           | list/vendor_id-72/p...
        
             | warmwaffles wrote:
             | Other libc implementations exist that do not have many.
             | 
             | Musl[1] for one.
             | 
             | [1] https://www.cvedetails.com/product/39652/Musl-libc-
             | Musl.html...
        
             | lxgr wrote:
             | To quote the grandparent post:
             | 
             | > especially if going through the lowest level function
             | that directly wrap the syscall.
             | 
             | I think the argument is that security bugs are unlikely to
             | occur in those low-level wrappers.
        
             | roblabla wrote:
             | Geez man, way to take things out of context. Let me fix
             | your quote:
             | 
             | > Going through the libc is very unlikely to actually
             | introduce vulnerabilities, especially if going through the
             | lowest level function that directly wrap the syscall.
             | 
             | Sure, glibc has a bunch of bugs. But the lowest level of
             | functions, that just wrap the syscalls, are very unlikely
             | to have bugs. Here's the `read` implementation, for
             | instance:
             | 
             | https://github.com/bminor/glibc/blob/21c3f4b5368686ade28d90
             | d...
             | 
             | All it does is delegate to the low-level syscall, with some
             | extra handling around to handle async calls (which can be
             | removed when compiling glibc yourself, but you're probably
             | not doing this).
             | 
             | Here's clone:
             | 
             | https://github.com/bminor/glibc/blob/21c3f4b5368686ade28d90
             | d...
             | 
             | This one's written in asm, and you can't really simplify it
             | all that much more.
             | 
             | All the functions that wrap the low-level syscalls are very
             | hard to get wrong, really. Where the glibc bugs come from
             | are the high-level functions, like pthread. But those can
             | trivially be bypassed if necessary.
        
               | sivizius wrote:
               | And where is the advantage then to have a `call
               | syscall_wrapper` over `mov rax, syscall_number; syscall`
               | if the ROP was able to return just before the call?
        
               | roblabla wrote:
               | I wasn't making any claim on whether this mitigation is
               | actually useful, I was simply stating it's unlikely to
               | introduce more vulnerabilities than the status quo.
               | 
               | But since you're asking, I can think of two upsides:
               | 
               | 1. It restricts the kernel attack surface available to
               | only those syscalls that are exposed through the wrapper.
               | (This is obviously dubious if the libc provides a generic
               | syscall function, I don't know if OpenBSD libc has one).
               | 
               | 2. It forces the ROP to either find the libc ASLR base,
               | or to find a gadget that calls into the target libc
               | function. This makes ROPs a lot harder to write.
               | 
               | As with any mitigations, they're mostly meant to make the
               | attacker's life miserable. They're not full protections,
               | and can often be bypassed. The point is to increase the
               | cost of the attack.
        
       | tus88 wrote:
       | Err....what about the stacks man?
        
       | kisamoto wrote:
       | Can someone please ELI5 the significance of this for me?
       | 
       | Thanks
        
         | Someone wrote:
         | In Linux, the system call interface is stable (on any given
         | architecture, but not across them. See
         | https://stackoverflow.com/questions/10281567/why-are-the-
         | sys...): the way you call any kernel function is guaranteed to
         | stay the same forever.
         | 
         | That means that programs that directly make system calls will
         | keep working on newer OSes.
         | 
         | On many (?most?
         | https://unix.stackexchange.com/questions/473137/do-other-
         | uni...) other operating systems, that's not the case; the OS
         | ships with a library that provides a stable interface, and
         | system call numbers, arguments, or calling conventions can
         | change (in theory, the interface could even change across
         | reboots or process runs). On openBSD, that library is Libc
         | (and, unfortunately, is a lot larger than just the OS
         | interface. IMO, in an ideal world, it should be split in two
         | parts, the OS interface and a C library)
         | 
         | Go wants to produce statically linked executables. It can't do
         | both that and link with LibC. It now changed to dynamically
         | link with LibC, guaranteeing that what you compile today will
         | run as well on next year's OpenBSD as it does on today's one.
         | 
         | On top of that, openBSD has a security feature where it
         | verifies that system calls are made via LibC. That feature
         | wasn't implemented as thoroughly as possible because go made
         | direct system calls. This change allows OpenBSD to tighten that
         | feauture.
        
           | ben_bai wrote:
           | I'm sorry OpenBSD also cranks libc versions whenever needed.
           | And it depends on what changed, if it'll run on an old
           | kernel. Since it links against a fixed version, keeping it
           | running involves recompiling or some trickery...
           | 
           | So basically executable are mostly only ever good for 1
           | stable release. So don't throw away your source code, you
           | gonna need it in 6 month.
        
           | justaj wrote:
           | So does OpenBSD's libc library differ a lot from say, Linux's
           | libc library? Meaning: Suppose you want to write an app that
           | can talk to both Linux's libc and OpenBSD's libc using some
           | sort of "common language" that is compatible with both, would
           | that be possible?
        
             | ben_bai wrote:
             | that common language is called POSIX
             | 
             | Edit: All libc basically adhere to it. Except for some
             | extentions that are not in POSIX but might be in
             | glibc/musl/FreeBSDs libc or some extentions that are
             | OpenBSD specific.
        
               | Someone wrote:
               | POSIX and C standards (IIRC, the two conflict in some
               | minor places). GNU libc also tries to adhere to Berkeley
               | UNIX (see https://www.gnu.org/prep/standards/html_node/Co
               | mpatibility.h.... Trigger warning: contains a remark on
               | text editors)
        
           | kisamoto wrote:
           | Thank you! That was really informative and I will check out
           | those links to learn more.
           | 
           | I've been developing with Go for a few years now but never
           | strayed too low level so was unsure how this change in Go
           | 1.16 would affect me. I doubt my code will run on OpenBSD in
           | the near future however I'm happy to know that if it does it
           | will be supported in the future.
        
             | ben_bai wrote:
             | go apps are and will be easy to run on OpenBSD. It's one of
             | the languages which are really as easy as "go get
             | githbu.com/..."
             | 
             | Edit: It's just the go mantra of compile once run forever
             | doesn't really work on OpenBSD. You do have to recompile
             | under certain conditions.
        
       ___________________________________________________________________
       (page generated 2021-02-02 23:02 UTC)