[HN Gopher] "Static Linking Considered Harmful" Considered Harmful
       ___________________________________________________________________
        
       "Static Linking Considered Harmful" Considered Harmful
        
       Author : yagizdegirmenci
       Score  : 134 points
       Date   : 2021-10-03 11:15 UTC (11 hours ago)
        
 (HTM) web link (gavinhoward.com)
 (TXT) w3m dump (gavinhoward.com)
        
       | davidwf wrote:
       | This is very situational, but I have recently been part of a
       | project that does a lot of C server-side development and we have
       | found that static linking our non-glibc dependencies has really
       | improved our developer experience. Using ceedling's dependency
       | plugin[1] and producing a single "statically" linked library has
       | made our C development much closer to using a language with a
       | more modern package manager. Don't get me wrong, if I was trying
       | to distribute binaries to machines I didn't control I'd
       | definitely be willing to invest in the Linux packaging "fun", but
       | for a server-side application it's been a good choice for our
       | team overall.
       | 
       | [1]
       | https://github.com/ThrowTheSwitch/Ceedling/tree/master/plugi...
        
         | PaulDavisThe1st wrote:
         | Yes and no. In Ardour, which has a substantial dependency tree
         | (80+ libraries), if we statically link, the edit/compile/debug
         | cycle becomes incredibly bogged down by static linkage. It
         | takes a long time (even with lld) to complete the link step. If
         | we use shared libraries/dynamic linkage, the edit/compile/debug
         | cycle is nice and fast, but application startup is a bit slower
         | (not by as much as static linkage is slower, however).
         | 
         | For users, it would be better if the application was statically
         | linked, at least in terms of startup cost. But because
         | developers do the edit/compile/debug cycle much, much more
         | often than users start the application, we opt for dynamic
         | linkage.
        
           | xyzzyz wrote:
           | You can dynamically link during development, and then ship
           | statically linked binary (with LTO etc). That's what Chrome
           | does, for example.
        
             | PaulDavisThe1st wrote:
             | It's not that simple for us (we have tried this).
        
       | sharikous wrote:
       | Correct me if I am wrong but OpenBSD introduced the constraint
       | that syscalls must come from a specific section of memory, that
       | assigned to their libc.
       | 
       | In a case like that static linking is not only unstable, as in
       | Windows or macOS, but also impossible.
       | 
       | And since that has security benefits we will probably see it
       | being migrated to other platforms in the near future.
        
       | ris wrote:
       | I eagerly await some concrete implementations of these new ideas.
       | 
       | In the meantime I'm waiting for the first _big_ vulnerability in
       | a widely used golang or rust library to see if downstream
       | projects pinning it _also_ release CVEs as they should (which
       | would probably DDoS the CVE system from the resulting avalanche)
       | _or_ they quietly bump it and move on (in which case their users
       | won 't get the nudge to upgrade). This is where someone says
       | "well you should just always be running the latest version of
       | everything", which is of course infeasible on a real life system
       | with thousands of installed packages.
       | 
       | And it's not that I don't completely sympathize or understand the
       | advantages of static linking from a developer's point of view -
       | you don't have to sell it to me there.
        
         | ghoward wrote:
         | Author here.
         | 
         | > I eagerly await some concrete implementations of these new
         | ideas.
         | 
         | I'm working on them now! They are not public because of a
         | licensing issue where I need to consult a lawyer first (thanks,
         | GitHub Copilot), but they will be public as soon as that's
         | sorted out.
        
           | Zababa wrote:
           | > They are not public because of a licensing issue where I
           | need to consult a lawyer first (thanks, GitHub Copilot), but
           | they will be public as soon as that's sorted out.
           | 
           | Do you mean that you built them using Copilot, or that you
           | don't want Copilot to use them? Or something else entirely?
        
             | ghoward wrote:
             | I don't want Copilot to use them. Sorry; should have said
             | that.
        
               | Zababa wrote:
               | No worries, it's nice to know that someone is working on
               | that with the help of actual lawyers!
        
         | DaiPlusPlus wrote:
         | The design of the Go build and packages system kinda renders
         | that point moot because the expectation is that provided you
         | follow Go's guidelines then a Go application should _always_ be
         | easily recompilable, and all external dependencies should
         | always be distributed in source-form, not as binaries (e.g.
         | https://github.com/golang/go/issues/2775 ).
        
           | Macha wrote:
           | Right, but if every application did distribute as source and
           | could just be recompiled, there would be fewer complaints
           | about the difficulties distributing Linux binaries
        
           | Zababa wrote:
           | > provided you follow Go's guidelines then a Go application
           | should always be easily recompilable, and all external
           | dependencies should always be distributed in source-form, not
           | as binaries
           | 
           | So it's a human solution, and not a technical one, which
           | means that it can and will fail.
        
             | DaiPlusPlus wrote:
             | That's like saying condoms are ineffective because you
             | included people too lazy to put them on in the first place
             | in your calculations.
             | 
             | Provided you (and your org) use Go as Google intends then
             | that isn't an issue.
        
               | Zababa wrote:
               | > That's like saying condoms are ineffective because you
               | included people too lazy to put them on in the first
               | place in your calculations.
               | 
               | I think it's closer to say that the existance of condoms
               | won't make AIDS and unwanted pregnancies disappear, even
               | if they were 100% effective. Which, if you're trying to
               | eliminate completly AIDS, is a fair claim to make.
               | 
               | My point is that you shouldn't assume that everyone is
               | going to follow the Go guidelines, because some people
               | won't, and you don't want your security to rely on that.
        
         | Wowfunhappy wrote:
         | > This is where someone says "well you should just always be
         | running the latest version of everything", which is of course
         | infeasible on a real life system with thousands of installed
         | packages.
         | 
         | Is updating a library with thousands of dependents, without
         | individually testing those dependents, any more feasible?
        
           | CyberShadow wrote:
           | You would have to do that anyway. With static linking, you
           | also have to rebuild those dependents and make sure their
           | users update them as well.
        
             | Wowfunhappy wrote:
             | I would rather the developers rebuild the dependents, test
             | them, make any necessary fixes, and send me the updated
             | (presumably static) binaries. As opposed to me, as an
             | individual user with no familiarity with the code base,
             | just switching out the pieces and preying nothing goes
             | wrong.
        
               | ris wrote:
               | And if the upstream developers are unresponsive, or only
               | provide the bump with the latest bleeding-edge version
               | which you can't upgrade to yet?
        
               | Wowfunhappy wrote:
               | On a system where security is critical, I would probably
               | need to re-evaluate my use of that software.
               | 
               | It doesn't necessarily need to be the upstream developer,
               | it could be some other organization analogous to a distro
               | maintainer. What matters is that _someone_ who is
               | familiar with the software and code base has actually
               | tested the update in a purposeful way.
        
               | AussieWog93 wrote:
               | As a small developer who handles my own support, I agree
               | with this sentiment completely!!
        
               | xg15 wrote:
               | You'll have the opposite problem though, that you have to
               | trust every single developer to properly manage their
               | dependencies and publish timely patches if there is a
               | vulnerability.
               | 
               | > _As opposed to me, as an individual user with no
               | familiarity with the code base, just switching out the
               | pieces and preying nothing goes wrong._
               | 
               | Wouldn't this be the task of your distro's mainteners,
               | not you? (Who _will_ have familiarity with all codebases
               | as well as options to contact the original developers if
               | there is an unsolvable issue)
        
           | ris wrote:
           | Well, it's what traditional distributions do every day.
           | 
           | In NixOS, we try to work the test suites into the build
           | process and rebuilds happen when any dependencies change, so
           | I'd like to think we get (almost) the best of both worlds.
        
             | Wowfunhappy wrote:
             | > Well, it's what traditional distributions do every day.
             | 
             | Which coincidentally is why I'm not using Linux right now.
             | :) A move towards static linking in the Linux world would
             | go a long way towards changing that.
             | 
             | (Everything I've read about NixOS seems great, thanks for
             | helping with that--it's just that it also has a large
             | barrier to entry.)
        
               | ris wrote:
               | (I'm merely a lowly maintainer in NixOS, shoulders of
               | giants etc.)
        
           | candiddevmike wrote:
           | That's the choice you're making everytime you add another
           | dependency--you are now bound to that repo and all of it's
           | upstreams. You need to follow along send stay on top of
           | Caves, API deprecations, etc. I don't think people weigh this
           | cost properly when doing a build vs integrate decision. Turns
           | out dependency management is a lot like the "when you sleep
           | with someone you're sleeping with everyone they've ever slept
           | with" ideology.
        
         | dcsommer wrote:
         | Are you sure about the "CVE explosion"? From the CNA counting
         | rules
         | https://cve.mitre.org/cve/cna/rules.html#section_7_assignmen...
         | :                 7.2.4 If multiple products are affected by
         | the same independently fixable vulnerability, then the CNA:
         | a. MUST NOT assign more than one CVE ID if the products are
         | affected, because they share the vulnerable code. The assigned
         | CVE ID will be shared by the affected products.
        
           | ris wrote:
           | I'm not sure what would actually happen in reality, whether a
           | single CVE would get endless addenda listing the packages
           | affected by an upstream vulnerability. I certainly see plenty
           | of new CVEs go past which are of the form "xyz had a vendored
           | version of abc, which was vulnerable to ..."
        
         | Zababa wrote:
         | > This is where someone says "well you should just always be
         | running the latest version of everything", which is of course
         | infeasible on a real life system with thousands of installed
         | packages.
         | 
         | Maybe the problem is have a system with thousands of installed
         | packages? If every application is in its own "space", then a
         | security failure won't affect the rest. If you want the best
         | security, you're going to have to do pretty big tradeoffs. If
         | you're not ready to do them, you're going to have to live with
         | an insecure system. I don't think there is a way around this.
        
           | kbenson wrote:
           | Part of the problem is not just getting all the different
           | applications and libraries up to date, but knowing which of
           | them need to be updated.
           | 
           | For a system that widely uses dynamic libs, that's fairly
           | easy to do. Check the version of the installed library, along
           | with any patches it might have, and you have a good idea of
           | whether it needs to be updated.
           | 
           | Now, imagine there's a zlib exploit. That's a _lot_ of things
           | in the system that need to be updated. It 's so ubiquitous
           | that even in a system that's almost entirely built with
           | dynamic libs, there are a few that use it as a static lib,
           | and you'll have to make sure those are updated too (go look
           | at the last zlib CVE and the major distro updates to it and
           | you'll see all those packages, sometimes a couple days later
           | as they discover them too).
           | 
           | A world where these are all statically compiled (or bundled
           | separately in containers) is much more complex. It's not
           | impossible, but you don't naturally get some of the work done
           | for you. To some degree we're already going this way with
           | containers though.
        
             | Zababa wrote:
             | > Part of the problem is not just getting all the different
             | applications and libraries up to date, but knowing which of
             | them need to be updated.
             | 
             | In which was? Is it because it's hard to know which version
             | of zlib was used in X application, or is it because there
             | is no centralized information about this? Maybe that's an
             | opportunity right there: build a graph of dependencies of a
             | system, and alert when something has been compromized and
             | which are the consequences.
        
               | kbenson wrote:
               | To some degree most distro's have this graph and it's
               | somewhat queryable (rpm has to agilities for this, if not
               | easily consumed). The difference is in how accurate and
               | queryable it is, and how easy it is to be fairly sure
               | everything is up to date.
               | 
               | It's not a show stopping problem, but it is something
               | that should be considered and thought closely about. This
               | is a security adjacent problem, and changes in it can
               | have consequences that are larger than "I'm running a
               | library that doesn't have the feature I want".
        
               | xg15 wrote:
               | > _build a graph of dependencies of a system_
               | 
               | You mean in the sense that every application that comes
               | with statically linked libraries should also come with
               | some metadata indicating which libraries/versions are in
               | there?
               | 
               | This would work, but then you have the problem metadata
               | always has: It can get out of sync with the actual thing,
               | developers can forget to properly maintain it (or even to
               | provide it at all) etc etc.
        
               | Zababa wrote:
               | You're right, I didn't answer the question of "who should
               | maintain that stuff?". I was imagining something like
               | libraries in JS or Rust, where people pin the
               | dependencies of their libraries, so with this, you can
               | build a graph of dependencies. I guess I have a very
               | "github" centric vision of things and that most software
               | isn't built this way, which would complicate things.
        
         | bregma wrote:
         | It would certainly make the embedded industry more secure if it
         | was always running the latest version of everything. Or at
         | least the ISPs, because imagine the bandwidth they're be able
         | to charge for as everything from your washing machine to your
         | watch to your lightbulbs to your car downloads new images
         | hourly.
        
           | ris wrote:
           | Just imagining the joy of broken firmware updates being
           | pushed out to all my lightbulbs...
        
         | mst wrote:
         | I believe when heartbleed happened quite a bunch of go progams
         | were yoinking in openssl because the go ssl tooling was still
         | relatively young.
         | 
         | I have no idea what actually happened there, I just recall some
         | moderately annoyed sysadmins figuring out how to rebuild go
         | applications by developers who'd since left the company.
         | 
         | (this is not a shot at go, this is merely a "there might
         | already be -some- data out there" wrt your question)
        
         | alkonaut wrote:
         | The difference between dynamically and statically linking
         | doesn't change the patching story so long as applications ship
         | with all their libs and share nothing.
         | 
         | The only difference is that a statically linked app is a large
         | binary blob in a directory while a dynamically linked app is
         | multiple smaller blobs in a directory.
         | 
         | Whether or not statically or dynamically linked apps are the
         | future, the idea of system wide shared libraries seems like
         | it's going away.
        
           | xg15 wrote:
           | > _The difference between dynamically and statically linking
           | doesn't change the patching story_
           | 
           | I think there are some edge cases where the mainteners will
           | always be caught between a rock and a hard place and the only
           | way to move forward is to involve the application developers
           | - at least as long as library compatibility can't be defined
           | more rigidly.
           | 
           | As a dev, you'd like to pin the versions of your dependencies
           | as tightly as possible, so you know the library will behave
           | in production exactly as it did in your tests.
           | 
           | As a maintener, you'd like applications to specify their
           | dependency versions as loosely as possible, so you can
           | transparently update a dependency in case of critical
           | patches.
           | 
           | This means, there can always occur a case where version x of
           | a library has a critical vulnerability but version x+1 breaks
           | the assumptions that the software is making.
           | 
           | No matter if the library is linked statically or dynamically,
           | this situation can't be solved by the maintener alone.
           | 
           | Of course the GP is also correct in a sense: If applications
           | bundle libraries or set tight version restrictions, it's now
           | the responsibility of the _developers_ to track critical
           | vulnerabilities in all of their dependencies and integrate
           | the patches as quickly as possible.
           | 
           | The problem with bundled libraries is that now we have to
           | trust every single developer to get this right.
        
           | IshKebab wrote:
           | Exactly. The issue is not static vs dynamic; it's bundled vs
           | unbundled.
           | 
           | You could probably even do an "unbundled" statically linked
           | system, e.g. imagine Debian but everything is statically
           | linked. Doesn't matter that everything is statically linked -
           | if there's a vulnerability in libpng you can still easily
           | update dependencies. Just uses more network / disk space.
        
             | dralley wrote:
             | > Doesn't matter that everything is statically linked - if
             | there's a vulnerability in libpng you can still easily
             | update dependencies. Just uses more network / disk space.
             | 
             | It's not that simple
             | 
             | * Someone, somewhere - probably distro volunteers - have to
             | manage this process of rebuilding dozens, hundreds, or
             | thousands of packages, and knowing which packages need to
             | be updated to begin with. The amount of work required from
             | distros (again, mostly volunteers) would be an order of
             | magnitude greater even if the process was automated.
             | 
             | * Likewise, the amount of resources needed to rebuild all
             | of those packages, and distribute them, would be orders of
             | magnitude greater.
             | 
             | * The amount of extra work (both for humans and computers)
             | required would delay the rollout of patches.
             | 
             | * And in the real world, a lot of users don't have
             | limitless network bandwidth and disk space.
             | 
             | * And auditing systems for security vulnerabilities would
             | be geometrically more difficult. Now you need to track a
             | recursive bill of materials for every package on the
             | system.
             | 
             | If there's a security bug in libc or something along those
             | lines - which are fairly common- distros would need to
             | rebuild practically every package in the OS. It's all well
             | and good to talk about how it would be no big deal but I
             | think if it was ever actually implemented no user would
             | accept that trade-off.
        
             | Wowfunhappy wrote:
             | I agree, but if you're bundling the dependencies why not
             | just statically link them too? I've never understood the
             | way macOS does it (dynamic linking, but app bundles ship
             | with the libraries they need included).
        
               | alkonaut wrote:
               | This is also how windows apps always worked. It's good
               | for deployment because an update might not touch more
               | than a few files.
               | 
               | Also not all language ecosystems support static linking
               | at all, e.g .NET
        
               | dragonwriter wrote:
               | > Also not all language ecosystems support static linking
               | at all, e.g .NET
               | 
               | The "language ecosystem" of .NET supports static linking
               | via .NET Native, doesn't it?
        
               | mastax wrote:
               | .NET native is a dead end technology that only supported
               | old UWP apps.
               | 
               | There's the new single file builds but those just
               | concatenate the dynamic libs together into one file.
               | 
               | There's also the new .NET linker for trimmed builds, but
               | that requires special annotations from many libs to work.
        
               | xg15 wrote:
               | Not a mac user, but one upside I can see is that this
               | forces apps to at least declare which libraries they are
               | using. If everything is statically linked, it might be
               | hard to figure out where on the system a vulnerable
               | library is being used at all.
        
               | ziml77 wrote:
               | In some cases it's required such as if you're using a
               | closed-source library that doesn't provide a static
               | version or if you're calling LGPL code from non-GPL-
               | compatible code.
        
           | bbarnett wrote:
           | shared ram too. it adds up.
        
             | mpyne wrote:
             | It does, but we're already moving to a world where deployed
             | apps are one of the very few things running in a container,
             | which is itself running somewhere in a VM, so there's less
             | sharing to achieve.
        
               | ithkuil wrote:
               | Yeah. In that scenario the distinction between dynamic
               | and static linking is moot. In both cases you need to
               | update the container image and when you do you only fixed
               | that container image and you still need to update all the
               | other images
        
           | mercurialfck wrote:
           | There's absolutely no need for just one-ring-to-rule-them-all
           | copy of libz.so. Shared libraries can be versioned and
           | garbage collected like how habitat (hab) does it: in separate
           | directories.
        
       | forrestthewoods wrote:
       | I care far more about about bundled vs unbundled than dynamic vs
       | static.
       | 
       | I am passionately in-favor of bundled. Unbundled distribution is
       | what has led to software being so outrageously impossible to
       | execute that the only sane path to distribution is to create a
       | bundled docker image. https://xkcd.com/1987/
        
       | flohofwoe wrote:
       | Unfortunately not even Linux allows full static linking (as far
       | as I'm aware at least) as soon as libraries like OpenGL are
       | involved.
       | 
       | Dynamic linking with core system libraries which are guaranteed
       | to exist(!) is fine though, for everything else dynamic linking
       | mostly has downsides.
       | 
       | Also: it's _only_ Linux where this is an issue, because it 's
       | nearly impossible to build small self-contained executables which
       | run on another Linux machine, thanks to glibc's versioning mess
       | (at least there's MUSL as an alternative to create portable
       | command line tools).
        
         | Macha wrote:
         | Didn't Go have to throw in the towel on not dynamically linking
         | libc on macOS as there's no guarantee of syscall stability at
         | all?
        
           | patrakov wrote:
           | I don't know about macOS, but they also did this on OpenBSD.
           | The reason is a recent security addition in OpenBSD, where
           | they made it illegal (as in: violate it, and the process will
           | be killed) to issue syscalls to the kernel from anything
           | other than libc.
        
             | Someone wrote:
             | https://lwn.net/Articles/806776/ discusses that feature and
             | says "Switching Go to use the libc wrappers (as is already
             | done on Solaris and macOS)", so yes, they did that on
             | macOS, too.
             | 
             | Reading https://golang.org/doc/go1.11#runtime, that's
             | fairly recent (since go 1.11)
        
           | flohofwoe wrote:
           | Don't know, I only know that I can build a macOS command line
           | tool on one machine, and run it on a different machine
           | without problems, while on Linux it's very easy to run into
           | the glibc version compatibility problem.
        
             | onei wrote:
             | I always understood the glibc versioning problem to be
             | solved by just building on the oldest common ancestor. If
             | you have rhel5 through 8, you build on rhel5 and move on.
             | If you depend on more than glibc... Well rhel5 is less fun.
        
               | ghoward wrote:
               | Author here.
               | 
               | Unfortunately, building on the oldest common ancestor
               | will still not save you from ABI breaks. An example is
               | [1]. If the size of `intmax_t` changes between that
               | oldest ancestor and the machines you are running on, get
               | ready for pain.
               | 
               | [1]: https://thephd.dev/intmax_t-hell-c++-c
        
               | ptsneves wrote:
               | Indeed, the only difference is that you might be using
               | less optimal system calls, but there is no reason to be
               | blocked and not target ancient kernels. I linked to 2.6
               | this year and everything ran fine.
        
         | p_l wrote:
         | At least one can statically link the application except for
         | forcibly-dynamic libs like libGL (also, apparently using
         | dlopen() is often _faster_ than naive dynamic linking...)
        
         | tyingq wrote:
         | It's also tricky if you need to call gethostbyname() and expect
         | system local behavior.
        
         | edflsafoiewq wrote:
         | Well yeah, the GL calls are _actually different_ on different
         | machines right? Isn 't that the absolute first consideration
         | when deciding if you should dynamically link?
        
           | flohofwoe wrote:
           | I'd rather expect that the 'user-facing' GL implementation
           | (GL2.x, GL3.x, ...) is identical and can be statically
           | linked, and that only the graphics driver layer differs
           | between machines (which would be similar to an executable
           | which links the C library statically, which in turn talks to
           | the operating system through syscalls).
           | 
           | But to be fair, dynamically linked GL wouldn't be a problem,
           | if this wouldn't also pull in a dynamically linked glibc (it
           | always comes back to glibc unfortunately).
        
             | cesarb wrote:
             | > I'd rather expect that the 'user-facing' GL
             | implementation (GL2.x, GL3.x, ...) is identical and can be
             | statically linked, and that only the graphics driver layer
             | differs between machines (which would be similar to an
             | executable which links the C library statically, which in
             | turn talks to the operating system through syscalls).
             | 
             | The problem is that the "graphics driver layer" (except a
             | small amount of security-sensitive pieces) runs in user
             | space, in the same process which is doing the
             | OpenGL/Vulkan/Metal/etc calls. The way modern graphics
             | hardware works is that the application writes a very
             | complex and hardware-dependent set of command buffers,
             | which are then submitted to the hardware, and only this
             | final submission goes through the kernel (for security
             | reasons). If the application had to call the kernel for
             | each and every step of the creation of the command buffers,
             | it would kill the performance.
        
             | creshal wrote:
             | Isn't this the point of libglvnd?
        
           | simias wrote:
           | That's normally what kernel abstractions are for, but
           | unfortunately for various historical reasons graphics API
           | often bypass the kernel in part or in full. There's also the
           | problem that graphics are often very performance-sensitive so
           | costly abstractions may not be desirable.
        
       | PaulDavisThe1st wrote:
       | > I don't think Ulrich Drepper could have foreseen all of the
       | problems with DLL Hell (though there were signs on Windows),
       | 
       | Drepper's article is from 2006. "DLL Hell" was beyond fully
       | understood at that point, certainly by Drepper and large numbers
       | of other well informed programmers.
        
         | ghoward wrote:
         | Author here.
         | 
         | In everything in the post, I attempted to give Drepper the
         | benefit of the doubt, though there are cases where I don't want
         | to since he sounds (to me) paternalistic in the way Apple and
         | Microsoft do.
         | 
         | So perhaps he understood, but I didn't want to assume that,
         | especially since it might inflame the discussion, which is
         | exactly what I was trying to avoid doing ( _again_ ).
        
           | PaulDavisThe1st wrote:
           | DLL Hell was well understood in the mid-90s. I think you need
           | to give him more benefit of the doubt on this point.
        
             | torstenvl wrote:
             | If someone glosses over an issue, it's either innocent
             | (didn't know about it, didn't understand it) or deceptive
             | (intentionally ignored it). Giving Drepper the benefit of
             | the doubt here means assuming he didn't know or understand
             | the problem he failed to adequately address.
        
               | PaulDavisThe1st wrote:
               | Here is most of the relevant text from TFA:
               | 
               | > Ulrich Drepper claims, rightly, that you only need to
               | update a shared library to apply security fixes to all
               | executables that rely on that library.
               | 
               | > This is a good thing! It is also a good vision.
               | 
               | > Unfortunately, it is only a vision because it's not the
               | whole story.
               | 
               | > As of yet, there is no way to automatically determine
               | if the ABI or API of a shared library changed between
               | updates. This is called DLL Hell.
               | 
               | The final two words are a link to the wikipedia page on
               | "DLL Hell". The final sentence of the intro section there
               | states:
               | 
               | > DLL Hell is the Windows ecosystem-specific form of the
               | general concept dependency hell.
               | 
               | That is, TFA defines "DLL Hell" using a wikipedia page
               | that explicitly states that it is a Windows-specific
               | version of a more general problem.
               | 
               | Drepper's article from 2006 fully tackles the general
               | problem, albeit without (as the TFA puts it): "a way to
               | automatically determine if the ABI or API of a shared
               | library changed between updates." Drepper's primary
               | suggestion there is a naming/versioning scheme which
               | describes specifically whether the ABI/API has changed in
               | ways that matter for shared linkage. It's not automatic -
               | that part is true. But is is a solution, widely used in
               | Linux and *nix more broadly.
               | 
               | Did Drepper address the Windows specific parts of "DLL
               | Hell". He did not, but that's because "DLL Hell" is
               | Windows specific (as other comments here fully clarify),
               | and he was not writing about how to fix the full scope of
               | that particular nightmare. He likely understood "DLL
               | Hell" as well as anyone, nevertheless.
        
               | torstenvl wrote:
               | I think you're missing the point. When the only
               | alternatives are intellectual dishonesty or ignorance,
               | assuming ignorance is the more generous option.
               | Therefore, assuming he _didn 't_ know _is_ giving him the
               | benefit of the doubt.
               | 
               | The reasoning behind your differing assumption is beside
               | the point, _even though your reasoning is more likely to
               | be true_.
        
               | PaulDavisThe1st wrote:
               | I think you're missing my point.
               | 
               | The TFA's claim is that Drepper didn't get all the issues
               | with DLL Hell. My claim is that Drepper did get all of
               | the issues, and addressed those that were not Windows-
               | specific (e.g. per-process COM servers etc.)
        
               | torstenvl wrote:
               | No. I am not addressing your point because your point is
               | not logically related to the topic under discussion.
               | 
               | Specifically, your post said:
               | 
               | > _DLL Hell was well understood in the mid-90s. I think
               | you need to give him more benefit of the doubt on this
               | point._
               | 
               | The substantive question about whether he adequately
               | addressed the issues with dynamic linking is beside the
               | point. Feel free to agree or disagree with the above
               | poster. Not relevant.
               | 
               | The post I am responding to is about, _given ghoward 's
               | position that X did not adequately address topic Y_,
               | whether it is "giving X the benefit of the doubt" to
               | further assume (a) X _did_ know about topic Y; or (b) X
               | _did not_ know about topic Y.
               | 
               | Because (a) implies dishonesty but (b) merely implies
               | ignorance, "giving X the benefit of the doubt" means you
               | should assume (b) until proven otherwise.
               | 
               | If you want to respond to other topics, feel free to hit
               | the reply button below someone else's post.
        
               | PaulDavisThe1st wrote:
               | The point under discussion is this line (and the
               | surrounding text) from TFA:
               | 
               | > I don't think Ulrich Drepper could have foreseen all of
               | the problems with DLL Hell (though there were signs on
               | Windows),
        
             | cesarb wrote:
             | > DLL Hell was well understood in the mid-90s.
             | 
             | Also, there are some additional facets of DLL Hell which
             | happened in the Windows of the mid-90s which are not as
             | relevant to Linux. What made DLL Hell so bad there was that
             | installing any random _program_ could _replace_ globally
             | shared libraries, sometimes even with an _older_ version.
             | That is, installing a game could make an unrelated
             | productivity app stop working, because the game helpfully
             | installed newer (or older!) versions of the shared
             | libraries it needed, into the same globally shared
             | directory in which nearly all DLLs lived. It got so bad
             | (even DLLs which came with the operating system were being
             | overwritten) that Microsoft IIRC initially introduced a
             | system which detected when this happened, and _replaced the
             | DLLs again_ with a clean copy it had stashed somewhere else
             | (and later, made these files more directly protected).
             | 
             | That's before considering the disaster that is in-process
             | COM servers; presenting a standard "open file" dialog, or
             | doing some printing, is enough to make Windows load
             | arbitrary DLLs into your process (shell extensions and/or
             | printer drivers), and these often don't have the highest
             | code quality. And then there are some things which inject
             | arbitrary DLLs into _every_ process...
             | 
             | Compared to that, the dynamic linking issues in the Linux
             | world are much more bearable. You don't see arbitrary
             | programs overwriting the global copy of something like
             | libgtk or openssl or zlib or libc, the only arbitrary
             | dynamic libraries being loaded into a process are things
             | like the NSS ones (usually from a small well behaved set)
             | or plugins from the graphics libraries (also usually from a
             | small well behaved set), and the only dynamic library being
             | injected into every process is the vDSO from the kernel.
        
             | ghoward wrote:
             | I'm having a hard time understanding because if DLL Hell
             | was well understood by 2006, it seems I should have given
             | him _less_ benefit of the doubt, but you say I should give
             | him _more_.
             | 
             | Apologies.
        
         | spacechild1 wrote:
         | I would even claim DLL didn't really exist anymore in 2006. I
         | remember it from the late 90s, but haven't encountered it in
         | the last 15 years. Fortunately, people have learned their
         | lessons.
        
       | themulticaster wrote:
       | > Second, ldd, the dynamic linker, can be manipulated into
       | executing arbitrary code.
       | 
       | ldd is _not_ the dynamic linker, it 's only a tool to debug the
       | process of dynamic linking. The dynamic linker is ld-linux.so
       | (exact name depends on glibc version, architecture, etc.)
       | 
       | Also, I think the linked article [1] about the security of ldd is
       | somewhat useless. The ldd(1) manpage [2] is very explicit about
       | the security of ldd and tells you not to run ldd on untrusted
       | executables:
       | 
       | > [...] Thus, you should _never_ employ ldd on an untrusted
       | executable, since this may result in the execution of arbitrary
       | code.
       | 
       | It's a little amusing how the linked blog post explains how to
       | create a malicious executable that runs arbitrary code when
       | inspected with ldd, noting that "I researched this subject
       | thoroughly and found that it's almost completely undocumented. I
       | have no idea how this could have gone unnoticed for such a long
       | time." and concluding with "Never run ldd on unknown
       | executables!" - all while the manpage literally mentions that
       | precise known limitation in the third paragraph!
       | 
       | To be fair, you could argue that this limitation is not widely
       | known and people should be made aware of the risks of ldd, but on
       | the other hand you can hardly criticize ldd when its manpage is
       | that explicit about the issue.
       | 
       | [1] https://catonmat.net/ldd-arbitrary-code-execution
       | 
       | [2] https://www.man7.org/linux/man-pages/man1/ldd.1.html
        
         | ratmice wrote:
         | While you are technically correct, ldd is literally a bash
         | script which runs ld-linux.so...
        
           | throwaway09223 wrote:
           | ld-linux.so's job is to link and run an executable. It's not
           | a vulnerability that it runs executable it's been handed. If
           | you run "ld-linux.so /bin/ls" you're just running "ls"
           | There's no security issue with this behavior. It's an
           | interpreter like any other - same as /usr/bin/python.
           | 
           | The argument around ldd having a vulnerability is that it
           | appears to be an introspective tool. It is not immediately
           | obvious that it just executes "LD_TRACE_LOADED_OBJECTS=1 ld-
           | linux.so /bin/ls" to print what the linker does during an
           | actual, live linking operation.
           | 
           | ldd documenting this fact to remind people seems reasonable
           | to me. There are other tools like readelf which can inspect
           | objects without executing their contents. Dynamic linking is,
           | well, dynamic and it can depend on code executed at runtime
           | -- so it is necessary to do so to get an accurate report of
           | what will happen.
        
         | ghoward wrote:
         | Author here.
         | 
         | Thank you for pointing out my mistake. I'll fix it.
        
           | Hello71 wrote:
           | ld isn't the dynamic linker either, it's the object linker.
           | the name of the dynamic linker is ld.so or ld-linux.so. this
           | is also documented in the ldd man page.
        
             | eqvinox wrote:
             | Fun fact: despite the library path and name, ld-linux.so is
             | executable. Try running                 /lib64/ld-
             | linux-x86-64.so.2
             | 
             | On your favorite Linux box. (Adjust path if needed for
             | different architecture / distro.)
        
               | cataphract wrote:
               | In fact, /lib64/ld-linux-x86-64.so.2 is set as the
               | interpreter of the dynamic executables, so executing
               | ./dyn_exec ends up being equivalent to /lib64/ld-
               | linux-x86-64.so.2 ./dyn_exec. The explicit form is
               | sometimes useful, for instace if you want to execute the
               | file with an interpreter other than the one specified in
               | the executable, or if you want to pass one of the
               | parameters that ld.so accepts.
        
               | rkeene2 wrote:
               | It's also useful if you accidentally chmod -x chmod...
        
         | eqvinox wrote:
         | Secure - and more useful - replacement: lddtree.
         | 
         | Part of "pax-utils" or similar package on some distros. It's
         | actually a shell script wrapper around scanelf (also pax
         | utils), which reads the appropriate bytes from ELF files
         | without just executing the ELF file.
        
           | rkeene2 wrote:
           | I usually just do: objdump -x <binary> | grep NEEDED but this
           | is a great tip !
        
             | colonwqbang wrote:
             | Often you need to find not only the soname but the actual
             | .so file that is found and linked. This can be affected by
             | environment variables like LD_PRELOAD and LD_LIBRARY_PATH.
             | Unfortunately I don't know of a way to check this without
             | ldd. If somebody knows of a tool that can do this without
             | executing the binary, please share.
        
       | [deleted]
        
       | rwmj wrote:
       | I read this interesting article recently about all the security
       | problems that static linking, bundling and pinning cause for
       | Linux distributions:
       | 
       | https://blogs.gentoo.org/mgorny/2021/02/19/the-modern-packag...
        
         | ghoward wrote:
         | Author here, and I am also a Gentoo user on my daily driver.
         | 
         | I read that article when it came out, and it felt disingenuous.
         | 
         | mgorny helps maintain Gentoo, which of all distributions, does
         | rebuild software as necessary.
         | 
         | It would be a simple change to portage to rebuild all
         | dependencies of a library when that library is updated,
         | regardless of if the user uses static linking or not. In fact,
         | there is a project to have a statically-linked musl-based
         | Gentoo.
         | 
         | So it's possible and (I would argue) would be an easy change.
         | mgorny would not have to worry about much because he could, in
         | essence, treat all libraries as dynamic libraries. The only
         | pain to be felt would be by users who would see their build
         | times increase (to rebuild dependents).
         | 
         | As a user, I would _happily_ take that tradeoff. I already
         | build my browser, which takes 5-8 hours; I 'll happily rebuild
         | dependents of updated libraries.
        
           | AshamedCaptain wrote:
           | > I already build my browser, which takes 5-8 hours; I'll
           | happily rebuild dependents of updated libraries
           | 
           | This is ridiculous. I most definitely do NOT want to rebuild
           | literally EVERY SINGLE PACKAGE on my system the moment they
           | fix a typo in libz.
           | 
           | With static linking literally every other Gentoo upgrade
           | becomes a world rebuild. See Nix. It is just insane and the
           | main reason I don't use NixOS (without a 3rd party binary
           | cache/host) for anything other than very small systems.
        
             | ghoward wrote:
             | > This is ridiculous. I most definitely do NOT want to
             | rebuild literally EVERY SINGLE PACKAGE on my system the
             | moment they fix a typo in libz.
             | 
             | I understand. You don't. I do.
             | 
             | What my post is arguing for is not one or the other, but
             | giving the user the choice.
             | 
             | I think you would be okay with making the choice of _not_
             | rebuilding everything, and I would make the opposite
             | choice.
             | 
             | I want to live in a world where we can both choose.
        
               | AshamedCaptain wrote:
               | But you can already rebuild the entire world on every
               | package change if you want (for sadomasochistic reasons?
               | it basically makes gentoo with say KDE _unusable_); and I
               | fail to see if there would be any practical difference on
               | whether you statically link, you dynamically link to a
               | hash-based soname a la nix, or you dynamically link --
               | you are just rebuilding everything on every library
               | change, and you are already assuming ASLR is useless.
        
           | stefantalpalaru wrote:
           | > It would be a simple change to portage to rebuild all
           | dependencies of a library when that library is updated
           | 
           | No change needed. It can already be done with the special
           | ":=" dependency slot: https://devmanual.gentoo.org/general-
           | concepts/dependencies/#...
        
           | mst wrote:
           | > It would be a simple change to portage to rebuild all
           | dependencies of a library when that library is updated
           | 
           | I believe e.g. freebsd's portmaster takes a -r switch to mean
           | "rebuild this and all its dependents" (I presume you meant
           | revdeps here), which gets you quite a long way.
        
         | Aqueous wrote:
         | But should distributing and ensuring the security of software
         | that users install on their own systems even be the
         | responsibility of operating system maintainers?
        
           | rwmj wrote:
           | Isn't it obvious? Or do you suggest that end users should be
           | responsible for checking and patching every package on their
           | system? Given that Red Hat has a large department just to do
           | that for RHEL, I can't imagine where end users would begin.
        
             | Aqueous wrote:
             | No, I'm suggesting that the authors of the applications are
             | responsible for patching and distributing their own
             | application like on every other major OS.
        
           | eqvinox wrote:
           | Yes.
           | 
           | If I installed software through my OS package manager, where
           | else would I get security updates from?!
        
             | Aqueous wrote:
             | Like on every other major OS the publisher of those
             | applications. And I mean they should be distributing the
             | applications as well. Not the OS publishers.
        
       | api wrote:
       | My personal view is that only standard core system libraries
       | should be dynamically linked. Anything that is not totally
       | ubiquitous should be static.
       | 
       | That seems to be the Go and Rust default approach, or close to
       | it. Link libc and such, but build in more obscure things.
       | 
       | The idea of dynamically linking everything doesn't scale. It asks
       | too much of distribution maintainers and makes them the choke
       | point for every little upgrade.
        
         | PaulDavisThe1st wrote:
         | So, you link against a GUI toolkit that can be used to display
         | images. Images in many different formats, including some that
         | haven't been used in more than a decade and are unlikely to
         | ever be encountered.
         | 
         | Do you want the GUI toolkit linkage to automatically include
         | every object module for every possible image file format in
         | your static image, or do you want the GUI toolkit to use
         | runtime dynamic linkage (explicit or implicit) to import only
         | the modules required for the image formats actually
         | encountered?
        
           | TillE wrote:
           | If I'm distributing binaries as a commercial product, I
           | definitely want the former.
        
             | PaulDavisThe1st wrote:
             | There are many "commercial products" that run on Linux
             | whose installation includes various *.so files that will be
             | dynamically linked at run time. Are you saying this is a
             | bad idea?
        
             | TeMPOraL wrote:
             | What about formats that don't exist yet? What about giving
             | you the ability to supply your own encoder/decoder for some
             | niche format you're using?
        
         | josefx wrote:
         | > That seems to be the Go and Rust default approach
         | 
         | I think Go tried to go fully static but ran into problems as
         | most mainstream OSes except for Linux do not provide a stable
         | system call interface, so applications have to link at least
         | one dynamic library. The Linux system call interface is stable,
         | but expects a C style stack layout and that also bit the Go
         | devs. multiple times. So you can probably go fully static if
         | you are only targeting Linux.
        
           | cesarb wrote:
           | > I think Go tried to go fully static but ran into problems
           | [...] So you can probably go fully static if you are only
           | targeting Linux.
           | 
           | There's yet another problem Go ran into on Linux: resolving
           | hostnames. The official way to resolve hostnames in a glibc-
           | based system is to read /etc/nsswitch.conf, dynamically load
           | the libraries referenced there, and call each of them in the
           | correct order. Go tried to emulate that mechanism, going
           | through glibc only when it saw a NSS module it didn't know,
           | but that caused issues on musl-based systems which used a
           | different mechanism
           | (https://github.com/golang/go/issues/35305). And it cannot
           | ever be fully static on glibc-based systems, since there's
           | always the chance of someone adding a new module to
           | /etc/nsswitch.conf (for instance, some Linux distributions
           | have now added the newer "resolve" module to that line, so Go
           | would have to implement yet another one to keep being fully
           | static).
           | 
           | The Rust approach of dynamically linking to the shared C
           | libraries (statically linking only the Rust libraries)
           | avoided all these issues.
        
       | mst wrote:
       | > Operating systems can statically map libraries.
       | 
       | I have "fond" memories of running perebaseall on cygwin that this
       | sentence reminded me of.
        
       | cesarb wrote:
       | The first section, on security fixes ("[...] you only need to
       | update a shared library to apply security fixes to all
       | executables that rely on that library"), starts talking about
       | security fixes but then suddenly switches to talking about ABI
       | and API breaks. But that sleight-of-hand hides the fact that many
       | (perhaps even most) security fixes do _not_ break the ABI or API;
       | they are completely contained to the implementation (one obvious
       | exception would be if the security issue was caused by bad API
       | design, but even then often there are ways to fix it without
       | breaking the ABI).
        
         | rewma wrote:
         | > But that sleight-of-hand hides the fact that many (perhaps
         | even most) security fixes do not break the ABI or API; they are
         | completely contained to the implementation (one obvious
         | exception would be if the security issue was caused by bad API
         | design, but even then often there are ways to fix it without
         | breaking the ABI).
         | 
         | Right you are. I was also perplexed when I read that non
         | sequitur. The author's reference to DLL Hell also suggests
         | there's some confusion in his analysis of the underlying
         | problem, given that DLL Hell is very specific to windows and at
         | best is only orthogonally related to ABI. The author's
         | references to API changes make even less sense, and definitely
         | cast doubt over his insight into the issue.
        
           | salawat wrote:
           | Differentiating DLL and SO hell is getting a bit beyond
           | pedantic as they are implementations of the same fundamental
           | abstraction. Any substantial difference in merely one of
           | implementation details.
        
             | rewma wrote:
             | > Differentiating DLL and SO hell is getting a bit beyond
             | pedantic (...)
             | 
             | It really isn't. Unlike linking problems, where the problem
             | is focused on how you need to fight your dependencies to be
             | able to resolve symbols, DLL Hell has been for over a
             | decade a dependency resolution problem that is solved at
             | the packaging level.
             | 
             | More importantly, on Windows, where sharing DLLs is not a
             | thing, you can simply drop a DLL in the app dir and be done
             | with it. In fact, it's customary for windows apps to just
             | bundle all their dependencies.
        
           | pjmlp wrote:
           | And was kind of fixed with application manifests, on XP.
        
         | ghoward wrote:
         | Author here.
         | 
         | Sure, most security fixes do not break the ABI or API. But it
         | only takes _one_ , and then you have a security issue.
         | 
         | So I can understand your sentiment that it was sleight-of-hand,
         | but I argue that it's not.
         | 
         | Also, we only _think_ that most security updates don 't break
         | API or ABI because we have no automatic way of _checking_. It
         | 's all thoughts and prayers. I'm a religious person, but
         | thoughts and prayers are _not_ a good way to do computer
         | security.
        
           | gnufx wrote:
           | > we have no automatic way of checking
           | 
           | I do for Fedora packages I maintain. There can be false
           | positives, like on symbols that shouldn't be exported, but
           | you can look at the API to start with. (The library author
           | should do that, of course.)
        
           | zozbot234 wrote:
           | If a security fix _must_ break the API, you can bump up the
           | SONAME which forces all packages depending on your shared
           | library to recompile against the new version. This is
           | comparable to the requirements under static linking, but you
           | only have to do it when absolutely needed.
        
           | throw0101a wrote:
           | > _Sure, most security fixes do not break the ABI or API. But
           | it only takes one, and then you have a security issue._
           | 
           | As someone who has been a Unix sysadmin for almost twenty
           | years now, I don't recall every installing a patch or package
           | update that broke the ABI/API.
           | 
           | If one should happen to occur next week, I'll still go with
           | leaning towards linked libraries, as their convenience out-
           | weights (has out-weighted) the once-in-twenty-years
           | occurrence of breakage.
           | 
           | > _It 's all thoughts and prayers. I'm a religious person,
           | but thoughts and prayers are_ not _a good way to do computer
           | security._
           | 
           | Ditto. But "faith" and "trust" are (almost) synonyms. And I
           | trust the updaters of libraries to be able to get out a
           | security update in short order more than I trust the
           | possibility of all the package maintainers that link to it to
           | be able to coordinate a quick update.
           | 
           | When OpenSSL's Heartbleed occurred it was one package update
           | and restarting of a bunch of processes (helped by the
           | _checkrestart_ utility in the _debian-goodies_ package). That
           | 's a lot quicker than updating a bazillion dependents.
        
             | ghoward wrote:
             | I understand where you are coming from. That's actually why
             | I gave my ideas to get the best from both types of linking.
        
             | xgbi wrote:
             | Not related to the OP but I've had many issues with non
             | standard vendors placing breaking changes in their
             | patch/minor releases. You think you're updating to a
             | relatively normal version and it actually breaks your
             | entire production.
             | 
             | I'm looking at you, MySQL, docker, Mongodb, or harbor.
             | 
             | I'm not complaining, those are provided free of charge and
             | we are happy when it works, but semver is really all that
             | matters in this: either you adhere and it's smooth sailing,
             | or you don't and then please don't use semver-looking
             | numbering.
        
               | dralley wrote:
               | There's a difference between vendor provided packages and
               | distro provided packages. Distros pay a lot more
               | attention to breaking changes.
        
               | [deleted]
        
           | taviso wrote:
           | I've (probably?) found more linker vulnerabilities than
           | anyone else, but don't really understand your argument. I
           | definitely understand library search path vulnerabilities
           | (I've literally found bugs in $ORIGIN parsing, e.g.
           | CVE-2010-3847 was one of mine!).
           | 
           | We think security updates don't break API/ABI because they're
           | backported to all previous major versions that are still
           | actively supported. This isn't "thoughts and prayers", it's
           | work that the packagers and maintainers do. I can't tell you
           | how useful being able to fix a vulnerability in every program
           | is, just think of updating every application that uses libpng
           | on your desktop, or every server application that uses
           | OpenSSL.
           | 
           | I just can't imagine wanting to give that up, because of name
           | mangling ABI issues!
        
             | gnufx wrote:
             | I hope that puts it to rest; I can hope. Unfortunately, my
             | experience is that people who haven't fixed multiple
             | security problems, and probably have never heard of taviso,
             | know a lot more about security than people who have...
        
               | ghoward wrote:
               | I _have_ heard of taviso. That doesn 't mean I can't
               | disagree with him.
        
               | gnufx wrote:
               | Sorry, I was talking generally. I try to avoid being
               | personal in posts.
        
           | [deleted]
        
           | froh wrote:
           | That's why you run a trusted Linux distro in production, like
           | sles or rhel or ubuntu, their core value proposition is
           | _exactly_ this: Provide binary compatible drop in replacement
           | security updates for shared objects over the lifetime of the
           | platform. They even maintain kernel internal abi
           | compatibility for the released and maintained product, so
           | your exotic-hardware.ko module doesn't need to be updated.
           | 
           | Also there is mature tooling around ABI compliance checking:
           | https://lvc.github.io/abi-compliance-checker/
           | 
           | This is used to document the ABI stability of mature projects
           | like so: https://abi-laboratory.pro/index.php?view=tracker
           | 
           | E: re thoughts and prayers, the mature Linux distro all run
           | extensive revalidation for their security updates. Breaking
           | customer binary applications with a security update is the
           | absolute worst case scenario. For yocto or similar cost free
           | projects, or Linux distros that are source, not binary, that
           | may be a different story. But for sles, rhel, ubuntu it's not
           | thoughts and prayers but a maintenance verification and
           | release process on their side and a monthly bill on yours.
        
       | tinus_hn wrote:
       | Considered harmful essays considered harmful:
       | 
       | https://meyerweb.com/eric/comment/chech.html
        
         | colejohnson66 wrote:
         | I was wondering when something like this would be written.
         | We've come full circle.
        
         | ghoward wrote:
         | Author here.
         | 
         | I read that essay a while ago and reread it before publishing
         | my post.
         | 
         | I don't think the author makes a good case. They make bare
         | assertions without examples and without evidence, so it's
         | unconvincing.
         | 
         | For example, they say that a "Considered Harmful" essay:
         | 
         | > often serves to inflame whatever debate is in progress, and
         | thus makes it that much harder for a solution to be found
         | through any means.
         | 
         | But this could be true of any essay.
         | 
         | More to the point, I think that the _tone_ of the essay or post
         | matters more. Case in point, my  "Dynamic Linking Needs to Die"
         | post probably inflames the debate more than this new
         | "Considered Harmful" post.
         | 
         | Maybe "Considered Harmful" essays have a _reputation_ for
         | inflaming the debate, and there 's an argument to be made for
         | that. But I personally don't think my post is one that follows
         | that reputation.
         | 
         | Also, because I was writing a post about a pre-existing
         | "Considered Harmful" essay, it was natural to use it,
         | especially since I needed to make it clear that my previous
         | post is harmful too.
        
           | jasode wrote:
           | _> Maybe "Considered Harmful" essays have a reputation for
           | inflaming the debate, and there's an argument to be made for
           | that. But I personally don't think my post is one that
           | follows that reputation._
           | 
           | Neither your post title nor your content bothered me but just
           | as fyi... some people are very irritated by _" snowclones"_.
           | My previous comment with various examples:
           | https://news.ycombinator.com/item?id=19071532
        
             | ghoward wrote:
             | Yeah, I suspected people might be. But since I was
             | answering a "Considered Harmful" essay, it was too easy.
             | Eh, oh well.
        
           | IshKebab wrote:
           | Either way, "considered harmful" is a tired and unoriginal
           | cliche. Using it twice in one title is just stonkingly
           | uncreative.
           | 
           | I agree with most of the article but the title sucks!
           | 
           | Also I'm not sure how you wrote so much about using LLVM IR
           | as a portable executable format without mentioning
           | WebAssembly or PNaCl once!
        
             | ghoward wrote:
             | > Either way, "considered harmful" is a tired and
             | unoriginal cliche. Using it twice in one title is just
             | stonkingly uncreative.
             | 
             | > I agree with most of the article but the title sucks!
             | 
             | You are correct. I don't claim to be creative, especially
             | with post titles. To be honest, I'm surprised this post got
             | as much traction as it did.
             | 
             | > Also I'm not sure how you wrote so much about using LLVM
             | IR as a portable executable format without mentioning
             | WebAssembly or PNaCl once!
             | 
             | Because, in my opinion, they will not solve the problem.
             | 
             | WebAssembly is too constrained.
             | 
             | PNaCl uses LLVM, which would need to be redesigned to do
             | all that I want.
        
       | lamontcg wrote:
       | I sorta think that static linking needs to die, but dynamic
       | linking also needs to be a lot better.
       | 
       | More of the choices that get made during autoconf needs to be
       | made lazily at link time. That would make executables and dynamic
       | libraries more like containers or lego blocks that could be
       | rearranged without compilation. Which would fix a lot of issues
       | that people hate about package managers and bloated distros along
       | the way.
        
       | gnufx wrote:
       | There are examples of the sort of reason I want dynamic linking
       | under https://news.ycombinator.com/item?id=28734875 and you also
       | really want it for profiling/tracing, and possibly debugging,
       | e.g. in HPC with the MPI PMPI mechanism.
       | 
       | libabigail is one tool for checking that a library's ELF
       | versioning hasn't lied about ABI compatibility when you're
       | building packages, for instance.
       | 
       | Unfortunately people typically don't follow Drepper's advice (as
       | far I remember the article) on building shared libraries,
       | particularly using symbol versioning a la glibc.
        
         | ghoward wrote:
         | Author here.
         | 
         | Yeah, there are reasons to use dynamic linking, but I'm still
         | not sure why dynamic linking gives you better profiling and
         | tracing.
         | 
         | > Unfortunately people typically don't follow Drepper's advice
         | (as far I remember the article) on building shared libraries,
         | particularly using symbol versioning a la glibc.
         | 
         | I mention this in the post, but I quoted another person, Ian
         | Lance Taylor, who wrote the gold linker, as saying that symbol
         | combining, a portion of symbol versioning, is an _unsolved
         | problem_.
         | 
         | Even if developers don't follow Drepper's advice, I wouldn't
         | put the blame on them because of that fact; I would argue that
         | it's more accurate to say they _can 't_ really follow his
         | advice.
        
           | dchapp wrote:
           | > Yeah, there are reasons to use dynamic linking, but I'm
           | still not sure why dynamic linking gives you better profiling
           | and tracing.
           | 
           | It's not so much that the tracing becomes _better_ , but that
           | it becomes feasible at all. Two specific situations come to
           | mind, both MPI-adjacent. (1) Running a PMPI-based tool on
           | code you can't recompile yourself (e.g., you need a Q
           | clearance to see the source, but not to actually execute it--
           | weird I know, but not that uncommon in the DOE labs.); and
           | (2) running multiple PMPI-based tools simultaneously which
           | are composed at runtime via PnMPI.
        
             | gnufx wrote:
             | Exactly, but even if you can rebuild it, you don't want to
             | under most circumstances. In principle with static binaries
             | you can use dyninst, for instance, but in practice you may
             | not be able to for various reasons. Then as a system
             | manager you'd like to have global profiling of what runs --
             | for various reasons, including putting the result for a
             | program in front of a user. If it's all dynamically linked,
             | you can hook in and do that by default, and with various
             | levels of insistence, depending on the hooks.
        
             | ghoward wrote:
             | I addressed these arguments. I think that if you don't have
             | access to the source, it's a security problem.
             | 
             | That includes not having access to the source because of
             | security clearances. As far as I am concerned, your
             | superiors in the DOE are adversaries to you and your
             | machine.
             | 
             | Other people will have different viewpoints on that, and
             | that's fine.
        
               | gnufx wrote:
               | I want free software, and I'm aghast at what some people
               | run, but this is not the real world. How many examples do
               | you want? If you talk security, what's the threat model?
               | What I run on a decently managed (with the aid of
               | dynamically linked libraries) compute cluster should only
               | put my data at risk.
               | 
               | As I understood it, there actually has been a push for
               | free software solutions on CORAL systems, but I don't
               | remember where that came from.
        
           | gnufx wrote:
           | Drepper's scheme obviously works tolerably well with glibc,
           | in particular, even if he didn't know what he was talking
           | about.
           | 
           | If you don't have symbol versioning, and you want to
           | statically link something with parts that depend on
           | incompatible versions of a library, you're obviously stuffed.
        
       | [deleted]
        
       | arduinomancer wrote:
       | It seems a pain point of static linking is that if a library
       | changes you need to rebuild everyone
       | 
       | Maybe a stupid idea but has there ever been thought of
       | distributing applications with source code included?
       | 
       | That way you could have a system that automatically rebuilds the
       | application if a static library changes.
       | 
       | Meaning when a user sees "updating" it's actually rebuilding the
       | app underneath.
        
       | ChrisSD wrote:
       | > However, there is one big elephant in the room: right now,
       | there are a lot of platforms that do not fully support static
       | linking because you have to dynamically link libc or other system
       | interfaces. The only one I know of that supports static linking
       | is Linux.
       | 
       | > The ironic thing is that operating systems only supported a
       | stable syscall ABI, rather than requiring programs to dynamically
       | link system libraries, they would have less problems with ABI
       | breaks.
       | 
       | > With a statically-linked binary, you can copy it to another
       | machine with a Linux that supports all of the needed syscalls,
       | and it will just run.
       | 
       | I think this is a misunderstanding. With Linux there is a stable
       | interface at the kernel boundary. Other OSes have their stable
       | interfaces in user space. The difference is mostly irrelevant in
       | this context. In one case you dynamically call into the kernel,
       | in another case you dynamically link a user space library. Either
       | way the code you're calling isn't static but the interface is.
        
         | AshamedCaptain wrote:
         | And to assume that only the syscall ABI layer is important is
         | quite the understatement. I have software from the 90s that
         | would work perfectly fine if it weren't for the fact that
         | /etc/mtab is now a symlink. And that is just an example of the
         | most smallish change -- not going to enter into the bazillions
         | of binaries that are broken due to changes in audio APIs (even
         | though the syscall numbers are still the same).
         | 
         | Static linking for future-proofing/compatibility has never been
         | a valid argument in my experience. Not only it is much easier
         | to fix (and debug) a broken dynamically linked binary than a
         | statically linked one, I would even dare to say that statically
         | linked binaries break much more frequently than dynamically
         | linked ones when you change the rest of the system. Even when
         | you include the breakage from libraries changing under the feet
         | of the dynamic binaries.
        
         | CyberShadow wrote:
         | There is a distinction between a fully-static binary and a non-
         | fully-static one, which is what I think the article refers to.
         | A fully static binary will not need to call the dynamic linker
         | at all, and tools such as "file" or "ldd" will identify such
         | binaries differently. A fully static binary will also run even
         | in the complete absence of any userspace support for its
         | architecture (assuming the hardware and kernel do support it) -
         | e.g. a static AArch64 binary will run on a 32-bit ARM
         | distribution, if the CPU supports AArch64.
        
           | [deleted]
        
           | ghoward wrote:
           | Author here.
           | 
           | This is exactly what I was getting at. Thank you.
        
           | ChrisSD wrote:
           | True, however...
           | 
           | > assuming the hardware and kernel do support it
           | 
           | Assuming kernel support is doing a lot of heavy lifting
           | there. To put it another way, why is a syscall better than
           | any other stable ABI? If both kernel and stable library are
           | distributed together then why _should_ it be considered
           | different?
        
             | garethrowlands wrote:
             | Indeed, Linux is the outlier here. Maybe it's better. But
             | the designers of, say, Solaris, Mac and Windows didn't
             | think so.
        
               | SAI_Peregrinus wrote:
               | Even Linux isn't really the outlier. The syscall
               | convention is stable, but you're not statically linking
               | the kernel into your binary.
               | 
               | A "library OS" like FreeRTOS does exactly that: the
               | kernel is just another library, with functions that you
               | call like any other static dependency. You can only run
               | one process (the OS & your userspace code are that
               | process).
               | 
               | Really the "fully static" side is only seen in practice
               | in RTOSes and similar embedded systems work. Being able
               | to dynamically load more than a single process is just
               | too handy to give up entirely. I don't think the opposite
               | extreme has even been tried (every single symbol in its
               | own .so, even within the kernel), the overhead would be
               | ridiculous.
        
             | ghoward wrote:
             | Author here.
             | 
             | Kernel support is actually easy: since Linux hardly ever
             | removes syscalls, just build your fully-static executable
             | on the oldest Linux you have, and then deploy it on all of
             | your relevant machines.
             | 
             | The syscalls used by all of your libraries, if they work on
             | that oldest Linux, would work on the newer ones.
             | 
             | In fact, this is why AppImage "Best Practices" includes
             | building on the oldest system. [1]
             | 
             | [1]: https://docs.appimage.org/reference/best-
             | practices.html?high...
        
               | gnufx wrote:
               | If you build against, say, RHEL5, you presumably acquire
               | vulnerabilities in relevant libraries, give up hardware
               | support, and still can't guarantee it will run correctly.
               | That's at least because Linux interfaces aren't stable in
               | general, specifically the pseudo-filesystem ones,
               | thinking of real examples.
        
             | Brian_K_White wrote:
             | If a kernel and library really are always distributed
             | together, so religiously iron clan that you can and should
             | treat them as a single object, then why are they not in
             | fact a single object?
             | 
             | A syscall may not be inherently too different from any
             | other interface, but a single interface is certainly
             | different from two interfaces.
        
         | PaulDavisThe1st wrote:
         | The word "static" is being conflated with "stable".
         | 
         | A "static interface" isn't a thing. There's stable and unstable
         | interfaces, and static and dynamic linkage. Not strongly
         | related.
        
         | amelius wrote:
         | I'm distributing my code as a bootable USB drive.
        
       | 0xbadcafebee wrote:
       | I wish I had three hours to waste on why almost all of these
       | points are either innacurate or willfully misleading. But I
       | don't, so I'll just point out the single biggest reason why
       | static linking is harmful.
       | 
       | Encryption libraries.
       | 
       | If OpenSSL, or LibreSSL, or Go's encryption modules, or _any
       | gigantic encryption library_ has a vulnerability, you basically
       | have to recompile, distribute, and then have your users download,
       | every single god damn networking program.
       | 
       | But it's worse than that. Because everyone wants to ship a
       | statically compiled Go app, nobody packages for the distros. So
       | now every user needs to go track down where they downloaded each
       | of their static Go apps, _safely_ download the update, verify its
       | checksum or signature via a 3rd party, and upgrade the app.
       | 
       | That is a logistical nightmare. Not only are apps going to remain
       | vulnerable for way, way longer, more users will be compromised by
       | phishing and other attacks because there's no package manager to
       | safely automate all these updates (distros sign and verify
       | packages for you).
       | 
       | I've actually written a Shell quasi-package manager/installer
       | just for statically compiled apps (cliv) so in theory that
       | problem could be somewhat solved... But nobody even knows about
       | that program, so the manual update problem still stands.
        
         | AshamedCaptain wrote:
         | If you ever statically link OpenSSL, your binary now has a
         | builtin expiration date. You are doing your users a disservice.
        
           | marcosdumay wrote:
           | To be fair, if you dynamically link to OpenSSL, your binary
           | has a builtin expiration date too, and it's very likely the
           | same date.
        
             | AshamedCaptain wrote:
             | Not really. I have made updated versions of libSSL that
             | allow use of new ciphers with the old ABI, for example.
             | (Retro purposes). But for example this has been done to
             | extend the life of other binary-mostly platforms (e.g.
             | webOS). As long as I can change libSSL separately, this is
             | trivial.
        
             | nly wrote:
             | OpenSSL ABI has been fairly stable within a given series.
             | 
             | Something compiled against 1.1.0a (Sep 2016) has a fair
             | chance of running on a system that has 1.1.0l (Aug 2021). 5
             | years of binary compatibility is nothing to be sniffed at
             | for a C library.
             | 
             | https://abi-
             | laboratory.pro/index.php?view=timeline&l=openssl
        
         | KronisLV wrote:
         | > If OpenSSL, or LibreSSL, or Go's encryption modules, or any
         | gigantic encryption library has a vulnerability, you basically
         | have to recompile, distribute, and then have your users
         | download, every single god damn networking program.
         | 
         | Good, then you'd know that these 50 of your networking apps
         | have each been tested by their developers' test suites and have
         | no bugs that have been introduced as a part of this dependency.
         | Essentially, there would be a server farm out there that'd run
         | all of the tests so you wouldn't have to deal with prod
         | breaking because of some new dependency version not taking
         | every possible configuration into account.
         | 
         | Of course, that implies:                 - that there are
         | automated builds in place       - that there are automated
         | tests in place       - that both of the above are run
         | regularly, on every dependency update       - that both of the
         | above are actually meaningful (think close to 100%
         | functionality coverage in tests)
         | 
         | The current situation of relying upon dynamically linked code
         | is only prevalent because we as an industry have decided not to
         | even attempt to do the above. I'd say that it's a matter of not
         | being willing to put in effort, not being willing to test our
         | software and packages, and not being willing to develop
         | software more slowly but thoroughly. Everyone wants "good
         | enough" now, rather than "almost perfect" in 10 years,
         | regardless of whether we're talking about developing software,
         | or the tooling around it.
         | 
         | > But it's worse than that. Because everyone wants to ship a
         | statically compiled Go app, nobody packages for the distros.
         | 
         | In my eyes, that just adds to the above - there are so many
         | deployment targets out there, all of these different packaging
         | systems that are used by different distros and all have
         | numerous idiosyncrasies as opposed to a single good package
         | manager, that people have been fed up with it and now just
         | either ship Docker containers or expect you to download
         | binaries.
        
         | ghoward wrote:
         | Author here.
         | 
         | I think you are strawmaning my arguments.
         | 
         | Personally, I would include encryption libraries in the "system
         | libraries" (I used that term on purpose) that could be
         | dynamically linked.
         | 
         | However, even if they are, you _still_ want to recompile every
         | user whenever that library is updated because of possible ABI
         | and API breaks. And some of those encryption libraries have a
         | history of API breaks.
         | 
         | Also, I am building the ideas I laid out in the post, which
         | should make it easier to handle dynamically-linked _and_
         | statically-linked encryption libraries.
        
         | chrisseaton wrote:
         | If you just update an encryption library and ship only that...
         | then who's tested all the applications against the new version?
        
           | salawat wrote:
           | Not the library author's problem. That's the consumer's
           | problem.
           | 
           | I'm not saying it's great, but it's how it works.
        
             | chrisseaton wrote:
             | > That's the consumer's problem.
             | 
             | Well this is the explanation for why people are static
             | linking! They don't want to run someone else's entirely
             | untested code and bear all responsibility for any problems.
        
               | salawat wrote:
               | For the record? I'm totally fine with that. As I'm
               | basically one of the static linking group, or "use
               | dynamic libraries in a way non-distinguishable from
               | static linking".
               | 
               | If I'm relying on it, it's getting tested and I'm not
               | going to dork with it without good reason once it is.
        
       | Wowfunhappy wrote:
       | Are there any Linux distros that actually distribute (most)
       | software as (mostly) static binaries? Right now, the only way to
       | do it seems to be compiling everything myself...
        
         | hhvn wrote:
         | https://sta.li/ Experimental, probably not that usable.
         | 
         | http://sabo.xyz/ More of a complete distro, but the dev himself
         | says he still uses a couple of programs dynamically linked.
        
         | ObscureScience wrote:
         | I think there are only experiments. From what I know Oasis
         | Linux is furthest along https://github.com/oasislinux/oasis
         | 
         | Another resources is https://sta.li/ I thought there was a
         | Archlinux-based attempt, but can't find references to that.
        
         | lrem wrote:
         | Why would you want that?
        
           | Wowfunhappy wrote:
           | Simplicity. If everything is statically linked, then updating
           | package A can never break package B.
           | 
           | As I see it, the entire world of Docker containers exists
           | primarily to solve the problem that static linking already
           | solved.
           | 
           | Something like Nix would work as well, and I think I would
           | love Nix if I knew how to use it. But I don't, and I don't
           | really have the time to learn.
        
             | lrem wrote:
             | I don't think that makes a lot of sense in a distro. If
             | package A gets an update, it means a sufficiently bad bug
             | was discovered in it. If it happens to be a library, you
             | want to make sure you update every package B that uses it.
             | Which, conveniently, the distro solves for you too.
        
               | Wowfunhappy wrote:
               | This is true, but only if I'm actually getting
               | _everything_ from the distro repository and I 'm able to
               | keep all of it on the latest version. In practice, I
               | sadly tend to run into situations where I need stuff from
               | other places.
        
               | lrem wrote:
               | But then you don't want the software from the distro to
               | be statically linked, but the software from other places.
               | In which case it doesn't matter that much what does the
               | distro do.
        
               | Wowfunhappy wrote:
               | Basically, the distros aren't perfect and I sometimes run
               | into conflicts. Also, with static binaries it would be
               | much easier to pin specific software at old versions
               | without affecting anything else, even just temporarily
               | for testing.
               | 
               | IMO, when stuff is self-contained, everything becomes
               | easier! This isn't to say some stuff like OpenGL can't be
               | dynamically linked, but some day I'd love to use a distro
               | with a static-first approach, because right now it simply
               | isn't an option.
        
             | Brian_K_White wrote:
             | AppImage, Flatpack, Snaps... all seem to be largely
             | reinventing the primary qualities, good and bad, of static
             | binaries in a new and much more complicated form.
             | 
             | At least a container provides more function than mere
             | bundling, but a lot of containers actually get used merely
             | for bundling.
        
           | blablabla123 wrote:
           | I think programs in /bin used to be usually statically
           | linked. When updating the glibc for instance, this makes the
           | change far more doable. In general dealing with static
           | binaries is much less pain, especially when they haven't been
           | compiled locally/by the distributor for this particular
           | version of the distribution.
        
             | drivebycomment wrote:
             | /sbin used to be statically linked on unix. The entire
             | point of /sbin is the independence from almost everything
             | else so that you can rely on them even when the system
             | boots up in a weird state.
        
       | ghoward wrote:
       | Author here. I don't know why my own submission of this [1]
       | didn't catch and this one did. Oh well. XD
       | 
       | I'm open to answering questions, and I'll be hanging in the
       | thread.
       | 
       | [1]: https://news.ycombinator.com/item?id=28728801
        
         | comex wrote:
         | I am extremely skeptical of this calculation by Drew DeVault:
         | 
         | > On average, dynamically linked executables use only 4.6% of
         | the symbols on offer from their dependencies. A good linker
         | will remove unused symbols.
         | 
         | Percent of exported symbols used is a _terrible_ proxy for
         | percent of library functionality used.
         | 
         | As an example, an SDL hello world program [1] uses 3 functions
         | out of 750 exported by libSDL2. Does that mean it only uses
         | 3/750 = 0.4% of libSDL2's functionality? Of course not; when I
         | tried compiling it against a static libSDL2 instead of dynamic,
         | the executable's binary size increased by a full 70% of the
         | size of libSDL2.so. [2]
         | 
         | Now, on one hand I admit that's an extreme example, as a real
         | SDL program would use a higher number of functions. And SDL
         | itself is not very amenable to dead code stripping; a good
         | chunk of the hello-world program is for things like sound or
         | input that the it doesn't actually use.
         | 
         | Still, even the hello-world program legitimately needs a bunch
         | of functionality, including support for both X11 and Wayland,
         | and setting up hardware accelerated surfaces using OpenGL. And
         | a real program would need more. Also, I'm not even counting the
         | long list of shared libraries linked by SDL (ranging from
         | libX11 to libFLAC); a SDL-based program will use 0% of their
         | symbols (since it doesn't them directly), but may need a
         | substantial portion of their functionality.
         | 
         | More importantly, this general pattern, where exported APIs are
         | just the tip of the iceberg, applies to many libraries.
         | 
         | I tried a similar experiment with zpipe, the official example
         | program for zlib - a much smaller and shallower library. It
         | ended up using 23% of symbols but 53% of the implementation.
         | 
         | On the other end of the spectrum, the problem would be far
         | worse with a real GUI library, where even, say, a simple text
         | input box truly requires massive amounts of functionality
         | including display, layout, Unicode font rendering, menus,
         | internationalization, and keyboard and mouse input. Plus, of
         | course, a variety of input method editors if you want to
         | support languages like Japanese.
         | 
         | In a statically linked world you would probably want to
         | implement most of that as IPC to shared daemons. Such a design
         | could work; there isn't much that really needs the performance
         | benefits of being in-process. But no major GUI libraries are
         | written that way, because in a world with shared libraries
         | there's no need.
         | 
         | [1]
         | https://gist.github.com/comex/c64b5a7e409d5d48ad6ebf37ec068b...
         | 
         | [2] Test harness also at [1]. This is with LTO; without LTO the
         | percentage is much higher. I also had to disable SDL's "dynapi"
         | feature that makes static linking secretly actually dynamically
         | link. Which they put in for good reasons, albeit mostly
         | applicable only to proprietary software where the user can't
         | just recompile/relink.
        
       | lgg wrote:
       | Every time I read one of these essays extolling the virtues of
       | static linking it makes me pretty sad. Engineering is about trade
       | offs, and sometimes it does make sense to statically link, but
       | the fact that those trade offs are so lopsided on Linux has very
       | little to do with dynamic linking vs static linking as generic
       | concepts, and more to do with the fact that the design of ld.so
       | is straight out of the 1990s and almost nothing has be done to
       | either exploit the benefits dynamic linking brings, nor to
       | mitigate the issues it causes.
       | 
       | On Darwin derived systems (macOS, iOS, tvOS, watchOS) we have
       | invested heavily in dynamic linking over the last two decades.
       | That includes features we use to improve binary compatibility
       | (things like two level namespaces (aka Direct Binding in ELF) and
       | umbrella frameworks), middleware distribution (through techniques
       | like bundling resources with their dylibs into frameworks), and
       | mitigate the security issues (through technologies like
       | __DATA_CONST).
       | 
       | Meanwhile, we also substantially reduced the cost of dynamic
       | linking through things like the dyld shared cache (which makes
       | dynamic linking of most system frameworks almost free, and in
       | practice it often reduces their startup cost to below the startup
       | cost of statically linking them once you include the cost of
       | rebasing for PIE), and mitigate much of the rest of the cost of
       | dynamic linking through things like pre-calculated launch
       | closures. It does not hurt that my team owns both the static and
       | dynamic linkers. We have a tight design loop so that when we come
       | up with ideas for how to change libraries to make the dynamic
       | linker load them faster we have the ability to rapidly deploy
       | those changes through the ecosystem.
       | 
       | As I explained in here[1] dynamic linking on macOS is way faster
       | than on Linux because we do it completely differently, and
       | because of that it is used way more pervasively on our systems (a
       | typical command line binary loads 80-150 dylibs, a GUI app around
       | 300-800 depending on which of our OS you are talking about). IOW,
       | by mitigating the costs we drove up the adoption which amplifies
       | the benefits.
       | 
       | And it is not like we have squeezed all the performance out of
       | the dynamic linker that we can, not by a long shot. We have more
       | ideas than we have time to pursue, as well as additional
       | improvements to static linking. If you have any interest in
       | either static or dynamic linking we're looking for people to work
       | on both[2].
       | 
       | [1]:
       | https://www.realworldtech.com/forum/?threadid=197081&curpost...
       | 
       | [2]: https://jobs.apple.com/en-us/details/200235669/systems-
       | engin...
       | 
       | (edit: fixed link formatting)
        
         | ghoward wrote:
         | Author here.
         | 
         | I'm curious: did you read to the end of the post where I lay
         | out ideas about how to get the benefits of both in one?
        
           | lgg wrote:
           | Yes. They are all reasonable ideas, and we actually have
           | experience with most (all?) of them:
           | 
           | * Distributing IR applications: We do a form of this by
           | supporting bitcode for App Store submissions on iOS, tvOS,
           | and watchOS. For watchOS it was a great success in the sense
           | that it allowed us to transparently migrate all the binaries
           | from armv7k (a 32 bit ABI on a 32 bit instruction set) to
           | arm64_32 (a 32 bit on a 64 bit instruction set), but we had
           | to carefully design both ABIs in parallel in order to allow
           | that to be efficient. It also introduces serious burdens on
           | developer workflows like crash reporting. Those probably are
           | not significant issues for people deploying binaries to
           | servers, but it can be pretty difficult for developers trying
           | to aggregate crash statistics from apps deployed to consumer
           | devices. It also causes security issues with code provenance
           | since you have to accept locally signed code, have the
           | transforms performed by a centrally trusted entity who signs
           | it, or you have to limit to your optimizations to things you
           | can verify through a provable chain back to the IR.
           | 
           | * Split stacks: We don't do this per se, but we do something
           | semantically equivalent. When we designed the ABI from arm64e
           | we used PAC (pointer authentication codes) to sign the return
           | addresses on the stack, which means that while you can smash
           | the stack and overwrite the return pointer, the
           | authentication code won't match any more and you will crash.
           | 
           | * Pre-mapped libraries: We have done this since then Mac OS X
           | Developer Releases back in the later 90s, though the
           | mechanism has changed a number of times over the years.
           | Originally we manually pre-assigned the address ranges of
           | every dylib in the system (and in fact we carefully placed
           | large gaps between the __TEXT and __DATA of the library and
           | have the libraries in overlapping ranges so that we could
           | place all the system __TEXT in single adjacent region and all
           | the __DATA in a second adjacent region that way we could
           | exploit the batch address translation registers on PPC
           | processors to avoid polluting the TLBs). When the static
           | linker built a dylib we look in a file with the mappings and
           | build the dylib with segment base addresses that we looked up
           | from the that manually maintained list, and when the OS
           | booted we would pre-map all the libraries into their slots so
           | every process just had them mapped.
           | 
           | That was a huge pain in the neck to maintain, since it meant
           | whenever a library grew too large it was would break the
           | optimizations and someone would need update the file and
           | rebuild the OS. It also only dealt with rebasing, to deal
           | with binding we locally ran update_prebinding which users
           | hated because it was slow, sysadmins hated because it means
           | we routinely rewrote every binary on the system. That was
           | also before anyone had deployed ASLR or codesigning.
           | 
           | Nowadays we use the dyld shared cache to achieve similar
           | ends, but it is a far more flexible mechanism. We essentially
           | merge almost all the system dynamic libraries at OS build
           | time into one mega-dylib that we can pre-bind together and
           | sign. We also use VM tricks to allow the system to rebase it
           | on page in rather than the dynamic linker doing any work, and
           | we pre-map it into every address space by default.
           | 
           | We even perform a number of additional optimizations when we
           | build it, such as analyzing the dependencies of every
           | executable in the base system and pre-calculating a lot of
           | the work dyld and even the dynamic runtimes like ObjC would
           | have to do in order to avoid doing them during app launch.
           | 
           | So in short, I think your ideas there have more merit than
           | you perhaps suspect, and from experience I can say if/when
           | you implement them you implement them you might find your
           | views on the trade offs of dynamic vs static linking change.
        
             | ghoward wrote:
             | Thank you.
             | 
             | And after reading your response, I might give up on at
             | least one of those ideas.
        
       | osigurdson wrote:
       | I really dislike the tired re-use of the term "considered
       | harmful". A better title may be "I don't like dynamic linking -
       | here's why".
        
       | Jach wrote:
       | I'd like to see solutions like SDL2's become more widespread:
       | https://old.reddit.com/r/linux_gaming/comments/1upn39/sdl2_a...
       | At runtime, the first call into SDL sets up a jump table for all
       | SDL functions afterwards. When SDL is statically linked, it uses
       | those static functions by default, but it's possible to specify
       | an environmental variable pointing at an SDL dynamic lib, and the
       | jump table will use those functions instead.
        
       | dhsysusbsjsi wrote:
       | Personal end user perspective: troubleshooting dylib errors is
       | the most common cause of wanting to throw the computer out the
       | window and quit programming.
        
         | cratermoon wrote:
         | See also: DLL Hell in Windows and, more generally Dependency
         | Hell. Dynamic linking errors are a subset of general dependency
         | problems.
        
           | spacechild1 wrote:
           | As others have already mentioned elsewhere, DLL hell hasn't
           | really been a thing anymore on Windows for years. I
           | personally can remember it, but that's more than 15 years
           | ago.
        
         | eqvinox wrote:
         | "end user perspective" + "quit programming"? Do you
         | troubleshoot dylib errors on your regular end user software and
         | just happen to be a programmer unrelated to that?
         | 
         | (If you're troubleshooting dylib errors on things you're
         | developing, that's not an end user perspective?)
        
           | salawat wrote:
           | Yes. If you've never had a clashing Debian package build
           | before, you've not been paying attention. I've made it my
           | mission in life to try to get people more aware of linkers,
           | and the fact that when you build something the one way it
           | works on your system it doesn't mean it will work that way on
           | someone else's.
        
             | eqvinox wrote:
             | I've never had a clashing Debian package build before, and
             | I build quite a few Debian packages. Some for a wider
             | audience and some for my own use.
             | 
             | All packages that leave the confines of my own systems are
             | built the "correct" way, in a clean minimal chroot with
             | sbuild/schroot/cowbuilder. The docs on how to set that up
             | are pretty good, and after you've done it once it's pretty
             | easy to repeat for extra target distros (e.g. older Debian,
             | Ubuntu.)
             | 
             | What would you point out to me as part of your mission?
             | 
             | [Just to be clear, a package is a package for a particular
             | target distro. There's no such thing as "a .deb for
             | software XYZ". Only "a buster .deb for software XYZ", or "a
             | bullseye .deb for software XYZ", etc.]
        
           | dizzy3gg wrote:
           | End user of a linker?
        
             | rewma wrote:
             | > End user of a linker?
             | 
             | I'd argue that if you're not packaging your software or
             | testing your software's dependencies, either you're doing
             | something extremely exotic that lies far outside anyone's
             | happy path or "dylib error" should not even be a keyword in
             | your vocabulary.
        
               | alisonkisk wrote:
               | The windows version, DLL Hell, is well known to non
               | programmers.
        
               | pjc50 wrote:
               | Roughly the same thing happens on Windows and is called
               | "DLL hell", occasionally witnessed by end users.
        
               | rewma wrote:
               | DLL Hell ceased to be a practical concern over a decade
               | ago, particularly given that Windows provides tight
               | control over its dynamic linking search order.
               | 
               | https://docs.microsoft.com/en-
               | us/windows/win32/dlls/dynamic-...
               | 
               | DLL Hell is not a linking problem, it's a packaging
               | problem.
        
           | gbear605 wrote:
           | There's a middle level of "troubleshooting dylib errors on
           | open source software I'm installing for personal use because
           | I'm a masochist who isn't happy with closed source software"
        
             | eqvinox wrote:
             | Not sure that's a solvable problem, or even a problem that
             | should be solved. It's the distributions' job to create
             | packages to build a working whole system. If you decide to
             | take that on yourself... I mean... yeah, you gotta learn
             | how to handle DSO issues. Because you decided you want to
             | do it yourself.
             | 
             | I don't think I've ever run into DSO problems with any
             | reasonable distro packaging (which, in this case, excludes
             | old Gentoo -- current Gentoo doesn't have those problems
             | anymore either.)
        
               | AnIdiotOnTheNet wrote:
               | But if the distro doesn't package software you want to
               | use, either because it only has an out of date version,
               | modified version, or was just never packaged, then you're
               | left dealing with this nonsense.
               | 
               | In saner OS's where they don't rely on unpaid third party
               | middlemen between the developer and the user this is
               | rarely a problem.
        
               | cratermoon wrote:
               | > In saner OS's where they don't rely on unpaid third
               | party middlemen between the developer and the user this
               | is rarely a problem.
               | 
               | Windows seems to fit your criteria for an OS without
               | unpaid third parties, and there still exists a thing
               | known as DLL Hell.
        
               | spacechild1 wrote:
               | I haven't really encountered DLL hell in Windows for the
               | last 15 years or so. Most apps simply bundle all their
               | dependencies locally and don't install them system wide.
        
       | KronisLV wrote:
       | Why are we still developing software like we used to 40 years
       | ago?
       | 
       | Why not just import external dependencies into your project at a
       | function/class level rather than at a package one: if you only
       | use FooPackage.BarClass and FooPackage.bazMethod(), you should be
       | able to choose to make your project depend on just those two
       | things, ideally in a completely transparent way by your IDE.
       | 
       | Then having to manage the full scope of packages and their
       | versions becomes less relevant, because the IDE can check whether
       | a new version has been released in the package manager of choice,
       | check whether those two things have changed and if they have,
       | then demand that the developer have a look at the code changes to
       | refactor code if necessary, otherwise not requiring any action on
       | their part. Furthermore, if you ever need to migrate to a
       | different package or even rewrite the functionality itself, you
       | could just look at the few signatures that you use, rather than
       | having to remove the package entirely and see what breaks in your
       | IDE. Why can't we have itemized lists of everything that's called
       | and everything that we depend on, as opposed to the abstraction
       | of entire packages?
       | 
       | Better yet, why even depend on binary blobs or hidden code
       | (that's ignored by your IDE) in the first place? Why not just
       | download the source for every package that you use and upon
       | updates be able to review the code changes to the package much
       | like a regular Git diff?
       | 
       | Of course, this would probably require getting rid of reflection
       | and other forms of dynamic code, which i fully support, since
       | those have never been good for much in the first place and only
       | destroy any hopes of determinism and full control flow analysis.
       | 
       | As for the possible counterargument of this being hard to do: it
       | wouldn't really be with more granular packages. Instead of trying
       | to shove an ecosystem inside of a single package, why not split
       | it into 10-20 bits of reusable code instead? Smaller packages,
       | which would be easier to review and manage.
       | 
       | Context: i dislike how Spring and Spring Boot in Java force a
       | huge ecosystem of fragile dependencies upon you, with their
       | reflection which ensures that your apps will break at runtime,
       | for example, when moving from Spring Boot 1.5 to 2.0.
       | Furthermore, in the JS world, the node_modules folders are
       | probably 10-1000x too large for what's actually necessary to
       | display a webpage with some interactive behaviour.
       | 
       | Disclaimer: i have no delusions about any of the above being
       | technically feasible right now. Perhaps "Why?" would be a good
       | question to ask. In my eyes, perhaps the industry is too set in
       | its current ways and as long as we don't have languages and
       | entire ecosystems that approach established practices from a
       | wildly different angle, no one can actually contest the decades
       | of already built packages and tools.
       | 
       | On the bright side, WireGuard kind of displaced OpenVPN somewhat
       | and there are numerous benefits to it being smaller, which is a
       | good example of getting rid of bloated software. Furthermore, Nix
       | may or may not do that to alternative approaches to package
       | management, but its usage still remains really low.
       | 
       | In my eyes, the only long term solution is to be able to tell
       | exactly what's necessary for your code to build and work, and to
       | test every dependency update that comes out against this
       | automated process. Static dependencies but at the speed of
       | updates of dynamic dependencies. Then you'd just have to wait for
       | a day until the devs fix the newest regressions of a new
       | dependency release before getting a new statically linked app, as
       | opposed to using dynamic linking and finding that some dependency
       | breaks your app as it happens in prod.
        
         | nemetroid wrote:
         | Joe Armstrong had some similar thoughts: "Why do we need
         | modules at all?".
         | 
         | https://erlang.org/pipermail/erlang-questions/2011-May/05876...
        
           | KronisLV wrote:
           | You know, the "all functions have unique distinct names" idea
           | is pretty interesting!
           | 
           | Honestly, that reminds me of the PHP standard library a bit,
           | which feels oriented towards procedural programming at times.
           | Now, the language and the actual implementation of the
           | functions aside, i found that pattern really pleasant to work
           | with, especially since it let me work more in line with
           | functional programming principles as well - pure functions
           | and no global state (even if passing data around was more
           | difficult).
           | 
           | Now, i won't say that the approach fits every project or
           | domain out there, but at the very least having a clear idea
           | of what your code depends on and what's going on inside of
           | your code base is an idea that i still stand by. Sadly, the
           | only tool that i found that at least tries to make this
           | easier was Sourcetrail, which is essentially dead now:
           | https://www.sourcetrail.com/
        
         | flohofwoe wrote:
         | > if you only use FooPackage.BarClass and
         | FooPackage.bazMethod()...
         | 
         | That's how it works already with static linking (just under the
         | hood), the linker will discard unused code and data (with LTO
         | this happens automatically for all dead code and data,
         | otherwise some special compiler/linker/librarian-options may be
         | needed).
         | 
         | A DLL on the other hand cannot predict what API functions will
         | actually be called on it, so everything must be included.
        
         | Brian_K_White wrote:
         | As an end user, I would LOVE to be able to run a calculator app
         | without having to pull in 500M of kde libraries and even
         | daemons.
         | 
         | But this is pretty much what a static binary already does, did,
         | 40 years ago. The developer has access to the full universe,
         | but the only thing that ends up in the binary are the bits they
         | actually used. Problem solved, 40 years ago.
        
         | PaulDavisThe1st wrote:
         | > Why not just import external dependencies into your project
         | at a function/class level rather than at a package one: if you
         | only use FooPackage.BarClass and FooPackage.bazMethod(), you
         | should be able to choose to make your project depend on just
         | those two things, ideally in a completely transparent way by
         | your IDE.
         | 
         | TFA is about programs that require a linker at either compile
         | and/or run time. The references in the rest of your comment to
         | Java and JS suggest that your experience with programming in
         | languages that require linking against compiled libraries might
         | be limited.
         | 
         | What you describe is _precisely_ what the linker is for! It
         | looks at the code you wrote (and compiled), sees that it calls
         | FooPackage.bazMethod(), and then goes and looks for that call
         | along the linker search path using a combination of implicit
         | and explicit library names. If it 's the static linker, it
         | imports the code for that call, and then recurses to check the
         | dependencies of FooPackage.bazMethod(). If it's the dynamic
         | linker, it does the same, except differently (runtime pointer
         | swizzling etc. etc)
         | 
         | When you link your program against libfootastic.a, the linker
         | is doing exactly what you suggest, and in most IDEs that step
         | is going to happen automatically. When you link your program
         | against libfootastic.so, the linker will do exactly what you
         | suggest, but at runtime.
         | 
         | > Better yet, why even depend on binary blobs or hidden code
         | (that's ignored by your IDE) in the first place? Why not just
         | download the source for every package that you use and upon
         | updates be able to review the code changes to the package much
         | like a regular Git diff?
         | 
         | Many large end-user desktop applications do precisely this. For
         | Ardour (I'm the lead dev), our build system downloads 80+
         | source code packages and builds them locally to create a 2GB
         | installed dependency tree of compiled libraries and ancillary
         | files. There are a lot of arguments against doing this, and
         | some arguments in favor.
         | 
         | However, we can't compile each file from each dependency as
         | part of the Ardour build because the build systems for those
         | dependencies are complex and independent. There's no way to
         | force the developers of libfftw (Fastest Fourier Transform in
         | the West) to somehow prepare their code structures so that we
         | can trivially replicate their entire build system as part of
         | ours. If there was only a single build system for EVERYTHING,
         | sure, maybe this would be feasible. But there isn't, and I'm
         | completely certain that there never will be.
        
           | ensiferum wrote:
           | Yep wrangling the dependencies really becomes a tremendous
           | pain at scale. Chromium for example does the same thing and
           | they have enough engineering backing to be able to throw
           | enough manpower into it so that that can get each and every
           | single dependency in the same build set using the same build
           | files and tools (gyp/gn/ninja). While feasible for an org
           | such as Google this is totally unfeasible for a small shop
           | with Limited engineering man hours. Really wish there was an
           | easier way to deal with all of this. Sigh
        
       | 3np wrote:
       | ""Static Linking Considered Harmful" Considered Harmful"
       | Considered Harmful:
       | 
       | This is the blog version of Re: Re: Fwd: Re: in email and has to
       | stop here.
       | 
       | See also https://meyerweb.com/eric/comment/chech.html
        
         | Brian_K_White wrote:
         | Incorrect.
         | 
         | Ok fine, explaination: "Re:" are not art. "Re:" are not added
         | deliberately and for intentional artistic effect, and are not
         | references (despite the literal) or homages.
        
         | ghoward wrote:
         | Author here.
         | 
         | See my reply to https://meyerweb.com/eric/comment/chech.html at
         | https://news.ycombinator.com/item?id=28737279 .
        
       | rascul wrote:
       | According to Linus, "Shared libraries are not a good thing in
       | general".
       | 
       | https://lore.kernel.org/lkml/CAHk-=whs8QZf3YnifdLv57+FhBi5_W...
        
         | ghoward wrote:
         | Author here.
         | 
         | I quote that email twice in my post. I decided not to quote
         | that part of the email just in case it might inflame the
         | debate.
         | 
         | But yes, he did say that.
        
       ___________________________________________________________________
       (page generated 2021-10-03 23:01 UTC)