[HN Gopher] Anatomy of Linux Dynamic Libraries
___________________________________________________________________
Anatomy of Linux Dynamic Libraries
Author : begoon
Score : 101 points
Date : 2022-10-12 10:28 UTC (1 days ago)
(HTM) web link (developer.ibm.com)
(TXT) w3m dump (developer.ibm.com)
| matheusmoreira wrote:
| Very useful article! I wish I had read this a long time ago,
| would've made a lot of things much easier to understand.
|
| I'd like to comment on this:
|
| > When the relocations are complete, the dynamic linker allows
| any loaded shared object to execute optional initialization code.
| This functionality allows the library to initialize internal data
| and prepare for use.
|
| > This code is defined in the .init section of the ELF image.
| When the library is unloaded, it may also call a termination
| function (defined as the .fini section in the image).
|
| There are huge problems with this stuff. Are these sections still
| used? I know you can add functions to these sections with a GCC
| attribute but I don't know why anyone would do that. Do other
| languages like C++ use these sections?
|
| https://blogs.oracle.com/solaris/post/init-and-fini-processi...
| monocasa wrote:
| Yes they're still used. C++ heavily uses them as you've
| guessed. Off the top of my head QEMU also uses them as a way to
| register all the machine variants it supports internally
| without having big per arch tables in source that everyone
| needs to constantly have merge conflicts with.
| cryptonector wrote:
| The Solaris Linker Guide is still the best document written on
| ELF and dynamic linking. I highly recommend spending some time
| with it.
| akshaykumar90 wrote:
| Can you share a link if available?
| cryptonector wrote:
| https://docs.oracle.com/cd/E53394_01/html/E54813/index.html
| gnufx wrote:
| Outside C++, for instance, constructor functions are useful in
| LD_PRELOAD objects to fix up things before a binary's entry
| point runs.
| gnufx wrote:
| Concerning ldd, see the security warning in ldd(1). I'm not sure
| if it's still an issue in current GNU binutils, but lddtree is
| supposed to be safe and the tree may be more useful.
| chrsig wrote:
| This is a good primer on the subject.
|
| One of the major projects I maintain at $work is an apache
| module, which has given me so much grief because of the fact that
| it gets dynamically loaded (and then unloaded, and then reloaded)
|
| It's been very enlightening, but there's no shortage of
| frustration.
|
| One major points is that a dynamically loaded library can have
| undefined symbols that are found (or not!) at runtime. This can
| introduce a lot of trouble when trying to make a test binary.
| Especially in apache's case, since any of the `ap_` symbols are
| only compiled into the executable, and aren't available as a
| static or shared library.
|
| In that situation, the only recourse that I'm aware of is to
| supply your own implementations of those functions that get
| compiled into the test binary ( _not_ the dynamic library under
| test, since that will create a collision when it 's loaded)
|
| Another thing to consider is that linux/elf shared libraries can
| be either dynamically loaded or dynamically linked. This is not
| true on all platforms, so it's not advisable to lean into it if
| portability matters.
|
| I recommend reading the ld.so manpage[0] -- in particular it
| outlines many environment variables to control the runtime
| linker. Notably, LD_DEBUG can be a life saver.
|
| Lastly, I'll also warn against using any global/static memory in
| a dynamically loaded library. It can be unloaded/reloaded, and
| that can create a lot of havoc, especially if a pointer to that
| memory gets saved somewhere that survives the unload, and then is
| accessible after the library gets reloaded.
|
| libprotobuf is a major victim of these types of issues - which is
| why (or a major contributor) to why the libprotobuf-lite library
| exists.
|
| [0] https://man7.org/linux/man-pages/man8/ld.so.8.html
| matheusmoreira wrote:
| > I'll also warn against using any global/static memory in a
| dynamically loaded library
|
| Yes!! I hate it when libraries do this. Libraries should have
| no state at all.
| stevewatson301 wrote:
| This article, along with [1] (HN discussion [2]) is what finally
| made the linking process "click" in my brain.
|
| [1] https://werat.dev/blog/how-wine-works-101/
|
| [2] https://news.ycombinator.com/item?id=33156727
| coley wrote:
| The first time I learned about dynamic loading was while watching
| an episode of Casey Muratori's Handmade Hero - a series where he
| builds a video game, in C, from scratch.
|
| He compiles the game code as a dll and dynamically loads it at
| runtime in the Win32 platform layer code. This way he can keep
| platform code and game code separate and reload the game code at
| runtime if any changes are made. Being new to this technique, I
| was impressed to say the least.
|
| I always assumed the same functionality was available in the
| linux environment, but I hadn't bothered to look it up. Now I
| know.
|
| For those who want to learn how to do this in Win32, here's the
| episode I mentioned above -
| https://www.youtube.com/watch?v=WMSBRk5WG58
| gnufx wrote:
| From the other side of it, there's "How to Write Shared
| Libraries": https://akkadia.org/drepper/dsohowto.pdf
|
| Edit: LD_PRELOAD would have been useful to mention in the
| article. It's a useful mechanism, e.g. for profiling by hooking
| relevant functions, like the PMPI mechanism in the MPI standard.
| doix wrote:
| LD_PRELOAD is also a great way to modify programs you don't
| have the source for. I used to work for a place that use
| Perforce as their source control platform. Perforce has/had a
| CLI tool you could use to open specific GUI tools called
| p4vc[0].
|
| The first time you ran the tool, it'd spawn a webserver in the
| background listening on localhost:7999 (this is now actually
| documented, back then I found out via strace). The problem was
| that we had many people working on the same machine (in
| different X11 servers).
|
| So whoever was first to run the p4vc on a machine would spawn
| the background server. Anyone that tried to use p4vc in the
| future would communicate with a server spawned by someone else.
| Since the authentication didn't match, it would create a login
| prompt (on the DISPLAY of the person that owns the webserver,
| not the person trying to run the command). It was super
| annoying.
|
| I created a tiny .so file that would intercept 'bind' and
| 'connect', check the port to see if it was 7999, and if it was,
| use the uid of the user instead. Otherwise just transparently
| pass along all the calls. Problem fixed!
|
| I felt like a wizard, one of the few times in my career that
| "arcane" knowledge came in handy.
|
| [0]
| https://www.perforce.com/manuals/p4v/Content/P4V/advanced_op...
| gnufx wrote:
| Yes, there's a bunch of common implementations which use it,
| like (Debian packages) eatmydata, fakeroot, datefudge,
| substitute mallocs. (One implementation of) fakeroot spawns a
| demon.
|
| Oh, and if you have something statically linked, so
| LD_PRELOAD is useless, the usual subversion tool is dyninst.
| ghotli wrote:
| I have to care about this sort of thing at great length in my
| current work. Targeting hundreds of linux / embedded
| environments. I picked up a tip or two and in general this is a
| very well written overview of the moving pieces I wish I'd found
| when I first had to dig into this domain
| photochemsyn wrote:
| Very helpful article, clear and informative. Understanding at
| least the basics of how the linker works at a basic level is very
| illuminating, and for embedded programming even more so. Found
| this recently which is a nice complement to this article:
|
| https://microcontrollerslab.com/bare-metal-embedded-systems-...
___________________________________________________________________
(page generated 2022-10-13 23:01 UTC)