[HN Gopher] The Journey Before main()
___________________________________________________________________
The Journey Before main()
Author : amitprasad
Score : 297 points
Date : 2025-10-25 19:33 UTC (1 days ago)
(HTM) web link (amit.prasad.me)
(TXT) w3m dump (amit.prasad.me)
| hagbard_c wrote:
| On the subject of symbols:
|
| > Yeah, that's it. Now, 2308 may be _slightly_ bloated because we
| link against musl instead of glibc, but the point still stands:
| There's a lot of stuff going on behind the scenes here.
|
| Slightly bloated is a slight understatement. The same program
| linked to glibc tops at _36_ symbols in _.symtab_ :
| $ readelf -a hello|grep "'.symtab'" Symbol table
| '.symtab' contains 36 entries:
| amitprasad wrote:
| Ah I should have taken the time to verify; It might also have
| something to do with the way I was compiling / cross-compiling
| for RISC-V!
|
| More generally, I'm not surprised at the symtab bloat from
| statically-linking given the absolute size increase of the
| binary.
| vbezhenar wrote:
| I wonder how many C projects prefer to avoid standard library,
| just invoking Linux syscalls directly. Much more fun to write
| software this way, IMO.
| forrestthewoods wrote:
| You had me with "avoid C standard library" but lost me at
| "incoming Linux syscalls directly".
|
| Windows support is a requirement, and no WSL2 doesn't count.
|
| C standard library is pretty bad and it'd be great if not using
| it was a little easier and more common.
| pmc00 wrote:
| You can do this in Windows too, useful if you want tiny
| executables that use minimum resources.
|
| I wrote this little systemwide mute utility for Windows that
| way, annoying to be missing some parts of the CRT but not
| bad, code here: https://github.com/pablocastro/minimute
| gpm wrote:
| I thought windows had an unstable syscall interface?
| LegionMammal978 wrote:
| It looks like that project does link against the usual
| Windows DLLs, it just doesn't use a static or dynamic C
| runtime.
| pmc00 wrote:
| Windows isn't quite like Linux in that typically apps
| don't make syscalls directly. Maybe you could say what's
| in ntdll is the system call contract, but in practice you
| call the subsystem specific API, typically the Win32 API,
| which is huge compared to the Linux syscall list because
| it includes all sorts of things like UI, COM (!), etc.
|
| The project has some of the properties discussed above
| such as not having a typical main() (or winmain), because
| there's no CRT to call it.
| Dwedit wrote:
| Pretty much yeah.
|
| You have your usual Win32 API functions found in
| libraries like Kernel32, User32, and GDI32, but since
| after Windows XP, those don't actually make system calls.
| The actual system calls are found in NTDLL and Win32U.
| Lots of functions you can import, and they're basically
| one instruction long. Just SYSENTER for the native
| version, or a switch back to 64-bit mode for a WOW64 DLL.
| The names of the function always begin with Nt, like
| NtCreateFile. There's a corresponding Kernel mode call
| that starts with Zw instead, so in Kernel mode you have
| ZwCreateFile.
|
| But the system call numbers used with SYSENTER are indeed
| reordered every time there's a major version change to
| Windows, so you just call into NTDLL or Win32U instead if
| you want to directly make a system call.
| antihero wrote:
| > Windows support is a requirement
|
| Why, exactly?
| AnimalMuppet wrote:
| > Windows support is a requirement...
|
| For what?
|
| There is _some_ software for which Windows support is
| required. There are others for which it is not, and never
| will be. (And for an article about running ELF files on RiscV
| with a Linux OS, the "Windows support" complaint seems a bit
| odd...)
| throwawaysoxjje wrote:
| A requirement from whom? To do what?
| rfl890 wrote:
| You can make CRT-free Win32 programs, read this guide[1] and
| you're all set. I've written a couple CLI utilities which are
| completely CRT-free and weigh just under a few kilobytes.
|
| [1]: https://nullprogram.com/blog/2023/02/15/
| forrestthewoods wrote:
| Great post!
| matheusmoreira wrote:
| _Almost_ freestanding. It still requires you to link
| against kernel32 and use the functions it provides. This is
| because issuing system calls directly to the Windows kernel
| is not supported. The kernel developers reserve the right
| to change things like system call numbers, so they can 't
| be hardcoded into the application.
| forrestthewoods wrote:
| > Almost freestanding. It still requires you to link
| against kernel32
|
| Nitpick: the phrase "link against kernel32" feels like a
| Linux-ism. If you're only calling a few function you need
| to load kernel32.dll and call some functions in it. But
| that's a slightly different operation than linking
| against it. At least how I've always used the term link.
|
| You're not wrong in principle. But Linux and Windows do a
| lot of things differently wrt linking and loading libs.
| (I think Windows does it waaay better but ymmv)
| matheusmoreira wrote:
| Loading means creating a memory image of the library.
| Linking means resolving the symbols to addresses within
| that memory image.
|
| Loading a library and calling some functions from it _is_
| linking. The function pointer you receive _is_ your link
| to the library function.
| 1718627440 wrote:
| Linker is short for Link Loader, so I don't now what your
| definition of linking is, if it doesn't include loading.
| 1718627440 wrote:
| > (I think Windows does it waaay better but ymmv)
|
| Can you elaborate on that?
|
| Btw., I don't want to bash Windows here, I think the
| Windows core OS developers are (one of) the only good
| developers at Microsoft. The NT kernel is widely praised
| for its quality and the actual OS seems to be really
| solid. They just happen to also have lots of shitty
| company sections that release crappy software and bundle
| malware, ads and telemetry with the actual OS.
| rfl890 wrote:
| Kernel32.dll is loaded into all Windows processes by
| default, so you actually can have a valid, working
| Windows binary with 0 entries in the import table. See
| here[1] for a "Hello world" program written as such.
|
| [1]: https://gist.github.com/rfl890/195307136c7216cf243f7
| 594832f4...
| WJW wrote:
| Obviously only a requirement if you intend your software to
| run under windows. But if you don't, why bother. Not all
| software is intended to be distributed to users far and wide.
| Some of it is just for yourself, and some of it will only
| ever run on linux servers.
| forrestthewoods wrote:
| > some of it will only ever run on linux servers.
|
| I've spent quite a lot of time dealing with code that will
| ever run on Linux which did not in fact only ever run on
| Linux!
|
| Obviously for hobby projects anyone can do what they want.
| But adult projects should support Windows imho and consider
| Windows support from the start. Cross-platform is super
| easy unless you choose to make it hard.
| WJW wrote:
| I don't think we are talking about the same type of
| software? The type I was talking about will only ever run
| on Linux because it's a (HTTP-ish) server that will only
| ever run on Linux.
|
| Probably a server that is only ever run by a single
| company on a single CPU type. That company will have
| complete control of the OS stack, so if it says no
| Windows, then no Windows has to be supported.
| forrestthewoods wrote:
| cool
| matheusmoreira wrote:
| > But adult projects should support Windows imho and
| consider Windows support from the start.
|
| Hope whatever "adult" is working on the project this is
| getting paid handsomely. They'd certainly need to pay me
| big bucks to care about Windows support.
|
| In any case, Linux system call ABI is becoming a lingua
| franca of systems programming. BSDs have implemented
| Linux system calls. Windows has straight up included
| Linux in the system. It looks like simply targeting Linux
| can easily result in a binary that actually does run
| anywhere.
| forrestthewoods wrote:
| > Windows has straight up included Linux in the system.
| It looks like simply targeting Linux can easily result in
| a binary that actually does run anywhere.
|
| Kind of. But not really. WSL2 is a thing. But most code
| isn't running in WSL2 so if your thing "runs on windows"
| but requires running in a WSL2 context then oftentimes it
| might as well not exist.
|
| > They'd certainly need to pay me big bucks to care about
| Windows support.
|
| The great irony is that Windows is a much much much
| better and more pleasant dev environment. Linux is
| utterly miserable and it's all modern programmers know.
| :(
| 1718627440 wrote:
| There is also WSL1 and Cygwin and MinGW/MSYS2.
|
| And no WSL2 is not a newer version of WSL1, they are
| entirely different products.
| forrestthewoods wrote:
| MinGW is awful. Avoid. Cygwin is honestly not really
| something that has come up in my career.
|
| I don't know why Linux people are so adamant to break
| their backs - and the backs of everyone around them - to
| try and do things TheLinuxWay. It's weird. IMHo it's far
| far far better and to take a "when in Rome" approach.
|
| My experience is that Linux people are MUCH worse at
| refusing to take a When in Rome approach than the other
| way. The great tragedy is that the Linux way is not
| always the best way.
| 1718627440 wrote:
| I found MinGW to be quite nice, but YMMW.
|
| > to try and do things TheLinuxWay
|
| It's not really about TheLinuxWay. It's more that
| Microsoft completely lacks POSIX tools at all and the
| compiler needs to have a complete IDE installed, which I
| would need a license for, and the compiler invocation
| also doesn't really correspond to any other compiler.
| codedokode wrote:
| Try playing audio or displaying image on the screen using
| only documented syscalls. And make it work on all
| platforms you mentioned.
| matheusmoreira wrote:
| Displaying an image on the screen is not _that_ difficult
| a task. Linux has framebuffer device files. You open
| them, issue an ioctl to get metadata like screen geometry
| and color depth, then mmap the framebuffer as an array of
| pixels you can CPU render to. It 's eerily similar to the
| way terminal applications work.
|
| It's also possible to use Linux KMS/DRM without any user
| space libraries.
|
| https://github.com/laxyyza/drmlist/
|
| The problem with hardware accelerated rendering is much
| of the associated functionality is actually implemented
| in _user space_ and therefore not part of the kernel.
| They unfortunately force the libc on us. One would have
| to reimplement things like Mesa in order to do this. Not
| impossible, just incredibly time consuming.
|
| Things _could_ have been organized in a way that makes
| this feasible. Example: SQLite. You can plug in your own
| memory allocation functions and VFS layer. I 've been
| slowly porting the SQLite Unix VFS to freestanding Linux
| in order to use it in my freestanding applications.
| vidarh wrote:
| I've worked on dozens of "adult" projects for 30 years,
| only 2 of which ever needed to run against the Win32 API,
| and only one of which ever ran on Windows. There's a
| whole world of people out there who don't care about
| Windows compatibility because it's usually not relevant
| to the work we do.
| jjmarr wrote:
| Tons of driver code does this.
| 1718627440 wrote:
| I generally try to stay portable, but file descriptors are just
| to nice, to not use them.
| Retr0id wrote:
| File descriptors are part of the linux syscall API, not libc.
| Are you thinking of FILE?
| 1718627440 wrote:
| I did mean file descriptors.
| Retr0id wrote:
| Then I'm confused by what you meant, because you can use
| fds with or without libc.
| 1718627440 wrote:
| I don't want to bypass libc in general, because I care
| about portability, but fds are just a nice interface, so
| I still use them instead of FILE, which would be the
| portable choice. My calls are still subject to OS
| choices, that differ from the kernel, since I don't
| bypass libc.
| ajross wrote:
| The "syscall API" is part of libc too. The read syscall is
| a trap, you put arguments in the right registers and issue
| the correct instruction[1] to enter the kernel. That's not
| something that can be expressed in C. The read() function
| that your C code actually uses is a C function provided by
| the C library.
|
| [1] "svc 0" on ARM, "int 0x80" on i386, etc...
| Retr0id wrote:
| syscalls are an implementation detail of some libc impls
| on some platforms, but the C spec does not mention
| syscalls.
| cyphar wrote:
| The libc syscall wrappers are part of the libc API, but
| on Linux, syscalls are part of the stable ABI and so you
| can freely do __asm__(...) to write your own version of
| syscall(2) and it is fully supported. Yeah, __asm__ is
| probably not in the C spec, but every compiler implements
| it...
|
| For instance, Go directly calls Linux system calls
| without going through libc (which has lead to lots of
| workarounds to emulate some glibc-specific behaviour --
| swings and roundabouts I guess...).
|
| Other operating systems do not provide this kind of
| compatibility guarantee and instead require you to always
| go through libc as the syscall ABI is not stable (though
| ultimately, you can still use __asm__ if you so choose).
|
| In any case, file descriptors are definitely not a libc
| construct on Linux.
| 1718627440 wrote:
| Yes, you can. Then you don't write against the OS, but
| against the kernel. It sometimes works, because the
| kernel is a separate project, it sometimes doesn't, you
| gave an example yourself.
|
| > In any case, file descriptors are definitely not a libc
| construct on Linux.
|
| File descriptors come definitely from the kernel, but
| they do also exist as a concept in libc, and I was
| referring to them as such. I was saying that I depend on
| non-portable libc functions, even though I value
| portability, because the API is just so nice. I did not
| want to indicate, that I am doing syscalls directly.
| matheusmoreira wrote:
| > That's not something that can be expressed in C.
|
| I've often made the argument that compilers should add
| builtins for Linux system calls. Just emit code in the
| right calling convention and the system call instruction,
| and return the result. Even high level dynamic languages
| could have their JIT compilers generate this code.
|
| I actually tried to hack a linux_system_call builtin into
| GCC at some point. Lost that work in a hard drive crash,
| sadly. The maintainers didn't seem too convinced in the
| mailing list so I didn't bother rewriting it.
|
| > The read() function that your C code actually uses is a
| C function provided by the C library.
|
| These are just magic wrapper functions. The actual Linux
| system call entry point is language agnostic, specified
| at the instruction architecture level, and is considered
| stable.
|
| https://www.matheusmoreira.com/articles/linux-system-
| calls
|
| This is different from other systems which force people
| to use the C library to interface with the kernel.
|
| One of the most annoying things in the Linux manuals is
| they conflate the glibc wrappers with the _actual_ system
| calls in Linux. The C library does a lot more than just
| wrap these things, they dynamically choose the best
| variants and even implement cancellation /interruption
| mechanisms. Separating the Linux behavior from libc
| behavior can be difficult, and in my experience requires
| reading kernel source code.
| 1718627440 wrote:
| > I've often made the argument that compilers should add
| builtins for Linux system calls. Just emit code in the
| right calling convention and the system call instruction,
| and return the result. Even high level dynamic languages
| could have their JIT compilers generate this code.
|
| You can only do that, when you compile for a specific
| machine. In general you are compiling for some abstract
| notion of an OS. JITs always compile for the machine they
| are running on, so they don't have that problem. There is
| code, that is compiled directly to your syscalls specific
| to your machine, so that abstract code can use this. It's
| called libc for the C language.
|
| > One of the most annoying things in the Linux manuals is
| they conflate the glibc wrappers with the actual system
| calls in Linux. The C library does a lot more than just
| wrap these things, they dynamically choose the best
| variants and even implement cancellation/interruption
| mechanisms. Separating the Linux behavior from libc
| behavior can be difficult, and in my experience requires
| reading kernel source code.
|
| In my experience there are often detailed explanation in
| the notes section. From readv(2): NOTES
| POSIX.1 allows an implementation to place a limit on
| the number of items that can be passed in iov.
| An implementation can advertise its limit
| by defining IOV_MAX in <limits.h> or at run time via the
| return value from sysconf(_SC_IOV_MAX). On
| modern Linux systems, the limit is 1024. Back
| in Linux 2.0 days, this limit was 16. C
| library/kernel differences The raw preadv()
| and pwritev() system calls have call signatures that
| differ slightly from that of the corresponding GNU C
| library wrapper functions shown in the
| SYNOPSIS. The final argument, offset, is un-
| packed by the wrapper functions into two arguments in the
| system calls: unsigned long pos_l,
| unsigned long pos These arguments
| contain, respectively, the low order and high order 32
| bits of offset. Historical C library/kernel
| differences To deal with the fact that
| IOV_MAX was so low on early versions of Linux,
| the glibc wrapper functions for readv() and writev()
| did some extra work if they detected
| that the underlying kernel system call failed
| because this limit was exceeded. In the case of
| readv(), the wrapper function allocated a
| temporary buffer large enough for all of the
| items specified by iov, passed that buffer in a call to
| read(2), copied data from the buffer to the
| locations specified by the iov_base fields of
| the elements of iov, and then freed the buffer. The
| wrapper function for writev() performed the
| analogous task using a temporary buffer and a
| call to write(2). The need for this extra
| effort in the glibc wrapper functions went away
| with Linux 2.2 and later. However, glibc continued to
| provide this be- havior until version 2.10.
| Starting with glibc version 2.9, the wrap-
| per functions provide this behavior only if the
| library detects that the system is running a
| Linux kernel older than version 2.6.18 (an ar-
| bitrarily selected kernel version). And since glibc
| 2.20 (which re- quires a minimum Linux kernel
| version of 2.6.32), the glibc wrapper
| functions always just directly invoke the system calls.
| matheusmoreira wrote:
| > You can only do that, when you compile for a specific
| machine.
|
| You always compile for a specific machine. There is
| always a target instruction set architecture. It decides
| the calling convention used for Linux system calls.
| Compiler can even produce an error in case the target is
| not supported by Linux.
|
| > In general you are compiling for some abstract notion
| of an OS.
|
| This "abstract notion of an OS" boils down to the libc.
| Freestanding C gets rid of most of it. Making system
| calls is also perfectly valid in hosted C. Modern
| languages like Rust also have freestanding modes.
|
| > In my experience there are often detailed explanation
| in the notes section.
|
| That's the problem. Why is the Linux stuff just a bunch
| of footnotes in the Linux manual? It should be in the
| main section. The glibc specifics should be footnotes.
| 1718627440 wrote:
| Specific machine meaning defined set of installed
| software, versions in install locations.
|
| Abstract notion of OS meaning Debian 12. Not Linux kernel
| commit ####, GNU libc commit ####, dpkg commit ####, apt
| commit ####, Apache httpd commit #### with patch ### to
| ### from Debian 4 version ### and Ubuntu 21 version ###,
| SQLite3 with special patches ### installed in
| /opt/bin/foo, ... (you get the idea).
|
| > That's the problem. Why is the Linux stuff just a bunch
| of footnotes in the Linux manual? It should be in the
| main section. The glibc specifics should be footnotes.
|
| Because you look at the OS manual, not at the
| documentation of the kernel. Notes and Bugs are also not
| footnotes in man pages. They are pretty important and are
| basically the first free-form section where you can tell
| about the ideas, ideals and history. The first part a
| pretty strict, formal description of the calling
| semantics.
| matheusmoreira wrote:
| Let's systematize this.
|
| Compilers build for target triples such as x86_64-linux-
| gnu. It is of the form isa-kernel-userspace. If kernel is
| linux, the builtin can be used. The isa determines the
| code generated by the compiler, both in general and for
| the builtin. The userspace can be anything at all,
| including none. Sometimes compilers build for target
| quadruples which also include a vendor, and that
| information is also irrelevant.
| 1718627440 wrote:
| I am not sure you understand my point. Inlining libc
| definitions for syscalls is fine when you only care about
| Debian 12 commit hash ####. It will break as soon as you
| think your machine is running Debian 12 and you updated
| it, so surely it includes the latest userspace-patches.
| It will also break when a user uses the OS configuration
| to change the behaviour of some OS functionality, but
| your code is oblivious to that matter, because your code
| bypasses the OS version of libc.
|
| Modifying the OS is fine, if this is what you want to do,
| but it comes with tradeoffs.
|
| ----
|
| You wrote earlier:
|
| > actually tried to hack a linux_system_call builtin into
| GCC at some point. [...] The maintainers didn't seem too
| convinced in the mailing list so I didn't bother
| rewriting it.
|
| I am not sure what exactly this means. There is
| syscall(2) in the libc, if you want to do this. If you
| want to inline the wrappers you can pass -static to the
| compiler invocation.
| electroly wrote:
| Not exactly the same, but on Windows if you use entirely Win32
| calls you can avoid linking any C runtime library. Win32 is
| below the C standard library on Windows and the C runtime is
| optional.
| okanat wrote:
| This is one of the cornerstones that guarantee Windows can
| easily upgrade the C runtime and make performance and
| security upgrades. Win32 APIs have a different function
| calling ABI too.
|
| So only part of that gets "bloated" is Win32 API itself
| (which is spread across multiple DLLs and don't actually
| bloat RAM usage). Most of the time even those functions and
| structures are carefully designed to have some future-
| proofness but it is usual to see APIs like CreateFile,
| CreateFile2, CreateFile3. Internally the earlier versions are
| upgraded to call the latest version. So not so much bloating
| there either.
|
| When the C runtime and the OS system calls are combined into
| the single binary like POSIX, it creates the ABI hell we're
| in with the modern Unix-likes. Either the OSes have to
| regularly break the C ABI compatibility for the updates or we
| have to live with terrible implementations.
|
| GNU libc and Linux combo is particularly bad. On GNU/Linux
| (or any other current libc replacements), the dynamic loading
| is also provided by the C library. This makes "forever"
| binary file compatibility particularly tricky to achieve.
| Glibc broke certain games / Steam by removing some parts of
| their ELF implementation:
| https://sourceware.org/bugzilla/show_bug.cgi?id=32653 . They
| backed due to huge backlash from the community.
|
| If "the year of Linux desktop" would ever happen, they need
| to either do an Android and change the definition of what a
| software package is, or split Glibc into 3 parts: syscalls,
| dynamic loader and the actual C library.
|
| PS: There is actually a catch to your " C runtime is
| optional." argument. Microsoft still intentionally holds back
| the ability of compiling native ABI Windows programs without
| Visual Studio.
|
| The structured exception handlers (equivalent of Windows for
| SIGILL, SIGBUS etc.. not for SIGINT or SIGTERM though) are
| populated by the object files from the C runtime libraries
| (called VCRuntime/VCStartup). So it is actually not possible
| to have official Windows binaries without MSVC or any other C
| runtime like Mingw-64 that provides those symbols. It looks
| like some developers in Microsoft wanted to open-source
| VCRuntime / VCStartup but it was ~vetoed~ not fully approved
| by some people: https://github.com/microsoft/STL/issues/4560#
| issuecomment-23... , https://www.reddit.com/r/cpp/comments/1l
| 8mqlv/is_msvc_ever_g...
| 1718627440 wrote:
| > split Glibc into 3 parts: syscalls, dynamic loader and
| the actual C library.
|
| What is left of the C standard library, if you remove
| syscall wrappers?
|
| > ABI hell
|
| Is that really the case? From my understanding the problem
| is more, that Linux isn't an OS, so you can't rely on any
| *.so being there.
| okanat wrote:
| > > split Glibc into 3 parts: syscalls, dynamic loader
| and the actual C library.
|
| > What is left of the C standard library, if you remove
| syscall wrappers?
|
| Still quite a bit actually. Stuff like malloc, realloc,
| free, fopen, FILE, getaddrinfo, getlogin, math functions
| like cos, sin tan, stdatomic implementations, some string
| functions are all defined in C library. They are not
| direct system calls unlike: open, read, write, ioctl,
| setsockopt, capget, capset ....
|
| > > ABI hell
|
| > Is that really the case? From my understanding the
| problem is more, that Linux isn't an OS, so you can't
| rely on any *.so being there.
|
| That's why I used more specific term GNU/Linux at the
| start. There is no guarantee of any .so file can be
| successfully loaded even if it is there. Glibc can break
| anything. With the Steam bug I linked this is exactly
| what happened. Shared object files were there, Glibc
| stopped supporting a certain ELF file field.
|
| There is only and only one guarantee with Linux-based
| systems: syscalls (and other similar ways to talk with
| kernel like ioctl struct memory layouts etc) always keep
| working.
|
| There is so much invisible dependence on Glibc behavior.
| Glibc also controls how the DNS works for the programs
| for example. That also needs to be split into a different
| library. Same for managing user info like `getlogin`.
| Moreover all this functionality is actually implemented
| as dynamic library plugins in Glibc (NSSwitch) that rely
| on ld.so that's also shipped by Glibc. It is literally a
| Medusa head of snakes that bite multiple tails. It is
| extremely hard to test ABI breakages like this.
| 1718627440 wrote:
| > malloc, realloc, free
|
| Wrapper around sbrk, mmap, etc. whatever the modern
| variant is.
|
| > fopen, FILE
|
| Wrapper around open, write, read, close.
|
| > stdatomic implementations
|
| You can argue, these are wrappers around thread syscalls.
|
| > math functions like cos, sin tan, some string functions
| are all defined in C library
|
| True for these, but they are so small, they could just be
| inlined directly, on their own they wouldn't necessarily
| deserve a library.
|
| > That's why I used more specific term GNU/Linux at the
| start.
|
| While GNU/Linux does describe a complete OS, it doesn't
| describe any specific OS. Every Distro does it's own
| thing, so I think these is what you actually need to call
| an OS. But everything is built so that the user can take
| the control over the architecture and which components
| the OS consists of, so every installation can be a
| snowflake, and then it is technically its own OS.
|
| I personally consider libc and the compiler (which both
| make a C implementation) to be part of the OS. I think
| this is both grounded in theory and in practice. Only in
| some weird middle ground between theory and practice you
| can consider them to not be.
| vbezhenar wrote:
| > malloc, realloc, free > Wrapper around sbrk, mmap, etc.
| whatever the modern variant is.
|
| I don't think that's correct. While `malloc` uses `brk`
| syscall to allocate large memory areas, it uses non-
| trivial algorithms and data-structures to further divide
| that areas into smaller chunks which actually returned.
| Using syscall for every `malloc`/`free` is quite an
| overhead.
|
| > fopen, FILE
|
| > Wrapper around open, write, read, close.
|
| They're not just wrappers. They implement internal
| buffering, some transformations (for example see "binary"
| mode, "text" mode.
|
| > stdatomic implementations
|
| > You can argue, these are wrappers around thread
| syscalls.
|
| No, they're wrappers around compiler intrinsics which
| emit specific assembly instructions. At least for any
| sane architecture.
|
| > I personally consider libc and the compiler (which both
| make a C implementation) to be part of the OS. I think
| this is both grounded in theory and in practice. Only in
| some weird middle ground between theory and practice you
| can consider them to not be.
|
| C is used a lot in embedded projects. I even think that's
| the majority of C code nowadays. These projects usually
| don't use libc (as there's no operating system, so
| concept of file or process just doesn't make sense). So
| it's very important to separate C compiler and libc and C
| compiler must be able to emit code with zero
| dependencies.
| 1718627440 wrote:
| Yeah sure they do more things than only doing the
| syscall, that's the point of an abstraction. But they
| still provide the functionality of the syscalls, just in
| the abstraction that you want it to be exposed as in the
| programming language. That's what I would consider a
| wrapper.
|
| > C is used a lot in embedded projects.
|
| Sure that's a freestanding implementation, which primary
| distinction is that it doesn't rely on the libc. The
| notion of the libc being part of the OS in the wider
| sense, still holds water here, since here no OS
| corresponds to no libc.
| saagarjha wrote:
| mmap and sbrk would be very poor implementations of
| malloc.
| 1718627440 wrote:
| We are talking of wrappers on top of mmap and sbrk. Of
| course you wouldn't use mmap and sbrk instead of the
| abstraction. It's really the same as the difference
| between fread and read.
| liqilin1567 wrote:
| I'm sick of glibc compatibility problems. Are there any
| recommended replacements?
| emidln wrote:
| .interp to a glibc/libc you ship or static linking. These
| days it's probably faster (in dev time) to just run a
| container than setting up a bespoke interp and a parallel
| set of libraries (and the associated toolchain changes or
| binary patching needed to support it).
| liqilin1567 wrote:
| Running a container is exactly my current solution as
| well.
|
| Are there any other solutions that don't depend on glibc?
| retatop wrote:
| Guix (and I assume Nix as well, but I only know Guix) can
| create a package that is completely self contained
| including glibc. You can even have it be an AppImage
| https://guix.gnu.org/manual/devel/en/html_node/Invoking-
| guix...
| 1718627440 wrote:
| Glibc is half of GNU/Linux. You can of course use another
| libc, but it will be a different OS.
| codedokode wrote:
| > Glibc broke certain games / Steam by removing some parts
| of their ELF implementation:
| https://sourceware.org/bugzilla/show_bug.cgi?id=32653 .
| They backed due to huge backlash from the community.
|
| It would be better if you specified which part was removed:
| support for executable code on stack. This is used in 99%
| cases by malware so it is better to break 1% of broken
| programs and have other 99% run safer.
| josefx wrote:
| The comments on that bug report mention several language
| runtimes getting broken. Preventing languages that are
| generally safer than C from working seems rather
| counterproductive to overall security.
| saagarjha wrote:
| > If "the year of Linux desktop" would ever happen, they
| need to either do an Android and change the definition of
| what a software package is, or split Glibc into 3 parts:
| syscalls, dynamic loader and the actual C library.
|
| The dynamic loader used to be its own library, FWIW. It got
| merged into the main one recently.
| matheusmoreira wrote:
| I once wrote a liblinux project just for this!! It was indeed
| extremely fun. Details in my other comment:
|
| https://news.ycombinator.com/item?id=45709141
|
| I abandoned it because Linux itself now has a rich set of
| nolibc headers.
|
| Now I'm working on a whole programming language based around
| this concept. A freestanding lisp interpreter targeting Linux
| directly with builtin system call support. The idea is to
| complete the interpreter and then write the standard library
| and Linux user space in lisp using the system calls.
|
| It's been an amazing journey. It's incredible how far one can
| take this.
| codedokode wrote:
| I think using syscalls directly is a worse idea than loading
| shared libraries, and new kernel features, like ALSA (audio
| playback), DRM (graphics rendering) and other use libraries
| instead of documenting syscalls and ioctls. This is better
| because it allows intercepting and subverting the calls, adding
| support for features even if the kernel doesn't support it,
| makes it easier to port code to other OSes, support different
| architectures (32-bit code on 64-bit kernel), and allows
| changing kernel interface without breaking anything. So
| Windows-style approach with system libraries is better in every
| aspect.
| mmsc wrote:
| It's also possible to pack a whole codebase into "before main()"
| - or with no main() at all. I was recently experimenting doing
| this, as well as a whole codebase that only uses main() and calls
| itself over and over. Good fun: https://joshua.hu/packing-
| codebase-into-single-function-disr...
| 1718627440 wrote:
| That is a really fun read and honestly doesn't even seem to be
| complicated and brittle. Just rename every function to
| main(100+n, ...).
| thatxliner wrote:
| Just wondering, how did you get that domain name? I've been
| looking for registrars offering .hu
| hashstring wrote:
| whois data points to https://www.domain.hu.
| slater wrote:
| https://nic.hu/index_en.html ?
| khaledh wrote:
| > A note on interpreters: If the executable file starts with a
| shebang (#!), the kernel will use the shebang-specified
| interpreter to run the program. For example, #!/usr/bin/python3
| will run the program using the Python interpreter, #!/bin/bash
| will run the program using the Bash shell, etc.
|
| This caused me a lot of pain while trying to debug a 3rd party
| Java application that was trying to launch an executable script,
| and throwing an IO error "java.io.IOException: error=2, No such
| file or directory." I was puzzled because I _know_ the script is
| right there (using its full path) and it had the executable bit
| set. It turns out that the shebang in the script was wrong, so
| the OS was complaining (actual error from a shell would be "The
| file specified the interpreter '/foo/bar', which is not an
| executable command."), but the Java error was completely
| misleading :|
|
| Note: If you wonder why I didn't see this error by running the
| script myself: I did, and it ran fine locally. But the
| application was running on a remote host that had a different
| path for the interpreter.
| mscdex wrote:
| Also be aware that kernel support for shebangs depends on
| CONFIG_BINFMT_SCRIPT=y being in the kernel config.
| 1718627440 wrote:
| Note, that this is not a Java specific problem, it can occur
| with other programs as well. "No such file or directory" is
| just the nice description for ENOENT, which can occur in a lot
| of syscalls. I typically just run the program through strace,
| then you will quickly see what the program did.
| gjf wrote:
| For those interested, I did a breakdown of the hashbang:
| https://blog.foletta.net/post/2021-04-19-what-the/
| itopaloglu83 wrote:
| I like doing this with old microcontrollers like PIC16 series
| etc. You said see how to stack pointer, timers, and variables
| etc. all are configured.
| fweimer wrote:
| > The ELF file contains a dynamic section which tells the kernel
| which shared libraries to load, and another section which tells
| the kernel to dynamically "relocate" pointers to those functions,
| so everything checks out.
|
| This is _not_ how dynamic linking works on GNU /Linux. The kernel
| processes the program headers for the main program (mapping the
| PT_LOAD segments, without relocating them) and notices the
| PT_INTERP program interpreter (the path to the dynamic linker)
| among the program headers. The kernel then loads the dynamic
| linker in much the same way as the main program (again without
| relocation) and transfers control to its entry point. It's up to
| the dynamic linker to self-relocate, load the referenced share
| objects (this time using plain mmap and mprotect, the kernel ELF
| loader is not used for that), relocate them and the main program,
| and then transfer control to the main program.
|
| The scheme is not that dissimilar to the #! shebang lines, with
| the dynamic linker taking the role of the script interpreter,
| except that ELF is a binary format.
| amitprasad wrote:
| You're right, and I knew this back in February when I wrote
| most of this post. I must have revised it down incorrectly
| before posting; will correct. Bit of a facepalm from my side.
| mkoubaa wrote:
| I've always wondered why there weren't more popular loaders to
| choose from given that on Linux loaders are user-space
| BobbyTables2 wrote:
| I suspect it is because they get really hairy.
|
| Loading ELFs and processing relocations is actually not too
| bad. It's fun after the initial learning curve.
|
| Then one has to worry about handling of "dlopen" and the
| loader creating the data structures it cares about. Yuck!!!
|
| It's kinda a shame because the glibc loader is a bit bloated
| with all the audit and preload handling. Great for
| flexibility, not for security.
| matheusmoreira wrote:
| It's hard to describe how complex this stuff is. Shared
| object loaders are essentially primitive package managers,
| topologically sortinf dependencies and everything...
|
| https://blogs.oracle.com/solaris/post/init-and-fini-
| processi...
| ksherlock wrote:
| There's also binfmt support, which can check a supposedly
| executable file against some magic and auto-launch an
| interpreter (like wine or java or dosemu). I looked into it
| for something once but in my case the magic wasn't good
| enough.
|
| https://www.kernel.org/doc/html/latest/admin-guide/binfmt-
| mi...
| delusional wrote:
| Its super awesome for qemu. It let's you chroot into a arm
| root (full of arm binaries) on you x86 machine and just run
| it like normal. No VM required.
| mananaysiempre wrote:
| Part of it is the Glibc loader's carnal knowledge of Glibc
| proper; there's essentially no module boundary there. (That's
| not _completely_ unjustified, but Glibc is especially hostile
| there, like in its many other architectural choices.) Musl
| outright merges the two into a single binary. So if you want
| to do a loader then you're also doing a libc.
|
| Part of it for desktop Linux specifically is that a lot of
| the graphics stack is very unfriendly to alternative libcs or
| loaders. For example, Wayland is nominally a protocol
| admitting multiple implementations, but if you want to not be
| dumb[1] and do GPU-accelerated graphics, then the ABI ties
| you to libwayland.so specifically (event-loop opinions and
| all) in order to load vendor-specific userspace drivers,
| which entails your distro's preferred libc (probably Glibc).
|
| [1] There can of course be good engineering reasons to be
| dumb.
| codedokode wrote:
| Why do you need "vendor-specific userspace drivers"? I
| thought graphic acceleration uses OpenGL/Vulkan, and non-
| accelerated graphics uses DRM? And there are no "drivers"
| for Wayland compositors?
| matheusmoreira wrote:
| OpenGL and Vulkan are implemented as libraries in user
| space as the Mesa project.
| fweimer wrote:
| With containers, you usually get incompatible dynamic loaders
| in the containers (see mananaysiempre' comment; the glibc
| dynamic linker sees rather active development in some LTS
| distributions). This wouldn't be possible if the loader were
| part of the kernel.
|
| Non-ELF loaders are fairly common, too. It's how Wine works,
| and how Microsoft reuses PE/COFF SQL Server binaries on
| Linux.
| matheusmoreira wrote:
| Yeah it turns out the kernel doesn't care about sections at
| all. It only ever cares about the PT_LOAD segments in the
| program header table, which is essentially a table of arguments
| for the mmap system call. Sections are just dynamic linker
| metadata and are never covered by PT_LOAD segments.
|
| This seems to be a common misconception. I too suffered from it
| once... Tried to embed arbitrary files into ELF files using
| objcopy. The tool could easily create new sections with the
| file contents just fine, but the kernel wouldn't load them into
| memory. It was really confusing at first.
|
| https://stackoverflow.com/q/77468641
|
| There were no tools for patching the program header table, I
| ended up making them! The mold linker even added a feature just
| to make this patching easy!
|
| https://www.matheusmoreira.com/articles/self-contained-lone-...
| turbert wrote:
| Its been a while since I've touched this stuff but my
| recollection is the ELF interpreter (ldso, not the kernel) is
| responsible for everything after mapping the initial ELF's
| segments.
|
| iirc execve maps pt_load segments from the program header,
| populates the aux vector on the stack, and jump straight to the
| ELF interpreter's entry point. Any linked objects are loaded in
| userspace by the elf interpreter. The kernel has no knowledge of
| the PLT/GOT.
| matheusmoreira wrote:
| That's right!
|
| https://lwn.net/Articles/631631/
|
| https://github.com/torvalds/linux/blob/master/fs/binfmt_elf....
|
| Especially relevant for dynamic linkers is the AT_PHDR and
| AT_BASE auxiliary vector entries which provide the address of
| the executable's program header table and the address of the
| interpreter, respectively.
|
| https://lwn.net/Articles/519085/
| archmaster wrote:
| This is awesome! To anyone interested in learning more about
| this, I wrote https://cpu.land/ a couple years ago. It doesn't go
| as in-depth into e.g. memory layout as OP does but does cover
| multitasking and how the code is loaded in the first place.
| fuzzy_biscuit wrote:
| I love cpu.land! Thanks for creating such a fun resource.
| bignerd_95 wrote:
| As someone who teaches this stuff at university, I see students
| getting confused every single year by how textbooks draw memory.
| The problem is mostly visual, not conceptual.
|
| Most diagrams in books and slides use an old hardware-centric
| convention: they draw higher addresses at the top of the page and
| lower addresses at the bottom. People sometimes justify this with
| an analogy like "floors in a building go up," so address
| 0x7fffffffe000 is drawn "higher" than 0x400000.
|
| But this is backwards from how humans read almost everything
| today. When you look at code in VS Code or any other IDE, line 1
| is at the top, then line 2 is below it, then 3, 4, etc. Numbers
| go up as you go down. Your brain learns: "down = bigger index."
|
| Memory in a real Linux process actually matches the VS Code model
| much more closely than the textbook diagrams suggest.
|
| You can see it yourself with:
|
| cat /proc/$$/maps
|
| (pick any PID instead of $$). ...
|
| [0x00000000] lower addresses ...
|
| [0x00620000] HEAP start
|
| [0x00643000] HEAP extended | (more allocations => higher
| addresses) ...
|
| [0x7ffd8c3f7000] STACK top (<- stack pointer)
| | the stack pointer starts here and moves upward
| (toward lower addresses) when you push
|
| [0x7ffd8c418000] STACK start ...
|
| [0xffffffffff600000] higher addresses ...
|
| The output is printed from low addresses to high addresses. At
| the top of the output you'll usually see the binary, shared libs,
| heap, etc. Those all live at lower virtual addresses. Farther
| down in the output you'll eventually see the stack, which lives
| at a higher virtual address. In other words: as you scroll down,
| the addresses get bigger. Exactly like scrolling down in an
| editor gives you bigger line numbers.
|
| The phrases "the heap grows up" and "the stack grows down" aren't
| wrong. They're just describing what happens to the numeric
| addresses: the heap expands toward higher addresses, and the
| stack moves into lower addresses.
|
| The real problem is how we draw it. We label "up" on the page as
| "higher address," which is the opposite of how people read code
| or even how /proc/<pid>/maps is printed. So students have to
| mentally flip the diagram before they can even think about what
| the stack and heap are doing.
|
| If we just drew memory like an editor (low addresses at the top,
| high addresses further down) it would click instantly. Scroll
| down, addresses go up, and the stack sits at the bottom. At that
| point it's no longer "the stack grows down": it's just the stack
| pointer being decremented, moving to lower addresses (which, in
| the diagram, means moving upward).
| 1718627440 wrote:
| That's how stacks on my desk grow and how everything grows in
| reality. I wouldn't numerate stacked things on my desk from the
| top, since this constantly changes. You also wouldn't name the
| first branch of a tree (the plant) to be the top-most one.
|
| In your example "the stack grows down", seems to be wrong in
| the image.
| bignerd_95 wrote:
| Thanks! I tried to rewrite the final sentence
| 1718627440 wrote:
| Yeah, but does that really help? The phrases "growing
| down/up" still exist and now you defined them to mean the
| opposite. This issue still didn't go away, since heap and
| stack still grow in different directions. Can't you just
| start drawing from the bottom of the blackboard, and it
| will be obvious? Coordinate systems also typically work
| that way.
| ofalkaed wrote:
| Starting at the bottom of the blackboard would be
| backwards from how it prints in the terminal when you cat
| /proc/<pid>/maps.
| 1718627440 wrote:
| The way it's printed in the terminal is honestly
| backwards to me. This seams to only come from the scroll
| direction of the terminal, and because this is not a
| drawing, but a simple list. Every other tool like a
| debugger show it in the opposite direction and in all
| illustrations I have read it's that way too.
| bignerd_95 wrote:
| So you use debuggers. Good. Then you can confirm that the
| program counter is incremented after each instruction,
| and that you read assembly from top to bottom. That means
| smaller addresses are at the top and larger addresses are
| at the bottom. This matches my layout, and it also
| matches what you see in the terminal in /proc/<pid>/maps.
| 1718627440 wrote:
| Incrementation direction is orthogonal to up/down. When I
| disassemble a program, the addresses are all over the
| place and not ordered.
|
| > That means smaller addresses are at the top and larger
| addresses are at the bottom.
|
| No, my mental model is the exact opposite and this
| matches the jargon out there.
|
| > also matches what you see in the terminal in
| /proc/<pid>/maps
|
| I think of this as a sorted list, not as a display or
| description of a model.
|
| When I drive on a road, I think of think of things on the
| road near me to have lower addresses in my coordinate
| system and things further away as having higher
| addresses. When I write a list of things I see, this will
| be from left-to-right then from top-to-bottom on a sheet
| of paper, because it is a list that follows the writing
| directions of my language/script. When I look at a
| traffic sign things nearer to me will be at the bottom
| and things far away at the top, because that's the
| agreed-upon mental model of a road. When I look at my
| navi, things near me will be in the center and things far
| away from me at the edge of the display.
|
| When I write down points in the first sector of the
| coordinate system, I might order things according to the
| x-coordinate ascending top-to-bottom. That doesn't mean I
| would draw the axis inverted.
|
| The correspondence of physical addresses to position is
| entirely non-linear and also three dimensional so there
| is no natural top and bottom especially when we are
| talking about virtual addresses.
|
| When I get taught a new concept I want to get to know the
| model everyone uses. I will not like a teacher, that
| tells me a different ordering which is different from how
| everyone else does it, because this the output of some
| random command on some random OS, which actually shows a
| list, not a graph of a memory model. (Sorry that's harsh,
| of course I still appreciate didactic simplification.)
|
| Maybe the issue is that you consider the stack to be so
| important to determine the model of the whole process
| space. When I would draw a stack on it's own, I DO draw
| it from top-to-bottom. But when I draw a whole process
| space, I do not, because everything else is
| mapped/allocated from bottom-to-top. When you invert the
| direction of the mental model, yes the stack now grows
| from bottom-to-top. But no, the other things are now
| allocated from top-to-bottom instead. This are more
| things: the text, libraries, mmap'd files, and most-used
| thing: the heap are all allocated inverted now. And the
| most important thing, the index for all that: the
| addresses now are allocated from top-to-bottom.
| krackers wrote:
| Yeah this is sort of the same objection I had in
| https://news.ycombinator.com/item?id=45709016
|
| Although thinking about it more, the fact that address
| indexing is now from top-to-bottom is actually consistent
| with how I imagine indexing for Nx1 array (or
| equivalently, how you index matrices in math).
|
| Thinking about it more, I don't think there is any real
| convention, even diagrams are split. You are going to
| have to learn to "flip" things either way, the same way
| matrix indexing differs from cartesian plane indexing.
| Same way different graphical systems have different
| conventions for where the origin is. People colloquially
| say things like "high mem", and you'll have to translate
| this to your mental model.
|
| It's why I suggested a horizontal, left-to-right
| visualization. I think everyone would agree that lower
| addresses on the left, higher addresses on right.
|
| Also could you elaborate more on what you mean by
| debuggers showing it the opposite direction? If you do
| `info proc mappings` in gdb it is also lower addresses at
| top. This might be debugger specific though.
| bignerd_95 wrote:
| Yes, I draw the heap starting at the top of the board and
| the stack starting at the bottom of the board and grow
| them toward each other. That works fine in a one-off
| explanation.
|
| The problem is that most textbooks draw the opposite, so
| the student leaves my lecture, opens a book or a slide
| deck, and now "down" means a different thing.
|
| It gets worse when they get curious and look at a real
| process with /proc/<pid>/maps. Linux prints mappings from
| low address to high address as you scroll down (which
| matches my representation). That is literally reversed
| from the usual textbook diagram. Students notice and ask
| why the book is "wrong."
|
| So I've learned I have to explicitly call this out as
| notation.
|
| Same story as in electronics class still teaching
| conventional current flow (positive to negative), even
| though electrons move the other way (negative to
| positive). Source:
| https://www.allaboutcircuits.com/textbook/direct-
| current/chp.... Historical convention, and then pedagogy
| has to patch it forever.
| amitprasad wrote:
| I think I got stuck in the same rut that I learned address
| space in whilst writing that diagram. I would tend to agree
| with you that your model makes much more sense to the student.
|
| Related: In notation, one thing that I used to struggle with is
| how addresses (e.g. 0xAB_CD) actually have the bit
| representation of [0xCD, 0xAB]. Wonder if there's a common way
| to address that?
| bignerd_95 wrote:
| Yes, I reached the same conclusions the hard way while
| exploiting memory corruption bugs. Once I understood how
| misleading these representations can be, everything finally
| became clear.
|
| About the address notation you're describing, I'm not sure I
| fully get the problem. Can you spell out the question with a
| concrete example?
|
| This is what the address space of a real bash process looks
| like on my machine:
|
| __
|
| $ cat /proc/$(pidof bash)/maps
|
| 5e6e8fd0f000-5e6e8fd3f000 r--p 00000000 fc:00 3539412
| /usr/bin/bash
|
| 5e6e8fd3f000-5e6e8fe2e000 r-xp 00030000 fc:00 3539412
| /usr/bin/bash
|
| 5e6e8fe2e000-5e6e8fe63000 r--p 0011f000 fc:00 3539412
| /usr/bin/bash
|
| 5e6e8fe63000-5e6e8fe67000 r--p 00154000 fc:00 3539412
| /usr/bin/bash
|
| 5e6e8fe67000-5e6e8fe70000 rw-p 00158000 fc:00 3539412
| /usr/bin/bash
|
| 5e6e8fe70000-5e6e8fe7b000 rw-p 00000000 00:00 0
|
| 5e6e94891000-5e6e94a1e000 rw-p 00000000 00:00 0 [heap]
|
| 7ec3d1400000-7ec3d16eb000 r--p 00000000 fc:00 3550901
| /usr/lib/locale/locale-archive
|
| 7ec3d1800000-7ec3d1828000 r--p 00000000 fc:00 3548995
| /usr/lib/x86_64-linux-gnu/libc.so.6
|
| 7ec3d1828000-7ec3d19b0000 r-xp 00028000 fc:00 3548995
| /usr/lib/x86_64-linux-gnu/libc.so.6
|
| 7ec3d19b0000-7ec3d19ff000 r--p 001b0000 fc:00 3548995
| /usr/lib/x86_64-linux-gnu/libc.so.6
|
| 7ec3d19ff000-7ec3d1a03000 r--p 001fe000 fc:00 3548995
| /usr/lib/x86_64-linux-gnu/libc.so.6
|
| 7ec3d1a03000-7ec3d1a05000 rw-p 00202000 fc:00 3548995
| /usr/lib/x86_64-linux-gnu/libc.so.6
|
| 7ec3d1a05000-7ec3d1a12000 rw-p 00000000 00:00 0
|
| 7ec3d1a2b000-7ec3d1a84000 r--p 00000000 fc:00 3549063
| /usr/lib/locale/C.utf8/LC_CTYPE
|
| 7ec3d1a84000-7ec3d1a85000 r--p 00000000 fc:00 3549069
| /usr/lib/locale/C.utf8/LC_NUMERIC
|
| 7ec3d1a85000-7ec3d1a86000 r--p 00000000 fc:00 3549072
| /usr/lib/locale/C.utf8/LC_TIME
|
| 7ec3d1a86000-7ec3d1a87000 r--p 00000000 fc:00 3549062
| /usr/lib/locale/C.utf8/LC_COLLATE
|
| 7ec3d1a87000-7ec3d1a88000 r--p 00000000 fc:00 3549067
| /usr/lib/locale/C.utf8/LC_MONETARY
|
| 7ec3d1a88000-7ec3d1a89000 r--p 00000000 fc:00 3549066
| /usr/lib/locale/C.utf8/LC_MESSAGES/SYS_LC_MESSAGES
|
| 7ec3d1a89000-7ec3d1a8a000 r--p 00000000 fc:00 3549070
| /usr/lib/locale/C.utf8/LC_PAPER
|
| 7ec3d1a8a000-7ec3d1a8b000 r--p 00000000 fc:00 3549068
| /usr/lib/locale/C.utf8/LC_NAME
|
| 7ec3d1a8b000-7ec3d1a8c000 r--p 00000000 fc:00 3549061
| /usr/lib/locale/C.utf8/LC_ADDRESS
|
| 7ec3d1a8c000-7ec3d1a8d000 r--p 00000000 fc:00 3549071
| /usr/lib/locale/C.utf8/LC_TELEPHONE
|
| 7ec3d1a8d000-7ec3d1a90000 rw-p 00000000 00:00 0
|
| 7ec3d1a90000-7ec3d1a9e000 r--p 00000000 fc:00 3551411
| /usr/lib/x86_64-linux-gnu/libtinfo.so.6.4
|
| 7ec3d1a9e000-7ec3d1ab1000 r-xp 0000e000 fc:00 3551411
| /usr/lib/x86_64-linux-gnu/libtinfo.so.6.4
|
| 7ec3d1ab1000-7ec3d1abf000 r--p 00021000 fc:00 3551411
| /usr/lib/x86_64-linux-gnu/libtinfo.so.6.4
|
| 7ec3d1abf000-7ec3d1ac3000 r--p 0002e000 fc:00 3551411
| /usr/lib/x86_64-linux-gnu/libtinfo.so.6.4
|
| 7ec3d1ac3000-7ec3d1ac4000 rw-p 00032000 fc:00 3551411
| /usr/lib/x86_64-linux-gnu/libtinfo.so.6.4
|
| 7ec3d1ac4000-7ec3d1ac5000 r--p 00000000 fc:00 3549065
| /usr/lib/locale/C.utf8/LC_MEASUREMENT
|
| 7ec3d1ac5000-7ec3d1ac6000 r--p 00000000 fc:00 3549064
| /usr/lib/locale/C.utf8/LC_IDENTIFICATION
|
| 7ec3d1ac6000-7ec3d1acd000 r--s 00000000 fc:00 3548984
| /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
|
| 7ec3d1acd000-7ec3d1acf000 rw-p 00000000 00:00 0
|
| 7ec3d1acf000-7ec3d1ad0000 r--p 00000000 fc:00 3548992
| /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
|
| 7ec3d1ad0000-7ec3d1afb000 r-xp 00001000 fc:00 3548992
| /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
|
| 7ec3d1afb000-7ec3d1b05000 r--p 0002c000 fc:00 3548992
| /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
|
| 7ec3d1b05000-7ec3d1b07000 r--p 00036000 fc:00 3548992
| /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
|
| 7ec3d1b07000-7ec3d1b09000 rw-p 00038000 fc:00 3548992
| /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
|
| 7ffd266f8000-7ffd26719000 rw-p 00000000 00:00 0 [stack]
|
| 7ffd2678a000-7ffd2678e000 r--p 00000000 00:00 0 [vvar]
|
| 7ffd2678e000-7ffd26790000 r-xp 00000000 00:00 0 [vdso]
|
| ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0
| [vsyscall]
|
| ___
|
| Each line is a memory mapping. The first field is the start
| address. The second field is the end address. So an entry
| like
|
| 7ffd266f8000-7ffd26719000
|
| means "this mapping covers virtual addresses from
| 0x7ffd266f8000 up to 0x7ffd26719000."
|
| The addresses are always increasing:
|
| - left to right: within a single line you go from lower
| address to higher address
|
| - top to bottom: as you go down the list you also go to
| higher and higher addresses
|
| Exactly like reading a book: left to right and then top to
| bottom.
| 1718627440 wrote:
| The issue amitprasad is pointing out is when you read
| addresses byte-wise and you determine that they are in
| little-endian.
| bignerd_95 wrote:
| If you're referring to little-endianness, it means the CPU
| stores multi-byte values in memory with the least significant
| byte first (at the lowest address).
|
| This convention started on early Intel chips and was kept for
| backward compatibility. It also has a practical benefit: it
| makes basic arithmetic and type widening cheaper in hardware.
| The "low" part of the value is always at the base address, so
| the CPU can load 8 bits, then 16 bits, then 32 bits, etc.
| starting from the same address without extra offset math.
|
| So when you say an address like 0xABCD shows up in memory as
| [0xCD, 0xAB] byte-by-byte, that's not the address being
| "reversed". That's just the little-endian in-memory layout of
| that numeric value.
|
| There are also big-endian architectures, where the most
| significant byte is stored at the lowest address. That
| matches how humans usually write numbers (0xABCD in memory as
| [0xAB, 0xCD]). But most mainstream desktop/server CPUs today
| are little-endian, so you mostly see the little-endian view.
| amitprasad wrote:
| Not so much the confusion of _what_ little endian is, but
| how we tend to represent it in notation. Of course this
| confusion was back when I was first learning things in high
| school, but I imagine I'm not alone in it
| krackers wrote:
| The stack does grow down though no matter what, in the sense
| that the pushing decrements the stack pointer. You can
| represent this as "up" in your diagram, but I don't think this
| makes it any easier conceptually because by analogy to a simple
| push/pop on an array, you'd naively expect higher addresses to
| contain more recent stack contents.
|
| The core of the issue is that the direction stack growth
| differs from "usual" memory access patterns which usually
| allocate from lower to higher addresses (consider array access,
| or how strings are laid out in memory. And little-endian
| systems are the majority)
|
| But if we're going with visualization options I prefer to
| visualize it horizontally, with lower addresses on left. This
| has a natural correspondence with how you access an array or
| lay out strings in memory.
| bignerd_95 wrote:
| Please try to draw, step by step, a process where lower
| addresses are at the top and higher addresses are at the
| bottom. You'll see that this makes everything much easier to
| understand.
|
| Do not confuse this with push and pop on an abstract stack
| data structure. That is not the same as the process stack. On
| a real process stack, newer data is stored at LOWER
| addresses. In fact, every push decrements the stack pointer
| (the address is decreased).
|
| If you want an example, think about how a string is placed
| and accessed on the stack. First, the stack pointer is
| decremented to reserve space (so in my diagram this "moves
| up" visually). Then the string can be read byte by byte by
| incrementing an index from the lower address toward the
| higher address. This is exactly like reading a book: left to
| right, top to bottom. If you flip memory upside down,
| everything becomes unnatural to understand: you would have to
| read the string from the bottom to the top.
|
| Try decompiling a program with Ghidra. Open the disassembly
| view and look at the addresses on the left. Lower addresses
| are shown at the top. Higher addresses are shown at the
| bottom. In my diagram this matches perfectly. Everything is
| consistent and you never have to mentally flip the memory
| layout.
|
| Years of practice led me to this, not just theory.
| ramanvarma wrote:
| did you see the relocations for the main binary applied before or
| after the linker resolves its own symbols? the ordering always
| feels like black magic when you step through it in a debugger
| matheusmoreira wrote:
| Hacking this stuff is _so_ fun!!
|
| > Depending on your program, _start may be the only thing between
| the entrypoint and your main function
|
| I once developed a liblinux project entirely built around this
| idea.
|
| I wanted to get rid of libc and all of its initialization,
| complexity and global state. The C library is so complex it has a
| primitive form of package management built into it:
|
| https://blogs.oracle.com/solaris/post/init-and-fini-processi...
|
| So I made _start functions which did nothing but pass argc, argv,
| envp and auxv to the actual main function:
|
| https://github.com/matheusmoreira/liblinux/blob/master/start...
|
| https://github.com/matheusmoreira/liblinux/blob/master/start...
|
| You can get surprisingly far with just this, and it's actually
| possible to understand what's going on. Biggest pain point was
| the lack of C library utility functions like number/string
| conversion. I simply wrote my own.
|
| https://github.com/matheusmoreira/liblinux/tree/master/examp...
|
| Linux is the only operating system that lets us do this. In other
| systems, the C library is part of the kernel interface. Bypassing
| it like this can and does break things. Go developers once
| discovered this the hard way.
|
| https://www.matheusmoreira.com/articles/linux-system-calls
|
| The kernel has their own nolibc infrastructure now, no doubt much
| better than my project.
|
| https://github.com/torvalds/linux/tree/master/tools/include/...
|
| I encourage everyone to use it.
|
| Note also that _start is an arbitrary symbol. The name is not
| special at all. It's just some linker default. The ELF header
| contains a pointer to the entry point, not a symbol. Feel free to
| choose a nice name!
| yawpitch wrote:
| You've got a broken link in your markdown, round about the phrase
| "lang_start function (defined here)".
| Animats wrote:
| From the title, I thought this was going to be about the parts of
| a program that run before the main function is entered. Static
| objects have to be constructed. Quite a bit of code can run.
| Order of initialization can be a problem. What happens if you try
| to do I/O from a static constructor? Does that even work?
| amitprasad wrote:
| This is heavily language runtime dependent -- there's nothing
| that fundamentally stops you from doing anything during the
| phase between jumping to an entry point and the main()
| abnercoimbre wrote:
| Indeed the craziest among us occasionally abuse this fact, so
| long as the compiler implementation lets us.
| Animats wrote:
| Right. This tends to come up with packages, which just by
| virtue of being loaded, set up to do something such as log,
| print, catch errors, or phone home to something.
| nneonneo wrote:
| For a fun example of a crash that can occur before main() even
| starts: https://stackoverflow.com/questions/12570374/floating-
| point-...
|
| The poster was receiving a SIGFPE (floating point exception) on a
| C program that is simply "int main() { return 0; }". A fun little
| mystery to dive into!
___________________________________________________________________
(page generated 2025-10-26 23:01 UTC)