[HN Gopher] Patching GCC to Build Portable Executables
___________________________________________________________________
Patching GCC to Build Portable Executables
Author : latchkey
Score : 83 points
Date : 2023-07-13 19:45 UTC (3 hours ago)
(HTM) web link (ahgamut.github.io)
(TXT) w3m dump (ahgamut.github.io)
| develatio wrote:
| There is a ticket in Go's repo about this:
| https://github.com/golang/go/issues/51900
| yjftsjthsd-h wrote:
| In case the author sees this:
|
| > Of course, my patch isn't perfect. It can't handle some
| anonymous structs, enums, const ints, or amazing things like
| using SIGILL as an array index [...]
|
| Is "SIGILL" a typo?
| hkgjjgjfjfjfjf wrote:
| [dead]
| inglor_cz wrote:
| The SIGILL signal is raised when an attempt is made to execute
| an invalid, privileged, or ill-formed instruction. SIGILL is
| usually caused by a program error that overlays code with data
| or by a call to a function that is not linked into the program
| load module.
|
| [0]
| https://support.sas.com/documentation/onlinedoc/ccompiler/do...
| spacechild1 wrote:
| Another common cause is trying to execute instructions that
| the CPU doesn't support, e.g. trying to run AVX2 code on a
| machine that only supports SSE4.
| inglor_cz wrote:
| Yeah, the only situation when I saw SIGILL "live" was my
| attempt to run some AES instructions on an emulator that
| didn't support them.
| yjftsjthsd-h wrote:
| Thanks, didn't know that was an actual signal!
| fit2rule wrote:
| [dead]
| 1vuio0pswjnm7 wrote:
| "But I've spent a good chunk of time removing obvious
| counterexamples, and a lot of popular software builds
| seamlessly."
|
| Publishing a list of software that will successfully compile
| might be helpful. Perhaps this already exists.
| ahgamut wrote:
| You can build software like vim, emacs, ninja, bash, git, gcc
| etc -- here's a list of software I got to build with this
| technique: https://github.com/ahgamut/superconfigure
|
| The superconfigure script is just a wrapper around the usual
| configure script used to build your software, supplying flags
| like --enable-static.
| appleflaxen wrote:
| This is incredible cool.
| KerrAvon wrote:
| The switch-to-if thing is pretty inelegant. It seems like you
| could fix this by mapping to a platform-neutral set of constants,
| so you could dereference the runtime symbol with a function --
| `switch(CosmoMappErrno(EINVAL))` and `case COSMO_EINVAL:` and so
| avoid turning the code into goto mush.
| ahgamut wrote:
| Dereferencing the runtime symbol with a function sounds
| interesting! If you can show me an example of where it works,
| I'd be happy to try it out. I like the if-else-goto arrangement
| because it fit in perfectly with the other parts of gcc -- if
| you look at my patch[1], you will find that I had to change
| very little of gcc's existing code to add this capability.
|
| [1]: https://github.com/ahgamut/gcc/tree/portcosmo-11.2
| outworlder wrote:
| Do we even care if the replacement is done automatically?
| jsmith45 wrote:
| But that won't help with compiling unmodified code. Nobody
| should worry about gotos being created inside the compiler. It
| is doing that all over the place anyway.
|
| Unless of course you mean that the compiler should be
| recognizing switches like this, and instead of rewriting them
| to if trees, it should be rewriting them to switches, changing
| the labels to use special cosmo specific constants for each of
| the values, and wraping the input value to the switch with a
| call to a function that maps the current runtimes platform's
| values for these over to corresponding cosmo specific constants
| (and letting other values pass though unchanged).
|
| That... actually might be a simpler compiler transformation to
| implement. There would be complexity in needing to recognize
| which of the multiple sets of of constants are being used, and
| applying the right call or calls to the switch input to map
| them. It also require complexity on the library side though
| (creating these ). Not sure if the author would want to
| implement it, given they have a working implementation of the
| other transformation.
|
| Lastly such a mapping approach would have risks of
| inappropriately mapping, or trickiness like having to invert
| the input before mapping for code that does the negative errno
| value thing.
| charonn0 wrote:
| N.B.: "Portable Executable" or "PE" is also the name of the
| Windows executable format.
| epcoa wrote:
| Looks like they changed the story headline to Actually Portable
| Executables (which is the original "correct" name). Other than
| an audio format relatively few use I suppose APE isn't the most
| clashy file format name.
| 1vuio0pswjnm7 wrote:
| Question: How are "PE" portable. Is it a Microsoft-centric
| definition, ignoring the existence of other OS. Any interesting
| history behind the term.
|
| I'm more familiar with the meaning of portable as it appears to
| be used in APE. For example,
|
| . 151 "portable" foldoc "The Free On-line Dictionary of
| Computing (30 December 2018)"
|
| portability
|
| portable
|
| <operating system, programming> The ease with which a piece of
| software (or {file format}) can be "ported", i.e. made to run
| on a new {platform} and/or compile with a new {compiler}.
|
| The most important factor is the language in which the software
| is written and the most portable language is almost certainly
| {C} (though see {Vaxocentrism} for counterexamples). This is
| true in the sense that C compilers are available for most
| systems and are often the first compiler provided for a new
| system. This has led several compiler writers to compile other
| languages to C code in order to benefit from its portability
| (as well as the quality of compilers available for it).
|
| The least portable type of language is obviously {assembly
| code} since it is specific to one particular (family of)
| {processor}(s). It may be possible to translate mechanically
| from one assembly code (or even {machine code}) into another
| but this is not really portability. At the other end of the
| scale would come {interpreted} or {semi-compiled} languages
| such as {LISP} or {Java} which rely on the availability of a
| portable {interpreter} or {virtual machine} written in a lower
| level language (often C for the reasons outlined above).
|
| The act or result of porting a program is called a "port". E.g.
| "I've nearly finished the {Pentium} port of my big bang
| simulation."
|
| Portability is also an attribute of {file formats} and depends
| on their adherence to {standards} (e.g. {ISO 8859}) or the
| availability of the relevant "viewing" software for different
| {platforms} (e.g. {PDF}).
|
| (1997-06-18)
| throwaway8356 wrote:
| Portable is a generic term. The PE format is the same for x86
| and ARM64 and other architectures.
|
| https://learn.microsoft.com/en-us/windows/win32/debug/pe-
| for...
|
| > The name "Portable Executable" refers to the fact that the
| format is not architecture specific
|
| https://en.m.wikipedia.org/wiki/Portable_Executable
|
| > On Windows NT operating systems, PE currently supports the
| x86-32, x86-64 (AMD64/Intel 64), IA-64, ARM and ARM64
| instruction set architectures (ISAs). Prior to Windows 2000,
| Windows NT (and thus PE) supported the MIPS, Alpha, and
| PowerPC ISAs. Because PE is used on Windows CE, it continues
| to support several variants of the MIPS, ARM (including
| Thumb), and SuperH ISAs.
|
| In my opinion it qualifies as portable
| dataangel wrote:
| Is there any practical purpose for APE? Like who is shipping
| single executables that need to run on N OSes?
| Timon3 wrote:
| I'm trying to release future side projects that way. I even
| build a Redbean executable for any client-only SPAs. The idea
| of this stuff _just working_ on so many systems is amazing.
| nicoburns wrote:
| Problem with this is that self-modifying code will likely
| trip security blocks on most platforms, which is probably
| more annoying for users than having to download separate
| executables.
| jcelerier wrote:
| i've been asked this hundreds of times in my life, by
| customers, bosses, etc...
| jjnoakes wrote:
| Storing executables on file systems shared across operating
| systems or architectures gets a lot easier if you don't have to
| have special paths for each os+arch combination.
| saagarjha wrote:
| Doesn't the executable get overwritten after first
| invocation?
| ahgamut wrote:
| I wrote this post: the title should be "Patching GCC to build
| Actually Portable Executables", because it refers to Cosmopolitan
| Libc and jart's Actually Portable Executable format.
|
| With my gcc patch, you can now build software like vim, emacs,
| ninja, bash, git, gcc etc with Cosmopolitan Libc, via their usual
| autotools/cmake-style build system. The built executables should
| run on Linux, FreeBSD, MacOS, OpenBSD, NetBSD, and also Windows
| (although I haven't tested Windows yet.)
|
| Here's a list of software I got to build with this technique:
| https://github.com/ahgamut/superconfigure
|
| The superconfigure script is just a wrapper around the usual
| configure script used to build your software, supplying flags
| like --enable-static.
|
| If you want to build gcc using Cosmopolitan Libc -- try out this
| repo: https://github.com/ahgamut/musl-cross-make/tree/gccbuild
___________________________________________________________________
(page generated 2023-07-13 23:00 UTC)