[HN Gopher] Single binary executable packages
___________________________________________________________________
Single binary executable packages
Author : feross
Score : 130 points
Date : 2022-03-02 15:46 UTC (7 hours ago)
(HTM) web link (notes.volution.ro)
(TXT) w3m dump (notes.volution.ro)
| ComradePhil wrote:
| The author doesn't seem to understand linux containers.
|
| > docker run just-another-2g-vm-image-running-a-fully-fledged-os-
| with-all-the-systemd-glory-masked-as-a-container;
|
| Docker doesn't run a "vm" image and it definitely doesn't run
| another kernel (the kernel isn't included in docker images
| because it's designed to run on the host kernel). It's more
| closer to chrooted process (if you just consider the process)
| than anything... combined with some resource isolation.
|
| > that certainly won't lose any data when you purge it by mistake
|
| If you don't understand what you are doing, you can "purge data
| by mistake" doing other things as well.
|
| Docker images come with ALL the pros listed in the article with
| fewer cons. For example, they are isolated from the get go by
| design, so fewer security concerns than running a binary on your
| system a your user without namespace isolation (namespace
| isolation is what's happening with docker). You can choose what
| files and devices process running in docker has access to.
|
| The download size may be bigger than a statically linked single
| executable... but if you are running a lot of different processes
| in docker, they may share the underlying layers which may mean
| that you end up using less disk space overall.
| ciprian_craciun wrote:
| > The author doesn't seem to understand linux containers. >
| Docker doesn't run a "vm" image and it definitely doesn't run
| another kernel [...]
|
| Yes I do understand them!
|
| In fact, in a former life (i.e. during the long forgotten time
| when Linux containers were just a novelty and LXC was seen as
| the "future") I was a big proponent of containers.
|
| What I was not a big supporter of is containers, that instead
| of being as you well say "more closer to chrooted processes",
| instead they are almost stand-alone VM images lacking a kernel.
| In fact Fly.io does exactly this -- they take a Docker
| container, add a Linux kernel, and run everything in a
| Firecracker "micro" VM.
|
| ----
|
| > that certainly won't lose any data when you purge it by
| mistake > If you don't understand what you are doing, you can
| "purge data by mistake" doing other things as well.
|
| Yes, butter-fingers are a thing... But with Docker it's so easy
| to delete a container by mistake (that although shouldn't have
| state it does); on the other side, if you are about to issue
| `rm -R -f .` and you are inside `/var/lib/mysql` you do have
| quite a few opportunities that perhaps you are doing something
| stupid (you have to explicitly write `rm`, add `-R` and
| manually type `/var/lib/mysql`)...
|
| ----
|
| > The download size may be bigger than a statically linked
| single executable... but if you are running a lot of different
| processes in docker, they may share the underlying layers which
| may mean that you end up using less disk space overall.
|
| OK, say that for "server applications" (that happen to use tens
| of micro-services) this makes sense.
|
| But how about a "tool application", say a static-site builder;
| does it now make sense to use Docker to run this tool?
| elitepleb wrote:
| actually pdrtable executables are this and more
|
| https://justine.lol/ape.html
| ihateolives wrote:
| Oh god. I wish people woukd just stop trying to be cool.
| Actmally pdrtable ehecmtable? This is prime
| https://www.reddit.com/r/grssk/ material.
| jart wrote:
| I love how the unconventional name lets me know which people
| think that names are what matters.
| cpach wrote:
| It's an extremely cool concept. I don't really see any reason
| to use it in prod though.
| dec0dedab0de wrote:
| It would be nice as an all in one installer, if you have an
| app you need to run on windows mac and linux in your
| enterprise, you could have one task in your ansible playbook,
| or whatever you use. without having to make sure you copy the
| correct one.
|
| Unfortunately it would also would be really good for viruses.
| AdamH12113 wrote:
| The convenience of binary executables is pretty clear in the DOS
| and Windows ecosystems, where games and native utilities have
| been distributed this way for decades. I have Windows binaries
| I've been carrying around for twenty years that still work on
| Windows 10, and old DOS programs run just fine in DOSBox. A key
| characteristic of such programs is that they include most of the
| libraries they use in the installation, with only a few common
| dependencies like the C runtime or Win32 left to the rest of the
| system. FFMPEG is a great example.
| nusaru wrote:
| Meanwhile, Apple has gone through two deprecations of CPU
| architectures: first PowerPC and then Intel. Macs are nice, but
| I wish they had the same commitment to backwards compatibility.
| spijdar wrote:
| Well, 3 actually, you missed the 68000 architecture.
|
| I'm not sure it really matters, in the end. How often would
| people want to run PowerPC-era Mac software on modern
| hardware natively if they could? The HN audience will include
| a disproportionate number of people who do, so you guys
| aren't included ;-)
|
| I haven't used a Mac in several years but it seems like the
| whole Apple mindset is more appliance like, that if you
| bought old software you just keep running it on the original
| old hardware, and given the problems with Microsoft's eternal
| backwards compatibility approach (applications and the OS
| bundling forever ancient code and keeping ancient interfaces
| around forever, and ending up with a disproportionate number
| of API calls numbered foo2, foo3, foo4ex), I'm not sure
| either way is strictly superior to the other. There are
| trade-offs with both.
| torginus wrote:
| Not PowerPC, but 32-bit programs are no longer supported
| since Big Sur - since they decided to no longer ship 32 bit
| libraries. There's software on the Mac that's 4-5 years old
| and cannot be run.
|
| And it's not an academic one - a large percentage of
| somewhat older Mac-native Steam games don't even start.
| userbinator wrote:
| 4 if you count 6502 (Apple II).
| throwawayboise wrote:
| > How often would people want to run PowerPC-era Mac
| software on modern hardware natively if they could?
|
| I think some creative people, e.g. writers, might like to
| be able to continue to run their favorite old word
| processors if they could. Every so often you'll hear about
| a writer who's still using Wordstar or WordPerfect or some
| other old software because they know it inside out and it
| does what they need.
| spijdar wrote:
| Oh, definitely. I'm just wondering how many Mac people
| would want to run said software on new hardware, as
| opposed to using their old hardware.
|
| Of course, using old hardware has plenty of disadvantages
| (the increasing scarcity over time as parts become
| impossible to acquire) but even now you can find power
| macs for cheap. Your average ebay listing is pretty
| ridiculous but I snatched a G4 ibook there 2 years back
| for $14 USD, essentially in like new condition.
| abruzzi wrote:
| There are some universes where old software or hardware is
| still heavily used or valuable. One area where I dabble is
| electronic music. Sure there are lots of new things, but a
| 50 year old guitar is still a viable instrument, so why not
| a 30 year old synthesizer? So in this universe, some people
| go to great lengths to maintain software that supports
| these instruments. I have a Yamaha VL1. Released in the mid
| 90s, it was Yamaha's flagship "tech preview" for waveguide
| physical modeling software. You could play the instrument
| and tweak some very basic parameters without a computer,
| but if you want to actually edit the modeling, you need the
| "expert editor" which requires MacOS System 7 and a direct
| serial MIDI port or Apples long gone "midi manager"
| software. I keep an old 68k laptop around for that purpose
| alone.
|
| Korg released an audio DSP playground in a PCI card called
| the OasysPCI which never got OSX drivers, so I have a MDD
| PPC Mac to run that. There are probably better things
| running natively today, hey instruments are things that
| shouldn't be obsoleted, since they all have their own
| sound. Most of the rest of the ancient software I run is
| run under Wine (which worked quite well up to Mojave, but
| became difficult when Apple killed 32 bit support) because
| Microsoft has done better at retaining backwards
| compatibility. So now I have a laptop permanently stuck on
| Mojave.
| ketralnis wrote:
| Hell, iOS will refuse to run binaries compiled as little as
| two years ago
| alerighi wrote:
| Well, you can do the same on Linux. The Linux kernel system
| call ABI doesn't usually break support for old software, and as
| long as you have a statically linked binary you can run even
| software that was compiled targeting Linux 2.4 (if I recall
| correctly).
|
| The fact is that is usually done in the Linux ecosystem: if you
| have access to the source code, what is the pro? It's just
| simple to recompile the software targeting the release of the
| operating system you are using. By dynamically linking system
| libraries you will get smaller executable and save RAM since
| they are shared (not only on disk, but also in memory, that is
| what shared means!)
| pinephoneguy wrote:
| That's really not a great idea on Linux. A _ton_ of stuff is
| handled in userspace. There 's really no better way to
| distribute Linux apps than source code.
| SCHiM wrote:
| Well, the problem is that it's not simple to recompile.
| There's a number of different build systems. Dev libraries
| you might not have installed, and that might no longer exist.
| Warnings that turned into errors in your version of the
| compiler, and all sorts of other nastiness.
|
| I dread every time I need to install a (older) package from
| source. I think around a quarter of the time it turns into a
| multiple hour adventure of frustration, only to discover the
| library it was missing is called $NAME-dev on my distro...
| alerighi wrote:
| As far as I know gcc doesn't turn warnings into errors from
| one release to another. In fact the default is that no
| warning is considered an error, unless you use options to
| force a different behavior. This is something that Apple
| seems to do (last time I used a mac I've seen that the
| Apple compiler that is a proprietary build of clang have
| some warning considered errors enabled by default), but I
| never seen in any Linux distribution.
|
| For code, you can compile by specifying the correct C
| standard. Unless the build system is badly written and
| doesn't add the correct -std=cXY to gcc of course (but it's
| trivial to fix).
|
| Regarding libraries, that is a real problem. Because to
| compile and old software you should use old version of the
| libraries, and they are usually not easy to get (and then
| it's a pain to compile them, and put the correct
| environment variables to make the software you are building
| link to that libraries and not the one in the system).
|
| I'm always able to compile a software, even an old one, on
| my system, and it never took me more than a couple of
| minutes. But I recognize that I'm a pretty experienced
| Linux user, so sure for the average user having a binary
| package is more easy.
| Quekid5 wrote:
| I'm guessing that they're talking about -Werror. (Which
| should never be specified in the build config you
| distribute, but is fine for tightly controlled
| environments like your DEV and CI environments.)
| saurik wrote:
| > ...only to discover the library it was missing is called
| $NAME-dev on my distro...
|
| You lost me at this part: if the dependency is easy to
| install and the issue is you just don't know that all of
| the headers are in -dev packages this seems like the
| easiest problem to never have again. Really: right up until
| there I was rooting you on.
| bombela wrote:
| I have been personally and my whole professional life on
| Linux for 15y. And I have the exact same feeling when I
| have to compile something.
|
| The waste of time to compile anything is staggering. And
| more often than not I give up on failure after 2h. Because
| one of the recursive depency is impossible to build. Or I
| got tired of git cloning or wget'ing 50 differents stuff.
| userbinator wrote:
| For me, close to 30 years as some small utilities I wrote when
| I first started Win32, I still use regularly. Most of them are
| below 100KB and link to MSVCRT, the system C library.
| phkahler wrote:
| Solvespace (CAD program) comes as a single executable for
| windows: https://solvespace.com/index.pl
|
| I dread the day we have to put together an installer for it. If
| we ever decide to use GTK on all platforms we'll have to since
| the GTK developers don't seem to think static linking is a
| useful thing for them to support.
| jeroenhd wrote:
| GTK on Windows is one of those things that irk me to no end.
| I've installed several tools that come with the entirety of
| GTK (and even a dbus instance that tries to start on boot) in
| the program files folder for that specific program.
|
| The GTK folks oppose static compilation, and that's a
| perfectly fine point of view because they can support their
| opinions quite well. However, without an official "Microsoft
| visual c++ runtime" equivalent, distribution of GTK apps is
| just so annoying on Windows! You can get the entire GTK
| library suite just fine on most Linux distros but when it
| comes to non-Linux platforms, you're left to your own
| devices, and that means every platform comes up with its own
| solution, incompatible with the rest.
| mywacaday wrote:
| Not quiet single binaries but https://portableapps.com/ has been
| a great resource over the years
| mwcampbell wrote:
| > I don't think a rich GUI application like LibreOffice, Firefox,
| Thunderbird, Slack, Discord, Skype, GMail, etc., can be delivered
| as a single executable.
|
| The old Delphi-based Skype for Windows _was_ a single executable.
| At least, most of it was in that big .exe.
| smasher164 wrote:
| Right, but applications can treat win32.dll as essentially a
| system-call layer. Windows also gives you a strong
| compatibility guarantee with those libraries.
|
| Userspace Linux is different in that not only are these
| libraries typically installed separately, they all have varying
| degrees of compatibility between versions.
| dheera wrote:
| Yeah, anything can be delivered as a single executable, it just
| requires some tools to package it as such.
| chazeon wrote:
| I really hope I can create executable packages, but when some
| bulky math libs in Python are involved, it really becomes very
| hard.
| dheera wrote:
| You can include all the bulky math libs in the executable. That
| way if you want some version of Tensorflow and some other
| executable wants some other version of Tensorflow, there isn't
| a conflict.
|
| IMO it is no problem if your software is several hundred
| megabytes in size, I have 8 TB of hard drive. Storage is cheap
| as long as you aren't using Apple.
| selfhoster11 wrote:
| I've got a 256GB SSD as my boot drive. Storage is not cheap.
| immibis wrote:
| Until your bundled version of Tensorflow isn't compatible
| with your installed version of CUDA.
| dheera wrote:
| So have both CUDA and non-CUDA versions bundled, along with
| the specific CUDA and cudnn version it needs. Still not
| that large. Maybe a couple gigs at most for everything, and
| a LOT of headache saved in return which I think is a fine
| deal.
|
| I mean, very often the Tensorflow version a piece of
| software wants _doesn 't_ actually have a version available
| for my currently-installed CUDA version. I'll have CUDA
| 10.1, 11.0, and 12.6 already on my system for other pieces
| of software, and your software is hellbent dependent on
| Tensorflow 2.3 and CUDA 11.1 and I'll end up having to
| install 11.1 in addition anyway just to run your software.
| immibis wrote:
| Until your bundled version of CUDA isn't compatible with
| your installed graphics card.
| dheera wrote:
| If so you wouldn't be able to run it anyway and you'd
| have to go to CPU only, whether or not you bundle stuff.
| mjg59 wrote:
| Simultaneously arguing that Flatpak isn't the future because it
| relies on runtimes that may duplicate resources, and then
| suggesting that you should statically link in all your
| dependencies (creating far more duplication than Flatpak
| runtimes) seems... odd?
| stitched2gethr wrote:
| In practice I'm not sure there would be much duplication. Even
| for tools using the same libraries they are very often using
| different versions.
| teddyfrozevelt wrote:
| In practice, the deduplication is pretty good. I have 78
| Flatpaks installed (apps and runtimes), and here's my dedup
| stats (calculated from https://gist.github.com/powpingdone/00
| 1a46aa7db190b9c935f71c...):
| =========================================== no dedupe:
| 16.0 GB (17228518418 B) dedupe: 11.7 GB (12594871766
| B) singlelet: 8.1 GB (8677649129 B) orphan:
| 1.8 GB (1889889290 B)
| ===========================================
| deduplicated size ratio: 73.10 singlelet space usage:
| 68.90 singlelet file ratio: 63.56 orphan space
| usage: 15.01 orphan file ratio: 2.63
| cfstras wrote:
| What are the golang- packages about in OpenSUSE? Go doesn't
| support dynamic linking (except for non-go-pieces like glibc or
| via cgo). Do they contain source-code? Or pre-built .a's? Both
| would have a hard time not conflicting with go's own build
| tooling, right?
| ciprian_craciun wrote:
| I've just installed `golang-github-burntsushi-toml` that should
| be the well known Go TOML library; asking `rpm -q -l` shows
| mainly `*.go` files, thus I assume they are mainly used as
| build dependencies?
| cfstras wrote:
| Hmm... That makes sense. Though if you are building from
| source, the only gain you have over regular go tooling is
| downloading from your favorite mirror rather than github&co.
| I hope it doesn't install these source-packages for prebuilt
| binaries...
| stagger87 wrote:
| What about a fully self contained folder? No need to statically
| link everything, on Windows and on Linux you can dynamically link
| to local files in a provided folder. Plus the folder provides a
| place to store local/temp files as well. This is how I deploy
| software on Windows and Linux, and I find it saves a lot of
| headaches.
| liuliu wrote:
| Lack of compression (or at the mercy of the file system) tends
| to be an issue. That is why people choose to use squashfs and
| then have a writable overlayfs on top of it.
| dj_mc_merlin wrote:
| > Rust -- say goodbye to cross-compiling, but if you stay away
| from OS provided libraries, you are kind of covered;
|
| What? Rust can cross compile perfectly fine.
|
| > C / C++ -- certainly possible, but you'll need on retainer a
| UNIX graybeard, or at least an autotools-foo master;
|
| If you write C and don't know how to cross compile your code you
| shouldn't be writing it.
|
| > Python -- you're already covered, just read the zipapp
| documentation;
|
| Python actually can't cross build (reliably).
|
| > Java -- possible, but you'll need a startup script to just call
| java -jar some-tool.jar; (also not a good fit for short-lived
| tools, mainly due to startup times;)
|
| Use GraalVM.
| hathym wrote:
| > Python -- you're already covered, just read the zipapp
| documentation;
|
| pyinstaller --onefile will create a signle file executable with
| an embedded python runtime and libraries (no need to install
| python) https://pyinstaller.readthedocs.io/en/stable/usage.html
| phkahler wrote:
| >> If you write C and don't know how to cross compile your code
| you shouldn't be writing it.
|
| It would be great if GCC supported all target architectures out
| of the box, but what would be needed for multiple OS/platform
| support?
| mypalmike wrote:
| > If you write C and don't know how to cross compile your code
| you shouldn't be writing it.
|
| This statement is the programmer equivalent of manspreading.
| dj_mc_merlin wrote:
| I'll admit you have a point.
|
| I didn't mean to come off as elitist, but I have never met
| anyone I'd trust to write a C program that didn't have big
| obvious bugs that didn't know how to cross compile/statically
| link a C executable. If you haven't, that means you haven't
| spent enough time actually coding to be able to feel safe
| about what you're writing. Which is fine. But then use a
| language you know or spend more time learning about your
| tools before complaining. C is simple, everything around it
| is difficult.
| doliveira wrote:
| What's your experience with GraalVM? In mine, the need to
| painstakingly track every single reflection use by getting the
| stack traces at runtime (and never be quite sure if you got all
| of them) pretty much killed it for me
| [deleted]
| smasher164 wrote:
| Have you tried the Tracing Agent?
| https://www.graalvm.org/22.0/reference-manual/native-
| image/A...
|
| It will track all uses of reflection during a program's
| execution. You can then use the resultant config as the
| starting point for the reflect-config.json.
| doliveira wrote:
| Oh, that's nice. I was using it through Quarkus a while
| ago, not sure if I missed it or it wasn't available yet.
| That seems like a good starting point, although I assume it
| can't assure no runtime errors in production
|
| Edit: it seems to me that GraalVM would benefit from a
| repository of classes and corresponding reflection configs.
| Like DefinitelyTyped from the Typescript world
| allisdust wrote:
| That rust comment does sound weird. It's one of the easiest to
| compile across platforms. All you need is the rust toolchain
| and may be openssl sometimes.
| tekknik wrote:
| The easiest to cross compile would be golang, it's as easy as
| setting 1-2 environment variables. Last I used rust it
| required tool chain setup, which if you're not familiar with
| the process and be quite difficult.
| brettdong wrote:
| I tried cross-compiling a Rust project from ARM macOS host to
| ARM Linux target, compiling source files is still fine, but
| during linking stage it threw lots of lots of errors. And
| it's a pure Rust project without any system library
| dependencies. Seems that the system linker also needs to
| support the target platform, or cross compiling won't work.
| marcosdumay wrote:
| Rust has the option of using a different linker for each
| target platform.
|
| But of course, your linker must support it. I don't know
| where to get a linker for Linux ARM in macOS. On Debian
| it's apt install.
| pjc50 wrote:
| > If you write C and don't know how to cross compile your code
| you shouldn't be writing it.
|
| Cross-compiling in C is always miserable to set up, especially
| around the area of libraries. And it gets worse if one of the
| compilers is from a different vendor.
| qbasic_forever wrote:
| I don't think people realize just how much work tools like
| crosstool NG do to take away the pain of cross compiling. I
| remember trying to cross compile stuff in the early 2000s
| before all that tooling existed and.... oh boy, it was
| nightmare fuel. A good docker container setup makes it even
| easier these days.
| ciprian_craciun wrote:
| By cross-compiling one usually understands compiling for the
| *same OS* but different architecture. What about cross-
| compiling for a *different OS*?
|
| Can I *easily* compile a non-trivial C/C++ application from say
| Linux to OSX or OpenBSD, for simplicity say all of them are x86
| 64bit? (Bonus point if the application in question happens to
| use LMDB or another non-trivial library that plugs intimately
| into the OS API.)
|
| About Rust, I've once tried to compile from Linux to OSX a
| quite trivial application (basically a glorified `md5sum`), and
| the experience wasn't pleasant, and it involved lots of
| "internet searching" and basically stumbling around... (I don't
| remember if I succeeded, but I definitely wouldn't try it
| again.)
| MaulingMonkey wrote:
| > By cross-compiling one usually understands compiling for
| the _same OS_ but different architecture.
|
| I don't even consider that to rise to the level of "cross
| compiling".
|
| Getting started with emscripten to target WASM for C and C++
| is rather a chore of dependency wrangling IME. Targeting WASM
| from Rust, OTOH, is trivial. Targeting windows from linux
| with Rust is also quite straightforward, as has been
| experimenting with targeting consoles or Android from
| Windows.
|
| Targeting a MIPS32 OpenDingux target from Windows was much
| more of a chore. The toolchain with libs, headers, etc. that
| I used is just a *.tar.bz2 that expects to be extracted to
| /opt/gcw0-toolchain of a linux distro specifically, and
| embedded absolute paths all over the place make changing that
| difficult. I do resort to WSL on Windows, basically only
| because of those embedded paths:
| https://github.com/MaulingMonkey/rust-opendingux-test
|
| Acquiring the appropriate libs and headers to link/compile
| against for cross compiling is always an adventure, but Rust
| isn't making things any worse IME.
| fwsgonzo wrote:
| Last time I tried to cross compile RISC-V with Rust it just
| didn't work at all. I even know how linkers work and tried my
| best with custom linker scripts and arguments, but the
| toolchain would not build a rv64gc. Perhaps they have fixed the
| issues now, though.
|
| I would like to add that Zig is extremely cross-compilable. You
| can even use it to cross compile C/C++ projects.
|
| And Nim is fairly easy to cross compile. Just output C or C++
| and cross compile that. I don't know why people are saying that
| cross compiling C or C++ is so miserable. Yes, it's not always
| a one-liner, but that depends on the toolchain. For example,
| there is crosstool-ng. Also Linux distros tend to have cross
| compilers for many architectures right on the box these days.
|
| Here is how to cross-compile to 64-bit RISC-V on my Ubuntu
| machine: sudo apt install
| gcc-10-riscv64-linux-gnu g++-10-riscv64-linux-gnu
|
| Debugging RISC-V and a heap of other architectures:
| sudo apt install gdb-multiarch
| isomel wrote:
| Never done RISC-V, but for ARM, this is trivial, one just
| have to rustup the right target stdlib, and then pass
| --target to cargo and done. Well, at least that's that easy
| if what you try to compile don't have C dependencies. For C
| dependencies, there is cross <https://github.com/cross-
| rs/cross> which I had good experiences with.
| pizza234 wrote:
| > Last time I tried to cross compile RISC-V with Rust it just
| didn't work at all.
|
| YMMV; my experience was actually the opposite, approximately
| an year ago. I was actually really surprised of being able to
| build na executable for RISC-V (that I then copied to a VM,
| and run) all by adding 4 lines to Cargo's config.toml.
| lioeters wrote:
| > NodeJS -- technically, I know it's possible; but I can almost
| bet my right arm that given today's ecosystem it's not feasible
|
| Caxa - Package Node.js applications into executable binaries -
| https://github.com/leafac/caxa
| danenania wrote:
| pkg is another one: https://github.com/vercel/pkg
|
| It can cross-compile too.
| cle wrote:
| > Use GraalVM.
|
| GraalVM can't cross-compile. If you want to compile it for a
| particular platform, it must be compiled _on that platform_.
| fxtentacle wrote:
| I agree 99% with this article, but the part about packaging C++
| is wrong. I'd say use "vcpkg" (yes, it's Microsoft but also works
| on Linux) and then CMake. CLion is a nice IDE for that kind of
| stuff and it's reasonably easy to setup cross-compiling or GitHub
| pipelines with it.
|
| The result:
|
| C++ source pushed to git => GitHub pipelines => freshly compiled
| binaries for all platforms
|
| That said, I have also re-written some things in Go just for its
| ease of packaging and cross-compilation. The fact that libraries
| "just work" is like a Zen experience to me. Also, "go generate"
| really makes it easy to merge small resource files into your one-
| file binary app. BTW, I come from python and Ruby, where "WTF did
| mkmf.rb crash?" is a daily thing...
| at_a_remove wrote:
| I wholly agree, although single directory releases are also
| acceptable.
|
| Last shop I worked at was ... well, they were of an age and came
| from an era when installation meant "drop some files in a
| directory, set up an icon on a desktop, what do you mean
| 'dependencies' and 'framework' and 'installer'?" I did my best to
| accommodate, because that seems more reasonable to me, but it
| certainly wasn't simple.
|
| I do not dislike the concept of the Registry, it is an
| interesting tradeoff against the skillion dialects of .ini files
| and other configuration, but much further than that, it feels
| like installation of software is akin to a drop of ink in the
| water -- you can't really get it back out again.
| FpUser wrote:
| This is what I do. All my programs are single file exe. The worst
| case if something else is needed (like assets) that single exe
| retrieves it from the web and it's been like this forever
| buzzert wrote:
| You can also integrate the assets into the binary itself using
| something like gresources.
| immibis wrote:
| That's terrible. Your website will disappear and your
| executables will be useless.
| FpUser wrote:
| Nope. Once installed it does not need the Internet. If there
| is a connectivity it will use it, if not it will still work
| (well except the collaborative functionality).
|
| You are knocking wrong door. Try SAAS instead for your
| critic.
| pjc50 wrote:
| Not mentioned yet seems to be C#, which has the dotnet core
| "single file" packaging option. You need to read the small print
| on that, though, as it (a) unpacks into a temp dir at runtime and
| (b) last time I tried required a couple of .so files alongside
| the executable, missing the point entirely.
| ripley12 wrote:
| This is a little out of date; the .NET single-file situation is
| pretty good as of .NET 5 (which came out in 2020).
|
| These days, if you publish a single-file app all .NET libraries
| live in the executable and do not need to be unpacked at
| runtime. Native libraries still need to get unpacked at runtime
| (if your application uses any; many don't), but you just set
| the IncludeNativeLibrariesForSelfExtract build property and
| everything else happens automatically for you.
|
| I'm working on getting IncludeNativeLibrariesForSelfExtract set
| by default, upvote here if you think that's a good idea:
| https://github.com/dotnet/sdk/issues/24181
| goodpoint wrote:
| 1) Bundling dependencies into a blob is a disaster for security
| and maintainability.
|
| The blob might be a statically-linked executable, a flatpak, a
| docker container and so on.
|
| All this methods go against OS-based packages and prevent end
| users to update vulnerable libraries.
|
| It is well proven that things like containers on docker hub age
| like fish, leaving many unpatched images around.
|
| This is why dynamic linking and Linux distributions exist! Stop
| reinventing the wheel in worse way.
|
| 2) Network bandwidth for updates is far from abundant for less
| wealthy 4 billion people on the planet.
|
| 3) Same for drive space. A lot of phones/SBC/IoT devices have
| limited storage.
|
| 4) Same for I/O bandwidth. (E.g. even on a new PinePhone bulky
| applications take many seconds to start.)
| jcelerier wrote:
| > if possible, especially important for Linux with it's glibc
| variation across distributions, try static linking; works
| flawlessly with Go (including cross-compiling), somewhat with
| Rust / C / C++, and most-likely it's already fulfilled with
| interpreted languages like Python / Erlang / Deno / Java;
|
| it's not possible for GUI apps as you would have to link against
| the OpenGL drivers statically (and thus your software would only
| work on one brand of graphics card)
| adamddev1 wrote:
| I found myself wanting this for pandoc while trying to install it
| to a docker container. It would be nice if there was just a
| single binary I could download, but it seemed like all the
| installation methods used package managers or massively complex
| install steps?
| ohCh6zos wrote:
| I think the author raises some excellent points, but it still
| feels somehow wasteful to me.
| closeparen wrote:
| I think the opaque and esoteric nature of packaging, which
| makes is so damn hard to build packages like this, actually
| prompts incredible amounts of waste when companies need to
| deploy their own code on their own machines. You get
| ridiculously overpowered solutions - container schedulers and
| the like - when all that was really wanted was binary packaging
| + init scripts or service units.
| brimble wrote:
| I'm in the camp that uses Docker almost entirely as a cross-
| platform (cross-distro, at the very least), very clean (i.e.
| everything ends up in one place and the location of all data
| & config is very well documented and easily configured in a
| standard way for pretty much all packages) package manager.
|
| I could probably get similar benefits from learning, say,
| dpkg and systemd really, really well... but then that'd only
| work on dpkg-using Linux distros. Meanwhile, docker commands,
| docker-based shell scripts, and docker-compose files work
| just about identically ~everywhere, and can even work nearly-
| transparently on systems that don't actually support linux
| containers, like macOS and Windows, via virtualization and
| some command shims.
|
| I hardly even care that it uses containers. The parts of it I
| like don't actually require that--though they _do_ encourage
| people packaging for Docker to document things well and to
| cut through a certain amount of various packages ' config
| bullshit. Take Samba--it's like 100x easier to configure &
| manage for common use cases using Docker than most distros'
| default packages & config files. There's no actual reason
| that needs to be the case, it's just that the process of
| containerizing it brought that as a side-effect, but that's
| one of the _main_ reasons I 'm using Docker, not the
| containerization _per se_.
| ComradePhil wrote:
| I really wish Microsoft doesn't discontinue wsl1 and
| instead develops it further to make it the "Wine"
| equivalent (for running Linux apps on Windows without
| emulation or virtualization).
|
| Wsl2 relies on a hypervisor (i.e. it has to run a Linux VM
| underneath) so running docker apps on Windows comes with
| all kinds of problems from poor performance to file
| permissions being messed up.
|
| A mature wsl1 would be able to run almost any Linux app on
| Windows natively.
| gompertz wrote:
| May I suggest the utility Eee
| (http://www.erikveen.dds.nl/eee/index.html).... Combines multiple
| files into one executable. Works great!
| smasher164 wrote:
| 1. Not every library can be statically linked. Graphics drivers
| come to mind, but most libraries for the linux desktop _expect_
| to be dynamically linked (e.g. wlroots, libinput). So not only is
| there a technical limitation, but a practical one as well,
| considering that all of these projects would have to support it.
| These projects rely on dynamic linking as a way to support
| system-wide configuration.
|
| 2. Being able to patch dependencies is important. This isn't so
| much a feature to help developers, but one to help users. You
| could technically do this without dynamic linking, but have to
| make it easy to recompile a project with an updated dependency
| (which most package managers don't). Not being able to patch a
| package's dependencies puts the developer accountable for your
| system administration issues, which is not scalable.
| [deleted]
| _3u10 wrote:
| This is basically what docker does and it's a godsend.
| Banana699 wrote:
| >basically what docker does
|
| With the added hassle of installing Docker and learning and
| managing all of its details, and what you get in the end only
| works on Linux.
|
| Meanwhile every OS in existence support running its executable
| format with zero other bookkeeping.
| _3u10 wrote:
| Yeah, that's the crux of the issue. Windows mostly does
| installers to get around its problems with single binaries.
| Only OSX has a reasonably decent solution for copying apps
| between machines.
|
| On Linux the only reasonable solution is an
| install/executable script and/or docker.
| chaxor wrote:
| How do you ensure that the client has docker though? Make a
| cross compiled binary that downloads and installs docker and
| then runs your image for them? Run it through a browser on
| another server? The starting point there leads to similar
| problems.
|
| Also, docker seems to be on the way towards non free (or
| already in a?) /non-foss , so will probably soon be completely
| out of favor with everyone. Perhaps other tools like
| jails,bhyve, etc will rule the space soon.
| _3u10 wrote:
| Well, the vast majority of people aren't running operating
| systems that require the use of docker or even really support
| it. So now you're looking at 5% of users who would need to
| install docker.
|
| Of those users 90% are running Debian. So you insert apt
| install docker into your script. The remaining .4% are using
| red hat. So you insert yum install docker into your bash
| script.
|
| Then you add a line to start your image, now put the bash
| script on your website, voila, single file program that works
| anywhere.
| JMS2021 wrote:
| Except the version of docker that ships with Ubuntu apt is
| out of date. The snap install doesn't add your user to the
| docker group.
| _3u10 wrote:
| So is the kernel...
|
| Once you have root just do whatever you want. replace the
| binary, install another version, upgrade the kernel, and
| then run your app.
| stephc_int13 wrote:
| The *nix application packaging style was biggest disappointment
| when starting to play with Linux back in the 90s.
|
| I know there are many arguments to support this style, but I
| simply dislike it from an aesthetic point of view, too messy.
| bob1029 wrote:
| We don't have single _binary_ releases, but we do have single
| _folder_ releases that are guaranteed to work out of the box on
| any given windows server. For me, the number of actual files is
| not of consequence. Just being able to run something and know its
| going to run is the key bit.
|
| The new Self-Contained Deployments feature in .NET made our lives
| infinitely easier as a B2B software vendor. We no longer have to
| have sweaty-hands conversations with the customer about which
| .net frameworks are installed or even allowed (?!) in their
| environments. I literally click my mouse... 4 times in visual
| studio and have a zip file containing everything we need to send
| to the customer.
|
| We also moved to SQLite recently, so that one zip file is
| literally the entire kingdom. You could grab a blank VM from
| Azure and our product would startup and run on it without any
| extra bullshit.
| gameswithgo wrote:
| Yep, that is good enough. What makes Linux so horrible of an
| experience for many desktop users is so much of the software is
| built with a server mindset, with a vast array of dynamic
| linking that often does not work. If the software is an up to
| date part of your particular distro's package manager, great.
| But when it isn't, total nightmare.
|
| And your end user desktop application just doesn't need to be
| built that way!
| stronglikedan wrote:
| Is this the same approach as portable apps?
| ripley12 wrote:
| There's something nice about the ergonomics of a single file
| compared to a single folder though. A single file is trivial to
| distribute as-is; you don't even need to put it in an archive.
| You can put multiple single-file apps in a directory on your
| PATH, no futzing with symlinks etc.
|
| There are certainly tradeoffs involved and it's not always the
| right decision, but these days I usually try to publish
| software as a single file until I can't.
| bob1029 wrote:
| > There's something nice about the ergonomics of a single
| file compared to a single folder though
|
| Agreed. We are going to move to proper single file (or as
| close as we can get) as soon as any sort of meaningful AOT
| compilation is available:
|
| https://github.com/dotnet/runtimelab/tree/feature/NativeAOT
|
| https://github.com/dotnet/runtimelab/issues/248
|
| There is a "PublishSingleFile" option, but that is just a zip
| file in disguise.
| ripley12 wrote:
| > There is a "PublishSingleFile" option, but that is just a
| zip file in disguise.
|
| I don't quite agree. These days (as of .NET 5), if you
| publish a single-file app all .NET libraries live in the
| executable and do not need to be unpacked at runtime.
| Native libraries still need to get unpacked at runtime (if
| your application uses any; many don't) but I haven't found
| that to be a big deal in practice.
| bob1029 wrote:
| > and do not need to be unpacked at runtime
|
| Thanks for the correction/clarification. I had not
| realized this was a thing now. Will give this option
| another look.
| c0l0 wrote:
| I've argued this stance once before, but I feel it needs
| reiterating on this topic: I for one am 100% convinced that
| whatever the (FOSS) software future has in store for the world
| MUST have strong support for dynamic linking.
|
| If you look at something like `apt-cache show podman | grep
| ^Built-Using:` (Output: https://paste.debian.net/plain/1225449)
| on Debian 11, you will see why. Imagine a few of those components
| shared between tens of packages, and security problems discovered
| im some. It's got to be any package maintainer's worst nightmare.
| "Traditional" distros run by volunteers and in participation of
| small-ish companies will not be able to cope with that kind of
| need for rebuild churn - and yet they will live amongst us for
| many years, if not decades, to come.
| duped wrote:
| I've argued the opposite stance numerous times, because shared
| dependencies are far rarer than they might appear to be while
| requiring custom packaging for different distros is a burden I
| don't want to bear as a package author. I want to have one
| build for all distros, not custom packaging for every one that
| has a slightly different definition of how software is supposed
| to be packaged.
|
| In my experience, non-developer users don't care about package
| managers (they want it to be invisible) but do care about one
| click install. The only way I've found to satisfy those users
| is to distribute applications outside the distros' package
| managers and statically link everything as much as possible.
|
| It saves me and my users time and money. Package management
| with inverted dependency trees is user and developer hostile,
| and it's why shit like podman and docker have to exist in the
| first place.
| convolvatron wrote:
| maybe we can also mention that semver is more of a rough
| guidance than anything else. you dont really know if an X+Y
| that runs fine will really continue to run with X+Y' unless
| you tests that cover that workload and you've actually run
| them.
| infogulch wrote:
| First, can this not be reduced to an automation and compute
| problem? A new version of a library with a bug fix or security
| fix is released, so rebuild a bunch of packages. Where's the
| issue? Statically linked binaries can be produced from dynamic
| libraries, so the "rebuild a bunch of packages" can be further
| reduced to "re-link a bunch of packages". Optimize bandwidth
| consumption by building locally and validating against a
| reproduceable build transparency log.
|
| Second, shouldn't the majority of the time rebuilding packages
| be dedicated to _testing_ each application with the new
| library? Would you just skip that step if you used dynamic
| libraries?
| jahewson wrote:
| If traditional distos don't scale then perhaps we really should
| be looking at alternative approaches? We need systems that
| serve the users ahead of the maintainers.
| ciprian_craciun wrote:
| At the moment, given that the majority of open-source
| software is hosted on GitHub, perhaps GitHub releases (as in
| downloads from the releases tab) might be one such possible
| "alternative software repository". It's only missing an
| official "installer"; but there are a few unofficial
| alternatives like `wget` and `curl`. :)
|
| ----
|
| I'm not endorsing GitHub, I'm quite neutral about it, however
| at the moment it does offer a "standardized" experience. So
| much so, that whenever I see a project announced on HN and
| the link takes me to a landing page, I immediately look for a
| GitHub link and switch to that.
| ciprian_craciun wrote:
| [I'm the author of the article in question.]
|
| I want to highlight a few ideas that perhaps I didn't cover very
| well in the article:
|
| * static linking -- the concept of static linking appears in
| total 7 times in the article, once in the summary, once qualified
| with "if possible", three times as comments on a few example
| projects that do happen to employ it, and once discussing Go's
| advantage; the article is not about static linking!
|
| * dynamic liking -- it is completely fine, just don't depend on
| non-mainstream libraries, and don't require the version released
| yesterday as most stable distributions won't have it (for the
| next couple of years);
|
| * packaging and dependency minimalism -- as I've noted on
| Lobsters (<https://lobste.rs/s/adr60v/single_binary_executable_pa
| ckages...>), the article is mainly about the simplicity of
| building, packaging, publishing, deploying; try to read it both
| from the point of view of the developer and the user;
| ripley12 wrote:
| I enjoyed the article, thanks.
|
| I think you could add an entry to the "Languages suitable for
| single binary executables" section; these days, .NET (C#, F#)
| single-file applications work very well. This is a fairly
| recent development (single-file support has been good for ~1.5
| years now) but I've been very satisfied.
| https://docs.microsoft.com/en-us/dotnet/core/deploying/singl...
| mattgreenrocks wrote:
| > Languages suitable for single binary executables
|
| > C/C++: certainly possible, but you'll need on retainer a UNIX
| graybeard, or at least an autotools-foo master
|
| I don't understand this. The "C/C++ is black magic!!" meme just
| never seems to go away.
|
| They both support static linking and always have. It mostly comes
| down whether you dependencies will work with it, and setting
| appropriate compiler flags. In other words, not terribly
| different than anything other supporting language.
| doliveira wrote:
| > The "C/C++ is black magic!!" meme just never seems to go away
|
| It's true, though. Proof of that is how many tools are made
| basically because no one wants to deal with Makefiles and
| autotools. I'd argue that even Docker fits this category of
| tools made to hide C/C++ builds away
| jcelerier wrote:
| > It's true, though. Proof of that is how many tools are made
| basically because no one wants to deal with Makefiles and
| autotools.
|
| no one wants to deal with Makefiles and autotools because as
| of 2022 they don't support spaces in paths and the most used
| desktop OS on the planet happens to have spaces in the
| default path for installing software
| trinovantes wrote:
| IIRC Microsoft specifically added the space in the default
| path so that developers know they _need_ to handle file
| paths with spaces
| TillE wrote:
| Yeah you just...link to static libraries. Which have usually
| already been built and installed alongside the shared
| libraries.
|
| Also why would you be using autotools in 2022? CMake has been
| the de facto standard for about a decade now.
| bally0241 wrote:
| Came here to say this, anyone still using autohell in 2022
| either is a stubborn grey beard or is seriously masochistic.
| Building static or shared libs/programs with CMake is treated
| simply as an option when calling the linker. I personally
| don't know anyone sane who wants to deal with the mental
| gymnastics needed by libtool. And don't get me started on the
| stupid permissions games played by 'make distcheck' where a
| simple git --archive would do just fine.
| lenkite wrote:
| Using CMake is like trying to do one-handed pushups to
| setup your build. Use https://mesonbuild.com/ and make your
| life comfortable as a couch potato.
| mwcampbell wrote:
| One big problem is the global symbol namespace when statically
| linking. If I'm not mistaken, Rust and Go are able to work
| around this by mangling their symbols. But it's way too easy
| for a C or C++ library to pollute the global namespace.
| Granted, on Linux, the symbol namespace is also global when
| dynamically linking, but that's not the case on Windows; I'm
| not sure about Darwin.
|
| Combine that with the multiplicity of build systems, and it's
| easier to just use dynamic linking. Go and Rust solve that by
| having a standard build system.
| dboreham wrote:
| Hmm. I wrote C and C++ for 20 years and while I haven't used
| either in some time, I don't remember this ever being a
| problem, except when trying to port Unix code that expected
| errno to be global to Windows (but that's the opposite
| problem of the one you're talking about).
| rkeene2 wrote:
| Darwin also uses heirarchical namespaces by default; On Linux
| you can do this to a limited extent with dlmopen() but not as
| much as you may want (e.g., to link in a library linked to
| another libc -- but this is unique to Linux, most platforms
| only have the single libc)
| _3u10 wrote:
| It's probably more realistic to have a single folder executable
| like .app "files" on OSX.
|
| If the app has assets it gets a little crazy quite quickly.
| teakettle42 wrote:
| BeOS/Haiku appends assets to the end of the ELF file with a
| ToC and some easy-to-use APIs for accessing them.
|
| I always wish that'd taken off.
|
| I guess AppImage isn't dissimilar in that it appends a
| squashfs image to be mounted with FUSE.
| immibis wrote:
| Windows also does this. Linux doesn't, preferring to KISS
| because with demand paging there's really no reason you
| can't just put them in your rodata section.
| zozbot234 wrote:
| AppImage does pretty much that out of the box.
| rathboma wrote:
| AppImages are so good. They 'just work'. They're super easy
| to build and distribute too. I think more Linux
| distributions should embrace them as the de-facto app
| standard to replace snap and flatpak.
| AnIdiotOnTheNet wrote:
| There actually is/was two standards for this in Linux:
| GNUStep Application Bundles[0] and ROX-Filer AppDirs.
| Currently AppImage is trying to keep the dream alive, but it
| bundles the files in a squashfs image attached to the
| executable.
|
| [0] Which of course came from NeXT Application Bundles, which
| is also where OSX Application Bundles came from.
| Kenji wrote:
| [deleted]
| BeeOnRope wrote:
| Creating a portable single-executable in C/C++ on Linux is not
| particularly easy.
|
| The dominant libc for Linux, glibc, says "static linking is
| highly discouraged", see [1] among many other sources for the
| subtle problems it causes.
|
| There are solutions like [2], [3], and [4] but they all have
| downsides and don't compare at all to e.g., the go (or even
| rust) single-executable story.
|
| ---
|
| [1] https://stackoverflow.com/questions/57476533/why-is-
| statical...
|
| [2] Link glibc anyway and try to avoid the pitfalls.
|
| [3] Link glibc dynamically but everything else static, and
| build your exe using an ancient toolchain (or better, a new
| toolchain built for an ancient libc) so that it works on most
| distros.
|
| [4] Give up on glibc and statically link an alternative libc
| like musl.
| kristov wrote:
| Or just dynamic link to libc and static link the rest as much
| as you can. If a binary dynamically links to libc, but is
| otherwise statically linked, all good no?
| admax88qqq wrote:
| Pretty good. Glibc does update occasionally. I've hit
| issues in the past trying to run old proprietary binaries
| that required libc.so.5
|
| But yeah that's pretty rare tbh.
| BeeOnRope wrote:
| The major library version number (e.g., 5 vs 6) isn't
| enough: the symbols even within lib.so.6 are versions and
| some may update every glibc version, so in general you
| may have problems building against a new libc 6 then
| running on a system with an old libc 6 (you'll get a
| runtime error from the dynamic loader complaining a
| missing versioned symbol).
| BeeOnRope wrote:
| Yes, that was option [3]. The problem is that glibc symbols
| are versioned and update fairly regularly, so your binary
| likely won't run on anything with a much older glibc, and
| glibc versions vary a lot across distros (e.g., some are
| much slower moving so use very old glibc).
|
| To work around this you need to try to build on an old of a
| distro as possible, so you use only old symbols... but you
| don't want to build on an ancient compiler since it might
| not support the language version or compiler flags you
| need, so you first build a new old toolchain on the old
| distro against the old libc, or find a docker container
| from someone who has already gone though this pain.
|
| Of course, this also means you can't use new libc features:
| libc is more than just the C runtime, it's also the shim
| layer between userspace and the kernel, so you might miss
| out on newer system calls (or have to write the wrappers
| manually, etc).
|
| It gets even worse for C++.
| irq-1 wrote:
| *Edit: I was wrong. Just checked go 1.7 and it made a static
| binary; no dynamic links. Don't know when that changed.
|
| > [3] Link glibc dynamically but everything else static
|
| Go does this. Zig links musl [4] by default but gives you the
| option to link glibc. I don't know about Rust.
| paxswill wrote:
| Rust links glibc dynamically, and can use musl for some
| platforms. By default most (all?) of the musl targets are
| linked statically, but they should be able to use dynamic
| linking for it as well.
| marcosdumay wrote:
| The good news is that there has been a long time since glibc
| had breaking changes, and the formal policy of the kernel
| reflects on it and makes future breaking changes unlikely.
| BeeOnRope wrote:
| It depends what you mean by breaking changes, right? Do you
| mean they try to avoid breaking ABI in a _forwards
| compatible_ way? That is, building against new glibc then
| runing against old glibc should work?
|
| I don't think so: they only make it ABI backwards
| compatible (compile against old glibc, run against new) and
| API compatible (recompile your source against different
| versions should work).
|
| I just checked by glibc and it had > 200 symbols versioned
| 2.34 which is the current glibc version. If you use any of
| those you can't run against older glibc.
___________________________________________________________________
(page generated 2022-03-02 23:01 UTC)