[HN Gopher] Static Executables with SBCL v2
___________________________________________________________________
Static Executables with SBCL v2
Author : todsacerdoti
Score : 107 points
Date : 2021-02-25 09:11 UTC (13 hours ago)
(HTM) web link (www.timmons.dev)
(TXT) w3m dump (www.timmons.dev)
| jwr wrote:
| Back when I programmed in CL and participated in comp.lang.lisp,
| the topic of Static Executables was a recurring thing. I never
| understood why people were so obsessed with it, but somehow this
| became the Major Showstopper for wider CL adoption, in the eyes
| of some users.
|
| Most people who actually used CL for building stuff were somewhat
| amused, and tried to find out what was so important about
| encapsulating everything into a single executable and why other
| languages didn't have to do it, but CL absolutely did.
|
| I thought this was a red herring, but indeed for some reason it
| did affect acceptability. Glad to see a solution that looks
| fairly good.
| TeMPOraL wrote:
| > _why other languages didn 't have to do it, but CL absolutely
| did_
|
| Possibly a feedback loop on lack of popularity :). Other
| languages do have to do this in some way or another, with
| respect to their runtimes. The dependencies can be dynamically
| loaded, but the entry point - from the user's POV - still has
| to be invocable by double-clicking or by ./, or you'll lose
| audience. Also size considerations; people are now more willing
| to download a 100+ MB trivial application than they were just a
| few years ago.
|
| On the runtime end, C/C++ solve this by having the runtime
| being part of the system, Java by somehow managing to make
| itself required everywhere, so everybody has a JRE already.
| Rust also compiles down to regular executables. Python got
| popular enough that at least Linux users are likely to have
| several runtimes already installed - but then Windows users
| don't, and any serious app will ship the runtime with it, etc.
|
| In my experience, it always boiled down to this: how to ship a
| portable Lisp application in a single package (by "portable", I
| mean no installation step), where the user just has to double-
| click on something and have it run, and at no point they have
| to know or care it's written in Lisp? This, IIRC, was
| meaningfully non-trivial, especially if you wanted to interop
| with external dependencies by FFI. I remember that at one
| point, I shipped my code to a non-Lisper in form of a
| Dockerfile which handled installing SBCL for them. Not only
| they were grateful, they've managed to pass on that program to
| another friend who (back then) was not much of a coder at all,
| and they both used it. That's the qualitative difference easy
| deployment makes.
| jchw wrote:
| With some languages the problem is solved by simply being so
| ubiquitous that the runtime is already on your machine. I think
| this is true of C, C++, Python, Java... The second best thing
| is a fully self-contained executable like you get with Go or
| Rust. (I do not think it is a coincidence that both Go and Rust
| have been massively successful, though I think statically
| linked binaries are only a minimal part of that.)
|
| The third best thing is shipping the binary alongside the app.
| The worst thing I can think of is just not shipping a runtime.
|
| (Of course when it comes to package maintainers, it may still
| be desirable to ship a separate runtime package that everything
| links to, so as long as the language provides ABI stability for
| the runtime... but I like to think of package maintainer use
| cases as being separate from normal usage.)
|
| And of course, for developers who primarily write apps that
| don't "ship" to external users, it would be normal to feel
| puzzled about this.
| jimbokun wrote:
| Static executables seems like one of the key selling points
| driving adoption of Go, for example.
| 7thaccount wrote:
| Single executables make distribution really easy. Sure, a lot
| of languages like Python don't have this, but it's also easier
| to tell someone to download and install Python to run a script
| than the SBCL process. The entire thing from emacs/slime to the
| package manager and so forth was just very complicated to me.
| Things like Portacle didn't help much either.
| jwr wrote:
| Yeah, I heard that argument -- but seriously, how many people
| run or "install" single executables these days? What actually
| gets distributed as a single executable? I honestly can't
| think of anything, on any system, except perhaps ngrok which
| I use for development.
| qbasic_forever wrote:
| Almost every single tool in the k8s ecosystem is a simple
| statically linked go-based executable. Core stuff like
| kubectl, helm, etc. Read any k8s guide and step 1 is
| usually "go download all theses exes from github releases"
| (or more in vogue now, install a global NPM package that
| has the executable in its bin dir).
| phone8675309 wrote:
| The previous version of the article
| (https://www.timmons.dev/posts/static-executables-with-
| sbcl.h...) explains a few contexts where static binaries
| are desirable.
| yellowapple wrote:
| > but seriously, how many people run or "install" single
| executables these days?
|
| Pretty much everyone who's ever used an AppImage on Linux,
| for one.
|
| This is also still the norm for macOS, from the user's
| point of view (yes, the application is technically a
| folder, but as far as most end users are concerned it's a
| single "thing" that gets dropped into their Applications
| folder). Still the norm for Windows, too, at least for
| installers (and "portable" apps, while not as common as
| ones installed to Program Files, certainly ain't obscure,
| even in this day and age).
| tumdum_ wrote:
| In my case: $ find ~/bin /usr/bin/ /bin/
| -executable | xargs -I XXX file XXX | rg "statically
| linked" | awk '{ print $1 }' /home/t/bin/floki:
| /home/t/bin/yj: /home/t/bin/helm:
| /home/t/bin/fzf: /home/t/bin/kubectl:
| /usr/bin/containerd-shim-runc-v1: /usr/bin/hyperfine:
| /usr/bin/docker-init: /usr/bin/containerd-shim:
| /usr/bin/containerd-shim-runc-v2: /bin/busybox:
| 7thaccount wrote:
| In my eyes, that's part of the problem. A single executable
| is a lot easier to deal with.
|
| Having someone download Anaconda Python (massive install)
| to run a script is a huge fail in my eyes.
| jgalt212 wrote:
| I think that's true for the technical crowd, but much less
| so for the general populace.
| sidpatil wrote:
| I don't see how, unless you're counting Windows
| application installers as single-file executables.
| rjsw wrote:
| I help develop a CL application that is delivered as a static
| executable built using SBCL, it doesn't need any other shared
| libraries so save-lisp-and-die is enough.
| eps wrote:
| PSA: SBCL stands for Steel Bank Common Lisp - http://www.sbcl.org
|
| PSA: PSA stands for Public Service Announcement. Just in case.
| zoren wrote:
| > The name "Steel Bank Common Lisp" is a reference to Carnegie
| Mellon University Common Lisp from which SBCL forked: Andrew
| Carnegie made his fortune in the steel industry and Andrew
| Mellon was a successful banker.
|
| https://en.wikipedia.org/wiki/Steel_Bank_Common_Lisp
| oblio wrote:
| Warning - rant ahead:
|
| I've been in this field for 15 years now. At some point the
| puns are a bit overbearing. We have enough cognitive overhead
| anyway, it's inherent in our field.
|
| They could have named it Carnegie Mellon Common Lisp, but
| hey, someone wanted to feel clever.
|
| Though this is super minor compared to Ruby (gems & co.) and
| especially to Chef (where you work with cookbooks, recipes,
| etc.) And of course, the granddaddy of them all, Unix.
| Because of course, less is more (all of my non-techie friends
| roll their eyes when I tell them about that one).
| dunefox wrote:
| I think Steel Bank Common Lisp is a good name and the pun
| makes me like it even more. How is "Steel Bank" any
| different from a made up name? It's not.
| mbeattie wrote:
| Well it's a fork of cmucl. Carnegie Mellon university
| Common Lisp. So they couldn't have named it the same as the
| thing it was derived from.
| yellowapple wrote:
| Maybe they should've called it Andrew Andrew Common Lisp?
| [deleted]
| dTal wrote:
| Do names based on wordplay _really_ cause you cognitive
| overhead? How are they any worse than totally arbitrary
| names? How is a pun a more distracting etymology than any
| other kind? Does it bother you that New York is named for
| King James II, as he was the Duke of York at the time?
| nathell wrote:
| Haven't used SBCL in a while, so just to confirm: is this a
| better alternative to SB-EXT:SAVE-LISP-AND-DIE?
| fisxoj wrote:
| Without looking at the details, this work probably still uses
| something like that, under the hood. In mainline sbcl, dumping
| an executable with save-lisp-and-die will create an executable
| that depends on a few dynamic libraries and, if you use the
| ffi, whatever dynamic libraries you're using though that. With
| the already existing static-program-op in one of the cffi
| packages (grovel, maybe?), you can get an executable with all
| of the libraries linked through the cffi linked in staticly
| (you won't need those .so files when you distribute the
| executable) but it will still need a few dynamic libraries when
| run. Glibc and libm at least, I think.
|
| As I understand it, this work is attempting to remove even
| those last few dynamic dependencies so you can distribute the
| dumped executable and be done.
|
| So, better if you want simpler distribution at the cost of
| larger executables.
| zulu-inuoe wrote:
| `save-lisp-and-die` is still how you do the image dump, but the
| process here is:
|
| 1. compile & load all lisp code
|
| 2. dump list of external symbols
|
| 3. save-lisp-and-die into a .core file
|
| 4. build new SBCL w/ symbols from step #2 linked in (using
| sbcl.o, which is the SBCL runtime)
|
| 5. run new SBCL, using the core from step #3
|
| 6. save-lisp-and-die into an executable
|
| edit. fixed formatting
| jart wrote:
| Musl is tired and Cosmopolitan wired, since it takes the fully
| static binary concept to a whole new level. SBCL should consider
| using it, since it lets your binaries run on Windows, Mac,
| FreeBSD, OpenBSD, and NetBSD too.
| https://github.com/jart/cosmopolitan See
| https://justine.lol/ape.html which talks about how the executable
| format is also a valid zip file, which was used to create a
| JavaScript interpreter that loads sources from itself in a single
| file. Another option for SBCL is to just use Cosmopolitan's
| research to implement the same thing from scratch. See files like
| https://github.com/jart/cosmopolitan/blob/master/libc/sysv/s...
| and
| https://github.com/jart/cosmopolitan/blob/master/libc/sysv/c...
___________________________________________________________________
(page generated 2021-02-25 23:02 UTC)