[HN Gopher] Following the Unix philosophy without getting left-p...
___________________________________________________________________
Following the Unix philosophy without getting left-pad (2021)
Author : goranmoomin
Score : 57 points
Date : 2022-11-10 18:36 UTC (4 hours ago)
(HTM) web link (raku-advent.blog)
(TXT) w3m dump (raku-advent.blog)
| nonameiguess wrote:
| This is conflating two things: the granularity of system
| utilities and the granularity of packages. Those are not the same
| thing. GNU core-utils provides 24 separate utilities into a
| single package. Add in sed, grep, and bash, which has built-ins
| providing virtually everything else, and you're pretty close to
| full POSIX with only four packages, thus four dependencies.
|
| At the language level rather than system utilities, think of
| something like boost. You install one dependency, and you get
| hundreds of individual libraries, each of which does only one
| thing and one thing well, and you can link only what you need
| into your own program. UNIX philosophy followed, without turning
| your dependency tree into combinatorial explosion.
| pjmlp wrote:
| On top of that calling either man or info on those utilities
| will show how far beyond they are of doing just one thing.
| bryanlarsen wrote:
| Yes, that is one of two conclusions in the article.
| niccl wrote:
| total nitpick, but it's unfortunate that he didn't credit XKCD
| for the image (https://xkcd.com/2347/), since it's under creative
| commons with attribution licence
| (https://creativecommons.org/licenses/by-nc/2.5/)
| spullara wrote:
| It is directly linked to the XKCD page.
| amadeuspagel wrote:
| The only problem with left-pad was that NPM let people delete
| packages that other packages depend on. That's literally it.
| kerblang wrote:
| Nicely done, hits the right points, and.. what would I add?
| Nothing, really. In these cases HN folk tend to resort to bizarro
| contrarianism in the effort to chat up the article. I hereby
| plant the _anti-_ bizarro-contrarian flag.
| ummonk wrote:
| It writes a 10 page essay for what can be described in one or
| two paragraphs - and is obvious to basically anyone who
| advocates for good standard libraries.
| [deleted]
| pvg wrote:
| If you want a flag to rally around, the 'no meta' one is better
| and a lot less effort.
| Tainnor wrote:
| The article raises some interesting points but I still disagree
| that comparing the mess of JavaScript dependencies[0] to GNU
| utils is warranted - the level of trust is fundamentally
| different. I have some reasonable intuition that the authors of
| GNU packages _as well as the distro maintainers_ do a good job
| writing, packaging and vetting these tools. That 's not really
| the case with left-pad or is-array. I can trust that an update to
| cat or ls will not break my system because enough people have
| looked at it, but I really don't know what's gonna happen when I
| update left-pad (without even knowing it, because a it's a
| transitive dependency).
|
| The other point is that GNU utils are self-contained binaries and
| don't usually include transitive dependencies 20 levels deep.
|
| Even when the problems these packages solve are not 100% trivial,
| unless you trust that library to be continuously maintained,
| maybe consider not adding the dependency, instead taking
| inspiration from the code and writing your own, maybe even
| simplified version of it (careful with that, you don't want to
| write your own crypto, for example - but when we're talking about
| a three-file "compare XML files" test util that hasn't been
| updated since 2014, just copying that code into your project
| might be the better approach).
|
| [0]: And it really is JavaScript - I don't know of any other
| language with the same kind of explosion of micro-packages. I'm
| not saying that other languages never experience dependency hell
| - of course they do - only that it's usually not as horrifying as
| it is with JS.
| amyjess wrote:
| > I can trust that an update to cat or ls will not break my
| system because enough people have looked at it, but I really
| don't know what's gonna happen when I update left-pad (without
| even knowing it, because a it's a transitive dependency).
|
| The biggest difference here is that distros vet their
| maintainers, often with a lengthy process similar to hiring
| someone at a company, while NPM and similar repos are entirely
| self-serve, where anyone can register an account and just
| publish whatever packages.
| iudqnolq wrote:
| > The biggest difference here is that distros vet their
| maintainers, often with a lengthy process similar to hiring
| someone at a company
|
| This is very surprising to me. During the debacle over Rust
| in the Python cryptography package maintainers were saying
| it's unreasonable to expect them to read dependency
| changelogs.
| Teongot wrote:
| What was the origin of the idea that the Unix philosophy is "do
| one thing and do it well"?
|
| If anything, the UNIX philosophy is more like "do one thing, and
| do it to a text file".
| dctoedt wrote:
| > _What was the origin of the idea that the Unix philosophy is
| "do one thing and do it well"?_
|
| Relatedly: Brian Kernighan's _Unix: A History and a Memoir_ was
| an enjoyable read.
|
| https://www.amazon.com/UNIX-History-Memoir-Brian-Kernighan/d...
| (not an affiliate link).
| TylerE wrote:
| do one thing, and do it with a file handle.
| jbverschoor wrote:
| aka functional
| dementis wrote:
| If you look at the wiki on the UNIX philosophy it does say
| "Store data in flat text files" and "Write programs to handle
| text streams, because that is a universal interface."
|
| You can do a surprising amount with flat files. For decades HP
| / HPE has been using flat files for Serviceguard / SGLX which
| is a high-availability cluster software that gets used in some
| very large Enterprise environments.
| a1369209993 wrote:
| > You can do a surprising amount with flat files.
|
| For example, tables in relational databases like sqlite are
| representationally isomorphic to tabular files like the
| output from `ls -al`. Sqlite has a lot of useful performance
| optimizations (including eg, not having to encode strings as
| "\x20" or the like if you want arbitrary bytes), but those
| come at the cost of a data format that you can't easily (not
| even notice that you needed to and did) reverse engineer,
| when either writing new software to consume or emit it, or
| even just visually inspecting it in a editor that doesn't
| already speak that format.
| foooobaba wrote:
| Programs/functions like left-pad do not follow the unix
| philosophy (which the author did say). The idea of the unix
| philosophy is to write less programs. Not write a ton of tiny
| programs, but a small number of composable programs. So instead
| of left-pad, right-pad, middle-pad, and anything else to get
| specific need you can use sprintf which can format a string in
| general, one program that can do implement the features of all of
| those.
|
| It is about "coding the permitter, not the area", so if you have
| NxM features to build, don't write N*M individual
| programs/functions, instead write N+M programs/functions that can
| be composed together to get all NxM features. This video explains
| it well: https://youtu.be/3Ea3pkTCYx4
| TrianguloY wrote:
| One thing missing in the article is the all-or-nothing of
| dependencies in most languages. A package with millions of micro
| functions is nice, but if your program only uses 10 of them it
| doesn't matter, you program has all. Some dependency managers
| allows you to "disable" components, but they need to be manually
| picked and usually you can only disable chunks, not individual
| functions.
|
| Wouldn't be nice to let the compiler package only the functions
| really used in the code and discard the rest? Do any language
| provide this? (C/C++?)
| dathanb82 wrote:
| It's available (usually as an optional feature) in lots of
| languages where dependencies are linked statically. The most
| standard terminology I'm aware of for it is "dead code
| elimination"
|
| In Javascript, webpack (as well as other module bundlers)
| supports it, though they call their dead code elimination
| mechanism "tree shaking".
| dwohnitmok wrote:
| The three solutions the author lists at the end of the article
| are themselves often in tension with each other, that is an
| expressive language is at odds with having a single great
| standard library and having an agreed-upon single utility
| package.
|
| An expressive language almost inevitably leads to more than one
| way of doing things and those different ways of doing things are
| often fundamentally mutually incompatible (i.e. a better way of
| writing the code will not save you). That essentially fragments
| the standard library and any single utility packages.
|
| See e.g. the absolute profusion of alternative "standard
| libraries" for Common Lisp, Haskell, etc. Even ones that rein
| things in a little bit like Clojure still fragment when it comes
| to things like concurrency, with a lot of 3rd-party libraries
| purporting to replace core libraries (e.g. core.async vs
| manifold).
| dementis wrote:
| The article seems to miss a very important factor of the Unix
| philosophy and that is "Make every program a filter" which means
| you also have to "Write programs to work together". Having a
| program that can do one thing well without left-pad is a
| worthwhile endeavor but if you can't leverage it for more than
| its original intended purpose has it truly reached its full
| potential?
| [deleted]
| raggi wrote:
| the at&t unix video marketed this idea by way of composing a
| spell checker in a shell script. even at the time, many users did
| not follow this pattern because the ergonomics for spell checking
| in this manner were already sub-par.
|
| what you get if you follow this pattern is git, not left-pad. git
| high level commands were derivative higher order combinations of
| underlying programs that are quite conformant to this aspect of
| the unix philosophy. their newer replacements in many cases no
| longer compose from single purpose programs, instead importing
| aspects of their function as code, and specializing new behavior
| in order to better meet user interface demands. This is a direct
| repetition of the spell checker UX story.
| zokier wrote:
| From my point of view the problem is not micropackages, but the
| constantly changing micropackages making the foundation of your
| software ever shifting around. The problem with leftpad was not
| any problem with the library itself, but the library changing (in
| this case getting deleted). I posit that embracing the culture of
| vendoring dependencies, and even fully absorbing them to your
| projects, leads in the longer term healthier ecosystem. Again
| leftpad, if everyone had vendored it then it would have been
| largely a non-event. Of course this way of thinking puts much
| higher demand on correctness and also probably some novel
| innovations regarding patch management when as a library
| maintainer you can not just rely on ability to push out new
| versions from your cathedralesque project.
| fedeb95 wrote:
| The point isn't big or small packages. Is unregulated public
| registries. Some quality/security should be enforced. Like some
| linux distributions do.
| kazinator wrote:
| Each one-job Unix program has to do about four things:
|
| 1. parse command line arguments
|
| 2. parse an input format
|
| 3. do its one job
|
| 4. format output
|
| Two or three out of these are superfluous and could be shared
| with a larger program if it were just a function in it.
|
| For instance, regarding (1) if the program just has a "dry run
| -n" and "verbose debugging -v" flag, and it becomes a function in
| the larger program which already has those flags, all the
| function has to do is refer to the configuration.
|
| Regarding (2), the function probably has access to some data
| structure constructed by the larger program rather than raw text.
|
| Regarding (4), the function may be able to put out a data
| structure for whose type the larger program already has an output
| format.
|
| Unix itself showed that the one-program-for-one-task is a weak
| idea when Awk was developed: a programmable utility that allows a
| single program running in one process to avoid using numerous
| Unix tools.
| [deleted]
___________________________________________________________________
(page generated 2022-11-10 23:01 UTC)