[HN Gopher] Bringing the Unix philosophy to the 21st century (2019)
       ___________________________________________________________________
        
       Bringing the Unix philosophy to the 21st century (2019)
        
       Author : todsacerdoti
       Score  : 146 points
       Date   : 2021-08-22 15:22 UTC (7 hours ago)
        
 (HTM) web link (blog.kellybrazil.com)
 (TXT) w3m dump (blog.kellybrazil.com)
        
       | c0l0 wrote:
       | So I've been writing shell scripts for about two decades, about
       | 75% of that time professionally. Parsing the unstructured, text-
       | based output of utilities is not the problem for anyone who's had
       | maybe a few weeks of training. Most `... | grep ... | cut ... |
       | sed ... | awk ...`-abominations the post laments can be replaced
       | by a single informed call to `awk`, making everything a lot more
       | elegant and concise.
       | 
       | Having JSON as an intermediate representation to work on instead
       | is not going to save anyone - what we'd _really_ need is for all
       | tool versions /variants output on all platforms (all GNU/Linux
       | distros, all the BSDs, all embedded Linux variants, all
       | commercial UNICES, etc.) to be the same, all the time. That's not
       | going to happen, so shell scripting is going to stay messy.
       | 
       | Also, for my INTERACTIVE shell, anyone can try to pry free-form,
       | text-based and semi-structured output from my cold, dead hands.
       | JSON or YAML output might be an acceptable compromise between
       | being easy to parse and bearable for human consumption, but for
       | my daily work, I would rather have my tools make it easy for _me_
       | , the human part in the whole equation, and not some parsing
       | logic that might not even (need to) exist. Shell scripting
       | provides most of its value from the fact that since I'm in its
       | repl-of-sorts all the time, I can translate that familiarity to
       | scripts and executables effortlessly, and I would not want that
       | going away. But I am rather certain it would, if we had JSON (or
       | another form of more structured data interchange syntax) adopted
       | as the "universal" interface between UNIX tools.
        
         | [deleted]
        
         | 1MachineElf wrote:
         | About this few weeks training you mentioned for Unix-style
         | parsing of unstructured text - what resources do you recommend?
        
           | tuatoru wrote:
           | The GNU Awk manual has several practical examples in it - as
           | manuals should. (Looking at you, nearly all man pages in
           | linux distros.)
        
         | kaliszad wrote:
         | I can see the potential in Babashka and other Clojure-based
         | systems from the approach point of view. There needs to be a
         | simple integrated editor or much better readline implementation
         | though that needs to work out of the box. That should improve
         | the interactivity in a major way. We also need to have a set of
         | very simple tools to work with files, network connections (e.g.
         | traffic dumps, netcat) and other typical shell tasks. This
         | isn't really as efficient when used interactively in the
         | Clojure ecosystem or anywhere else from what I have seen
         | besides just using e.g. ls, du, df, fdisk, tcpdump, nc ...
         | 
         | More long term, I have high hopes of having an environment
         | above the kernel that basically does what /proc and /sys and
         | others do currently. Programs equivalent to those in /bin
         | /usr/bin etc. would be just dynamically loadable modules or
         | built into the basic tool set. No shell scripts anymore, no
         | random filesystems with custom formats for everything, just a
         | JITed VM that has a strong set of tools but anybody can extend
         | it with either stuff written in some Clojure-like language, or
         | something compiling to the VM or something native that has some
         | kind of interface (FFI?) to be usable from the programs running
         | on the VM (e.g. for cryptographic stuff or stuff that needs to
         | be as efficient as possible). We would also need something like
         | SSH but for structured data that would support a SHELL/ REPL-
         | like workflow as a byproduct but really be meant for more or
         | less high performance, efficient communication (e.g. useable
         | even for large file copy operations and such). In the end,
         | parts of this system could connect to a in kernel VM (BPF?) and
         | execute there but we would interact with them using the nice,
         | structured REPL.
         | 
         | This would be a huge undertaking but I can't really see, how we
         | can radically improve the efficiency of work with the current
         | systems. It seems, we are mostly just patching old approaches
         | to do new tricks and to me, it seems to be falling apart. The
         | complexity we impose upon us is crushing and I don't think all
         | of it is necessary.
        
           | dimitar wrote:
           | Babashka is really nice in that it has json and yaml
           | libraries built-in, so you end up using your Clojure data
           | structures throughout the script, and yet still
           | consuming/emitting json/yaml if you need to.
        
         | emodendroket wrote:
         | PowerShell is just so much nicer to use than anything where
         | text munging is the only way to do things, and it's just as
         | "pluggable" as Unix shell commands. And it can output text (or
         | JSON or XML or YAML) or whatever you want easily by piping into
         | relevant commands (or just not piping anywhere if you wanted
         | text). I don't imagine those are going anywhere anytime soon
         | but I think it is wrong to say the system cannot be improved
         | upon.
        
         | athrowaway3z wrote:
         | I know enough awk to know it solves my problem, but use it to
         | little to remember how, and the development experience is never
         | very clean.
         | 
         | What i would like is an cli awk ide. (Rolls of the tongue
         | right? )
         | 
         | Ideally i could write ... | awk-ide "scripts/thing.awk" | ...
         | 
         | If it exists, run it. Otherwise have it open an editor session
         | that runs an awk program as i type it on some buffered data,
         | shows the result, and have some hints on what syntax/variables
         | are available.
        
           | cle wrote:
           | I use this occasionally in Emacs with awk-mode to live edit
           | awk programs against a data file:
           | 
           | https://github.com/danlamanna/live-awk-mode
        
         | 1vuio0pswjnm7 wrote:
         | Can JSON compare with line-based data, which many original UNIX
         | utilities seem to target. JSON's design assumes the user can
         | read the entire file into memory. It's really easy to exhaust
         | resources with JSON. And fast, crash-proof JSON parsers become
         | more challenging to write.
         | 
         | Whereas it's not nearly as easy to exhaust memory with line-
         | based data processing nor to crash utilities that read line-by-
         | line, e.g., sed. If lines are too long, I can chop them down to
         | a reasonable size on some sentinel.
         | 
         | IMO, JSON, like Javascript, is web/browser centric. For someone
         | who rarely uses a browser or Javascript and is comfortable with
         | UNIX, e.g., yours truly, JSON is not particularly advantageous.
         | For large data, line-based is more robust (and memory-
         | efficient) than JSON, IME.
         | 
         | Better than JSON is netstrings or bencode.
         | 
         | https://en.wikipedia.org/wiki/Netstrings
         | 
         | https://en.wikipedia.org/wiki/Bencode
        
         | kbrazil wrote:
         | I don't think the issue is that it's hard to manually parse.
         | The problem is that it's hard for someone else to read your ad-
         | hoc parser years later and reason about what you did if they
         | need to modify it.
         | 
         | Disclaimer: I am the author of the article and JC.
        
           | pdimitar wrote:
           | This is even more true for the ungodly long `jq` incantations
           | that people write.
           | 
           | It's like I get it, the old way is ugly and not always easy
           | to decipher but at least it's shorter and your chances of
           | understanding it are better.
           | 
           | I've had both -- the classic piped chain of UNIX commands and
           | various JSON-producing tools piped to `jq`. The former were
           | still easier to work with.
        
             | hackerbrother wrote:
             | (Sorry, this is my first time trying to do a formatted
             | comment here.)
             | 
             | What I like to do is comments like:
             | 
             | /*                 * Collates            * [            *
             | {             *        id: 4            *        dept:
             | 'oncology',            *        name: 'Joe S.'            *
             | }            *    .            *    .            *    .
             | *  ]            *            *  into            *
             | *  {            *     4: {  // id            *        'Joe.
             | S': { // name            *           dept: oncology
             | *        }            *     .                 *     .
             | *     .                 *   }            */
             | 
             | ( Then insert horrible one-liner that does the
             | transformation. )
        
               | [deleted]
        
             | kbrazil wrote:
             | Yes, I have seen those too! That's why I also wrote Jello,
             | which is like jq but uses pure python without the
             | boilerplate. Python is nearly universal now and typically
             | easy to read, though more verbose. Jq is just as much a
             | write-once tool as awk and perl for more complex queries.
             | For simple attribute calls, though, it's both terse and
             | readable.
        
               | hnlmorg wrote:
               | This is why I wrote murex shell
               | (https://github.com/lmorg/murex), it's an alternative
               | $SHELL, so you'd use it in place of Bash or Zsh, but it's
               | optimised for modern DevOps tools. Which means JSON and
               | YAML are first class citizens.
               | 
               | It's syntax isn't 100% POSIX compatible so there is some
               | new stuff to learn but it works with all the existing
               | POSIX tools and is more readable than AWK and Perl but
               | while also being terse enough to write one liners.
        
               | NegativeLatency wrote:
               | I wrote something similar by description in ruby, I'd be
               | curious to see your python implementation.
               | 
               | I considered python but Ruby's easier chaining with
               | map/filter/etc made it easier for me to use when writing
               | just one line with it to transform some json.
               | 
               | https://github.com/nburns/utilities/blob/master/rson
        
               | kbrazil wrote:
               | Sure thing!
               | 
               | https://github.com/kellyjonbrazil/jello
               | 
               | Other languages are superior in their handling of
               | maps/arrays, but Python is just so damned popular now I
               | thought it was a good choice to democratize JSON
               | handling.
               | 
               | https://blog.kellybrazil.com/2020/03/25/jello-the-jq-
               | alterna...
        
         | JJMcJ wrote:
         | > awk
         | 
         | Or Perl. That's what Perl was originally created for.
         | 
         | Or sed. If awk is too mainstream for you.
         | 
         | Or Python.
         | 
         | Or just write the ugly pipeline in your script, and promptly
         | forget how it actually works. I've done that a lot.
        
           | u801e wrote:
           | > Or sed
           | 
           | After learning how to use ed in scripts, I've found it's
           | actually easier to use it compared to sed because it
           | effectively has random access through the input rather than
           | going from beginning to end.
           | 
           | But both awk and Perl have the advantage of storing values in
           | variables over sed or ed.
        
             | Koshkin wrote:
             | sed, being a _stream_ editor, is more efficient than ed, in
             | that it does not need to keep the entire file in memory.
        
         | jl6 wrote:
         | > what we'd really need is for all tool versions/variants
         | output on all platforms (all GNU/Linux distros, all the BSDs,
         | all embedded Linux variants, all commercial UNICES, etc.) to be
         | the same, all the time
         | 
         | The industry has approximated this state of affairs by
         | approximating a GNU+Linux monoculture. Doesn't matter that BSD
         | tar behaves differently if ~nobody uses it.
        
         | coliveira wrote:
         | You mentioned one pain point of shell scripting, variability
         | among platforms. The other pain point is that different tools
         | have different methods to treat errors and warnings. This makes
         | debugging shell scripts a nightmare, compare to scripting
         | languages like python. If you need to debug your work, you need
         | to check several levels of script vs. commands, and use
         | different ways to check for errors depending on what is causing
         | the problem.
         | 
         | If I had to decide on a tool to replace shell script I would
         | vote for tcl, since it maintains many of the advantages of the
         | shell but provides a better handling of the programming aspect.
         | Unfortunately these days it seems that you either use shell or
         | some full-scale language like python.
        
       | eadmund wrote:
       | Why not go all the way and use a format capable of expressing
       | code _and_ data? I refer, of course, to S-expressions.
       | 
       | They also have the benefit of properly handling numbers. Some
       | might look at the absence of maps as a negative, but I think
       | alists are preferable anyway due to their constant ordering.
        
         | pjmlp wrote:
         | Specially because it keeps being forgotten that Lisp Machines
         | and Interlisp-D workstations shells were basically a graphical
         | based REPL.
         | 
         | To put it in 2021 terms, Jupiter Netbooks in 1980's instead of
         | PDP-11 green phosphor terminals.
        
         | kbrazil wrote:
         | That's a good debate to have. I settled on JSON due to its
         | readability and ubiquity in web APIs. It's something people all
         | the way down and up the stack are very familiar with these
         | days.
        
           | eadmund wrote:
           | True, folks _are_ very familiar with JSON, but it does have
           | problems, and the best time to pick the best solution is
           | before one has to deprecate a lesser solution. Computing is
           | built atop a pile of decisions which made sense at the time
           | and cannot be changed now due to compatibility (spaces in
           | Makefiles, anyone?): there is no time like the present to
           | simply choose to do the most correct thing.
           | 
           | 'But folks won't use it!' Well, they might not. But if one
           | gives folks a choice between the capability they need using
           | an unfamiliar technology and not having the capability at
           | all, they will learn the unfamiliar tech.
        
         | selfhoster11 wrote:
         | I'd rather avoid mixing the two. Shell injection is already a
         | danger.
        
         | quasarj wrote:
         | Mostly because S-expressions are terribly hard for humans to
         | read... and everyone is already familiar with javascript
         | syntax.
        
         | lfnoise wrote:
         | Yes. I never understood why JSON over s-exprs. The absence of
         | maps is not a negative. S-exprs can represent maps. There are
         | no maps in JSON, really anyway. It is just text. How that data
         | is represented in memory is the output of parsing. You could
         | just as well parse (dict (a 1)(b 2)(c 3)) into a hash table if
         | you wanted. You could also have sets (set 1 2 3) or whatever
         | other data structure.
        
           | pjmlp wrote:
           | Fear of parentheses, basically.
        
             | u801e wrote:
             | But json already has quotes, commas, brackets and braces
             | :-)
        
               | pjmlp wrote:
               | Yeah, apparently finding misplaced quotes, commas,
               | brackets and braces is magically easier than misplaced
               | parentheses. :)
        
           | iainmerrick wrote:
           | You _could_ do any of those things, but you have to pick a
           | convention and other people have to agree on it.
           | 
           | JSON is nice in that it has just enough structure to do a
           | good number of tasks in one obvious way. The biggest omission
           | is probably some kind of time and/or date type (but ISO8601
           | in a string is the obvious solution there).
           | 
           | It's not a coincidence that JSON was reverse-engineered from
           | a language with convenient literals for dictionaries and
           | arrays, and most languages provide those two collection types
           | because they cover most use cases, so JSON fits most
           | languages fairly well.
           | 
           | It's just handy having _both_ arrays and dictionaries
           | available, rather than stretching one data structure to cover
           | both, whatever Lua or Lisp might say.
        
       | natt941 wrote:
       | I can understand why the idea of more structured, object-like
       | input and output is appealing, but after using PowerShell for a
       | while, my take is that it's much harder to manipulate objects
       | into a consistent format than it is to manipulate text. For
       | instance, if you want to compare Azure DNS records with DNS
       | records from a Windows server, it's a huge pain because Get-
       | AzDnsRecordSet and Get-DnsServerResourceRecord return objects
       | with different structures. Same problem if you want to pipe
       | output of one util to another which expects slightly different
       | format of input. More generally, text is great for loose
       | coupling; structured objects, less so.
        
         | laumars wrote:
         | There are better shells out there for handling structural data,
         | like Murex and Elvish
        
         | crdrost wrote:
         | It's that the languages suck.
         | 
         | I've tried writing a language around this but it's... rusty.
         | You kind of want to pipe Get-AzDnsRecordSet and Get-
         | DnsServerRrsourceRecord to the same interface structure and
         | then build another transform to some other structure from the
         | interface... The program wires together the transforms. So the
         | programming paradigm becomes rather unconventional, you only
         | write transformations of "events", you address other shell
         | commands by listening to outputs of their "eventspace," etc.
         | ... You get a sort of strange logic programming language where
         | the pipe operator has to be slightly specialized every time
         | because it always needs to transform a little as it pipes
        
         | zamadatix wrote:
         | I think that's why the article proposes adding command line
         | options and alternative APIs to output JSON objects not forcing
         | all things the use objects all the time.
         | 
         | For any serious data retrieving or scripting the light object
         | wrapping will be superior. For quick one off use maybe the
         | plain text option will be quicker to reason about.
        
         | jayd16 wrote:
         | You can always just thunk down to the text representation and
         | do things that way. Having the structure is a strict plus, no?
        
           | ori_b wrote:
           | This is a much more complex text representation.
           | 
           | Mandating tab separated columns with a consistent quoting for
           | embedded spaces would be a net benefit. And it would match
           | today's tools well.
        
         | zozbot234 wrote:
         | > More generally, text is great for loose coupling; structured
         | objects, less so.
         | 
         | I disagree. Parsing "loose coupled" text and converting it to
         | the format that a different tool expects is a rather non-
         | trivial problem, and one that's often poorly specified to begin
         | with; converting structured outputs is generally
         | straightforward in comparison.
        
       | rurban wrote:
       | Thankfully he didn't propose XML. Unfortunately it looks like he
       | (and many others) really thinks that JSON is better, even if it's
       | underspecified, thus leading to possible insecurities. See the
       | recent jsonsec thread. (Undefined key ordering and duplicate
       | handling)
       | 
       | So I have to bring in jsmn.h to parse protocols? Sorry no. Been
       | there, done that. We are pushing too much unnecessary JSON around
       | already.
       | 
       | Unix is also about KISS. Structured data are lines and
       | paragraphs.
        
         | ajuc wrote:
         | > even if it's underspecified, thus leading to possible
         | insecurities
         | 
         | what it replaces (unstructured text) is much less secure so I
         | don't think that counts against json
        
           | azalemeth wrote:
           | But at least the parsers are very, _very_ battle tested at
           | least
        
       | xrayarx wrote:
       | Article from 2019 basically advertises
       | https://github.com/kellyjonbrazil/jc which converts output of
       | many unix cli commands into json, depends on Python
        
       | antonios wrote:
       | FreeBSD has started supporting JSON output for various tools via
       | `libxo` in their base system for quite some time now.
        
       | freshhawk wrote:
       | Uh ... what about when I don't want to load the entire stream
       | into memory before the next stage starts running? Are these
       | implicit json arrays that stream out?
       | 
       | Or do we now have incompatible json shell tools and streaming
       | text tools as a permanent fixture?
       | 
       | I'm excited about structured output ideas, but json? I'd much
       | rather have streams of whitespace separated words than json.
       | That's in that "No type system is better than a bad type system"
       | metaphorical area.
       | 
       | I'll take grepping with a theoretically brittle regex over this
       | jq[1] any day of the week.
       | 
       | [1] jq -nc --stream 'inputs | select(length==2) | select(
       | [.[0][0,2,4]] == ["results", "data", "row"]) | [ .[0][6], .[1]] '
        
         | selfhoster11 wrote:
         | For the first point, there's no reason why we can't use JSON
         | array wrapped output. In fact, we likely should use something
         | like this for uniformity. Loading everything into memory is
         | also not too problematic assuming a working swap space and
         | reasonably well-architectured output schema.
         | 
         | For the second point, whitespace sensitivity is the one mistake
         | that greatly pisses me off with Unix. I should be able to pass
         | arguments and filenames with as many spaces as I want. We are
         | in the 21st century and occasionally do use spaces in
         | filenames.
        
         | kbrazil wrote:
         | I agree JSON is probably not right for every type of program
         | output, but the age of web APIs has shown us that is probably
         | great or adequate 90% of the time. If something is spewing out
         | long lines of data I think JSON Lines would be a good option so
         | you don't need to read the whole structure into RAM. But any
         | other structured output that has a healthy community and
         | ecosystem supporting it would be better than just space
         | delimited lines, or worse - groups of lines you need to deal
         | with.
        
         | woodruffw wrote:
         | I agree with you in broad strokes, but as a piece of anecdata:
         | I've had a lot of success building tools that emit and consume
         | JSONL[1] instead of entire JSON documents. JSONL preserves the
         | Unix pipeline's inherently parallel design (people tend to
         | forget this, even when waxing about the Unix philosophy!) but
         | gives us all of the nice typing of a JSON stream.
         | 
         | That being said, I too will take a `sed` or `awk` one-liner
         | over some of the `jq` monstrosities that I've seen.
         | 
         | [1]: https://jsonlines.org/
        
       | [deleted]
        
       | mcswell wrote:
       | While you're at it, bring Unix into the 21st century, in which we
       | use Unicode (UTF-8 encoding). The number of Unix utilities that
       | badly support Unicode is, well, painful to those of us who deal
       | with non-ASCII data all the time.
        
         | 10000truths wrote:
         | Which utilities don't support UTF8? I've never had encoding
         | issues when using coreutil binaries e.g. grep or sed or awk. I
         | guess if you want to display emojis or something, that might
         | not work properly, but that's an issue with your terminal, not
         | with the utility you're running.
        
         | anthk wrote:
         | Unix "2.0" (plan9) invented utf-8.
        
       | seppoonbi3 wrote:
       | I have noticed that people intuitively use unix philosophy
       | _inside_ their programs. Not often you see well written software
       | with functions that do not respect the unix philosophy (if only
       | they would be programs and not functions).
        
         | tyingq wrote:
         | I agree that they handle some amount of unix philosophy
         | intuitively. But, for example, lots of command line programs
         | don't handle SIGPIPE.
        
         | dan-robertson wrote:
         | Emacs is in many ways anti-Unix-philosophy and I think the way
         | it's software tends to be structured is no exception. Functions
         | can get ad-hoc extensions or modifications all over the place
         | with advice and hooks and dynamic scoping[1]. The only
         | similarity is that in Unix most things are files and in Emacs
         | most things are buffers.
         | 
         | [1] Unix has weird ad-hoc mechanisms too like environment
         | variables or your PATH containing modified versions of
         | programs, and it has programs that do a thousand and one
         | things, but I claim those are mostly violations of the Unix
         | philosophy.
        
           | zozbot234 wrote:
           | The Unix philosophy includes the use of software libraries in
           | one's programs, albeit only resorted to when truly necessary.
           | That often involves resorting to many of these tricks,
           | usually for the sake of greater software reuse.
        
       | richardanaya wrote:
       | I was literally just thinking about this a few days ago. I'm
       | super excited by https://www.nushell.sh/ . I think they are
       | hitting on an order of magnitude improvement paradigm of shells
       | that fit very nicely with the theme of this article.
        
         | bluecatswim wrote:
         | I've always been looking for anushell.
        
         | sonofajojo wrote:
         | This reminds me of powershell
        
         | bassman9000 wrote:
         | https://www.nushell.sh/book/
         | 
         | This goes against the philosophy explicitly mentioned in OP's
         | article. E.g. avoid tabular formats.
         | 
         | This is systemd against init again. A powerful, but
         | overreaching shell that becomes unreplaceable and bloated with
         | concerns. In constrast, traditional *nix/GNU programs work
         | well, and interact well, in every shell.
        
           | selfhoster11 wrote:
           | You see traditional Unix/GNU programs as working well and
           | interacting will. I see a programming paradigm designed for
           | six-char identifiers, not designed for whitespaces and
           | punctuation in names, running in something that emulates a
           | physical terminal that emulated a paper-based terminal.
           | Terseness is there, but ergonomics of use for non-utter-
           | experts could be improved.
           | 
           | I think that the shell paradigm could be so much better. If
           | there's a momentum to make a change from that, I'll jump on
           | the bandwagon. Any change is better than the current state of
           | the shell.
        
       | Koshkin wrote:
       | Apparently, Plan9 has failed in this regard. What went wrong?
        
         | pjmlp wrote:
         | I guess the team moved on to Inferno, plus this kind of shells
         | don't play well with UNIX related culture.
         | 
         | Xerox PARC workstations and Lisp Machines already solved the
         | problem via their REPLs.
        
       | ape4 wrote:
       | That `jc` command that converts non-json output to json is neat.
       | But, um, seems slightly kludgy to be obvious. What if the
       | `ifconfig` format changes slightly. I suppose its a stop-gap
       | until all commands have a json output option.
        
         | kbrazil wrote:
         | If the ifconfig output changes slightly, then it will probably
         | break the thousands of ad-hoc parser implementations buried in
         | scripts worldwide. With this approach, only a central parser
         | library (open source) needs to be maintained so it can be fixed
         | quickly and robustly.
         | 
         | That being said, the goal is that command line tools that
         | output useful data for scripts should have a structured output
         | option like JSON while still keeping the human text output
         | option so something like JC doesn't even need to exist.
        
           | ape4 wrote:
           | You have a point but when when I write a script to parse a
           | command's output I don't read/parse every line. I'd probably
           | grep for the one line I want.
        
         | hjek wrote:
         | True. When I try it with `date` on Debian Sid I get parser
         | error:                   $ jc -p date         jc:  Error - date
         | parser could not parse the input data. Did you use the correct
         | parser?                  For details use the -d or -dd option.
        
           | ape4 wrote:
           | This `date | jc --date` seems to be the way
        
           | kbrazil wrote:
           | Odd. What locale are you using? Should work fine with C or
           | en_UTF8, per the readme.
        
       | phendrenad2 wrote:
       | "Up until about 2013 it made just as much sense as anything to
       | assume unstructured text was a good way to output data at the
       | command line..."
       | 
       | But in 2013 a certain data format called JSON was standardized as
       | ECMA-404..."
       | 
       | "Had JSON been around when I was born in the 1970's Ken Thompson
       | and Dennis Ritchie may very well have embraced it as a
       | recommended output format to help programs "do one thing well" in
       | a pipeline."
       | 
       | This whole post hinges on the theory that JSON is a revolutionary
       | technology that no one had created something like before, and no
       | one even considered creating before.
       | 
       | But that seems completely wrong, right?
       | 
       | "The Xerox Network Systems Courier technology in the early 1980s
       | influenced the first widely adopted standard. Sun Microsystems
       | published the External Data Representation (XDR) in 1987. XDR is
       | an open format, and standardized as STD 67 (RFC 4506)"
       | 
       | https://en.wikipedia.org/wiki/Serialization
        
         | kbrazil wrote:
         | I don't follow. Your examples are of data formats standardized
         | in the 80's while Unix was developed in the 60's and 70's. JSON
         | even existed before 2013, but the fact that it became a
         | standard in addition to being popular is the point I was
         | making.
        
       | [deleted]
        
       | jrm4 wrote:
       | So, this is not "bringing it to the 21st century." I feel like
       | what would actually do that is related to the idea of don't try
       | to succeed perfectly, try to fail elegantly?
       | 
       | I want some sort of non-destructive way for "the shell" to take a
       | best guess at what's going on when I make a typo, or when the
       | data's not formatted quite right, like a live linter that's there
       | all the time.
        
       | zokier wrote:
       | > (iii) [...] Don't hesitate to throw away clumsy parts and
       | rebuild them.
       | 
       | Some forgotten wisdom right there.
        
       | jwilk wrote:
       | FWIW, "ip -j route" should work correctly since v4.17:
       | 
       | https://git.kernel.org/pub/scm/network/iproute2/iproute2.git...
        
       | dleslie wrote:
       | JSON is a terrible intermediary format since its structure is
       | incompatible with streaming data; and demands full structure
       | parsing before consuming the data.
        
       | davidzweig wrote:
       | Related, I am wondering if somebody is aware of a project to wrap
       | cli utilies in a minimal 'TUI', along the lines of this:
       | https://github.com/chriskiehl/Gooey , but staying in the
       | terminal. You'd pass the name of the program to run as the first
       | argument. Haven't thought this through fully.
       | 
       | EDIT: well, some discussion about this:
       | https://github.com/chriskiehl/Gooey/issues/296
        
       | maple3142 wrote:
       | Parsing ad-hoc text formats always seem very fragile, as single
       | spacr character (or even newline character) could easily break
       | the pipeline. Using a proper data structure could make scripts
       | works more "properly" when encountering these edge cases.
       | 
       | Which format to use a as representation of data structure might
       | be debatable, but I think JSON is a reasonable choice.
       | (Formatted) JSON is readable by human and could be easily parsed
       | by programs.
        
       ___________________________________________________________________
       (page generated 2021-08-22 23:00 UTC)