[HN Gopher] Tips on adding JSON output to your CLI app
___________________________________________________________________
Tips on adding JSON output to your CLI app
Author : kbrazil
Score : 64 points
Date : 2021-12-03 21:29 UTC (2 days ago)
(HTM) web link (blog.kellybrazil.com)
(TXT) w3m dump (blog.kellybrazil.com)
| wruza wrote:
| CLI and JSON would be amazing if terminals made a step forward
| too. Because both raw json and triangled in-browser console json
| logging just suck for daily reading. A new terminal could either
| detect patterns or use explicit cues in json to format
| structures, and show raw data on demand. E.g. this json5:
| [{_repr:"ls:file", name:"foo.txt", type:"text/plain", size:512,
| access:"664", ...}] could be presented as usual ls does, but
| processed further as json. A whole lot of representations (and
| editors) - including ui-based - could be added to the system
| (e.g. /usr/local/share/repr/ls:file (+x)) to format any sort of
| data, instead of formatting it in-program with getopt and printf.
| And when there is no repr file, well, you still have triangles
| mode. We're too stuck with text=text=text idiom.
| Structure=text=ui would be so much better.
|
| (I'm aware of powershell and am ignoring it consciously)
| nicoburns wrote:
| Nushell does at least some of this
| laumars wrote:
| Such a shell already exists
|
| https://github.com/lmorg/murex
|
| You'd have to learn a new shell syntax but at least it's
| compatible with existing CLI tools (which Powershell isn't)
| umvi wrote:
| In addition to an option for writing output as JSON, consider
| also adding an option for streaming output to stdout. Those two
| features were added to GCC9 gcov and are what enabled me to write
| a tool that parallelizes coverage report generation.
|
| In practice this enabled generating coverage reports orders of
| magnitude faster than traditional gcov wrappers like lcov
| throw0101a wrote:
| See also libxo:
|
| > _The libxo library allows an application to generate text, XML,
| JSON, and HTML output using a common set of function calls. The
| application decides at run time which output style should be
| produced. The application calls a function "xo_emit" to product
| output that is described in a format string. A "field descriptor"
| tells libxo what the field is and what it means._
|
| * https://github.com/Juniper/libxo
|
| Then add an "--output-format" option.
| GordonS wrote:
| A +1 from me on something like `--format` - pipe auto-detection
| feels unnecessary and like an inevitable footgun.
|
| As just one example, the Azure CLI defaults to human-readable
| output, but has an "output" parameter so you can have JSON if
| you want - I've never once wanted any kind of format auto-
| detection, and I have to say that I still don't.
| lunfard000 wrote:
| come to the powershell-side, we have ConvertFrom-Json
| enriquto wrote:
| If you ever need to fight against this annoying json trend (e.g.,
| when your tool only emits certain information in stupid
| ungrepable json), consider filtering the output through a gron so
| that it becomes saner.
| trulyme wrote:
| Thank you, sounds very useful for quick queries:
| https://github.com/TomNomNom/gron
| hencoappel wrote:
| Grep is for simple text, jq allows much more powerful
| searching/selection. Don't get me wrong, I believe it should be
| a choice and not force JSON on users, but for some it's useful.
| redact207 wrote:
| Jq makes life easier. I'd say for complex output, using filters
| is more readable than using awk to extract random positions of
| substrings
| dylan604 wrote:
| why is JSON ungrepable?
|
| grep key file.json | awk -F: '{print $2}'
|
| if you're already searching for a key, seems like you're just
| wanting the value.
|
| granted, i hardly ever (have i actually ever??) interact with
| JSON this way, so i'm not exactly familiar with pitfalls.
| nrclark wrote:
| JSON is greppable if all you need a simple key-value from a
| known format and indentation. It's much harder if you don't
| know the indentation/line breaks, or if it's whitespace-free,
| or if your key can ever appear in your data.
| fragmede wrote:
| The pitfall is that JSON has _zero_ guarantees for how often
| line breaks do and don 't occur, and is often used to
| represent hierarchical data. Grepping for 'key: foo', and
| some liberal use of _-A_ and _-B_ may find you what you 're
| looking for, but _grep_ is simply the wrong tool for that
| job. (And how do you handle a key with newlines in it?) _jq_
| [0] is the right tool, but jq 's syntax is it's own, and is
| harder to use (unless you use it regularly).
|
| [0] https://stedolan.github.io/jq/
| dylan604 wrote:
| when all you have is a hammer, everything looks like a
| nail. i can see rudimentary attempts at trying to get some
| data on a system you might not have full control over and
| are just needing something. i'm sure we've all been logged
| into a remote system maintained by someone else. cavemen
| throwing rocks at the spaceship type scenarios.
| b3morales wrote:
| When it's not formatted, just emitted as a single line.
|
| (Granted grep still _works_ , but...not nicely.)
| dylan604 wrote:
| gotcha. i'm imagining a pretty gnarly regex to return the
| data after the first colon after the match but only up
| until the next comma or square bracket or curly bracket.
| yeah, that's unpleasant.
|
| thankfully, there are tools like jq to help maintain one's
| sanity
| akyoan wrote:
| To be honest I find it funny that anyone would attempt to
| "parse" such a simple format with the wrong tool when
| there are plenty that can extract data in dot notation.
| b3morales wrote:
| Another DON'T, silently switching between JSON and human-readable
| depending on whether the output destination is a pipe. Just an
| extra hassle when I'm writing my downstream command. Or could be
| phrased as a DO: give the user a switch to pick the output
| format, if you have both.
| GauntletWizard wrote:
| I started to try to make an argument for pipe autodetection,
| but I just can't. It seems like a useful feature but is
| actually a trap. Shell scripts that are going to rely on json
| output should always explicitly specify that they want json
| output, and giving them the ability to shortcut that by
| autodetecting a pipe will only make it easy to ignore that -
| And then break when some other format comes into vogue. Human
| readable output should generally be the default, unless the
| programs are explicitly designed as only part of a pipeline.
|
| Having multiple outputs is a great feature, though. I'm
| especially fond of tooling in Kubernetes that allows you to
| nicely pipe things in and out in multiple formats.
___________________________________________________________________
(page generated 2021-12-05 23:00 UTC)