[HN Gopher] The Dawn of a new Command Line Interface (2017)
___________________________________________________________________
The Dawn of a new Command Line Interface (2017)
Author : harporoeder
Score : 120 points
Date : 2021-04-13 10:29 UTC (12 hours ago)
(HTM) web link (arcan-fe.com)
(TXT) w3m dump (arcan-fe.com)
| xbar wrote:
| All the Arcanspace stuff is cool. The problems are real.
|
| The pieces and principles force me to think about many of the
| "skills" I have acquired in order to work on various shells and
| desktops as burdens.
|
| I can't bring myself to change to (or develop) pipeworld/durden-
| esque environments (yet?), I am drawn to them because I have a
| strong, natural alignment to the principles the author espouses
| and develops against.
|
| Keep up the good work.
| _pmf_ wrote:
| Some very true points. The fact that terminal protocols are
| modeled on top of linear output streams is a historical accident.
| Koshkin wrote:
| TN5250 being one of the famous exceptions.
| Semaphor wrote:
| This didn't get any comments when it was submitted in 2017, but
| there were some 2 years ago:
| https://news.ycombinator.com/item?id=21645181
| underwater wrote:
| > Those who don't understand Unix are condemned to reinvent it,
| poorly.
|
| What a low value and needlessly dismissive comment. And based
| on HN's track record it probably means Arcan actually has a
| shot.
| donatj wrote:
| It's a pretty common quote from Introducing Regular
| Expressions (2012) by Michael Fitzgerald, used as a quip.
| oblio wrote:
| He he, recency bias, I guess?
|
| It's from The Art of Unix Programming (2003) by Eric S.
| Raymond. It's actually in the first chapter. And the
| supposedly original quote is:
|
| > Those who do not understand Unix are condemned to
| reinvent it, poorly.
|
| > -- Henry Spencer Usenet signature, November 1987
|
| http://www.catb.org/~esr/writings/taoup/html/philosophychap
| t...
|
| Henry Spencer being:
| https://en.wikipedia.org/wiki/Henry_Spencer . I imagine
| that's why the quote is also in that regexp book.
|
| It's a super arrogant quote used, I'd say, 95% of the time
| by people who don't deeply know other OSes (besides shallow
| memes and jokes about them).
| donatj wrote:
| LOL, I knew I knew the quote, didn't remember from where
| and WikiQuote gave me that source. Should be corrected
| but I'm on mobile at the moment, I'll do so later if no
| one does in the meantime. That'll teach me to trust the
| internet.
|
| https://en.wikiquote.org/wiki/Unix
| cb321 wrote:
| The general structure of the quip/aphorism/joke/snark
| pre-dates the industrial revolution dating back (at
| least!) to Edmund Burke's [1] "Those who don't know
| history are doomed to repeat it."
|
| [1] https://en.wikipedia.org/wiki/Edmund_Burke
| mixmastamyk wrote:
| Sounds similar to:
| https://en.wikipedia.org/wiki/George_Santayana
|
| Looks like Burke lived a century+ earlier.
| oblio wrote:
| Yeah, I know, but the difference is that Burke was
| actually right :-D
| leoc wrote:
| TAOUP didn't make it famous: it was probably about as
| well known as it is now well before 2003. The cited
| Usenet post is real https://groups.google.com/g/sci.space
| .shuttle/c/L8-Upf8gZoY/... .
| Koshkin wrote:
| Even less recently, we have had a very similar sounding
| Greenspun's Tenth Rule.
| oblio wrote:
| The Common Lisp one? Another arrogant and these days
| probably outdated quote that's sometimes trotted out on
| the internet.
| Koshkin wrote:
| > _outdated_
|
| No, CL's popularity hasn't changed over the past 30
| years.
| oblio wrote:
| Yes, you're completely right. The same people using in
| 1990 are using it today.
| rbanffy wrote:
| I have seen more than one generation say this and then,
| after a while, attain enlightenment. Feel free to be the
| next one.
| oblio wrote:
| Programming languages are tools. Tools need to be used to
| build wonderful works of art or industry to be useful or
| have any realistic shot at being considered great tools.
|
| The fact that the overwhelming majority of great
| applications, services, any kind of software we create is
| not written in Lisp should at some point in the first
| century after the creation of Lisp, lead Lisp
| afficionados to some kind of enlightenment, one would
| hope.
|
| Unlikely, since it's already been 60 years, so they only
| have 40 years left.
| Koshkin wrote:
| Agreed. (MS-DOS may have "reinvented UNIX, poorly," but there
| do exist other OS and other approaches to UI that enjoy a
| large and happy user base.)
| bjornjajayaja wrote:
| Honestly one could argue that DOS has some merit beyond
| Unix. It's simple command line interface expects users will
| program in other languages like C or assembly. It's more
| directly interaction with the machine. Sure it had
| limitations like file name lengths etc back in the day, but
| I think many Unix shells are overly complex and ought to
| yield to more suitable languages like Perl.
| mixmastamyk wrote:
| Agreed in general, however shell languages (should) have
| a different focus, such as running processes.
|
| There's been a few attempts at this, but I settled on the
| fish shell as it has a good balance and syntax
| highlighting for interactivity.
|
| I still write simple scripts in sh and complex ones in
| Python however.
| hashkb wrote:
| Many of the standard unix tools are not written in sh.
| They're written in C. Am I missing something in your
| comment?
| ticviking wrote:
| > It's simple command line interface expects users will
| program in other languages like C or assembly.
|
| I think the point here is that DOS explicitly rejects the
| composition that pipes enable, in exchange for the
| ability to just hand "the whole system" over to a
| programmer.
| mixmastamyk wrote:
| DOS had pipes since version 2.0 I believe.
|
| But the main point, that DOS was basically a thin program
| loader that "got out of the way" is sound. Think of it
| like a UEFI shell environment, with 640k+ limits.
| unilynx wrote:
| Dos pipes were just writes and reads to temporary files.
| No parallel processes
| mixmastamyk wrote:
| Ok, but didn't concern the end user, as far as I can
| remember.
| CJefferson wrote:
| I'm interested in progress in this area, but I'm disappointed the
| article seems to ignore the existence of powershell, which does
| some things different to traditional linux shells -- I would at
| least like a discussion of how this related to powershell.
| Koshkin wrote:
| To be fair, the article seems to be more about the terminal
| emulation and the limitations it puts on the CLI rather than
| about the shell language.
| rbanffy wrote:
| Proper VT emulation gives you multiple fonts, mouse and
| windows.
| yourapostasy wrote:
| The "key issues to address" list explicitly calls out many of
| the challenges Powershell manages with its object-based view of
| the world. It looks like Arcan's SHMIF layer is meant to let
| developers using Arcan capture rich data and metadata before it
| gets smooshed into the VTxxx emulation layer and smeared away
| in a sea of screen formatting characters.
|
| Where Arcan cannily (IMHO) differs from Powershell is it
| acknowledges the transition from existing toolchains will be
| organic instead of demanding wholesale operational changes to a
| different processing model. So it splits the data and metadata
| processing pipeline, exposes it at key points to the developers
| using Arcan, and lets an emulation layer cover legacy use
| cases, while enabling Arcan developers to create much richer
| data representations underneath the covers that can be tapped
| if desired by users via Lua manipulating the many different
| Arcan components (I think). This opens up hybrid GUI/TUI/other-
| interface best-of-breed/mixed-mode options that are
| unreasonably challenging to implement (especially with sync
| logic) with current UI stacks.
|
| I like this approach more than Powershell's, but it remains to
| be seen how Arcan handles the guidance of the data and metadata
| ontology and resultant taxonomy; it is a much deeper problem
| space than just defining a default namespace, a way to
| modularly plug into it with extensions, then standing back and
| letting developers have at it. The way Powershell handled it,
| introspection and discoverability are out of band instead of a
| batteries included experience, and module management is still
| split out of band between Nuget, PrivateGallery, custom
| solutions, hand-curated Git+Jenkins lash-ups, _etc._
|
| For my small-scale, personal projects like for my family's
| intranet, the lack of an opinionated default for at-scale
| management of these issues is not a problem. But at my clients
| with dozens, hundreds and even thousands of developers, it
| presents software engineering management challenges. Solvable,
| just a developer experience hygiene factor I'd rather not deal
| with because solving it adds zero business value, it's just the
| cost of doing business.
| donatj wrote:
| Is it possible to return structured data in PowerShell from
| non-.Net languages? I am genuinely asking here, it's been years
| since I've poked it. If not though that seems like an enormous
| stumbling block.
| kaba0 wrote:
| Well, it can export as json, xml and I'm sure some other
| formats as well, so in principle it can.
| rbanffy wrote:
| That sounds interesting, but either programs deal with
| schema transforms, or we'll have to write a lot of that
| kind of logic in XSLT or jq (kind of the same stuff we do
| with cut, sed, awk, sort and grep for unstructured
| streams). Not sure it's that useful a feature unless the
| domain of the programs is narrow.
| kaba0 wrote:
| To be honest, I've been thinking about a shell based on
| the JVM. Especially now with GraalVM, you can have truly
| polyglot programs running on top of it, with inlinig
| cross-language boundaries! But of course native code
| execution should be "bootstrapped" into that world
| somehow. (Actually, it can use LLVM IR and that as well
| can run a C function on js objects which is phenomenal,
| but people do want to run native software from a shell)
| rbanffy wrote:
| While you'll be able to invoke methods on returned
| objects, if you are piping stuff, you'll still need to
| deal with hammering the output of one program into the
| input of another.
| TeMPOraL wrote:
| I think it could be useful - from PowerShell's
| perspective, all you need is to wrap your "classical"
| executables to somehow accept and emit .NET objects, and
| you can carry on from there. Importantly, no more writing
| parsers in the pipeline or in your scripts.
|
| I could see a whole library of such wrappers being built
| for common executables (and new popular OSS CLI tools
| could provide their own wrapper along with the tool).
| rbanffy wrote:
| You can wrap your .NET, JVM, Python, Perl or C
| functionality in a small program that links to the
| library you want, that deserializes your XML or JSON,
| reads your command, and acts on the incoming object
| stream, spitting it to stdout in a neutral, preferably
| human readable, format that can then be piped into the
| next command.
|
| Oh! Wait! This is what tools like cut, jq, grep, awk, and
| sed to.
|
| And they won't tie you to a certain object model.
| TeMPOraL wrote:
| > _Oh! Wait! This is what tools like cut, jq, grep, awk,
| and sed to._
|
| Yes! Except, you have to use them to write one-shot
| parsers that are specific to both what you're piping from
| _and_ what you 're piping to! For n commands you may want
| to arbitrarily mix, that's n2 parsers to write.
|
| > _And they won 't tie you to a certain object model._
|
| From the point of view of a system user, tying to a
| certain object model is a _feature_. It facilitates
| commands interacting well with no parsers in between. But
| even if you don 't have a common object model, just
| wrapping the commands to produce structured output can
| already be helpful, because the parsers you'll be writing
| to glue commands will operate on structured, higher-level
| concepts. It's easier to map between two different
| representations of what a "file" or "process" is, than it
| is to manually dig them out of an unstructured text
| stream.
|
| To use a stupid example, assuming hypothetical "kill-and-
| announce" that echoes a process name and sends a SIGKILL
| to its PID: ps aux | awk '{ ...extract
| columns... }' | sed 'rename columns' | xargs kill-and-
| announce ...
|
| Would be harder to figure out than:
| wrapped-ps | wrapped-kill-and-announce
|
| Or even with incompatible object models:
| wrapped-ps | select Target={id:$_.PID, name:$_.name} |
| wrapped-kill-and-announce
| fiddlerwoaroof wrote:
| This is one of those eternal debates I think: on the one
| hand you have OOP and certain sorts of FP advocates
| advocating for extremely rich data models that eliminate
| certain classes of errors by design. On the other hand,
| you have the Unix notion of using a very generic data
| type (see also Perlis's "rather 100 functions on one data
| structure than ten functions on ten data structures") to
| maximize flexibility, at the cost of a certain amount of
| footguns.
|
| Personally, I'm discovering more and more that CL's REPL
| in SLIME is my preferred shell in part because the CL
| ecosystem has a rich array of operations on a small
| number of data-types (without forcing everything into
| binary streams), but also the ability to define ad-hoc
| datatypes as needed and a powerful system for
| manipulating them and integrating the custom data types
| into existing protocols.
|
| Also, the claim that you need n^2 encoder/decoders isn't
| actually true: conventions around data formats can reduce
| it to O(n) in the number of conventions.
| TeMPOraL wrote:
| > _This is one of those eternal debates_
|
| Myself I know I'm stuck here. I've been thinking in
| circles for a while. At the moment, I'm leaning towards
| rich type system and structured data, but I'm not sure if
| it can be made fully convenient for the shell.
|
| Do you know if anyone attempted to summarize this debate
| and distill some conclusions, or reframe it in a way that
| would allow to make progress?
|
| > _I'm discovering more and more that CL's REPL in SLIME
| is my preferred shell_
|
| In terms of command line or scripting? SLIME is awesome,
| particularly with its interactive Inspector :). And I
| wrote my fair share of "scripts" as CL functions. But I
| can't seem to get used to using the REPL as a shell; I
| feel something is missing. Most likely convenience
| functions (that I didn't bother to write for myself yet).
|
| That whole paragraph you wrote applies to another Lisp
| system, though: _Emacs_. In recent years, I 've been
| leaning on it more and more for day-to-day operations.
| For instance, I rarely use shell for file operations - I
| mostly stick to dired. I end up scripting common
| operations straight in Emacs Lisp, and bind them to ad-
| hoc UIs (using the Hydra package). And of course utilize
| well-known packages like Magit (for Git). This recent
| experience makes me think that we could pull more common
| shell interactions and turn them from 1D to 2D interface.
| And I keep dreaming about merging it with PowerShell, in
| a way that allows Emacs to ingest and manipulate .NET
| objects nicely, while at the same time exposing its
| internals to PowerShell.
|
| > _Also, the claim that you need n^2 encoder /decoders
| isn't actually true: conventions around data formats can
| reduce it to O(n) in the number of conventions._
|
| I think this assumes conversions are lossless, which
| isn't the case in UNIX piping. When you invoke "tools
| like cut, jq, grep, awk, and sed", it's almost always to
| filter out most of the text in the pipeline.
| pjmlp wrote:
| Powershell is great, it is the closest of the REPL experience
| from Xerox PARC, Lisp Machines and ETHZ workstations.
|
| The experience could be replicated in UNIX shells, but noone
| sems to care to actually go down that path.
| ticviking wrote:
| I mean it's possible to run PowerShell on Linux, but
| PowerShell has some pretty significant rough edges, including
| a pretty painful performance story for something I want to
| use as a UI.
| pjmlp wrote:
| It is also a matter of culture, as mentioned you could have
| a UNIX shell being structured, able to directly interact
| with shared objects, IPC with running applications, yet
| very few do it.
| TeMPOraL wrote:
| I feel that the closest we can get to a Lisp machine with
| modern components right now would be to get PowerShell, Emacs
| and StumpWM (or EXWM) to deeply interoperate.
| rkangel wrote:
| It is worth noting that this article is from 2017.
| p_l wrote:
| Which is over 5 years after Powershell became common.
| INTPenis wrote:
| Yes but some would argue Psh didn't become useful until
| release 6 in 2018.
| indymike wrote:
| As long as I have to write C# wrappers to launch commands
| like I would in unix with xargs or parallel in PowerShell
| it's still not the ticket.
| p_l wrote:
| And neither of those are native parts of shells but
| external applications, dare I say, just like the C#
| wrappers. (Especially parallel, since it's nowhere close
| to default install, and has very pesky license notice).
|
| OTOH, PS 7.0 added ForEach -Parallel
| indymike wrote:
| Yeah, but & is a thing and will do the job nicely as long
| as you don't try to pipe output to the same file... also,
| you can invoke pretty much any command with xargs... the
| issue was escaping escaped stuff with PowerShell.
| Hopefully things have gotten better.
| TeMPOraL wrote:
| You shouldn't need to, unless your PowerShell
| installation doesn't support Invoke-Command, or ForEach-
| Object -Parallel...
| indymike wrote:
| Yeah... shouldn't need to but... there are issues with
| escaping command line parameters with invoke-command and
| ForEach-Object that prevent parallelizing older cmd-style
| command line applications. The work around is to write a
| wrapper in a compiled language and use PowerShell to
| invoke the wrapper... There's a lot of really cool stuff
| in PowerShell.
| TeMPOraL wrote:
| > _there are issues with escaping command line parameters
| with invoke-command and ForEach-Object that prevent
| parallelizing older cmd-style command line applications._
|
| Can you tell me more? So I know what to expect when I hit
| such issue myself.
|
| > _The work around is to write a wrapper in a compiled
| language and use PowerShell to invoke the wrapper..._
|
| Wonder if you need to really break out of PowerShell for
| this.
|
| Recently when doing some scripting at work, I ended up
| writing .NET code straight in the script. It looks like
| this: $someCSharp = @" using
| System; //... public class Foo : Bar {
| public override object Baz(Quux arg) { //...
| } } "@ Add-Type $someCSharp
| -ReferencedAssemblies C:\Path\To\Some\dependency-I-
| need.dll
|
| This makes PowerShell compile your code into an in-memory
| assembly and make it available in the session, just as if
| it was an external .NET type. You could probably make a
| module out of it. Hell, cmdlets are .NET types
| underneath, you can make new ones from C#, so you could
| probably make such ad-hoc .NET classes available as
| PowerShell cmdlets.
|
| I suppose using a proper .NET development environment for
| this would be more kosher; I arrived at the solution
| described above because my needs were simple, like
| binding .NET regex engine to SQLite (to get REGEXP
| operator in queries), or convincing Invoke-RestMethod to
| stop checking certificate validity.
|
| Still, I love the capability, the fact that I can _just_
| drop into C# in the middle of the script. It 's one of
| those things that make me think of Lisp machines and
| Common Lisp the experience of being able to _deeply_
| interact with your environment, and the joy of writing
| "scripts" that are automatically AOT-compiled (like with
| SBCL implementation of Common Lisp).
| p_l wrote:
| That would be... an opinion. Not very grounded one unless
| one's whole base for it was "I couldn't use it on Linux".
| crazyloglad wrote:
| There was no point in bringing up powershell at the time, its
| legacy is an entirely different beast from vtXYZ and would
| invite the next "why isn't hyper.is covered" and "why isn't
| upterm covered and ..."
|
| The fabric under attack was the bastard display server protocol
| itself and not the parts that tie in with the strong points of
| Powershell, i.e. shell UI design or data exchange format
| specifics.
|
| This story was continued in a post from yesterday that fizzled,
| https://arcan-fe.com/2021/04/12/introducing-pipeworld/#pipel...
| marlowe221 wrote:
| I work in a Windows shop (security compliance reasons for
| federal government contracts). Before I got this job I was
| pretty much only developing in Linux.
|
| Honestly? Powershell is pretty cool, though it feels a little
| strange to admit that. It has it's own very powerful syntax and
| virtually all of the common Bash commands are now aliased by
| default, so it's pretty smooth transition for the basic stuff.
| richardwhiuk wrote:
| See https://github.com/PowerShell/PowerShell/pull/1901
| nottorp wrote:
| At a quick look this means you'll need to use a special ssh
| client to use their framework?
|
| Terminals are decoupled from the actual application for a good
| reason...
| em-bee wrote:
| yes, but that's exactly why we are stuck and can't evolve.
|
| if we want to get something better than current terminals, a
| new protocol and client are needed.
|
| take mosh for example. it uses a different protocol than ssh
| too, which adds some interesting features. and it replaces ssh
| (except for relying on ssh to make the initial authentication)
|
| there is no reason why another alternative to ssh could not be
| created. if it's good, it will catch on. just like mosh which
| is slowly growing, and also like ssh was in the very beginning
| when it slowly grew to replace telnet.
| nottorp wrote:
| Yeah but that thing seems to suffer from architecture
| astronautitis and a desire to take over every function just
| like systemd...
|
| While, you know, ssh carries your bits and the terminal
| displays said bits, just like it displayed the telnet bits
| before. Decoupled is good.
| em-bee wrote:
| fair point. actually, i think i may have misunderstood the
| problem. ssh can tunnel anything, so whatever new protocol
| is devised, it should actually work though ssh.
|
| also i am not disputing that a new system could not be
| decoupled. i am just concerned that it should not be
| required to work with ssh if that becomes a limiting
| factor.
|
| architecture astronautitis is certainly bad, but maybe it's
| possible to create an implementation with a limited subset
| of features that are actually practical.
| taeric wrote:
| I'm not sure I agree with some of the key issues. In particular,
| the points on code/data and user input versus pasted input.
|
| By and large, I like knowing that I am in control of what I send
| a program on the terminal. It is actually maddening when I go to
| paste into a word processor and it doesn't just bring over the
| text. Or when I go to script a program, but it doesn't act sanely
| when driven by a script instead of by a user. Where sanely is,
| exactly the same.
|
| There are times when I have taken advantage of eshell to inline
| images and such. That said, I don't use many tui style programs,
| other than emacs. And I greatly appreciate that the majority of
| emacs buffers are textual buffers.
| globular-toast wrote:
| This seems to be more about text-based user interfaces rather
| than command-line interfaces. I wonder if the author has
| considered Emacs? A lot of people think of Emacs as a "text
| editor", but that's far from the truth of what it is. It's really
| a Lisp interpreter and toolkit especially designed for making
| text-based user interfaces.
| yourapostasy wrote:
| It looks like the problem space the author is addressing is
| broader than Emacs; they seem to want to make TUI's one of many
| different, first-class-citizen presentation layer options,
| interacting with many different input mechanisms. A Lego
| approach to UI frameworks basically as far as I can tell.
|
| It would be interesting if there was an Arcan-philosophy UI
| framework in CL for an updated Climacs on top of a bare metal
| Lisp environment, for a turtles-all-the-way-down CL experience.
| It would open up interesting interactions with Emacs that are
| an unreasonably big pain to implement today, like buffers that
| manipulate view ports into observability data under a pluggable
| UI that renders graphically a graph data-oriented view of the
| discovered structure of the data under an investigation taking
| notes into a Jupiter-like notebook with rich multimedia data,
| for instance.
| taeric wrote:
| I'd actually be surprised if you couldn't do some of this
| with emacs today. Falling back to the text is just hella
| repeatable in ways that most binary forms are not.
| MayeulC wrote:
| One thing I've been thinking of: how about making typed file
| descriptors?
|
| Besides stdio and stdout, one of the best-accepted conventions on
| modern operating systems running a graphical shell seems to be
| the clipboard.
|
| Programs can offer different mime types trough a clipboard, so
| why not do so trough a pipe or file descriptor? That would bring
| us a bit closer to powershell, while remaining backwards-
| compatible. cat pipe.dot <- text/plain
| | dot <- text/plain, text/dot, image/svg+xml, application/pdf,
| application/postscript, etc | xclip
|
| Instead of xclip in the above example, one could use an image
| viewer, a multiplexer a bit like tee (1) but to redirect to
| standard input, etc. One could force a pipe to filter out some
| mimetypes, or use a program dedicated to that. One could imagine
| bidirectional communications over a channel.
| xeyes | ssh somehost record-audio.sh | mail
| someone@somewhere nmtui | guitoolkit rsync |
| guitoolkit ssh somehost rsync | guitoolkit
|
| Basically, building on the strengths of UNIX.
| Someone wrote:
| The clipboard nowadays is populated lazily (you do not want
| that _dot_ call to generate Postscript, svg, pdf, png, gif, and
| whatever else it can generate while, in the end, you'll need
| only one of them), so you would have to keep those processes
| running until you know nobody needs the data anymore.
|
| That would be quite a change from the traditional pipe model.
| MayeulC wrote:
| Oh, but the traditional pipe model can also produce data
| lazily (wait until a given FD is open before producing
| output), so that wouldn't be that much of an issue.
|
| That wouldn't change much from the pipe model. At least on
| wayland, wl-copy forks to the background. If you wanted the
| producer to finish producing data and exit, surely you could
| make a flag for it in the receiving program, or pipe it
| trough one that does.
| Someone wrote:
| This is different. You need communication back through the
| pipeline to inform the data producers what data you want.
|
| Also, in your example ending in _| xclip_ , xclip cannot
| know what format data it has to ask from dot until it gets
| pasted somewhere. Alternatively, it would either have to
| guess what format(s) the user would like to paste later or
| it would have to cover all bases and ask _dot_ for output
| in every format it supports.
| TeMPOraL wrote:
| Come to think of it, you could abuse NTFS Alternate Data
| Streams for this.
|
| Imagine: > # See alternate data streams >
| Get-Item diagram.dot -Stream * Stream Length
| ------ ------ :$DATA 123
| text/plain 4444 image/svg+xml 65432
| image/png 456134 > # Send the whole bulk
| > Get-Item diagram.dot -Stream * | ... > #... or
| pick just one: > Get-Content diagram.dot -Stream
| "image/svg+xml" | Send-Email ....
|
| Would have to ensure everything is aware of the streams and the
| convention, though, so that you wouldn't accidentally truncate
| them when copying files around.
| monocasa wrote:
| Not just NTFS, but pre OSX Macs would do stuff like this.
| Like a formatted document would have a text/plain fork, and
| the formatting information in another fork. Or an executable
| could keep it's symbol table or relocation information in
| other forks.
| TeMPOraL wrote:
| Thanks! I just checked, and it seems that ADS were
| introduced in NTFS in order to support Mac resource forks.
|
| https://en.wikipedia.org/wiki/NTFS#Alternate_data_stream_(A
| D...
| MayeulC wrote:
| Ah, that's interesting, thanks for mentioning it. That's
| pretty much what I mean, but on arbitrary file descriptors
| instead of files.
|
| I agree that both ends need to be aware of these extra
| formats to make use of them, but just like with clipboards,
| programs could always fall back to pretending only text/plain
| exists :)
| HelloNurse wrote:
| Typing, in the programming language sense, is a major can
| of worms that a general purpose shell usually avoids with a
| deliberately bare bones type system (e.g. text, binary,
| directory and symlink).
|
| How can a shell do anything serious with _asserted_ file
| types that can be missing, wrong, malformed, unknown or
| mendacious? It 's hard to go beyond offering a best-effort
| "open with the associated program" application as an
| alternative to vi, cat etc.
| MayeulC wrote:
| > How can a shell do anything serious with asserted file
| types that can be missing, wrong, malformed, unknown or
| mendacious? It's hard to go beyond offering a best-effort
| "open with the associated program" application as an
| alternative to vi, cat etc.
|
| How would that be any worse than the status quo, though?
|
| Programs already output, depending on their invocation
| and the utility at hand, binary data in arbitrary
| formats, shell escape sequences, text, ASCII art, etc.
|
| The only issue I can think of is for portability of the
| scripts, if a program gets updated to support a new type
| as input or output, and that causes issues compared to
| the testing environment. But that's arguably already an
| issue too, and it would be rather simple to annotate
| expected types.
| jodrellblank wrote:
| Why not simply make the pipe explicitly HTTP?
| first | second | third
|
| "first" starts up a HTTP server, "second" is a HTTP client
| which makes a standard HTTP request to "first" declaring in the
| headers what content types it can support, and "first" returns
| a standard HTTP response declaring in the headers what kind of
| content it is sending. "Second" is also a HTTP server for
| "third" in the same way, and so on down the pipeline. All the
| OS has to do is coordinate a free TCP port for each pipe. The
| HTTP client can simply be a lightweight headless Chrome
| Electron bundle, and the whole pipeline gets all the benefits
| of HTTP (standard UTF-8 encoding, compression), with only a
| small overhead of cookie policies, cross pipe scripting, modern
| auth tokens, 2-factor authentication between client and server,
| SAML, and a tiny surface area increase for security concerns
| but only a few dozen MB or so.
|
| Plus the pipeline then has ongoing two-way communication up and
| down it.
| karatinversion wrote:
| This is awful, I love it.
| MayeulC wrote:
| The idea isn't that bad.
|
| Obviously the following can't be taken at face value:
|
| > lightweight headless Chrome Electron [...] with only a
| small overhead of cookie policies, cross pipe scripting,
| modern auth tokens, 2-factor authentication between client
| and server, SAML, and a tiny surface area increase for
| security concerns but only a few dozen MB or so.
|
| However, there's probably much to be learned from the history
| of the web when it comes to security and compatibility
| between different implementations and successive evolutions
| of a protocol. http certainly has merits, given its long
| history. It isn't flawless either, of course.
|
| Even features such as compression could make sense at first
| glance, though they would be a better fit for a dedicated
| tool like piping to zstd, since you control both ends of a
| pipe. Range requests, multiplexing, server-sent events, etc,
| could have some use. And it isn't like making a http server
| or client is so hard with a small library, given you already
| pull in libc or worse.
|
| http probably wouldn't be suitable as-is, certainly not over
| TCP (though the fopen/fclose dance is quite close to it), but
| making it a fully fledged bidirectional protocol could be
| interesting, especially if you can somehow build on top of
| the system IPC on a microkernel. It could be a way to forward
| hardware access and such.
| devenblake wrote:
| This is a dangerous joke to make / idea to perpetuate given
| the audience of HN.
| TeMPOraL wrote:
| Agreed. I'm tempted to simultaneously upvote and flag this
| idea :).
| hnlmorg wrote:
| This is something I've been working on with my shell,
| https://github.com/lmorg/murex
|
| The problem isn't so much finding a way of passing metadata but
| how do you retain compatibility with existing GNU / UNIX tools?
| It would be easy to create an entirely new shell (like
| Powershell) that throws out backwards compatibility but that
| wouldn't be very practical on Linux/UNIX when so much of it is
| powered by old POSIX standards.
|
| I addressed this by having a wrapper around pipes. Anything
| that connects out to a classic executable is a traditional
| POSIX byte stream with no type data sent. Anything sent to a
| command that is written to support my shell can make use of
| said meta data.
|
| As for handling graphical components, that's a little easier.
| Many command line programs already alter their behaviour
| depending on whether STDOUT is a TTY or not. I take things a
| little further and have a media handler function, called
| `open`, that inlines stuff in a shell that can be (it will
| render images in the terminal) but if STDOUT is a TTY then it
| will pipe the image data instead.
| MayeulC wrote:
| Wow, your shell seems to be pretty far along.
|
| > The problem isn't so much finding a way of passing metadata
| but how do you retain compatibility with existing GNU / UNIX
| tools
|
| Honestly? I'd have suggested a new syscall that provides an
| interface similar to _isatty()_. It seems a matter of adding
| a few constants to the _ioctl_ syscall, prototype it and
| prove it is useful, then upstream it. Better make a crude
| prototype, see if that 's any useful, and ask upstream what
| they think of it.
|
| If the new ioctl isn't available, nothing changes, programs
| continue exchanging whatever they expect. If the new
| interface is available, most programs won't know about it,
| and would continue their business as usual. However, programs
| that know about it could get more metadata trough it, and
| communicate in a smarter way. Looks like a standard way of
| introducing new functionality, can be deprecated if needed,
| and keeps backwards-compatibility, at the expense of a tiny
| bit of complexity (dwarfed by having to support multiple
| formats, which some tools already do).
|
| The way you do it seems quite fine too, not sure which would
| be better.
|
| Staying with traditional file descriptors also has the issues
| that you are limited to unidirectional communication, and
| only one file descriptor, so maybe you need a protocol like
| html, as a sibling joked.
|
| > As for handling graphical components, that's a little
| easier. Many command line programs already alter their
| behaviour depending on whether STDOUT is a TTY or not.
|
| Well, that's kind of my point: you could output more than
| curses interface if you wanted, something like an xml ui
| file, or a list of commands on a side channel. But I make no
| judgement on this: it was just to illustrate what kind use-
| cases this would enable.
| sime2009 wrote:
| A possible way of doing what you describe might be to add an
| extra file descriptor or two when invoking processes. Along
| side the normal stdin, stdout, and stderr, you might have a
| couple more where a protocol can be used to negotiate and share
| typed data. It might even be possible to do it a backwards
| compatible way. (If in an app the extra file descriptors are
| missing, then fall back to old school stdin/stdout.)
|
| This blog from years ago describes something similar:
| http://acko.net/blog/on-termkit/ i.e. expanding the process
| model
| MayeulC wrote:
| Thank you for the blog post, it resonates quite well with me.
|
| I thought a bit about this, but that becomes a bit
| complicated without additional metadata on a file descriptor
| (is it _really_ that additional interface, or just a random
| FD we inherited?). It might be possible to achieve in a
| robust way with mount namespaces and named pipes, but that 's
| another can of worms...
|
| That said, it would likely work 90% of the time that way.
| sime2009 wrote:
| I pay attention to things in this area as I've been working
| a terminal which aims to do new and interesting things with
| text and also non-text datatypes. See "Shell integration"
| and below at https://extraterm.org/features.html
|
| Integrating some new shell projects like murex and nushell
| with Extraterm is on my TODO list.
| willis936 wrote:
| One issue with windows command line I've run into recently is
| text rendering performance. Running cxxmatrix at a high
| resolution brings even a modestly powerful machine to its knees.
| cxxmatrix crashes alacritty, but I have found that running VcXsrv
| and urxvt from wsl yields _much_ better cxxmatrix performance
| than when running from wsl.
|
| Here is a related issue. There are many cousin issues.
|
| https://github.com/microsoft/terminal/issues/2047
| dan-robertson wrote:
| I think emacs is a good example of a terminal program (not really
| command line) that made a good success out of becoming more gui
| and I think it shows a lot of the limitations of the terminal
| emulators. In a gui you easily get all the colours and not many
| weird duplicate keyboard shortcuts (exceptions are eg TAB vs C-i
| and M- vs ESC). You can have images and clicking and middle click
| paste and multiple fonts and some OS integration. But the
| interface still remains a fundamentally mutable text-based thing.
|
| For something completely different: https://hisham.hm/userland/
| legulere wrote:
| What I find an interesting is command interfaces in GUI
| applications. For instance ctrl + shift + p / cmd + shift + p
| menu of visual studio code or the new windows terminal, or the
| commands in slack. A GUI gives way more options for displaying
| information especially for auto-complete even while staying in a
| keyboard-only interface.
| simias wrote:
| I feel towards the un _x shell the same way I feel towards Vim:
| there 's quite a bit of cruft, some poor decisions, quite a bit
| I'd do differently if I decided to write my own implementation
| from scratch. But on the other hand it mostly works well and I've
| learned over the years how to work around the awkward parts.
|
| It's easy to make a laundry list of the ways the traditional unix
| shell falls short. The problem is creating an alternative that's
| superior enough to warrant learning something new and consensual
| enough that it can get mainstream appeal.
|
| The article acknowledges that:
|
| >There is arguably a rift here between those that prefer the
| 'shove it in a browser' or flashy UIs that animate and morph as
| you interact, and those that prefer staring into a text editor.
| It seems to me that the former category gets all the fancy new
| toys, while the latter mutters on about insurmountable levels of
| legacy. What I personally want is many more "one- purpose" TUIs
| and for them to be much easier to develop. They need to be
| simpler, more consistent, obvious to use, and more configurable.
| That's nice and dreamy, but how are "we" supposed to get there?
|
| But then I argue that they jumps the shark a bit. As far as I can
| tell the Arcan project seems more preoccupied with reaching
| feature-parity with Xorg than actually improving the CLI. And I'm
| sure for some people that's great, but for me personally that
| makes it a funny novelty thing that I don't see myself using. It
| reminds me when I first saw compiz ~15 years ago, where you had
| virtual desktops on a rotating 3D cube with all sorts of weird
| visual effects[1]. It looked cool, but I never actually used it
| because it wasn't practical and I had no use for it.
|
| On the other hand I'd love for a better CLI implementation,
| better pipes, easier debugging, a more expressive scripting
| language, better task control, better feedback.
|
| But then we went full circle to the core of the issue: everybody
| agrees that the un_x CLI sort of sucks, but we don't really agree
| on what exactly needs fixing.
|
| [1] https://youtu.be/B-FVJoZ-zwI
| dale_glass wrote:
| That's odd, I found the Compiz cube to be helpful actually.
|
| As a graphics effect it's about as basic as it can get -- even
| the venerable GLX Gears is more complex. And I found it to
| actually increase usability because it visually and very
| intuitively showed you the movement, so it made organization a
| bit neater. It made it easier to keep a mental map of what was
| where, because it could be arranged spatially.
|
| If anything, it didn't go far enough. I wanted more faces, and
| the possibility of working in a "zoomed out" state, for when I
| need to look at a whole bunch of stuff at once.
|
| The real novelty stuff to me were the Enlightenment effects
| like the water effect on the bottom of the screen -- now that
| chewed up CPU and did nothing of actual use.
| Nursie wrote:
| I liked the cube for the same reason - virtual desktops just
| kinda clicked when I could rotate the cube to get to the
| stuff running just off screen. I've never really bothered
| with them before (or since).
|
| You could make the 'cube' have an arbitrary number of faces,
| but I don't think there was the zoom mode you talk about.
| dale_glass wrote:
| I recall not being able to use the top and bottom faces,
| for instance. But maybe that was just the KDE
| implementation.
| Nursie wrote:
| True, you could not when using compiz with gnome either,
| as I did back then.
|
| You could add more faces to the cube so that it became a
| slice of a pentagonal prism, hexagonal prism, heptagonal,
| octagonal etc. Or take one away so you were working on
| the sides of a triangular prism.
|
| But you're right, there was no vertical freedom of
| movement. You couldn't use a Rhombicuboctahedron, cool as
| it might be :)
| (https://en.wikipedia.org/wiki/Rhombicuboctahedron)
| hyakosm wrote:
| I was searching for the quote because the text is italic, and I
| realized it's because you wrote UNIX with a star two times (for
| Unix-like ?)
| [deleted]
| theamk wrote:
| this is interesting -- I was expecting to see a replacement for
| existng ttys, and was planning to complain about ssh forwarding,
| redirection, etc... instead, they explicitly say:
|
| > Recall that the prime target now is local text-oriented,
| command line interfaces - not changing or tampering with the awk
| | sed | grep | ... flow, that's an entirely different beast.
|
| So I'd call this a "TUI" (text user interface), and the proposed
| schema replaces X11 or Wayland, not xterm.
___________________________________________________________________
(page generated 2021-04-13 23:01 UTC)