[HN Gopher] The Linux Command Line
___________________________________________________________________
The Linux Command Line
Author : Brajeshwar
Score : 198 points
Date : 2022-12-28 14:09 UTC (1 days ago)
(HTM) web link (www.linuxcommand.org)
(TXT) w3m dump (www.linuxcommand.org)
| kaycebasques wrote:
| How Linux Works (also published by No Starch) covers a lot of
| useful commands too: https://nostarch.com/howlinuxworks3
| aquova wrote:
| I just got this book for Christmas, it's a really excellent
| guide into the internals of a Linux system.
| quijoteuniv wrote:
| Want to change the world? Write books like these!
| channel_t wrote:
| I consider this book pretty much required reading for junior devs
| and others in engineering who don't have a decent handle on basic
| Unix skills for whatever reason--which seems to be a more common
| occurrence these days than earlier in my career.
| chasil wrote:
| It is a very kind gesture to make this book freely-available.
|
| It does have a small problem in the number of "bashisms" that
| it quietly promotes.
|
| The entire Debian/Ubuntu family has chosen the Almquist shell
| because of speed and standards compliance, and many things in
| this book will not work there.
|
| It would be helpful to have a book that is clear that the
| POSIX.2 shell is not Bash.
| deathanatos wrote:
| IMO, the year is 2022, almost 2023. Trying to do POSIX sh is
| tying both arms behind your back. Set the shebang to
| #!/bin/bash, and now you've got at least got only one arm
| tied behind your back.
|
| If your environment somehow lacks bash, consider _that_ the
| bug, and get bash.
|
| Or, untie both arms, and use a language that's not going to
| take every opportunity to stab you in the back with subtle
| idiocies dating from the last millennium.
|
| (But I do agree that, if you're going to put down a bash-ism,
| it's bash, not POSIX sh.)
| chasil wrote:
| May I suggest that you attempt to convince The Austin
| Group/Open Group of your position, to update what was
| originally known as the POSIX.2 shell standard?
|
| https://pubs.opengroup.org/onlinepubs/9699919799/utilities/
| V...
|
| http://www.opengroup.org/austin/faq.html
| SSLy wrote:
| The only entity that still respects the Open Group is the
| sod submitting the SUS/POSIX submissions for macOS on
| behalf of Apple.
| _dain_ wrote:
| posix is a messy codification of a situation that existed
| at a particular point in time. it didn't arrive ex
| nihilo.
|
| if enough people decide to just use bash, then that will
| be the new standard. we don't have to ask for some
| committee's permission to advance.
| hsbauauvhabzb wrote:
| > Or, untie both arms, and use a language that's not going
| to take every opportunity to stab you in the back with
| subtle idiocies dating from the last millennium.
|
| Are you still referring to bash here?
| mustache_kimono wrote:
| > It would be helpful to have a book that is clear that the
| POSIX.2 shell is not Bash.
|
| Maybe the one thing I love about Linux culture (compared to
| the BSDs, illumos, etc.) is that it generally doesn't require
| everyone to adhere to some high UNIX religion.
|
| Writing a shell script of more than 50 lines is hard to do
| correctly. Bash sometimes makes that a little easier. And
| it's almost ubiquitous. Forcing new Linux users to learn a
| POSIX compliant shell, when they are only likely to be using
| Linux and bash, is simply cruel and unusual. Just as
| including only `vi` in the base system instead of `nano` is a
| silly self-own.
|
| I see this unreflective question far too often: "Why aren't
| people using my system and using Linux instead?" And the
| answer is pretty simple, Linux at least made efforts to reach
| out to people where they were, in hundreds of low effort
| ways, whereas your system probably didn't.
|
| "Everyone should be using Linux" and/or "should love the
| command line" is pretty silly, but just as silly is "people
| should learn the POSIX shell" first. A shell they are likely
| never to use interactively. Because of a hypothetical
| portability concern? It's nuts.
| tremon wrote:
| _just as silly is "people should learn the POSIX shell"
| first_
|
| Where did the GP make that assertion? I don't see anywhere
| in the post where they say "please rewrite all the shell
| examples in POSIX-compliant sh". All I see is a request to
| inform the reader that not every shell is bash.
| chasil wrote:
| The origin of this was actually capitalism that bordered on
| religion.
|
| In the UNIX Wars of old, Sun and AT&T attempted to take
| unreasonable control of System V, which included the Korn
| shell.
|
| DEC, IBM, HP, and others responded with POSIX.
|
| https://en.wikipedia.org/wiki/Unix_wars
|
| The impact upon the Korn shall was actually to _reduce_
| functionality, and a number of features were removed
| (arrays, coprocesses, etc.). The reason for this is that
| the source code for the Korn shell was configured to
| compile to a 64k text segment (for 286-Xenix), and
| readability /maintainability was sacrificed.
|
| Due to this lineage of the POSIX shell, it is very friendly
| for embedded systems and other constrained environments in
| ways that its descendants and competitors are not (and can
| never be).
|
| It deserves to be distinguished from Bash, without
| question.
| mustache_kimono wrote:
| > It deserves to be distinguished from Bash, without
| question.
|
| Absolutely, and I agree with your sentiment at a broad
| level -- although I think the book notes who the book is
| for, and that `bash` and `sh` are distinct, it doesn't
| explain the why of maybe considering `sh` for other uses
| which aren't really the focus of the book, and perhaps it
| should?
|
| > Due to this lineage of the POSIX shell, it is very
| friendly for embedded systems and other constrained
| environments
|
| I'm wondering if this will matter as much in the future.
| Are people really going to be programming 16 bit
| microcontrollers when a 32 bit parts are becoming so
| cheap? How constrained are we really going to be?
| chasil wrote:
| Here is my copy of Busybox for Windows:
| C:\Users\chasil>busybox bash ~ $ a[1]=1 bash:
| a[1]=1: not found ~ $
|
| Despite what Busybox claims, that is very much not Bash.
| That is Almquist, with a few added bashisms.
|
| I was in a clothing store a few months ago, and their
| credit card readers had crashed. They were showing the
| Busybox copyright notice.
| kccqzy wrote:
| In college during the introduction to command-line class
| the professor simply required everyone to write only POSIX-
| compliant shell scripts. And your shell scripts must work
| with newlines in file names. We did fun things like parsing
| (a subset of) HTML using only sed (did you know about sed's
| hold space and pattern space?). It was basically an
| intellectual puzzle for geeks with too much time on hand.
| BossingAround wrote:
| > The entire Debian/Ubuntu family has chosen the Almquist
| shell because of speed and standards compliance, and many
| things in this book will not work there.
|
| So Debian/Ubuntu ship completely without Bash? According to
| [1], it's Bash, but it might be outdated.
|
| [1] https://wiki.debian.org/Bash
| chasil wrote:
| Bash is shipped in Debian/Ubuntu, but as /bin/bash, not as
| /bin/sh.
|
| What is shipped as /bin/sh is described here:
|
| http://gondor.apana.org.au/~herbert/dash/
|
| From the EPEL RPM:
|
| "DASH is a POSIX-compliant implementation of /bin/sh that
| aims to be as small as possible. It does this without
| sacrificing speed where possible. In fact, it is
| significantly faster than bash (the GNU Bourne-Again SHell)
| for most tasks."
| John23832 wrote:
| Hm but the default shell is still bash. It would make
| sense that "bashisms" don't work on "sh". You should use
| bash or bash derivatives (zsh) for bashisms.
|
| I think the bigger stumbling block is that people often
| can't differentiate bashisms from basic shell scripting.
| quesera wrote:
| FWIW, zsh is not a bash derivative.
|
| Both are spiritual descendants of Bourne shell directly,
| with some ksh, csh, and tcsh mixed in, in varying
| proportions.
| gdavisson wrote:
| bash is the default interactive shell for users, but dash
| is the default shell for scripts with a #!/bin/sh shebang
| (or no shebang, unless they're run from bash), system()
| calls, etc. This means bashisms will work _sometimes_ ,
| which can be worse than not working at all.
|
| I agree about the importance of differentiating bashisms
| from basic scripting, but I think the root cause of that
| is documentation (including this book) that don't make
| the distinction.
| layer8 wrote:
| The book is very clear about the fact that it's using
| Bash, even noting when a feature requires a newer
| version, and uses #!/bin/bash for scripts throughout.
| John23832 wrote:
| ...But the foreword for any script could just as easily
| be `#!/bin/bash`... you're choosing not to use bash. If
| you're on a system with no bash, and are forced to use
| `sh`, then just don't use bashisms...
|
| I'm not trying to be difficult, but I just don't see a
| problem. Explicitly use bash (or a derivative) if you
| want bashisms, don't if you do not.
| ghostpepper wrote:
| What's a good alternative book to learn linux command line
| without bashisms?
| chasil wrote:
| I really don't know of one. There should be one.
| seba_dos1 wrote:
| It's not a problem as long as you remember to declare bash as
| the interpreter instead of relying on sh to be symlinked to
| bash.
| John23832 wrote:
| Modern Ubuntu/Debian ship with bash?
|
| ``` ubuntu@primary:~$ uname -a
|
| Linux primary 5.15.0-56-generic #62-Ubuntu SMP Tue Nov 22
| 19:56:13 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
|
| ubuntu@primary:~$ echo $SHELL
|
| /bin/bash ```
|
| dash is the default `/bin/sh` but the default shell is still
| bash. (man sh gives you dash's man)
| chasil wrote:
| Bash is the default interactive shell assigned to new users
| in /etc/passwd.
|
| This is different from /bin/sh, which is Almquist.
| _dain_ wrote:
| So just put `#!/usr/bin/env bash` at the top of the file
| _dain_ wrote:
| what kind of circumstance are people in where they can use
| posix sh but not bash? in the year 2022? I don't understand
| it. fucking everything has bash nowadays. and if you're on
| some machine that unaccountably doesn't have bash, then why
| can't you just sudo apt install bash, or scp the binary over
| and chmod +x it?
|
| if you can't even do that, maybe reevaluate what life choices
| got you to this place. life is too short to write strictly
| posix sh.
| quesera wrote:
| > posix sh but not bash?
|
| Many non-Linuxes, and some Linuxes too.
|
| > why can't you just sudo apt install bash
|
| Linux is not the whole world. Debian is not the whole
| Linux. Interactive shells on servers are not guaranteed to
| be available. Control over the installation image is not
| guaranteed, or always desirable.
|
| > life is too short to write strictly posix sh
|
| Alas. Life is too short for writing or responding to
| uninformed rants. Yet here we both are.
|
| POSIX shell scripting has its place. It's not as difficult
| or complicated as you imagine. The portability tradeoff is
| valuable or essential in many real-world cases.
| velavar wrote:
| Do you think that this is because they don't really need to
| anymore with all the nice GUI/tools around? Is there an
| advantage to bring able to do something on a command line vs
| other ways?
|
| Asking because I'm at a bit of a crossroads - I have a good
| handle on about 5% of the command line knowledge which gets me
| through 80% of the stuff I need to do. I'm wondering if
| learning to use more commands is worth the effort when I can
| already get the task done without using the command line?
| _dain_ wrote:
| Consider each program as a node in a graph, with the edges as
| the possibility of interop -- output of program A working as
| input of program B. With UNIXy command line programs this is
| pretty close to a fully-connected graph. So, per Metcalfe's
| Law, the value scales with the square of the number of
| programs you know how to tape together.
|
| Command line programs compose like this, GUIs don't.
| dinkumthinkum wrote:
| I think a lot of very junior developers now really just don't
| know very much about computers and electronics beyond social
| media.
| type-r wrote:
| When contemplating a future investment into learning
| something, I like to consider the long-term benefits. Do we
| believe that command lines will become more or less important
| in the following years? My bet is that they will continue to
| fade over time, as they have been for years (decades?). I
| don't think there's much sense to invest more than the
| minimum unless there's a specific use case in mind.
| chasil wrote:
| The Bourne shell was a relative latecomer to UNIX, released
| in 1979, replacing the [Ken] Thompson shell.
|
| It lacks many things provided by a POSIX shell (nearly all
| of which are present in the later Korn shell).
|
| There are many entire operating systems that survived only
| a fraction of this time. I think that it will remain
| relevant for decades.
|
| https://en.wikipedia.org/wiki/Bourne_shell
| gofreddygo wrote:
| more commands are a side effect of more capability.
|
| If you get a sense of things you can do with (|, & , >) and a
| dozen or so commands like
|
| sort | uniq | cut | paste | join | head | tail | sed | awk |
| wc | tr | fmt | col | nl | pr | find | grep | tar | export
|
| The value you get in return is intense for the effort
| invested.
|
| Plus, in the 10 years i've been maintaining code, bash
| scripts are the minority that still work as they used to with
| almost no change needed.
|
| That single handedly is worth my money.
| foobarian wrote:
| These days I would include jq in that list as well, near
| the front in fact
| deathanatos wrote:
| And xsv, if you're doing anything with CSV.
|
| A lot of cut invocations I see are on CSV files, and are
| all bugs out of the gate.
|
| (But use JSON, if you can. It's a step up, and easier to
| work with, particularly with jq.)
| zzo38computer wrote:
| Pipes, redirection, scripts, etc are some of the most
| important reasons why command-line is better in many ways
| than GUI. In some cases GUI is helpful, but often, command-
| line is much better in many ways.
|
| For this and other reasons, I will often write programs as
| command-line using standard I/O for most data. For example, a
| music playing program which reads the music file from stdin
| and writes the audio data to stdout which is taken by aplay
| to play the audio on the speaker. (In this way, you can also
| add other thing in between such as special effects, if
| desired.)
|
| (It is also useful for other programs (interactive or GUI or
| whatever) to have the opportunity to open pipes with other
| user-specified programs; Meirloom-mailx is one interactive
| command-line program does it (I often use that feature to
| display pictures attached to messages), and Free Hero Mesh
| (which has GUI) also exclusively uses such pipes to
| import/export levels and pictures and move lists.)
| Nihilartikel wrote:
| It's tough to quantify the knowledge levels, but, I'd say
| that a practical understanding of bash loops, piping,
| grepping, and 'cut' ing text. Is a good start for basic dev
| work.
|
| I've seen hours wasted writing jankey one off c++ utils, for
| lack of knowing that grep/awk exist.
|
| Being able to tail a dev log and filter it for errors is also
| a part of seasoned developer competence too, I'd say.
| randomluck040 wrote:
| I wouldn't call myself seasoned developer but tail (and
| grep) saved so much time during my masters thesis, I don't
| even know how I'd checked for new entries in my log files
| other than using cat. When working with anything posting a
| log, I'm very grateful to know about tail.
| [deleted]
| amalgamated_inc wrote:
| I think that you probably have 80% of the gross capability,
| but you will gain a huge level of refinement. I'm about 10-15
| years into the command line (depending on how you count, lol)
| and I keep finding more stuff that just wouldn't stick 5
| years ago, because I simply didn't have those problems yet.
|
| E.g. there was a time in my life when I didn't get what
| `xargs` even did, I literally couldn't grok it. Now, I can't
| imagine life without `-0`!
|
| The composability of UNIX commands is just so amazing. Every
| command builds on every other command. I haven't seen that
| with any other tools or program.
| tester457 wrote:
| What is the most helpful thing you do with xargs and -0?
| somrand0 wrote:
| depends on how close you wanna get to the servers, and then
| the hardware?
|
| I suppose it's a super rare occurrence now, but back when I
| was starting on this, I would mess up my X server and was
| forced to fix it from the command line.
| xcambar wrote:
| Because developer tools are increasingly polished and easily
| accessible without in-depth knowledge of the arcanes of the
| command line. And that's great to welcome more people in our
| field.
|
| The downside is that once they're welcome, the climb of the
| learning curve is barely started.
| kevin_thibedeau wrote:
| Those polished tools hide a lot of the underpinnings so that
| nothing is understood. We get header-only C libraries (not
| C++) because people don't understand build tooling and want
| something that works on easy mode.
| xcambar wrote:
| What's so wrong about easy mode?
|
| For me, it's hard to make good software today because the
| bar is actually higher, in the sense that it must be usable
| by a much wider audience and I can't be lazy about
| usability or even worse, let it unnecessarily complicated
| because "people should know about it or learn".
| kevin_thibedeau wrote:
| Need to migrate your IDE project to a CI system? How do
| you do that if you only know how to click through a
| wizard? Everything has to be spoon fed and if it isn't
| they are paralyzed.
| xcambar wrote:
| > Everything has to be spoon fed and if it isn't they are
| paralyzed.
|
| Fixed it for you: [...] and WHEN it isn't they LEARN.
| sweezyjeezy wrote:
| I'd say that's a bit orthogonal - command line tools are
| also hiding underpinnings from you. The issue is about the
| UX, not the functionality. Seemingly every command line
| tool has built it's own language of incantations on how to
| use it. I've been using grep, tar, curl, find, xargs etc.
| at least a few times per week for over a decade now now. I
| shouldn't have to look at the man pages this regularly just
| to remember the letters I need to type on something I've
| done 30 times before.
|
| I don't have this problem with the equivalent libraries in
| the programming languages I use, give me 'easy mode'. We
| need new binaries with a modern UX, or a better IDE for
| using the command line.
| kevin_thibedeau wrote:
| Command line tools are composable. They give you
| superpowers by being able to rally a collection of
| unrelated programs into working for your benefit.
| Monolithic GUIs can't do this.
| genericacct wrote:
| GPT and other similar technology will be the new IDE
| soon. Command line gpt tools are already out there and
| they will write bash scripts in seconds.
| BeFlatXIII wrote:
| Easy mode allows one to get to the productive work faster.
| mkesper wrote:
| I was astonished last week again how bad GUIs for git are.
| Take VSCode: It encourages you to enter just the title of the
| commit message. That's the boring part for people used to
| describe WHY that change was made in the long message.
| jalino23 wrote:
| this true. getting started in the industry has a lot of
| assumptions about command line knowledge, every other tutorial
| or book kinda assumes it. it wasn't until 3 years later that I
| realized that I should have learned it sooner.
| vram22 wrote:
| It's a good book.
|
| I had done a review of it as a blog post under the O'Reilly
| Bloggers program which existed some years ago. But it and other
| such book reviews disappeared after a reorg of the site. So only
| a stub remains:
|
| https://www.oreilly.com/pub/blogger/vasudevram
| whitepoplar wrote:
| I found this book to be excellent when I read it, as are most of
| No Starch's books.
| argonaut07 wrote:
| How would you unit test/mock your scripts, following the
| guidelines of this book?
| bottlepalm wrote:
| I'd be cool to type something like `// spin up a lamp stack
| docker image`, and have below it the ghosted auto-completed
| command powered by chat gpt or something.
|
| I'd go so far as to say all command line apps should have an
| -explain suffix, that doesn't actually run the command it just
| parses it and outputs how it understands it in plain text.
| Combined with the above to help avoid AI mistakes.
| dlivingston wrote:
| I'm sorry, but that sounds awful. I turn to the terminal when I
| want precise and exact control over commands - the above seems
| highly fuzzy and error-prone.
| bottlepalm wrote:
| That's why I added --explain, so no matter what it is
| perfectly clear what the suggested command will do.
| nonrandomstring wrote:
| > should have an -explain suffix
|
| Adding -h or --help generally does something useful
| bottlepalm wrote:
| That gives you a list of all the commands, no something short
| and sweet about exactly what the command you have written
| will do.
|
| If --explain is part of the argument parser then it would be
| relatively easy to implement.
| [deleted]
| jimmaswell wrote:
| I bet you could open a bash file with copilot enabled and do
| that right now.
| amalgamated_inc wrote:
| Can't wait for AI to auto-guess that I meant to pass `-f /`
| to `rm`..
| Overtonwindow wrote:
| Related: "In the Beginning... Was the Command Line"
|
| https://web.stanford.edu/class/cs81n/command.txt
| amalgamated_inc wrote:
| So great! Maybe his best work. I re-read it every year or so.
| In fact, I'll re-read it right now.
| saeranv wrote:
| This is what made me interested in using Linux for the first
| time!
| jeroenjanssens wrote:
| Related: Data Science at the Command Line
|
| https://datascienceatthecommandline.com/
|
| Disclaimer: I wrote it. Plugging it here because (a) it's
| related, (b) it's free to read online, and (c) would not have
| existed without HN.
| ykonstant wrote:
| Very interesting work!
| amelius wrote:
| I still get "Argument list too long" on a machine with 64GB of
| memory.
|
| This is not acceptable in this day and age.
| forinti wrote:
| It might protect you from shooting yourself in the foot.
| dinkumthinkum wrote:
| Most command line programs are open source so you can do
| something about that if it really makes sense.
| jeffbee wrote:
| You can't fix this in the tool you are trying to invoke
| because the tool never runs. The error E2BIG is returned from
| the syscall `execve` because it fails to run the program.
|
| If you just want to work around this on some architectures
| you can simply raise the stack limit which defaults to 8MiB.
|
| If you want to do this in a portable way, learn to use
| `xargs` or `find`.
| amelius wrote:
| > If you want to do this in a portable way, learn to use
| `xargs` or `find`.
|
| xargs and find are a hack here, because it will start the
| target program multiple times. They only work if CMD FILE1
| FILE2 has the same meaning as CMD FILE1; CMD FILE2.
|
| The fix with the stack limit is also a hack. What other
| system requires you to specify a hard limit, these days?
| genericacct wrote:
| -N 1 will invoke the command with only one arg at a time
| chungy wrote:
| Pretty sure that limit doesn't exist anymore.
| deathanatos wrote:
| I think it does. The man page for execve(2) documents it. On
| my system, the documented limit is 2 MiB.
|
| OTOH, I do sort of see the argument for "why is there an
| arbitrary limit here?", beyond just "it's limited by
| available memory".
|
| OTOH, what on Earth is one doing, passing more than 2 MiB of
| arguments to a program? Surely one would be better served
| with piped input, at that point...
| ketanmaheshwari wrote:
| I made these slides on Linux Terminal Tools that might also be
| handy alongside the book:
| https://github.com/ketancmaheshwari/lisa19/blob/master/LPT_L...
| Icathian wrote:
| This was the book that kick-started me along to becoming a
| software engineer. Fantastic book and I can't recommend it
| enough.
___________________________________________________________________
(page generated 2022-12-29 23:01 UTC)