[HN Gopher] Bash functions are better than I thought
___________________________________________________________________
Bash functions are better than I thought
Author : todsacerdoti
Score : 277 points
Date : 2021-10-31 16:29 UTC (6 hours ago)
(HTM) web link (cuddly-octo-palm-tree.com)
(TXT) w3m dump (cuddly-octo-palm-tree.com)
| chaps wrote:
| Oh yeah, bash functions are great and absolutely abusable.
| Sometimes you need some grand hacks to get it to work well, but
| when it works well, it can do some magic. You can even export
| functions over ssh!
|
| I wrote this a few years back which ran on bunches of hosts and
| fed into an infrastructure network mapper based on each hosts'
| open network sockets to other known hosts. It wasn't really
| feasible to install a set of tools on random hosts.. but I still
| had root ssh access across the board. So I needed something tool
| agnostic, short, auditable, and effectively guaranteed to work:
|
| https://github.com/red-bin/lsofer/blob/master/lsofer.sh
| throwawaygh wrote:
| _> You can even export functions over ssh!_
|
| What does "export" mean in this context? Do you just mean `ssh
| host cmd` where cmd is some shell that happens to contain (and
| call) a function?
| xyzzy_plugh wrote:
| Probably implying use of `declare -pf` to turn functions into
| strings which can be evaluated by a remote shell.
| chaps wrote:
| Exactly -- check out script in the link, it shows how it
| would be used. I'm not sure why the first `remote_cmd` is
| called (probably local testing and forgot to delete it), so
| ignore that.
|
| Try this and you'll see how it returns a dramatic amount of
| bash as its output: remote_cmd="`typeset -f`
| ; function test123 { echo hi! ; }" && echo -e $remote_cmd
| throwawaygh wrote:
| Ah, makes sense! Yeah, that's nice. You can do something
| similar with scripting languages by piping source code into
| the interpreter. Super useful back in the day for doing
| crap on machines in an LSF cluster when the NFS was
| down/slow.
| 2Gkashmiri wrote:
| I want to disable my USB controller so after every boot, I am
| forced to open terminal, type Sudo -s , give password and unbind
| the USB.
|
| Is there a way to automate this as a shell script without having
| to type the password?
|
| I assume
|
| Sudo -s | password | unbind string but that doesn't work.
| jacob019 wrote:
| use an rc.local script
| Hackbraten wrote:
| You want to use your service manager for that kind of tasks.
| For example, systemd or sysvinit on Linux, or launchd on macOS.
| kafkaIncarnate wrote:
| Look into NOPASSWD in the sudoers manpage. You can just put the
| code in a script then give %wheel (or whomever) NOPASSWD access
| to run it. This can also be thrown in sudoers.d for ease of
| copying and managing config across machines.
| sp332 wrote:
| You can configure udev to blacklist the device.
| https://projectgus.com/2014/09/blacklisting-a-single-usb-dev...
| qbasic_forever wrote:
| Yep, do this OP... don't try to hack in some automated script
| that now has a race condition with device setup. udev has a
| ton of hooks for enabling, disabling and doing anything when
| device state changes.
| anardil wrote:
| You can take this even further by rewriting bash functions at
| runtime if you like. `declare` gives you direct access to the
| function body
|
| https://www.anardil.net/2018/dynamic-shell-scripting.html (self
| plug)
| throwaway984393 wrote:
| Getting anything out of a subshell that isn't from STDOUT is
| impossible. So you can't define an array in a subshell and then
| use it outside the subshell, and you can't return an array (or
| anything that isn't a string) from a subshell. If you only use
| subshells and want to use any kind of data structure that isn't a
| string passed from STDOUT, you have to do it globally. And
| subshells are slow. So nobody uses subshells.
|
| If you use Bash for programming, you have to stop thinking in
| terms of the holier-than-thou software engineer, whose ego
| believes that a superior, "clean" design makes a superior
| program. You should embrace globals. You should switch between
| using or not using the enforcement of set variables or program
| exit status. You should stop using Bashisms and subtle, obscure
| language features unless you absolutely have to.
|
| Bash is not a "real" programming language, so do not treat it as
| one. Do not look for hidden features, or try to do things in cute
| ways that nobody else uses. There is no superior method or hidden
| knowledge. Just write extremely simple code and understand the
| quirks of the shell.
| gpderetta wrote:
| Well you can use arbitrary file descriptors to get things out
| of a subshell, not just stdout. I had to do it when I had to
| get something out of band, while redirecting stdout through a
| pipeline.
|
| Mind, if possible, it is even uglier than using stdout.
| Groxx wrote:
| yeah, I was wondering if there was some reason that something
| like this wouldn't work: err_f="$(mktemp)"
| third_f="$(mktemp)" out="$(cmd 2>"$err_f"
| 3>"$third_f")" err="$(cat "$err_f")" third="$(cat
| "$third_f")"
|
| Seems like that should work fine, and it's not even that odd
| looking or hard to understand. But I haven't tried, not sure
| if there's some surprise lurking in the depths of tempfiles
| or something.
| laumars wrote:
| > _Bash is not a "real" programming language, so do not treat
| it as one._
|
| Bash is definitely a different paradigm to your average
| imperative or functional language and thus requires a different
| approach but I wouldn't go so far as to say it's not "real".
|
| It certainly fits the criteria of a programming language even
| if it does have more warts than a pantomime witch.
| craftinator wrote:
| It is Turing Complete...
| ratww wrote:
| Yes.
|
| Compared to other languages, Bash can be incredibly worse for
| some things, but much better for others.
|
| People who don't realise that won't be able to get the best
| out of their toolset.
| arendtio wrote:
| throwaway984393 <-- just a troll...
|
| Edit: It seems I should elaborate.
|
| > Getting anything out of a subshell that isn't from STDOUT is
| impossible. So you can't define an array in a subshell and then
| use it outside the subshell, and you can't return an array (or
| anything that isn't a string) from a subshell. If you only use
| subshells and want to use any kind of data structure that isn't
| a string passed from STDOUT, you have to do it globally.
|
| Yes, but that is no problem at all, because that is the way
| shell scripts work and if you do it in a functional way, where
| the function has no state, it is great (we a few exceptions).
|
| > And subshells are slow. So nobody uses subshells.
|
| Shell scripts are slow in general, subshells don't make an
| exception but also don't have a large impact. And people do use
| subshells.
|
| > If you use Bash for programming, you have to stop thinking in
| terms of the holier-than-thou software engineer, whose ego
| believes that a superior, "clean" design makes a superior
| program. You should embrace globals. You should switch between
| using or not using the enforcement of set variables or program
| exit status. You should stop using Bashisms and subtle, obscure
| language features unless you absolutely have to.
|
| If you use Shell scripts, you should understand that this
| language has been designed decades ago and that professionals
| advise to use it just in short scripts to connect binaries.
|
| > Bash is not a "real" programming language, so do not treat it
| as one. Do not look for hidden features, or try to do things in
| cute ways that nobody else uses. There is no superior method or
| hidden knowledge. Just write extremely simple code and
| understand the quirks of the shell.
|
| That part is mostly okay, but "real" doesn't get the point, as
| it is a real language, just one with many problems. However,
| given its superior strength we haven't overcome it yet...
| fstrthnscnd wrote:
| > However, given its superior strength we haven't overcome it
| yet...
|
| "Superior strength" is probably just a matter taste, however
| I would really like to read your explanation on why you think
| so.
| worrycue wrote:
| > Bash is not a "real" programming language, so do not treat it
| as one.
|
| Is there a reason we aren't using a shell with a proper
| programming language for scripting?
|
| I can't say I'm a fan of bashscript. I wonder why we are using
| this weird language.
| nextaccountic wrote:
| ruby in some cases feel just like shell scripting, even more
| than perl.. you can even do piping to chain two commands
| together https://stackoverflow.com/questions/4234119/ruby-
| pipes-how-d... (edit: using the shell library
| https://github.com/ruby/shell so, not out of box in ruby,
| unfortunately, which is a mortal sin for replacing bash in
| scripts =/)
|
| Here are the ways to run a program in ruby
| https://stackoverflow.com/questions/7212573/when-to-use-
| each...
| pajko wrote:
| There's csh.
| jrochkind1 wrote:
| Because cross-OS (not just cross-linux-distro but even that)
| standards-making doesn't exist anymore in a real non-broken
| way, and we're stuck with whatever standards of 20+ years
| ago. Whether official or de facto (technically bash is just
| de facto although `bash --posix` or `sh` is an official real
| standard). There basically isn't any "innovation" happening
| in this space in a way that could really result in anything
| as pervasive as bash. Maybe also cause it's just not
| "interesting" anymore, the 'sexy' things are many levels of
| abstraction higher than they were when bash came to
| dominance. It feels like now we're just stuck with it
| forever, indeed.
| sodapopcan wrote:
| My feeling is that it is about convenience when typing at the
| terminal. Having a unified language for both scripting and
| entering commands is quite convenient. Also, bare strings
| while typing at the terminal is very convenient! It would be
| really annoying if we always had to add quotes for string
| args when we're typing at the terminal: $ cat
| "file.txt"
|
| ...and I don't even know what options would look like...
| maybe: $ ls "*.txt", ["l", "a"]
|
| I dunno... this would mean we'd probably need parens now for
| precedent-type concerns: $ ls("*.txt", ["l",
| "a"]) > "file.txt"
|
| Anyway, that is why I understood that it's this way. I can't
| point to a resource about it, though.
| fstrthnscnd wrote:
| > It would be really annoying if we always had to add
| quotes for string args when we're typing at the terminal
|
| With bash (and I suspect with every other popular shells
| out there), it actually means something different. In your
| second example, if you don't use quotes, the shell will do
| the extensions, but if you do, the process will have to do
| it (and in the case of ls, it cannot).
|
| In some case it's good to have the shell taking care of it,
| but sometimes (eg, when using 'find') it's not.
| pajko wrote:
| You have to add quotes in bash if the filename contains a
| space. Variables should also be quoted for the same reason,
| because their values might contain spaces. "${var}"
| dahfizz wrote:
| Only in a script. On the command line, tab expansion will
| escape any special characters in a filename.
| fragmede wrote:
| don't forget to handle filenames with newlines in them as
| well!
| jandrese wrote:
| You look like you are just about to discover Powershell.
| psyclobe wrote:
| Powershell is how I instrument cross platform CI
| scripting, with the power of the module concept and
| manifess now you can write real powerful type safe apis
| that can run anywhere. It has its quirks though...
| throwaway984393 wrote:
| If you want to choose a programming language, there are
| hundreds to choose from. You could write a small program
| ("script") in Brainfuck. But that would be really annoying,
| and it would take you a long time to get anything done. Other
| languages may also take a long time to write, test, execute,
| and maintain. And they may be better at some things than
| others.
|
| You have to step back and remember the point of all this. Why
| are you programming? To solve a problem, and make a human's
| life easier. What is the problem you are trying to solve? How
| can we make the human's life easier? In this case, it's "I
| want to combine bunch of programs together in a command-line
| interface, to make it easier to use these programs to get my
| work done on the command-line." So, what solution should we
| choose for this scenario?
|
| Lots of different "scripting" languages exist (we used to
| call them "glue languages"). Today Python is the most
| popular, and before it was PHP and Perl. But who cares if
| they're popular? What are they good for? Why would you choose
| one over the other?
|
| Python is a general purpose language which is easy to learn,
| easy to write, and easy to read. Well that sounds nice, but
| it's very "general" and not specific to our use case. PHP is
| a language designed for use in web development. Perl is a
| language designed for system administration-type tasks.
|
| Bash scripting is designed for making it easy to combine
| already-existing programs in a command-line shell, with no
| modification or creation of programs required. The only
| "dependency" is the Bash interpreter (which happens to also
| be the command-line interface we started our problem with!
| how convenient!) and an existing collection of software tools
| that work with each other through a command-line interface.
| Every part of it is explicitly _not_ designed to "make
| programming easier". Instead, it is designed to make
| "combining programs in the command-line" easier. The result
| is things like every single word you type is potentially just
| an outside program being executed, or arguments to that
| program. No other language has this quirky behavior, because
| they're not designed solely to make command-lines easier to
| use.
|
| I know Perl well, and the language is fantastically useful as
| a system scripting language. It combines some of the best
| features of common shell scripting programs with more
| programming language-specific features. It is designed with
| shortcuts and "magic" to make quick work of command-line
| scripting tasks. But ultimately Perl was not built into the
| command-line, so its utility is always a bit stilted. Going
| between the shell and the Perl code and back to the shell
| isn't as flexible as if the Perl code was embedded in the
| command-line. Perl also doesn't have as simple a facility for
| pipes as a command-line shell does. Oh, and a Perl install is
| rather large, isn't available everywhere by default
| (anymore), and has all the usual dependency problems. So even
| though I know Perl very well, if I need to just combine
| existing programs in the command-line, I always use Shell
| scripting instead of Perl. (There are Perl shells which solve
| this problem, but then everyone needs to learn Perl, and Perl
| can be.... idiosyncratic)
|
| You could also use Awk for a large number of general
| programming tasks, but again it's designed for a different
| specific problem: pattern-scanning and processing, not
| generally combining programs together.
|
| So we use shell scripting to solve the problem because it was
| designed specifically for our problem, it's ubiquitous, and
| simple to use. You could use any other programming language,
| but it wouldn't fit our scenario as well. Once your use case
| changes, and you are no longer only trying to combine
| existing programs together, or the existing programs don't do
| what you want them to do, then you need to use a different
| language that better fits that new use case.
| rovr138 wrote:
| Perl used to be used a lot to automate things. I feel Python
| has taken some of that, but also have seen Bash being picked
| up more.
| midasuni wrote:
| I tend to start writing things in bash (piping into things
| like grep, tac, sed, etc
|
| However if I'm not careful they get painfully complex and I
| tend to regret not writing it in perk. As such I now switch
| to using Perl when I get to the function stage, or anything
| other than the most simple bash arithmetic.
| heavyset_go wrote:
| It's because Bash is everywhere and is thus the next least
| common denominator after sh.
| chasil wrote:
| Bash is not everywhere.
|
| There are two very distinct places where Bash is not, that
| I have found.
|
| Bash is not in busybox. It pretends to be, but what is
| really there is the Almquist shell, with a bit of syntactic
| sugar to silence the common complaints.
|
| Bash is also not in Debian's shell (/bin/sh). In that
| place, there is no tolerance of bashisms.
|
| It is important to know the POSIX shell for these reasons.
| chasil wrote:
| Stephen Bourne was a great fan of ALGOL, and had C
| preprocessor directives to "ALGOLify" C.
|
| https://research.swtch.com/shmacro
|
| The Bourne shell thus adopted ALGOL syntax, distinct from
| anything else in UNIX.
|
| David Korn brought a pile of C, and wedged it into a max 64k
| program space (for Xenix), and his upward-compatible Korn
| shell (ksh88) had many, many features.
|
| Then there were the "UNIX Wars"...
|
| https://en.wikipedia.org/wiki/Unix_wars
|
| The fallout of this was the choice of a few Korn features,
| but not all (no arrays, coprocesses, [some] eval, regex, et
| al.)
|
| The wars defined the POSIX shell. Yes, it can be infuriating.
| gfodor wrote:
| This reminds me of the arguments made against writing "real"
| code in JavaScript in the early days of the web, until
| Crockford came along and wrote "The Good Parts." There is no
| reason to think that a few idioms and curating features could
| go a long way to leading to a much better, less hacky paradigm
| for bash shell scripting.
| hsn915 wrote:
| Bash is much older than JavaScript. If it was going to turn
| into a real programming language, it would have by now. It
| hasn't.
|
| Also, JavaScript really is not a very good programming
| language. We are just stuck with it because it's the only
| language the browser understands. (Well, until recently:
| things are going to change with the introduction of wasm).
|
| But for the shell, we're not stuck with any one language.
| Whatever you want to do can be programmed in your favorite
| language. You can easily write a python script instead of a
| bash function.
| raziel2p wrote:
| > things are going to change with the introduction of wasm
|
| people have been saying this for years already and nothing
| seems to have come of it :(
| tambourine_man wrote:
| Yeah. Turns out, having an interpreted runtime in the
| browser is quite useful and sending bytecode across the
| wire not as practical.
|
| We've been forgetting and relearning that for the past
| two decades.
| ReactiveJelly wrote:
| Just don't use Bash.
| gavinray wrote:
| A lot of the time you don't even have the luxury of assuming
| "bash" is present, you've got only "sh".
|
| Since all valid "sh" syntax is valid "bash", you've got to
| restrict yourself to only sh-valid code in the event the host
| doesn't have bash.
|
| I have gotten used to doing this -- these scripts usually
| wind up being run in Docker containers that don't have "bash"
| installed for the sake of minimalism.
|
| IE, you can't use [[ $predicate ]], but instead [ $predicate
| ], etc. Lot of subtle differences.
| Spivak wrote:
| Not all sh syntax is valid bash -- in bash [[ is reserved
| but not in sh. And if you want syntax that is valid but
| doesn't do the same thing in sh vs bash there's plenty of
| examples.
| mistrial9 wrote:
| I would agree with this and use python instead, but for uuhh
| "python problems with strings". IF you are 'blue sky' open,
| making new things, yes, sure. no BASH. But for things I built
| 8 years ago that are non-trivial, and I don't prioritize an
| entire rewrite, BASH works very well. And guess what, every
| two years that 8-years-ago just comes along with it. BASH is
| here to stay.
| chasil wrote:
| The POSIX shell can sometimes achieve with a few lines what
| would take dissertations to do in C.
|
| The supersets of POSIX are often even more effective.
|
| Ugly as the tool may be, in the box it stays.
| necheffa wrote:
| LOLWUT?
|
| At least _try_ to understand what you are talking about before
| you criticize. Your rant demonstrates your complete lack of
| understanding on the topic.
|
| The, apparently, "hidden" knowledge is that Unix shells are
| designed to process text. And so you serialize complex data
| structures over FIFOs from child to parent. That can be as
| simple as comma/colon/whatever delimited bytestreams chopped up
| with awk or as complex as JSON and parsed with jq.
| zsz wrote:
| Sir! Yes, sir!
| fiddlerwoaroof wrote:
| If you take the time and effort to understand how bash wants
| you to think, you can learn how to right elegant scripts that
| are as maintainable as anything else.
| jefftk wrote:
| I've written a lot in bash over the years, and I feel like I
| understand it pretty well. But I would never say that even
| elegant bash scripts are as maintainable as "anything else".
| It is a clunky programming environment, born of compromises,
| with many traps that are easy to miss in code review.
| tempodox wrote:
| There's `shellcheck`, as part of Syntastic, for vim. I find
| it quite useful. You could call it a Bash linter.
| pletnes wrote:
| Shellcheck is great and is a standalone cli program, as
| well as a website if you don't want to install it
| locally. https://www.shellcheck.net/
| lazyweb wrote:
| I'd rather always check my code locally if the
| alternative is pasting it into a random web from.
|
| On the other hand, I wonder if the web page keeps track
| and could offer "best of bash mistakes".
| naniwaduni wrote:
| A random web form can exfiltrate the data you paste into
| it (and whatever your browser lets it gather). A local
| program can exfiltrate ... approximately everything of
| value on the machine?
| [deleted]
| rowanG077 wrote:
| Can you point me towards an elegant bash script? I honestly
| have never seen one that is more then a few lines.
| gavinray wrote:
| I like to think this thing I wrote isn't so bad and is
| fairly readable, even if you don't know anything about
| shell programming:
|
| https://github.com/GavinRay97/hasura-ci-cd-
| action/blob/a9731...
|
| This is out of necessity. I'm not the sharpest tool in the
| shed, so I have to go out of my way to write things such
| that when I come back to them in months or years, I still
| understand what they do.
|
| For this same reason, is also why I never use shorthand
| flags for scripts.
|
| I have no clue what IE: _" -s -y -o"_ might do (or even
| worse, "-syo", dear god my eyes! Also -- is that one
| special command, or a series of individual commands?!).
|
| But _" --silent --assume-yes --output-file"_ is pretty easy
| to grok immediately.
| confidantlake wrote:
| I agree with you that it seems readable, nice job on it.
| But there is comment there that says
|
| _Oh man this is so ugly_
|
| Not something I would expect in an "elegant" script.
| Don't think that it is your fault, just it is very hard
| to write anything elegant in bash.
| gavinray wrote:
| Ah yes, that section.
|
| Hahaha -- fair point. Readable: maybe. Elegant? No. Bash
| is far from elegant =P
| mdpye wrote:
| I would add a variable ENDPOINT, initialised to "" and
| set to " --endpoint $THEENDPOINTVALUE" if the endpoint
| value was passed. Then include that in every invocation?
|
| Sorry if I missed something in the logic, reading on my
| phone, but from the comment, this feels like something I
| do frequently...
| jamesrr39 wrote:
| While I understand the sentiment, I'm not sure how bash could
| ever be as maintainable as a something written in e.g. Python
| (or even better, a strongly-typed language).
|
| The thing with bash is, it's great for tying things together
| and quick bits and pieces, but it's not set up for writing
| maintainable code. Arrays, functions, even if statements
| comparisons can all be done in bash (as first-class
| features), but are just... easier in other languages. And
| then think about the refactoring, linting, testing tools
| available in bash vs other languages. And then on top of
| that, there's the issue of handling non-zero return codes
| from programs you call; do you `set -e`, and exit on any non-
| zero return code even if you wanted to continue, or not `set
| -e`, ignoring any errors as your script just continues.
|
| Personally, when I feel I want to use a function (or array,
| or other similar, non-trivial thing), in bash, it's time to
| reach for another language.
|
| Having said that, there are some nice programs written in
| bash. https://www.passwordstore.org/ being one that comes to
| mind.
| einpoklum wrote:
| > Personally, when I feel I want to use a ... non-trivial
| thing ... in bash, it's time to reach for another language.
|
| Then you must not be writing any bash at all. Functions are
| useful for almost anything beyond a one-liner script.
|
| Typical functions used in innumerable non-trivial scripts:
|
| * print_usage()
|
| * die() (see: https://stackoverflow.com/q/7868818/1593077)
|
| :-)
| jamesrr39 wrote:
| You are kind of right, I don't write much bash, but I do
| write some simple scripts that I can call quickly and
| easily (e.g. start this program with these args, write
| the log file here with the filename as the current date,
| etc). Although regarding "Then you must not be writing
| any bash at all"; I'm not sure how you could have deduced
| this!
|
| With regards to `print_usage()` and `die()`, yes, I would
| reach for Python 3 then. The `argparse` module and
| `throw` are first-class members of the stdlib/language
| and are better and more standard between programs than if
| I threw together these myself (and with `throw` you get a
| stack trace, which is nice).
| mkl wrote:
| > e.g. Python (or even better, a strongly-typed language).
|
| Python is strongly typed. Maybe you meant "statically"? (As
| opposed to dynamically.)
| jamesrr39 wrote:
| Yes, you are right, and statically is what I meant,
| thanks for the correction.
| ilyash wrote:
| Sorry. "elegant" here triggered.
|
| https://ilya-sher.org/2021/03/19/running-elegant-bash-on-
| sim...
| MereInterest wrote:
| To have variable typos result in errors with `set -u`, then
| expand an array that is defined but may be empty, I need
| write it as `${ARRAY[@]+"${ARRAY[@]}"`. (Further explanation:
| https://stackoverflow.com/questions/7577052/bash-empty-
| array...) But maybe bash arrays are too new a feature, and
| should be avoided, so let's look at something simpler, like
| command-line arguments.
|
| Parsing command-line arguments is easy, but parsing them
| correctly is ridiculously hard. `getopts` doesn't handle long
| flags, so it's out. `getopt` doesn't handle arguments with
| spaces in them, unless you have a version with non-standard
| extensions, so it's out. You're left with manual parsing, and
| it's a royal pain to make sure you handle every expected case
| (short/long flags, clusters of short flags, trailing
| arguments to be passed through to a subprocess, short/long
| options whose value is in the next argument, long options
| whose value is after an equals sign, and several others I'm
| probably forgetting about). And this is just to get the
| arguments into your script, before you actually do anything
| with it.
|
| I agree that bash has effective idioms, and learning those
| idioms can make scripts easier to write. I strongly disagree
| that bash is "as maintainable as anything else", and scripts
| beyond a few hundred lines should be rewritten before they
| can continue growing.
| drran wrote:
| You can use bash-modules arguments library to generate
| argument parser for you.
| hutrdvnj wrote:
| But in reality you never see such scripts.
| Osiris wrote:
| One way to answer this would be to find the largest (in terms
| of LoC) Bash script you can find and try to add a feature or
| fix a bug.
| michaelcampbell wrote:
| > So nobody uses subshells.
|
| I'll count myself among nobodies, I guess. I use them a fair
| bit when part of said subshell cd's somewhere, and I don't feel
| like using pushd/popd, either of which might fail to operate
| based on my mistakes, but subshells seem to never fail to exit,
| eventually.
| 1vuio0pswjnm7 wrote:
| "So nobody uses subshells."
|
| djb has always used them, even in the shortest of scripts.
|
| Otherwise I agree with everything in this comment.
|
| There is a reasonable argument to avoid using bash for non-
| interactive scripts. The benefits of any additional bash
| features, so-called "bashisms", arguably do not outweigh the
| costs of making these scripts non-portable. One of the many
| advantages of shell scripts is that they are portable and have
| tremendous longevity; shell scripts can last a long, long time.
| There are no version changes and aggressive feature creep to
| worry about as is routinely the case with programming
| languages. The scripts just keep working, every day, and we can
| forget we are even using them, e.g., they are being used in the
| various ways by UNIX-like OS distributions.
|
| One of the today's programmer memes was "Get Stuff Done". Maybe
| the shell was not meant for today's programmers. But for
| "sysadmins", or "DevOps", or whatever term anyone comes up with
| in the future, people who can "administer/operate" computers
| running a UNIX-like OS for themselves or for someone else like
| a client or an employer, the Bourne shell works for its
| intended purpose, better than anything anyone has come up with
| in the last 50+ years. There's a lot of stuff that "gets done"
| with the shell, whether it is on someone's own computer, their
| client's or their employer's.
|
| Attempts at "shell replacements", cf. alternative shells,
| usually look like interpreters for programming languages, not
| shells. Perhaps this is not a coincidence.
|
| Bourne shell is relatively small and fast. Computers with small
| form factors often use UNIX-like OS and when they do they
| usually include shells. Many programmers seem to dislike the
| notion that such a layer exists. They often try to blame the
| shell instead of their own lack of interest in learning to use
| it.
|
| The shell is boring, and "boring" is sometimes the wisest
| choice. Most software has expanded to consume available
| resources (often the developer's choice not the user's). This
| makes computer performance gains difficult for the end user to
| discern, e.g., decade after decade, routine tasks seem to take
| the same amount of time. However the shell and many "standard"
| UNIX utilities have not changed as much in the same period.
| Subshells may have been "slow" many years ago. IMHO, they do
| not feel slow today. Routine tasks performed with the shell
| seem to run faster, as one would expect after hardware upgrade.
|
| I posit: Life is too short to learn every programming language
| du jour but it is long enough to learn the Bourne shell,
| reasonably well.
| oweiler wrote:
| Bash scripts and shell scripts are not portable.
|
| They have to shell out to other commands in order to do
| something meaningful.
|
| Depending on your OS, these commands may not exist, or have a
| different set of commandline flags.
| jrochkind1 wrote:
| Wait, a subshell is a child process, right?
|
| I would imagine there would be performance implications of
| defining every bash function as a subshell, is why it's not
| universally recommended to define functions this way?
|
| No?
| [deleted]
| arendtio wrote:
| If you care about performance, you better don't write shell
| scripts. The typical task of a shell script is to start
| programs and connect them in a meaningful way. This is in
| itself a pretty expensive task performance wise.
|
| So arguing about the cost of sub-shells is somewhat besides the
| point.
| zegl wrote:
| I did a quick test, and function calls using the subshell type
| of function seems to be about twice as slow as normal function
| calls. #!/bin/bash fib1() {
| if (( $1 < 2 )) ; then echo $1
| return 0 fi echo $(( $(fib1
| $(($1-1))) + $(fib1 $(($1-2))) )) return 0
| } fib2() ( if (( $1 < 2 )) ; then
| echo $1 return 0 fi
| echo $(( $(fib2 $(($1-1))) + $(fib2 $(($1-2))) ))
| return 0 ) time fib1 15 time
| fib2 15
|
| Result: 610 real 0m1.585s
| user 0m0.590s sys 0m0.972s 610
| real 0m3.168s user 0m1.068s sys 0m2.025s
| jrochkind1 wrote:
| It probably doesn't matter too much if you have only a
| handful of function invocations in your exec. But if you have
| a a couple orders of magnitude more... RAM is going to be an
| issue too maybe.
|
| Creating a new process for every single function invocation
| _seems_ crazy to me, and but might actually be just fine for
| many "ordinary" use cases? (Although might not have been on
| computers of 20+ years ago, which might also be why it's not
| something advised, so much of bash tradition is decades old?)
| kevincox wrote:
| Of course the subshell is copy-on-write so the ram
| requirements shouldn't be huge. But assuming some stack you
| are using at least some stack in each process you are
| looking at 4k per call which adds up fairly quickly.
| dataflow wrote:
| Yeah, try subshells on MSYS2 and see how performance goes.
| chubot wrote:
| Yes, and that's my reaction too. While I can see the rationale
| for always starting another process, in practice I haven't
| found the leakage to be a big problem.
|
| Actually Oil functions don't use dynamic scope, but this is
| done in-process, not with another process:
|
| https://www.oilshell.org/release/latest/doc/variables.html#p...
|
| ----
|
| Also nested functions don't really add much useful to shell.
| It's purely a matter of textual code organization and doesn't
| affect the semantics. I define all functions at the top level.
| fragmede wrote:
| subshells don't propagate information out which makes them hard
| to work with in many cases. Ie _((foo=42)); echo $foo_ is not
| going to have 42.
| laumars wrote:
| If your hot path needs that level of micro-optimisation then
| you're far better off rewriting it an language that compiles
| instead of interprets and forks. Even Ruby and Perl would run
| circles around Bash.
| smarx007 wrote:
| Additionally, 'local -n' allows to define a local variable as a
| reference, useful with arrays, for example.
| gavinray wrote:
| I write a LOT of bash/shell scripts. And I don't like it, it's
| just part of what I have to do.
|
| Learning a handful of bash idioms and best-practices has made a
| massive impact for me, and life much easier. The shell is
| something you cannot avoid if you're a programmer or other sort
| of code-wrangler.
|
| You can interact with it + be (mostly) clueless and still get
| things done, but it's a huge return-on-investment to set up
| "shellcheck" and lookup "bash'isms", etc.
|
| ----
|
| _(Off-topic: I am convinced Ruby cannot be beaten for shell-
| scripting purposes. If I had a wish, it would be that every
| machine had a tiny Ruby interpreter on it so I could just use
| Ruby. I 'm not even "a Ruby guy", it's just unreasonably
| good/easy for this sort of thing. And I keep my mind open for
| better alternatives constantly.)_
|
| Example of near-identical script in bash vs Ruby:
|
| https://github.com/GavinRay97/hasura-ci-cd-action/blob/maste...
|
| https://github.com/GavinRay97/hasura-ci-cd-action/blob/maste...
|
| I'm not sure how much closer to describing your exact intent in
| English a language can get than:
| successfully_made_executable = system 'chmod +x
| /usr/local/bin/hasura' abort 'Failed making CLI executable'
| unless successfully_made_executable
|
| Though I have NOT written Perl (either old Perl, or Raku/Perl 6)
| but I do believe it may be roughly this semantic too.
|
| EDIT: Looks like Perl/Raku is essentially the same as Ruby in
| this regard. So besides it being a whacky language, take that for
| what you will: $successfully_made_executable =
| shell "chmod +x /usr/local/bin/hasura" die 'Failed making
| CLI executable' unless $successfully_made_executable.exitcode is
| 1
| jrochkind1 wrote:
| > I'm not sure how much closer to describing your exact intent
| in English a language can get than:
|
| > successfully_made_executable = system 'chmod +x
| /usr/local/bin/hasura'
|
| > abort 'Failed making CLI executable' unless
| successfully_made_executable
|
| I guess, but if I was writing scripts like this, I'd really
| want to write a helper method something like:
| def system_or(cmd, fail_msg) system cmd || abort
| fail_msg end # and then
| system_or 'chmod +x /usr/local/bin/hasura', 'Failed making CLI
| executable'
|
| Instead of having to write every single `system` call as
| boilerplate two liners with an immediate-throwaway local var.
| nickjj wrote:
| You can also turn a Bash function into a script command with
| almost no effort, such as making a file called "run" and putting
| this in it: #!/usr/bin/env bash
| set -eo pipefail function hey { echo
| "Hey!" } TIMEFORMAT=$'\nTask completed in
| %3lR' time "${@}"
|
| Now after running a `chmod +x hey` can run it with: ./run hey
|
| Feel free to replace "time" with "eval" too if you don't want
| your command timed.
|
| This is a really useful pattern because it means you can create a
| "run" script with a bunch of sub-commands (private or public),
| auto-render help menus and create project specific scripts
| without any boilerplate. Bash also supports having function names
| with ":" in the name so you can namespace your commands like
| "./run lint:docker" or "./run lint:frontend".
|
| I have a practical example of this sort of thing here:
| https://github.com/nickjj/docker-flask-example/blob/main/run
|
| I've written about this pattern in more detail here
| https://nickjanetakis.com/blog/replacing-make-with-a-shell-s....
| It's basically a less limited Makefile for when you want to make
| project specific aliases. This is something I use all the time
| now.
| francislavoie wrote:
| I do the same thing, but slightly differently.
| https://github.com/francislavoie/laravel-websockets-example/...
| louwrentius wrote:
| At one time, I did learn myself to write shell scripts. I even
| wrote this 3Kl line monstrosity [0]
|
| However, I would strongly advice to master a proper programming
| language. I respect the article and the efforts of the author,
| but I feel that it is the past.
|
| I mastered Python a bit and the ability to just use things like
| dictionaries, proper parsing libraries and such, instead of
| kilometers of fragile pipes, it is so much better.
|
| I understand something like Python may feel total overkill, but
| that 10 line shell script suddenly needs quite a bit of error
| handling and some other features and before you know it, you wish
| you started out with python or something similar.
|
| [0]: https://github.com/louwrentius/ppss
| gjvc wrote:
| does this do the same as https://www.gnu.org/software/parallel/
| ?
| barosl wrote:
| > Subshells are, as the name suggests, running in a subshell.
| They don't strictly have to be OS subprocesses
|
| Are there really cases where subshells are invoked within the
| same process? In my experience, it has never been the case.
| That's why I've been trying to minimize the use of subshells
| because spawning a new process is a bit slow.
| Xophmeister wrote:
| Here's an interesting bit of...abuse :) Since Bash is effectively
| stringly-typed, it can be used as a functional programming
| language, with pipes similar to function composition.
|
| e.g.: wtfp.sh #!/usr/bin/env bash
| map() { local fn="$1" local input
| while read -r input; do "${fn}" "${input}"
| done } reduce() { local fn="$1"
| local init="$2" local input local
| output="${init}" while read -r input; do
| output="$("${fn}" "${output}" "${input}")" done
| echo "${output}" } filter() {
| local fn="$1" local input while read -r
| input; do "${fn}" "${input}" && echo "${input}"
| done } add() { echo $(( $1 + $2 ))
| } increment() { echo $(( $1 + 1 ))
| } square() { echo $(( $1 * $1 )) }
| even() { return $(( $1 % 2 )) }
| sum() { reduce add 0 } map
| increment | map square | filter even | sum
|
| ...then: $ printf "%s\n" 1 2 3 4 5 | ./wtfp.sh
| 56
| dllthomas wrote:
| I enjoyed how my bash 2048 came out:
| https://github.com/dlthomas/bash2048/blob/master/2048.sh
| ridiculous_fish wrote:
| Fun shell function fact: used to be if you `break` or `continue`
| in a function without a loop, bash would _find_ the loop:
| breaker_breaker() { break; } foo() { breaker_breaker; }
| while true; do echo Loop foo done
|
| bash dynamically crawled up the call stack until it hit a break-
| able loop. If you squint it almost looks like exception handling!
| Anyways this no longer works in bash, though it still does in
| zsh.
| Sniffnoy wrote:
| Heh, kind of like dynamic scoping, except as applied to control
| flow...
| dllthomas wrote:
| > If you squint it almost looks like exception handling!
|
| It also reminds me of TCL's "uplevel":
| https://www.tcl.tk/man/tcl8.4/TclCmd/uplevel.html
| barosl wrote:
| That's very impressive, it almost feels like a legit useful
| hack. Why was it not fixed in zsh? Did they decide it was
| working as intended?
| naniwaduni wrote:
| Well, bash's new behaviour (noisily complain and do nothing,
| reporting success) doesn't do anything _useful_ , so
| implementing it gratuitously breaks old code to no gain.
|
| Dynamic stack-crawling is more or less the most popular
| historical behaviour (though afaict the Bourne/Korn lineage
| just silently fails breaks outside the enclosing function);
| it's just generally consistent with everything else in the
| shell that's globally/dynamically scoped. Even in the shells
| where you can't break out of your enclosing function, `eval
| break` still breaks through loops in your current context,
| and that kind of looks like a function you called that called
| break if you squint.
|
| (POSIX expressly leaves this case undefined with a carveout
| for break/continue inside the body of a function lexically
| contained within a loop.)
| dorianmariefr wrote:
| Just write scripts in Ruby
| jrochkind1 wrote:
| Installing ruby/predicting the version of ruby
| installed/writing ruby that can run on any version installed...
| is unfortunately non-trivial.
|
| I think pretty much the only reason people write bash is
| because you have an incredibly high chance of bash being
| installed and a version of bash being installed that will run
| whatever bash you write just fine.
|
| Perl is honestly almost as reliable to be there predictably and
| compatibly... but I guess people would rather write bash than
| Perl?
| ziml77 wrote:
| Bash might be consistent, but what about the programs you're
| calling out to? Even basic utilities have different options
| between BSD and GNU Coreutils. Something like git might not
| have options you're expecting due to differences between
| versions. Or if you need to download a file using HTTP, you
| will run into a problem when you run on a machine that has
| wget when your script was expecting cURL.
|
| And yes, you have these sorts of problems with other
| languages, but my point here is that Bash doesn't free you
| from them.
| CSSer wrote:
| Is there any risk of Bash ever going away? It seems like it's
| the de facto shell. I remember considering whether or not I
| should learn Perl at one point. It didn't even feel like a
| choice with Bash. Trendy shells seem like they have no choice
| but to support it too.
|
| I feel like what usually makes me reach for something beyond
| Bash is really a matter of wanting or needing some dependency
| that wasn't written in it for whatever reason. Usually this
| happens right at the point where the script/utility starts to
| turn into a library/program, so it's trivial to just
| transpose the control flow into whatever language is required
| at that point and go from there. This of course raises the
| type of concern you mentioned about Ruby, but at that point
| it's hopefully worth the trouble to address.
| bombcar wrote:
| The only "danger" to bash is if zsh completely overtakes it
| - and even then scripts will probably be written in bash-
| compatible zsh.
| dorianmariefr wrote:
| bash is no longer the default shell on macOS
| aaaaaaaaaaab wrote:
| Just don't forget to install the correct Ruby version, of
| course keeping the system's Ruby intact, so better use RVM for
| that, and of course you shouldn't install random gems globally,
| so better do the whole thing via Bundler, and of course don't
| forget to check in the Gemfile.lock too.
|
| Or you can just use bash and keep your sanity.
| [deleted]
| smarx007 wrote:
| Actually, you can expect a recent version of Python 3 from
| most distros these days. Stdlib is quite powerful and no hurt
| would come from a small number of globally installed libs
| with stable APIs, like click and requests.
| superkuh wrote:
| The key word there is "these days". There have been a lot
| of computers built and software set up in the last ~20
| years that are still getting used today and will be for a
| long time. Not every environment is using $latest.
|
| Despite this you can expect your bash scripts written today
| to work on any of these machines and their ancient OS
| installs. This is primarily because people writing in Bash
| care enough to not use new features, not the lack of new
| features in Bash.
| throwawaygh wrote:
| _> Just don't forget to install the correct Ruby version_
|
| This justification for bash has always been perplexing to me.
| If I'm operating in an environment where my ONLY reliable
| infra invariant is "bash will probably work", cleaning up the
| org's infrastructure clusterfuck is probably my #1 priority.
|
| (Or I guess in a few cases the fleet are not boxen but little
| embedded/iot devices, in which case you probably don't want
| to be running _any_ of these sorts of scripts for a whole
| host of reasons...)
|
| _> of course you shouldn't install random gems globally...
| Or you can just use bash and keep your sanity._
|
| What are some scenarios where you'd need a gem in Ruby but
| Bash just works?
|
| The only situations I can think of are cases where you're
| using Bash to call out to some executable that does fancy
| things. Which (1) you can do from any scripting language
| anyways, and (2) means you're just shifting dependency
| management from the package manager to the image/Dockerfile.
|
| But actually, the combination of justifications here is
| particularly perplexing. If the org has a stable way of
| handling system images, then you'll know which version of
| $scripting_language should exist and where it's installed.
| The only way you end up with language version woes is if you
| don't have standardized infra. BUT... if you don't have
| standardized infra, _and_ Bash can do things that Ruby can 't
| without special Gems, then it stands to reason that you're in
| a situation where your Bash scripts depend on the magic state
| of individual unicorn boxes?! Which is particularly fragile
| and frightening and far worse than installing some local gems
| or whatever!
|
| IDK. The purported benefits of Bash always sound like they
| flow out of environments where there are basically no fleet-
| wide infrastructure invariants.
|
| "Just use $scripting_language" might be the best advice in
| this thread just as a sort of canary in the coalmine. I.e.,
| if your org can't "Just use $scripting_language" because
| "which version?" then the team will probably benefit
| tremendously in an infinite variety of ways from an afternoon
| of infrastructure cleanup. Regardless of whether they use
| bash or a scripting language going forward :)
| jandrese wrote:
| > This justification for bash has always been perplexing to
| me. If I'm operating in an environment where my ONLY
| reliable infra invariant is "bash will probably work",
| cleaning up the org's infrastructure clusterfuck is
| probably my #1 priority.
|
| That is not your job. Your job is to get that machine
| working using whatever is already installed. Adding a new
| package means going to the production committee with your
| proposal and justification and analysis of the increased
| threat surface.
|
| The defaults are everything.
| throwawaygh wrote:
| _> > This justification for bash has always been
| perplexing to me. If I'm operating in an environment
| where my ONLY reliable infra invariant is "bash will
| probably work", cleaning up the org's infrastructure
| clusterfuck is probably my #1 priority._
|
| _> That is not your job._
|
| This sort of thing is definitely your job have the word
| "Principal" in your job title, and probably also if the
| word "Senior is in there as well ;-)
|
| And in any case everyone is responsible for excellence in
| the milieu in which their team operates. If Senior or
| even fresh grad Jr. comes to me with a solid good idea
| I'll champion for it as if it were my own baby. And then
| recommend/fight for rapid promotion in the case of Jr or
| put in a good word for promotions for the Sr.
|
| If you recommend your org have standardized images with
| well-documented info about language versions etc. and the
| answer you get from your management/tech leadership is
| "not your job", I recommend finding a new job.
|
| _> Adding a new package means going to the production
| committee with your proposal and justification and
| analysis of the increased threat surface._
|
| The context of my quote was "knowing which version of
| ruby/perl/python is installed". There's _almost
| certainly_ a version of one of those on your standard
| linux machine, and everyone pushing to prod should damn
| well be able to look up exactly which one.
|
| _> Adding a new package_
|
| The general debate here goes way beyond adding a new
| package. Good infra needs WAY better invariants than
| "definitely bash is installed in the usual place". If a
| concern is "IDK which version of Ruby is installed on the
| machines I'm targeting" then either you're fighting fires
| and need to keep every intervention _really damn simple_
| or else your org as _Real Issues_. In either case, bash
| is the enemy.
|
| _> The defaults are everything. _
|
| Those defaults aren't handed down from Gods. Your org
| chooses them.
| t-3 wrote:
| The advantages of bash are almost all related to it NOT
| being a "real" programming language. The terseness, ease of
| writing self-modifying code and anonymous functions, lack
| of typing, flexible syntax, easy interoperability with any
| other language and program through any available interface,
| are not really desirable for writing stable and
| maintainable code. They are hugely desirable for quickly
| hacking something together, testing and learning, and the
| numerous simple scripting tasks involved in system
| administration.
| throwawaygh wrote:
| _> They are hugely desirable for quickly hacking
| something together_
|
| Undeniably, yes. I was there in the 90s ;-)
|
| But hacking things together in a way that's _robust_ is
| difficult, and bash isn 't a good match for that
| difficulty.
|
| These days I mostly operate in the realm of "how can I
| enable others to hack things together without blowing
| tens of millions of our dollars and their very early
| career on a stupid mistake".
| kaba0 wrote:
| But it is so full of landmines that even those quick
| dirty hacks _will_ fail.
|
| Also, don't even start me up on self-modifying code, we
| have one at a work project and it sometimes just fails
| and results in inserting the same echo statement at each
| run, resulting in every bootstrap displaying 2^n
| messages, depending on when have I last cleaned it up...
| totony wrote:
| This is one of the reasons I think nix is a game changer. Not
| need to care about that, just put everything in nix-shell.
| hackbinary wrote:
| Or Python, or tcl, or (eeek) Perl, or JavaScript.
|
| Or anything that can handle integers and floating point
| calculations natively.
| aaaaaaaaaaab wrote:
| Bash handles integers natively.
| nanomonkey wrote:
| Babashka for Clojure is my new favorite scripting language.
| agumonkey wrote:
| Seriously impressive
| VWWHFSfQ wrote:
| > Essentially, the only "advantage" of not running your functions
| in a subshell is that you can write to global variables.
|
| subshells are ridiculously slow because it's a forked child
| process. imagine forking for every function call in your
| program....
|
| > I simply do not understand why people keep recommending the {}
| syntax at all.
|
| because it's almost always what you actually want.
| SubiculumCode wrote:
| I'd argue that most of the work in a bash program is done by
| functions like find, grep, etc, and that the time to fork is
| not all that relevant. We don't program the same kinds of
| things in bash that we might in C++
| fiddlerwoaroof wrote:
| Yeah "fork is slow" is the sort of microbenchmark that is
| mostly irrelevant for shell scripts: every command you run in
| a script is basically a fork.
| plorkyeran wrote:
| Assuming that fork is fast everywhere is how you end up
| with things like ffmpeg's configure script that runs in
| seconds on linux and _minutes_ on Windows.
| dataflow wrote:
| > every command you run in a script is basically a fork
|
| Not for built-in commands.
|
| > Yeah "fork is slow" is the sort of microbenchmark that is
| mostly irrelevant for shell scripts
|
| Maybe if you're on a Linux kernel, but not everywhere else.
| Spivak wrote:
| Where are you running bash where forks are expensive?
| Like sure Windows exists but bash on WSL is running a
| Linux kernel.
| dataflow wrote:
| Not quite. WSL2 uses a Linux kernel. WSL1 uses a Windows
| kernel and fork is much slower there. Also there's
| userspace variants like MSYS2, Cygwin, etc.
| chasil wrote:
| When I am using busybox sh/bash, I am very, very careful
| not to fork unless I must.
|
| For mass processing, I will use xargs to minimize the
| number of processes created.
| VWWHFSfQ wrote:
| if you want to fork your function call then you do it
| explicitly with $(my_function). I'm aware that people are
| always discovering things for the first time but there is
| literally decades of thought that has gone into why bash
| behaves the way it does. and there's a pretty good reason
| why the bash authors decided not to make function calls
| fork by default...
| [deleted]
| chaps wrote:
| Sure it's slow on startup, but if you design your functions to
| pipe to each other as if you were writing purely-functional
| code, startup time is mostly irrelevant. Function startup
| happens once and from there they just feed down to the
| parallel-by-default pipe stream.
| memco wrote:
| Interesting idea. I have thought about doing the trap cleanup but
| found it cumbersome to reason about when there are many functions
| so this is helpful. I would like to have seen a complete example
| at the end rather than just explaining why it's cool and then
| leaving it to the reader to imagine what it looks like.
|
| Besides cleanup, one thing I think I would love to see is a good
| mechanism for logging. I have started to build a file for
| functions and then other files, which source that as a library
| and calls the functions as needed. I would love to be able to
| tell the library functions to log something if the parent file
| wants it, print to stderr or stdout by default or be silent if
| the caller wants that instead.
| RNCTX wrote:
| I have used bash to write an OCR processor that called a python
| wrapper around tesseract, and then turned the pdf output into
| json to go into a solr search database by parsing the output with
| sed.
|
| No, it wasn't smart to do so.
|
| Yes, it still works.
|
| No, I don't know how it works.
| boardwaalk wrote:
| People do underestimate the shell. Particularly I see people
| shoehorning collections of commands into a Makefile, when a shell
| script would work just fine.
|
| On the other hand, with a lot of glue work I do, I eventually
| want to something more complex (use lists and maps, complex
| string building and regexes, date handling) and while you /can/
| do that in bash, I might as well start in Python and have
| everything in one language and take advantage of things like code
| sharing via modules. (And yes you can share code in shell, but
| again it's not as nice.)
| fragmede wrote:
| _Might as well_ , yes, but I've found writing shell scripts in
| Python to be cumbersome because whatever flavor of
| _os.system()_ I end up using just doesn 't work well
| syntactically. I can run a command and pipe to a bunch more
| commands way easier in a shell because I'm already using a
| shell when interacting with the computer. Perl had this figured
| out, but proved unable to continue evolving (aka adding types,
| like Python/Ruby/JavaScript have managed to.)
|
| If there's a modern library/workflow that makes this not the
| case, I'm all ears!
| ilyash wrote:
| > writing shell scripts in Python to be cumbersome because
| whatever flavor of os.system() I end up using just doesn't
| work well syntactically.
|
| This is exactly how I felt. Bash can't handle structured data
| well. Python (being general purpose programming language)
| can't handle calling external programs well because it's not
| the "focus" of the language. My shameless plug solution is a
| "real" programming language that can do both well (along with
| proper handling of exit codes and more goodies for "devops"y
| scripting).
|
| https://github.com/ngs-lang/ngs
|
| More about this feeling when you don't have a well fitting
| language for "devops"y scripting - https://ilya-
| sher.org/2017/10/10/why-i-have-no-favorite-prog...
| laumars wrote:
| There's a few of these types of shells floating about these
| days.
|
| I personally use murex but Elvish seems popular too.
| ilyash wrote:
| NGS is programming-language-first while other modern
| shells are typically shell-first. Multiple dispatch would
| probably be the most prominent manifestation of this
| approach in NGS.
| nrclark wrote:
| I use Make extensively for glue scripts or build scripts that
| call other tools. Make gives you four big advantages over a
| pure shell-script:
|
| 1. Tab completion. All major bash-completion packages know how
| to show Makefile targets in response to a TAB.
|
| 2. Parallel, serial, or single-target execution.
|
| 3. Automatic dependency resolution between tasks. Tasks that
| build files can also use timestamps to see what needs
| rebuilding.
|
| 4. Discoverability. Anybody who sees a Makefile will usually
| understand that something is supposed to be run from the
| Makefile's directory. Chances are good that they'll check the
| tab-completions too. There are conventions for standard targets
| like 'clean' and 'all'.
|
| If you have a project with a build-process that has a bunch of
| small tasks that you might sometimes want to run piece-by-
| piece, Make is the perfect tool IMO.
| totony wrote:
| Since make does not sanitize input or handle error, I use it
| only for parallelism/dependency management and offload all
| build to shell scripts. I've found this to be way more
| maintainable.
| jandrese wrote:
| Discoverability goes out the window the instant someone uses
| something like automake unfortunately. Then the makefile
| becomes an absolute mess of dummy targets and near gibberish.
| thangalin wrote:
| Bash can also employ a kind of function pointer, which can
| greatly simplify command-line argument parsing. Consider the
| following simplified example: ARG_ARCH="amd64"
| ARGUMENTS+=( "ARG_ARCH,a,arch,Target operating system
| architecture (${ARG_ARCH})"
| "log=utile_log,V,verbose,Log messages while processing" )
| utile_log() { echo "$1" } noop() { return 1 }
| log=noop
|
| By creating a comma-delimited list of command-line arguments, the
| parsing logic and control flow can be influenced from a single
| location in the code base. The trick is to eval-uate
| "log=utile_log" when the "-V" command-line argument is provided
| (or assign ARG_ARCH to the user-supplied value after "-a"). Using
| the $log variable invokes the function, such as in:
| preprocess() { $log "Preprocess" }
|
| If "-V" isn't supplied, then every invocation of $log simply
| returns without printing a message. The upshot is a reduction in
| conditional statements. Function pointers FTW. I wrote about this
| technique in my Typesetting Markdown series:
|
| https://dave.autonoma.ca/blog/2019/05/22/typesetting-markdow...
|
| Here's a rudimentary implementation and example usage to wet your
| whistle:
|
| https://github.com/DaveJarvis/keenwrite/blob/master/scripts/...
|
| https://github.com/DaveJarvis/keenwrite/blob/master/installe...
| Izkata wrote:
| It also works with dynamic names, if you need a slightly
| different entry point and don't want to risk conflicting with
| other function names: run_foo() { echo foo; }
| run_bar() { echo bar; } name=foo run_$name
| dontcare007 wrote:
| Bash scripting helped me out of a big problem a couple of decades
| ago. It's nice to have a tool around that doesn't change out from
| under you.
| gjvc wrote:
| in bash, strongly prefer functions to aliases
| agumonkey wrote:
| let's make a script that migrate .aliases to named functions
| arp242 wrote:
| zsh has "always", which behaves as "try .. finally" and such in
| some other languages: { foo }
| always { cleanup stuff }
| sundarurfriend wrote:
| `always` is a much better name for that feature.
| rackjack wrote:
| As stated: posix compliant functions with local variables can be
| done with something like: foo () ( x="hello";
| echo $x )
|
| because this creates a subshell. Fun!
| kzrdude wrote:
| This-all is actually not exactly about bash functions, but POSIX
| shell functions:
| https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...
|
| `local` is not in POSIX, but is so common in "posix" shells that
| many use it, just a sidenote.
| notatoad wrote:
| >`local` is not in POSIX
|
| So maybe it _is_ about bash functions and not posix functions,
| because it 's refencing features that exist in bash but not
| posix?
| pdkl95 wrote:
| > Well, because it simply doesn't work for them: returning from a
| function does not trigger the EXIT signal.
|
| It doesn't trigger EXIT, but it does trigger RETURN. Just trap
| both: #!/bin/bash foo() {
| trap "echo 'Cleanup!'" RETURN EXIT #return
| #exit echo "Kill me with ^C or \"kill $$\""
| while true ; do : ; done } foo # should
| print 'Cleanup!' on SIGTERM, # returning, or
| calling exit
| xeyownt wrote:
| Wow, this is great! Did not know that, thanks for sharing!
___________________________________________________________________
(page generated 2021-10-31 23:00 UTC)