[HN Gopher] TIL: timeout in Bash scripts
___________________________________________________________________
TIL: timeout in Bash scripts
Author : lr0
Score : 206 points
Date : 2025-05-26 11:34 UTC (11 hours ago)
(HTM) web link (heitorpb.github.io)
(TXT) w3m dump (heitorpb.github.io)
| chii wrote:
| could you instead just add a count to how many times the sleep
| was invoked, and then add that check into the `until` condition
| to quit after X numbers of sleeps?
|
| You dont need to timeout here, and you won't need to subshell
| another bash to just get the timeout to work.
| xelxebar wrote:
| For the author's stated purpose, it sounds like this would work
| well. However, curl can take a long time to timeout of
| whatever, depending on server state. I'm curious how people
| would approach a guaranteeing a maximum time to response or
| error.
| chii wrote:
| In the case of curl, there is a timeout parameter for
| requests and connections etc. It's what i'd use for checking
| for a server being up.
|
| But in the general case, where the command being invoked does
| not have such an option, then it does make a lot of sense to
| do a check like that via the `timeout` utility.
| diggan wrote:
| For curl:
|
| --connect-timeout = Times out if the connection wasn't
| established within N seconds
|
| --max-time = Times out if the entire request wasn't
| completed within N seconds
|
| But then I don't remember if connect-timeout takes DNS
| lookups into account, or TLS handshakes. I seem to remember
| there is another sort of timeout that tends to be hard to
| get right when the connections are flaky/drops a lot of
| packets, so you end up having to wrap curl anyways if you
| want a hard limit on the timeout.
| SoftTalker wrote:
| I've had many, many cases of processes in Linux getting
| wedged up bad enough that they cannot be killed. Usually
| this seems to involve them waiting on I/O.
| cb321 wrote:
| It's not POSIX, but both bash & zsh have real time queries
| built-in (of course, $(date) is an even older school
| substitute): t1=$((EPOCHSECONDS + 60))
| while [ $EPOCHSECONDS -lt $t1 ] do # curl ... &&
| break # or whatnot done
|
| There is also EPOCHREALTIME which gives you a floating point
| to micro or nanoseconds (for bash/zsh), but only Zsh provides
| FP arithmetic. There are string-manipulation workarounds, of
| course. And, yes, with Zsh you might need a `zmodload
| zsh/datetime` in there.
|
| These variables seem "under known". EDIT: For example, you
| can get a quickie wall time measurement from a Zsh shell
| function like this: dt () {
| t0=$EPOCHREALTIME "$@" printf "%.7f
| wallSec\n" "$((EPOCHREALTIME-t0))" >&2 }
|
| And then you can actually run in your shell
| dt echo hi, moon
|
| without even a single fork/clone. (which you can confirm with
| an off to the side `strace -fv -o/dev/shm/dt.st -p
| WHATEVER_PID`, although I guess the culture these days is
| often to have even prompt printing launch a zoo of activity)
| crabbone wrote:
| Not really (at least, not very easily). There's no guarantee
| that for whatever reason curl won't hang.
|
| To do it properly, you'd need some code before the loop to
| start a separate process that would check on the parent
| process... but, really, you don't want to go there, not in Bash
| anyways.
|
| But, assuming curl won't hang, you could compare timestamps.
| It's better than counting iterations (in terms of emulating
| timeout command).
|
| But then, you might want to get fancy and implement exponential
| backoff or whatever other strategy you fancy to not overload
| the whatever thing you are polling... again, probably not in
| Bash.
| miduil wrote:
| What I usually do when I need a retry logic is
| for i in {0..60}; do true -- "$i" # shelleck
| surpression if eventually_succeeds; then break; fi
| sleep 1s done
|
| Not super elegant, but relatively correct, next level is
| exponential back off. Generally leaves a bit of composability
| around.
| miduil wrote:
| Note this will still require timeout for eventually_succeeds
| depending on the application.
|
| In Bash, or literally whenever you are dealing with
| POSIX/IO/Processes, you need to work with defensive coding
| practices.
|
| Whatever you do has consequences
| mdaniel wrote:
| Up to you but I think the way shellcheck wants that problem
| solved is by using _ as in for _ in
|
| https://github.com/koalaman/shellcheck/wiki/SC2034#intention...
| frou_dh wrote:
| Apparently timeout(1) is part of GNU Coreutils. I wasn't sure
| after reading whether it was part of Bash itself.
| mdaniel wrote:
| Also, watch out because like many things the timeout command
| and args differs between /usr/bin/timeout or gtimeout in Brew
| (that's where the "g" prefix comes from). I haven't used BSD in
| order to know what it's story is
| aidenn0 wrote:
| Prefixing GNU coreutils with "g" is common on most non-Linux
| Unix systems; it prevents conflicts with the base system
| (gmake/gtar vs make/tar).
| jonhohle wrote:
| But also sucks because the g-prefixed versions aren't
| installed on Linux systems which means scripts that rely on
| them are not portable.
| mdaniel wrote:
| Thankfully bash tolerates that, if the script author
| cares, e.g. gnu_sed=gsed if !
| command -v $gnu_sed; then
| gnu_sed=$(detector_wizardry) fi $gnu_sed -Ee
| ...
| chasil wrote:
| > It's a shame we can't use timeout with until directly
|
| The until keyword is part of the POSIX.2 shell specification,
| which does not include any sort of timeout functionality. It
| could be implemented in bash, but it would not be portable to
| other shells (Debian dash being the main concern).
|
| This is the reason that it is implemented as a separate
| utility.
|
| Search for "The until loop" below to see the specification.
|
| https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...
| majke wrote:
| My fav little-known trick is to test various syscalls fail with
| strace fault injection, like: $ strace -e
| trace=clone -e fault=clone:error=EAGAIN
|
| random link: https://medium.com/@manav503/using-strace-to-
| perform-fault-i...
| jonhohle wrote:
| This is incredible and something I'd wish I'd known about a
| long time ago. I'd often stub out functions like this knowing I
| couldn't test the failure branch, but try to limit that to as
| small an area as possible.
|
| Thanks!
| ycombinatrix wrote:
| This is great. Anyone know of an equivalent in Windows?
| dwattttt wrote:
| Application Verifier provides fault injection, as well as
| detection for a bunch of conditions
| (https://learn.microsoft.com/en-us/windows-
| hardware/drivers/d...).
|
| It's only intended for native/unmanaged code though.
| pveierland wrote:
| Literally just added some command timeouts in a new kubernetes
| setup. This POSIX shell script implementation of await-cmd.sh /
| await-http.sh / await-tcp.sh is mature and quite handy in some
| scenarios:
|
| https://github.com/vegardit/await.sh
| noufalibrahim wrote:
| I used to use timeout 1800 mplayer show.mp4 ;
| sudo pm-suspend
|
| As my poor man's parental control to let my kids watch a show for
| 30 minutes without manual supervision when they were younger.
| Useful command
| gbraad wrote:
| That is probably the best described use case!
| AtlasBarfed wrote:
| Is there a language with a less standardized standard library
| than bash?
|
| Is there an attempt anywhere to build a slightly modern standard
| library for bash scripts?
|
| You know besides stack overflow?
| cpach wrote:
| Would it really be worth the effort? When this level of
| complexity is reached, I personally think it's better to use a
| more capable language, such as Python or Ruby.
| ninkendo wrote:
| Not only that, but the strength of bash is its ubiquity. (Or
| for that matter, posix sh, if you want even more ubiquity.)
| If we started adding lots of features to bash, it wouldn't
| make sense to use them unless you are positive every place
| using your script has a new-enough version installed. Which
| defeats the purpose of the main use case for bash in the
| first place, which is (IMO) for portable scripts that will
| run on any Unix-like system.
| AtlasBarfed wrote:
| I may not like bash, but I sure as hell use it a lot...
|
| And I like messy langs. My favorite language is groovy.
| crabbone wrote:
| Neither Python nor Ruby offer a simple interface to
| concurrency, serialization and terseness. Not even close. And
| aren't moving in the desired direction. Perl could have
| tried... but it's a separate can of worms, and still not
| quite there.
|
| PowerShell is a missed opportunity. A project with a ton of
| resources dedicated by a company with bottomless coffers...
| which ended up being sub-par.
|
| I wish there was a sensible alternative, but I haven't found
| one yet.
| oweiler wrote:
| Bash has no standard library. It has builtins, and commands.
| And commands are just external tools.
| queuebert wrote:
| Even '[' is an external binary in /usr/bin typically.
| AStonesThrow wrote:
| "Standard library" is sort of a C-specific term.
|
| "builtins" are primitives that Bash can use internally
| without calling fork()/exec(). In fact, builtins originated
| in the Bourne shell to operate on the current shell
| process, because they would have no effect in a subprocess
| or subshell.
|
| In addition to builtins and commands, Bash also defines
| "reserved words", which are keywords to make loops and
| control the flow of the script.
|
| https://www.gnu.org/software/bash/manual/bash.html#Reserved
| -...
|
| Many distros will ship a default or skeleton .bashrc which
| includes some useful aliases and functions. This is sort of
| like a "standard library", if you like having 14 different
| standards.
|
| https://gist.github.com/marioBonales/1637696
|
| '[' is an external binary in order to catch any shell or
| script that does not interpret it as a builtin operator.
| There may be a couple more. Under normal circumstances, it
| won't actually be invoked, as a Bash script would interpret
| '[' as the 'test' builtin.
| t-3 wrote:
| Busybox/coreutils/the userspace of your platform are the
| "standard library". The shell is basically just there for
| control flow and IO, everything else is just programs on your
| computer.
| beej71 wrote:
| POSIX and the Single Unix Specification are pretty much all you
| have
|
| I write a lot of shell scripts and they tend to be POSIX-
| compliant. For dependencies, you can use the `command` command
| to fail elegantly if they're not installed.
| alganet wrote:
| https://github.com/shellfire-dev
|
| https://github.com/shellspec
|
| https://oils.pub/
|
| There's probably more.
|
| The shell has a ____wide____ userbase with many kinds of users.
| Depending on your goal, the rabbit hole can go very deep (how
| portable across interpreters, how dependant on other binaries,
| how early can it work in a bootstrap scenario, etc).
|
| These are mine:
|
| https://github.com/alganet/coral
|
| https://github.com/alganet/shell-versions
|
| https://github.com/Mosai/workshop
| minaguib wrote:
| In _this_ particular case, you could just tell curl to internally
| timeout the request (via `-m`) instead of trying to manage the
| timeout on the process level
| aidenn0 wrote:
| Not really, since it's calling `curl` in a loop, and they want
| the loop to timeout. There's possibly a set of options to curl
| to make it retry for a certain amount of time but I don't know
| it off the top of my head.
| PeterWhittaker wrote:
| I tend to do something like this. Normally, I wouldn't include
| the extra _jobs_ calls and extra _echo_ calls, these are just to
| show what is happening. #!/usr/bin/env bash
| runUntilDoneOrTimeout () { local -i timeout=0
| OPTIND=1 while getopts "t:" opt; do case
| $opt in t) timeout=$OPTARG;; esac
| done shift $((OPTIND - 1)) runCommand="$*"
| $runCommand & runPID=$! echo checking jobs
| jobs # just to prove there are some echo job check
| complete while jobs %- >& /dev/null && ((timeout > 0));
| do echo "waiting for $runCommand for $timeout
| seconds" sleep 1 ((timeout--))
| done if (( timeout == 0 )); then echo
| "$runCommand timed out" kill -9 $runPID
| wait $runPID else echo "$runCommand
| completed" fi echo checking jobs
| jobs # just to prove there are none echo job check
| complete } declare -i timeopt=10 declare
| -i sleepopt=100 OPTIND=1 while getopts "t:s:" opt; do
| case $opt in t) timeopt=$OPTARG;; s)
| sleepopt=$OPTARG;; esac done shift
| $((OPTIND - 1)) runUntilDoneOrTimeout -t $timeopt sleep
| $sleepopt
| aidenn0 wrote:
| Note that if you need to pass variables into the bash -c
| invocation, the best way to do it is to append them. e.g.
| bash -c 'some command "$1" "$2"' -- "$var1" "$var2"
|
| I use "--" because I like the way it looks but the first
| parameter goes in argv[0] which doesn't expand in "$@" so IMO
| _something_ other than an argument should go there for clarity.
|
| Note that bash specifically has printf %q which could
| alternatively be used, but I prefer to use bourne-compatible
| things when the bash version isn't significantly cleaner.
| fragmede wrote:
| Busybox uses argv[0] to know what to run, so you can feed it
| "ls" as argv[0] and it'll run "ls" (or "mv"/"cp"/etc).
| tryauuum wrote:
| why didn't he opt to use `timeout --signal=SIGKILL` and instead
| wrapped everything in extra bash to make it more killable?..
| teo_zero wrote:
| According to TFA you can only kill processes and "until", being
| a builtin, doesn't spawn any new process.
| epr wrote:
| I'm generally not a huge fan of inlining the command or
| cluttering up my local directory with little scripts to get
| around the fact that it must be a subprocess you can send a
| signal to. I use a wrapper like this, which exports a function
| containing whatever complex logic I want to time out. The funky
| quoting in the timeout bash -c argument is a generalized version
| of what aidenn0 mentioned in another comment here (passing in
| args safely to subproc). #!/usr/bin/env bash
| long_fn () { # this can contain anything, like OPs until curl
| loop sleep $1 } # to
| TIMEOUT_DURATION BASH_FN_NAME BASH_FN_ARGS... to () {
| local duration="$1"; shift local fn_name="$1"; shift
| export -f "$fn_name" timeout "$duration" bash -c
| "$fn_name"' "$@"' _ $@ } time to 1s long_fn
| 5 # will report it ran 1 second
| abbeyj wrote:
| You need `"$@"`, not just `$@` at the end of the command.
| Otherwise it will split any arguments that have spaces in them.
| E.g. try long_fn() { echo "$1"
| sleep "$2" } to 1s long_fn "This has
| spaces in it" 5
| yonatan8070 wrote:
| I recently used timeout + tcpdump to bandaid over a race
| condition where sometimes a video streaming service started
| before the camera was ready and got stuck in a loop. So I just
| captured the video stream's port with tcpdump, then used timeout
| and tcpdump's exit code to tell if it's working or not
| js2 wrote:
| Retry is also a nice little utility that makes the retry loop
| easier:
|
| https://github.com/minfrin/retry
| arjie wrote:
| A friend recently showed me https://google.github.io/zx/api and
| it's actually quite enjoyable to use. Very close to a shell and
| LLMs know it quite well.
| artursapek wrote:
| ah reminds me of the jQuery glory days
| jiehong wrote:
| This reminds me of bun with its shell api for JavaScript [0].
|
| [0]: https://bun.sh/docs/runtime/shell
| craigds wrote:
| FYI curl actually helpfully has a `--retry-connrefused` flag to
| avoid doing this loop in the shell entirely
| febusravenga wrote:
| This is my attempt to reinvent wheel from several years ago:
| https://github.com/zbigg/bashfoo/blob/master/timeout.sh
|
| This is very complex, because if you.write lots of functions that
| call functions, you really just want to run something that
| inherits while env from your process, that's why there is control
| and sleep process and naive race to decide which finished
| first...
|
| That's probably reason I ignored built-in timeout...
| TacticalCoder wrote:
| I've got, since forever, an advanced Bash prompt. But I also
| don't want my Bash prompt to have any visible delay. So back in
| the days I came up with time outs working with milliseconds
| (which, AFAIR, isn't the case for the timeout command whose
| granularity is seconds at best?). It involved processes and
| killing etc. but it got me what I wanted: either an instant
| prompt with all the infos I want of an instant prompt which may
| miss one or two infos. I much prefer that to the _" my prompt
| contains no information because that's quicker"_.
|
| Been working flawlessly since 20 years: so flawlessly that I
| don't remember how it works.
| sllabres wrote:
| From personal experience I would always recommend an output of
| how many retries were necessary, if one expect zero. Otherwise
| the retry loop can hide a problems like an unreliable service or
| network until it's too late.
| leitasat wrote:
| Have you heard of exponential back off? Tl;dr make the sleep time
| dependent on the mumtof retries
| robinhouston wrote:
| I've been playing around with trying to make a timeout using just
| bash builtins, motivated by the fact that my Mac doesn't have the
| timeout command.
|
| I haven't quite been able to do it using _only_ builtins, but if
| you allow the sleep command (which has been standardised since
| the first version of POSIX, so it should be available pretty much
| anywhere that makes any sort of attempt to be POSIX compliant),
| then this seems ok: # TIMEOUT SYSTEM #
| # Defines a timeout function: # # Usage: timeout
| <num_seconds> <command> # # which runs <command>
| after <num_seconds> have elapsed, if the script # has not
| exited by then. _alarm() { local timeout=$1
| # Spawn a subshell that sleeps for $timeout seconds #
| and then sends us SIGALRM ( sleep
| "$timeout" kill -ALRM $$ ) &
| # If this shell exits before the timeout has fired, #
| clean up by killing the subshell subshell_pid=$!
| trap _cleanup EXIT } _cleanup() { if [
| -n "$subshell_pid" ] then kill
| "$subshell_pid" fi } timeout() {
| local timeout=$1 local command=$2 trap
| "$command" ALRM _alarm "$timeout" } #
| MAIN PROGRAM times_up() { echo 'TIME OUT!'
| subshell_pid= exit 1 } timeout 10
| times_up for i in {1..20} do sleep 1
| echo $i done
| anotherevan wrote:
| TIL Bash has `until` as well as `while`!
| oso2k wrote:
| Another fun way to test connectivity in pure bash (need a
| revision from the past 15 years) is timeout 5
| bash -c 'cat < /dev/null > /dev/tcp/google.com/80'
|
| Replace google.com and port 80 with your web or tcp server (ssh
| too!). The command will error/time out if there isn't a server
| listening or you have some firewall/proxy in the way.
___________________________________________________________________
(page generated 2025-05-26 23:00 UTC)