[HN Gopher] OpenBSD cron(8) now supports random ranges with steps
___________________________________________________________________
OpenBSD cron(8) now supports random ranges with steps
Author : gslin
Score : 99 points
Date : 2023-05-08 15:40 UTC (7 hours ago)
(HTM) web link (undeadly.org)
(TXT) w3m dump (undeadly.org)
| kevin_thibedeau wrote:
| How would this work with a step that isn't divisible into the
| range for the field? Given minutes 0~59/25 with a random offset
| of 0 there will be an event at 0, 25, and 50 minutes past the
| hour. On the next hour does it start at 0, 15, or a new random
| offset? i.e. constant offset, constant step, or regenerated
| offset.
| aldto wrote:
| I read it as the 25 is just a simple step. So the first hour in
| your example would run at 0, 25, and 50. The next hour would
| run at 15 and 40, etc.
| g0xA52A2A wrote:
| Another nicety OpenBSD's cron gained not long ago was the "-s"
| flag to ensure only a single instance of command will be run
| concurrently.
|
| https://man.openbsd.org/man5/crontab.5#s
| pengaru wrote:
| That's gross. If the operation shouldn't run concurrently it
| should use an exclusive flock or similar. It's not just cron
| that can cause concurrent execution, and if that matters you
| generally want to robustly prevent it - not just if cron is the
| executor.
| mjw1007 wrote:
| I think it's fine for the case where running the operation
| more than once at a time isn't incorrect, but potentially
| wastes resources.
| thiagocsf wrote:
| It's not gross. It's a simple solution to a simple and common
| cron problem
|
| Just because it doesn't consider external executions, it
| doesn't mean it's a bad solution. Horses for courses etc
| pengaru wrote:
| > It's not gross. It's a simple solution to a simple and
| common cron problem
|
| No, it's gross. By providing that facility in the wrong
| place it discourages implementing it in the right place to
| people who come at the problem from the cron perspective.
|
| Wrap the command in a flock-running script. That script
| goes in the crontab entry. When you're inevitably debugging
| your cron-scheduled command - paydirt! The command
| serializes itself still while you're manually testing
| instead of shitting itself.
| eska wrote:
| Isn't that the same? Just because you check a file lock
| in your script doesn't mean that other invocations of the
| program without the script will check the lock.
| Apofis wrote:
| Lock files in scripts are actually pretty unreliable,
| learned this the really hard way and the lesson cost
| $10's of thousands of dollars.
| pengaru wrote:
| No, it's not the same. The crontab entry reflects what
| you'd run to reproduce what the crontab does.
| justsomehnguy wrote:
| Ah, yes, lets add exclusive flock support to _all_ binaries
| in the existence, just in case.
|
| You are confusing running a process with running _a task_
| from scheduler.
| pengaru wrote:
| > Ah, yes, lets add exclusive flock support to all binaries
| in the existence, just in case.
|
| You are aware that flock(1) is a thing, right? And that a
| defacto core tenet of UNIX is composability of disparate
| programs that do one thing well?
| justsomehnguy wrote:
| > You are aware that flock(1) is a thing, right?
|
| No. Maybe I heard and even knew what it is, but I never
| needed it.
|
| You know why?
|
| My task scheduler supports running only one instance of
| the task so I don't need to reinvent the wheel every time
| I need something to run on schedule.
|
| Just 'somebinary args args args', 'run only one instance'
| and I'm done.
|
| > flock - manage locks from shell scripts
|
| Well, yep. Reinventing the wheel each other time.
|
| Thanks, I have some more important things (like browsing
| HN) than writing shitty shell scripts.
| trasz4 wrote:
| >Well, yep. Reinventing the wheel each other time.
|
| Erm, no - flock(1) is a UNIX tool, a component if you
| will. It's the exact opposite of reinventing the wheel.
|
| Not understanding the operating system you're using is
| fine, but it's good to at least know what you don't know.
| pengaru wrote:
| > No. Maybe I heard and even knew what it is, but I never
| needed it.
|
| ...
|
| > Thanks, I have some more important things (like
| browsing HN) than writing shitty shell scripts.
|
| Probably a good idea, before you dig that hole of
| incompetence any deeper.
|
| We're talking about crond here, which is already running
| the scheduled commands via sh, it's inherently involving
| the shell. Welcome to *NIX.
| justsomehnguy wrote:
| > efore you dig that hole of incompetence any deeper.
|
| "Hur-dur! Look at me! Me mighty UNIX wizard! Bow to me!"
|
| > running the scheduled commands via sh, it's inherently
| involving the shell. Welcome to *NIX.
|
| Thanks, while you are running _shell commands_ I 'm
| running systems. I'm not interested in writing shell
| scripts for things what were solved _decades ago_.
| gsich wrote:
| Better look into a mirror ...
|
| Using cron (or systemd) to ensure only 1 instance is
| running is a valid approach.
| joosters wrote:
| _a defacto core tenet of UNIX is composability of
| disparate programs that do one thing well?_
|
| cron goes completely against that principle - after all,
| you can schedule jobs with the 'at' command, and to make
| a repeating task, you just make it exec 'at' again each
| time it is called. cron is for the lazy, no real UNIX
| hacker would dream of using such an extravagant single-
| use program. /s
| mlyle wrote:
| If your job is a real script: sure, handle locking in it.
|
| If it's a single command or pipeline... adding the layer of
| indirection to have a script that runs flock is more opaque.
| Might as well put it in cron and trust cron to only run it
| once.
| progmetaldev wrote:
| You are of course correct, but at the same time, you might
| (or probably not) be surprised at the number of "system
| administrators" that are thrown into the job without really
| having the capability to expand too far on their knowledge.
| Having the option in cron may help those administrators that
| specifically search Google for cron usage, and never come
| across flock.
|
| EDIT: In addition, the Task Scheduler in Windows has this
| type of option, so it may help those sys admins coming from
| that environment, leveraging their existing knowledge
| mekster wrote:
| An external program but there's run-one program in Ubuntu repo.
| hnlmorg wrote:
| Today I learned the -n flag. Is this also an OpenBSD exclusive?
|
| I have to say, these flags are a really nice enhancement to
| cron.
| networked wrote:
| FreeBSD and NetBSD also implement "-n". There doesn't seem to
| be a cross-platform port of OpenBSD cron like there are of
| doas and OpenBSD ksh. (Anybody want to try making one?)
| Cross-platform fcron has "erroronlymail".
|
| https://man.freebsd.org/cgi/man.cgi?query=crontab&apropos=0&.
| ..
|
| https://man.netbsd.org/NetBSD-9.3-STABLE/crontab.5
|
| http://fcron.free.fr/doc/en/fcrontab.5.html#FCRONTAB.5.ERROR.
| ..
| znpy wrote:
| From this thread is emerging that pretty much most operating
| systems had this for a while in some forms... except maybe
| openbsd (and a few others?).
|
| Btw congrats to the openbsd people for catching up.
| networked wrote:
| Nice! To me this is a small but meaningful differentiator for
| OpenBSD. It is similar to the "random" feature in my favorite
| cron, fcron:
| http://fcron.free.fr/doc/en/fcrontab.5.html#FCRONTAB.5.RANDO....
|
| In crons that do not support something like this, you can
| introduce a random delay with a Perl oneliner. For example,
| # Start within the first 10 minutes of a matching hour. 0
| */8 * * * perl -e 'sleep rand 10*60' && ~/.config/jobs/fetch-tcl-
| logs
| yakubin wrote:
| And if you have jot, but not Perl, like on a bare FreeBSD
| install: 0 */8 * * * sleep "$(jot -r 1 0 599)"
| && ~/.config/jobs/fetch-tcl-logs
| zeroimpl wrote:
| Might as well just use bash sleep $((RANDOM
| % 600)) && fetch-tcl-logs
| rollcat wrote:
| Given how taxing the "thundering herd" effect can be on mirrors,
| websites (RSS readers!), you'd think this sort of thing should've
| been in cron since at least mid-90s.
|
| Once again, OpenBSD with the simple, obvious solution, that
| everyone else kinda overlooked. I hope every other cron out there
| copies and ships this as soon as possible.
| throw7 wrote:
| I first ran into randomization delays in cronie (the ~ is how
| it's implemented also and there's a RANDOM_DELAY variable for
| use too), after redhat had switched to it at some point years
| ago. Personally, never really used it, but it's nice it's
| there.
| rollcat wrote:
| Intriguing! I had no idea! Again, I wish the concept was more
| popular back when cron and the Internet were younger, and
| having it built-in and readily available goes a very long
| way. If someone doesn't introduce you to the concept, you
| have little chance of knowing better until you find yourself
| at the receiving end of a spike, yelling at the clouds.
| dralley wrote:
| >OpenBSD with the simple, obvious solution, that everyone else
| kinda overlooked
|
| Systemd Timers have had this for a while.
| trasz4 wrote:
| Yeah, but with systemd timers it's just another ad hoc hack.
| djbusby wrote:
| I'm not hot for systemd but this https://www.freedesktop.or
| g/software/systemd/man/systemd.tim... looks robust, far
| from ad-hoc hack.
| usr1106 wrote:
| RandomizedDelaySec=
| lockhouse wrote:
| They said simple and obvious, which much of systemd is not.
|
| Personally I find the OpenBSD solution far more elegant and
| UNIXy than the systemd one, but to each their own.
| ijustlovemath wrote:
| What's more simple than an ini file?
| rodgerd wrote:
| Reinventing the same features many times over in many
| different places, of course.
| ratorx wrote:
| Why does the complexity of the rest of systemd matter when
| comparing the implementation of this feature?
|
| What would you say is wrong with RandomizedDelaySec?
| mekster wrote:
| UI is too complicated to type some word concatenated
| keyword than being visually intuitive.
| cratermoon wrote:
| I believe the philosophy was to leave it up to the service to
| behave sensibly, including things like have a circuit breaker,
| use some kind of backoff/retry, and generally be robust in the
| face of resource contention.
|
| It kind of feels like this is putting the policy of "don't all
| go at once" into the cron mechanism, which is just starting
| jobs at desired times.
| nvy wrote:
| It's a neat idea to be sure but I fail to see how this will
| have any material impact.
|
| Firstly, OpenBSD is a niche OS, meaning the absolute magnitude
| of OpenBSD cron jobs out "in the wild" is relatively low.
|
| Second, my understanding is that this is a client-side feature.
| I.e. if I run a service, this feature only benefits me if a
| significant portion of my users opt into it.
|
| Third, I have an unsubstantiated suspicion that cron usage
| relative to systemd usage is also on the decline.
| toast0 wrote:
| freebsd cron(8) has -j for the daemon to add a random sleep of
| up to 60 seconds on each task. This was added in FreeBSD 5.3,
| committed 19 years ago. (FreeBSD 5.3 released November 6, 2004)
|
| https://github.com/freebsd/freebsd-src/commit/f5896baf9c429c...
| chaps wrote:
| A random sleep of up to 60 seconds doesn't really solve the
| problem the OpenBSD changes do, especially when your jobs
| take longer than 60s. For example, instead of
| "0-59/10" in the minutes field, "0~59/10" can be used to run
| a command every 10 minutes where the first command starts at
| a random offset in the range [0,9]. The high and low numbers
| are optional, "~/10" can be used instead.
| cyberpunk wrote:
| A random sleep of up to 10 seconds turned a 150gbit/sec
| spike into a 5gbit/sec spike on the Akamai bill from a
| newspaper app I once worked on...
| mlyle wrote:
| That's surprising. You'd think spreading the workload
| start over 10 seconds would lower the size of spikes
| (integrated over a second) by at most a factor of 10.
|
| But the above point is still true: many jobs take a few
| minutes to run. 60s of dispersion in start time is better
| than nothing, but you really want more.
|
| (In this case, things are still quantized to a minute
| boundary, so you'd really want _both_ ).
| compumike wrote:
| This is a nice feature at the minute-resolution level. I think
| something at the second-resolution level would be helpful too.
| For example, I have a cronjob on my Raspberry Pi at home that
| runs every minute and does a simple check-in with Heii On-Call so
| I get alerted if there's a FIOS outage or the pi breaks. I ended
| up writing a little bash script like this:
| #!/bin/bash set -e
| HEIIONCALL_API_KEY="redacted_api_key_goes_here"
| HEIIONCALL_TRIGGER_ID="redacted_trigger_id_goes_here"
| AUTHORIZATION_HEADER="Authorization: Bearer
| ${HEIIONCALL_API_KEY}" CHECKIN_URL="https://api.heiioncal
| l.com./triggers/${HEIIONCALL_TRIGGER_ID}/checkin" if
| [ "$1" != "--now" ]; then RANDOM_SLEEP=$[ ( $RANDOM %
| 55 ) + 1 ] echo "Sleeping ${RANDOM_SLEEP} seconds
| before checkin..." sleep ${RANDOM_SLEEP}s fi
| echo "Checking in..." exec curl \ -X POST \
| --retry 5 --retry-connrefused --retry-max-time 15 --retry-delay 1
| \ -H "${AUTHORIZATION_HEADER}" \
| "${CHECKIN_URL}"
|
| This script ~/bin/heiioncall-checkin.sh gets called by crond
| every minute at exactly :00 seconds, so my expected maximum
| timeout between check-ins is approximately 120 seconds. And I can
| skip the sleep with "--now" flag for testing. But I'd much rather
| have this random offset behavior be something optionally built-in
| to cron, I suppose.
| psychphysic wrote:
| This is useful. Many services that run periodically can end up
| with odd patterns emerging.
|
| E.g. Tor network capacity varies over the month due to bandwidth
| management tending to happen at the begining/end of months
| 1letterunixname wrote:
| Some people call it splay. It helps to not have all running
| instances hug to death telemetry and configuration management
| servers.
___________________________________________________________________
(page generated 2023-05-08 23:00 UTC)