[HN Gopher] More dynamic cronjobs
___________________________________________________________________
More dynamic cronjobs
Author : 0928374082
Score : 85 points
Date : 2025-12-27 06:10 UTC (16 hours ago)
(HTM) web link (george.mand.is)
(TXT) w3m dump (george.mand.is)
| victorbjorklund wrote:
| Cool. Had no idea you could run commands inside a CRON
| expression.
| garganzol wrote:
| Running a command is the main idea of cron. In this case, the
| author runs composite commands like: test &&
| action
|
| Where 'test' is another shell command that returns 0 or 1. This
| is not a special cron syntax, it's just the inherent capability
| of the Unix shell.
|
| In any case, this whole approach is very clever and shows the
| beauty of The Unix Way.
| WolfCop wrote:
| Does anyone maintain a programmatically accessible list of
| holidays for their company? Similar to the HOLIDAYS.txt in the
| article, but it would allow for things like "don't run this the
| day before or during a company holiday."
|
| I work at a company with different holidays in certain countries,
| which would complicate things, and require something more
| structured than a list of dates. But having that accessible could
| be useful.
|
| Has anyone tackled that, or come across a solution?
| jaredsohn wrote:
| Ruby has https://github.com/bokmann/business_time but when I
| looked at it, custom code was needed to calculate holidays that
| were offset because they are on the weekend.
| glawre wrote:
| At our company we have enough systems reliant on holiday dates
| that we have a Holiday system that emits events when there are
| changes.
|
| This happens surprisingly often, given that religious dates
| change and there are holidays/closures for storms in some
| regions.
| sublinear wrote:
| I had something come up recently that I think sounds similar.
| That project needs several time-sensitive jobs. When any one of
| them runs, the first thing it does is check a holidays.json
| file.
|
| It parses the file using jq and compares its entries with the
| current time according to GNU date. At the root is the names of
| the jobs. Each job has its own list of holidays. Each of these
| holiday items in the job's respective list has keys for the
| display name of the holiday, the formatted date to compare to,
| and in a few cases the ISO day-of-week and a string containing
| a modulo arithmetic function (e.g. don't run the friday before
| Christmas, etc.).
|
| Sorry, yes that means I call eval on that string and yes that
| means some of these are repeated in the same file under the
| arrays for the other jobs. Also, such lists will have to be
| maintained and the exact observed dates cannot always be known
| ahead of time beyond about a year since people can change their
| minds for various reasons (think bank holidays). Depending on
| your use case you may also want to define a start time and end
| time for a window of when this should or shouldn't run (i.e.
| business hours).
|
| I don't know if that helps. I know it's hacky, but I don't
| think there's a nice way to handle things like "second monday
| after 4th of july, but if the 4th also happens to be monday
| then it should instead be the second tuesday". God help you if
| you also need to handle each holiday being observed in
| different timezones. At least at the end of the day none of
| this would be much code, just very terse code dense with
| meaning.
| godelski wrote:
| I think this could be solved quite similar to the OP and better
| done with systemd. Spitballing, but I think the best thing to
| do would be to write the timer in a standard file but have the
| activation time be written in an override file. That way you
| can ensure you are just editing that file with your scraper
| (should be able to hit the API if it is something like a google
| calendar or outlook).
|
| I think the systemd timer would give you the benefit here as
| you can write the time in varying formats. Timezones, UTC,
| local, or whatever. That should give you the structure you
| need, if I'm understanding your problem correctly.
|
| While systemd has more boilerplate than cron I think it has a
| lot of advantages that make it worth it. Best to just have a
| skeleton of these jobs (I keep some in my dotfiles) and then
| you have it. Or have the LLM write it (ironically one of the
| few instances I'll advocate for letting the AI write the code).
| You can do everything in the article and so much more.
|
| https://man.archlinux.org/man/systemd.time.7
| stackskipton wrote:
| I've been at several companies that have tried.
|
| One just did it with code where all the processes had
| holiday.json which would be checked at each launch, if it was
| holiday, it would do no work and exit.
|
| Other one is operator that would monitor if it was supposed to
| be a holiday and either change systemd or Kubernetes to suspend
| the jobs.
|
| I'd recommend code over messing with the system, much more
| flexible.
| petepete wrote:
| The GOV.UK website has a list of UK bank holidays on it. If
| you add a .json to the end of the URL you get this, a thing
| of beauty.
|
| https://www.gov.uk/bank-holidays.json
| krick wrote:
| Not sure what do you mean. I.e., what exactly is supposed to be
| the tricky part. Yes, I've dealt with quite a few processes
| like that, but I never tried, or would ever want to to put this
| into crontab. In fact, I don't know how the author intended his
| article, but if you consider doing _that_ for production, I
| strongly advice you not.
|
| What you do instead, is you schedule the cronjob for the most
| generic case, e.g. each day. And if it does not need to run 3
| days before holidays with crescent moon when wind is blowing
| from the south, it is just the part of business logic of the
| process, which you write in the any proper programming language
| that you prefer (or that the system is written in anyway).
|
| Now, how do you manage the list itself depends on the details
| and I've done all sorts dirty things that one probably
| shouldn't do (cutting corners), but in the most flexible case
| it is just some CRUD-type page in your back-office system, with
| a real UI, and there is a person (usually in the bookkeeping
| department of the company) who has it among his
| responsibilities to maintain the schedule. You store it in some
| proper SQL database and cache it aggressively, so the the
| myriads of cronjobs don't bother it more than necessary.
| AndrewDavis wrote:
| Great post. And if you want some control support for your
| cronjobs perl App::Cronjob[1] can provide features such has
| exclusive locking, so a job won't run if the previous run is
| still going, or provide a timeout, and some options for sending
| mail on success or failure
|
| [1]https://metacpan.org/pod/App::Cronjob
| https://metacpan.org/dist/App-Cronjob/view/bin/cronjob
| stevenjgarner wrote:
| This is great! I'm sure like a lot of programmers, I had been
| fulfilling the requirement for similar conditional logic by
| having a simple recurring cron job run other code or database
| queries with the conditional logic that this post demonstrates
| can be done directly in cron.
| bblb wrote:
| Also check out the 'chronic' command from moreutils. No more dev
| nulls.
| hermannj314 wrote:
| I learned something cool about cron filtering and a nice api I
| didn't know existed - date.nager.at
| hiAndrewQuinn wrote:
| I keep toying with the idea of writing a cron that implements a
| Poisson process. Say I give it a parameter of 3600; our `pcron`
| would ensure the jobs occur randomly but average out to once per
| hour, making the timing of the next run independent of the last
| via the memoryless property of the exponential distribution.
|
| The next sleep interval would be calculated probably as as t =
| -\lambda \ln(U) (where U is a uniform random variable). This way
| you ensure that the probability of the job firing in the next 10
| seconds is the same whether the last job finished an hour ago or
| just five seconds ago. But \lambda remains the _average_ amount
| of time between jobs.
|
| It's compelling to me because it solves thundering herd problems
| at the architectural level, and also because it simply seems like
| a lot of fun to have to code very defensively against such chaos.
| Switching back to a deterministic schedule after surviving such
| chaos probably leads to a much more robust system overall.
| tomashubelbauer wrote:
| This sounds like a really fun idea.
| jpalomaki wrote:
| Learned once the hard way that it makes sense to use "flock" to
| prevent overlapping executions of frequently running jobs. Server
| started to slow down, my monitoring jobs started piling, causing
| server to slow down even more. */5 * * * * flock
| -n /var/lock/myjob.lock /usr/local/bin/myjob.sh
| cluckindan wrote:
| Have you tested how this behaves on eventually consistent cloud
| storage?
| atherton94027 wrote:
| I'm confused, is EBS eventually consistent? I assume that
| it's strongly consistent as otherwise a lot of other linux
| things would break
|
| If you're thinking about using NFS, why would you want to
| distribute your locks across other machines?
| dherls wrote:
| I would definitely recommend not putting complex logic like this
| in your cron definitions. Much more annoying to find and debug in
| the future. I prefer to write a short wrapper script that
| contains the test logic instead and track/version control it
| zbentley wrote:
| Good advice. You can also check in and version your crontabs
| (or timer units or whatnot) directly.
| threemux wrote:
| Embedding a test like that is something I've never considered -
| very cool.
|
| These days I tend to use systemd timers on Linux though. Despite
| my love/hate relationship with systemd, timers and service files
| are really nice.
___________________________________________________________________
(page generated 2025-12-27 23:01 UTC)