[HN Gopher] Deno Cron
___________________________________________________________________
Deno Cron
Author : 0xedb
Score : 184 points
Date : 2023-11-29 16:03 UTC (6 hours ago)
(HTM) web link (deno.com)
(TXT) w3m dump (deno.com)
| lambtron wrote:
| hey, andy from the deno team here. we're really excited to land
| this. happy to answer any questions or pass questions along to
| the team!
| threatofrain wrote:
| Any plans for an SQL db service?
| lambtron wrote:
| we've gotten a bunch of interest for this and the team is
| discussing it :)
| ushakov wrote:
| use sqld (libsql)?
| throwawaaarrgh wrote:
| Personally I would stay within your niche and focus on
| documentation, guides, and outreach to convert more people
| to your stack. A few simple but very reliable easy
| solutions is better than a lot of not great solutions.
| kavaruka wrote:
| how does it works on self-hosted environment?
| igorzi wrote:
| We're gathering more feedback for supporting self-hosted cron
| environment. Can you describe how you would use it?
| msoad wrote:
| is there an API to receive the state of this and/or other cron
| jobs?
| igorzi wrote:
| Not currently, but this is on the roadmap
| nwjsmith wrote:
| Is the spikiness of cron schedules going to cause you
| operational problems? You're going to end up with a _lot_ of
| jobs scheduled at "0 0 * * *".
| igorzi wrote:
| Yes, we're anticipating more spiky workloads because of this.
| Deno Deploy is already designed to handle spikes, but we also
| have a few additional mitigations in place for Cron. For
| example, we will limit concurrent dispatches for the same
| project/user/organization, which may slightly delay the
| execution of specific cron tasks.
| SheinhardtWigCo wrote:
| Charge 25% more for every 0 in the scheduling expression,
| problem solved :-)
| SahAssar wrote:
| Or add a default randomness factor that makes it run
| within a certain time (like 60s or 5m) of the target,
| perhaps with an additional charge to run at the exact
| time if people have that requirement.
| IshKebab wrote:
| Why on earth would you copy Cron's incomprehensible syntax?
| You're writing code in a statically typed language! You can
| make a nice type checked API that actually is understandable
| without having to use one of the many many crontab generator
| UIs.
|
| Crazy API design!
|
| Edit: seems like there is a WIP typed API but I still have no
| idea why you would lead with the terrible one...
| yawnxyz wrote:
| I've been looking for something like this! Is it possible to
| run rclone to send Google Drive documents to an S3 compatible
| endpoint? (~20GB of files)
| iobdas wrote:
| Excited to use this, nice work Deno team!
| Dig1t wrote:
| This is pretty neat, I can see this maybe being useful for
| database cleanup/housekeeping, what other example use cases are
| there?
| lambtron wrote:
| pulling from an API / scraping data, sending
| emails/notifications/reports, backing up databases/snapshots,
| checking availability of various services, pinging website,
| pre-warm up apps/scale applications, various maintenance tasks.
| skybrian wrote:
| When using Deno Deploy, which region does a cron job run in? Is
| there a way to configure it?
| igorzi wrote:
| It currently runs in us-east by default. Specifying a region is
| on our roadmap.
| ushakov wrote:
| Does it work locally?
| lambtron wrote:
| yes as of 1.38!
| vasergen wrote:
| Wanted to ask the same, because didn't find it in the article
| simonw wrote:
| I found the code for that here:
| https://github.com/denoland/deno/tree/v1.38.3/ext/cron
| aleksiy123 wrote:
| Nice, I was just wondering if there was an opportunity for a
| product similar to vercel for batch jobs.
|
| This isn't quite that but close.
|
| From my experience there 5 main components for larger scale
| system.
|
| - Request server
|
| - database
|
| - queue
|
| - background worker
|
| - offline batch jobs
|
| I feel like so far serverless has the first 3 fairly well
| developed but the last two are still underdeveloped.
|
| But maybe I'm just not aware of other solutions?
| goostavos wrote:
| EventBridge + Lambda or ECS gets you pretty far for the last
| two imo
| vvpan wrote:
| But then you need to set it up. On small fast moving teams
| having something that just works from the code is a massive
| win and I'd rather not cross the infra/code barrier.
| danfarrelly wrote:
| This is pretty much what Inngest is (https://www.inngest.com/).
| Runs on Deno as well.
| aleksiy123 wrote:
| Ingest is definitely nice. Its design is an
| orchestrator/scheduler where you offload your workers as
| serverless functions.
|
| The only issue is for background jobs you need to design it
| in such way to not run in to timeouts.
|
| Which makes it slightly more complex then just having a
| single executable running periodically.
|
| There is also https://www.defer.run/ which run your code on
| their infra and don't have timeouts. But they only support
| node/bun at the moment.
| danfarrelly wrote:
| We'll also be supporting long-running services as well in
| the near future which subscribe to updates from Inngest,
| rather than get invoked via HTTP.
|
| Currently, to get beyond the ~5 min limit per step, you'd
| need to deploy to something non-serverless like Fly.io,
| Render, or your own instance running Express.js or similar.
| TheIronYuppie wrote:
| This is really interesting - we've tried really hard to solve
| some of these with Bacalhau[1] - a much simpler distributed
| compute platform. Would love your feedback!
|
| [1] https://github.com/bacalhau-project/bacalhau
|
| Disclosure: I confounded Bacalhau
| aleksiy123 wrote:
| Huh, this is pretty cool.
|
| I took a quick look through the docs and from what I
| understand you can just submit docker containers to be run on
| a cluster. The default one is a public cluster. In some way
| it seems similar to Google Borg.
|
| What I don't understand though is do you have to pay? Who
| provides the compute?
|
| Maybe possible to build a vercel type service on top of the
| private cluster where people can just submit their code?
| slig wrote:
| You should check Windmill.dev.
| aleksiy123 wrote:
| I will check out windmill. Though it does look somewhat
| complex.
|
| One tangentially related is I'm looking for somewhere to just
| run a python script once a day.
|
| I'll see if I can do that with windmill.
| kkoppenhaver wrote:
| (disclaimer: Dev Advocate for Airplane)
|
| Seems like that would be a good fit for Airplane's
| Schedules: https://www.airplane.dev/schedules.
|
| It's pretty much as straightforward as deploying your
| Python script (and any dependencies with requirements.txt)
| and setting up your schedule for when you want it to run!
| aleksiy123 wrote:
| I will try this as well thank you!
| slig wrote:
| Yes, you can do that quite trivially on WM.
| jrklabs_com wrote:
| We have been using a self-hosted Windmill instance for
| running batch processes in Production for a while now. It
| has been a very nice upgrade from cron jobs for us (adding
| captured job outputs, status info, ability to re-run jobs,
| etc).
| rubenfiszel wrote:
| That's exactly what windmill.dev is (https://windmill.dev)
| cja wrote:
| My first question when I am considering a product like this is
| how can I get an email when there's an error in my application.
| Why isn't that in the top level feature list of everything like
| this, or am I unusual in thinking that this is an essential
| feature of any hosting system?
| throwawaaarrgh wrote:
| Some hosting systems may support that out of the box but I
| don't know what they are. Typically you would use metrics,
| monitors and alerts and create a notification alert based on
| certain conditions. No reason they couldn't provide some out of
| the box, but it's not common. At the very least it would need
| to stop after a short number of alerts or they'd end up
| spamming customers if it crashes on a loop
| evbogue wrote:
| I've started tossing https://ntfy.sh/ alerts into my Deno apps
| to get push notifications for things I'm interested in
| skybrian wrote:
| My wife has had good luck using Pushover [1], which seems
| similar. I wonder how these services compare?
|
| [1] https://pushover.net/
| evbogue wrote:
| Pushover looks very similar to Ntfy, but I haven't tried it
| yet.
| cja wrote:
| I actually use Pushover for a project but it just does the
| notification part and requires me to catch all errors and
| send them to Pushover. I'm looking for the hosting service
| to catch unhanded errors automatically
| kbenson wrote:
| I think the answer to that is the same as the answer to "how do
| I send an email in Deno?", since this lets you run arbitrary
| code.
|
| System level cron needs that feature built in because you're
| just telling it a command to run, and you're limited in how you
| deal with error and output handling of that command. That's not
| a problem when you're in the language you're using to define
| the action. Just catch your errors, or grab all STDOUT/STDERR
| with some solution and do with it what you want on success or
| error.
| fiddlerwoaroof wrote:
| The downside is that when it has to be done on a job-by-job
| basis, you can't rely on the cron system handling failure
| modes you didn't anticipate.
| kbenson wrote:
| It's code. There is not necessarily a single solution that
| makes sense. The thing you're calling could throw an
| exception, or it could return a failure value. If it fails,
| you may want to set off some extra routine that retries, or
| perhaps you want to handle some specific errors and retry
| or notify, but other errors are critical and cause failure,
| not just notification. Having it handle those automatically
| is not a feature in some cases, it's a problem.
|
| System cron is _simple_ , by design, because as a DSL
| there's a benefit to not complicating it. If you're already
| in Javascript/Typescript, a lot of the benefits of that are
| mitigated by the benefits of having much more control over
| exactly how it functions in every care.
|
| As an example of this, there's a bunch of cron "helpers" to
| deal with the shortcomings of cron's simplistic approach,
| such as those in moreutils[1].
|
| P.S. Personally I wouldn't have even called this
| implementation cron and use the cron syntax for it, since
| that just makes people assume cron usage and the cron
| scheduling format is not an asset if you're already in a
| language where you could just pass in a structure with the
| specific fields you want set by name.
|
| 1: https://rentes.github.io/unix/utilities/2015/07/27/moreu
| tils...
| obblekk wrote:
| How long does the job get to run?
|
| Would be really awesome if this could be used to offload long
| lived jobs from the request.
|
| Defer.run is building this for vercel and it's really cool.
| lucacasonato wrote:
| You can use Queues for this :)
|
| https://deno.com/blog/queues
| vorticalbox wrote:
| I wish this had a built in rate limit, I wanted to hit an api
| with lots of test data, I didn't need to actually wait for
| the response and I didn't really care how long they took.
|
| I just needed to limit how fast the http requests went.
|
| I guess I could do this with the delay and calculate how long
| it would need to wait.
| leroman wrote:
| Nice, years ago I wrote a TypeScript (originally CoffeeScript re-
| written to TypeScript) library to generate schedule times for a
| cron between two time points, should be easy to write a service
| like this with it.
|
| https://github.com/romansky/JsCron
| vvpan wrote:
| I am a lead on a small startup team and one of the biggest pain
| points is dealing with infrastructure. We have no dedicated
| devops person and much of that work falls on me and other people
| who would be better writing code. I think the cloud paradigm is
| experiencing a shift. Few of us want to deal with cloud
| infrastructure (whether clicking around or via Terraform or
| equivalent) to execute a function every X minutes - its
| unnecessary developer hours. The functionality is so common that
| it should just work. Perhaps stemming from that there seem to be
| two recent trends:
|
| 1. Services like this cron with Dyno Deploy or Vercel where the
| cloud things are abstracted away for you.
|
| 2. I have no data to back it up but it seems like JVM is
| experience a bit of a comeback with a few companies adopting
| Kotlin for backend and, also subjectively, talk about Elixir has
| increased here on Hacker News (we've all seen that table that
| shows how the Erlang VM is cron/background jobs/logging
| service/KV store all rolled into one). And if those are not the
| trends I feel like they should be.
|
| Both of those are very appealing to me as a team lead. Honestly,
| I have never been a fan of managing services in AWS/GCP/etc - the
| setup overhead rarely seemed to outweigh the pros, at least until
| some heavy amount of data start moving around. I might be stating
| the obvious - less work is definitely better, but pretty much
| every company I have consulted for in the last few years had a
| (crappy) bespoke cloud setup that was a massive time sink.
| mattgreenrocks wrote:
| For sure. As someone who has stayed away from this part of the
| biz, infrastructure always looked like makework to me.
| Necessary, mind you, but still makework that was indicative of
| poor/insufficient computing primitives for services.
|
| > it seems like JVM is experience a bit of a comeback with a
| few companies adopting Kotlin for backend
|
| JVM 21 adding virtual threads, and Spring Boot 3.2 using them
| with one line of config is huge. We can now write simple code
| that looks blocking and let the runtime handle it instead of
| writing async function/await everywhere. Personally I'm loving
| Spring Boot 3.2 with Kotlin, especially for the fact that I can
| bundle scheduled jobs, API, and frontend all in one place for
| my indie projects. Plus the JVM world of devs seems to have a
| somewhat decent appreciation of how to make web services that
| aren't rife with unnecessary coupling.
|
| The bad? There is a legendary amount of cargo-culting blog
| posts and questionable advice around Spring Boot, often for
| older versions.
| mike_d wrote:
| It is really disappointing to me when I hear takes like this.
| As an industry we are splitting the roles of programmers and
| engineers so that programmers can just throw spaghetti at the
| wall and it is the engineers problem to happily run it.
|
| We need more well rounded people that also fundamentally
| understand how their code is executing on the backend so
| performance and cost can be optimized.
| marcosdumay wrote:
| We need decently designed environments where running stuff is
| trivial. It's not practical to expect the same person to be
| able to deal with both all the bullshit that comes from
| making the spaghetti and all the bullshit that comes from
| making it adhere to the wall.
| everforward wrote:
| It is practical, a lot of us do it. Besides, the amount of
| bullshit it takes to make the spaghetti stick to the wall
| is dictated by how it was made.
|
| Make nice spaghetti and it sticks to the wall effortlessly.
| Make bad spaghetti and it takes 4 people and a dozen rolls
| of duct tape to get it up there.
| marcosdumay wrote:
| > the amount of bullshit it takes to make the spaghetti
| stick to the wall is dictated by how it was made
|
| I really don't know where the bullshit coming from. Often
| it looks like every single person is fighting for their
| lives against it, and yet it keeps growing.
|
| But it's mostly dictated by overarching decisions that
| constrain both how it runs and how it's made (AKA,
| outside somewhere). Of course, there exist people on both
| sides that create problems all by themselves, but those
| are always easy to solve.
| everforward wrote:
| > But it's mostly dictated by overarching decisions that
| constrain both how it runs and how it's made (AKA,
| outside somewhere).
|
| While annoying, there's often a good reason behind those
| if you can drill down to the engineers that asked for
| that rule.
|
| Just as an example, it's easy to provide a single HA
| solution at the infrastructure tier... provided the infra
| people can make a few assumptions about your app
| (stateless, can handle multiple copies running at once,
| etc). It's virtually impossible to build HA that makes no
| assumptions, and it's incredibly burdensome to support 85
| different HA implementations with varying sets of
| assumptions.
|
| It makes business sense to have infra do the HA instead
| of each developer team, and that means restrictions on
| how apps are built and run.
|
| What you're experiencing is the platform-ization of a
| bunch of aspects of the app.
|
| Making nice spaghetti usually consists of finding the
| rules, finding the people who made them, and then
| grabbing them and asking "what does it take for this app
| to be in prod 15 minutes after we push the last commit?".
| It sounds like it will be a lot of people because there's
| a lot of rules, but there's usually like a half-dozen
| teams that make all the rules.
|
| It will likely force you to build differently than you
| would have, but it will also mean that you're more likely
| to have 0 friction and you're more likely to get help
| with the friction you do encounter.
| eropple wrote:
| _> It 's not practical to expect the same person to be able
| to deal with both all the bullshit that comes from making
| the spaghetti and all the bullshit that comes from making
| it adhere to the wall._
|
| I don't think I agree. Complexity--many parts, opposes
| "simple"--is part of the job; I'm pretty comfortable
| reasoning at the product layer (I've done devrel and
| product management) and working down to the cloud or bare
| metal (sometimes you find yourself wielding strace and
| wondering what mistakes got you here). Granted, that I have
| a handle on that complexity is part of why I'm a
| staff/principal engineer: to do that, and to help others
| who haven't achieved that level of understanding yet.
|
| But by trying to remove complexity, IME you invariably
| introduce _complication_ --relative difficulty in
| comprehending a given part, opposes "uncomplicated"--and
| that's where the demons lie. You give "the spaghetti maker"
| a nice, perfectly spherical pasta extruder; the amount of
| complication introduced by attempting to insulate the
| person on that side of the house only introduces problems
| for the other, who in turn must introduce complication to
| maintain it. That itself in turn introduces new constraints
| to the guy who just wants to turn the crank on the
| spaghetti maker and now _his_ world reintroduces the
| complexity the initial perfectly-spherical-tool tried to
| remove.
|
| Better, instead, to embrace that there are multiple moving
| parts that _themselves are allowed to be uncomplicated_ ,
| and work from there. Some complication is inevitable and
| irreducible, but if you don't treat each part of your
| system as being on an island, you can manage it and put it
| where it has the least impact.
| jacobyoder wrote:
| > We need more well rounded people that also fundamentally
| understand how their code is executing on the backend so
| performance and cost can be optimized.
|
| There's also some sort of balance that needs to be struck. As
| a 'web developer'... to be 'well balanced', I need to
| understand (or dig in to) the minutia of:
|
| * JS build tools, syntax, oddities, versions, and be able to
| troubleshoot these in a variety of environment (works locally
| with these architectures, and runs on production, but the CI
| pipeline changed to accommodate someone else and now I have
| to unstuck all this).
|
| * SQL - I need to understand the implications and nuance of
| various indexing strategies because... hey, PG 14 changed and
| now our caching approach needs tuning. Should I be using
| b-tree indices, or something else? Should my indices be
| clustered? How do I manage replication and backups? And I
| need to understand the tradeoffs of ORMs vs hand-rolled, vs
| stored procedures, time-series databases, performance impact
| of views and materialization. And security.
|
| * Application level - I need to understand various aspects of
| security at the application layer, and balance security vs
| usability vs accessibility vs performance vs feedback from
| various program/product managers and end users. Oh, and I
| need to be able to troubleshoot the application in a variety
| of contexts. Oh... and mobile. I need to be conversant and
| 'well rounded' in various flavors of mobile development
| (hybrid vs native vs whatever).
|
| * System updates - what? I'm still on JDK 11? WTF? Why aren't
| you on the current versions? We need to update pronto,
| because there's too many security issues we'll get dinged on!
|
| * UI - I need to be a CSS master, and if I'm not already on
| the tailwind bandwagon, I risk losing my job. Or... at least,
| I'll fall behind as I try to incorporate the new tailwind UI
| work the new person did because they couldn't be bothered to
| learn bootstrap 5. And... accessibility - need to be an
| expert in that.
|
| * Deployment - hey, I now need to be able to understand
| multiple deployment approaches and processes, be fluent in
| docker/k8s and various tools, be able to troubleshoot these
| self-sufficiently. Doesn't matter if these are necessarily
| the right tools for the situation - someone else dictated
| this and lobbying for anything else means you're afraid to
| learn new things.
|
| And... I should be cheery and helpful and positive about all
| of this being _my_ responsibility. Because when I ask for
| folks from another team to help, I get told I need to 'own'
| the project, and I can't just 'throw it over the wall' and
| expect someone else to do something for me. If I push back on
| anything, I risk getting labelled "old" and "a dinosaur" and
| "afraid of change" and "unwilling to learn new things".
|
| tldr - if you actually have teams of people with related yet
| diverse skills, consider letting each play to their
| strengths, help support them in their strengths, and organize
| around everyone doing what they're best at. The 'well
| rounded' thing has its limits.
| franciscop wrote:
| My answer tends to be either I own the project and thus I'm
| able to deploy to Heroku/Vercel/etc, or you own the
| project, setup, tech and so I'm going to need support. It
| actually worked pretty nice in the past, but _probably_
| because I 'm really good at React while being just mediocre
| at DevOps, so I can convince managers that a couple of
| hours/month of help with DevOps is worth a lot more than me
| not doing frontend. (Actually, in one case it worked really
| bad for me).
| vvpan wrote:
| It does not matter if you are "well rounded". If you can
| schedule something with a line of code then that is
| objectively faster to develop and maintain than to (for GCP
| for example):
|
| - setup cloud cron job
|
| - setup pubsub channel
|
| - create pubsub subscription
|
| - add environment variables specifying the subscription
|
| - make all that work in terraform and run migrations
|
| And that process is different for every cloud provider.
| FuriouslyAdrift wrote:
| What you are descibing is an SRE... in the old Google
| definition.
| stillwithit wrote:
| Barnacle of the recent boom where cheap money meant every
| team and business were 2-3x as big as necessary, everyone got
| an assistant (even the assistants) and lattes all day
|
| Some projects seems to be pulling "ops" functionality into a
| new monolith that is also the languages native environment
| (deno, vercel)
|
| Still all Linux at the hardware layer and oodles of scripting
| language state, HTML/CSS. The web is a bloated mess after the
| "throw spaghetti at wall to try and gain market advantage,
| boost engagement" efforts of the last 10-12 years
|
| MBAs ran software engineering as if machines were squirting
| soup into cans.
| dyeje wrote:
| There's a new crop of PaaS products that aim to provide a layer
| of sanity on top of cloud providers (e.g. Aptible). Might be a
| good fit for you.
| klabb3 wrote:
| > 1. Services like this cron with Dyno Deploy or Vercel where
| the cloud things are abstracted away for you.
|
| I agree with the sentiment, but to me it's a shame this even
| qualifies as abstraction in the first place. Periodic execution
| of a function is extremely basic and already exists in almost
| every programming environment. If this needs to be a service
| (instead of a library call), there is something deeply wrong
| elsewhere, imo. We all laughed about leftpad, but at least that
| was native code and not "as-a-service", dependent on networking
| and proprietary code on someone else's machine.
|
| I have nothing against Deno in particular, and I'm also not
| saying this is easy from an implementation perspective (due to
| complexity elsewhere). I'm just surprised that trivial features
| - from an end-user perspective - justify such a ruckus.
| notpachet wrote:
| I wonder why more people don't just set up a cheap VM and run
| this sort of thing via crontab.
| Swizec wrote:
| Does crontab on a self-managed VM guarantee at least once
| delivery? For many, if not most, use-cases that guarantee
| is critical.
| notpachet wrote:
| No. But I don't think there's any environment out there
| that is going to 100% guarantee that your cron succeeds.
| Even if it is reenqueued on failure, it could just keep
| hitting the same problem and crashlooping. You still need
| some kind of failure reporting, and a human to jump in to
| fix whatever went wrong.
| fesc wrote:
| The same reason why most people stopped manually editing
| some random files via FTP to do deployments: to get a
| proper reproducible, automated and monitored production
| environment.
| codedokode wrote:
| Cloud setups are not reproducible, because you cannot
| reproduce the same env locally or in the other cloud.
| notpachet wrote:
| I think there's a threshold below which this is just
| unnecessary infrastructure overhead, and I'd posit that
| most cron use cases fall below this threshold. If yours
| is above it, well and good.
| turtlebits wrote:
| Because managing servers adds a lot of tech overhead (OS
| and security updates/access control/disk/HA/monitoring) +
| provisioning/config + deployment.
|
| That said, there will be a time when your SaaS costs more
| than managing infra.
| everforward wrote:
| > Periodic execution of a function is extremely basic and
| already exists in almost every programming environment.
|
| Periodic execution is surprisingly complicated. What happens
| when your node is down when it was supposed to execute and
| then comes back up? What happens around timezone changes and
| leap years? What does this thing do if a run takes so long
| that the previous run is still going? How do you get
| logs/notifications if it fails?
|
| "Best effort" periodic execution is super easy, where you try
| to make a thing happen every period but it's okay if it
| doesn't happen as long as it does happen at some point in the
| near future (i.e. periodic clean up jobs, or maybe batch
| emails).
|
| There's also a salient argument that you shouldn't run things
| inside the same process/VM if you can avoid it. It'd be a
| real shame if the production app went down because the
| cleanup cronjob went haywire and choked out the CPU.
| twic wrote:
| I am a big fan of avoiding service proliferation and cloud
| complexity, and just making really good use of the
| facilities provided by your language and OS of choice.
|
| But for exactly the reasons you outline, scheduling jobs is
| one of the few things where i would be eager to use some
| reliable, observable, carefully-engineered service.
| klabb3 wrote:
| > What happens when your node is down when it was supposed
| to execute and then comes back up?
|
| If you want at-least-once scheduling you'd probably use a
| library that writes a last-completed time to your db.
|
| > What happens around timezone changes and leap years?
|
| For wall-time scheduling you'd need to handle it, yes. Why
| would a service be better than a library though?
|
| > What does this thing do if a run takes so long that the
| previous run is still going?
|
| I think we'd need either another db entry or simply prevent
| overlap directly in the library.
|
| > How do you get logs/notifications if it fails?
|
| Same way as other things in your app that fails I guess.
| Perhaps logging a critical event?
|
| If you're talking about external health monitoring then
| yes, you'd need to have a separate service by definition,
| preferably with a completely different provider/host.
| AgentME wrote:
| Scheduled actions on a scalable deployment aren't straight
| forward, unless you just want all your live instances to run
| the scheduled actions at the same time in parallel. You'd
| need the instances to coordinate with each other to make sure
| they aren't redundantly running the same actions, and you
| might need to scale the number of instances based on the
| scheduled actions. (Consider the case where the service is
| scaled down to 0 live instances, or where all the instances
| are busy with load and you want more instances to come up to
| deal with the scheduled actions.) In this case, you want cron
| handled by something aware of all your instances and can
| scale them (or you want it running outside of your scalable
| deployment and have it hit your load balancer which is aware
| of all your instances and can scale them, etc).
| conradfr wrote:
| For cron jobs in Elixir you can use:
|
| - [0] Quantum, quite simple and supports cron syntax.
|
| - [1] Oban, which is a job runner, requires a DB, but also
| supports periodic jobs with a cron syntax.
|
| (probably others but that's the ones I've used)
|
| [0] https://github.com/fanduel/quantum-elixir
|
| [1] https://github.com/sorentwo/oban
| seqizz wrote:
| > one of the biggest pain points is dealing with infrastructure
|
| > We have no dedicated devops person
|
| I understand that not every startup can afford an extra person
| for infra-heavy tasks, but this should not be surprising. Any
| tool which looks like an abstraction to you might get complex
| by time, and steal your time. The cloud was supposed to fix
| some of these, look where we are now.
| RadiozRadioz wrote:
| Or you could just use an actual host running actual cron. No
| clicking, no terraform, just crontab -e. The last thing small
| short-staffed startups with no infra people should be doing is
| messing around setting up cloud infra to run something on a
| schedule.
| ketzo wrote:
| Okay, but who's taking care of the actual host? I know this
| is HN and we're all server admin wizards, but there is _some_
| effort and maintenance required.
|
| This is theoretically even easier than that. Yay for a
| spectrum of options!
| jokethrowaway wrote:
| That's an overblown problem, especially for a small
| business which is not going to have the infra problem of
| someone with scale problems.
|
| We've just been sold lies that pushed us to buy expensive
| clouds - and are not that much simpler than good old tools
| ketzo wrote:
| I agree that "it's _way_ too much effort to _ever_
| consider running your code on your own servers / EC2
| instances" is overblown, and has very much favored cloud
| providers.
|
| But for the _specific_ problem of "I wanna execute some
| code on a regular schedule, but I want it to be
| production-ready", Deno Cron does seem much quicker and
| lighter than provisioning a host, setting up health
| checks, setting up remote access controls for new devs,
| etc.
|
| There are obviously many people who would find it very
| easy to do all the "server-work" I'm describing. I am not
| one of them! This service looks awesome to me.
| vvpan wrote:
| It's not an option if you have more than one machine and you
| want the machines to be uniform. If they are not uniform
| means it's back to square one - bespoke setups. And then I
| want reproducibility across environments and environments
| that are easy to set up (for testing for example). Basically
| I think it's not an option.
| codedokode wrote:
| Why does a small startup need several machines though?
| MuffinFlavored wrote:
| > Honestly, I have never been a fan of managing services in
| AWS/GCP/etc - the setup overhead rarely seemed to outweigh the
| pros
|
| I just `ssh` into prod and `git pull` and either `docker
| compose up` or `kubectl apply` or `terraform apply` or `helm
| update` or `argocd sync`
| ketzo wrote:
| Okay, this is either really good satire, or you are wildly
| overestimating the average startup engineer's knowledge of
| each of those tools and their configurations
| bauble wrote:
| I don't understand the reluctance to hire a system
| administrator. If you have two or more technical folks, at
| least one of them should be a sysadmin (or "devops" if you
| prefer). Pretending that it's an unskilled role that anyone
| should pick up reflects a poor understanding of the industry.
| pech0rin wrote:
| The "crappy bespoke cloud setup" is usually the result of
| people whose jobs are usually just regular app devs, but had
| been thrown into the infrastructure side head first. I think
| the key is to actually have someone who knows what they are
| talking about.
|
| Unlike other people I don't think you need to hire someone
| directly. I do devops freelancing so you could hire someone
| like me. Or look online for various people that do it. I mean
| if its a toy app without many customers then who cares, but
| once you start scaling infrastructure gets to me more and more
| important.
| koolba wrote:
| > How exactly does Deno Deploy know there's a cron in your code,
| even when there's no web server handling requests?
|
| > When a new production deployment of your project is created, an
| ephemeral V8 isolate is used to evaluate your project's top-level
| scope and to discover any Deno.cron definitions. A global cron
| scheduler is then updated with your project's latest cron
| definitions, which includes updates to your existing crons, new
| crons, and deleted crons.
|
| Having cron jobs defined in the code itself just feels weird. I
| get the convenience of doing things "in one place", but it's such
| an orthogonal concept. But I suppose combining things is part of
| the allure of deno any way.
|
| Now that aside, the Flavor Flav dinosaur is the best thing I've
| seen all week.
| mgkimsal wrote:
| Where better to put cron work? In another repo? If the cron
| jobs are 100% unrelated to the codebase, yeah, it might be
| weird. If the jobs are "run the foo() method every 2am"...
| having that info colocated with the foo() method itself makes a
| lot of sense, imo.
| nightpool wrote:
| Just a normal yaml file...? Why go through the complexity of
| putting it in another codebase when you can just make a
| single config file?
|
| There are probably some benefits of using the Deno.cron
| syntax for some functions, and benefits of using a config
| file for others, but acting like the only other option is
| "put it in a completely seperate codebase" doesn't make any
| sense to me.
| rafram wrote:
| Or even a crontab! You know, the thing that this is named
| after.
| nightpool wrote:
| I think there are sensible reasons people don't like
| using unlabeled, whitespace-delimited formats that
| require an ascii-art diagram to explain in the modern
| day.
| everforward wrote:
| To me, it would depend on whether you always wanted to deploy
| the cronjobs with the codebase. I think I usually want the
| option to deploy them separately.
|
| Just as an example, if you have 8 nodes or whatever, your
| cronjob is going to get updated when the first one is
| deployed, not the last. The other nodes may or may not
| support that cronjob until they get the update.
|
| Going the other direction, if you have to roll back a single
| node for whatever reason, now you have conflicting cronjob
| versions.
|
| I typically want those rollouts to be separate so that I can
| handle failures in the app deployment before I update the
| cronjob.
| codedokode wrote:
| You can put ordinary crontab into your repo.
| jacobyoder wrote:
| That too - but keeping the items that deal with the code
| with the code seems simplest for most use cases.
| JLCarveth wrote:
| Have the team at Deno not heard of systemd timers?
| charcircuit wrote:
| I agree. "0 * * * *" to schedule something hourly is a poor for
| developer experience. systemd timers use a more human readable
| "hourly". "0 1 * * *" to run at 1:00 daily becomes "1:00".
| systemd timer calendar events read like actual timestamps.
| shepherdjerred wrote:
| I want to love Deno. It seems to really modernize TypeScript
| development. I just haven't had good experiences with it.
|
| I spent a day or two migrating a TypeScript project [-1] over to
| Deno from NodeJS. Here's what I had to do:
|
| * Change all of my local imports to something that would work
| with Deno. That meant appending `.ts` or `index.ts` for folders,
| except in some cases where Deno requires `.js`
|
| * Modify my monorepo to play nice with Deno -- an import map does
| this easily. Deno has documentation around import maps, but I had
| to figure out the solution myself. Also, import maps are
| currently broken in JetBrains IDEs.
|
| * Change my NPM imports to something that would work with Deno.
| The most straightforward thing to do was just change `import
| "foo"` to `import "npm:foo"`, but this felt hacky so eventually I
| used https://esm.sh, which worked for some packages but not
| others.
|
| * Figure out how to get Lodash working with TypeScript. It's not
| simple. [0]
|
| * Use a hack to get typeorm with sqlite working. [1]
|
| * Try out `deno compile`, only to determine that it somehow works
| differently than `deno run`. My project simply wouldn't work with
| `deno compile`, probably because my project has some system
| dependencies that don't get properly linked/included.
|
| * Setup my scripts to properly lint my project. Deno has
| formatting/linting built-in, but it's split across many commands
| with different usages. For example, I can run `deno fmt` to
| format my entire project, but I have to give Deno a path to run
| `check` or `lint`, e.g. `deno check src/index.ts `
|
| * Patch a third-party library that was setting an HTTP header to
| `null`. NodeJS handles this case just fine, but Deno throws an
| error [2].
|
| * Attempt to build my UI (astro + react) with Deno. It seems some
| people have gotten this partially working, but I gave up and
| stayed on NodeJS for building my UI. This led to me:
|
| * Using dnt [3] to build a Deno package for consumption with
| NodeJS/npm frontend. This was actually surprisingly simple; kudos
| to the author of the dnt library.
|
| After all of that work, I finally was able to use Deno in my
| project. It was really cool! Unfortunately, both VS Code and
| IntelliJ with Deno are essentially unusable [4]. Or, at least,
| unacceptably slow compared to what I had with NodeJS.
|
| [-1]: https://github.com/shepherdjerred/glitter-boys/tree/sj/deno
|
| [0]: https://stackoverflow.com/a/66073607
|
| [1]:
| https://github.com/typeorm/typeorm/issues/6123#issuecomment-...
|
| [2]: https://github.com/Sansossio/twisted/issues/97
|
| [3]: https://github.com/denoland/dnt
|
| [4]: https://github.com/denoland/vscode_deno/issues/895
| tengbretson wrote:
| This really seems to muddy the distinction between the data layer
| and the application layer. I'm sure they've put a lot of thought
| into this, but it seems like there could be some bizarre edge
| cases.
| manicennui wrote:
| Not using something more intuitive than cron's schedule format
| seems like a missed opportunity.
| igorzi wrote:
| This is being worked on:
| https://github.com/denoland/deno/issues/21122. Should be
| available with the next Deno release.
| phoe-krk wrote:
| The AI-generated imagery at https://deno.com/blog/cron/cover.png
| is actually painful the longer you look at it. The multiple watch
| hands might be an artistic effect, but the clock lacks the tenth
| hour and has two elevenths, the dinosaur's legs make absolutely
| no sense, and the water reflections show a different pattern on
| the dinosaur legs and a completely different set of clock hands
| than what is visible above.
|
| I know that displaying artwork is not the point of the article,
| but please, at least make it believable when you look at it for
| more than one second.
| handsaway wrote:
| It's hard to ignore when it takes up my entire screen and I
| have to scroll to see anything else. AI artwork is popping up
| everywhere and I don't think it looks as cool as people seem to
| think. It comes off incredibly tacky.
| cantSpellSober wrote:
| Perhaps bad AI art will go down as a tacky nostalgia, the
| "'under construction' gif" of their time
| ketzo wrote:
| I mean, it's a hero image above a technical article. What
| percentage of readers even _did_ look at it for more than one
| second?
|
| I thought it was awesome, even though it was pretty obviously
| AI-generated for the reasons you stated.
| FragenAntworten wrote:
| It's especially disappointing to see Deno doing this, because
| in the past they've used endearing art made by an artist,
| hashrock, which I liked very much.
| asim wrote:
| Wrote similar when I was working on an API platform. Used http
| callbacks. https://m3o.com/cron/api#Schedule
| seanw265 wrote:
| I like a lot of what Deno does, but I'm having some trouble
| understanding why this (along with Deno's KV and Queues offering)
| is part of the Deno runtime.
|
| The blog post touts this as a feature to reduce the amount of
| code to get up and running, but a library would accomplish the
| same thing. A first-party library could even directly integrate
| with Deno Deploy in the same way that this appears to.
|
| Why pollute the runtime with something unrelated?
| ramesh31 wrote:
| >Why pollute the runtime with something unrelated?
|
| Because money.
|
| They are venture funded and need to show some kind of path to
| profitability.
| syrusakbary wrote:
| I was pleasantly surprised to see this, but it seems that the JS
| context in each run of the cron script is cloned so cron jobs are
| actually run in a separate "process".
|
| Namely, I ran this example. let s = 0, r = 0;
| Deno.cron("Add second", "* * * * *", async () => {
| console.log(`Add second ${s++}`); });
| Deno.serve(async (_req) => { return new Response(`Seconds
| Running: ${s} / Number of requests: ${r++}`); })
|
| Example deployed here: https://weak-stoat-26.deno.dev
|
| So, effectively, it seems that Deno.Cron clones the JS context
| but any changes done to that context will not be reflected in the
| Deno.serve. And, if that's the case, why don't handle Cron jobs
| in a different flow rather than mixing them with Deno.serve? (I
| think it can lead to unintended side-effects)
|
| I'd love if someone from the Deno team could provide more context
| here, maybe I missed something?
| igorzi wrote:
| If you need to manipulate state - you should use Deno KV or
| some other persistent database. V8 isolates are created on-
| demand to handle Deno.cron and Deno.serve requests, potentially
| in different regions. You should not make assumptions that the
| same isolate will be used to handle all requests.
| its-summertime wrote:
| Overlapping as an option would be nice, if I'm using cron to pull
| data to append to a database, and the append fails, I can
| normally retry. However, without overlap, I can't sit there doing
| retries since that will block future cron runs.
|
| Timezones would be nice, "This helps avoid issues with time zones
| which observe daylight saving time." But it also causes issues
| with daylight saving time. I can't run something at 9 in the
| morning, I need to run it 1-2 hours before-hand, calculate the
| offset, and then queue a run 1-2 hours later. Or I can run 2
| crons and have one or the other die early. Point being, all
| solutions become very hacky very fast.
|
| It would be nice if an object was passed to the handler that had
| a targetDate for the ideal start time would be, so runs can be
| easily labeled and separated.
|
| Yes I could round the Date.now() to the last minute, however, if
| Deploy ever goes over the minute boundary, that's all kaput.
|
| In a similar vein: Systemd timers have some nice features,
| AccuracySec/RandomizedDelaySec/FixedRandomDelay, some options
| similar to that would be nice (of course, with minute resolution
| instead) (and of course, fixed delay can be pre-calculated, but
| it would be nice to just say 30m and have deno runtime manage
| that for me)
|
| https://www.freedesktop.org/software/systemd/man/latest/syst...
| phatboyslim wrote:
| Anyone here using Deno in a production workload? Genuinely
| curious about your experience.
| ilaksh wrote:
| Great, but do they really have to leave the cron schedule format
| as the only option for specifying? It seems like a great idea if
| it's 1975 and you are dealing with limitations of that time. And
| I think it's good to keep it as an option.
|
| But why not include any kind of schedule specifier that is a bit
| less cryptic and error prone? Is it really so much code?
|
| Maybe someone has a package already that can wrap this or output
| a schedule from a human readable description or something. Maybe
| something like `human-to-cron`.
| alexgarcia-xyz wrote:
| A JSON schedule format is coming soon:
| https://github.com/denoland/deno/pull/21340
| bsnnkv wrote:
| There is no doubt that this is a cool technical feat, but I don't
| think you'll catch me handling cron-like scheduling like this
| alongside application code.
|
| I can see from comments like vvpan's that there is a fatigue
| around managing services in AWS/GCP/etc, but I have gone very
| much in the other direction in the last few years towards bare
| metal servers managed with NixOS. I feel much more confident in
| handling scheduled jobs on bare metal with Systemd timers than
| tying myself even deeper into a specific language+deployment
| solution ecosystem.
___________________________________________________________________
(page generated 2023-11-29 23:01 UTC)