[HN Gopher] Temporal .NET - Deterministic Workflow Authoring in ...
___________________________________________________________________
Temporal .NET - Deterministic Workflow Authoring in .NET
Author : kodablah
Score : 103 points
Date : 2023-05-03 16:18 UTC (6 hours ago)
(HTM) web link (temporal.io)
(TXT) w3m dump (temporal.io)
| swyx wrote:
| congrats Chad! great to see you ship this. I wonder if anyone
| familiar with Azure Durable Functions would be willing to give
| this SDK a side by side comparison!
| foota wrote:
| One must imagine Sisyphus happy.
| 0xcoffee wrote:
| I didn't really understand the part about 'What Problem Does
| 'Ref' Solve?'
|
| What would be the downside of writing it such as:
|
| `ExecuteActivityAsync<OneClickBuyWorkflow>(e =>
| e.DoPurchaseAsync, etc...)`?
| kodablah wrote:
| That's basically what is happening here, except you create `e`
| ahead of time instead of a do-anything-you-want lambda. But
| they essentially the same thing, though I think
| `ExecuteActivityAsync(MyActivities.Ref.DoPurchaseAsync,
| etc...)` is clearer that you are referencing an activity. And
| since it's just a delegate that an attribute is on, you can
| write a helper to do what you have instead.
| Merad wrote:
| If you're worried about someone doing something other than
| calling a method in the lambda it's pretty straightforward to
| use an Expression<Func<T, TResult>> and validate the contents
| of the expression. Seems like that would make for a much
| better dev experience than this static Ref property pattern
| that you don't see anywhere else in C#.
| kodablah wrote:
| It's not just that worry, it's the other reasons on top of
| that detailed in the post. But note that GP's post doesn't
| _call_ the method, it just references it which is basically
| what we are doing in the post. But for things like Durable
| Entities which _do_ call the method, yes we can prevent
| multi-use of the lambda arg but there are other problems
| (forcing interface/virtual, can't allow other args, sync vs
| async, etc).
| Merad wrote:
| I'm not really arguing that your pattern _doesn't work_,
| I just think you've created an unnecessary new pattern to
| solve an already solved problem. Using expressions with
| lambdas is pretty standard in C# when you need to
| reference a method or property without calling it
| immediately. Entity Framework is an obvious example,
| mocking libraries like Moq, and at least one that I know
| of (Hangfire) uses expressions to serialize info about a
| method so that it can be invoked later, possibly on a
| different machine or even in a totally different app. All
| of the things that you mention can be validated in an
| expression, if you feel the need.
| kodablah wrote:
| Thanks! Other commenter has also convinced me to
| investigate expression approach. I was hoping not to
| force people to create lambdas for each thing they want
| to invoke from a code readability POV, but it sounds like
| the ecosystem wants to force that.
| 0xcoffee wrote:
| The main reason I'm curious is because they write
|
| 'We solve this problem by allowing users to create instances
| of the class/interface without invoking anything on it. For
| classes this is done via
| FormatterServices.GetUninitializedObject and for interfaces
| this is done via Castle Dynamic Proxy. This lets us
| "reference" (hence the name Ref) methods on the objects
| without actually instantiating them with side effects. Method
| calls should never be made on these objects (and most
| wouldn't work anyways).'
|
| Which sounds like a lot of heavy lifting. It seems something
| like public class WorkflowBuilder<T> where
| T : class { // Somehow workflow gets the
| real instance private T Instance;
| public async Task<TResult>
| ExecuteActivityAsync<TResult>(Func<T, Task<TResult>> func) =>
| await func(Instance); public async
| Task<TResult>
| ExecuteActivityAsync<TResult>(Func<Task<TResult>> func) =>
| await func(); public async Task
| ExecuteActivityAsync(Func<Task> func) => await func();
| public async Task ExecuteActivityAsync(Func<T, Task> func) =>
| await func(Instance); } public
| record Purchase(string ItemID, string UserID);
| public class PurchaseActivities { public
| static WorkflowBuilder<PurchaseActivities>
| OneClickBuyWorkflow => new
| WorkflowBuilder<PurchaseActivities>();
| public async Task DoPurchaseAsync(Purchase purchase)
| { await
| OneClickBuyWorkflow.ExecuteActivityAsync(e =>
| e.DoPurchaseAsync(purchase)); }
| public static async Task DoPurchaseAsyncStatic(Purchase
| purchase) { await
| OneClickBuyWorkflow.ExecuteActivityAsync(() =>
| DoPurchaseAsyncStatic(purchase)); } }
|
| Would pretty much achieve the same thing.
| kodablah wrote:
| > Which sounds like a lot of heavy lifting
|
| How is `e` created for the `e =>
| e.DoPurchaseAsync(purchase)` lambda? You're going to have
| to do that lifting anyways to create an instance for `e`
| that isn't really a usable instance. Unless you use source
| generators which we plan on doing.
|
| I think what you have there is a lot more heavy lifting.
| Also note that workflows and activities are unrelated to
| each other. Workflow can invoke any activities. The code
| you have is a bit confusing because `PurchaseActivities`
| should be completely unrelated to workflows.
| 0xcoffee wrote:
| >You're going to have to do that lifting anyways to
| create an instance for `e` that isn't really a usable
| instance.
|
| But this should never happen, these is no reason to
| create an unusable instance. The real instance should be
| resolved in the same way however the current workflow
| resolves it.
| kodablah wrote:
| Activities may actually run on completely different
| systems than where the workflow calls execute on. All
| "execute activity" is from a workflow perspective is
| telling Temporal server to execute an activity with a
| certain string name and serializable arguments.
| Everything else is sugar. So if you're gonna use a type
| caller-side to refer to the name and argument types, you
| can't instantiate it fully (its constructor may have side
| effects for when it really runs).
|
| You can jump through a bunch of hoops like requiring
| interfaces which is what some frameworks do. But in our
| case, we just decided to make it easy to reference the
| method without invoking it or its instance.
| 0xcoffee wrote:
| Lamba/Func/Expressions are doing exactly this in C#,
| there is no instantiation required. Creating a unusable
| Ref object is jumping through hoops.
|
| You can parse an expression to serialize it and run it on
| a different server etc
|
| See e.g. https://github.com/6bee/Remote.Linq
| kodablah wrote:
| Hrmm, I will investigate using an expression tree for
| this (now's the time while it is alpha). I was hoping to
| avoid people having to create lambdas. I hope I don't run
| into overload ambiguity with the existing
| `ExecuteActivity` calls where you can just pass an
| existing method as Func<T, TResult> param. I will
| investigate this approach, thanks!
|
| Of course the "Ref" pattern is user-choice/suggested-
| pattern, it's not a requirement in any of our calls that
| just take simple delegates however you can create those
| delegates. So I may be able to work it in there.
| progmetaldev wrote:
| If it's possible to use both approaches without
| cluttering your API, that may be the best solution. I
| know a lot of more junior devs that have a difficult time
| wrapping their head around expressions in C#. Excellent
| library!
| jcparkyn wrote:
| As another point of reference, Hangfire also uses
| expressions for a similar use-case and it seems to work
| quite well.
|
| E.g. https://docs.hangfire.io/en/latest/background-
| methods/passin...
| BackgroundJob.Enqueue<EmailSender>(x => x.Send(13,
| "Hello!"));
| borissk wrote:
| Wonder how does it compare to https://github.com/UiPath/CoreWF
| kodablah wrote:
| Hrmm, I admit not knowing a lot about Windows Workflow
| Foundation, but from a quick glance I suspect they both have
| roots in the same place - deterministic workflow authoring that
| invokes activities like Amazon SWF and Azure Durable Task
| Framework. So they are probably quite similar, yet CoreWF seems
| to prefer YAML or limited code-based DSL whereas Temporal
| prefers to provide the full language with determinism
| constraints.
| time0ut wrote:
| Temporal has been amazing to work with so far. We've been looking
| forward to the .NET SDK for a while now so we can maintain parity
| with our Java and Node ecosystems.
| lorendsr wrote:
| For those not familiar with workflows as code, a workflow is a
| method that is executed in a way that can't fail--each step the
| program takes is persisted, so that if execution is interrupted
| (the process crashes or machine loses power), execution will be
| continued on a new machine, from the same step, with all
| local/instance variables, threads, and the call stack intact. It
| also transparently retries network requests that fail.
|
| So it's great for any code that you want to ensure reliably runs,
| but having methods that can't fail also opens up new
| possibilities, like you can:
|
| - Write a method that implements a subscription, charging a card
| and sleeping for 30 days in a loop. The `await
| Workflow.DelayAsync(TimeSpan.FromDays(30))` is transparently
| translated into a persisted timer that will continue executing
| the method when it goes off, and in the meantime doesn't consume
| resources beyond the timer record in the database.
|
| - Store data in variables instead of a database, because you can
| trust that the variables will be accurate for the duration of the
| method execution, and execution spans server restarts!
|
| - Write methods that last indefinitely and model an entity, like
| a customer, that maintains their loyalty program points in an
| instance variable. (Workflows can receive RPCs called Signals and
| Queries for sending data to the method ("User just made a
| purchase for $30, so please add 300 loyalty points") and getting
| data out from the method ("What's the user's points total?").
|
| - Write a saga that maintains consistency across services / data
| stores without manually setting up choreography or orchestration,
| with a simple try/catch statement. (A workflow method is like
| automatic orchestration.)
| mirsadm wrote:
| It is a replica of Amazon Simple Workflow as far as I can see.
| Which is not a bad thing, SWF is great and does not get much
| attention.
| kodablah wrote:
| That's no coincidence, Temporal is founded by the creators of
| Amazon Simple Workflow. See https://temporal.io/about.
| euroclydon wrote:
| Seems like this would depend on some storage guarantees, but I
| can't find anything about that
| lorendsr wrote:
| Correct, the workflow's guarantee to always complete
| executing independent of hardware failures is dependent on
| the database not losing data. You host your workflow code
| with Temporal's Worker library, which talks to an instance of
| the Temporal Server [1], which is an open-source set of
| services (hosted by you or by Temporal Cloud), backed by
| Cassandra, MySQL, or Postgres. [2] So for instance increasing
| Cassandra's replication factor increases your resilience to
| disk failure.
|
| [1] https://github.com/temporalio/temporal
|
| [2] https://docs.temporal.io/clusters#persistence
| pjc50 wrote:
| Ah, so that's what the technique is called. I've seen a similar
| approach used for point of sale systems that basically
| persisted on every button press, so if one crashed you could
| bring up exactly the same state on a different one simply by
| logging in.
| lukevp wrote:
| What happens if you have a workflow in progress that you want
| to change the implementation of? Eg. If I had a workflow that
| was waiting for 30 days but then decided I wanted that interval
| to be 7 days instead?
| lorendsr wrote:
| After you deploy the new workflow code, you can reset [1] a
| workflow's execution state to before the DelayAsync statement
| was called, and then the workflow will sleep for 7 days from
| now.
|
| That doesn't take into account the time it's already been
| waiting. For when you want to do something on a schedule and
| be able to edit the schedule in future, there's a Schedules
| feature that allows you to periodically start a workflow,
| like a cron but more flexible. [2] In this case, the workflow
| code would be simpler, like just ChargeCustomer() and
| SendEmailNotification().
|
| [1] https://docs.temporal.io/cli/workflow#reset
|
| [2] https://docs.temporal.io/workflows#schedule
| kodablah wrote:
| Temporal determines a workflow is non-deterministic if upon
| code replay the high-level commands don't match up with what
| happened during the original code execution. In this case,
| technically it's safe to change the code this way because
| both still result timer commands. But what the timer was
| first created with on existing runs is what applies (there
| are reset approaches though as mentioned in other comment).
| However if, say, you changed the implementation do start a
| child workflow or activity before the timer, the commands
| would mismatch and you'd get a non-determinism error upon
| replay.
|
| There are multiple strategies to update live workflows, see
| https://community.temporal.io/t/workflow-versioning-
| strategi.... Most often for small changes, people would use
| the `Workflow.Patched` API.
| xupybd wrote:
| Temporal looks great but it takes a long time to find out what it
| is. The website has a lot of information but the navigation is
| not good.
| omgbobbyg wrote:
| [dead]
___________________________________________________________________
(page generated 2023-05-03 23:00 UTC)