[HN Gopher] The Workflow Pattern
       ___________________________________________________________________
        
       The Workflow Pattern
        
       Author : kiyanwang
       Score  : 61 points
       Date   : 2023-10-05 08:17 UTC (1 days ago)
        
 (HTM) web link (blog.bittacklr.be)
 (TXT) w3m dump (blog.bittacklr.be)
        
       | jdeaton wrote:
       | One of the biggest issues I've observed with workflow systems is
       | an inability to get return values from each subprocess and then
       | decide based on those which subprocesses to instantiate next. It
       | always seems like the workflow execution graph is static in my
       | experience. Is that correct?
        
         | noselasd wrote:
         | The couple workflow engines I've used supports that. It sounds
         | crazy to not support something that obvious
        
         | sisve wrote:
         | Maybe before, but i feel that is not a problem anymore.
         | Windmill.dev solves this with easee. Very good UI on top of the
         | workflow engine.
        
         | brad0 wrote:
         | That sounds very similar to AWS Step Functions to me. You
         | define a workflow graph in JSON, then run the workflow and hand
         | out subprocesses to lambdas etc.
        
       | cannonpalms wrote:
       | I built a feature of our platform on top of Apache NiFi[1] and
       | had a great experience. I would have preferred a non-JVM
       | language, as that'd have been easier for us in a containerized
       | environment, but I have no complaints about NiFi otherwise.
        
         | claytonjy wrote:
         | Interesting; for me NiFi is tied with Airflow as my most-hated
         | software I've had to use professionally.
         | 
         | This was a few years ago but NiFi only supported editing the
         | DAG from a GUI; you could store the resulting XML in git, but
         | no human could read or edit it. Python steps were stuck on
         | Python 2.6 or 2.7. Hard to debug pileups. Is it better now?
        
       | jusonchan81 wrote:
       | One of my favorite workflow engines that has a really simple way
       | to do things was not listed here, so I'll call it out - Netflix
       | Conductor (https://github.com/Netflix/conductor).
       | 
       | Its capabilities comes to light when you model really complex
       | workflows and one real value is how its all very visual not just
       | during modeling but when running it. The history remains visible
       | and you can even see how the whole flow evolved.
        
         | pid-1 wrote:
         | My company has 200+ DAGs on Airflow, so that ship has sailed
         | for me.
         | 
         | That said, Conductor looks really cool, thanks for sharing.
        
       ___________________________________________________________________
       (page generated 2023-10-06 23:00 UTC)