https://github.com/harshadmanglani/polaris Skip to content Toggle navigation Sign in * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} harshadmanglani / polaris Public * Notifications * Fork 1 * Star 35 * High performance workflow orchestrator for Golang 35 stars 1 fork Branches Tags Activity Star Notifications * Code * Issues 0 * Pull requests 0 * Actions * Projects 0 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Actions * Projects * Security * Insights harshadmanglani/polaris This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main BranchesTags Go to file Code Folders and files Name Name Last commit Last commit message date Latest commit History 10 Commits .gitignore .gitignore README.md README.md datastore.go datastore.go executor.go executor.go executor_test.go executor_test.go go.mod go.mod go.sum go.sum metadatamanager.go metadatamanager.go models.go models.go registry.go registry.go registry_test.go registry_test.go test_utils.go test_utils.go View all files Repository files navigation * README [Polaris] An extremely light weight workflow orchestrator for Golang. Inspired from https://github.com/flipkart-incubator/ databuilderframework Getting Started From the root of your Go module, run: go get github.com/harshadmanglani/polaris@v1.0.0 Usage // dataStore = DataStore{} - use your database by implementing the IDataStore interface polaris.InitRegistry(dataStore) polaris.RegisterWorkflow(workflowKey, workflow) executor := polaris.Executor{ Before: func(builder reflect.Type, delta []IData) { fmt.Printf("Builder %s is about to be run with new data %v\n", builder, delta) } After: func(builder reflect.Type, produced IData) { fmt.Printf("Builder %s produced %s\n", builder, produced) } } response, err := executor.Run(workflowKey, workflowId, dataDelta) Use cases 1. You have multi-step workflow executions where each step is dependent on data generated from previous steps. 2. Executions can span one request scope or multiple scopes. 3. Your systems works with reusable components that can be combined in different ways to generate different end-results. 4. Your workflows can pause, resume or even restart from the beginning. Limitations 1. Workflow versioning is tricky to implement: 1. Unless you can afford a 100% downtime ensuring all active workflows move into a terminal state, deploying new code requires ensuring backward compatibility. 2. What this means is - you'll need to a deploy a version of code that is backward compatible for older non terminal workflows while newer ones will execute on the new code. 3. Once the older workflows have completed, a deployment to clean up stale code will be required. 2. The level of abstraction is lower in this framework compared to Cadence, Conductor: 1. Workflows can be made fault oblivious if there is an external (reliable) service giving callbacks per workflow id. 2. Instrumentation can be set up by adding your custom code to push events via listeners. Terminologies * Data - The basic container for information generated by an actor in the system. * Builder - An actor that consumes a bunch of data and produces another data. It has the following meta associated with it: + Name - Name of the builder + Consumes - A set of Data that the builder consumes + Produces - Data that the builder produces + Optionals - Data that the builder can optionally consume; one possible use case for this: if a builder wants to be re-run on demand with the same set of consumable Data already present, add an optional Data in the Builder and restart the workflow by passing an instance of the optional Data in the DataDelta + Access - Data that the builder will just access and has no effect on building the topologically sorted ExecutionGraph * Workflow - A specification and container for a topology of connected Builders that generate a final data. It has the following meta: + Name - Name of the workflow + Target Data - The name of the data being generated by this data flow + Resolution Specs - If multiple builders known to the system can produce the same data, then this can be used to put an override specifying which particular builder will generate a particular data in context of this data flow. + Transients - A set of names of Data that would be considered Transients for this case. (See later for a detailed explanation of transients) * DependencyHierarchy - A graph of connected and topologically sorted builders that are used by the execution engine to execute a flow. * DataSet - A set of the Data provided by the client and generated internally by the different builders of a particular ExecutionGraph How does the framework perform at scale? The framework itself has extremely low overhead. Since execution graphs are generated pre-runtime, all the orchestrator will do at runtime is use the graph and available data to run whichever builders can be run. Benchmarking pending. About High performance workflow orchestrator for Golang Topics go golang workflow workflow-engine orchestrator workflow-orchestration workflow-orchestrator Resources Readme Activity Stars 35 stars Watchers 2 watching Forks 1 fork Report repository Releases 1 tags Packages 0 No packages published Languages * Go 100.0% Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.