https://github.com/windmill-labs/windmill Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Case Studies + Customer Stories + Resources * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this organization All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} windmill-labs / windmill Public * Notifications * Fork 96 * Star 2.7k Open-source developer platform to turn scripts into workflows and UIs. Open-source alternative to Airplane and Retool. windmill.dev License Unknown and 2 other licenses found Licenses found Unknown LICENSE AGPL-3.0 LICENSE-AGPL Apache-2.0 LICENSE-APACHE 2.7k stars 96 forks Star Notifications * Code * Issues 19 * Pull requests 11 * Discussions * Actions * Projects 1 * Security * Insights More * Code * Issues * Pull requests * Discussions * Actions * Projects * Security * Insights windmill-labs/windmill This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 94 branches 170 tags Code * Local * Codespaces * Clone HTTPS GitHub CLI [https://github.com/w] Use Git or checkout with SVN using the web URL. [gh repo clone windmi] Work fast with our official CLI. Learn more about the CLI. * Open with GitHub Desktop * Download ZIP Sign In Required Please sign in to use Codespaces. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @rubenfiszel rubenfiszel and rubenfiszel chore(main): release 1.100.1 (#1563) ... c9a1966 May 12, 2023 chore(main): release 1.100.1 (#1563) * chore(main): release 1.100.1 * Apply automatic changes --------- Co-authored-by: rubenfiszel c9a1966 Git stats * 3,226 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .devcontainer feat(dev): setup devcontainer (#549) September 19, 2022 16:58 .github remove uffizzi May 11, 2023 22:42 .vscode flow builder args handling improvements January 28, 2023 23:22 backend chore(main): release 1.100.1 (#1563) May 12, 2023 23:52 benchmarks feat(backend): Redis based queue (#1324) April 11, 2023 12:24 cli chore(main): release 1.100.1 (#1563) May 12, 2023 23:52 deno-client fix(cli): fix cli folder sync May 8, 2023 21:13 docker fix openbb build May 9, 2023 13:57 frontend chore(main): release 1.100.1 (#1563) May 12, 2023 23:52 functions/api fix cloudflare preview April 27, 2023 12:19 go-client fix go-client with new openapi February 21, 2023 12:25 imgs edit main video February 21, 2023 01:28 lsp chore(main): release 1.100.1 (#1563) May 12, 2023 23:52 python-client chore(main): release 1.100.1 (#1563) May 12, 2023 23:52 typescript-client add initial typescript client May 7, 2023 01:17 .dockerignore first commit May 5, 2022 04:25 .env update readme April 24, 2023 01:35 .gitignore feat(frontend): introduce mysql as a script language (#982) December 4, 2022 19:56 CHANGELOG.md chore(main): release 1.100.1 (#1563) May 12, 2023 23:52 CLA.md transferring copyright from ruben to windmill labs July 30, 2022 14:09 Caddyfile update caddyfile April 8, 2023 14:00 Dockerfile chore(deps): bump node from 19-alpine to 20-alpine (#1459) April 24, 2023 21:12 LICENSE Update LICENSE March 1, 2023 09:53 LICENSE-AGPL first commit May 5, 2022 04:25 LICENSE-APACHE transferring copyright from ruben to windmill labs July 30, 2022 14:09 NOTICE transferring copyright from ruben to windmill labs July 30, 2022 14:09 README.md add OAUTH_JSON_AS_BASE64 May 11, 2023 23:03 depot.json Switch to Depot for Arm image build (#1104) January 15, 2023 17:55 docker-compose.yml update self-host April 8, 2023 13:52 init-db-as-superuser.sql feat(frontend): app splitpanes (#1248) March 2, 2023 01:30 openflow.openapi.yaml chore(main): release 1.100.1 (#1563) May 12, 2023 23:52 version.txt chore(main): release 1.100.1 (#1563) May 12, 2023 23:52 View code [ ] Windmill - Turn scripts into workflows and UIs that you can share and run at scale Main Concepts Show me some actual script code CLI Running scripts locally Stack Security Sandboxing Secrets, credentials and sensitive values Performance Architecture How to self-host Docker compose Kubernetes (k8s) and Helm charts Postgres without superuser Commercial license OAuth for self-hosting Resource types Environment Variables Run a local dev setup only Frontend Backend + Frontend Contributors Copyright README.md windmill.dev . Open-source developer infrastructure for internal tools. Self-hostable alternative to Airplane, Pipedream, Superblocks and a simplified Temporal with autogenerated UIsm and custom UIs to trigger workflows and scripts as internal apps. Scripts are turned into UIs and no-code modules, no-code modules can be composed into very rich flows, and script and flows can be triggered from internal UIs made with a low-code builder. The script languages supported are: Python, Typescript, Go, Bash, SQL. Docker Image CI Package version Discord Shield Try it - Docs - Discord - Hub - Contributor's guide Windmill - Turn scripts into workflows and UIs that you can share and run at scale Windmill is fully open-sourced (AGPLv3) and Windmill Labs offers dedicated instance and commercial support and licenses. Windmill Diagram main.mp4 * Windmill - Turn scripts into workflows and UIs that you can share and run at scale + Main Concepts + Show me some actual script code + CLI o Running scripts locally + Stack + Security o Sandboxing o Secrets, credentials and sensitive values + Performance + Architecture + How to self-host o Docker compose o Kubernetes (k8s) and Helm charts o Postgres without superuser o Commercial license o OAuth for self-hosting o Resource types + Environment Variables + Run a local dev setup o only Frontend o Backend + Frontend + Contributors + Copyright Main Concepts 1. Define a minimal and generic script in Python, Typescript, Go or Bash that solves a specific task. Here sending an email with SMTP. The code can be defined in the provided Web IDE or synchronized with your own github repo: Step 1 2. Your scripts parameters are automatically parsed and generate a frontend. Step 2 Step 3 3. Make it flow! You can chain your scripts or scripts made by the community shared on WindmillHub. Step 3 4. Build complex UI on top of your scripts and flows. Step 4 Scripts and flows can also be triggered by a cron schedule '_/5 _ * * *' or through webhooks. You can build your entire infra on top of Windmill! Show me some actual script code import * as wmill from "https://deno.land/x/windmill@v1.62.0/mod.ts"; //import any dependency from npm import cowsay from "npm:cowsay@1.5.0"; export async function main( a: number, // unions generate enums b: "my" | "enum", // default parameters prefill the field d = "default arg", // nested objects work c = { nested: "object" }, // permissioned and typed json db: wmill.Resource<"postgresql"> ) { const email = Deno.env.get("WM_EMAIL"); // variables are permissioned and by path let variable = await wmill.getVariable("f/company-folder/my_secret"); const lastTimeRun = await wmill.getState(); // logs are printed and always inspectable console.log(cowsay.say({ text: "hello " + email + " " + lastTimeRun })); await wmill.setState(Date.now()); // return is serialized as JSON return { foo: d, variable }; } CLI We have a powerful CLI to interact with the windmill platform and sync your scripts from local files, github repos and to run scripts and flows on the instance from local commands. See more details CLI Screencast Running scripts locally You can run your script locally easily, you simply need to pass the right environment variables for the wmill client library to fetch resources and variables from your instance if necessary. See more: https://docs.windmill.dev/docs/advanced/local_development/ Stack * Postgres as the database * backend in Rust with the following highly-available and horizontally scalable architecture: + stateless API backend + workers that pull jobs from a queue in Postgres (and later, Kafka or Redis. Upvote #173 if interested ) * frontend in Svelte * scripts executions are sandboxed using google's nsjail * javascript runtime is the deno_core rust library (which itself uses the rusty_v8 and hence V8 underneath) * typescript runtime is deno * python runtime is python3 * golang runtime is 1.19.1 Security Sandboxing Windmill uses nsjail on top of the deno sandboxing. It is production multi-tenant grade secure. Do not take our word for it, take fly.io's one Secrets, credentials and sensitive values There is one encryption key per workspace to encrypt the credentials and secrets stored in Windmill's K/V store. In addition, we strongly recommend that you encrypt the whole Postgres database. That is what we do at https://app.windmill.dev. Performance Once a job started, there is no overhead compared to running the same script on the node with its corresponding runner (Deno/Go/Python/ Bash). The added latency from a job being pulled from the queue, started, and then having its result sent back to the database is ~50ms. A typical lightweight deno job will take around 100ms total. Architecture [diagram] How to self-host We only provide docker-compose setup here. For more advanced setups, like compiling from source or using without a postgres super user, see documentation Docker compose curl https://raw.githubusercontent.com/windmill-labs/windmill/main/docker-compose.yml -o docker-compose.yml curl https://raw.githubusercontent.com/windmill-labs/windmill/main/Caddyfile -o Caddyfile curl https://raw.githubusercontent.com/windmill-labs/windmill/main/.env -o .env docker compose up -d --pull always Go to http://localhost et voila :) The default super-admin user is: admin@windmill.dev / changeme From there, you can follow the setup app and creat other users. Kubernetes (k8s) and Helm charts We publish helm charts at: https://github.com/windmill-labs/ windmill-helm-charts Postgres without superuser If you do not want, or cannot (for instance, in AWS Aurora or Cloud sql) use a postgres superuser, you can run ./init-db-as-superuser.sql to init the required users for windmill. Commercial license To self-host Windmill, you must respect the terms of the AGPLv3 license which you do not need to worry about for personal uses. For business uses, you should be fine if you do not re-expose it in any way Windmill to your users and are comfortable with AGPLv3. To re-expose any Windmill parts to your users as a feature of your product, or to build a feature on top of Windmill, to comply with AGPLv3 your product must be AGPLv3 or you must get a commercial license. Contact us at ruben@windmill.dev if you have any doubts. In addition, a commercial license grants you a dedicated engineer to transition your current infrastructure to Windmill, support with tight SLA, audit logs export features, SSO, unlimited users creation, advanced permission managing features such as groups and the ability to create more than one workspace. OAuth for self-hosting To get the same oauth integrations as Windmill Cloud, mount oauth.json with the following format: { "": { "id": "", "secret": "", "allowed_domains": ["windmill.dev"] //restrict a client OAuth login to some domains } } and mount it at /usr/src/app/oauth.json. The redirect url for the oauth clients is: /user/ login_callback/ The list of all possible "connect an app" oauth clients To add more "connect an app" OAuth clients to the Windmill project, read the Contributor's guide. We welcome contributions! You may also add your own custom OAuth2 IdP and OAuth2 Resource provider: { "": { "id": "", "secret": "", // To add a new OAuth2 IdP "login_config": { "auth_url": "", "token_url": "", "userinfo_url": "", "scopes": ["scope1", "scope2"], "extra_params": "" }, // To add a new OAuth2 Resource "connect_config": { "auth_url": "", "token_url": "", "scopes": ["scope1", "scope2"], "extra_params": "" } } } Resource types You will also want to import all the approved resource types from WindmillHub. A setup script will prompt you to have it being synced automatically everyday. Environment Variables Api Server Environment Variable name Default Description / Worker /All DATABASE_URL The Postgres database All url. DISABLE_NSJAIL true Disable Nsjail Worker Sandboxing SERVER_BIND_ADDR 0.0.0.0 IP Address on which to Server bind listening socket PORT 8000 Exposed port Server The number of worker per Worker instance NUM_WORKERS 3 (set to 1 on Eks to Worker have 1 pod = 1 worker, set to 0 for an API only instance) DISABLE_SERVER false Binary would operate as Worker a worker only instance The socket addr at which to expose METRICS_ADDR None Prometheus metrics at All the /metrics path. Set to "true" to expose it on port 8001 Output the logs in json JSON_FMT false format instead of All logfmt The base url that is BASE_URL http://localhost:8000 exposed publicly to Server access your instance The base url that is reachable by your workers to talk to the BASE_INTERNAL_URL http://localhost:8000 Servers. This help Worker avoiding going through the external load balancer for VPC-internal requests. The maximum time of execution of a script. TIMEOUT 300 When reached, the job Worker is failed as having timedout. The timeout after which a job is considered to be zombie if the worker ZOMBIE_JOB_TIMEOUT 30 did not send pings Server about processing the job (every server check for zombie jobs every 30s) If true then a zombie job is restarted RESTART_ZOMBIE_JOBS true (in-place with the same Server uuid and some logs), if false the zombie job is failed The number of ms to sleep in between the last check for new jobs in the DB. It is SLEEP_QUEUE 50 multiplied by Worker NUM_WORKERS such that in average, for one worker instance, there is one pull every SLEEP_QUEUE ms. The maximum number of MAX_LOG_SIZE 500000 characters a job can Worker emit (log + result) If Nsjail is enabled, DISABLE_NUSER false disable the nsjail's Worker clone_newuser setting Keep the job directory KEEP_JOB_DIR false after the job is done. Worker Useful for debugging. License key checked at LICENSE_KEY (EE only) None startup for the Worker Enterprise Edition of Windmill The S3 bucket to sync S3_CACHE_BUCKET (EE only) None the cache of the Worker workers to The rate at which to tar the cache of the TAR_CACHE_RATE (EE only) 100 workers. 100 means Worker every 100th job in average (uniformly randomly distributed). The signing secret of SLACK_SIGNING_SECRET None your Slack app. See Server Slack documentation The domain of the cookie. If not set, the COOKIE_DOMAIN None cookie will be set by Server the browser based on the full origin DENO_PATH /usr/bin/deno The path to the deno Worker binary. PYTHON_PATH /usr/local/bin/python3 The path to the python Worker binary. GO_PATH /usr/bin/go The path to the go Worker binary. The GOPRIVATE env GOPRIVATE variable to use private Worker go modules The netrc content to NETRC use a private go Worker registry PIP_INDEX_URL None The index url to pass Worker for pip. PIP_EXTRA_INDEX_URL None The extra index url to Worker pass to pip. PIP_TRUSTED_HOST None The trusted host to Worker pass to pip. The path environment PATH None variable, usually Worker inherited The home directory to HOME None use for Go and Bash , Worker usually inherited The max number of DATABASE_CONNECTIONS 50 (Server)/3 (Worker) connections in the All database connection pool A token that would let SUPERADMIN_SECRET None the caller act as a Server virtual superadmin superadmin@windmill.dev The number of seconds to wait before timeout TIMEOUT_WAIT_RESULT 20 on the Worker 'run_wait_result' endpoint The number of max jobs in the queue before rejecting immediately the request in QUEUE_LIMIT_WAIT_RESULT None 'run_wait_result' Worker endpoint. Takes precedence on the query arg. If none is specified, there are no limit. Custom DENO_AUTH_TOKENS DENO_AUTH_TOKENS None to pass to worker to Worker allow the use of private modules Override the flags passed to deno (default --allow-all) to tighten DENO_FLAGS None permissions. Minimum Worker permissions needed are "--allow-read=args.json --allow-write= result.json" Registry to use for NPM dependencies, set if you have a private NPM_CONFIG_REGISTRY None repository you need to Worker use instead of the default public NPM registry Specify dependencies that are installed PIP_LOCAL_DEPENDENCIES None locally and do not need to be solved nor installed again Specify python paths (separated by a :) to be appended to the PYTHONPATH of the ADDITIONAL_PYTHON_PATHS None python jobs. To be used Worker with PIP_LOCAL_DEPENDENCIES to use python codebases within Windmill Whitelist of headers INCLUDE_HEADERS None that are passed to jobs Server as args (separated by a comma) Whitelist of workspaces WHITELIST_WORKSPACES None this worker takes job Worker from Blacklist of workspaces BLACKLIST_WORKSPACES None this worker takes job Worker from Webhook to notify of events such as new user INSTANCE_EVENTS_WEBHOOK None added, signup/invite. Can hook back to windmill to send emails (Enterprise Edition only) Interval in seconds in between bucket sync of the GLOBAL_CACHE_INTERVAL 10*60 cache. This interval * Worker 2 is the time at which you're guaranteed all the worker's caches are synced together. The worker groups WORKER_TAGS 'deno,go,python3,bash,flow,hub,dependency' assigned to that Worker workers CUSTOM_TAGS None The custom tags Server assignable to scripts. The time in seconds JOB_RETENTION_SECS 606024*60 //60 days after which jobs get deleted. Set to 0 or -1 to never delete The time in between polling for the WAIT_RESULT_FAST_POLL_INTERVAL_MS 50 run_wait_result Server endpoints in fast poll mode The time in between polling for the WAIT_RESULT_SLOW_POLL_INTERVAL_MS 200 run_wait_result Server endpoints in fast poll mode The duration of fast WAIT_RESULT_FAST_POLL_DURATION_SECS 2 poll mode before Server switching to slow poll Exit worker if no job EXIT_AFTER_NO_JOB_FOR_SECS None is received after Worker duration in secs if defined Base64 encoded JSON of the OAuth OAUTH_JSON_AS_BASE64 None configuration. e.g Server OAUTH_JSON_AS_BASE64=$ (base64 oauth.json | tr -d '\n') to encode it Run a local dev setup only Frontend This will use the backend of https://app.windmill.dev but your own frontend with hot-code reloading. 1. Install caddy 2. Go to frontend/: 1. npm install, npm run generate-backend-client then npm run dev 2. In another shell sudo caddy run --config CaddyfileRemote 3. Et voila, windmill should be available at http://localhost/ Backend + Frontend See the ./frontend/README_DEV.md file for all running options. 1. Create a Postgres Database for Windmill and create an admin role inside your Postgres setup. The easiest way to get a working postgres is running cargo install sqlx-cli && sqlx migrate run. This will also avoid compile time issue with sqlx's query! macro 2. Install nsjail and have it accessible in your PATH 3. Install deno and python3, have the bins at /usr/bin/deno and /usr /local/bin/python3 4. Install caddy 5. Install the lld linker 6. Go to frontend/: 1. npm install, npm run generate-backend-client then npm run dev 2. In another shell npm run build otherwise the backend will not find the frontend/build folder and will crash 3. In another shell sudo caddy run --config Caddyfile 7. Go to backend/: DATABASE_URL= RUST_LOG=info cargo run 8. Et voila, windmill should be available at http://localhost/ Contributors [6874747073] Copyright Windmill Labs, Inc 2023 About Open-source developer platform to turn scripts into workflows and UIs. Open-source alternative to Airplane and Retool. windmill.dev Topics python open-source platform typescript postgresql low-code self-hostable Resources Readme License Unknown and 2 other licenses found Licenses found Unknown LICENSE AGPL-3.0 LICENSE-AGPL Apache-2.0 LICENSE-APACHE Stars 2.7k stars Watchers 15 watching Forks 96 forks Report repository Releases 170 v1.100.1 Latest May 12, 2023 + 169 releases Packages 3 Used by 14 * @miracle0519 * @abrapartners * @VikingDadMedic * @HarshCasper * @Spread0x * @gabfl * @tmvanetten * @jasim + 6 Contributors 31 * @rubenfiszel * @dependabot[bot] * @fatonramadani * @adam-kov * @HurricanKai * @sqwishy * @mrl5 * @github-actions[bot] * @lplit * @jaller94 * @ryanrich + 20 contributors Languages * JavaScript 75.0% * Svelte 13.4% * Rust 7.4% * TypeScript 4.0% * Python 0.1% * Dockerfile 0.1% Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.