[HN Gopher] Show HN: Workflow Use - Deterministic, self-healing ...
___________________________________________________________________
Show HN: Workflow Use - Deterministic, self-healing browser
automation (RPA 2.0)
Hey HN - Gregor & Magnus here again. A few months ago, we launched
Browser Use (https://news.ycombinator.com/item?id=43173378), which
let LLMs perform tasks in the browser using natural language
prompts. It was great for one-off tasks like booking flights or
finding products--but we soon realized enterprises have somewhat
different needs: They typically have one workflow with dynamic
variables (e.g., filling out a form and downloading a PDF) that
they want to reliably run a million times without breaking. Pure
LLM agents were slow, expensive, and unpredictable for these high-
frequency tasks. So we just started working on Workflow Use: -
You show the browser what to do (by manually recording steps; show
don't tell). - An LLM converts these recordings into deterministic
scripts with variables (scripts include AI steps as well, where
it's 100% agentic) - Scripts run reliably, 10x faster, and ~90%
cheaper than Browser Use. - If a step breaks, workflow will
fallback to Browser Use and agentically run the step. (This self-
healing functionality is still very early.) This project just
kicked off, so lots of things will break, it's definitely not
production-ready yet, and plenty of stuff is still missing (like a
solid editor and proper self-healing). But we wanted to share
early, get feedback, and figure out what workflows you'd want to
automate this way. Try it out and let us know what you think!
Author : gregpr07
Score : 40 points
Date : 2025-05-16 16:05 UTC (6 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| pzullo wrote:
| Cool stuff!
| cdolan wrote:
| This is amazing. We've been using BrowserUser to try and create
| deterministic playwright scripts for months with mixed results.
|
| So, so, so excited to see this
| deepdarkforest wrote:
| Very cool. 1) How do you deal with timings? If a step includes
| clicking on a link or something that needs loading, then if you
| just fire off the generated playwright code at at once, some
| steps might fail because the xpath is not there yet. So to be
| safe, i'm guessing you would have to wait using the difference in
| the timestamps in the json. 2. For self healing, we worked on
| something similar, and we found it's very easy to get off the
| rails if one step fails because if your assertions of if the fix
| was correct are off, then the next and next steps will also fail
| etc. The most stable way was to just regenerate _all_ steps if a
| step fails in 2 attempts (2 runs of the flow) consecutively. If
| the xpath is broken for a step, very likely the subsequent ones
| won 't be worth healing individually.
| gregpr07 wrote:
| 1) we made this sick function in browser use library which
| analyses when there are no more requests going through - so we
| just reuse that!
|
| 2) yeah good question. The end goal is to completely regenerate
| the flow if it breaks (let browser use explore the "new"
| website and update the original flow). But let's see, soo much
| could be done here!
|
| What did you work on btw?
| deepdarkforest wrote:
| 1. Oh yes right. I remember trying it out thinking it was
| going to be brittle because of analytics etc but it filters
| for those surprisingly well.
|
| 2. We are working on https://www.launchskylight.com/ ,
| agentic QA. For the self onboarding version we are using pure
| CUA without caching. (We wanted to avoid playwright to make
| it more flexible for canvas+iframe based apps,where we found
| HTML based approaches like browser-use limited, and to
| support desktop apps in the future).
|
| We are betaing caching internally for customers, and
| releasing it for the self-onboarding soon. We use CUA actions
| for caching instead of playwright. Caching with pixel native
| models is def a bit more brittle for clicks and we focus on
| purely vision based analysis to decide to proceed or not. I
| think for scaling though you are 100% right, screenshots
| every step for validating are okay/worth it, but running an
| agent non-deterministically for actions is def an overkill
| for enterprise, that was what we found as well.
|
| Geminis video understanding is also an interesting way to
| analyze what went wrong in more interactive apps. Apart from
| that i think we share quite a bit of the core thinking, would
| be interested to chat, will DM!
| crazymoka wrote:
| So I can use this and it will pull new data from a database I can
| use to have it fill out a form? And I can trigger it when I need
| it to run with the updated form information?
| petethomas wrote:
| It's not mentioned here but Kapwork contributed the beginnings of
| this work in a PR a couple weeks ago: https://github.com/browser-
| use/browser-use/pull/1437. Thank you Gregor & Magnus for the
| great tech and all you're doing for the community.
| rammy1234 wrote:
| seems similar to selenium plugin for firefox, minus the scripting
| it generates.
| vasusen wrote:
| Very cool evolution!
|
| Really great to see the fallback to the agentic run when the
| automation breaks. For our e2e testing browser automation at
| Donobu, we independently arrived at the same pattern and have
| been impressed with how well it works. Automatic self-healed PR
| example here: https://github.com/donobu-inc/playwright-
| flows/pull/6/files
|
| edit: typo
| ProofHouse wrote:
| Can utilizing Chrome Extensions be added? It seems no one has and
| would be a critical bridge to many browser tasks
| nico wrote:
| Yes. Also, would love something that can run directly in my
| browser with my sessions
|
| There's a lot of websites that are super hostile to automation
| and make it really hard to do simple, small, but repetitive
| stuff with things like playwright, selenium, chromedriver
___________________________________________________________________
(page generated 2025-05-16 23:00 UTC)