[HN Gopher] Show HN: PageAgent, A GUI agent that lives inside yo...
___________________________________________________________________
Show HN: PageAgent, A GUI agent that lives inside your web app
Title: Show HN: PageAgent, A GUI agent that lives inside your web
app Hi HN, I'm building PageAgent, an open-source (MIT) library
that embeds an AI agent directly into your frontend. I built this
because I believe there's a massive design space for deploying
general agents natively inside the web apps we already use, rather
than treating the web merely as a dumb target for isolated bots.
Currently, most AI agents operate from external clients or server-
side programs, effectively leaving web development out of the AI
ecosystem. I'm experimenting with an "inside-out" paradigm instead.
By dropping the library into a page, you get a client-side agent
that interacts natively with the live DOM tree and inherits the
user's active session out of the box, which works perfectly for
SPAs. To handle cross-page tasks, I built an optional browser
extension that acts as a "bridge". This allows the web-page agent
to control the entire browser with explicit user authorization.
Instead of a desktop app controlling your browser, your web app is
empowered to act as a general agent that can navigate the broader
web. I'd love to start a conversation about the viability of this
architecture, and what you all think about the future of in-app
general agents. Happy to answer any questions!
Author : simon_luv_pho
Score : 62 points
Date : 2026-03-05 17:01 UTC (5 hours ago)
(HTM) web link (alibaba.github.io)
(TXT) w3m dump (alibaba.github.io)
| simon_luv_pho wrote:
| This is highly experimental right now, but here are some quick
| links for anyone wanting to dig deeper:
|
| - GitHub: https://github.com/alibaba/page-agent
|
| - Live Demo (No sign-up): https://alibaba.github.io/page-agent/
| (you can drag the bookmarklet from here to try it on other sites)
|
| - Browser Extension:
| https://chromewebstore.google.com/detail/page-agent-ext/akld...
|
| I'd be really interested in feedback on the security model of
| client-side agents giving extension-bridge access, and taking
| questions on the implementation!
| jadbox wrote:
| I tried setting the LLM to "http://0.0.0.0:8080" and the
| extension crashed and now continues to crash at startup.
| jauntywundrkind wrote:
| Not exactly the same but I'd also point to Paul Kinlan's FolioLM
| as a very interesting project in this space. A very nice browser
| extension,
|
| > _Collect and query content from tabs, bookmarks, and history -
| your AI research companion. FolioLM helps you collect sources
| from tabs, bookmarks, and history, then query and transform that
| content using AI._
|
| https://github.com/PaulKinlan/NotebookLM-Chrome
| https://chromewebstore.google.com/detail/foliolm/eeejhgacmlh...
| simon_luv_pho wrote:
| Thanks for sharing! We need more projects like this in the JS
| ecosystem.
| klueinc wrote:
| I've been trying to arrive to something like this with my own
| sidepanel extension called Klue but its more of a user notes +
| web page context approach. Nice to see another take on this!
| https://chromewebstore.google.com/detail/cackjmmgcmnkjnffabk...
| pscanf wrote:
| Very cool!
|
| I'm particularly impressed by the bookmark "trick" to install it
| on a page. Despite having spent 15 years developing for the
| browser, I had somehow missed that feature of the bookmarks bar.
| But awesome UX for people to try out the tool. Congrats!
| simon_luv_pho wrote:
| Thanks!
|
| Bookmarklets are such an underrated feature. It's super
| convenient to inject and test scripts on any page. Seemed like
| the perfect low-friction entry point for people to try it out.
|
| Spent some time on that UX because the concept is a bit hard to
| explain. Glad it worked!
| MeteorMarc wrote:
| Confusing name because of the existence of pageant, the putty
| agent.
| kirth_gersen wrote:
| Came here to say missed opportunity to call it "PAgent". Rolls
| off the tongue better than Page Agent.
| simon_luv_pho wrote:
| I'm 2 years too late for that one...
| simon_luv_pho wrote:
| Darn. Pageant would've been a nice name though. Maybe `page-
| agent.js` is more relevant in web dev community.
| mmarian wrote:
| I think page agent is good. I've never heard of putty's
| pageant. And I think it's better to distinguish it from
| general meaning of pageant (for beauty).
| simon_luv_pho wrote:
| Thanks!
| graypegg wrote:
| I think every successful Show HN post ends up with a "thought
| this was about X" or "didn't look up the name first?"
| comment. Consider it a win! I don't think anyone will mistake
| a tool for putty with your tool, but you might share a google
| search page with it.
| coreylane wrote:
| Looks cool! Are you open to adding AWS Bedrock or LiteLLM
| support?
| simon_luv_pho wrote:
| Thanks!
|
| It supports any OpenAI-compatible API out of the box, so AWS
| Bedrock, LiteLLM, Ollama, etc. should all work. The free
| testing LLM is just there for a quick demo. Please bring your
| own LLM for long-time usage.
| dzink wrote:
| Is this Affiliated with the Chinese company Alibaba? Any chance
| data goes there too?
| simon_luv_pho wrote:
| Full transparency: I work at Alibaba and published this under
| Alibaba's open-source org. I sometines maintain it during work
| hours, so yes, Alibaba technically pays me for it. That said,
| this is my project -- it's MIT-licensed, includes no backend
| service, and is open for anyone to audit.
|
| The free testing LLM endpoint is hosted on Alibaba Cloud
| because I happen to have some company quota to spend, but it's
| not part of the library. Bring your own LLM and there is zero
| data transmission to Alibaba or anywhere else you haven't
| configured yourself.
|
| I highly recommend using it with a local Ollama setup.
| mentalgear wrote:
| > Data processed via servers in Mainland China
|
| Appreciate the transparency, but maybe you could add some
| European (preferably) alternatives ?
| simon_luv_pho wrote:
| Please use your own LLM api instead!
|
| The free testing LLM is Qwen hosted by Aliyun. Qwen and
| DeepSeek are the only ones I can afford to offer for free. It's
| just there to lower the try-out barrier; please DO NOT rely on
| it.
|
| The library itself does NOT include any backend service. Your
| data only goes to the LLM api you configured.
|
| I tested it on local Ollama models it works fine.
| simon_luv_pho wrote:
| I'm looking into a European testing endpoint. The legal and
| compliance requirements are quite hassle, and persuading my
| company to pay for that infrastructure is gonna be a tough
| sell.
| general_reveal wrote:
| I've been thinking about something like this. If it's just a one
| line script import, how the heck are you trusting natural
| language to translate to commands for an arbitrary ui?
|
| The only thing I can think of is you had the AI rewrite and embed
| selectors on the entire build file and work with that?
| simon_luv_pho wrote:
| Everything happens at runtime, on the HTML level.
|
| It uses a similiar process as `browser-use` but all in the web
| page. A script parses the live HTML, strips it down to its
| semantic essentials (HTML dehydration), and indexes every
| interactive element. That snapshot goes to the LLM, which
| returns actions referencing elements by index. The agent then
| simulates mouse/keyboard events on those elements via JS.
|
| This works best on pages with proper semantic HTML and
| accessibility markup. You can test it right now on any page
| using the bookmarklet on the homepage (unless that page CSP
| blocks script injection of course).
| Mnexium wrote:
| Curious - how does it perform with captchas and other "are you
| human" stuff on the web?
| simon_luv_pho wrote:
| I added in the system prompt that it should skip CAPTCHAs and
| hand control back to the user. Currently working on a proper
| human-in-the-loop feature. That's actually one of the key
| advantages of running the agent inside your own browser.
| Mnexium wrote:
| Makes sense.
|
| For curiosity's sake, have you had it try to attempt
| captchas?
|
| If so, what were the results?
| simon_luv_pho wrote:
| I haven't. I don't think it will work well.
|
| I use a text-based approach. Captchas like "crossroad"
| usually need a screenshot, a visual model and coordinate-
| based mouse events.
| CloakHQ wrote:
| the CAPTCHA question points to a deeper issue: even before
| the CAPTCHA, most sites are already fingerprinting the
| browser that's running the agent. if the underlying browser
| leaks automation signals - navigator.webdriver, Canvas
| fingerprint deviations, WebGL anomalies - the session gets
| flagged or soft-blocked long before a CAPTCHA is even served.
|
| the "inside your own browser" angle is actually the right
| intuition here. a real user's browser has built up a
| consistent fingerprint profile across sessions. the moment
| you run an agent in a context where those signals differ from
| that baseline, you're detectable. curious whether you've run
| into this on sites with aggressive bot detection, or whether
| the use case has mostly been internal/enterprise apps where
| that's not a concern?
| popalchemist wrote:
| Does it support long-click / click-and-drag?
| simon_luv_pho wrote:
| Not yet. Currently focused on the more common interaction
| patterns. PRs welcome though!
| popalchemist wrote:
| Gotcha. Still very cool! Congrats on the release.
| simon_luv_pho wrote:
| Thanks!
| jadbox wrote:
| Firefox support?
| moehj wrote:
| "Interesting architecture -- embedding the agent inside the app
| context rather than outside it makes sense for session-aware
| tasks. One question: how do you handle output validation before
| the agent acts on the DOM? Client-side agents acting on live
| state without a certification layer seems like a reliability risk
| in production. We've been building ARU (aru-runtime.com) as a
| runtime certification layer for exactly this -- curious if you've
| thought about that boundary."
___________________________________________________________________
(page generated 2026-03-05 23:00 UTC)