[HN Gopher] Show HN: Index - new SOTA Open Source browser agent
___________________________________________________________________
Show HN: Index - new SOTA Open Source browser agent
Hey HN, Robert from Laminar (lmnr.ai) here. We built Index - new
SOTA Open Source browser agent. It reached 92% on WebVoyager with
Claude 3.7 (extended thinking). o1 was used as a judge, also we
manually double checked the judge. At the core is same old idea -
run simple JS script in the browser to identify interactable
elements -> draw bounding boxes around them on a screenshot of a
browser window -> feed it to the LLM. What made Index so good: 1.
We essentially created browser agent observability. We patched
Playwright to record the entire browser session while the agent
operates, simultaneously tracing all agent steps and LLM calls.
Then we synchronized everything in the UI, creating an unparalleled
debugging experience. This allowed us to pinpoint exactly where the
agent fails by seeing what it "sees" in session replay alongside
execution traces. 2. Our detection script is simple but extremely
good. It's carefully crafted via trial and error. We also employed
CV and OCR. 3. Agent is very simple, literally just a while loop.
All power comes from carefully crafted prompt and ton of eval runs.
Index is a simple python package. It also comes with a beautiful
CLI. pip install lmnr-index playwright install chromium index
run We've recently added o4-mini, Gemini 2.5 Pro and Flash. Pro is
_extremely good and fast_. Give it a try via CLI. You can also use
index via serverless API. (https://docs.lmnr.ai/index-
agent/api/getting-started) Or via chat UI - https://lmnr.ai/chat.
To learn more about browser agent observability and evals check out
open-source repo (https://github.com/lmnr-ai/lmnr) and our docs
(https://docs.lmnr.ai/tracing/browser-agent-observability).
Author : skull8888888
Score : 30 points
Date : 2025-04-23 16:11 UTC (6 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| purplecats wrote:
| got a video demo that isn't behind a sign up wall?
| skull8888888 wrote:
| here's a demo of a chat UI
| https://x.com/skull8888888888/status/1910763169489764374
|
| here's a demo of CLI
| https://x.com/skull8888888888/status/1914728292193628330
| skull8888888 wrote:
| there's also a demo right in the repo https://github.com/lmnr-
| ai/index
| noleary wrote:
| > Index is the SOTA open-source browser agent for autonomously
| executing complex tasks on the web.
|
| I've written a handful of pretty hacky Python scripts that just
| pull down all of the HTML content from a page and toss it over to
| OpenAI. As you can imagine, these were all extremely simple
| tasks, e.g., "find out if there's a login button"
|
| What's a good example of a complex task that Index is well-suited
| for? What's the threshold of minimal complexity where you guys
| are a really good fit?
| skull8888888 wrote:
| - research task, agent is smart enough to understand which
| links to click next without the need to hardcode the parsing
| and navigation logic
|
| - any task that requires UI interaction, button clicking,
| filter selection, form filling and so on. Just prompt it, it's
| surprisingly very robust and self-healing.
|
| - complex long-running task that require extensive context -
| e.g. researching one topic and then creating spreadsheet,
| creating a presentation for a topic and so on.
|
| Essentially, any task that can be done within a browser
| environment that previously required flacky hardcoded
| predefined scripts. Also, website testing is a great example.
| androng wrote:
| Can it actually do something difficult like apply for jobs? So
| far I know of five or so websites that claim they can apply to
| jobs for you like sonara.ai and usemassive.com and Skyvern AI but
| when you try to actually use them all they can do is the one-page
| job applications and not the much more common Workday 10-page job
| applications with annoying "create an account" and annoying
| questions like "Do you have any relatives that work at Sony" and
| annoying "fill out all your work experience" where you have to
| click 50 times for one application. That's like half of all job
| applications.
| https://jobs.spectrum.com/job/-/-/4673/76746020384?utm_sourc...
| skull8888888 wrote:
| I'm pretty confident it can do it. Try it out and see for
| yourself. Just install the package, run cli and give it your
| prompt.
|
| pip install lmnr-index playwright install chromium index run
|
| Also try experimenting with different models. So far, Gemini
| 2.5 Pro is the best in terms of quality/speed. Claude 3.7 is
| also pretty good.
___________________________________________________________________
(page generated 2025-04-23 23:00 UTC)