[HN Gopher] Show HN: A discovery-focused search engine for Hacke...
___________________________________________________________________
Show HN: A discovery-focused search engine for Hacker News
We (Nick, Dens, Denzell, Fede, Drew, Aaryan, and Daniel) have been
building HN Discovery, a discovery-focused search engine for Hacker
News, in our spare time for the past 6 months and are excited to
show it! It adds the following features relative to the existing
keyword search interface and preserves the existing ones: - no-JS
version (hnnojs.trieve.ai) - site:{required_site} and
site:{negated-site} filters - public analytics - LLM generated
query suggestions based on random stories - recommendations -
dense vector semantic search - SPLADE fulltext search - RAG AI
chat - order by descendant count client code (FOSS self-hostable)
- https://github.com/devflowinc/trieve-hn-discovery engine code
(BSL source-available) - https://github.com/devflowinc/trieve
There is an extended about page with detailed information on
features, how much it costs to run, etc. here -
https://hn.trieve.ai/about.
Author : skeptrune
Score : 100 points
Date : 2024-08-29 17:08 UTC (5 hours ago)
(HTM) web link (hn.trieve.ai)
(TXT) w3m dump (hn.trieve.ai)
| ghoomketu wrote:
| This is impressive! I've frequently encountered challenges with
| Algolia search not locating specific items, but this appears to
| offer a much more detailed search capability.
|
| I've bookmarked this site and hope it remains available when I
| need it, unlike many great Show HN posts that vanish after six
| months or so.
| skeptrune wrote:
| Glad you found it useful! Fulltext search will almost certainly
| be up in perpetuity, however, we may drop the semantic index if
| it doesn't get much usage as that's significantly more
| expensive to host.
| dewey wrote:
| > $6835.39/month
|
| This seems way higher than I expected. Cloud pricing is out of
| control when Postgres is already > $500 for a small instance that
| could be run for a fraction if it wouldn't be a cloud provider.
| skeptrune wrote:
| Pretty sure we will have to co-locate soon in general. Cloud
| provider costs are near unsustainable for our business at
| least.
| leobg wrote:
| I'm willing to bet that I can reduce your costs by at least
| 10x. I'd go so far as to say this thing should be able to
| handle HN front page traffic at < $300 / month, including all
| real-time vector search.
|
| That is, if this 6k number is actually true. Part of me
| (forgive me) is in fact wondering if maybe this is an
| advertisement for your SaaS and you're inflating this number
| to make people think there's no way they can build a thing
| like that themselves. But, giving you the benefit of doubt,
| if you are truly paying this, you're overspending by more
| than an order of magnitude. Most likely too many middlemen.
|
| Email is in my profile if you want to talk about it.
| skeptrune wrote:
| It is 100% not made up to make our SaaS more attractive.
| Our shared SaaS only goes up to 1M vectors, so it's not
| like it's cheaper anyways. We would currently charge the
| raw cost + ~20% for us to host something at HN scale.
| Almost none of the cost is due to serving the traffic; it's
| all just rented high mem compute instances and GPUs. We
| could serve ~1k QPS on the current infra.
|
| Our terraform and helm are public in the repo - https://git
| hub.com/devflowinc/trieve/tree/main/terraform/gcl...
| Ishaank1999 wrote:
| the quality of the search is really good! really useful to see
| how opinions change over time on HN (try typing in 'rust is').
| shiandow wrote:
| Oh thanks! I had one heck of a difficult time trying to find the
| author and title of 'the glass bead game' by Herman Hesse. It's
| pretty hard to find with simple keyword based search.
|
| Though short comments seemed to score a bit too highly IMHO. It
| took a while to find a query that found the long rambly comment I
| needed.
| skeptrune wrote:
| We spent quite a bit of time trying to make length
| normalization better, but there's still a lot of room for
| improvement. The default behavior was super biased towards
| longer rambly comments and we may have over-corrected. I
| appreciate the note.
| airstrike wrote:
| Congrats on building and shipping! I love how "at home" the
| styling feels.
|
| I searched for `Excel` since that's a topic I care about and tend
| to follow frequently.
|
| The first ("most relevant") link is a post with 1 point and 0
| comments from 2020. The second link has 1 comment and 2 points,
| from 2017.
|
| The top two links on Algolia's search have about 1000 points each
| and are way more topical.
|
| I tried to hit "Back" to make this comment, and saw the site
| broke my browser's navigation. I was forced to right click on
| "Back" (or spam the back button) to get back here... so not a
| great experience, overall
| skeptrune wrote:
| - the version without search as you type and JS at
| https://hnnojs.trieve.ai/ has a more normal back button
|
| - Algolia ranks by points by default while we are ranking by
| relevance score which is the difference. You can order our
| results by points if you want to with the order by select
| component that says "relevance".
|
| We went back and forth on making points sorting default and
| ended up deciding against it, but maybe we should have. Our
| thinking was that since it's focused on "discovery" it was
| worth prioritizing relevance, but I can see how it can feel the
| result quality isn't as great. HN is really good at
| highlighting interesting links.
|
| Best fix would have been LTR, but we made incorrect decisions
| early on which made the rewrite a bit too hard -
| https://trieve.ai/launching-trieve-hn-discovery/#relevance-q...
| vercantez wrote:
| This is impressively fast. Well done!
| globalise83 wrote:
| OK, so I have a very specific product area that I am both
| interested in from a discovery perspective and know (modestly)
| quite a lot about, and just ran a query on it, and I must say,
| this search engine provided a really interesting and thought-
| provoking set of results. Good job.
| skeptrune wrote:
| Awesome to hear!
| gsuuon wrote:
| This is nice - I for one appreciate the results not being ranked
| by points, since some interesting stories don't get any traction.
| skeptrune wrote:
| It's a tough call. I personally agree that default sorting on
| relevance score is better for discovery. It's easy enough to
| switch the ordering to points if desired.
| autoexec wrote:
| This is very cool! Since HN's own search doesn't work without JS
| enabled this is a massive help
| skeptrune wrote:
| Hell yeah. We were super passionate about making sure it would
| work without JS as well as with.
| pflanze wrote:
| Thanks for the nojs version!
| skeptrune wrote:
| Of course!!
___________________________________________________________________
(page generated 2024-08-29 23:00 UTC)