[HN Gopher] Show HN: I made a better Perplexity for developers
___________________________________________________________________
Show HN: I made a better Perplexity for developers
Hi HN, I am Jiayuan, and I'm here to introduce a tool we've been
building over the past few months: Devv (https://devv.ai). In
simple terms, it is an AI-powered search engine specifically
designed for developers. Now, you might ask, with so many AI
search engines already available--Perplexity, You.com, Phind, and
several open-source projects--why do we need another one? We all
know that Generative Search Engines are built on RAG (Retrieval-
Augmented Generation)[1] combined with Large Language Models
(LLMs). Most of the products mentioned above use indexes from
general search engines (like Google/Bing APIs), but we've taken a
different approach. We've created a vertical search index focused
on the development domain, which includes: - Documents: These are
essentially the single source of truth for programming languages or
libraries; I believe many of you are users of Dash
(https://kapeli.com/dash) or devdocs (https://devdocs.io/). -
Code: While not natural language, code contains rich contextual
information. If you have a question related to the Django
framework, nothing is more convincing than code snippets from
Django's repository. - Web Search: We still use data from search
engines because these results contain additional contextual
information. Our reasons for doing this include: - The quality of
the index is crucial to the RAG system; its effectiveness
determines the output quality of the entire system. - We focus
more on the Index (RAG) rather than LLMs because LLMs evolve
rapidly; even models performing well today may be superseded by
better ones in a few months, and fine-tuning an LLM now has
relatively low costs. - All players are currently exploring what
kind of LLM product works best; we hope to contribute some
different insights ourselves (and plan to open source parts of our
underlying infrastructure in return for contributions back into
open source communities). Some brief product features: - Three
modes: - Fast mode: Offers quick answers within seconds. - Agent
mode: For complex queries where Devv Agent infers your question
before selecting appropriate solutions. - GitHub mode(currently in
beta): Links directly with your own GitHub repositories allowing
inquiries about specific codebases. - Clean & intuitive UI/UX
design. - Currently only available as web version but Chrome
extension & VSCode plugin planned soon! Technical details
regarding how we build our Index: - Documents section involves
crawling most documentation sources using scripts inspired by
devdocs project's crawler logic then slicing them up according
function/symbol dimensions before embedding into vector databases;
- Codes require special treatment beyond just embeddings alone
hence why custom parsers were developed per language type
extracting logical structures within repos such as architectural
layouts calling relationships between functions definitions etc.,
semantically processed via LMM; - Web searches combine both
selfmade indices targeting developer niches alongside traditional
API based methods. We crawled relevant sites including blogs forums
tech news outlets etc.. For the Agent Mode, we have actually
developed a multi-agent framework. It first categorizes the user's
query and then selects different agents based on these categories
to address the issues. These various agents employ different models
and solution steps. Future Plans: - Build a more comprehensive
index that includes internal context (The Devv for Teams version
will support indexing team repositories, documents, issue trackers
for Q&A) - Fully localized: All of the above technologies can be
executed locally, ensuring privacy and security through complete
localization. Devv is still in its very early stages and can be
used without logging in. We welcome everyone to experience it and
provide feedback on any issues; we will continue to iterate on it.
[1]: https://arxiv.org/abs/2005.11401
Author : jiayuanzhang
Score : 155 points
Date : 2024-05-08 15:19 UTC (7 hours ago)
(HTM) web link (devv.ai)
(TXT) w3m dump (devv.ai)
| danenania wrote:
| Looks very cool! Congrats on the launch.
|
| "For complex queries where Devv Agent infers your question before
| selecting appropriate solutions."
|
| Could you expand on this a bit? What does "infers your question"
| mean?
|
| It's not all that clear to me from the site or your post when
| Fast Mode vs. Agent Mode should be used. Is Fast Mode for
| answering conversational questions and Agent Mode for answers
| that involve writing code?
| datadrivenangel wrote:
| Pretty cool!
|
| Looks like there's an opportunity to improve the fast mode by
| caching the results for simple searches.
| moneywoes wrote:
| any insights in how you built it?
| jiayuanzhang wrote:
| Hi, Devv founder here.
|
| I've outlined some initial ideas in this post and may develop a
| more detailed article later on. Stay tuned!
| cpursley wrote:
| This is great. I'd love to see a higher level architectural
| writeup/talk (but not stack specific) about how to build a live
| search RAG system like this, perplexity, etc.
| jiayuanzhang wrote:
| I've previously given a talk on RAG and am eager to adapt it
| into an article in the future.
| unshavedyak wrote:
| Your implementation strategy sounds interesting! I'll give it a
| try. While reading your design it made me interested if i as a
| user could prompt new indexes for libaries i use.
|
| Ie if the quality RAG index is your primary offering, then as a
| user i imagine my experience will depend on how well you have
| indexed things i care about. Maybe my language of choice (Rust)
| has decent indexes, but some random Crate i try to use might not.
|
| I'd love to be able to queue up index ingests of standard API
| sources like docs.rs/crates.io and be notified when that ingest
| completes.
|
| Will give it a try today, congrats on the launch!
| jiayuanzhang wrote:
| Hi, Devv founder here.
|
| Thank you for your valuable feedback; it's an excellent
| suggestion! In fact, we've already begun implementing this
| feature with our initial step being the introduction of GitHub
| Mode. This new functionality will enable seamless integration
| with your personal GitHub repositories. We've developed a
| bespoke indexer tailored to various programming languages to
| enhance this experience.
|
| Furthermore, we can expand this capability to include
| documentation and other resources as well. The architecture is
| designed to be extensible, so all that's needed is the creation
| of additional indexers to support these materials.
| lxe wrote:
| Very polished. Would love to know more about the tech behind
| this.
| bschmidt1 wrote:
| Great UI/UX and very nice work!
|
| I just tested it by typing "llama cpp gpu support" that's it.
|
| Flawless instructions for Python, but when I followed up with
|
| "in node"
|
| It didn't know about node-llama-cpp. Is there a general knowledge
| cutoff, and/or is loading developer-specific stuff a manual
| process?
| jiayuanzhang wrote:
| I use Agent Mode to rephrase the query, and it appears to have
| provided the correct answer.
|
| The results: https://devv.ai/search?threadId=dl3vwbdu52ww
|
| P.S. Agent Mode is a superior option to Fast Mode. It
| meticulously examines your questions and assigns an appropriate
| agent to provide answers, leveraging GPT-4 technology in its
| operations.
| bschmidt1 wrote:
| Nice! Yeah without it, it wanted me to make my own node-
| llama-cpp: https://devv.ai/search?threadId=dl3rah43egw0
|
| So agent mode is better for more recent stuff that you might
| find in a search engine?
| jiayuanzhang wrote:
| Agent Mode has a "thinking" process, so it will be more
| intelligent than Fast Mode.
| fallingsquirrel wrote:
| Since you asked for feedback: I tried it out by asking about Zig
| conditional compilation, and it hallucinated some syntax that
| doesn't exist.
|
| https://devv.ai/search?threadId=dl3rtxmcsruo
|
| EDIT: The syntax came from a language proposal in a github issue
| from 8 years ago, so I guess it's not fully hallucinated. But
| still not the best choice of what source to use.
| jiayuanzhang wrote:
| Hello, I'm the founder of Devv. Thank you for your feedback;
| however, I'm not acquainted with Zag, so I can't identify the
| incorrect syntax.
|
| I utilized Agent Mode to rephrase the query, and here are the
| results: https://devv.ai/search?threadId=dl3rtxmcsruo
|
| P.S. Agent Mode is a superior option to Fast Mode. It
| meticulously examines your questions and assigns an appropriate
| agent to provide answers, leveraging GPT-4 technology in its
| operations.
|
| ---
|
| > The syntax came from a language proposal in a github issue
| from 8 years ago, so I guess it's not fully hallucinated. But
| still not the best choice of what source to use.
|
| Yes, if the source contains errors, the response may generate
| inaccurate information. We are continually refining our re-
| ranking algorithm within the Retrieval-Augmented Generation
| (RAG) system to select the most reliable sources.
| williamstein wrote:
| It's "Zig" not "Zag". https://ziglang.org/ Zig is under heavy
| development, but there's a single page
| https://ziglang.org/documentation/0.12.0/ that is a
| reasonably comprehensive source of truth about the current
| state of the language.
| binary132 wrote:
| Someone should come up with a language called Zag just so
| they can say "zigged when he should have zagged"
| unshavedyak wrote:
| Seems like it would be a great library for Zig. Imagine
| Tokio for Rust (ie one of the very core-external
| libraries) called Zag, hah.
| jiayuanzhang wrote:
| Apologies, that was a typo.
| hemloc_io wrote:
| I've noticed this w/ using LLMs for programming, esp while
| learning Zig.
|
| Outside of popular languages it seems like they always
| hallucinate.
| Alifatisk wrote:
| What is tha Fast option and Agent option?
| mdp2021 wrote:
| It seems quite useful, congratulations.
|
| I am also pleasantly surprised it is not suffering a "hug of
| death" following the presentation here. I am curious about the
| need in resources for your engine? What kind of hardware is it
| running on?
| jiayuanzhang wrote:
| frontend: next.js + react + vercel
|
| backend: go/rust/python + gin + mysql + pinecone + es + redis +
| aws
|
| llm: openai/azure + aws gpu + aws bedrock
| devmor wrote:
| It's not very good at giving the proper credence to version
| numbers.
|
| Granted I started with a hard one, but I asked it how to create a
| GTK3 interface with PHP, and it gave me instructions to download
| and use an abandoned project for GTK2, but described it as GTK3
| in the steps.
|
| I tried asking it some other questions about languages and
| applications specific to version numbers - it seems to provide
| incredibly ambiguous and version agnostic responses, or tells me
| essentially "you may or may not be able to do this, and you
| should check if you can" when the answer is clearly that it is
| not possible. Or it just ignores the version entirely and
| provides instructions that don't match up - hallucinating UI
| elements or commands that don't (or didn't yet) exist.
|
| For something targeted at developers, this is a gaping hole and
| is what I would consider a major oversight - the responses I'm
| getting are very similar in content to what I get from GPT and
| Ollama's generic models.
| mdp2021 wrote:
| It is quite possible that the lack of actual intelligence in
| the LLM is the obstacle in this context.
|
| I also just queried something with "perplexing" results in
| fact, but I tried the "generic" "knowledge" instead of the
| "specific" about coding: in the reply the engine included good
| pointers, but clearly without knowing why they were especially
| relevant - relevance which instead appeared in the linked
| references.
|
| It is an LLM+RAG based search engine: the value is only partly
| in the summary, which could even be misleading - as expected
| from lack of actual intelligence -, the value is in the linked
| resources.
|
| In other words, it "understands" your query better that a
| search engine of the past - and that is valuable. But for the
| actual solution you are querying for, the "summary" part could
| be good or could be defective: it is probably best to consult
| the linked material... Material that you could have not found
| immediately otherwise - it could have been tricky with past
| technology to express your need in a way that makes you obtain
| good search results.
| everforward wrote:
| That's kind of an interesting issue, I wonder if different
| tokenization would help. Like maybe putting a space between GTK
| and the number would put them in separate tokens and give
| better output.
|
| More generally, do text AI's not support weighting terms like
| the image AI's do? Over in Stable Diffusion that sounds like
| something where I'd add a weight like "How do I create a
| <GTK3:1.2> interface in <PHP:1.1>?"
| jiayuanzhang wrote:
| Have you tried Agent Mode? It offers greater intelligence and
| accuracy compared to Fast Mode.
|
| P.S. Agent Mode is a superior option to Fast Mode. It
| meticulously examines your questions and assigns an appropriate
| agent to provide answers, leveraging GPT-4 technology in its
| operations.
| trirpi wrote:
| Even without the AI generated content this is useful. Google
| seems to not index Github repositories so you can't search for
| specific variable names.
|
| Feedback: I tried to click one of the links under "source" but it
| kept jumping down as the LLM-generated content was added.
| jiayuanzhang wrote:
| Hi, Devv founder here. It works fine here, did you just click
| the "related questions"? This will generate a follow-up answer.
| anonu wrote:
| how do you see a Chrome extension working?
| canadiantim wrote:
| How does it compare to Greptile? Can I use it to ask questions
| about my own codebase?
| 2StepsOutOfLine wrote:
| Regarding: "Fully localized: All of the above technologies can be
| executed locally, ensuring privacy and security through complete
| localization."
|
| Does this mean you intend to let people self-host?
| jiayuanzhang wrote:
| Hi, Devv founder here.
|
| Yes, this is on our roadmap. We will launch "Devv for Teams" in
| the upcoming quarter. This new feature will enable seamless
| integration of internal team knowledge, including codebases,
| wikis, issue trackers, and logs.
| afro88 wrote:
| If self hosted Devv for Teams supports BitBucket, Confluence,
| JIRA and Azure DevOps, the company I work for (v large
| enterprise) would be incredibly interested.
| ActionHank wrote:
| I would install right away if this is the case.
|
| I really distrust putting my API keys into brand new and
| unknown websites, just seems like credentials harvesting to me.
| jiayuanzhang wrote:
| https://news.ycombinator.com/item?id=39923404
|
| You might want to check out this project.
| williamstein wrote:
| Is there something like this (maybe this?) that provides an API
| so I can integrate it like any other model into my own website
| (in this case, https://cocalc.com)? I tried asking the Phind.com
| devs, but got ignored.
| jiayuanzhang wrote:
| Hi, Devv founder here.
|
| That sounds interesting. Could you provide further details? By
| the way, integrating an API is part of our future plans. We
| plan to enable Devv integration with Slack, Linear, and
| websites in the future.
|
| Also, if you want to discuss more, feel free to email me at
| jiayuan@devv.ai
| danenania wrote:
| I would also love an API like this for integration with
| Plandex[1] (a terminal-based coding agent for complex tasks).
| Perplexity has an API but it only exposes various open source
| LLMs, not the search-enriched results from their main product.
|
| It would be really cool if, when starting a coding task with
| Plandex, relevant docs/context from a web search could be
| automatically included in context via this kind of API.
| Currently urls can be loaded into context with `plandex load
| [url]` but you have to figure out which urls would be helpful
| to load yourself.
|
| 1 - https://github.com/plandex-ai/plandex
| jiayuanzhang wrote:
| Great, we're on it. We'll be looking into this project and
| keeping you updated at our changelog hub:
| https://hub.devv.ai/changelog. As soon as our API goes live,
| we'll post the announcement there.
| algo_trader wrote:
| Is the Plandex Cloud still active ?
|
| I dont see a link to it on the github/website. DO i have to
| self host right now ?
|
| Good luck with this
| danenania wrote:
| Thanks! Cloud is active yes. Just install Plandex as
| described here https://github.com/plandex-
| ai/plandex?tab=readme-ov-file#ins... -- then run `plandex
| new` and you'll be prompted to start an anonymous trial on
| cloud.
| rcarmo wrote:
| I don't really get Perplexity - it is amazingly slick, but I get
| almost line-by-line identical output from Bing Chat, so I have to
| wonder how much differentiation they really afford (I haven't set
| up an account, just comparing free access). This, though, has
| mostly gotten what I asked it right (including some arcane C++
| stuff), so I will be giving it a try at home.
| tppiotrowski wrote:
| I use perplexity because it's the best free GPT that is
| anonymous (no login required)
| tarasglek wrote:
| My https://chatcraft.org offers free models and is open
| source. They start throttling under heavier usage tho.
|
| Gonna add some free models with search in future
| paxys wrote:
| I unironically like meta.ai better. It uses both Google and
| Bing for web searches, and is better at citing its sources.
| winterturtle wrote:
| I've been using https://yaddle.ai. It's got a nicer UI, free
| to use, and has a lot of models to try.
| niutech wrote:
| Both Bing Copilot and Perplexity Copilot uses GTP-4 under the
| hood. For more LLMs, check out https://labs.perplexity.ai
| dcsan wrote:
| does the bing API have citations? both you.com and pplx have
| that feature which chatGPT doesn't, tho its still in closed
| beta release for pplx
| yawnxyz wrote:
| I just signed up for the PPLX API and it surprisingly doesn't
| do internet searches... It just has slightly more recent info
| than GPT-4.
|
| Try searching for "Weather in [your city]" and compare it to
| Google or any weather app. It's consistently wrong.
| mg wrote:
| Have you thought about accepting the query as a get request?
|
| The 3 engines you mention (Perplexity, You.com and Phind) all do
| that. So do Google, Bing and DuckDuckGo. It makes it easier to
| link to results and build custom links.
|
| Also, I could add you to Gnod Search then:
|
| https://www.gnod.com/search/ai
| jiayuanzhang wrote:
| You can initiate a new search by using the URL
| `https://devv.ai/search/{query}`.
| mg wrote:
| Awesome, added:
|
| https://www.gnod.com/search/ai?q=Python%3A%20How%20do%20I%20.
| ..
| nilsherzig wrote:
| Ah nice, good work :) I might steal some design ideas for my own
| project haha https://github.com/nilsherzig/LLocalSearch
| noashavit wrote:
| Congrats on the launch! Great UI, better even than that of
| Perplexity :-)
| yoouareperfect wrote:
| Very nice! I asked it for "React Suspense" and the results were
| pretty ok!
| Aachen wrote:
| FYI, I can't view your terms because it claims my browser is
| incompatible. The website itself (devv), HN, OpenGL applications,
| youtube (JS-heavy), everything works fine but the plain text that
| your ToS and privacy need to be give that error message with no
| further information that I could pass on to debug it
|
| In case anyone knows, I'd be curious: does that mean no terms
| apply to my usage if I can't view them by reasonable means? Just
| whatever local law defaults apply? Earlier today I noticed the
| terms of the local zoo 404'd (while buying tickets online) and I
| wondered the same
| hangonhn wrote:
| Oh wow. This is quite decent. I asked it two questions that has
| historically tripped up either Google Gemini (what does an
| asterisks in the middle of a parameters list mean in Python) or
| ChatGPT (how to extend Fernet to use AES 256) and it got both of
| them right.
| factorymoo wrote:
| I asked it for an efficient way to sort a list in Python [1].
|
| I'm running the code it gave me to try it out on a small list,
| it's been 10 minutes and it's still running. Might be something
| worth looking into.
|
| Granted, the way I asked for this function was not the most
| natural.
|
| [1] https://devv.ai/search?threadId=dl4c8if11c00
| monsieurbanana wrote:
| It won't be worth looking into, because there's nothing they
| (devv.ai) can do, short of trying some automated self-
| improvement loop a la devin where the AI writes code, evaluates
| the code, fixes issues as they arise... Still not worth, it's
| not their core business.
|
| You're just hitting a limit of the LLMs, they won't give you
| bug-free code, specially not from the first time, specially not
| complex ones like galloping timsort.
___________________________________________________________________
(page generated 2024-05-08 23:01 UTC)