[HN Gopher] Consensus: Use AI to find insights in research papers
___________________________________________________________________
Consensus: Use AI to find insights in research papers
Author : birriel
Score : 43 points
Date : 2023-08-17 19:40 UTC (3 hours ago)
(HTM) web link (consensus.app)
(TXT) w3m dump (consensus.app)
| input_sh wrote:
| I felt really excited clicking on some of the suggested prompts,
| but that excitement quickly fell apart when trying to generate a
| summary for what I consider to be a fairly simple custom search:
|
| impact of airbnb listings on house prices - can't summarise, need
| to post it as a question.
|
| how do airbnb listings impact house prices? - can summarise,
| can't create a concensus, must use a yes/no question.
|
| do more airbnb listings increase house prices? - can summarise,
| can't create a concensus because there's not enough relevant
| search results. But the maximum is 20 (according to the info
| icon) and it found 11 highly relevant articles, so I _really_ don
| 't understand how there isn't enough relevant search results.
|
| And I gave up and deleted my account.
| [deleted]
| dvt wrote:
| > And I gave up and deleted my account.
|
| Same. Yet another "AI product" that has zero product-market fit
| and zero usefulness. What sucks is that even if you _do_ get a
| consensus, it 's not even tractable (e.g. does not properly
| cite sources), so where could I possibly even use the
| conclusion drawn?
| [deleted]
| didibus wrote:
| I've been doing this with chatGPT directly. But it's cutoff data
| is 2021. It's pretty good though at telling you even the research
| name so you can validate.
| SamBam wrote:
| As someone who is starting a master's in science and will be
| looking at lots of research papers, I've been wondering what
| the best use of this could be.
|
| If I have my own PDFs, I guess I could get ChatGPT to create
| summaries in some structured way, perhaps in a single file with
| citation:summary, and then send up that file with every
| question I ask?
| pmarreck wrote:
| I asked it about UBI and I guess there isn't enough data yet,
| because it called it "fiscally unbearable and morally
| unacceptable" (which seems to be what only 1 uncited source
| said)! At least it admitted consensus was low...
|
| https://consensus.app/results/?q=Does%20universal%20basic%20...
|
| Summary Top 10 papers analyzed Some studies suggest that
| universal basic income (UBI) can generate support for structural
| reforms and improve mental health, while other studies argue it
| may be fiscally unbearable, morally unacceptable, and increase
| wealth inequality.
| marcopicentini wrote:
| Why not reply directly with the answer learned from paper?
| Currently it's like search the more relevant paper, open the pdf
| and read it.
| semerda wrote:
| This is great. How is "disputed" calculated? (hopefully not news
| babble but another paper disputing it?)
| acqbu wrote:
| Is this another SaaS making API calls to OpenAI or is it more
| than that? If so, how?
| malux85 wrote:
| Embed articles and throw the results in a vector database.
|
| Throw up a search result that just uses cosine similarity on
| the vector search with questionable metrics and no explanations
| on how things are calculated.
|
| Charge yearly because you know people will churn after a month
| or two.
|
| Profit
|
| - Every "AI startup" in the last 2 months
| fzliu wrote:
| I'll play DA here - there's quite a bit of engineering
| surrounding these apps that can appear hidden to folks from
| the outside looking in. Various levels of prompt engineering
| and in-context learning might be necessary to get optimal
| results, and this could mean significantly more complexity at
| the application level.
| cj wrote:
| Every time I hear or read "prompt engineering", I can't
| help but cringe a bit. I'm not sure why, but it's the same
| reaction I would have if I heard someone say "Google search
| query engineering".
|
| Comparing to google search, there definitely is skill
| involved in knowing how to google things well. We're all
| accustomed to googling things many times per day so I think
| a lot of people forget that being able to google things and
| get the results you want is a skill that has to be learned.
|
| But I would never refer to being "good at writing google
| search queries" as any kind of engineering. But is becoming
| good at searching google any less difficult than getting
| good at writing LLM prompts?
|
| I'd love to hear the other side of the argument. How
| difficult is it to become good at "prompt engineering"? Why
| do we even call it "prompt engineering" instead of just
| "writing effective prompts"?
|
| Edit: I think the main gripe I have with the term "prompt
| engineering" is it makes the skill of writing prompts sound
| a lot harder than it actually is. Maybe I'm underestimating
| how difficult it is to learn how to write good prompts?
| xigency wrote:
| Effective prompting has more to do with theater than
| engineering.
| yuvalr1 wrote:
| > The current source material used in Consensus comes from the
| Semantic Scholar database
|
| For people who are familiar with this database, are the papers in
| it trustworthy?
|
| It's nice to search directly in scienific papers, but only if
| they're reputable.
___________________________________________________________________
(page generated 2023-08-17 23:00 UTC)