[HN Gopher] Scite: Smart Citations for Better Research
___________________________________________________________________
Scite: Smart Citations for Better Research
Author : polm23
Score : 53 points
Date : 2021-06-20 04:21 UTC (1 days ago)
(HTM) web link (scite.ai)
(TXT) w3m dump (scite.ai)
| rerx wrote:
| That name has just been burnt into my mind to stand for the
| Scintilla Text Editor: https://www.scintilla.org/SciTE.html
| Amazing how this is still getting releases after 20+ years.
| hutzlibu wrote:
| Yeah, I assumed by the title, SciTE added a feature to help
| with citations.
| uniqueuid wrote:
| On the tech side, this looks nice. But one thing people don't
| seem to take seriously enough is:
|
| If you're doing science and you don't know the literature, then
| you're doing a bad job.
|
| Understanding a paper won't work by reading AI summaries. Finding
| surprising new papers in your field should be very rare even
| without tech.
|
| [edit] Let me re-phrase the very negative statement above. There
| are a couple of important tasks in working with literature.
|
| One is finding cumulative truth, i.e. figuring out the overall
| state of research. This should be done in a true meta-analysis of
| the data, since references are sometimes citing articles
| incorrectly (i.e. claiming something that's not in the paper) and
| the linguistic complexity is enormous. Meta-analyes are a lot of
| manual work, even though you can extract some data from
| standardized tables.
|
| Another one is figuring out what new publications there are on a
| specific subject. Search engines like google scholar are
| immensely useful here, because you can follow the reference tree
| in both directions (citing and cited). This tool would help, but
| perhaps incrementally so (not a panacea).
|
| A third task would be to identify novel interesting papers that
| you didn't know about, perhaps from a different field. This tool
| might help, but only if an unknown interesting paper cites
| something you're looking at (i.e. if the networks are not
| disjoint).
|
| Apart from that, offering structural analyses and visualizations
| of science is something I believe should be done with open source
| and open data, because science is about trust and transparency -
| now more than ever.
| sleepingsoul wrote:
| Hi,
|
| Thanks for the thoughtful response.
|
| Disclaimer: I work at scite.
|
| I want to clarify one point in your comment because I know a
| lot of people read HN comments before clicking through: we do
| NOT offer AI summaries!
|
| scite is fundamentally built on the idea that not all citations
| are equal. We offer disambiguation into citations by extracting
| the in-text contexts when citations happen between two papers.
| With that, you can qualitatively see how a paper has been cited
| by others by seeing the citation contexts from each citing
| paper.
|
| As someone who used to be in research before at the National
| Cancer Institute, I certainly agree that scientists who are
| advanced in their field should know the literature inside out.
|
| But the utility of scite is I think quite a bit more than that:
|
| - If you're an expert scientist, you can configure alerts to
| know when new citations happen to papers that you care about
| (e.g. ones you published, or that are important in your field,
| etc). Except unlike Google Scholar alerts and such, you can now
| immediately see the citation contexts (as opposed to opening up
| each paper, and trying to locate the reference(s) and the in-
| text citations).
|
| - If you're a student or an experienced researcher trying to
| ramp up in a new field (which is increasingly important given
| how interdisciplinary science is these days), this is a useful
| tool in your arsenal to digest and discover new papers.
|
| - That last comment also applies if you are in the industry!
| e.g. someone in pharma doing drug discovery or evaluation and
| you're working on different drugs every <X> months.
|
| We do have a preprint we published if you'd like to read more
| about how scite works:
| https://www.biorxiv.org/content/10.1101/2021.03.15.435418v1
|
| I hope that was clear. I'm happy to answer any other questions.
| I'll try to come back and check periodically.
|
| - Ashish
| uniqueuid wrote:
| Thanks for taking the time to reply.
|
| I agree that those use cases are definitely valid. And I
| would also say that research is a technologically
| underdeveloped space. So any innovation here is great and
| will definitely help people!
|
| The only worry I have (from past experience) is that easy
| tools have a differential impact: That they facilitate bad
| research more than they facilitate sound research. My
| perception as a scientist is that we primarily need to
| address actively harmful science (as in: spurious results,
| p-hacking). Fostering _more_ science, on the other hand, is
| definitely a second or even third priority.
| sleepingsoul wrote:
| Yep, I agree. I would like to point out though that one of
| our aims with this is to buffer against that bad research
| you mention.
|
| Along with our extraction of citation statements, we have a
| deep learning model that classifies each statement as
| "supporting", "contrasting", or "mentioning" of the claims
| made in the original paper.
|
| Indeed there is a lot of nuance here, and we've tried to
| address this in a number of ways (including a manual flag
| process for review where someone believes it is incorrectly
| labeled), and we're constantly improving in this area. In
| general we would expect that research that is not sound
| will receive more contrasting citations rather than
| supporting citations because their findings won't be as
| well supported by future research.
|
| I should say explicitly that a "contrasting" citation is
| not a bad thing per se, especially given the nature of
| scientific discovery. But a study that was published
| through p-hacking whose findings are not reproducible
| should be easier to discern through subsequent contrasting
| citations. In today's world, all we might know is that it
| was cited a bunch, with the nuance buried underneath.
|
| - Ashish
| rmbyrro wrote:
| @sleepingsoul , how good is your database? Just tried a
| random paper I've read recently (DOI 10.1109/32.748920). It
| shows "no citations" in Scite, but Google Scholar shows
| "Cited by 1180": https://scholar.google.com/scholar?cites=905
| 2935302971671123...
| rmbyrro wrote:
| This seems to be very useful. I'm finding a much better
| experience browsing references made within a paper.
| sleepingsoul wrote:
| Hi there,
|
| Well, that's a tricky question to answer! :)
|
| At the moment we use DOIs to link papers / citations, and
| get our information on DOIs from crossref
| (https://www.crossref.org/).
|
| So every DOI there is also in our system (and any citation
| coming from a source without a DOI currently won't be
| picked up).
|
| From crossref we also have metadata on which publications a
| given paper references, so we can always show information
| on how many publications cite a given paper ("traditional
| citations").
|
| As for our database of Smart Citations (the extracted in-
| text citation statements) -- the answer is that it depends
| on our ability to access full-text articles to actually do
| the citation statement extraction. We're constantly growing
| in this area through new indexing agreements with
| publishers and so on.
|
| To your specific question about that DOI, here is the
| corresponding scite report for that paper:
| https://scite.ai/reports/a-unified-framework-for-coupling-
| dZ...
|
| I see that we identified 371 distinct citing publications
| (and extracted 374 in-text citation statements from those
| publications).
|
| Can you let me know where you saw it say "no citations"?
| That sounds like a bug... I'll take a look. You can email
| me here: ashish@scite.ai
|
| - Ashish
| rmbyrro wrote:
| I think you have an issue in your UI, or I don't
| understand how it's supposed to be used
|
| When I have the results from an article in my screen and
| use the top search to find another, it only updates the
| upper part of the page with the paper title, not the
| stats and summary below it
|
| I had searched for an article before that had no
| citations. That's why I continued to see "no citations"
| when I searched for the DOI I shared...
| sleepingsoul wrote:
| Hi,
|
| Thanks for that description, I managed to reproduce this
| and it is our fault not yours. You are using it
| correctly.
|
| We'll get it fixed ASAP!
|
| EDIT: Fixed now :)
| rmbyrro wrote:
| Thanks!
|
| Weird, I searched for that DOI moments before and it
| showed this result: "This publication either has no
| citations yet, or we are still processing them"
|
| Let me look into others
| rsa4046 wrote:
| This. As a recently retired academic research scientist (i.e.,
| still 'working', but no longer in the lab, and no longer paid
| LOL), this can't be emphasized enough. Most scientists are
| highly specialized: experts in their local area of interest,
| the command of which must be fairly deep. They are aware of
| which papers are about to appear in press, either because they
| were either a referee at some point in the review process, or
| received a 'heads up' preprint.
|
| Your statement on 'finding cumulative truth, i.e. figuring out
| the overall state of research' is a key point.To become newly
| acquainted with a research area (on one's own) is a daunting
| task, requires plowing through dozens and dozens of past
| research articles, review summaries, abstracts, texts, even
| conversing with authors of these works. But rapid progress is
| oft realized when people in two or more separate areas
| serendipitously interact in new projects, thus bringing
| expertise -- heretofore missing or weak -- to bear on a common
| problem. This is the intrinsically collaborative aspect of
| science that survives despite intense intramural competition.
| glup wrote:
| I find it interesting how this is premised on the "one big claim
| per paper" model, which for better or worse seems increasingly
| common. In my experience a lot of 60's-90's research is of the
| form "the results of our first study were consistent with [some
| specific paper], and our second experiment was not." This nuance
| is pretty hard for people to keep in mind, and often seems to get
| lost over time (as a postdoc in cognitive science at MIT, I've
| noticed a big difference between what papers _actually_ say, vs.
| what people say those papers say 30 years on). I 'm excited to
| see how well Scite actually works as I test it over the next
| week!
| darosati wrote:
| Disclaimer: I work at scite.
|
| We, at scite, actually designed scite to help folks see beyond
| the "one big claim per paper" model. Specifically, we designed
| scite to present show how specific results were either
| supported or contrasted by other papers, rather than showing
| that the publication itself was disputed or reproduced as a
| whole. If there was any feedback you could give on how we could
| present things so that folks don't get that impression we'd
| love it - feel free to DM me or let us know at hi@scite.ai as
| you test it out.
|
| In an ideal world we would present the claims, results, and
| argumentation as structured information to the user but we are
| not quite at that stage yet!
| elektor wrote:
| Highly recommend Scite, I use it on a nearly daily basis when
| doing lit reviews for my job.
| darosati wrote:
| Thanks! I am glad you are finding it useful (I work at scite)
___________________________________________________________________
(page generated 2021-06-21 23:01 UTC)