[HN Gopher] The Absurdity of Peer Review: What the Pandemic Reve...
___________________________________________________________________
The Absurdity of Peer Review: What the Pandemic Revealed
Author : akvadrako
Score : 16 points
Date : 2021-06-06 21:18 UTC (1 hours ago)
(HTM) web link (elemental.medium.com)
(TXT) w3m dump (elemental.medium.com)
| Tycho wrote:
| I mean, seems pretty obvious, on the face of it, that enshrining
| a practice like peer review would lead to disingenuous gate-
| keeping, normalcy bias, intellectual sloppiness, politicisation,
| cliques, groupthink, obscurantism and nerdification.
| tinus_hn wrote:
| If anything the pandemic made clear that science absolutely
| requires peer review, otherwise opinions get elevated to facts
| and at the end the reputation of science itself is in shambles.
|
| 'It is ridiculous to think covid-19 could have originated in a
| lab'
|
| 'Covid-19 spreads through touching surfaces'
|
| 'Hydroxychloroquine cures covid-19'
|
| 'Hydroxychloroquine does not cure covid-19'
|
| 'Any opinions that go against this popular scientific stance are
| dangerous and should be blocked'
| makomk wrote:
| One of the worst hydroxychloroquine studies - the now-retracted
| one in the Lancet claiming that patients receiving the drug
| were dying at a higher rate, which appears to have used
| completely fake data and where there was no possible way they
| could've got the data they purported to have used - was in fact
| peer reviewed. Peer review really doesn't do as much as people
| seem to think it does.
| AwaAwa wrote:
| The problem is that "peers" of a like mind are easily found,
| when churning papers is rewarded. Quantity beats Quality in
| today's world.
| xg15 wrote:
| So then what? Even if peer review does not solve the problem
| of bad research completely, it certainly mitigates it better
| than not doing anything.
|
| How would you solve this problem?
| TigeriusKirk wrote:
| By attributing less weight to the imprimatur of peer
| review.
|
| It's a sign that something might well hold up, but it's not
| definitive. I think people know this about their own field,
| and then immediately forget it when dealing with another
| field.
| User23 wrote:
| > Even if peer review does not solve the problem of bad
| research completely, it certainly mitigates it better than
| not doing anything.
|
| This isn't self-evident. In fact, it's conceivable that
| peer review could actually lower the quality of published
| research by delaying paradigm shifting work from seeing the
| light of day.
| IncRnd wrote:
| How? If someone is going to rely upon the conclusions of a
| paper, simply read that paper and evaluate the claims. i.e.
| stop the sole use of the titles of papers and their cliffs
| notes to support a point.
| throawayu7748 wrote:
| _It is ridiculous to think covid-19 could have originated in a
| lab'_
|
| Have there been any peer-reviewed studies that unambiguously
| _admit_ the possibility that Covid-19 may have originated in a
| lab? Although I think it 's something that needs serious
| consideration, the alleged lack of Medical Reliable Sources
| (basically medical journal articles) is being used aggressively
| by a small group of editors on Wikipedia to enforce the line
| that a lab leak is "conspiracy theory".
| B1FF_PSUVM wrote:
| > only one of Einstein's 300 or so published papers was ever
| peer-reviewed, which so disgusted him that he never submitted a
| paper to that journal again
|
| I never thought of this, and it is one of those shifts in habits
| and uses that go unremarked (until, of course, someone points
| them out like this piece did).
|
| Perhaps due to the industrialization of research after the middle
| of the XX century?
|
| Now that we don't really need to publish in journals, since
| anything can be made available on the internet, the filter of
| credibility can also be changed - for instance to something like
| being cited approvingly by other reputable researchers.
| jdikatz wrote:
| Even if peer review doesn't _actually_ signal paper quality, it
| 's still useful for science journalists to note whether a paper
| is in its final form or still a work in progress. Referring to
| unpublished papers as "pre-prints" or "working papers" could be a
| less biased way to do so.
|
| Also, it's hard to tell whether peer review actually improves
| paper quality by comparing published / unpublished papers in a
| world _with_ peer review, where everyone is writing with the
| knowledge they 'll be intensely scrutinized. Without some sort of
| detailed review process -- even one that's potentially deeply
| flawed -- researchers would have fewer incentives to be careful.
| xg15 wrote:
| Ok, so what's the alternative to peer review then? We'll "just
| know" when that paper/blog entry/tweet/4chan post is written by
| the next Einstein and we can take it for a fact?
| sharken wrote:
| There really isn't any good alternative, having one or more
| qualified peers go over the results is a lot more trustworthy.
|
| Of course it requires that there isn't any hidden agenda or
| that a specific result of the review is dictated.
| rossdavidh wrote:
| Peer review is necessary. Peer review is totally failing to do
| its job. There is nothing in the current structure of peer review
| which would suggest that it could be expected to do its job.
| Discuss.
| prof-dr-ir wrote:
| This is a timely article about an important issue in the
| sciences.
|
| The author begins by discussing the importance given in popular
| media to whether articles are peer reviewed or not, using
| somewhat absurd examples of foundational historical papers to
| stress the point.
|
| The author then makes a logical jump, and claims that "Science
| worships peer review". However neither the journal Science nor
| the discipline of the same name are capable of worshipping
| anything. The author might mean to say that "Scientists worship
| peer review"? But for such a claim the author provides little
| evidence, and further it would contradict the later statement
| that "simple inertia, and ..." are the reasons "we still have
| peer review".
|
| Subsequently the author tries to demonstrate that peer review is
| useless by stressing the multitude of problematic papers that did
| make it through peer review. This, of course, is a standard
| logical fallacy and again in direct contradiction with the latter
| statement that "we _do not know_ whether the science we produce
| is better with or without it " (emphasis mine).
|
| As I mentioned above, it is clear that the author raises an
| important point and at its root the message it tries to convey is
| certainly valid. However the article itself adds little to what
| scientists (and I would wager the informed public) already know.
| Besides the issues mentions above, I think a further weakness is
| that it claims to speak for all of science but I might imagine
| that in a field like mathematics the act of peer review can be
| more meaningful.
|
| Altogether, then, I am of the opinion that the above issues need
| to be corrected in a substantial revision before I can accept the
| article for publication in your journal.
|
| Sincerely,
| version_five wrote:
| I did my Ph.D in a relatively niche area of physics. We had
| conferences, attendeed by a handful of people from all over the
| world, and presented and discussed our research. None of this was
| "peer reviewed" in the sense that it had undergone a third party
| review like you'd have for paper or conference submission. But
| the whole point of the conferences was peer review. This is what
| I expect mostly happened in the days of Einstein and Watson. Just
| because there was not an editor assigning three reviewers,
| doesn't mean their work wasn't scrutinized by their peers.
|
| Now extrapolating this outside of niche science and into research
| that is underpinning policy decisions (or tenure and promotion
| decisions), of course one would expect it to undergo some kind of
| review. And if something had not, and was just a preprint being
| circulated, it's a pretty important consideration that its not
| yet part of established canon.
|
| Certainly I agree there are lots of flaws with peer review, that
| a system that originally evolved to build out the body of
| accepted knowledge in a field had now been formalized and used
| for other purposes, and that it can be gamed. But it don't find
| it credible to say that we just don't need it, and that we
| shouldn't distinguish between new results and better scrutinized
| ones.
___________________________________________________________________
(page generated 2021-06-06 23:01 UTC)