Post ARKhHVLKWE5G8E8PdA by moritz_negwer@mstdn.science
(DIR) More posts by moritz_negwer@mstdn.science
(DIR) Post #ARKVhX2lcTnNU962bY by tiago@social.skewed.de
2023-01-05T10:18:56Z
0 likes, 1 repeats
Taking a closer look at this paper, I really dislike it.https://www.nature.com/articles/s41586-022-05543-xIt's clear that newer papers *must* have a lower “disruption” score than older ones under a null model — they even confirm this in the supplemental material with a randomization test.When comparing with the null model they compute only the z-score, getting values at most 4 or so. It's also besides the point — as usual, small meaningless deviations from the null model can be statistically “significant.” Effect size ≠ statistical significance.Finally, according to their definition, review papers would be “disruptive” because they funnel a bunch of citations. And a paper that does not cite anyone but is universally cited would not be “disruptive”. 🤷@networkscience @academicchatter #networkscience #networks
(DIR) Post #ARKfJYr8EFuNhJrXyC by elduvelle@neuromatch.social
2023-01-05T12:05:59Z
0 likes, 0 repeats
@tiago @networkscience @academicchatter Interesting points, thank you for sharing! I need to read it.. I wonder what @LaurelineLogiaco thinks of it.
(DIR) Post #ARKhHVLKWE5G8E8PdA by moritz_negwer@mstdn.science
2023-01-05T12:28:38Z
0 likes, 0 repeats
@tiago @networkscience @academicchatter Interesting take, thanks for sharing. So for example, could the synchronized drop across all fields from the mid-90s (in Fig 2a) be due to longer citation lists, with papers being easier to find (internet) and track (citation software)?I'd imagine that the inclusion of more (marginally) relevant papers would reduce the "disruption" measure used here.
(DIR) Post #ARKu374fFfEKDHo7fM by rupertoverall@fosstodon.org
2023-01-05T14:51:40Z
0 likes, 0 repeats
@tiago @networkscience @academicchatter I wonder how this score is influenced by the length of (allowed) citation lists vs. the size of the literature in the field. There are often so many potential citations (massively increasing over time) but the author is restricted to selecting a 'representative' few.
(DIR) Post #ARLFBHGJM5zVAFzPN2 by tiago@social.skewed.de
2023-01-05T18:48:31Z
0 likes, 0 repeats
@networkscience @academicchatter Some people are asking why I think a null model explains the effect seen. The idea is very simple: The quantity they use to quantify how disruptive a paper is depends on how many citations to a "focal" paper also cite older papers that the focal paper also cites. Thus, if a citing paper "bypasses" the focal paper in the citation graph, it is considered less disruptive.However, a newly published paper can cite an arbitrarily large number of papers that can also be arbitrarily old, but on the other hand they can only be cited by a relatively much smaller number of newer papers, which at the moment of publication is exactly zero.Therefore, if we consider a null model where papers cite each other at random, with the only constraint that newer papers can only cite older ones (a scenario in which we could not claim that the nature of publication patterns is changing), then the probability that a citing paper will bypass the focal paper will be larger for newer papers than older ones. This is because newer papers can cite much older ones, which tend to attract more citations just by virtue of being old.(On top of this, citation dynamics are known to be subject to a preferential attachment — or rich get richer — dynamics, where the more cited a paper is, the larger is its probability of being cited in the future. This means that a few older papers tend to be much more cited than newer ones, and therefore they will attract even more of these "bypass" citations.)Because of the above, newer papers *must* have a lower score than older ones, just due to basic probabilistic reasons. It does not mean necessarily that the nature of research or publication is changing.
(DIR) Post #ARLG0yxMezmhRtmXwW by tiago@social.skewed.de
2023-01-05T18:57:51Z
0 likes, 0 repeats
@rupertoverall @networkscience @academicchatter It's related to how old a paper is. A new paper can cite works that are arbitrarily old or new, but this is not true for older papers which can cite only even older papers. I explain in more detail here: https://social.skewed.de/@tiago/109638091509211304
(DIR) Post #ARLGnCouQ1cuLWp0sa by nadlerlab@mstdn.science
2023-01-05T19:06:31Z
0 likes, 0 repeats
@tiago @networkscience @academicchatter Thank god I am not the only one... There's another thing: Have a look at the weird spikes at discrete values (1, 0.5 and so on) in the distribution in Figure 1... I think that papers with very low citation counts contribute heavily to the tails of the CDt distribution. The maximally disruptive papers are probably manuscripts that acrued one total citation...
(DIR) Post #ARLJDj2jR9avin3p20 by aledaus@neuromatch.social
2023-01-05T19:33:04Z
0 likes, 0 repeats
@tiago @rupertoverall @networkscience @academicchatter What you say makes a lot of sense.I wonder if this "recency" effect (or artifact) fades out when moving back in the past.Anyway, are there other "sociological" aspects we may extrapolate?For instance, fig.2a, Tech and Phys show huge CD decrease after the 60ies. Might that be due to the massive investments in basic sci-tech research driven by the war and immediate post war conditions??Biomed and Social seem to follow a more gradual pattern...Also, what happened to "language" use after the 70ies (fig3a)?? Did we stop creating new labels for the same thing?
(DIR) Post #ARP29MjFXOFelh7gTw by bwyble@neuromatch.social
2023-01-07T14:40:35Z
0 likes, 0 repeats
@tiago @networkscience @academicchatter I had wondered if changes in citation practices might be the cause of the trend.
(DIR) Post #ARRwi8ISpdnLxorwPo by ncrav@mas.to
2023-01-09T00:24:31Z
0 likes, 0 repeats
@tiagoThis definition of disruptive seems strangely associated with using citations as a metric for quality. Yet a disruptive paper - in the sense of moving beyond perceived bounds - would have by definition less material to cite from (otherwise it wouldn't be disruptive at all).@networkscience @academicchatter