[HN Gopher] Are Experts Real?
___________________________________________________________________
Are Experts Real?
Author : tlb
Score : 103 points
Date : 2021-01-11 15:17 UTC (6 hours ago)
(HTM) web link (fantasticanachronism.com)
(TXT) w3m dump (fantasticanachronism.com)
| hikerclimber wrote:
| i don't think so... you can gain expertise in anything.
| dragontamer wrote:
| Yes, experts are real.
|
| Unfortunately, experts are only expert in a few subjects, and
| experts (well... all humans even) are bad at knowing when they've
| left their field of expertise.
|
| Linus Torvalds for example is clearly an expert programmer. And
| then suddenly he talks about AVX512 and I wonder if he knows
| anything about low-level programming or optimization. (https://ww
| w.realworldtech.com/forum/?threadid=193189&curpost...).
|
| Case in point: with Rocket lake (on 14nm nonetheless), the power-
| limits associated with AVX512 have already been largely lifted.
| It didn't even take a few months before Linus's words became
| obsolete.
|
| https://travisdowns.github.io/blog/2020/08/19/icl-avx512-fre...
|
| 100MHz difference single-core, 0MHz difference multicore.
| Virtually no penalty for using AVX512 on new systems. Linus was
| ultimately trapped in the Skylake-X way of thinking, and not
| looking forward to the next chips.
| rhino369 wrote:
| Expert opinion isn't really worth anything until it's been
| stress tested by other experts. I worked with experts in the
| context of patent and trade secret litigation. I pick people at
| the top of their field from prestigious universities with
| impeccable credentials.
|
| If you ask them off the cuff questions they often are wrong.
| But if they investigate and think about it, they'll correct
| themselves.
|
| Experts are also just as susceptible to bias, motivated
| reasoning, groupthink, etc. as anyone else.
|
| This is why academic papers are supposed to be a back and forth
| discussion. "TRUST THE EXPERTS" has a plural for reason, and
| should be understood to mean trust the experts after they've
| done a thoughtful analysis with the best available evidence and
| have considered dissenting expert views.
| zug_zug wrote:
| Did you read the article? The article is specifically talking
| about cases where experts are incorrect _within_ their field of
| expertise and then cited as a source of truth in published
| papers.
|
| The question of the article isn't really a boolean, it clearly
| shows some experts are real and some aren't and asks the more
| important question -- how do we discern when the
| institutionalized "expert" in an area is really an expert?
| dragontamer wrote:
| > Did you read the article? The article is specifically
| talking about cases where experts are incorrect within their
| field of expertise and then cited as a source of truth in
| published papers.
|
| Yes.
|
| But I still think that Linus is a good character to discuss
| within the scope of the article. Linus is clearly considered
| an expert, and many people follow his bombastic words
| (retweeting, or using them to form the basis of their
| argument around here).
|
| "Because Linus said so" is a common way to end various
| arguments. Linus's opinion is truly "expert", people copy it
| without further analysis because... well... Linus kinda
| deserves it. He is an expert after all.
|
| It doesn't mean Linus is always right.
|
| --------
|
| The "feedback loop" of the open source world is pretty quick.
| If you're a bad programmer, people figure it out when they
| start working with your code.
|
| But the "feedback loop" of "chip ISA opinions" is slow.
| There's not really any way to prove, or disprove, whether or
| not AVX512 is a good or bad idea.
|
| Linus's expertise in programming doesn't necessarily
| translate to chip ISA design. And as such, Linus can get
| details (expected power-characteristics of future chips)
| dramatically wrong.
| vlovich123 wrote:
| I think you're over-focusing on the AVX512 argument. You
| even said yourself it was a product release that happened
| months after he made the comments. Was he supposed to
| understand what changes Intel made to AVX512 before they
| announced it? Also, he's not even talking about clock
| downscaling. He's saying that AVX512 is the wrong way to
| get significantly more performance vs the amount of space
| and power it consumes (definitely true for most
| applications and 100% true for kernel work) and that Intel
| would be better served focused on general perf (which isn't
| wrong given how they've been trounced by AMD and Apple).
|
| Consider also that Linus worked at Transmetta for a
| relatively long time trying to build a CPU company. I would
| trust his opinions on chip ISA design more than most
| software developers.
|
| You've basically taken his post out of context,
| misinterpreted it to be saying something he isn't, and
| claiming it's wrong because some details have changed on
| the ground _after_ he made it.
|
| This is the problem with expert opinion. It's so easy to
| misinterpret. It's also possible to get some details wrong
| without invalidating the larger point. Figuring out when
| the expert is wrong is hard, but claiming that Linus
| doesn't understand chip design just factually doesn't
| understand the context of Linus' perspective and his actual
| professional experience.
| dragontamer wrote:
| > Also, he's not even talking about clock downscaling
|
| Quote the post in question:
|
| > not with some AVX512 power virus that takes away top
| frequency (because people ended up using it for memcpy!)
|
| This is pretty obviously talking about the down-clocking
| issue.
|
| > You've basically taken his post out of context,
| misinterpreted it to be saying something he isn't, and
| claiming it's wrong because some details have changed on
| the ground after he made it.
|
| AVX on Sandy Bridge was worthless: it wasn't any faster
| than SSE because Sandy Bridge was still 128-bit (even on
| AVX code).
|
| Anyone who has followed ISA-development for decades would
| recognize that the first implementation of an ISA is
| often sub-par and sub-optimal. It wasn't until Haswell
| before AVX was faster than SSE in practice.
|
| AVX512 was first implemented on Xeon Phi. Then it was
| ported over to servers in Skylake-X. AVX512 from there on
| remained a "1st generation" product for a few years (much
| like AVX on Sandy Bridge, before Haswell). From the
| history of these ISA-extensions, we generally can expect
| downclocking issues or inefficiencies in the first few
| implementations.
|
| -----
|
| In fact, some portions of AVX still remain suboptimal.
| vgather for example, isn't really much faster than
| standard load/stores. Maybe they'll be suboptimal
| forever, or maybe some hypothetical future CPU could
| optimize it.
|
| Either way, I know that I shouldn't make an argument
| based on the "stupidity of vgather" instruction. There's
| an "obvious" optimization that could happen if that
| instruction becomes popular enough to deserve the
| transistors. At best, if I were to argue about vgather
| instructions, I'd preface it with "current
| implementations of vgather on Skylake-X", or similar
| wording.
|
| CPUs change over time, much like programming languages or
| compilers. I expect a degree of forward-looking ability
| from experts, not for someone to get lost in the details
| of current implementations.
| vlovich123 wrote:
| I don't think I've ever met an engineer who's a domain
| expert who talks like that. Unless you have actual
| specific information about future plans and reason to
| believe in them, expecting everyone to qualify their
| prognostication with "this generation but it might change
| in some hypothetical future" is just not useful. That's
| generally true of all technical topics. What's stated
| today can change in the future. These aren't physical
| limits being discussed.
|
| Finally, Linus' main advice is to stop focusing on AVX
| and focus on architectural improvements. It took an
| enormous amount of engineers to implement AVX512 and then
| even more to get rid of the power hit. I trust Linus (and
| back that up with my own experience and observations in
| the industry) that Intel would have been far better
| served focusing on architectural improvements like Apple
| did. It's a blind spot Intel has because they know how to
| bump clocks and create x86 extensions but they don't
| trust architectural improvements to deliver massive
| gains. Hell, this isn't the first time. Netburst (which
| was their attempt) screwed up so badly they had to return
| to the P3 architecture with the Core line. Intel has the
| engineers but their management structure must be so
| bloated an inefficient to fail with such regularity.
| dragontamer wrote:
| > I don't think I've ever met an engineer who's a domain
| expert who talks like that. Unless you have actual
| specific information about future plans and reason to
| believe in them, expecting everyone to qualify their
| prognostication with "this generation but it might change
| in some hypothetical future" is just not useful. That's
| generally true of all technical topics. What's stated
| today can change in the future. These aren't physical
| limits being discussed.
|
| If I, 5-years or 10-years ago, claimed ARM sucks because
| its in-order, then I'd be making a similar mistake as
| Linus did. There's nothing about the ARM-ISA that forced
| in-order execution, it was just the norm (because back
| then: it was considered more power-efficient to make in-
| order chips).
|
| There's nothing part of AVX512 that says "You must use
| more power than other instructions". That was just how it
| was implemented in Skylake-X. And lo-and-behold, one
| microarchitecture later, the power-issue is solved and a
| non-issue.
|
| Similarly, ARM chips over the past decade switched to
| out-of-order style for higher performance. Anyone who was
| claiming that ARM was innately in-order would have been
| proven wrong.
|
| ----------
|
| Maybe it takes experience to see the ebbs and flows of
| the processor world. But... that's what I'd expect an
| "expert" to be. Someone who can kind of look forward
| through that.
|
| I mean, I've watched NVidia GPUs go superscalar
| (executing INT + FP simultaneously). I've watched ARM go
| from largely in-order microcontrollers into out-of-order
| laptop-class processors. I've watched instruction sets
| come and go with the winds.
|
| ------
|
| If that's not good enough for you, then remember that
| Centaur implemented AVX512 WITHOUT power-scaling issues
| back in 2019, still predating Linus's rant by months. A
| chip did exist without AVX512 power throttling, and he
| still ranted in that manner.
| exmadscientist wrote:
| If you read his argument as less about the power virus effect
| and more about the uselessness of AVX512 for most general-
| purpose integer code, and accordingly about the opportunity
| cost of developing and fabricating AVX512 for clients who
| really will not use it, then, unfortunately, he's _right_.
|
| The quote "I'd _much_ rather see that transistor budget used on
| other things that are much more relevant, " in particular, is
| the key to the argument. Because that seems to be basically
| exactly what the Apple M1 does, and look what people are saying
| about _it_.
| dragontamer wrote:
| But Linus is talking about power-budget. And Ice-lake /
| Rocket lake are proving that AVX512 is well within the power-
| budget of these chips.
|
| M1 dedicates more chip-area to integer units, and also uses
| less power (thanks to the 5nm advantage). So what's going on
| with M1 isn't really a power-issue at all.
|
| When Linus says things like:
|
| > I want my power limits to be reached with regular integer
| code, not with some AVX512 power virus that takes away top
| frequency (because people ended up using it for memcpy!) and
| takes away cores (because those useless garbage units take up
| space).
|
| Its clear that he was unable to be forward thinking. He was
| unable to see that a future chip, like Icelake or Rocket
| Lake, could fix the AVX512 power-throttling so dramatically.
| (without even going to an advanced node !!!).
| topspin wrote:
| > But Linus is talking about power-budget.
|
| Linus covered a lot of ground in that message. He was
| correct on his history, whether that is history you'd like
| to remember or not. His transistor budget argument is very
| plausible, but without playing out an alternative history
| we can't know, and by we I mean you. At the time he wrote
| it he was correct on the power 'virus,' and I can't think
| of any good reason why he or anyone else owes Intel the
| benefit of the doubt that some future device would improve
| anything.
|
| The AVX512 fanboi contingent will forever seethe about what
| Linus wrote. Meanwhile Apple and AMD are eating Intel's
| lunch with no 512 bit SIMD instruction units at all. Linus
| had it right and and he is looking more correct every day.
| dragontamer wrote:
| > The AVX512 fanboi contingent
|
| Just so you know... I'm an AMD Threadripper fanboy, if
| anything. Since that's my personal computer. More
| importantly though, I'm a technologist. I experiment and
| play with many technologies, including SIMD compute, when
| I can.
|
| Either way, I don't think flamebait comments like that
| are very helpful for discussion. But I'll try to focus on
| the crux of your argument instead...
|
| > Meanwhile Apple and AMD are eating Intel's lunch with
| no 512 bit SIMD instruction units at all
|
| Apple has 4x 128-bit Multiply-and-accumulate per-clock-
| tick on its M1. AMD has 2+2 multiply and add x 256-bit
| SIMD.
|
| Those processors are actually very beefy in terms of
| SIMD-implementation.
|
| There's also the Fujitsu A64Fx, which is an ARM-
| implementation of 512-bit SVE. In fact, 512-bit ARM-SVE
| is about to be widely-deployed on Neoverse.
|
| SIMD-units are universally expanding across the industry.
| One of AMD Threadripper's biggest leaps forward from Zen1
| -> Zen2 was the upgrade from 2+2 128-bit SIMD to 2+2
| 256-SIMD (doubling its throughput).
|
| Intel is trying to stay ahead of the curve with 512, and
| they're really not that far ahead. Indeed: ARM's SVE is
| already caught up.
|
| -------
|
| NVidia's 32-wide SIMT is basically a 1024-bit
| implementation, while AMD's 64-wide SIMD is 2048-bit
| SIMD. With all of the "Deep Learning" going on, its
| actually a very important workload for a number of
| companies.
|
| Linus is pretty much saying that Deep Learning /
| Tensorflow / etc. etc. isn't important. Which is...
| backwards thinking methinks. I think that deep learning
| is overhyped, but still I see the benefits of general
| SIMD compute in every-day coding situations.
| cbozeman wrote:
| AVX seems to me to be a band-aid to the instruction length
| problem inherent in x86 CISC systems vs. RISC chips like M1.
|
| Then again, maybe the fact that the memory and GPU are all
| integrated into M1 also helps propel it over the top.
|
| How well would an AMD / Intel SoC perform if you threw a
| 5950X / 10900K, 16 - 128GB of LPDDR4X-4133, and a GTX 3090
| into a blender and baked up an enormous die with a huge
| watercooled solution?
|
| I dunno. I want to believe its the architecture that's
| succeeding, not the SoC by virtue of being an SoC, but I
| don't have the knowledge or ability to judge that.
| dragontamer wrote:
| > AVX seems to me to be a band-aid to the instruction
| length problem inherent in x86 CISC systems vs. RISC chips
| like M1.
|
| AVX is neither CISC nor RISC. Its SIMD.
|
| We're talking about 1970s Cray-systems or 1980s Connection
| Machine (CM-2 or CM-5). Its a completely different paradigm
| of compute. An entire history of computing that runs
| parallel to the CISC vs RISC wars.
|
| > Then again, maybe the fact that the memory and GPU are
| all integrated into M1 also helps propel it over the top.
|
| M1 has an awesomely wide 128-bit x4 pipeline SIMD execution
| backend. Apple is making sure its SIMD unit (for NEON ARM
| instructions) doesn't fall dramatically behind.
|
| M1 can perform 4x 128-bit Multiply-and-Accumulate
| instructions per clock tick. Its no slouch.
|
| AVX512 is better than ARM Neon on M1, but even Apple sees
| the importance of SIMD compute.
|
| The ARM world is talking about 512-bit SVE instructions.
| SIMD compute is obscure and niche: but its a parallel niche
| computation style with decades of history. Its the proper
| solution to a huge host of computational problems.
| cbozeman wrote:
| Wow, fascinating! Thank you for the reply. :)
| dragontamer wrote:
| It should be noted that SIMD-on-CPU is not to be the
| fastest SIMD in the world... but instead the closest SIMD
| to your data.
|
| SIMD today largely exists in the realm of GPUs: the
| NVidia 3090 is a dedicated SIMD-architecture for example.
| But as an "external" device on a PCIe 4.0 port, your CPU
| can only stream 30GBps worth of data to it (the max-
| bandwidth of PCIe 4.0 x16)
|
| The AVX512-units inside of an Intel chip is the same GPU-
| SIMD architecture, but smaller. More importantly, it
| shares L1 cache, and therefore has TBps worth of data-
| sharing.
|
| As such, AVX512 becomes useful if you need a fast CPU to
| calculate some things, and then switch off to SIMD-
| paradigm for some reason.
|
| ---------
|
| What's a good example of this "switchoff" ?? Well,
| Stockfish-nnue (now default in Stockfish 12) is a great
| example.
|
| You might know that SIMD-compute, in particular GPUs, are
| the one of the best architectures for calculating neural
| nets. But GPUs are terrible with branchy-code (which
| happens often in chess: "if(bishop) then (calculate
| bishop moves)" is rather difficult to do efficiently on a
| GPU! Its hard to explain why, but its called "branch
| divergence". Look it up if you feel like it).
|
| Anyway, a CPU is best at the "if(bishop)" kind of
| computation. But GPUs are best at neural nets.
|
| Stockfish-nnue beats Leela-Zero, by using AVX512 on a
| small-and-lightweight neuralnet, but still uses the CPU
| for the fast if() statements.
|
| This is only made possible because AVX512 and the rest of
| the CPU shares the same L1 / L2 cache, meaning you can
| get TBps worth of data-sharing, instead of GBs if you
| went "off-chip" to another computer (like calling a GPU-
| to calculate a neural net).
|
| Furthermore, the NNUE neural net in Stockfish is composed
| of if-statements: only PART of the neural net updates,
| which means it can never be efficiently implemented on a
| GPU. Only something with fast if() statements can
| implement NNUE. (EDIT: Well... maybe not "never", but
| it'd be non-obvious to code up a solution. I think its
| possible, now that I think of it... but its not done in
| typical GPU code these days)
|
| GPUs / classical SIMD compute want all of its data to
| update uniformly, without if() statements mucking up the
| logic. CPUs have giant branch-prediction and
| hyperthreading and other such features to speed up if-
| statements.
|
| -----
|
| A small SIMD for small SIMD-calculations, that combines
| with fast CPU-style branch-heavy code. A nightmare for
| GPUs to implement. Its something only possible with CPU +
| SIMD instruction assist.
|
| ------
|
| GPUs are likely still best for large-scale SIMD. Like
| giant matrix multiplications (or many matrix-
| multiplications, like in video games). Where if-
| statements are less common.
| [deleted]
| rusticpenn wrote:
| Regardless of the quality of knowledge, people with better social
| and networking skills will become better known than those
| without.
| TheTrotters wrote:
| Among the general public? Definitely.
|
| Social and networking skills also matter within your
| professional community but they are not a substitute for
| publishing in top journals/conferences.
| WC3w6pXxgGd wrote:
| Great article. I have had many debates with people within the
| last year about the concept of "experts," and have frequently
| found it almost impossible to break through the myth that
|
| 1. Experts exist, and you should believe everything they say.
|
| 2. Expertise is fungible, so an "expert" in one small sliver of
| healthcare is also somehow an overall expert in healthcare.
|
| An example of 2 was a conversation I had recently with someone in
| the healthcare industry who, in this single conversation, said
|
| - Cuba has better healthcare than the US
|
| - All doctors would be fine with the State forcing them to
| provide their services because doctors are not profit-driven and
| only care about The Greater Good.
| tboyd47 wrote:
| Great article.
|
| > Do you have it in you to ignore 800 epidemiologists, and is it
| actually good idea?
|
| Yes, I do, because I was tracking the wave of non-stop bullshit
| coming from scientists closely throughout all of 2020, and got
| completely fed up. But I'm not the norm (yet), and no, it's not
| actually a good solution.
|
| We're in the beginning of the story of the boy who cried wolf.
| Eventually the whole town ended up like me: fully willing to
| ignore and oppose the mainstream information channels. When a
| real wolf came, the boy told the truth once out of 1,000 times,
| and the townspeople all ignored him and got their sheep eaten up.
| We need to find a way to crowd-source our trust mechanism if
| "trust the experts" starts to fail.
|
| My suggestion is that any "expert" who volunteers an opinion that
| later gets falsified should be canceled permanently from the
| public sphere, with no forgiveness. They can go on publishing in
| scientific journals and selling books, but their opinion should
| not be accepted by the general public anymore.
|
| The other solution is to keep ignoring the boy and start watching
| your own sheep.
| moonshinefe wrote:
| So what main issues are you talking about that were "bullshit"
| advice from scientists in general exactly? You make a lot of
| super vague arguments with no specifics, and while it's nice to
| agree with children's fables like the boy who cried wolf,
| unless we agree on facts that happened, it's kind of a
| rhetorical argument with no real bearing in reality.
|
| Also, I've got to say, "cancelling" anyone who's ever wrong
| from being an "expert" in the public sphere sounds like a
| really poor idea. Everyone is wrong sometimes and that's a
| crazy standard which would have tons of its own perverse
| incentives, probably even worse than the ones scientific
| journals and the like have now. Do you want scientists on the
| record terrified of saying anything besides the most vague and
| unfalsifiable statements? It's an untenable situation if you
| bring it to its logical conclusion. I do agree with you that
| there ought to be more consequences for scientists willingly
| spreading bad science though, beyond what there is now--but the
| solution won't be simple.
|
| Given the vagueness of your post I'm not sure why you have the
| perspective that scientists produced a "non-stop wave of
| bullshit". From my perspective, I saw a bunch of demagogues and
| populists cynically manipulating a major demographic for
| personal gain while sidelining and ignoring the broader
| scientific advice and recommendations in the country the entire
| time, while over 350000 people died.
| [deleted]
| 13415 wrote:
| I'm baffled by your post because I honestly have made the
| opposite experience. I've been tracking expert advice on Covid
| and practically all experts have been right about almost
| everything related to the disease from the start of February
| until now. They even predicted the reactions of the population
| correctly over time, and the second wave in autumn in my
| region. There were minor disagreements about measures, but
| overall mostly agreement, and practically every expert has been
| and is still advising more stringent measures than the
| governments end up implementing. (Which is not surprising,
| given the different responsibilities at play.)
|
| I felt not only well-informed, I actually felt and still feel a
| bit over-informed about the topic. My country of residence
| handles the crisis relatively well, so it's kind of silly and
| depressing to me to continue to collect data and look at daily
| Covid figures.
|
| May I ask where you live? Maybe the US? Did you get your
| information mostly from print media, online media, or TV?
| tboyd47 wrote:
| > May I ask where you live? Maybe the US?
|
| How'd you guess? :)
| 13415 wrote:
| That explains the different experiences, I live in
| continental Europe and mostly followed local expert
| interviews and journal articles. It seems to me that the
| topic was politicized early in the US. I've followed
| Fauci's meandering diplomacy from time to time, as well as
| disagreements between federal and local responses. Maybe
| some trade-offs were made.
|
| There is one complaint I had about experts in Europe, too,
| and that is that they resorted to a "king's lie" about
| masks at the beginning of the pandemic. Certainly none of
| them said you shouldn't wear a mask, but they downplayed
| their importance when there weren't enough of them yet for
| the medical staff. IMHO, they shouldn't have done that.
| Instead, governments should have regulated mask trade and
| distribution from the start like in some Asian countries.
| tboyd47 wrote:
| > I've followed Fauci's meandering diplomacy from time to
| time, as well as disagreements between federal and local
| responses.
|
| It speaks volumes that you identified what country I'm
| from and the person I was really talking about just from
| a vague rant about "scientists."
|
| My rant may have been aimed at one or two people who
| appear to be compulsive liars at this point, but the
| author of the post is complaining about a problem with
| science in general. And it's not something you can
| attribute to anti-intellectualism or the perception of
| science; it really is a 'dereliction of duty" on behalf
| of scientists, and the madness of 2020 exposed these
| problems to the general public. More on that problem
| here: https://www.youtube.com/watch?v=7leWXMr3ayk
|
| As an aside, I would ask those people (not you) who
| blamed me for feeling this way if they really think
| browbeating people who speak out will fix the issue. Why
| do we all have to keep blaming each other? Why not work
| together to hold our experts to a higher standard?
| moonshinefe wrote:
| There wasn't a problem of the information not being there,
| there was a problem of 30-40% of the country believing angry
| demagogues over experts, and said demagogues strong arming
| the scientific community with political and social power.
| Even the most basic precautions like wearing a mask were
| politicized while the scientists practically begged for
| people to wear them. They left the WHO and refused to enter
| the vaccine pool. How do you fight that? Blame science? It's
| ridiculous and anyone claiming it's primarily science's fault
| I seriously question their good faith to debate here.
|
| I was here the entire crisis (and I'm not a US citizen btw)
| and had easy access to the same advice Canadians, Europeans,
| and whoever else in the West got as well. The information was
| here, but the political leaders often downplayed it or simply
| refused to follow it. What did we get? The numbers don't lie,
| and take a look at the results of the current political
| climate and the death count. This wasn't a "good way to go"
| being overly skeptical of scientists. Like, there are
| problems with science, but this is if we're being cute
| "throwing the baby out with the bathwater"--or if we're being
| less polite--so incredibly stupid it's largely contributing
| to the deaths of hundreds of thousands of people.
| JadeNB wrote:
| > My suggestion is that any "expert" who volunteers an opinion
| that later gets falsified should be canceled permanently from
| the public sphere, with no forgiveness.
|
| That will lead to one of two things: (1) information paralysis,
| where no-one authoritative is willing to speak in public for
| fear of making a mistake (and so the public sphere is ceded to
| the ignorant); or (2) constant churn, where experts are
| constantly removed from the public sphere (because everyone
| makes mistakes), and so we only hear from the people who
| haven't yet had enough experience to make a mistake.
| [deleted]
| tensor wrote:
| I suppose we are at a huge impasse as a society then. Because
| someone like me holds the opinion of non-experts as having zero
| value. I will not respect and do not want any policy being set
| by people with your attitude. I will continue to follow the
| science, and if I feel that people like you who ignore it are
| threatening my life, then I will do everything I can to stay
| far away from you and keep you far away from me.
|
| I'd be totally for splitting society into those who respect
| science and experts, and those who don't. I have no desire to
| live in your world.
| tboyd47 wrote:
| Why is the answer to make me the enemy just because I spoke
| my mind?
| cltby wrote:
| > I'd be totally for splitting society into those who respect
| science and experts, and those who don't
|
| Science not as a search for knowledge, but as a search for
| daddy figures to respect and obey. _Nothing_ is as harmful to
| progress as this blind devotion to expertise.
| cltby wrote:
| Many argue that the experts ultimately got it right (and are
| thus redeemed). I strongly disagree. Even when the public
| health types corrected course, their behavior left a bad taste
| in my mouth. There was no acknowledgement of having been wrong,
| no introspection on how their errors came to be, no discussion
| how they could be avoided in the future. Just aggressive
| enforcement of the new party line couched in appeals to
| authority. The whole enterprise reeks of blind dogmatism.
| tboyd47 wrote:
| > Many argue that the experts ultimately got it right (and
| are thus redeemed).
|
| Perhaps their opinion on that one matter they corrected was
| redeemed, but their general lack of understanding about the
| limits of their own knowledge was exposed, which is more
| damning for someone called an "expert."
| TheTrotters wrote:
| And in time-sensitive situations getting something
| "ultimately right" is akin to getting it wrong.
| ndiscussion wrote:
| Spot on, a broken clock _looks_ correct twice a day. I have
| zero confidence that they 're actually correct this time.
| DanielBMarkham wrote:
| This essay points out a pervasive problem that doesn't get much
| attention (aside from fear-mongering)
|
| Because it's a topic so easily prone to hyperbole, it helps if we
| can reduce it down to a few premises.
|
| - Science is science, but the _practice_ of science is a social
| system. People get tenures, jobs, promoted, invitation to speak,
| etc. by how well they do in the judgment of their peers
|
| - Political systems exist for political reasons, ie, if you don't
| have a reproducible, falsifiable test in a feedback loop
| somewhere, your career and goals are all about image. There's no
| alternative, either for you or your field of study (That doesn't
| mean other sciences aren't important or aren't really science.)
|
| - If somebody appears on TV or in the media playing the role of
| an expert, it is, by definition, because they wanted to appear on
| TV or in the media. The goals of being popular and asked about
| important questions and the goals of whatever your field are ...
| two different things. The reporter doesn't care who figured out
| X. They care who can bring excitement and emotion to talking
| about X.
|
| - All this leads directly to the problem of the average science
| news consumer: any communication you receive about science or
| research is the result of a lot of conflicting interests. "Being
| good science" is way down the list, sadly. I imagine the
| percentage of average readers who understand p-hacking is below
| 5%
|
| - None of this is a secret. The average news consumer knows it,
| even if they're not able to articulate it
|
| - Add in things like the reproducibility crisis, and we've got a
| huge problem as a species. We've got one flashlight to light our
| way in the darkness and it turns out there's a loose connection
| in there somewhere
| cbozeman wrote:
| This is a fantastic comment, and spot on I think. I concur with
| everything you've said.
| [deleted]
| breck wrote:
| The solution is to #abolishcopyright. If you don't allow people
| to exchange ideas freely, and you allow entities to control
| monopolies on distributing information, you are going to have a
| vastly suboptimal flow of information.
| ska wrote:
| > ...by definition, because they wanted to appear on TV or in
| the media
|
| I suspect this misses some nuance; there is a meaningful
| difference between people who appear on the media because they
| want exposure, and people who do it because they want their
| field of study to be well represented.
|
| For example there are people to accept interviews when they
| genuinely dislike the process, because they felt it was a duty
| that came along with their chair, etc. I doubt these are ever
| the most dynamic interviews, but they are common.
|
| I agree this doesn't really extend to the level of the same old
| bobble heads pulled up on prime time news TV, but don't thing
| you can paint all of the media exposure with this brush.
| sigotirandolas wrote:
| I see it as a spectrum, where to be an "expert" you need to
| both skilled in the field and a good communicator/politician.
|
| The problem is, in those areas where there's not a tight
| feedback loop as mentioned in the article, it's possible to
| build your way to the top by just being somewhat skilful in
| the field _and_ knowing how to appear confident, and how to
| (over)sell your work, and knowing when something is
| technically right but you should not say it because politics.
| And on your way to the top you 're going to out-compete those
| that are very skilful on the field, but don't have such good
| communication abilities.
| grawprog wrote:
| I think that comes down to self-proclaimed experts vs experts
| as decided by their peers.
|
| Many of the people I met who were considered experts and had
| a vast depth of knowledge never referred to themselves as
| experts and seemed do get embarrassed at the idea.
|
| Whereas, i've met several self-proclaimed experts who liked
| to talk a lot about how great they were, but never really
| showed the depths of knowledge the other kind of experts
| showed.
|
| At least from my experiences anyway in life.
| potta_coffee wrote:
| Peer incentives and desires can also be skewed / wrong.
| Just because there's some kind of consensus doesn't mean
| the conclusion is correct.
| DubiousPusher wrote:
| > I suspect this misses some nuance; there is a meaningful
| difference between people who appear on the media because
| they want exposure, and people who do it because they want
| their field of study to be well represented.
|
| I don't think there is a meaningful difference. First, I
| don't believe people can make choices in any kind of
| isolation. If you believe you are making a choice without
| your ego involved, you have simply deceived yourself.
| Congrats, you have a high functioning ego. Second, even if
| they could, audiences are not capable of concluding with any
| reasonable certainty which is motivating someone appearing on
| television. I don't mean that derisively, like people are
| stupid, I mean it is essentially beyond humans to do this.
|
| IMO, in the absence of actionable information, it is
| pointless to speculate. Basically, you cannot know it so it's
| not worth thinking about.
| ska wrote:
| I think I see your point - from the point of the viewer
| it's impossible to tell why someone has agreed to be
| interviewed.
|
| I guess my hypothesis is that you get better information
| from people whose reason for agreeing to interviews is not
| self promotion. If true, that's actionable.
| tpoacher wrote:
| I don't see how 'disliking the process' equates to 'not
| benefitting from the outcome'.
|
| I dislike interviews, but I like having a job.
| ska wrote:
| There isn't any direct benefit for many of the people I had
| in mind.
|
| Indirect benefits are harder to evaluate and will be nature
| involve a bunch of hand waving, but there is a huge
| difference between say holding a research chair, and having
| a consulting business as an "available expert".
| kurthr wrote:
| Well, since many of these "experts" are professors with
| tenure they can't really lose their job nor readily
| increase their salary. They might be able to more easily
| get funding for their research/students from nontraditional
| sources (e.g. industry grants), but many of them would not,
| because they value the estimation of their peers which
| might be (rightfully) hurt. Not all or even a majority even
| care about money or they wouldn't have chosen the
| profession. Some of this has changed with startup board
| positions, industry coalition funding, and academic
| promotional departments, but not quickly.
|
| All of that goes out the door with pundits and shills, who
| don't just appear once, but have chosen the field for
| fame/fortune... and for those who have largely chosen to
| sell their research topics/results for profit. However,
| those are relatively easy to spot by most viewers and
| journalists who care.
| moomin wrote:
| I think this argument conflates two things, one of which is true
| and slightly terrifying. The terrifying thing is how few
| scientists understand the statistical underpinnings of their
| studies. The conflation happens because he thinks this is
| stratified by field. In practice, there are plenty of
| sociologists with a decent grasp on this, and a truly astonishing
| number of biologists who don't. Don't believe me? Read the
| studies of biology papers and how many have serious problems in
| their statistical treatment.
|
| So we've got a serious problem with how science is being done.
| Probably doesn't affect whether or not expertise exists, or which
| fields it exists in. Unless we're going to start saying biology
| isn't real.
| dmitryminkovsky wrote:
| The Masks Fiasco of 2020 was indeed a noteworthy example of
| experts-gone-wild, but my favorite is still the Monty Hall
| problem [0], because unlike epidemiology where proof stems from
| evidence-based experimentation, the Monty Hall fiasco saw a wide
| range of experts in mathematics/statistics--a field where truth
| does not require experimentation but only logic-based proof--
| loudly and publicly denigrate a person with no expertise in their
| field, only to be wrong.
|
| But I think the an even more interesting question that this post
| does not cover is the role experts play in our culture. Neil
| Postman discusses experts at length in _Technopoly: The Surrender
| of Culture to Technology_ (1992), wherein he characterizes them
| as basically the shaman-priests who control the Overton window
| [1] in a world where morality and social acceptability no longer
| stem from things like religion:
|
| > ...expertise is a second important technical means by which
| Technopoly strives furiously to control information. There have,
| of course, always been experts, even in tool-using cultures. The
| pyramids, Roman roads, the Strasbourg Cathedral, could hardly
| have been built without experts. But the expert in Technopoly has
| two characteristics that distinguish him or her from experts of
| the past. First, Technopoly's experts tend to be ignorant about
| any matter not directly related to their specialized area. The
| average psychotherapist, for example, barely has even superficial
| knowledge of literature, philosophy, social history, art,
| religion, and biology, and is not expected to have such
| knowledge. Second, like bureaucracy itself (with which an expert
| may or may not be connected), Technopoly's experts claim dominion
| not only over technical matters but also over social,
| psychological, and moral affairs. In the United States, we have
| experts in how to raise children, how to educate them, how to be
| lovable, how to make love, how to influence people, how to make
| friends. There is no aspect of human relations that has not been
| technicalized and therefore relegated to the control of experts.
|
| ...
|
| > The role of the expert is to concentrate on one field of
| knowledge, sift through all that is available, eliminate that
| which has no bearing on a problem, and use what is left to assist
| in solving a problem. This process works fairly well in
| situations where only a technical solution is required and there
| is no conflict with human purposes--for example, in space
| rocketry or the construction of a sewer system. It works less
| well in situations where technical requirements may conflict with
| human purposes, as in medicine or architecture. And it is
| disastrous when applied to situations that cannot be solved by
| technical means and where efficiency is usually irrelevant, such
| as in education, law, family life, and problems of personal
| maladjustment. I assume I do not need to convince the reader that
| there are no experts--there can be no experts--in child-rearing
| and lovemaking and friend-making. All of this is a figment of the
| Technopolist's imagination, made plausible by the use of
| technical machinery, without which the expert would be totally
| disarmed and exposed as an intruder and an ignoramus.
|
| > Technical machinery is essential to both the bureaucrat and the
| expert, and may be regarded as a third mechanism of information
| control. I do not have in mind such "hard" technologies as the
| computer--which must, in any case, be treated separately, since
| it embodies all that Technopoly stands for. I have in mind
| "softer" technologies such as IQ tests, SATs, standardized forms,
| taxonomies, and opinion polls. Some of these I discuss in detail
| in chapter eight, "Invisible Technologies," but I mention them
| here because their role in reducing the types and quantity of
| information admitted to a system often goes unnoticed, and
| therefore their role in redefining traditional concepts also goes
| unnoticed. _There is, for example, no test that can measure a
| person's intelligence_. Intelligence is a general term used to
| denote one's capacity to solve real-life problems in a variety of
| novel contexts. It is acknowledged by everyone except experts
| that each person varies greatly in such capacities, from
| consistently effective to consistently ineffective, depending on
| the kinds of problems requiring solution. If, however, we are
| made to believe that a test can reveal precisely the quantity of
| intelligence a person has, then, for all institutional purposes,
| a score on a test becomes his or her intelligence. The test
| transforms an abstract and multifaceted meaning into a technical
| and exact term that leaves out everything of importance. One
| might even say that an intelligence test is a tale told by an
| expert, signifying nothing. Nonetheless, the expert relies on our
| believing in the reality of technical machinery, which means we
| will reify the answers generated by the machinery. We come to
| believe that our score is our intelligence, or our capacity for
| creativity or love or pain. We come to believe that the results
| of opinion polls are what people believe, as if our beliefs can
| be encapsulated in such sentences as "I approve" and "I
| disapprove. "
|
| [0]
| https://en.wikipedia.org/wiki/Monty_Hall_problem#Vos_Savant_...
|
| [1] https://en.wikipedia.org/wiki/Overton_window
| altarius wrote:
| One of my problems with this line of argument is that it provides
| a sort relativism of opinion, every opinion is just as valid as
| any other. And then you're at post-truth and post-fact sadly...
|
| If Carlsen is a fake, so is every chess player he has ever
| played, the whole system? Epidemiologists lied about masks (or
| they really didn't know better 3-5 months into the pandemic, also
| not good), discount everything else said? I've heard these
| arguments often lately :/
|
| Experts not being "real" implies that there's no absolute
| expertise whatsoever and implicitly legitimizes dismissing any
| argument whatsoever on a single failure and allows people to
| believe that any opinion (including their own) is just as valid.
| I think this is in part to blame for our "post-fact/post-truth"
| phenomenon. Many people decide experts or government were
| wrong/have lied before and can no longer be trusted. And when an
| outsider comes along with contrarian but convenient ideas, why
| not believe them, nobody else is to be trusted after all and
| every opinion is valid.
|
| Any expert opinion can be wrong but the prior is that an expert
| opinion is much more likely to be correct than your own, or a
| politician's, or a blogger's.
|
| NB: I want to note that the author's point is much more nuanced
| and (more importantly!) narrow, but it's not evident from the
| title or a superficial reading.
| [deleted]
| yboris wrote:
| Surprised that the classic "Conditions for intuitive expertise: A
| failure to disagree." paper by Daniel Kahneman and Gary Klein is
| not referenced
|
| https://psycnet.apa.org/record/2009-13007-001
|
| My takeaway: without short feedback loops it is hard to build
| expertise. If it takes 2 years to learn the individual you
| paroled reoffended, you can't learn (so there are virtually no
| expertise in parole officers or university admissions). In these
| scenarios, simple linear models outperform "experts" by a wide
| margin.
|
| See: "In Praise of Epistemic Irresponsibility: How Lazy and
| Ignorant Can You Be?" by Michael Bishop
|
| https://philpapers.org/rec/BISIPO
| eesmith wrote:
| Teller attributes Bohr as saying: "An expert is a person who has
| found out by his own painful experience all the mistakes that one
| can make in a very narrow field."
|
| I prefer something closer to "An expert is someone who knows some
| of the worst mistakes that can be made in his subject, and how to
| avoid them". 'Cause even experts make mistakes in their own
| field.
|
| This piece seems to think an expert cannot make a mistake in a
| given field of expertise, which is "utter nonsense, of course."
|
| I don't like this example: "If you look at the papers citing
| Viechtbauer et al., you will find dozens of them simply using
| N=59, regardless of the problem they're studying"
|
| Paging through the linked-to Google Scholar search finds 90
| results. Of the first 12 with readable text, only two seem
| correctly described that way (and max 3), suggesting under two
| dozen examples. A spot check of the remainder suggests that
| pattern holds. I'll show my work:
|
| #1 says "A sample size of 30 patients per group was chosen as
| recommended for pilot studies to achieve an appropriate level of
| statistical power [30]." They selected 60, and "1 patient was
| lost to follow-up" giving 59. So this wasn't N=59 but N=60. -
| https://medinform.jmir.org/2019/4/e14044/?utm_source=TrendMD... .
| 30 was chosen because "It corresponds to the detection of a
| potential difference of 21% between groups for a power of 80% and
| an alpha significance level of 5%." which is not the default
| example from Viechtbauer.
|
| #2 does not use N=59. The number 59 occurs because: "The study
| included 16 patients with diabetes (59 +- 8.8 years; 8 men and 8
| women)" -
| https://www.sciencedirect.com/science/article/pii/S095825921...
|
| #3 uses N=59, though I don't know if it was used incorrectly
| because I can only read the abstract and citation list -
| https://journals.sagepub.com/doi/pdf/10.1177/0306624X1985848...
|
| #4 specifically does NOT use Viechtbaueret, writing:
| "Viechtbaueret al., (2020) recommends sample size between 10-40
| as adequate for feasibility study. Linacre, (2015) suggests a
| sample of thirty (30) respondents as appropriate for general
| purposes. This study however uses 60 respondents becauseadequate
| sample and item produce a good item, person separation and
| reliability in Rasch measurement (Fisher, Elbaum, & Coulter,
| 2010)." -
| https://journals.sagepub.com/doi/pdf/10.1177/0306624X1985848...
|
| #5 uses N=43 ("With an alpha value of 0.05 and with an 80% power
| we need a sample size of at least 43 patients to detect a
| difference in validity and assess reliability") -
| https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6886622/ . This is
| not the default example from Viechtbauer.
|
| #6 uses N=59, using calculations directly from Viechtbaueret. It
| appears to be used correctly.
|
| #7 uses N=53, commenting "There is no gold standard for sample
| size calculation in pilot studies. Sample sizes as small as 10
| (Hertzog, 2008) and as large as 59 (Viechtbauer et al., 2015)
| have been recommended." -
| https://journals.lww.com/nursingresearchonline/FullText/2018...
|
| #8 uses N=30 for each region, writing "Tamano de la muestra: en
| cada region, se llevo a cabo un estudio piloto con
| aproximadamente 30 pacientes" -
| http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1... .
| The 59 occurs in, for example, "La reduccion tambien fue
| estadisticamente significativa (59%) al tener en cuenta el coste
| del tiempo de enfermeria".
|
| #9 points to the N=59 recommendation (correctly, for their
| experiment) but actually uses N=71 -
| https://www.mdpi.com/1422-0067/20/8/2024/pdf
|
| #10 appears to have the discussed N=59 problem as it doesn't
| mention the 0.05 effect - https://mhealth.jmir.org/2016/3/e109/ .
|
| #11 says "Pilot studies should include at least 12 patients
| [30,31], and cluster-randomized studies need more patients to
| correct for the clustering [32]. To have sufficient power for
| this pilot study, we intended to include 15 patients per nurse."
| where [31] is Viechtbauer. The value 59 is because "data of 149
| (98%) patients were available of whom 66 (44%) were in the
| intervention group; ++ = data of 135 (89%) were available of whom
| 59 (44%) were in the intervention group". -
| https://www.sciencedirect.com/science/article/pii/S026156141...
|
| #12 says "In line with recommended sample sizes of pilot
| feasibility trials, 83-85 40 patients is deemed sufficient to
| explore feasibility, acceptability and potentially efficacy of
| the intervention, assuming retention rates of 80%, the true
| population consent rate will be with a margin of error of 13%
| (95% binomial exact CI)". The only "59" comes from citations. -
| https://www.sciencedirect.com/science/article/pii/S026156141...
|
| I'm not sure how we're to have a take-home lesson when the false
| negatives so overwhelm the claimed evidence.
|
| I'm also hung up on: "Capitalism and evolution work so well
| because of their merciless feedback mechanisms." because of
| obvious questions like "does capitalism work well for slaves?"
| d23 wrote:
| A lot of the points are interesting, but I couldn't help but get
| hung up on the link to the covid analyist who supposedly beat the
| experts, at https://www.maby.app/covid/
|
| A couple of immediate red flags: the predictions stop in early
| May, and there appears to be no weighting of how "important" a
| given prediction is. Perhaps the former is just because a contest
| ended at that time. For the latter, of course, importance is
| subjective, but ranking these two as equivalent is silly:
| * How many cases per day will Washington state average for the
| week ending June 7? (Tom: good prediction, Experts: bad) *
| How many US COVID deaths will occur in 2020? (Tom: bad
| prediction, Experts: good)
| pa7x1 wrote:
| Experts are real, unfortunately knowledge in many fields is built
| with very poor standards of quality. The experts in those fields
| are vastly knowledgeable in falsehoods.
|
| While the general public questions if we should be funding the
| next big project in physics, the real question we should be
| asking is if we should be funding almost anything else as long as
| they are not willing to apply the same levels of rigor.
| ndiscussion wrote:
| That's right, unfortunately most of science has been debunked
| ie replication crisis. Sadly, the populace doesn't see it that
| way.
| orzig wrote:
| Whoa - [citation needed] on "Most of science has been
| debunked"
|
| We've learned to be much more skeptical, and the previous
| standards were scandalous, but there's a difference between
| "too much of" and "most of"
| nradov wrote:
| Ioannidis JPA (2005) Why Most Published Research Findings
| Are False. PLoS Med 2(8): e124.
| doi:10.1371/journal.pmed.0020124
|
| https://doi.org/10.1371/journal.pmed.0020124
| berekuk wrote:
| On the other hand, here's a hilarious response how
| Ioannidis is wrong in his claim from one of the most
| well-known replication crisis blogs:
| https://replicationindex.com/2020/12/24/ioannidis-is-
| wrong-m...
| nradov wrote:
| Why do you suppose they chose to publish this hit piece
| as a blog post instead of going through the peer review
| process with a real journal article? Or at least a formal
| letter to PLoS Medicine?
| ndiscussion wrote:
| Here's a start, almost every single field is implicated:
| https://en.wikipedia.org/wiki/Replication_crisis
|
| Economics, social sciences, even medicine.
| QuesnayJr wrote:
| Social psychology has a replication crisis, because they
| had a culture of never questioning a published result.
| You can see that the vast majority of that article is
| devoted to problems within psychology. (Medicine also
| seems to have problems.)
|
| In economics 1/3 of 18 studies failed to replicate? A
| study that small, what are the odds that study itself
| would replicate? Note that this claim is restricted to
| experimental studies, which are usually testing human
| decision-making on economic questions. This is the field
| of economics closest to social psychology. It seems
| plausible to me that the model of "get a bunch of people
| in a room and ask them some questions" is not a reliable
| methodology.
|
| Economics as a whole may have a problem with replication,
| but we can't tell from the information on that page.
| There are whole fields that aren't even mentioned -- no
| physics, no chemistry.
| Der_Einzige wrote:
| And don't forget machine learning!
|
| If your paper in ML doesn't provide code, I assume that
| it's not reproducible and the authors made a conscious
| decision to not release the code...
| bgorman wrote:
| One of my theories based on life experience is that many fields
| do not attract the brightest people to begin with, and
| consequently many of the ideas from these fields are not
| pressure-tested enough.
|
| In todays academia, it is basically impossible to fail getting a
| PhD in many fields provided you are ok being miserable for years
| at a time. At the same time academia is filled with a ton of bs
| like publicly funded research being released behind paywalls,
| professors receiving grants for projects that are already
| completed, students being denied graduation util their project
| can be submitted to the highest impact journal possible etc,
| having to do a long postdoc to get a tenure track position etc...
|
| For smart people that value financial freedom, academia is a
| terrible option at this point in time.
|
| The current coronavirus fiasco has laid bare the lack of critical
| thinking in the life-sciences fields.
| xyzelement wrote:
| I want to share this tid-bit. When I studied portfolio
| management, I had to learn not just how to talk to people about
| their risk tolerances, but how to ascertain whether they knew
| their own tolerances. The concept was called "calibration" - or -
| how well is someone able to know that they are wrong/don't know
| something. Basically, "high calibration" is the opposite of
| Dunning-Kruger in my mind.
|
| The text cited a study that measured calibration by profession.
| The highest calibrated profession was meteorologists. It makes
| sense if you think about it: I prognosticate rain tomorrow,
| tomorrow comes and it's sunny - I am forced to confront the fact
| that my prognosis was wrong. So not only is there a way to know
| who's better and who's worse, but also constant reminder that "I
| can be wrong despite my best work"
|
| The lowest calibration profession was doctors. Which also makes
| sense. Imagine I am a doctor and I miss an early sign of a
| disease. The patient dies 5 years later - I may not even know,
| and even if I do, very unlikely am I confronted with the specific
| point in time 5 years prior where I made the wrong call.
|
| So the thing that matters with expertise is - how often is the
| person getting feedback. In the article they cite chess as having
| "real experts" - of course. Because a chess player gets feedback
| on how good they are every time they win or lose a match.
|
| On the flip side, how often is someone in a very soft field
| getting feedback? What's the way for an "expert" in say Gender
| Studies, to wake up in the morning and be confronted with "the
| thing I did yesterday didn't work?"
|
| The other thing about experts is they are optimizing for a single
| thing. Eg ask your compliance person at work if you can do
| something, the answer is "NO" because they are optimizing for not
| being sued, but in reality that's not the only concern. Or in the
| pandemic, ask the economist if we should lock down - they'll say
| no because they understand the financial consequences. Ask a
| medical person and they'll say yes because they are focused
| solely on the number of sick people. In reality the world is
| complex and there are tradeoffs.
|
| Related, experts are often concerned with high-impact issues
| which may be different from individual issues. EG: in the USA, by
| default, kids get the Hep B vaccine on their first day of life.
| On country level this makes sense because there are a lot of
| mothers out there who have Hep B and don't know it. But when my
| wife was giving birth, we knew to opt out despite the
| recommendation because we understood that the experts are
| optimizing for a thing that doesn't relate to us.
| zug_zug wrote:
| > "This year, we've come to better appreciate the fallibility and
| shortcomings of numerous well-established institutions ("masks
| don't work")... while simultaneously entrenching more heavily
| mechanisms that assume their correctness ("removing COVID
| misinformation")."
|
| I suspect in a cultural moment of mass-skepticism of even valid
| systems (e.g. masks/election/vaccines/wikipedia) it's very hard
| to touch on the fallibility of those systems without putting
| people on the defensive. It shouldn't be in an ideal world --
| making these things more auditable and rigorous only makes them
| less prone to questioning in the long term, but I suppose many
| feel faith in the system is more important at this moment than
| incremental improvement.
|
| [However this really doesn't answer the n=59 conundrum, which I
| think requires a different lens]
| lordnacho wrote:
| I think stats is the culprit. Or lack of it. If there's one area
| of study that ties together observations and models, it's stats.
| But stats suffers from the same problem that programming does:
| people think it's a tool that they just need to know a little bit
| of, so they don't defer to the proper authorities in the area.
|
| Of course this is just anecdotes, but I do have a friend who did
| stats as a phd and this is the attitude that seems to be common:
| stats (and code) are minor tools that won't affect the outcome of
| our experiment.
| klmadfejno wrote:
| IMO Experts are more wise to the problems of their domain than
| intelligent. Intelligence relates to the execution of mental
| tasks and the recall of specific facts. Wisdom relates to the
| planning of the approach to solve a given task.
|
| It is whether or not the nature of the problem lends itself to
| expertise that defines if experts are particularly relevant for a
| given domain.
| 13415 wrote:
| I'd say that experts are mostly knowledgeable, experienced, and
| broadly respected within their scientific domain. I don't see
| how intelligence or wisdom are important factors - other than
| for the fact that most accomplished scientists are probably
| rather intelligent.
| Bakary wrote:
| I remember this concept being mentioned elsewhere as "friendly"
| and "hostile" learning environments
| JadeNB wrote:
| Epstein's excellent book _Range_ uses the terms, though I think
| they don 't originate there.
| breck wrote:
| Prediction: in 10 years no "expert" will not be putting their
| work in Git. Git provides a zero cost (in time and money) way to
| share your full work with audibility; where fixing mistakes
| continuously is expected and not something to be embarrassed
| about; where people can instantly clone to reproduce and test
| better ideas upon your work; where collaborating with proper
| attribution is done automatically, et cetera.
|
| Look in any field--epidemiology, computer science, physics,
| chemistry, medicine, et cetera. If the leading experts aren't
| using git yet, look for the ones that are. They will be the
| leading experts of tomorrow.
| [deleted]
| jdsully wrote:
| I like that prediction, but git is actually a bit of a step
| back from older more traditional source control systems.
| Rewriting history is far more common with git to the point
| where teams have to make explicit policies on when that is
| allowed.
|
| Even if git becomes common in science outside of code, I don't
| think it would overcome the social pressure to appear
| infallible.
| breck wrote:
| > Rewriting history is far more common with git
|
| How do you mean? It's impossible. You git push something, and
| then you leave an immutable trail that someone somewhere may
| have cloned so even if you were to force push or delete you
| can't erase it. Assuming it was of any importance and someone
| was watching (but if not of any importance, then no big deal
| to force push/delete).
| 0xffff2 wrote:
| Saying something is immutable because someone _may_ have
| saved an old copy before you mutated your own copy sounds
| very fragile to my ears.
| rubatuga wrote:
| If you push to GitHub it won't delete it
| nonpolitical wrote:
| This reminds me of my layperson puzzlement:
|
| Can we apply and verify the results from CERN experiments beyond
| experimental settings? If we can't, how can we know the results
| from CERN experiments aren't just artifacts of the experimental
| apparatuses?
| losvedir wrote:
| That's a good question. You're right that a random grad student
| in a lab can't replicate, e.g., the discovery of the Higgs
| boson!
|
| However, at least for the Higgs (and I believe this is standard
| for most such large experiments), there were actually two
| different groups using two different apparatuses, both at CERN.
|
| There was the CMS[0] detector and ATLAS[1] detector, used and
| analyzed by two different groups of physicists, resulting in
| two papers[2,3], published on the same day.
|
| [0] https://en.wikipedia.org/wiki/Compact_Muon_Solenoid [1]
| https://en.wikipedia.org/wiki/ATLAS_experiment [2]
| https://arxiv.org/abs/1207.7235 [3]
| https://arxiv.org/abs/1207.7214
___________________________________________________________________
(page generated 2021-01-11 22:02 UTC)