[HN Gopher] Please commit more blatant academic fraud
___________________________________________________________________
Please commit more blatant academic fraud
Author : EvgeniyZh
Score : 476 points
Date : 2021-05-30 09:15 UTC (13 hours ago)
(HTM) web link (jacobbuckman.com)
(TXT) w3m dump (jacobbuckman.com)
| intricatedetail wrote:
| Scientific community tainted itself by producing fake medical
| cannabis "research" as they could only get funding if their
| "studies" could prove negative effects.
| lowdose wrote:
| Out of curiosity could you refer me to some sources to read
| more about this.
| intricatedetail wrote:
| https://www.greenentrepreneur.com/article/357200
|
| This article scratches the surface.
| sillysaurusx wrote:
| One surprising discovery about AI: There's nothing inherently
| wrong with cherrypicking. I used to think it was this dirty
| thing. Oh, you're only showing your best results?
|
| But the best results are what matters. If you can create an
| amazing song from scratch by telling a computer what to do, you
| only need to do it once. The song is still good.
|
| And it's easy to automate. If you have a way of detecting a good
| song, or at least filtering out rubbish, then you can generate
| thousands of attempts to get the good result.
|
| I like to say "It's hard to pick cherries from a rotten tree."
|
| The flipside of this post, though, really hits home. I recently
| was super excited about a paper called FNet, which showed that
| "fourier transforms can be competitive with transformers", i.e.
| just replace multihead attention with a fourier transform. 7x
| faster on a GPU! Woo!
|
| Buried in section 3.1, they casually note that BERT with half the
| parameter count completely dominates them in terms of accuracy:
| https://twitter.com/theshawwn/status/1393315603973386240
|
| In other words, the paper was ... advertised in a misleading way,
| to put it kindly. Most of the transformers they compared with
| aren't the traditional multihead GPT style transformer that
| people think of when they hear "transformer". And the one that
| was (BERT) totally annihilated them.
|
| But it's also easy to jump to conclusions too quickly. I've been
| a hothead in the past, and called out a paper when it turned out
| that I just didn't understand enough. It's tricky to know for
| sure.
| coldtea wrote:
| > _But the best results are what matters._
|
| Not the kind of cherry-picking condemned here, which is more
| like your drug killing 90% of the animals it was tested on, and
| making a small difference in 2% of them, and you present it as
| only that 2% ever happened (and thus represent 100% of the
| cases).
| sillysaurusx wrote:
| Oh, you're right:
|
| _Trying that shiny new algorithm out on a couple dozen
| seeds, and then only reporting the best few. Running a big
| hyperparameter sweep on your proposed approach but using the
| defaults for the baseline. Cherry-picking examples where your
| model looks good, or cherry-picking whole datasets to test
| on, where you've confirmed your model's advantage. Making up
| new problem settings, new datasets, new objectives in order
| to claim victory on an empty playing field. Proclaiming that
| your work is a "promising first step" in your introduction,
| despite being fully aware that nobody will ever build on it._
|
| Yeah, I misread; sorry. Testing your model on a small dataset
| and presenting it as a general solution is all too common.
|
| That's why generative models are cooler than classifiers, to
| me at least. You can show the outputs visually, whether it's
| text or image or sound. But with classifiers, you're chasing
| an accuracy rating. I forgot that people often test on
| CIFAR-10 (smol data) without verifying on Imagenet (big-ish
| data) and then present their paper as very general.
|
| But, I do have something to say about that kind of dataset
| cherrypicking: Researchers often find it hard to test on
| large datasets because of limited compute resources. One of
| the pioneers of DDPM wasn't able to train on imagenet due to
| lack of GPUs. CIFAR-10 was the best they could do. So it's
| sometimes hard to tell whether there's intentional deceit, or
| just a shoestring budget. (I was surprised how much research
| a lot of people get done in spite of limitations.)
|
| Thanks for the correction; cheers.
| teruakohatu wrote:
| > . If you have a way of detecting a good song, or at least
| filtering out rubbish, then you can generate thousands of
| attempts to get the good result.
|
| So often the filtering technique is a human in the loop,
| manually going through the output. Humans become the hidden,
| undifferentiatable and undisclosed loss function.
| sillysaurusx wrote:
| Sure is! And you'd often be amazed at just how much data you
| can churn through, if you put your mind to it. I once watched
| Gwern classify thousands of poetry outputs over the course of
| a day or two, with nothing more than dedication and a bash
| script to mark "a" or "b".
|
| It took a long time to accept that this is both normal and
| effective. It's not some hidden secret to be ashamed of. The
| headline images in https://openai.com/blog/dall-e/ are
| probably the _best_ and most interesting images they could
| find; they were picked out by humans. It 's a positive
| statement to say that dall-e can do that some of the time,
| rather than "it fails X% of the time."
| underseacables wrote:
| I graduated in 2014, and was astonished at how much cheating
| there was in university. From thefts and bribery, to groups
| sitting in the back of the class whispering in another language,
| integrity at the university level has plummeted. When students
| openly cheat and advance, it's no wonder that behavior becomes
| permissible among faculty
| wincent wrote:
| Best part of the article is the footnote, where the author
| characterizes his own papers:
|
| > This paper is bullshit, this paper (a NeurIPS oral) is
| bullshit, this paper is complete bullshit, this paper is mostly
| good science but also has a sprinkling of bullshit. Apologies to
| my co-authors.
|
| https://jacobbuckman.com/2021-05-29-please-commit-more-blata...
| lixtra wrote:
| The author already gave it away in the abstract, though. _Up
| to_ two digits is a good indicator of bullshit (by both author
| and review). [1]:
|
| >> score of up to 93.30%
|
| [1] https://www.aclweb.org/anthology/D16-1254.pdf
| turbinerneiter wrote:
| If this, and the stuff coming up in this thread, is true about
| academia ... Trump is our fault.
|
| How can we tell the people to trust science, if science is a
| clusterfuck of underpaid PhD students who are incentivized to
| cheat?
| baybal2 wrote:
| The motivation behind it is very simple: eye-watering salaries in
| the field.
|
| Moneyed people are ready to pay these salaries because a lot of
| them have succumbed to hype equating "AI" (that phrase alone is
| complete bullshit) to something bordering on "magic," and
| bringing equally magical money making opportunities.
|
| The prospect for business owners to get rid of white collar
| workers in a manner not unlike how machines swept blue collar
| workers is just this much irresistible.
| alisonkisk wrote:
| The eye watering salaries come from passing a whiteboard
| interview, not publishing papers.
| YeGoblynQueenne wrote:
| This!
| caddemon wrote:
| The same issues exist in a ton of other academic fields though,
| most of which do not get crazy salaries. IMO academics are more
| concerned with recognition and other ego-stroking than they are
| with salary.
| StandardFuture wrote:
| So, in other words, there really is not any "publish or perish"
| in academia. It's simply that to _outcompete_ your fellow
| graduate students in getting a top job in industry (or a
| tenured position in academia) you need to have the most
| publications with the most "novelty". And that can be
| justified at any cost.
|
| So, is this, at the end of the day, that the ratio of students
| willing to study graduate level studies, and jobs that require
| graduate level studies is dramatically imbalanced?
|
| Sometimes, we as humans like to avoid the obvious discussion
| because it is the most uncomfortable discussion to have.
|
| The truth is that we need more restrictions and an even higher
| bar on education that artificially pipes the best of the best
| in corresponding numbers to estimated availability of jobs in
| 4-5 years.
|
| I am pretty sure that the medical world already does this.
| pitaj wrote:
| The AMA artificially restricts the supply of doctors through
| their licensing monopoly, in order to keep salaries high.
|
| This is bad and shouldn't be emulated in other industries.
| StandardFuture wrote:
| Would there be enough people willing to go into Medical
| School debt without a high salary on the other end?
|
| The entire point of game theory is to align incentives. It
| is not to simply paint pictures of ideals and deny that
| humans are human.
|
| I understand that the AMA has incentives that are not
| nearly as noble as simply ensuring the world has a steady
| stream of doctors. Nonetheless, the answer is not to simply
| say "the AMA has some impure incentives" as an actual
| dismissal of a different argument.
| unishark wrote:
| As long as there's a possibility of making more money by
| lying and cheating, people will lie and cheat. Especially
| when it's _a lot_ more money. But I don 't think there's any
| unemployment danger among CS PhD's to worry about. Unless
| they somehow managed to avoid learning how to use any tools
| or technologies of value to anyone. PhD's do not have to be
| at academia or a "top job" to be employed in a research or at
| least R&D capacity.
|
| By the way, tenured positions do not pay eye-watering
| salaries. It's still the same vicious low stakes.
| StandardFuture wrote:
| > But I don't think there's any unemployment danger among
| CS PhD's to worry about
|
| This is why my comment _specifically_ addressed _top
| positions_ and _tenured positions_ i.e., a position with a
| specific world-renown research group. Competition for the
| positions that putting all that time and effort into a PhD
| would be for.
|
| > employed in a research or at least R&D capacity.
|
| Sure, jobs exist. I am not seeing where I denied this fact
| anywhere.
|
| > tenured positions do not pay eye-watering salaries
|
| Tenure is what I am specifically referring too. It's a form
| of intellectual and economic freedom.
| unishark wrote:
| So you want to restrict people from getting a phd unless
| they can get a top job out of it? What about people who
| want to do research but aren't worth the
| pay/responsibility of top jobs? Is your goal to increase
| the value of a phd by limiting it to highest quality
| people possible (like with MD's)?
|
| Tenure is a benefit that is given instead of money to
| keep you from leaving for more money. It may be nice to
| have late in your career if you want to keep working
| forever or bizarrely want to stay at your research-
| focused school but focus on teaching. But money adds up
| nicely and has more flexibility. And like with job offers
| and promotions everywhere, it's something you generally
| can only get if they need you more than you need them.
| bonoboTP wrote:
| Yes, the would be the outside solution if you were an all
| seeing God. But in the real world we have to make do with
| local incentives. And that is leading to a ballooning of
| academia, professors want more grad students for more papers
| and more prestige, some of whom also turn into profs and it
| snowballs. It's great while the music is playing, but then
| when winter comes back, what will all these people do then?
| Things are already insanely competitive to the degree that
| many profs even say they couldn't make it into a graduate
| program now, you already need top conference papers just to
| get accepted at grad school (in better groups).
|
| In a way, I think a lot of the internal squabbles regarding
| diversity etc. in the AI community come from the animosity
| due to a shrinking cake and more hungry mouths.
| YeGoblynQueenne wrote:
| >> The motivation behind it is very simple: eye-watering
| salaries in the field.
|
| For a very broad meaning of "in the field". Academic salaries
| are kind of meh. The eye-watering salaries you mean are paid by
| large technology companies and I don't think those hire you on
| the merit of your publications, only. For example, allegedly
| google and facebook et al hire drop-outs from PhD programmes in
| highly rated institutions etc. so people who haven't yet had
| the chance to change the face of AI with their research, if I
| may be so bold.
| quantum_state wrote:
| A very funny article ... on a serious note, why would one want to
| cheat oneself by claiming false things in one's research
| endeavor?
| anonymousDan wrote:
| The author mentions a collusion ring in AI, but I'm pretty sure
| the article refers to a situation that arose in the computer
| architecture community (which itself happened quite a while
| back).
| zitterbewegung wrote:
| I think we need to rethink the core concept of Universities
| having publish or perish as a part of being a Professor. The
| reason why fraud exists is because of the incentive structure and
| due to the nature of research you have intelligent people that
| will game the system. So, we have to figure out how to change the
| incentive structure so that professors don't attempt to perform
| Academic Fraud or we h ave to figure out if fraud does exist
| after the fact.
|
| I have tried to reproduce ML / Deep learning research. I have a
| few heuristics that don't really tell me if fraud exists but
| either they just can't release the data due to special
| circumstances that I can understand or it isn't kosher. The
| biggest one is that you have to register or ask them for their
| test data. Another one is that the paper has been published for
| more than two years and there are no citations from other work.
| bonoboTP wrote:
| Keeping test data private is good practice actually. It keeps
| from hyperparameter optimization on the test set. Yes
| theoretically the dataset authors might still do it, but it's
| still better for the field as a whole to keep some test data
| secret and only use it for evaluation.
|
| Registering before getting data is common, so they can make you
| agree to some terms. It's not suspicious in itself.
|
| There are no hard and fast rules of thumb. You can look at the
| group, the people involved, the country, the writing style, the
| quality and thoroughness of ablation, is there code available
| etc. But nothing is 100%.
| ModernMech wrote:
| > So, we have to figure out how to change the incentive
| structure so that professors don't attempt to perform Academic
| Fraud or we h ave to figure out if fraud does exist after the
| fact.
|
| One possible way: increase funding to the arts and sciences.
| The total NSF budget was ~$8B, or 0.03% of US GDP. We could
| double that and no one would even notice. Increased funding
| means more money to go around, means higher acceptance rates
| for grants, means less incentive to write fraudulent papers.
|
| The best part is, you have the power to actually change this.
| Write your congressperson, vote for a congressperson who
| pledges to do this, run for congress yourself, or organize
| other people to do this with you.
| zitterbewegung wrote:
| Increasing the budget doesn't solve the problem. You would
| need to create a set of well funded auditors that have domain
| experience but the problem with that is no one would want the
| job.
| guilhas wrote:
| Trust the science
| xbar wrote:
| This is deserving a of a careful read and long consideration.
| Well said.
| plaidfuji wrote:
| > Undermining the credibility of computer science research is the
| best possible outcome for the field, since the institution in its
| current form does not deserve the credibility that it has.
|
| This is obviously somewhat tongue in cheek, but I'm not sure the
| author appreciates the full implications of how this would play
| out. Research funding for AI exists in competition with funding
| for all other fields. Revealing deep-seated BS would serve mostly
| to decrease funding commitments to AI projects in favor of other
| fields, or away from R&D entirely. It would be bad for all
| researchers (but if the true level of fraud is that high, and I
| have no doubt that it is, this may simply be an inevitable market
| correction).
|
| Other comments in this thread seem to miss the point as well:
| "why don't universities select for quality not quantity"? Well,
| the change has to start with the funding sources for the unis to
| care.
| Aperocky wrote:
| I think this is exactly the implication.
|
| The amount of people pouring into AI, or more specifically to
| Neural Network version of AI is unnatural, and it resembles a
| bubble in the classic economic sense.
| vinni2 wrote:
| Which is why several journals are pushing for reproducibility as
| a requirement and dedicated reproducibility tracks where there
| are opportunities to call bluff on established papers. As an
| academic from a small lab in an unknown university it is shocking
| for me when we are unable to reproduce performance numbers from
| papers by popular institutions.
| teachingassist wrote:
| > several journals are pushing for reproducibility as a
| requirement
|
| The current requirement for academic publishing is peer-review,
| i.e.:
|
| "Colleagues in my field looked over it briefly and said it's ok
| [because it doesn't challenge their work]"
|
| This inevitably leads to whole fields of science being
| corrupted and biased in ways that we can't easily identify.
|
| Science publishing standards could be improved in any number of
| ways: e.g. original data must be shared in an open format,
| publish your hypotheses before doing the research, publish the
| code, standardise the setting out of methods and statistics and
| results so that other scientists in and out of the field have a
| hope of following them.
|
| Nobody in the industry is motivated to change this, and hides
| behind the romantic ideal of a scientist pushing the boundaries
| and breaking the rules to discover novel theories.
|
| If such a scientist exists today, we can't identify them
| against the background of noise.
| alisonkisk wrote:
| Modern science _doesn 't_'have peer review!
|
| Peer review works within a community, scientists review each
| other's work to decide who to trust. But the public and
| funding agencies are not part of that review! Not even
| scientists in tangentially related fields (such as
| statisticians or methodology experts) are consulted.
|
| Furthermore, "peer review" is limited to pre publication.
| Follow up research by others has mininal effect on the impact
| of a paper on a scientist's career.
|
| So why to we believe or fund the results? Either we are not
| their peers and should not engage with them, or we should be
| reviewing their work as part of the publication process.
|
| We also have "peers" in criminal prosecution. Would you allow
| a criminal trial where only the scientist's colleagues were
| in the jury?
| alisonkisk wrote:
| Modern science _doesn 't_ have peer review!
|
| Peer review works within a community, scientists review each
| other's work to decide who to trust. But the public and
| funding agencies are not part of that review! So why to we
| believe or fund the results? Either we are not their peers
| and should not engage with them, or we should be reviewing
| their work as part of the publication process.
|
| We also have "peers" in criminal prosecution. Would you allow
| a criminal trial where only the scientist's colleagues were
| in the jury?
| bonoboTP wrote:
| The problem is that laypeople or bureaucrats can't judge
| highly specialized technical contributions. It's already
| difficult to do if your subfield is a different niche.
| teachingassist wrote:
| > It's already difficult to do if your subfield is a
| different niche.
|
| It's not _that_ difficult, at least in science and
| science-adjacent fields. The scientific method still
| follows the same principles.
|
| If you are used to reading papers, you can tell when
| someone has elided important detail, or not included the
| evidence for their claims - or, if someone [or a whole
| subfield] is using technical language to obfuscate their
| work.
| bonoboTP wrote:
| There are many levels of analysis to a paper. Yours is
| part of it but not the full picture. One other thing is
| the novelty aspect. To assess that, you need to know the
| current state of the field. Also the best practices,
| known methodological pitfalls, judge what is a valid
| chain of reasoning and what is too speculative etc. It's
| far from simple and I often see outsiders online
| criticizing largely irrelevant things while missing core
| issues. There's no shortcut to expertise in a field. You
| need first hand experience and knowledge of the state of
| the art. Otherwise, if papers have to talk to laypeople
| or generic scientists, we'll see even more smokes and
| mirrors. The idea of peer review is that a work should be
| challenged by those best equipped to challenge it.
| bonoboTP wrote:
| Reproducibility doesn't begin to solve the subtler issues
| mentioned in the article. It is necessary but far from
| sufficient.
| ModernMech wrote:
| There's a lot of discussion here about changing the incentive
| structure of academia to prevent this kind of fraud, but based on
| the discussion I've read so far I don't think a lot of people
| here understand the exact incentives that are involved, so I'd
| like to try to explain how academics actually operate.
|
| We've all head of "publish or perish", and there is a general
| understanding that you have to publish papers to get tenure, so
| that's where the incentive is. But it's a little more complicated
| than that.
|
| When you are hired as a tenure-track assistant professor at a
| university, you're given what's called a "startup" package.
| Assistant professors are essentially startup founders where their
| company is a lab. Your job as an assistant professor is to use
| that money to purchase equipment and to recruit PhD students (aka
| employees) so that you can launch your research agenda. This
| money pays for:
|
| - rent taken off the top (essentially paying for your lab space
| and other costs the University incurs)
|
| - a portion of your own salary
|
| - your students' tuition
|
| - your students' stipends
|
| - conference travel and fees
|
| - lab equipment (computers, desks, chairs, machinery, scientific
| apparatuses). To note here, for every dollar you spend on
| equipment, the University takes a percentage of that. So if I buy
| a chair for $100, the University will charge for example $10.
| This is a sort of tax for being affiliated with the University
| and getting free use of their resources (internet, libraries,
| subscriptions, etc.).
|
| - As well as other things.
|
| Crucially, this money is not free. The expectation is that you
| will in the next 6 or so years use that money to jumpstart a
| successful lab operation. A "successful" lab is one that is able
| to procure significant funding from government and institutional
| sources. Your first goal is to procure enough funding to recoup
| the initial startup package the University laid out. That's your
| bare minimum. Ideally, you want to procure funding far in excess
| of this by the time you're up for tenure, to prove to the
| University that your lab will bring in more dollars. Because
| you've got to earn your keep.
|
| Notice there hasn't been much discussion of papers so far. The
| initial startup package can be quite significant -- on the order
| of millions of dollars depending on the research agenda. As an
| assistant professor, you need to bring in millions of dollars
| from funding agencies like the NSF to make that back in
| multiples. But the NSF budget was only $8 billion dollars in
| 2020, and they need to spread that money around to new assistant
| professors across the entire country. Therefore, funding rates
| usually range from 10-20% for a proposal, which is quite low when
| your entire future is depending on it.
|
| This is where "publish or perish" comes in. When you are up for
| tenure, yes they'll be looking at the quantity and quality of
| your publications. However, you could have 1000 high quality
| publications, but if you don't bring in significant grant money,
| you will not be getting tenure. The way you get that grant money
| is by writing a good research grant proposal, and the way you
| back up and bolster those proposals is through publications.
|
| This is why there is so much fraud in publications. Not getting
| tenure at this stage means your research career is essentially
| over. You are as good as fired from your current role, and few
| Universities will take you on as a tenure track professor when
| you've already failed at your current institution. At this point
| you're in your mid to late 30s, and your best years (in terms of
| research ideas) are already behind you. It's time to enter
| industry or become a lecturer.
|
| So how to fix this?
|
| 1. More research funding for government agencies. Higher funding
| rates means less incentive to commit fraud.
|
| 2. A feedback loop of profits from industry to universities.
| Right now when technological advancements happen at universities,
| those flow out to the general public, then to companies, who
| figure out how to monetize them. The profits generated by that
| monetization are captured entirely by private corporations, even
| though the foundational innovations on which they are based are
| funded by Universities and by extension the public. If some of
| this profit were diverted back into Universities (maybe through
| increased corporate taxes), then this would help.
|
| 3. Lessen the incentive for Universities to essentially become
| hedge funds with schools attached. My University is currently
| sitting on a billion dollars that is locked up in investments.
| Harvard's endowment is currently 40 billion dollars. They could
| fund the entire country's research agenda 5x over with that. Yet
| professors are paying their own salaries with public money? And
| the University gets a cut? This is insane.
|
| 4. Offer an alternative off-ramp for academics who don't get
| tenure. If not getting tenure won't end your career you've spent
| your entire life building, then the incentive to commit fraud
| would be lessened as well.
|
| Anyway, tldr; there's not enough money to go around, so fraud
| exists.
| OminousWeapons wrote:
| I agree with everything you said. At the risk of being
| pedantic:
|
| > A feedback loop of profits from industry to universities.
| Right now when technological advancements happen at
| universities, those flow out to the general public, then to
| companies, who figure out how to monetize them. The profits
| generated by that monetization are captured entirely by private
| corporations, even though the foundational innovations on which
| they are based are funded by Universities and by extension the
| public. If some of this profit were diverted back into
| Universities (maybe through increased corporate taxes), then
| this would help.
|
| This isn't ENTIRELY true. Universities "own" research IP and
| they do have tech transfer offices which at least theoretically
| help them monetize their largest breakthroughs via researcher
| startups. Many large, well established labs will also perform
| collaborative industry research, effectively acting as a CRO,
| which they can charge a lot of money for (and the Universities
| can subsequently take overhead from). The huge, field leading
| labs with well known PIs will have multiple R01s and various
| grants but the vast majority of their funding will come from
| industry.
| ModernMech wrote:
| Yes, this is true. But I was thinking more along the lines of
| fundamental research. For example, DARPA spent a lot of money
| investing in transistor research in the 50s, which begat
| microcontrollers, which begat companies like MS, Apple, and
| Google. Those companies together are worth multiple trillions
| of dollars, yet wouldn't exist but for the publicly funded
| sector research that preceded them. Harvard's $40 billion
| looks piddly in comparison. And what kind of research does
| MS/Google/Facebook fund in the University? They'd like us to
| figure out how to optimize ad placements on their search
| engines and social media sites. I've seen far too many
| promising researchers swallowed by that kind of work, only to
| quit after realizing the direct application of their research
| is simply to make those companies more profitable while they
| are paid $25k per year and eating ramen 3x a day.
|
| We see the same thing happening right now in robotics. DARPA
| spent a lot of money jumpstarting driverless cars with the
| DARPA grand challenge in 2005. The researchers I know who
| were involved with that have left academia and are in the
| private sector now because that's where all the money is.
| There's no money flowing back in the academic direction even
| though there's a direct line from what they were doing in
| 2005 to what Toyota, GM, Google, Tesla, etc. are doing today.
|
| I'm not saying they shouldn't make profits and be worth a lot
| of money. They provide a lot of value. But the outcome
| definitely seems lopsided to me.
| OminousWeapons wrote:
| Thank you for the clarification. I agree with you.
| bonoboTP wrote:
| Yes, as someone in the system, it's all true and an open secret,
| everyone knows it and we complain about it over some beers all
| the time.
|
| But isn't something like this true everywhere?
|
| And blaming the individual is futile. I like Scott Alexander's
| Moloch [0] concept more useful. If you apply a high standard
| against your own research regarding thoroughness, not
| overclaiming etc. never looking at the test set etc., you won't
| beat the benchmark, which contains entries you know are too good
| to be true. But if you don't beat them, you'll get rejected. The
| pace is extreme, lots of groups work on very similar things and
| it's a race to publish very similar stuff first. As a PhD student
| you _need_ publications. You are in your late twenties, early
| thirties, you can 't afford to lose several years.
|
| The big lie is that there is so much discovery happening as
| suggested by the thousand upon thousands of papers at big
| conferences, most of which is never really engaged later on. We
| have to pretend that all grad students in every group can
| contributing some actual valuable, bulletproof novel thing to
| science every few months and pretend that you can have multiple
| papers at conferences while remaining painfully critical towards
| your own work. I mean how could it be that people happen to
| always find something that works out in the end? How can people
| sit into a project at the start of the PhD and immediately crank
| out a valuable novel contribution? It isn't realistic, but we
| have to pretend because this is the background assumption behind
| a PhD. But the current state of affairs is inflating away the
| value and prestige of a PhD. It means less and less because
| everyone knows that it's not just a measure of scientific
| research skill and knowledge.
|
| However, what people are suggesting, namely making it less based
| on peer review and more on some social media like system is also
| misguided. It's like the instagrammization of the field: the
| sexiness, the dopamine hit, the celebrities.
|
| But this isn't specific to AI or CS or academia. This is how
| humans are. It's how status hierarchies are always gamed. Think
| office politics, actual politics, dating.
|
| _Atvthe same time_ actual AI methods are really getting better,
| it 's not a lie and not just hype. It's actually quite
| paradoxical how well the system works despite the flaws. Vision
| algorithms work really damn well, its very obvious to anyone who
| remembers the pre-2010 times. Perhaps there is somewhat of a
| stagnation in the last few years but also a lot is happening in
| smaller niches, as the big deep leaning ideas are adapted and
| applied specifically to particular tasks. The methods work, but
| the literature is distorted. You need to be an insider to know
| what to actually believe.
|
| [0] https://slatestarcodex.com/2014/07/30/meditations-on-moloch/
| StandardFuture wrote:
| > everyone knows it and we complain about it over some beers
| all the time.
|
| Then why not speak out publicly against it?
|
| Obviously because you know that there are still a significant
| number of individuals that _want_ the system to work this way.
| Your fear of those individuals is why you don 't speak out
| against it.
|
| So who are they? Who are you scared of?
| bonoboTP wrote:
| Well, I'm not talking about outright fraud but the more
| subtler forms mentioned in the article. I think it's probably
| harder for outsiders to understand the nuance.
|
| It's not "who are they", its everyone. The incentives are set
| up wrong. Blaming individuals is insufficient. It's like
| dreaming if only politicians stopped lying. Academia is a
| hierarchy and a power system. People have interests,
| reputation, prestige and it often directly translates to
| money.
|
| Again, I'm not talking about blatant outright lies or fudging
| numbers.
|
| Thing is, after some rejections or sabotaging yourself a few
| times by being too critical with your own research and not
| having publications you will imitate what you see around you
| or you drop out due to a lack of publications. It's rewarded
| to be less critical with yourself. And ultimately you want to
| graduate after all that sunk cost. Even switching to industry
| isn't so easy without publications in prestigious venues to
| show for those years.
| StandardFuture wrote:
| > I think it's probably harder for outsiders to understand
| the nuance.
|
| No, it's simple. It's called game theory. So, who benefits
| from the publish or perish system that drives this
| profoundly fraudulent behavior and outcomes in science?
|
| You are avoiding the "who" question. Why not _maybe_ ,
| _perhaps_ , identify the _who_ and get them removed from
| their position?
|
| It may or may not even be a fellow scientist.
|
| > lack of publications
|
| uh huh, science is not and never was a stationary process.
|
| It's absolutely disgusting that modern academia is filled
| with so many cowards and buffoons that the state of modern
| science has reached such a low.
|
| > Even switching to industry isn't so easy without
| publications
|
| Well, this is stupid. The only thing useful to corporations
| is your _training_ and not your novelty.
|
| Any novelty is a cherry on top, cream-of-the-crop pick and
| has nothing to do with being "hirable". But, that is also
| just an unchangeable aspect of our competitive reality and
| is thus entirely irrelevant to this discussion.
| bonoboTP wrote:
| I disagree that this is for the lizardmen, the leftists,
| the woke or whoever. No, it's us all. We need
| publications because otherwise research is hard to
| evaluate for outsiders. If it's evaluated by peers, it's
| politics again.
|
| There's a generic problem in academia and there is a
| specific one in the explosively growing and lucrative
| ones like AI. We can't think clearly if we blur all sorts
| of complaints against academia into a generic complaint
| against elites or something. Publish or perish also comes
| from the desire to measure, quantify things objectively
| etc. It's a generic trend in the management community.
| It's easier to hide behind mechanistic procedures, we
| live the age of the bureaucrat and not of the strong
| leader, for various historic reasons. Of the doctrine of
| equality, not of Great Men theories. But it all leads too
| far.
|
| I recommend the SSC articles on Moloch for thinking
| deeper about this.
|
| Why do parents chase extracurriculars and test drilling
| for their kid? Because it's competitive. With lower
| stakes, backwater academia can have lower pressure amd
| perhaps more attention and care.
|
| We cannot abstract away the human. This is not a mere
| technical problem of a problem of morals in a few
| individuals. Read some novels, drama etc. We in the
| technical fields would benefit a lot from learning more
| from the humanities, literature, even classics from
| Antiquity. Man has not changed, our nature is still rich
| and complex and the same lesson that power corrupts.
|
| Re training: outside the US, PhD programs have no
| explicit training. Training stops at the master level.
| From day one you just work on methods in the hope to
| publish the results. Advisors usually don't have a lot of
| time for each individual PhD student. Maybe let's call it
| experience, not training.
| tlb wrote:
| The idea that a field can be reformed by making it worse until it
| suddenly faces a reckoning and emerges much better is ... I don't
| know where people keep getting the idea that this might work. It
| has never worked in any field ever in history.
|
| The thing that can happen: fields gradually split into rigorous
| and non-rigorous camps. Like with evidence-based medicine, or
| chemistry/alchemy. Depending on the field, either might prevail
| in the market. Medical research and bridge-building are mostly
| rigorous, programming is mostly non-rigorous.
|
| AI/ML has a range of rigor levels, from fairly good to total
| crap. I think people in the field have a reasonable idea which is
| which. It's frustrating to people outside the field that think
| they can just take the technique with highest reported
| performance numbers and expect good results.
| Kye wrote:
| It's interesting that programming used to be rigorous. Whole
| programs were written out and verified before anyone would
| invest the computer time. IBM had a range of flowchart
| templates for it! Then the growing availability of computing
| resources flipped the cost-benefit calculation in favor of
| getting it done at any cost, and quality control processes
| haven't been able to keep up.
| salawat wrote:
| It's funny you mention that. I'm often surprised how little
| literature there is out there on Quality in general, and how
| often I have to depart from Software in general and head into
| manufacturing/metrology to get any new/useful insight on
| Quality Control methodologies for software. There is,
| however, more to the story than Moore's Law at work.
|
| #1 In the early days, you had very little between you and the
| executing machine. Your programs were more ways of doing
| things to ensure a particular machine would get you a
| reasonable answer, and it was far easier to communicate the
| totality of the stack, software+hardware combined.
|
| #2 Compare the picture of the 60's-90's programmer to the
| programmer of today. You had to know the hardware, and do
| cartwheels to decompose your problem to be solvable within
| the constraints of the machine which you had to run it on.
| You were a professional optimization problem solver+a
| studious cross-referencer (libraries and dependency
| management were not as mature as they are now, nor were there
| as many Virtual Machine constructs to foster write once, run
| anywhere.
|
| #3 Libraries were a case of build it yourself, or you ported
| something else by figuring out the toolchain + operating in a
| much smaller network of professionals to reach out to for
| guidance.
|
| #4 There weren't many if any concessions to programmer
| "comfort" (IDE's), less static analysis (as far as I'm
| aware), and good luck finding documentation without paying
| for it.
|
| Now: >Many programmers are blissfully unaware of cache
| coherency, memory hierarchy, or the quirks of the
| hardware/filesystems they run on. The hardware is "the
| compiler writer's job" or "those driver writer's problem"
| (smh?), the filesystem is Someone Else's Problem...
|
| >More and more, the solution looks more and more like "throw
| more hardware" at the problem to create more abstraction,
| which requires more intermediate steps, which takes more
| compute...
|
| If for no other reason, software is disgustingly hard to
| Quality Control for because nowadays, it's more about having
| access to development talent to make the core system
| architecture, no matter how inefficient or bloated, maintain
| it, and extend it rather than boiling things down to least
| computtational overhead.
|
| When you've got a thing built on a constantly shifting Tower
| of Babel, where your artisan knows less than a 10th of what
| their total set of dependencies are doing, and oftentimes are
| selected for their willingness to sit down, crank out the
| requirements, and not balk; it truly is a miracle when
| Quality software actually happens. Test coverage alone isn't
| it. Nor is refactorability, or readability. You have to have
| the right Software for the right people, at the right time,
| for the right costs, to bring about the right constellation
| of jobs done, to create value. None of that value, in modern
| thinking, should be intrinsically tied to the people making
| it. In fact, I have a theory that the market is trying to get
| away from that by favoring designs where everything you need
| is in the automation code itself, as it obviates the concept
| of the "heroic wizard coder" as we know it. Part of this
| Quality too, is you have to actually care about the people
| you are implementing for. Any programmer can create a program
| that works (painfully). It takes a sharp one to make
| something intuitive, quick to learn, and a pleasure to work
| with without being godawful slow.
|
| I'd say Quality software is not something you set out to
| build (at least the way the Market driven ecosystem we have
| today works), but a happy accident when you combine all the
| right factors to get the job done you set out to do and keep
| it that way. But the roadmap to getting there is the hard
| part.
|
| If anyone ever wrote a dynamic code generator that worked
| like a database optimizer (imagine Select implementation FROM
| C Where INPUTS(x1,..,xn) AND OUTPUTS (y1,...,yn) WITH
| CONSTRAINTS (z1,...,zn)), we'd all be out of a job, possibly
| all the healthier for it.
|
| Who am I kidding, we'd all just be query writers and
| optimization engine babysitters for it.
| joe_the_user wrote:
| I loved your first paragraph but I choked your second.
|
| All modern fields of study use quantitative measurement so
| splits in practice now would have to do with different ideas of
| rigor rather than rigor-nonrigor.
|
| The problem of AI/ML isn't the blatant cheating but the way
| that the goal is often getting n% higher than soto on X
| benchmark. Just chasing benchmarks makes any connection to
| broad dubious imo. It might or might-not give you something
| practically useful but definitely give you something career-
| wise useful.
|
| But just as much, when the field is just a giant race where no
| cares about any broader understanding, cynicism seems like a
| natural result. The ideal of academia, for all it's failings,
| is to give people some amount of space to speculate and explore
| wider vistas.
|
| It would be good if X number of people had the space to explore
| a variety of visions of "AI" other than the dominant one. But
| despite the vast number of people being sucked into the field,
| my guess is this is getting harder, not easier. And, of course,
| the mere appearance of "rigor", of quantitative measurement, is
| not helping things, again in my opinion as someone of no
| authority at all in field.
| dalbasal wrote:
| In politics and other areas, the concept is known as
| "accelerationism."
|
| It tends to be associated with political extremism... "bring on
| the revolution/war/etc." As an angry or disillusioned response,
| I think it's a close relative of nihilism. Cover for being
| destructive.
|
| That's not to say "worse before better" isn't a thing... it's
| just not a thing we can do usually.
| Aperocky wrote:
| I have the opposite view.
|
| Is AI as a field not destined for destruction? There were
| good stuff, but it's been out for a while and now it's just
| mountains upon mountains of crap.
|
| What does society stand to lose?
| mjburgess wrote:
| Well it does apply to individual people.
|
| Since animal psychology is mostly design to keep animals _safe_
| (not, eg., to _flourish_ ), we can persist in severely bad
| circumstances merely because they arent unsafe.
|
| What "getting to rock bottom" does is drive a person to acute
| unsafety so they are able to overcome the inherent risk of
| change.
|
| This "personal accelerationism" does work, indeed, it may be
| the only thing that works.
|
| I do agree that this likely does not generalise to
| institutions.
| [deleted]
| ceilingcorner wrote:
| Most improvements to safety have come after serious disasters.
| Seatbelts, non-hydrogen zeppelins, life boats on cruise ships,
| etc.
| biztos wrote:
| > Medical research and bridge-building are mostly rigorous,
| programming is mostly non-rigorous.
|
| Medical research and bridge-building are increasingly dependent
| on programming. Maybe they are less rigorous than we like to
| think?
| raphlinus wrote:
| I'll give my reading, and a tip to the author.
|
| To me, this did not read as a serious proposal to actually
| commit more fraud. Rather, it was a "modest proposal" in the
| tradition of Swift, in which the actual call to action was for
| the field to be more critical of papers, especially to be on
| the lookout for all fraud, both the obvious kind and the more
| subtle variants, the latter of which also do great damage to
| the field.
|
| The tip: humor like this is fun and appreciated by people who
| run in the same circles as the author, but an essay like this
| will be read by a diverse cross-section of people. Some won't
| have the cultural references, some won't have English as a
| first language, etc. Almost always when I've snuck jokes into
| my writing, I've found it causes confusion.
|
| So I might have written this a slightly different way,
| something along the lines of: the community is structurally
| more equipped to deal with blatant than subtle fraud.
| Ironically, now that we're seeing more egregious examples of
| fraud, there's a better chance that things will get better; we
| would have tolerated the subtle types for a long time, as lots
| of people benefit from the status quo.
|
| If the author were actually legitimately calling for more
| fraud, then I apologize for misunderstanding.
| dahart wrote:
| I assumed it was tongue in cheek, but the end of the article
| really convinced me that the author is serious about his 'let
| it get worse so it can get better proposal' stance.
|
| "Widespread fraud would force us to re-strengthen our
| community's academic norms, transforming the way we do
| research, and improving our collective ability to progress
| humanity's knowledge.
|
| "So this is a call to action: please commit more academic
| fraud.
|
| [...]
|
| "Let's make explicit academic fraud commonplace enough to
| cast doubt into the minds of every scientist reading an AI
| paper. Overall, science will benefit."
|
| Given the number of people in academics and politics and
| software and just about any field who argue from the 'let's
| burn it down and rebuild it' position, even if it is satire,
| it's basically impossible to tell.
| jacobbuckman wrote:
| Author here -- it's intended as satire, don't worry, haha.
| I know that nobody would ever actually do this so I decided
| to just lean into it. If I had any real power by which to
| fix things, I would have a much more nuanced take.
| raphlinus wrote:
| > I know that nobody would ever actually do this
|
| Or would they?
| thih9 wrote:
| > I know that nobody would ever actually do this
|
| What gives you this certainty, especially given Poe's law
| [1]?
|
| [1]: https://en.m.wikipedia.org/wiki/Poe%27s_law
| dennis_jeeves wrote:
| >Medical research
|
| Lol.
| josalhor wrote:
| Well, I wonder if this could force people to start publishing
| their code so others can replicate the results. This would help
| greatly with the kinds of issues that are being discussed:
|
| - It would be much harder to hide makeup results and cherry-
| picked seeds
|
| - Useless research would not be as easy to hide
|
| - The real-world impact of the research would be more valued.
|
| - Researchers would be forced to recheck their code for it to
| be presentable, which could be a net benefit in terms of
| finding mistakes
|
| - and so on..
|
| I do imagine this could harm the number of replications of the
| research. However, I think there can be a net positive effect
| from such a policy. What do you think?
| the_snooze wrote:
| I'm active in the security and privacy field, and some
| conferences have started "artifact reviews" [1][2], where
| authors can optionally submit the data and code that go along
| with their work.
|
| It's definitely an improvement, but the devil is in the
| details. Should these be mandatory? If they're mandatory,
| then what quality standards should artifacts meet? Who's
| ensuring those standards are followed? How should proprietary
| code and sensitive human-subjects research data be handled?
| There's also the question of what code should be made public:
| Is it just the analysis that produced the metrics on the
| paper (e.g., R scripts, Jupyter notebooks, etc.)? Or should
| it also include the data collection and pre-processing? How
| about the code that didn't produce immediately publishable
| results, but could be useful for future work?
|
| [1] https://petsymposium.org/artifacts.php
|
| [2] https://www.usenix.org/conference/usenixsecurity21/call-
| for-...
| josalhor wrote:
| > where authors can optionally submit the data and code
| that go along with their work.
|
| So cool. I wish there were more steps in this direction.
|
| > Should these be mandatory?
|
| As an outsider, I would say no. However that should have a
| detrimental effect on the ability to get published if you
| are from an academic institution. I can totally see a gray
| area when it comes to non-academic publishers.
|
| > There's also the question of what code should be made
| public
|
| Shouldn't it be as much as possible? The way I see it is
| that the easier to replicate a paper, the more people can
| focus on its true utility. That is, by abstracting
| ourselves away over the details of the paper, what does
| this really contribute? From my point of view, publishing
| as much as possible maximizes the real-world impact of the
| research.
| miltondts wrote:
| Great news! There are journals that do this! E.g.
| https://academic.oup.com/gigascience
|
| They require the full means for replication to be open
| access, and reviewers take this very seriously! They will run
| your code, open github issues if they detect bugs etc.
| bonoboTP wrote:
| These discussions remind me of how there's a years long delay
| before these ideas first reach boards like HN and then
| popular magazines and the taxi driver.
|
| Yes, code releasing is a big talking point and most major
| groups release code. It doesn't solve the subtler
| problems,like claiming 1% improvements as a solid
| contribution, unequal hyperparameter tuning, cherry picking
| datasets, overclaiming the novelty, salami publishing,
| lacking ablations, not comparing against better methods etc
| etc. Releasing code is a step but not a panacea. At least it
| partially protects against fully made up numbers which is not
| nothing, but it's just scratching the surface. In many cases
| the code is not so easy to use and the original authors may
| not be willing to help you with reproducing it, ignoring
| emails and GitHub issues. If you can't reproduce it, they can
| always claim something. In one case I was told they rewrote
| the codebase in a new framework since they generated the
| results in the paper and there may be small discrepancies due
| to that. Apparently that was enough change that they were no
| longer the best in the benchmark comparison. But nobody cares
| at this point, the paper was published, they are on to their
| next project. People also have the attitude that it doesn't
| matter anymore, the field has moved on already, a retraction
| doesn't change much just hurts the author and the group. The
| vast majority of papers isn't used anywhere actually in
| practice, the field is moving so fast that any method can
| only remain on top for a couple of months.
|
| Also, I don't think AI really is worse at this. It's rather
| that we are more open about it because we have less to lose
| and professors don't have an iron grip, due to the existence
| of industry. In some of academia your whole future depends on
| whose ass you kiss, it's better in CS. It's a bit like sexism
| accusations directed at nerds, as if they particularly bad.
|
| The molochian cancer is everywhere. The less you see it may
| just indicate its more effectively hidden.
| ssivark wrote:
| In the steel-manned implementation, I would like the plots
| and tables to be auto-re-generated on publication, from
| submitted code. That would ensure that the results can have
| no "hidden state" and the paper cannot claim anything more
| than what their publicly shared implementation guarantees.
|
| This is par for the course in theoretical work; we should
| strive to apply the same standards of rigor to experimental
| work.
| wiz21c wrote:
| AI seems to me as quite experimental. Designing better
| benchmarks (that is less cheatable) will help. Publishing
| code will go against IP policies, that should be handled at
| the political level : do you research for profit or for
| knowledge ?
|
| But yeah, code adds transparency and a whole lot of cheats...
| The funny thing is that any well organized coder will
| automate report production, so it's not like it is hard to
| do...
| bumby wrote:
| Publishing the code may not be enough. Part of the problem is
| that replication of a study isn't usually rewarded in the
| academic sphere because of the obsession with novelty within
| publications. Most researchers want to spend their finite
| time on something they find interesting and also may help
| their career. One way may be to create a movement to foster
| good replication studies that can be published and rewarded.
| unishark wrote:
| Novelty is in the job description though. I think the
| problem, especially in AI/ML is the obsession with "State
| of the Art" algorithms. This can be gamed by making claims
| with numerical "experiments". If an idea had to stand on
| its own, and be interesting even if it may not make a
| better pattern-recognition product, most of them probably
| wouldn't make it. Indeed that's essentially the
| requirements presented by reviewers and editors: "prove we
| should read about this otherwise-uninteresting method by
| giving numerical test results where it wins".
| bumby wrote:
| I'm not stating novelty isn't be part of the aim of
| research, I just think we need to be aware of the
| perverse incentives it creates when it becomes the sole
| underlying criteria by which merit is judged. I think
| there's room for expanding the definition of what is good
| research to include replication of prior art because the
| field in general will be better off for it. I imagine
| novel research would still be more prized but it wouldn't
| relegate replication to being considered a waste of time.
| OminousWeapons wrote:
| > Well, I wonder if this could force people to start
| publishing their code so others can replicate the results.
| This would help greatly with the kinds of issues that are
| being discussed:
|
| The real solution to the problem is to set a rule that says
| if you use a lab's published code base AT ALL for new work,
| you must cite the people who produced it along with the PI of
| that lab. No exceptions; acknowledgements don't count. Labs
| (especially those outside of CS such as life sciences labs)
| view code as IP and they don't want to release their code
| bases for two reasons: First, they don't think they will be
| properly cited when people use it (valid concern). Second,
| they think that they will lose their monopoly on future
| publications using the code base (less valid concern).
| Mandatory citation would solve both of those problems. Until
| a rule like this is in place, most labs will fight you tooth
| and nail on releasing code.
| fullshark wrote:
| This struck me mostly as tongue in cheek, and the the author's
| main point is the blatant fraud is just the tip of the iceberg,
| the subtle fraud is happening under the surface and affecting
| almost every field.
| lettergram wrote:
| I manage an applied research group in the machine learning space,
| particularly AutoML and synthetic data generation. That means is
| we read the latest literature and create applications. Sometimes
| we build new research, most of the time we patent something new.
| But the goal is to get research to production as fast and as
| impactful as possible.
|
| The first thing I do when sorting research (papers, blog posts,
| presentations, etc) is sort the research by "B.S.", "maybe B.S.",
| "probably real".
|
| If you look at research papers in computer science, half are what
| have the following problem: "running a simulation over and over
| until the authors get the results They like".
|
| I can't tell you how many papers with thousands of citations are
| just blatantly wrong. They don't run corrections based on the
| number of simulations they run, they don't take into account
| other variables, etc
|
| This isn't limited to CS either. Biology, economics,
| environmental sciences, etc all suffer the same fate.
|
| The worst part about it, is it's PEER REVIEWED. Meaning, others
| agree this is the way to do things, which is why I don't trust
| academia almost at all.
| Q57C3HYc7g wrote:
| Maybe this is a dumb question, but what do you mean by "run
| corrections based on the number of simulations they run"?
| L-four wrote:
| Say you run 5 simulations and when comparing to the standard
| approach as a percent the samples are [-1%, -1%, +2%, +2%,
| +%6]. Your new method is not 6% better the standard it's at
| best +1.6% better.
| Jabbles wrote:
| For example, https://en.wikipedia.org/wiki/Look-
| elsewhere_effect and more generally
| https://en.wikipedia.org/wiki/Multiple_comparisons_problem
| christophilus wrote:
| It's a problem and is the reason I default to skepticism. What
| is the solution, though?
|
| Skepticism is crippling-- and definitely annoying to others.
| Drives my family crazy, because it comes off as belligerent
| contrarianism.
| eeegnu wrote:
| If you keep track of these categories and have amassed a large
| enough dataset, it sounds like a fun experiment to see if a NN
| can learn some feature of the papers text to predict if a paper
| is BS. Some combinations of citations may even be a telling
| feature.
| galimaufry wrote:
| > They don't run corrections based on the number of simulations
| they run, they don't take into account other variables, etc
|
| I think this looks like a bigger problem specifically because
| you are in AutoML.
|
| Suppose you are training a GAN. There's notoriously a certain
| amount of luck involved in traditional GAN training, because
| you need the adversary and the generator to balance each other
| just right. So people try many times until they succeed.
| Probably they were not even recording each attempt, so they do
| not report how many times they had to run before getting good
| results.
|
| From an AutoML point of view, this is BS work - the training
| procedure cannot be automated, and (apart from using the actual
| seeds) the work cannot be reproduced.
|
| But from the point of view of everyone else, maybe it is fine.
| They get a generator model at the end, it works, other people
| can run it.
| pnt12 wrote:
| >But from the point of view of everyone else, maybe it is
| fine
|
| I think from a practical perspective, it is fine. You want
| results and you have a black box algorithm that produces
| them, fine.
|
| From an academic perspective, AI research is a mess. The
| reason you try something is not from a logical theory, but
| due from a "hunch" or replicatinga similar algorithm applied
| in a parallel area. If it does not work, you change some
| parameters and run it more times. Still not working so maybe
| you extend the network to include some more inputs and hope
| for better results.
|
| I did my thesis in machine learning and was very disappointed
| with the state of the field.
| caddemon wrote:
| I don't think there's necessarily a problem with trying
| things on a hunch, some of the best results in science have
| been due to a hunch or even an accident. The problem comes
| from trying a dozen hunches and only writing up one, or
| like you say completely cherry picking hyperparameters.
| goalieca wrote:
| I despised how data sets and code were considered a gold-mine
| and researchers would often refuse to release them lest they
| give up a serious paper publishing advantage for their group.
| More often than not, i found code full of serious errors and
| completely lacking basic test cases and sanity. Sometimes the
| data sets were much larger than the subset chosen for
| publication.
| geofft wrote:
| There's another solution: give up on academia.
|
| The model of academia works well when some part of society is
| willing to fund researchers spending time on open-ended basic
| research, even if it produces no results. Saying "Don't engage in
| unethical behavior" is well and good, but if the incentives are
| towards unethical behavior, you shouldn't expect very much.
|
| Academic funding and employment is very closely linked to the
| number of published papers. The article says that a grad student
| who publishes three papers a year is a professor's dream - but on
| reflection, it should be clear that this is not because three
| papers a year is necessarily good science, it's just a good
| metric. The professor's lab looks good for publishing so much,
| the professor can ask for more funding from grant agencies, and
| the student is likely to get a good faculty job which reflects
| well on the professor.
|
| So it's extraordinarily difficult to stay in the field, have a
| well-funded lab, hire enough grad students, and get tenure if
| you're optimizing for the quality of your science over the number
| of publications. The "publish or perish" culture (literally, in
| at least one case, it seems) isn't driven by the practitioners;
| it's driven by the requirements of getting grants and getting
| academic jobs in an increasingly competitive market.
|
| Now in some fields you do need the facilities from a university
| to do your research, but for CS in general - and especially for
| AI - you'll be just fine in industry. All you need is a pile of
| cloud VMs, and industry can get you that. Perhaps you also need
| interesting data and real-world problems; industry can get you
| that too.
|
| Society today underfunds academia and overfunds industry. You,
| the individual researcher, are not going to be able to fix this.
| Go where the incentives are better aligned for you.
|
| (And if you're a Ph.D. student who sees no future in your life
| after being complicit in fraud, please, please reach out to
| industry. Your prior publications are nowhere near as high-stakes
| as in an academic job; you can contact the journal, get it
| retracted, drop out of the program, and have a great life ahead
| of you. What saddens me is that the student who allegedly took
| his own life was probably so deeply surrounded by academia that
| he didn't know there's a world who won't judge him for having one
| fewer paper on his CV and will even look on him _positively_ for
| reporting fraud and getting his advisor into career trouble.)
|
| ---
|
| That said, my argument is undermined a bit because the papers OP
| denounces have coauthors in industry. So I don't have an answer
| for why those coauthors went along with it.
| YeGoblynQueenne wrote:
| >> All you need is a pile of cloud VMs, and industry can get
| you that.
|
| That is true if by "AI research" you mean "applications", as
| opposed to contributing anything new. To contribute new
| knowledge what is needed is time and the resoruces to study the
| work of those who have contributed new knowledge before you,
| and to become an expert in that knowledge. In theory, that's
| half the job of a PhD student. The other half is creating new
| knowledge. Beating benchmarks is not the job of a researcher,
| despite what's the norm in machine learning research these
| days.
|
| >> So I don't have an answer for why those coauthors went along
| with it.
|
| Because the motivation of researchers in industry is money, not
| knowledge. Academia is capable of motivating researchers to
| create new knowledge. It is also capable of motivating them to
| commit academic fraud. But industry only understands one motive
| and can only offer one reward.
|
| Basically, if your university pushes you to publish or perish,
| you might publish bullshit, or perish, or find yourself a niche
| where you can publish something that isn't bullshit. Many
| researchers do this, but of course you don't hear about them
| because they're not in the news. In fact, you probably hadn't
| heard about Geoff Hinton, Yoshua Bengio, Yan Le Cunn and You-
| Again Schmidhuber before the big boom of deep learning, because
| at the time they were just such small-fry researchers (compared
| to what they are today- I once had a look at Hinton's
| publication record and he was anything but small fry
| academically, even in the 1980's, when the lore says he was
| roaming the academic wilderness isolated from the mainstream;
| it's all bollocks). For sure, no Google was throwing millions
| at them at the time and the machine learning community as a
| whole had more or less given up on neural networks, or rather
| the majority of machine learning researchers where happy to
| leave Hinton et al and Schmidhuber to work on neural nets,
| while everyone else was working on Bayes nets, then decision
| trees, then SVMs, etc.
|
| I digress, but what I'm trying to say is that "publish or
| perish" is the norm for researchers who are not motivated
| enough, for their own personal reasons, to make meaningful
| contributions. They are not the only kind of researcher and
| when progress happens, it comes from the other kind. But
| researchers in industry only have one motive, because industry
| offers only one incentive and is only interested in results
| that satisfy that incentive- regardless of how those results
| are brought about.
| geofft wrote:
| If the noble of heart can get good work done in academia
| despite the pressures on them, why can they not get good work
| done in industry despite the pressures on them?
| YeGoblynQueenne wrote:
| Because the "pressures" in industry is that you get fired
| if you don't do the work you're hired to do. There's still
| a modicum of "academic freedom" in academia, the freedom to
| pursue your research interests, whatever those may be. In
| industry, you pursue your employer's interests and if they
| are not also your interestes, tough.
|
| Speaking in this with the experience of working in the
| industry for a few years, then going into academia,
| precisely because I got bored doing other peoples' work and
| I wanted to do what I'm interested in.
| cycomanic wrote:
| I find it quite ironic that every time some unethical academic
| behaviour is published on HN, there's several posts about "move
| to industry this never happens here". I mean this is HN, a
| significant portion of the industry is working on optimizing
| for clicks and keeping people engaged (addicted?) to some
| platform or the other. Let's not even talk about the blatantly
| unethical (and often times illegal) things that happen in the
| startup world to increase evaluations, secure funding etc.
|
| Now academia has lots of issues, but to say industry is
| better... It's sort of like telling an amateur athlete who is
| upset by some competitors using caffeine (or amphetamines) to
| instead to professional sport, because they are more ethical.
| The stakes (and money) are much higher so people are more
| willing to cheat to gain an edge.
| geofft wrote:
| Sure, but as an individual researcher, the integrity of your
| scientific work is not compromised by the unethical goals of
| your employer or the unethical ways in which it raises
| funding. That's what I'm claiming. It's rather different from
| amateur vs. professional athletics, where both have the same
| goal of "run as fast as possible but within some nebulous
| concept of natural human ability": the goals are different
| between academia and industry.
|
| Or put another way: Of the many deep moral questions raised
| by the Manhattan Project, not a single one was "Did they
| commit academic fraud and claim that an atomic bomb was
| scientifically possible when it wasn't?" They were employed
| to actually get the job done, not to act as if a job were
| getting done.
|
| And at the end of the day, funding for university AI labs is
| largely driven by the existence of those same unethical
| industry goals - governments fund the work because it's good
| for the economy, students pay to take AI classes in their
| undergraduate degrees because it's an investment in their
| future careers, the industry donates everything from
| fellowships to entire buildings to academia, etc. I don't
| think you can cleanly wash your hands of industry's ethical
| concerns by staying in academia but working on the problems
| that industry finds interesting. If your goal is employment
| without _any_ ethical concerns, you 're going to have a very
| hard time.
|
| (In the case of this particular post, the co-authorship of
| papers with industry makes it clear that the research
| directly benefits industry.)
| rscho wrote:
| > as an individual researcher
|
| And how is an individual researcher funded, if I may ask?
| We already have many homeless teachers living in their
| cars, so I don't think the power of public opinion will
| rebalance financial incentives to make an even more useless
| profession (in the eyes of the general public) attractive.
| Joe-Schmoe-the-janitor won't see financially supporting
| lone nerds as a first priority.
| [deleted]
| geofft wrote:
| Huh, I thought I was pretty clear that I meant an
| individual researcher employed by industry to do their
| research (" _for CS in general - and especially for AI -
| you 'll be just fine in industry.... Go where the
| incentives are better aligned for you._") - was it not?
| rscho wrote:
| So, limiting research to projects small enough for a
| single individual, then? Because similar situations will
| and do arise in industrial research groups.
| geofft wrote:
| Ah, I see the confusion. By "individual researcher" I do
| not mean "independent researcher" - I mean the
| researcher, considering their motivations as an
| individual person. They can be part of a group of
| researchers, and they're generally working for some
| employer. After all, the "individual researcher" in
| academia is generally working with a group - e.g., the
| example given about the grad student who is "every
| professor's dream" only makes sense in the context of
| that relationship - and that produces the negative
| pressures described.
|
| In particular, I mean that a researcher in academia, as a
| person (an "individual researcher"), is motivated by the
| demands of academia to get grants and fill their CV and
| is therefore incentivized to conduct dishonest science to
| make that happen, and a researcher in industry, as a
| person (an "individual researcher"), is not directly
| incentivized to conduct dishonest science - perhaps
| there's dishonesty in how their employer gained funding
| or what they do with the research, but that doesn't
| compromise the accuracy of their research, motivate them
| to game the peer review system, etc. The researcher as an
| individual has the choice about whether to be in academia
| or industry.
|
| So, I don't think I follow how similar situations will
| arise in industrial research groups. (Though, as
| mentioned in my original comment, I'm probably missing
| something, because there were researchers from industry
| who coauthored these papers.) Even among a group of
| researchers in industry, the incentives should be to
| produce things of value to the employer, not to play the
| part of productive-looking researchers.
|
| I'm specifically not claiming that _independence_ will
| solve anything; the fundamental problem is funding, and
| (as you say) nobody is going to want to live out of their
| car to do good research. And you need access to
| facilities /tools of some sort to do your research; my
| claim is that industry can provide those at least as well
| as academia can, not that they are unneeded. Admit that
| you're constrained to work at a place that can fund your
| research and that no place exists that will pay you well
| and leave you to your own devices, and then find the
| place whose incentives to fund you are least likely to
| compromise your research integrity and most likely to
| reward you for actual good work. At the moment, at least
| in the society where I live, that happens to be industry.
| bonoboTP wrote:
| Depends on what industry you're talking about. Those that
| publish are working according to the same practices. Google,
| Facebook et al. need to pass peer review too and they care
| about papers as a dick measuring contest.
| Delk wrote:
| > What saddens me is that the student who allegedly took his
| own life was probably so deeply surrounded by academia that he
| didn't know there's a world who won't judge him for having one
| fewer paper on his CV and will even look on him positively for
| reporting fraud and getting his advisor into career trouble.
|
| Could be that, or could just be that the student couldn't
| imagine himself not doing academic research despite knowing of
| the rest of the world.
|
| It's sad nevertheless, of course, and I'm not trying to say
| you're wrong in any way. It's just that people's motives and
| the reasons they feel trapped can be difficult to guess.
| dalbasal wrote:
| The level of academic unhappiness with publishing and its
| surrounds is actually surprising to me.
|
| First, I never hear the counterargument.. that current publishing
| is OK. Does it exist, or is everyone unhappy while nothing
| changes?
|
| Second, I'm surprised at how much of a single institution
| academia seems to be. "Publish or Perish," for example, has to be
| supported by tenure committees, grant makers and such. Are they
| all the same?
|
| Michael Stonebraker^ suggests that tenure committees accept a
| limited number of papers on a resume. This is to encourage lower
| volume, higher quality publication. What stops _some_ tenure
| committees from implementing such changes.
|
| I just don't understand what's locking everything into place.
|
| ^https://www.youtube.com/watch?v=DJFKl_5JTnA&t=853s
| OminousWeapons wrote:
| The fundamental problem is that there isn't enough money in the
| system and there aren't enough professorships for the legions
| of PhD students that exist. We either need to cap the number of
| PhD students much more aggressively or we need to spool up many
| more institutes and increase funding dramatically. If you set
| the bar for tenure at people who are publishing in Nature or
| Science or prestigious journal X then people will just come up
| with ways to game that too.
|
| The other thing that goes unsaid is that huge swaths of
| researchers in the US are immigrants on visas. If they cannot
| find that next postdoc they will be kicked out of the country.
| This creates a massive incentive to do whatever you need to do
| to get your next opportunity.
| ModernMech wrote:
| It's not the papers per se that get you tenure -- it's the
| grants you bring in, and you can only get those by going
| through funding agencies, who are the ones that care about your
| publications. The grant process is like this: you write a
| proposal that says "I want to do this. Here's how it will work.
| I've published these papers to support my idea. Now give me $x
| million dollars". That pitch only works if there is a
| significant publication record to back it up.
|
| That's why there's a monoculture -- everyone is going through
| the same agencies: nsf, darpa, and a handful of other
| government/institutional funding sources. If you are bringing
| in 10s of millions of dollars worth of funding into the
| university with only a handful of papers, you're going to still
| have a lot of support for your tenure. The flip side is that
| you can publish all day every day, but if you're not bringing
| in grants you're not getting tenure.
| stevenbedrick wrote:
| Ding-ding-ding! This is indeed the root cause of much of the
| pressure that drives academic researchers to publish All Of
| The Things. Funding agencies care very, very, very much that
| you have a published track record of having done something
| similar to whatever it is that you are proposing to do, and
| regularly will reject otherwise solid applications due to the
| PI not having sufficient relevant publications. Not saying
| it's right or wrong (that's too big of a question for an
| Internet comment) but it's one of the main motivating forces
| behind anything that a researcher does.
| gravypod wrote:
| > If you are bringing in 10s of millions of dollars
|
| What's even more fun is what happens when you _do_ bring in
| those funds. It doesn 't go directly to your research and you
| will have restrictions on how you can spend the money. Some
| organisations take between 0% and 70% of off the top. Then,
| they require you to spend your money on specific areas. We
| wanted disk space for storing some data. They wanted to
| charge us >$1/GB + cost of backups to store this data and we
| couldn't just go to AWS or any generic cloud vendor.
| Al-Khwarizmi wrote:
| Well, this trend is more or less everywhere but it's not
| uniform, some countries have much more extreme "publish or
| perish" cultures than others.
|
| For example, in Spain practically everything related to the
| academic career (promotions, salary decisions, tenure, etc.) is
| measured by pure bean counting. Actual quality doesn't matter
| at all, it's all about having X papers in Y journal quartile
| during Z period. Evaluations are conducted by scientists but
| they might as well be clerks, because there is usually a
| grading scale on everything saying "each paper in this quartile
| gives X points" so the weight of the actual opinions of the
| evaluators is zero or close to zero (even if they read one of
| those papers and it's total bullshit, the regulations still say
| that they have to give it X points). As a result, the kind of
| subtle fraud described in the post is widespread, with most
| people gaming metrics and trying to publish in the crappiest
| possible journals that happen to have a high impact factor.
|
| In contrast, for example in France things seem to be more
| relaxed. I don't know the exact details as I'm not from there,
| but I have many colleagues from there and I do know that they
| can actually spend years thinking about very difficult
| problems, and then publishing a really good paper (even if it's
| in a journal or conference without such good metrics) and their
| career seems to be more or less fine. As a result, research
| coming from France in my field tends to generally be really
| high quality (IMHO) even though in Spain those people's CVs
| would be considered bad due to not optimizing metrics.
|
| The US seems to be somewhere in between, China looks similar to
| Spain, Northern European countries seem to be somewhere between
| the US and France in the spectrum, and so on.
| goalieca wrote:
| I ran into these issues in an adjacent field a generation ago.
| Many candidly spoke to me about having to be a 'team player'.
|
| This is all due to the criteria for survival and graduation.
| Let's be honest, not every idea will work and this does not
| reflect badly on the researcher who came up with it. But the only
| way to graduate and put food on the table is to keep getting
| funding through novel papers that have great results.
| StandardFuture wrote:
| "Novel papers" with "great results".
| MattGaiser wrote:
| > this does not reflect badly on the researcher who came up
| with it.
|
| > But the only way to graduate and put food on the table is to
| keep getting funding through novel papers that have great
| results.
|
| So in the way in which it matters, it does reflect badly on the
| researcher who came up with it.
| goalieca wrote:
| Science involves a lot of risks but we don't reward negative
| results, lessons learned from methodology problems, or failed
| reproductions except in rare studies. These issues come up
| all the time and many resort to tactics in the article to
| keep their funding / graduation plan alive.
| bonoboTP wrote:
| Well, does it reflect badly on a startup founder if their
| idea doesn't work out? But since things are subtler in
| science, it's possible to sell stuff and pollute the
| literature even if it didn't really work out. To do otherwise
| is altruistic but ultimately outcompeted by those with less
| qualms about it.
| MattGaiser wrote:
| > Well, does it reflect badly on a startup founder if their
| idea doesn't work out?
|
| Maybe not in SV, but I have heard plenty of people tell
| startup founders to "get a real job" or refer to trying a
| startup as a "figuring it out phase."
|
| When a startup some friends founded failed, they had to
| deal with people wondering if they chose the startup route
| as they couldn't get real jobs upon graduation.
|
| So, yes, in some places.
| bonoboTP wrote:
| Either way, even if the problem will be solved it will be the
| small guys who get screwed. The top dogs have built their careers
| on this bullshit, now they pull up the ladder and would want to
| make the playing field harder for anyone else.
|
| Typical strategy also with startup companies that skirt the dark
| gray zone of the law when they are small, but then when they
| extract enough money, they suddenly demand regulation so the rest
| of the up and coming are smacked down.
| Igelau wrote:
| That was my gut feeling about the ACM article that the author
| here refers to. The vague expose in ACM is made by someone in
| the comfortable position of "Royce Family Professor of Teaching
| Excellence in Computer Science at Brown University" about
| people who aren't anywhere near that. It is like saying: please
| stop treading water so desperately, you are splashing those of
| us in the lifeboats.
| igorkraw wrote:
| I've met Jacob in person and we had a super interesting
| discussion about RL that sadly petered out during the covid
| lockdown. This blog post very much fits my impression of him and
| I give mad probs to shitting on your own work (particularly the
| thermometer encoding was a pet peeve of mine).
| williesleg wrote:
| Global anything.
| blagie wrote:
| It's worth noting: In most of these cases, even when caught, the
| guilty parties and the bad papers remain unnamed. There are
| known-to-the-editors fraudulent papers in major journals still
| being cited.
|
| The consequences of committing academic fraud are minor, and the
| consequences of not doing so generally mean no academic jobs or
| tenure.
|
| There isn't a 99.99% innocent claim here (as at the end of the
| article). This stuff is widespread. This is much more accurate:
|
| "because everybody is complicit in this subtle fraud, nobody is
| willing to acknowledge its existence"
|
| Littman is at Brown, where the majority of the CS department
| engages in this !@#$%. I'm not trying to single out Brown. Parts
| of MIT, Stanford, etc. are even worse.
| MattGaiser wrote:
| With laws, there are full time police officers, investigators,
| and prosecutors working to catch people. And even then, most
| crimes go unsolved and unpunished.
|
| Even without the complicity, stopping fraud and bad behavior is
| a difficult problem. Part time reviewers and reputation seem
| woefully insufficient counters.
| qPM9l3XJrF wrote:
| Yes, I'm very skeptical of Buckman's claim that more fraud
| will help. Seems similar to a claim around 2015-2016 that
| voting for Donald Trump will inspire America to clean up its
| act politically. Institutions aren't biological systems that
| operate according to mysterious hormetic processes.
| Institutions are created by humans and thrive or decay based
| on whether they effectively channel human effort. Be the
| change you want to see in the world. And if you wreck
| something, and others apply blood sweat and tears to recreate
| it in a way that's better than the original, you don't get to
| take any credit as the wrecker.
| blagie wrote:
| I think the overwhelming problem is the amount of _trust_
| people have an academia, and especially, in elite academia.
| A Harvard research paper is _trusted_. The harm of people
| trusting fake science is high. I see it every day in my
| field.
|
| 1) If people realize it's more like a Facebook post, that
| will be better. People will be able to push back.
|
| 2) Alternatively, if we clean up this mess, that will be
| even better. Academia ought to be trustworthy.
|
| I don't see a path to #2 without a lot of dishonest people
| with tenure being laid off. At elite institutions, _most_
| people hired in the past decade or two cheated at least a
| little bit. I don 't see a path to get there without a
| high-profile scandal.
| MattGaiser wrote:
| I suspect that we see terrible events as changing things
| for the better because when something major happens, we
| notice a strong reaction.
|
| We forget all the not quite as awful things that were just
| let slide and normalized.
| FabHK wrote:
| That would be interesting, if universities had full time
| academic integrity officers whose job was to detect academic
| fraud.
| immmmmm wrote:
| after having spent a third of my life in academia i don't
| see this happening: everyone want as many as possible
| articles, if possible in the most high impact journals.
| detecting fraudulent or irrelevant pieces is against that
| goal.
| matthewdgreen wrote:
| I don't think people understand how hard this would be, or
| how much it would impede science. To give an analogy to
| coding: imagine a company with thousands of programmers,
| all using different languages and writing different types
| of code (mobile apps, back-end server apps, mainframe code,
| device firmware). Now assign one or two poorly-paid
| employees whose job is to review _all of their code_ and
| find issues with it.
|
| Can you imagine how annoying this would be? How much time
| you would spend simply explaining to these people what your
| code does, so they could understand the basics of what your
| program is even doing? How ineffective they would be at
| detecting actual fraud, and how quickly their processes
| would turn into yet another annoying layer of bureaucracy
| and checklist compliance?
|
| There are a lot of things Universities could do to assist
| scientists in producing better output. This is not one of
| them.
| MattGaiser wrote:
| Or just have some academic equivalent to SDETs. Have a
| replication and analysis team that checks that statistical
| quality, code, basic science, etc.
| seesawtron wrote:
| Previous thread: https://news.ycombinator.com/item?id=27295906
| huijzer wrote:
| I blame the incentives. Academia is not about selling facts, it
| is about selling papers. This is very similar to newspapers; they
| are about selling newspapers.
| thaumasiotes wrote:
| Hmmm.
|
| Here we see the following claim:
|
| > By partaking in a form of fraud that has left the Overton
| window of acceptability, the researchers in the collusion ring
| have finally succeeded in forcing the community to acknowledge
| its blind spot. For the first time, researchers reading
| conference proceedings will be forced to wonder: does this work
| truly merit my attention? Or is its publication simply the result
| of fraud?
|
| But I don't see how this follows. If I follow the link to the
| description of the actual fraud (
| https://cacm.acm.org/magazines/2021/6/252840-collusion-rings...
| ), it says essentially the opposite: the "fraudulent" papers are
| no different from papers published by ordinary means.
|
| > the review process is notoriously random. In a well-publicized
| case in 2014, organizers of the Neural Information Processing
| Systems Conference formed two independent program committees and
| had 10% of submissions reviewed by both. The result was that
| almost 60% of papers accepted by one program committee were
| rejected by the other, suggesting that the fate of many papers is
| determined by the specifics of the reviewers selected
|
| > In response, some authors have adopted paper-quality-
| independent interventions to increase their odds of getting
| papers accepted. That is, they are cheating.
|
| > Here is an account of one type of cheating that I am aware of:
| a collusion ring.
|
| > A group of colluding authors writes and submits papers to the
| conference.
|
| > The colluders share, amongst themselves, the titles of each
| other's papers, violating the tenet of blind reviewing
|
| > The colluders hide conflicts of interest, then bid to review
| these papers, sometimes from duplicate accounts, in an attempt to
| be assigned to these papers as reviewers.
|
| > The colluders write very positive reviews of these papers
|
| So the system is: conferences already can't tell the difference
| between a good paper and a bad paper. Researchers respond by
| adopting strategies for passing review that are irrelevant to
| paper quality (since paper quality doesn't count). But those
| strategies aren't _bad_ for paper quality. If I 'm reading
| conference papers, why would I worry about whether one of them is
| the product of review collusion?
| anonymousDan wrote:
| Ya the guy has barely read or understood the article he is
| basing all his complaints on.
| YeGoblynQueenne wrote:
| >> If I'm reading conference papers, why would I worry about
| whether one of them is the product of review collusion?
|
| Well, because if there's no incentive to write a good paper,
| very few good papers will ever get written and you'll waste
| your time reading crap papers.
|
| I get what you say: if the process is random anyway, then
| what's the problem? But the process is not random: authors
| maximise their chances to publish their paper if they partake
| in a collusion ring. Partaking in a collusion ring diverts
| resources from writing a good paper, therefore the quality of
| published papers goes down the drain. The most important
| resource necessary to write a good paper of course is the
| motivation to not publish crap. If that goes, everything else
| follows.
| [deleted]
| user-the-name wrote:
| The author is saying that all of the papers are bullshit,
| garbage, and a waste of time. He is saying that everyone is
| turning a blind eye to this. He is saying that now that the
| fraud has got this bad, it is out in the open, and people will
| be forced to deal with the fact that the entire field is filled
| with garbage.
| YeGoblynQueenne wrote:
| That's right. Btw, I think the author is wrong. This is not
| "out in the open" now. It's been "out in the open" since for
| ever. And those who act all surprised about it are probably
| the ones who knew it better than most.
|
| I don't mean the author, of course, but he is pointing to
| some of his own papers as bullshit. Why did he publish
| bullshit papers if he knew they were bullshit? Or doese he
| mean those papers are bullshit with hindsight? That, I can
| understand- "I wrote this paper ten years ago and reading it
| now, I cringe". Sure, that happens and it's only evidence of
| the person's progress. But to say that one's research was
| bullshit in the context of an article that describes academic
| fraud that leads to the publication of bullshit? That I don't
| get. Surely the author is not confessing to committing
| academic fraud himself! I didn't get that from the footnote
| anyway.
| jacobbuckman wrote:
| When I published those papers, I was new to the field. I
| was guided by the standards set by the community, by my
| mentors, by my peers. I was proud of each of those papers.
| At publication time, I believed I was doing good science,
| and the belief was re-affirmed by acceptance at top
| conferences.
|
| My thinking has evolved since then. The community norms
| have not.
| YeGoblynQueenne wrote:
| Oh, er, sorry for the harsh words, anyway I hope they
| didn't come across as too harsh.
|
| So I think you're saying the papers you linked to where
| "bullshit with hindsight", not that they were unethical
| at the time. Or at least you didn't think they were.
| tom_mellior wrote:
| > If I'm reading conference papers, why would I worry about
| whether one of them is the product of review collusion?
|
| Because the one you are reading may have crowded out a better
| one. Even if the current review system is essentially random,
| replacing it with something that is essentially a contest of
| well-connectedness is worse. Young researchers with good ideas
| but fewer connections, or people from less well-known
| institutions would have their ideas suppressed.
|
| So you should be worrying about stagnation, and about not
| reading what might actually be new and exciting.
| thaumasiotes wrote:
| None of that reflects negatively on the paper. There is no
| additional caution warranted when reading papers. That's just
| a question of "are you happy with the state of the world?".
| You can think about that question any time.
| YeGoblynQueenne wrote:
| >> None of that reflects negatively on the paper.
|
| I think it does. Good papers are written by people who care
| about doing good research, whose primary motivation is to
| do good research. Such people will not accept to collude or
| commit academic fraud to get their papers published,
| because they are idealistic and are in research because
| they want to do useful work. To such people, committing
| fraud is anathema, for personal reasons that have nothing
| to do with economic or other incentives.
|
| There are such people in academia but they are also crowded
| out, to borrow tom_mellior's turn of phrase, from others,
| who don't hesitate to commit academic fraud to get
| published and who don't give two flying figs about the
| quality of their own work. This is obviously a concerning
| state of affairs that can only be detrimental to the
| overall quality of research.
|
| So I'm sorry but you're dismissing the issue out of hand
| without having thought about all the consequences. Academic
| fraud is like, I don't know, broken windows? It just
| perverts everything around it and creates a black hole of
| bullshit that sucks everything in it. Good research cannot
| thrive in such conditions.
| [deleted]
| 2sk21 wrote:
| Thank goodness I am retired and don't have to publish stuff in
| the current environment. Thirty years ago when machine learning
| was a sleepy backwater, life was a lot easier for us.
| sillysaurusx wrote:
| As a present-day researcher (or at least someone who aspires to
| be), I'm eagerly waiting for the hype to die down, and for ML
| to become a sleepy backwater again. You were lucky to play
| around with ML during those days.
|
| The neat thing is, the next AI winter, we'll still have massive
| hardware rigs. That's very different from 30 years ago. It's a
| lot easier to discover new things when you can just test every
| possibility overnight, rather than carefully planning. So if
| "winter is coming," it will be a lot less harsh this time.
| YeGoblynQueenne wrote:
| >> The neat thing is, the next AI winter, we'll still have
| massive hardware rigs.
|
| So the next big thing in AI will still come from brute-
| forcing solutions by sheer power of compute. To clarify, you
| say this is a good thing?
| busterarm wrote:
| That kind of compute still costs money. You still have to
| find the funding for it.
| sillysaurusx wrote:
| Nah, a motivated hacker can use a Colab TPU or GPU for
| free. It's how I started.
|
| There's an ungodly amount of resources available now
| compared to even 10 years ago, let alone 30. The bottleneck
| is usually motivation.
| YeGoblynQueenne wrote:
| Note that all the advances that eventually led to today's
| successes and applications of deep learning happened 20
| or more years ago (e.g. LSTMs was in the '90s, the
| neocognitron in the '80s and so on).
|
| Then the hardware and the data came online and the field
| boomed, but no progress has been made in terms of new
| approaches and the overall state of research has
| stagnated with the inevitable results discussed in the
| article above (I can quote at least one Turing Award
| winner on that if that sounds just like me bloviating).
| In a sense, because it's now easy enough to get good
| results by throwing a bunch of data at a large computer,
| everyone's at it and nobody is looking for a way to
| obtain good results _without_ a lot of data and a big
| computer. Which of course is not sustainable, not least
| because it means only large corporations can obtain state
| of the art results (and to be honest, I don 't think any
| such have been obtained by "motivated hackers" using
| colab).
|
| No, my hope is that things will continue as they are now,
| until academics realise they've been kicked out of the
| game by the Big Players, and then go look for a way to
| compete that doesn't require gigantic amounts of compute.
| After all, academia _can_ still motivate people to do
| actual research and find actually new things. The same
| motivation is much harder in industry, that only cares
| about one thing and only knows how to hand out one kind
| of reward.
| kordlessagain wrote:
| "The hope is that the progress in hardware will cure all
| software ills. However, a critical observer may observe that
| software manages to outgrow hardware in size and
| sluggishness."
|
| https://en.m.wikipedia.org/wiki/Wirth%27s_law
| [deleted]
| zx2391 wrote:
| I experienced parts of CS academia and it was off-putting to say
| the least. Full of mediocre people who understood "the system" -
| not that much different from a direct-marketing operation, or a
| mafia - all while pestering CS professionals working in the weeds
| with their crazy-talk. Creating papers out of things, my
| colleagues and I talk while having a coffee. I'm not joking,
| intelligent guys get high on their own supply and will sell you
| ideas, that never gonna fly in the real-world with a straight
| face - and they will try it over and over again. It's
| embarassing.
|
| Personally, I'm trying out the route of getting the resources
| myself to do my research, much more work - but at least sane work
| with a level of independence that I feel is necessary to see
| things through.
| [deleted]
| fighterpilot wrote:
| I watched a lecture by an MIT database professor who recommends
| that the top 10 schools only allow up to 3 papers on CVs for
| junior faculty applications and 10 for tenure applications.
|
| He thinks this will be enough to start a culture shift towards
| quality over quantity, which could go some way towards addressing
| the fraud and collusion ring issues by removing the incentive for
| these behaviors.
| szarnyasg wrote:
| Was it Mike Stonebraker's talk "My 10 Fears about the Future of
| the DBMS Field"? (2018 Donald B. Gillies Memorial Lecture,
| presented at the University of Illinois):
| https://www.youtube.com/watch?v=DJFKl_5JTnA
| fighterpilot wrote:
| Yeah that's it.
|
| Some other notable points from that talk:
|
| - He thinks it's not an overly difficult problem to solve. It
| just needs some grassroots push from a handful of people in
| each of the top CS departments. It's just that not many are
| really trying to push for it. (I assume there would be people
| staked in the current system, though, who want to maintain
| the status quo.)
|
| - Nowadays he sees people trying to split research up into
| Least Publishable Units in order to maximize the number of
| papers.
|
| - Coming up with Postgres would've been impossible in today's
| climate since the time it took would entail an insufficient
| number of papers to get hired or promoted.
|
| - He thinks part of the reason that the culture has
| deteriorated is that Western universities have adopted
| cultural norms of East Asian universities, for whatever
| reasons.
| DistressedDrone wrote:
| If the universities wanted the problem fixed it would be fixed.
| I truly don't believe research universities are focused on
| producing value (other than to themselves).
| fighterpilot wrote:
| Apparently it wasn't always like this, and it used to be
| common for junior faculty hires to have only 0 or 1
| publications coming out of their PhD program.
|
| Why did the culture deteriorate so much and become so
| myopically focused on weakly informative metrics like
| publication count?
| atty wrote:
| When fields are small, they can rely on much higher
| fidelity signals for hiring (having significant
| interactions with the individuals at conferences and
| workshops, for instance). I did my PhD in a sub field of
| nuclear physics, and even though I only published one paper
| in my PhD, I got a job offer at an Ivy League university
| because I had done extensive work with their group in
| collaboration, and the group leaders liked me.
|
| However, now that many research fields are so massive that
| it is impossible to personally know the majority of
| individuals, institutions need other ways of judging
| individuals. The number of papers published is a weak
| signal, but it's better than nothing. Now that it's being
| so heavily gamed by so many individuals, that signal
| strength is decreasing even more.
|
| There's also a second strong corrupting factor that many,
| if not most, of these individuals do not want to become
| professors, they want to get a high paying job in industry,
| which means their short term output is far more important
| to them than their long term reputation in the field.
|
| I honestly don't know what can be done to fix this that
| wouldn't have negative side effects. But perhaps the side
| effects would be better than the situation we are in now.
| magila wrote:
| Because government grants demand it. Private patrons can
| trust their own judgment when deciding who to fund, but
| when it's taxpayer money being handed out people are
| understandably going to demand objective metrics to guard
| against corruption. In academia the objective metrics of
| choice are publication and citation count, so here we are.
| christophilus wrote:
| This is the answer, or a very large part of it.
| Incentives are set up such that this is an inevitable
| outcome.
| _dps wrote:
| Two likely contributors:
|
| 1) due to population effects, academic positions are much
| more competitive now than they were in say 1970; if you
| figure that the top 50 research universities are not
| generally expanding the number of professors, and that new
| professors generally also come from those top 50 research
| universities, then on average a top-50-research-university
| professor will generate one new such professor in a career,
| despite having 10-100x as many graduate students (this was
| different in the 70s when the university system was rapidly
| expanding).
|
| 2) the increasing desire for fairness in hiring and
| promotion (by itself, a good thing) means that you need to
| be able to resolve hiring and promotion disputes with
| something both objective and external to the university (in
| the same way some undergraduate institutions put more
| admissions weight on external and objective metrics like
| standardized tests compared to more easily game-able
| internal metrics like high school class grades)
| riversam wrote:
| I have a story of a CS "professor" and fraud.
|
| I am a student at University of the People. It's an online-only
| school which relies on volunteer instructors who are paid a small
| honorarium for each class. Instructor involvement varies from
| instructor-to-instructor, but most act mostly as moderators
| rather than instructors. This is due to the "peer learning"
| environment of the school.
|
| In 2015, the university either hired or hosted (or had some
| relationship with) an instructor who was fired a month later. He
| was fired because he'd lied about his credentials. He claimed to
| have a PhD from either Stanford or MIT, but had none.
|
| Prior to volunteering at the school, he'd worked on building up
| an online profile for himself. Yes, astroturfing.
|
| He wrote a few "academic" articles on Second Life. In one, he
| claims one of the founders of the game as his co-author or a
| contributor. He "published" this article to a couple of websites
| and then he posted it on Wikimedia Commons. It has even been
| cited in real academic works.
|
| He wrote a self-published book on Second Life which he submitted
| to the Library of Congress. He leverages this quite often.
|
| He claimed to hold a "world record" on an ACM ICPC challenge. And
| he managed to work this into the ACM ICPC article on Italian
| Wikipedia which has since been removed.
|
| He used Freebase and Wikidata to create "info boxes" about
| himself on search engines.
|
| He even spent time writing fake articles promoting himself on a
| websites including Blasting News and IMDb. On the latter he
| claims to have created a commercial with Julia Roberts.
|
| The best part is that while he was at University of the People he
| was also running a service to help students cheat services like
| Turnitin. This was something he prompted on LinkedIn, Reddit, and
| even Wikipedia. Yes, the traces are still there on Wikipedia.
|
| To this day, the guy holds a grudge against the university. He
| posts fake reviews on TrustPilot. He posts comments on the Reddit
| sub such as posing as a fake recruiter or going on about the
| university's subsidiary in Israel. He uses a plethora of socks
| and IPs to edit the university's Wikipedia pages and engage in
| edit wars. He has also tried to scrub some of his past actions by
| getting articles deleted.
|
| And this doesn't even get into his attacks on other institutions,
| attempt to run an Italian NGO/political party, and more. My mind
| continues to be blown as I dig into his activities.
| cryptica wrote:
| One big problem in our society is that there is a Darwinian
| selection process occurring which selects for psychopathy. It's
| only going to get worse.
|
| Psycopaths are attracted to power and are willing to do anything
| for it so they are more likely to get it. Power comes with money.
| Money allow psychopaths to have more children. The children are
| more likely to be psychopaths too so they will also end up in
| positions of power with more money. As automation increases, the
| system can afford to support an increasing number of psychopaths
| and they quickly take up all available positions of power via
| ruthless means. Psychopaths end up occupying all the important
| political and judicial positions. Altruistic people who follow
| the rules can't afford to compete in the market place, they have
| fewer children, they are discriminated against by psychopaths who
| recognize that they are different from themselves. Altruists
| become extinct as they are slowly replaced by machines which make
| altruism a redundant character trait; not necessary for a
| 'functioning' society. Now that we have machines to automate
| everything, the economy will no longer rely on 'exploitees' to do
| the hard value-adding work.
|
| Then a whole new class of ultra-psychopaths (e.g. serial killers)
| will become more common and proceed to wipe out the regular
| psycopaths through violent means. Humans will get progressively
| worse and end up driving themselves to extinction. There's not
| going to be any robot uprising; humans will make themselves
| extinct before that happens.
| selimthegrim wrote:
| You need to lay off the Peter Watts.
| fighterpilot wrote:
| Your thesis relies on the selective pressure for psychopathy
| being larger now than it was previously. Is that the case? It's
| harder nowadays to be an outright criminal or murderer and get
| away with it due to strength of the rule of law.
| Igelau wrote:
| Anecdata and not medical advice, but I used to think up a lot
| of things like this before I started taking SSRIs.
| [deleted]
| s5300 wrote:
| Well, the unfortunate part is that he's at least somewhat
| correct, and it's been found that psychopaths are much less
| likely to seek/adhere to mental health treatment such as
| SSRIs.
| [deleted]
| berndi wrote:
| Wow, the amount of depravity in some academic circles is
| astonishing.
|
| The article links [1] the case of a PhD student at the University
| of Florida who was forced to participate in such a publication
| collusion ring and was pressured to commit scientific fraud by
| fabricating results and submitting them to a conference [2],
| being threatened with physical harm should he decide to go
| public.
|
| This student saw no way out and decided to kill himself.
|
| Just a few days after the suicide, the department thought it
| would be appropriate for the student's own lab to have a "fun"
| excursion and to document it on Instagram [3].
|
| I'm lost for words.
|
| [1] https://medium.com/@tnvijayk/potential-organized-fraud-in-
| ac... [2] https://huixiangvoice.medium.com/the-hidden-story-
| behind-the... [3] https://www.instagram.com/p/Bz1EExLhdYD/
| guavaNinja wrote:
| Could someone please screenshot the Instagram post and re-
| upload it somewhere else for those of us who don't have an
| account. Thanks.
| aembleton wrote:
| https://imgur.com/a/tJfeQjg
| pvaldes wrote:
| He can't blame other that himself for commiting suicide.
|
| We should stop worshiping this kind of people. He has the
| option to join another team. One of the thousands of scientific
| teams not directed by a psycho and working on millions of
| interesting problems waiting to be solved. To leave and do
| other things was also an option. You can have a really
| fulfilling and happy live without being a scientist (Is more
| probable in fact).
|
| People with suicidal tendences had deep inner problems that
| didn't started necessarily in the university. Some are
| attention suckers, manipulative professionals that need to
| assume the protagonic role, idiots that decide to jump by a
| window to avoid facing their first real conflict in their
| lives.
|
| They dream about to punish papa, mama, the evil teacher and the
| cruel world that apparently owed them a career in science. They
| smile with the idea of everybody attending their funeral in a
| rainy day with sad faces in dread. The main motivation behind
| most (true) suicidal people is collective punishment.
|
| Some people can choose to feel miserable for the rest of their
| lives when this happens. Other will be wiser, break the endless
| stream of bullshit and refuse the role of punished. Both groups
| will eventually keep with their lives instead to feel guilty
| and miserable for an unexpected act that was beyond their acts
| or wishes.
| Cyril_HN wrote:
| In undergrad, my professor explained to me in very candid terms
| (from the position of being a generous mentor) that the optimal
| path to success in the Arts and Social Sciences is:
|
| 1. Find a niche only a couple of people operate in.
|
| 2. Make friends with them at all costs and work in their area.
|
| 3. Review each others work, amassing enormous citations in
| highly respected (albeit sometimes niche) journals.
| thomasahle wrote:
| I wonder if part of the current problem is not that the
| fields are too big. In a small field where people know each
| other, reputation becomes more important, and you can't just
| misrepresent your results, since everyone will read it and
| know you did it. Meanwhile, in a big field, it is easier to
| hide in the crowd. Write some papers that get accepted, but
| not read, and have some friends cite them.
| a9h74j wrote:
| I saw something similar in an academic's blogged advice ten
| or fifteen ago: "At graduate level, you should cultivate
| [peers and a concentration] such that your intellectual
| correspondence is publishable."
|
| I took that as plausible enough at the time to mention the
| advice once to a graduate student in philosophy. I did not
| take it at the time as immediately translatable into a
| [simply translated?] phrase I have heard since: "friendship
| corruption."
|
| There are some video lectures from a writing consultant
| employed at UChicago. He accepts that concentrations within
| fields are to some extent self-defined in the academic game
| -- in only slightly more generous terms. IIRC his strategic
| advice for those failing to publish is: learn to frame your
| abstracts as respecting but advancing the discussion. Perhaps
| in small enough niches _ingratiate yourself_ would need to be
| mentioned? Or show that you will play by being selectively
| generous with citations??
|
| Citations. The academic version of SEO and currency more
| valuable than money? (Recall where page rank came from.) One
| kind of power behind institution-sanctioned monsters?
| ItsMonkk wrote:
| > When a measure becomes a target, it ceases to be a good
| measure. - Goodhart's Law
|
| > The more any quantitative social indicator is used for
| social decision-making, the more subject it will be to
| corruption pressures and the more apt it will be to distort
| and corrupt the social processes it is intended to monitor.
| - Campbell's Law
|
| This is just another proof that you can't trust metrics.
| Build up a network of people that you can hold personally
| responsible for, and if any of them ever recommends low
| quality work then let them go. Let that network know
| whenever you discover a network of individuals who put out
| low effort work so that you and your network can disregard
| the entire network.
|
| Bottom up, not top down. Do not rely on citation count. Do
| not rely on views. Do not rely on upvotes or reviews from
| people you do not know. Do not rely on if they went to a
| prestigious university. Do not rely on the ability to pass
| a standardized test.
| galimaufry wrote:
| > "At graduate level, you should cultivate [peers and a
| concentration] such that your intellectual correspondence
| is publishable." ... I did not take it at the time as
| immediately translatable into a [simply translated?] phrase
| I have heard since: "friendship corruption."
|
| This seems misguided, and I certainly hope that this
| 'friendship corruption' concept never catches on. There are
| great papers that started as letters and were later
| completed by the sender, recipient or both. No one should
| feel ashamed about that, and no one should feel ashamed of
| developing friendships with their colleagues.
| a9h74j wrote:
| Well said. My comment was phrased in response to its
| parent comment looking askance at "amassing enormous
| citations." There _is_ love of truth and not all is
| corruption.
| matthewdgreen wrote:
| The problem here is that there is no bright-line
| distinction between "friendship corruption" and "doing
| high-quality collaborative research in a small field where
| colleagues are cordial."
|
| I'm not saying that one can't bleed into the other --
| sometimes they do. I'm saying that any approach you take
| that disincentivizes the bad behavior is also likely to
| harm good scientific collaborations as well. It's one of
| the downsides of peer review.
| splithalf wrote:
| This model has been applied to every field; all those "family
| run businesses" like the mafia, the American news industry,
| Harvard.
| btilly wrote:
| A similar dynamic in mathematics is why I chose to leave
| academia a quarter century ago.
| berndi wrote:
| How is it possible that professors with impeccable academic
| credentials get fired for jokes nearly instantly [1], yet this
| student's professor was allowed to carry on until he resigned
| two years later [2]?
|
| [1]
| https://en.wikipedia.org/wiki/Online_shaming#Tim_Hunt_contro...
| [2]
| https://eu.gainesville.com/story/news/education/campus/2021/...
| TrackerFF wrote:
| Much easier to prove the case of terrible jokes or non-PC
| behavior. Twitter has streamlined that stuff for years now -
| and what's more, that kind of behavior is much more relevant
| for more people, rather than academic fraud which might be a
| very niche thing, only relevant to small circles of people.
|
| One could ask - why is it that petty theft can land some
| people behind bars for years, while wealthy people committing
| tax fraud only get fined, or at most a couple of months in a
| cushy white-collar crime facility? Well - for one, the former
| crime is much easier to prove, especially if you're caught
| red-handed. The latter crime tends to be incredibly complex,
| and will cost a ton of resources to prosecute.
|
| I guess the same goes for your questions. Professors
| incriminating themselves on twitter or youtube - easy as pie.
|
| Collusion rings with respectable professors, that probably
| use students for dirty work and plausible deniability, and
| resources enough to fight their employer in the courts: hard
| fight
| gjulianm wrote:
| For one, you're cherry-picking two examples. Some professors
| don't get fired for jokes nearly instantly and others get
| immediately fired for abusing their students.
|
| Second, amplification plays a part. A story with "two sides"
| (such as the joke one) will draw far more public attention
| than another with only one side, just because there will be
| more debate and people using it to advance their agendas. In
| this case, it includes "PC Twitter" but also all of the
| people who scream "freedom of speech" whenever someone is
| criticized for saying something dumb or inappropriate.
| underseacables wrote:
| The tenure system. It protects more bad professors than it
| retains good ones.
| rscho wrote:
| No tenure, no exploratory science and no moonshot. Newton
| was a dangerous asshole. Would you exchange differential
| calculus for squeaky-clean academia? I don't think so ...
| sdenton4 wrote:
| Well, it's not a real choice... We also had Leibniz.
| b3morales wrote:
| There's an old quote to the effect that, if forced to
| choose one thing to destroy: either the _Principia
| Mathematica_ or the Sistine Chapel*, choose the
| _Principia_. The reason? The _Principia_ is a monumental
| achievement, but it 's also universal, and it can be
| rediscovered. The paintings are the unique creation of
| one moment in time, unduplicatable.
|
| *Or some other work of art; I can't remember the original
| exactly.
| iamadog3 wrote:
| There's a lot of questions that we need to address, cause
| and effect in the case of tenure and the general
| priorities of academia.
|
| First you implicitly assert that tenure effects the cause
| of emergent moonshots. How much evidence do we have to
| the contrary? Cursory research shows the modern
| application in the US dating to the vague "19th century"
| not a long timeframe. Tenure itself appears to have
| emerged in the same timeframe. The modern US application
| of tenure (secondary) was put in place in 1940.
|
| I'll grant you that we have seen a good deal of progress,
| but I don't know that you could make a robust argument
| that without tenure, that progress would be absent. I
| would assert that it falls into inconclusivity, and that
| to form an argument would require speculation and
| conjecture. All things are not made equal, and so finding
| a suitable control group to compare against would be
| impossible.
|
| We can look at history, though, and see that there was a
| plenitude of highly driven scientists publishing and
| advancing understanding prior to the advent of tenure.
| But to say that we can transpose that to the contemporary
| model itself is a conjecture.
|
| Simply, we do not know, and can not know.
|
| As to the priorities of academia, and tenured
| individuals, and the metrics that institutions use to
| enlighten themselves on the performance of individuals we
| seem to have come upon a system of perverse incentives.
| That is exactly everything, to me, it seems we had ought
| to avoid. Tenured academics can obviously be terminated,
| but not in frivolous contexts. They are expected to hold
| some degree of real responsibility. What tenure grants is
| their freedom of opinion, and the right to fail in their
| pursuit. As we know, science is the art of failing
| upwards in a controlled direction.
|
| "In all lines of academic investigation it is of the
| utmost importance that the investigator should be
| absolutely free to follow the indications of truth
| wherever they may lead. Whatever may be the limitations
| which trammel inquiry elsewhere we believe the great
| state University of Wisconsin should ever encourage that
| continual and fearless sifting and winnowing by which
| alone the truth can be found." --Theodore Herfurth, 1894
|
| But the cutthroat competition, the model of "publish or
| perish" elimination, the perverted demand for
| conclusivity all stand to imperil the actual aims of
| science. This is all somewhat incentivized by the tenure
| system, by personal interest, and by the implicit
| obligation to "realized" progress - except this is of
| course not real progress. It isn't as concrete as the
| fundamentals, the traces are laid much finer these days
| and replication of research appears more often
| infrequent, while the quality of publications is
| increasingly called into question and an economy of
| debauchery contaminates data for capital and personal
| gain. And thus the bastion of humanity is corroded while
| evermore maintaining its authority outward. A real hazard
| if you ask me.
|
| I believe it _all_ needs reform and serious reflection to
| build it back better.
| skystarman wrote:
| Comparing these personnel decisions from not only different
| universities and entirely different incidents but also an
| entirely different country seems unwise to me.
| hellbannedguy wrote:
| Tenure
|
| I had a professor in a chemistry lab that was proud of the
| amount of complaints he received against him.
|
| He literally threw the two ring binder it at me while I was
| in his office. It must have been a hundred pages. He kept it
| on the wall like a trophy.
|
| I talked to a Counselor at the school, and before I could
| complete my sentance, he said Dr. Berzergian? (I don't
| remember the exact spelling of his name.). The Conselor said
| he, and the Dr., almost got into fisticuffs over his
| attitude. He told me to take the course at another college.
|
| I realized later all his "problem" students were young males.
|
| Yes--I truly believe this was his twisted way of hitting on
| people.
|
| A few years later, I was in a bar in San Francisco talking
| about this professor whom really gave me a bad time. By
| chance, he knew of the guy, and told me about him.
|
| This was in the nineties, but oh boy if he acted this way
| today, and I stole that stack of complaints---well who knows?
|
| This professor caused students to change majors, and even
| drop out.
| zaphod12 wrote:
| I'm sorry to be this guy, but it's "tenure"
| whatshisface wrote:
| There's nothing in tenure that requires universities to
| force bad teachers to continue teaching.
| a1369209993 wrote:
| For that matter, there's nothing in tenure that requires
| universities to _allow_ bad teachers to continue
| teaching. Tell him to go research something, and if
| anyone thinks his attitude is worth dealing with for his
| expertice, they can approach him volutarily.
| rscho wrote:
| Getting tenure means (ideally) that the system considers
| you have proven yourself as a competent researcher and
| therefore accepts to release a bit of pressure on
| publishing to allow you to pursue more exploratory
| objectives. It does not mean you're allowed to behave
| badly and (officially) does not protect oneself from the
| consequences of such behaviour. Remove tenure and you
| remove the last bastion of real research we have left in
| our industrialized and quasi-corporate western research
| institutions.
|
| Removing tenure would completely trash western science
| and in practice yield total scientific leadership to
| eastern powers, who still have old style academic systems
| with strong tenure positions and less concern for
| academic mistreatment and "wokeness".
| raverbashing wrote:
| And this is what students pay thousands of dollars per year
| for?
|
| Yeah, there are bad teachers and there's this.
|
| Why this kinds of abuse is tolerated is anyone's guess. But
| school and college management are usually too coward to
| deal with those issues.
| jjoonathan wrote:
| Supply and demand. Everyone wants to do cutting edge
| research but nobody wants to pay for it. People who know
| how to get it paid for hold all the cards and can
| therefore get away with anything.
| wpietri wrote:
| I would encourage you to think more about the social use of
| "jokes". Once we get past the level of knock-knocks, jokes
| very often have social meaning. Look at people like George
| Carlin, for example. His "7 dirty words" routine was deeply
| political. It was a full-on assault on American government
| censorship and the cultural elements that demanded it. And
| looking at history, he's won. Humor can be very powerful.
|
| Jokes can also be used the other way, for social control.
| Growing up, I heard a lot of racist and sexist jokes, the
| practical effect of which was to demean: to create a place
| and put disfavored people in it. I'm old enough that nerds
| were a similar group, and I remember being the butt of a lot
| of jokes. When that happens, you're just supposed to take it;
| any objection to being demeaned is met with, "Why so
| sensitive! It's just a joke!"
|
| So in the case you cite, the problem wasn't him telling a
| joke. There are whole books full of jokes for speakers he
| could have used. It was him invoking rank sexist stereotypes
| and suggesting the solution to his inability to manage his
| feelings was to push women out of the labs that they've been
| working for decades to get equal access to. And indeed, are
| still working on. At my alma mater, just this week a CS
| professor was just pushed out after dozens of women
| complained about sexual harassment in recent years. [1] It
| took dozens because early complaints were dismissed. And
| there are far more stories of professors like that then there
| are of ones being booted for "joking" misogyny.
|
| [1] https://www.michigandaily.com/news/walter-lasecki-
| resigns-ef...
| meowkit wrote:
| > for social control
|
| I think you're reading way too deeply into this. You're
| referring to the Tim Hunt quote linked?
|
| Everything is about control or manipulation - its implicit
| to the human condition.
| throwaway894345 wrote:
| I agree that jokes can be subversive; I don't agree
| (perhaps you don't either) that subversive speech (jokes or
| otherwise) merit termination, which is to say I'm not an
| authoritarian. With respect to racism, I don't doubt that
| there are too many racist jokes, but the Twitter sphere
| tends to miss those in favor of jokes which are decidedly
| "antiracist" i.e., jokes which make fun of _racism_
| (whether left-wing racism or right-wing racism).
| zpeti wrote:
| Unpopular Opinion: people making these sexist jokes do
| actually think they are jokes. They aren't using them to
| push people down. Most people are nice people and don't
| realise they are being assholes and pushing people down.
|
| On the other hand it's completely fair that people feel
| pushed down by them.
|
| HOWEVER - this entire social justice movement is being used
| to outsource getting into conflict, and standing up for
| yourself. My guess is 80-90% of the time if you told
| someone who made a sexist joke that you are hurt by it,
| they would apologise (sincerely), and probably not do it
| again. But for that people actually need to get into a
| conflict situation, which is hard.
|
| But it would make life a lot easier if we just sorted out
| these issues at the source, with two people, explaining
| what hurts and why to someone.
|
| This modern solution of going to HR or to Twitter is not
| constructive to society, it creates massive divides, it
| also creates cowardly behaviour rather than encouraging
| actual people to talk to each other.
| pessimizer wrote:
| But this becomes a full-time job for minorities to
| explain what is bigoted to people they don't even want to
| be talking to, which
|
| 1) gets you attacked for seeing everything as bigoted -
| especially when you make mistakes because you can't know
| why everyone is doing everything, just see statistically
| stuff is happening to you and people who look like you
| more than everyone else, and
|
| 2) alienates you from your co-workers, who would prefer
| that you act according to the stereotypes they have of
| people like you and laugh at the jokes they're making
| about you (and your parents, and your parents parents,
| who were indisputably shat on.) They don't want to hang
| out with you because they can't relax around you. You're
| not going to get promoted unless the word comes from so
| far up you're going to get resented for it, and
|
| 0) it's just another burden to constantly be explaining
| how and why you're miserable to people, even (especially)
| the ones who consider it self-improvement to listen to
| you.
|
| The temptation is just to coon for people, say what they
| want you to say and do what they want you to do, and just
| silently hate them and hate yourself.
|
| > But it would make life a lot easier if we just sorted
| out these issues at the source, with two people,
| explaining what hurts and why to someone.
|
| This is problematic thinking. For example, black people
| are 15% of the US population. It isn't one-on-one, it's
| one-on-five-and-a-half at best. And really, if you're a
| middle class professional (let's say programmer) where
| there's a lower proportion of black people that would be
| indicated by relative populations, it's one-on-a-small-
| army-20%-of-them-heavily-redpilled-and-angry.
|
| I prefer to leave it to the twitter mob, although some of
| their positions are crazy, and it being twitter the
| people who are going to be the most vocal are going to
| have severe personality disorders (usually borderline.)
| It's still nice sometimes to have them deflect the
| belligerent white dude from you.
| skywhopper wrote:
| Number one was a highly offensive "joke" minimizing a truly
| serious problem that was made to a number of journalists to
| whom it definitely wasn't funny. Plenty of people are fired
| regularly for making complete fools out of themselves and the
| institutions they represent by offending important outsiders.
| TazeTSchnitzel wrote:
| Not all targets and not all campaigns are created equal? If
| someone has a lot of power within the institution, and the
| campaign comes mainly from people the institution doesn't
| care about... you can fill in the gaps.
|
| I seriously doubt that exactly _what_ someone has done is the
| major factor. People in weak positions can easily lose them
| over the tiniest of things; people in strong positions can
| get away with murder!
| rscho wrote:
| Well... non-PC jokes are a liability risk for the institution
| at large. Dead students are not. We have an identical
| phenomenon in hospitals: there are huge campaigns against
| sexism and the like, while there are multiple young
| professionals committing suicide from burn-out every year and
| noone does as much as bat an eyelash.
|
| Contrary to victims of discrimination, dead people can't
| easily organize into coordinated legal action.
| specialist wrote:
| Said another way: Attention economy. There's no rhyme or
| reason for why any one particular outrage bubbles to the
| top and becomes today's cause celebre.
| berndi wrote:
| You would think that allegedly driving a PoC student to
| suicide would count for something in the PC twitter
| sphere...
| throwaway894345 wrote:
| I get the distinct impression that the Twitter sphere
| isn't genuinely concerned about people of color, or else
| they would express concern over, say, inner city
| violence. Instead they work hard to brand any such
| concern as "far-right".
| andromeduck wrote:
| It's mostly just wingnuts & entitled folks trying to make
| a name for themselves.
| throwaway894345 wrote:
| Probably, but IMO they have too much power and their
| behavior is destructive.
| the_lonely_road wrote:
| Are Asians PoC? I thought they were not and that was
| literally the only reason the oft repeated phrase "black
| and other PoC" didn't literally translate into "not
| white".
| root_axis wrote:
| I have never heard the suggestion that Asian people
| aren't considered "PoC".
| throwaway894345 wrote:
| Some woke people will argue that they are "white
| adjacent" when convenient (e.g., the Harvard admissions
| scandal) but also that they are people of color when
| convenient (e.g., the Atlanta spa killings).
| [deleted]
| faeriechangling wrote:
| They are racial chimeras. Changing between majority and
| minority based on the speakers preferences.
| andromeduck wrote:
| According to Twitter we're "multiracial whites".
|
| BIPoC is also often used as a dogwhistle to exclude
| Asians and Latinos when convenient.
| anchpop wrote:
| Are Latinos not indigenous to the Americas? (genuinely
| asking)
| csande17 wrote:
| My understanding is that the term "Latino" includes both
| people indigenous to Latin America and people whose
| ancestors were colonists from Spain. The latter group is
| generally the larger/more well-known one -- that's why
| "Latino", a Spanish word, is used to describe them.
| bronzeage wrote:
| They can't be PoC because their higher than average
| success contradicts the narrative of victims.
| hackflip wrote:
| Only when it is politically convenient
| wyager wrote:
| They've recently started using the term "BIPOC" to
| clarify that Asians are excluded, and when they want to
| include Asians they'll say "BIPOC and AAPI".
| tomjakubowski wrote:
| No, by any good faith source that I have read, BIPOC does
| not at all exclude Asians. "The acronym BIPOC refers to
| black, indigenous, and other people of color and aims to
| emphasize the historic oppression of black and indigenous
| people."
|
| Incidentally, one of the co-founders of the "BIPOC
| Project" is an Asian-American woman.
|
| * https://www.thebipocproject.org/
|
| * https://dbpedia.org/page/Person_of_color
|
| * https://www.verywellmind.com/what-is-bipoc-5025158
| vimy wrote:
| European activists have started using BIPOC as well. They
| don't seem to realize that indigenous means white in
| Europe.
| andromeduck wrote:
| Isn't BAME the euro equivalent?
| vimy wrote:
| That's the UK term. Other European countries are more
| focussed on the US so they use all the US terms.
| wyager wrote:
| From the BIPOC project website: "We use the term BIPOC to
| highlight the unique relationship to whiteness that
| Indigenous and Black (African Americans) people have"
|
| If it included Asians they would just keep using POC.
| berndi wrote:
| Wikipedia says they are. However, the Supreme Twitter
| Council of Wokeness may have ruled otherwise at some
| point.
| PicassoCTs wrote:
| So many different terms and abbreviations, for what is
| just humans..
| forgingahead wrote:
| The "PC twitter sphere" as you describe them are only
| interested in one thing: doing what is easy and public
| for burnishing their own stature, and nothing else. No
| real problems will be solved by them, because they prefer
| having the public think they are solving them, rather
| than putting in the actual work and disciplined thinking
| to do it.
| throwaway45209 wrote:
| At a lunch for female journalists and scientists, Hunt gave a
| speech...
|
| "It's strange that such a chauvinist monster like me has been
| asked to speak to women scientists. Let me tell you about my
| trouble with girls. Three things happen when they are in the
| lab: you fall in love with them, they fall in love with you,
| and when you criticise them they cry. Perhaps we should make
| separate labs for boys and girls? Now, seriously, I'm
| impressed by the economic development of Korea. And women
| scientists played, without a doubt, an important role in it.
| Science needs women, and you should do science, despite all
| the obstacles, and despite monsters like me."
|
| That is not a joke.
| AussieWog93 wrote:
| That quote is a textbook example of self-deprecating
| British humour.
| throwaway45209 wrote:
| It's actually self-congratulating, while deprecating
| others.
| [deleted]
| hnuser123456 wrote:
| I can't see it that way. He is open about his own
| weaknesses as well. We don't have to look up to him for
| saying this, but that doesn't make it your typical one-
| sided bashing.
| dwighttk wrote:
| "Now, seriously..." it isn't a very _good_ joke, but what
| makes you think that isn 't a joke?
| [deleted]
| newswasboring wrote:
| People keep bringing out that this can be a joke, but he
| said this two days later
|
| "I did mean the part about having trouble with girls. It is
| true that I have fallen in love with people in the lab, and
| that people in the lab have fallen in love with me, and
| it's very disruptive to the science. It's terribly
| important that, in the lab, people are on a level playing
| field. And I found these emotional entanglements made life
| very difficult. I mean, I'm really, really sorry that I
| caused any offence - that's awful. I certainly didn't mean
| - I just meant to be honest, actually." [1]
|
| He was given a chance to clarify, he doubled down. While
| I'm not saying a joke should cause people to be fired, but
| this is clearly more than a joke.
|
| [1] https://www.bbc.com/news/uk-33077107
| Al-Khwarizmi wrote:
| So what part of falling in love with people in the lab,
| people in the lab falling in love with you, and that
| affecting concentration and productivity, is so evil as
| to get someone fired?
|
| Sorry, honest question. Being from a different culture, I
| honestly fail to get this kind of outrage. For me it's
| just a description of humans being human...
| timoth3y wrote:
| > So what part of falling in love with people in the lab,
| people in the lab falling in love with you, and that
| affecting concentration and productivity, is so evil as
| to get someone fired?
|
| Fundamentally, he is saying that since he has trouble
| keeping his emotions in check around women, the solution
| is to not allow women in the lab rather than developing
| his own managerial or social skills.
|
| It's understandable why some would question the wisdom of
| having this person responsible for developing the skills
| of female scientists.
|
| Should he have been fired? I don't know. Certainly not if
| tis joke was is only "offence", but I suspect there is a
| bit more history to the situation.
| b3morales wrote:
| It's painfully reductive and one-dimensional. What about
| envy and hate, hero worship, and other emotional
| attachments? Those have no effect on working environment?
|
| What about men and women who aren't romantically
| attracted to (respectively) women and men? Are gay men
| relegated to the women's lab? But only one per batch,
| lest they fall in love with each other? (And bisexual
| people can only be trusted to do science on their own.)
|
| In general don't we expect "professionalism" to include a
| level of managing your emotions? And this person is
| basically saying "I can't deal, therefore certain other
| people must be kept away so I don't get distracted". As
| well as tarring women in general as not being able to
| deal, which is unfair. I sure wouldn't want to work with
| this guy after hearing him say that.
| missedthecue wrote:
| Feelings of love are notoriously hard to "manage". This
| is the plot line of most Rom-Coms for instance.
| b3morales wrote:
| Romantic comedies are fiction, though, intentionally
| exaggerated for entertainment. I don't deny there's a
| kernel of truth to that kind of story -- or else they
| wouldn't be interesting at all -- but I hope we're not
| taking them as a model of workplace behavior.
| skystarman wrote:
| FUnny you leave out the part where he also said if you
| criticize a woman in the lab they cry...
|
| He made a terribly misogynistic "joke" and paid the
| consequences for it.
|
| You'd think someone with a Nobel prize wouldn't be so
| clueless
| prepend wrote:
| > FUnny you leave out the part where he also said if you
| criticize a woman in the lab they cry...
|
| Do you think women in the lab, when criticized, are more
| likely, less likely, or equally likely to cry?
|
| Is the scientist's comment mean spirited, or sexist, or
| just an observation?
|
| I think it's important to consider what the intent behind
| these jokes are. The Wikipedia article calls out
| statements from 29 other scientists that note how women
| (and men) were advanced within his lab and outside his
| lab.
|
| So if this person thinks that the women he's worked with
| cry when criticized, so we not want him to say that? It
| seems more like the goal should be to not stigmatize
| crying as that seems pretty reasonable for all genders,
| rather than to stigmatize talking about crying.
| whatshisface wrote:
| The rule is really simple: it makes people uncomfortable
| when you make generalizations about natural traits shared
| by the group they're in. Period. All groups (even groups
| people feel proud to be a part of), and all
| generalizations, even ones that sound positive or don't
| apply to the listener. I'm not going to list examples but
| if you're having a hard time thinking of them just
| imagine overhearing a conversation at a coffee shop about
| "those <something you are>, they're always <something you
| do or don't do>."
| zozbot234 wrote:
| > it makes people uncomfortable when you make
| generalizations about natural traits shared by the group
| they're in.
|
| It does not make _everyone_ uncomfortable, obviously. I
| thought we were not supposed to resort to stereotypes.
| [deleted]
| skystarman wrote:
| "WHen making terribly misogynistic comments his INTENT
| wasn't to be misogynistic, it's just based on his
| experience that women are driven primarily by their
| emotions and unable to handle pressure!"
|
| Really solid defense!
| prepend wrote:
| I'm not defending him and it's funny you think this is
| some pro/con situation.
|
| That being said misogyny requires intent, right. It means
| someone who hates or dislikes women. So if you make a
| statement that every time you criticize a woman, she
| cries and don't have ill intent toward women, then that
| isn't a sign of misogyny.
|
| I don't think women are any more likely to cry than men,
| but if there's research that shows it so, is that
| misogynistic?
|
| If you say "women are shorter than men" is that
| misogynistic?
|
| I think it largely depends on intent as if someone is
| trying to demean women or does hate women, that's a big
| difference. Saying women are shorter than men as part of
| some overall argument on inferiority is clearly
| misogynistic.
| newswasboring wrote:
| > and that affecting concentration and productivity, is
| so evil as to get someone fired?
|
| Hard to not assume malice when you accuse someone for
| implying something they explicitly said they don't want
| to imply.
| toss1 wrote:
| "Human nature, is what we are put on earth to overcome."
|
| --Katherine Hepburn, in The African Queen
|
| A few of the good words to live by
| iamadog3 wrote:
| Not really, no. Human nature is what we're dealt, we must
| embrace both the positive and negative aspects of it.
| Perhaps the inevitability of two people gravitating
| towards one another can be leveraged? Perhaps the
| disparities can elucidate us on unseen proclivities in
| different populations, things that can also be leveraged
| and positively.
|
| What we should avoid is cramming people into functionary
| roles and instruct them they must act as would a machine.
| No longer can they be compelling or compelled but only
| impelled as would be a gear turning in the insurmountable
| forces of the engine that drives.
| pasabagi wrote:
| I guess for me the thing that's really crappy about this
| quote is that it shows his underlying attitude - that
| women are basically always potential romantic partners.
|
| If he was into men, and he said he didn't want men in the
| lab because he might fall in love with them, you can sort
| of see how absurd it is, and how unpleasant it is to be
| the object of romantic fantasy when you're just trying to
| get on with your job.
| rscho wrote:
| > women are basically always potential romantic partners
|
| Ok, why would that not be the case? Laws? PC? Age
| difference? Love/biology doesn't care about social rules,
| and this has been shown time and again in every possible
| situation you could think of.
|
| > you can sort of see how absurd it is
|
| Huh, no I can't. What makes it different when you reverse
| the situation?
| pasabagi wrote:
| > Huh, no I can't.
|
| The point is, any human can be a romantic partner to any
| other. Therefore, his argument should be that no pair of
| humans should work together ever for risk of romantic
| entanglements.
|
| Except it doesn't work that way, because we're all really
| used to the idea that in the workplace, you treat your
| colleagues as colleagues, not as fantasy-future-partners.
|
| This isn't PC. It's just basic common sense, that he's
| lost his grip on, because he sees women _first_ as
| romantic partners or sex objects or whatever, and
| _second_ as scientists.
| bumby wrote:
| > _Ok, why would that not be the case?_
|
| Because of professionalism. The root of term used for
| "professions" like law, medicine, engineering etc. is
| that one professes to a code of ethics. That code should
| overrule base desires.
|
| We generally wouldn't accept a doctor who views and
| treats patients primarily as an income stream despite
| greed being a near-universal human drive and we shouldn't
| expect a professor to view subordinates as potential
| romantic partners. Acknowledging the drive exists isn't a
| reason to condone it.
| rscho wrote:
| > Acknowledging the drive exists isn't a reason to
| condone it.
|
| So, exactly what Hunt said in his speech.
| bumby wrote:
| Maybe you can help me understand the context better. From
| the GP post where he seems to advocate for separate male
| and female labs he seems to imply there isn't enough
| professionalism present to have co-ed labs.
|
| I'm saying that claim is more an implication of the
| person saying it and their (lack of) professional ethics
| than an indictment of the subordinates. It's very similar
| in my mind to the recent arguments about gender in
| military units
| rscho wrote:
| The context is that he in essence says that this problem
| has no good solution, but he thinks that the co-ed labs
| are the best alternative even with all the shortcomings
| that go with them. Everyone will be perfectly
| professional until someone falls in love and the PC
| solution crumbles to dust. And FWIW, I think he's right.
| bumby wrote:
| > _Everyone will be perfectly professional until someone
| falls in love_
|
| Isn't this the case with everything? I.e., if "everything
| is fine until it isn't" it's not really saying much of
| anything except he doesn't think he can create a culture
| of professionalism within his lab. Does this "welp, we
| can't do anything about our base desires" extend outside
| romantic relations? Would it be acceptable to claim
| "well, physical altercations are just going to happen
| because you know people will get mad at each other from
| time to time"?
|
| I'm not hiding behind professionalism, I'm saying it's
| reasonable to acknowledge those base desires while also
| expecting a higher standard of behavior.
| rscho wrote:
| > Would it be acceptable to claim "well, physical
| altercations are just going to happen because you know
| people will get mad at each other from time to time"?
|
| Are you willing to punish people with jail time or worse
| for falling in love and adopting the behaviour that goes
| with it? This is the other extreme of your argument, and
| there are many places in the world where this is the
| social norm.
|
| The PI can do everything he/she wants, love will happen
| and people will behave accordingly. The point is
| acknowledging that this is not a problem that arises at a
| single point in time allowing you to fire the offender,
| but that it happens along a continuum that will
| constantly decrease lab efficiency.
| bumby wrote:
| > _Are you willing to punish people with jail time or
| worse for falling in love and adopting the behaviour that
| goes with it?_
|
| No, because one is a criminal offense and the other is a
| breach of ethics. From that standpoint, it was a bad
| analogy. But I would hold someone accountable for being
| unprofessional in the workplace. To be clear, I'm not
| saying to punish people for falling in love, I'm saying
| you can hold them accountable for letting it affect the
| workplace and creating an unprofessional environment.
|
| > _The PI can do everything he /she wants, love will
| happen and people will behave accordingly._
|
| This is probably where we disagree. I think the PI holds
| some responsibility for setting the tone of the work
| culture. You may not be able to control people's feelings
| but you can make it clear that certain actions are not
| going to be tolerated. That's especially necessary in
| cases of fraternization. It's the PI's job to maintain
| the professional standards of the lab.
| QuinnWilton wrote:
| Where do non-straight people fit into this "solution"?
| rscho wrote:
| They don't. I fail to see how that's surprising given
| that straight people don't fit either.
| QuinnWilton wrote:
| Plenty of us have absolutely no problem working in same-
| sex environments, and plenty of straight people have no
| issues working in coed environments.
|
| This is very clearly a case of the professor being
| unprofessional and exploitative of his position of power.
| rscho wrote:
| Plenty and plenty, yes. Now, what's a solution that would
| work for everyone, males females and others alike?
|
| This is very clearly a professor acknowledging the
| problems that arise due to interindividual biology in
| work environments. Unlike PC supporters hiding the issue
| under the blanket of professionalism.
| zozbot234 wrote:
| The controversial part is where he suggests that this is
| a "problem" and might be a good reason to _exclude_ women
| from working in the lab alongside with males. That looks
| like he 's starting out with a very sexist attitude and
| trying to justify it with flimsy excuses.
| anewhnaccount2 wrote:
| That part was the joke. He's saying it's a real problem,
| but that's clearly the wrong solution. For him, it's
| obviously a bad solution, and so worthy of ridicule.
| wrs wrote:
| By framing that as "exclusion" are you assuming the
| women-only labs would be worse places to work than the
| men-only labs?
| b3morales wrote:
| As a policy it denies both women _and_ men the
| opportunity of working together. If you are a man and a
| woman happens to be working on the same problem that you
| are investigating, would you like to be excluded from
| learning from her? And vice versa.
| wrs wrote:
| Exactly. So his proposal is not a practical solution to
| the problem of human behavior affecting productivity, but
| it's not "sexist". Many single-sex schools exist and have
| strong proponents, but are rarely described as
| "excluding" people.
| h2odragon wrote:
| > a description of humans being human...
|
| Precisely. There's some sort of "if you have authority
| you must be better than me" feeling and "better than me"
| admits no flaws or human variety _at all_ , apparently.
| Some folks want perfect Gods to follow and keep failing
| to make them from people made of meat.
| frickinLasers wrote:
| It is difficult to distinguish genuine romantic feelings
| between two people, from the case of a superior using
| their position to get their genitals wet and a
| subordinate capitulating for fear of losing their job.
|
| Since feelings are only a biological impulse, and we
| humans frequently suppress our impulses in the form of
| self control, it's much easier to look for that oxytocin
| fix in a more appropriate arena.
|
| The military has been doing this for ages, forbidding
| officers from fraternizing with enlisted. And plenty of
| civilians abusing positions of power have proven the
| wisdom of such a policy.
| Veen wrote:
| It's difficult to distinguish genuine romantic feelings
| from exploitative lust everywhere. It's a constant of
| human experience and has very little to do with power
| dynamics.
| whatshisface wrote:
| > _It 's difficult to distinguish genuine romantic
| feelings from exploitative lust everywhere._
|
| Huh?
| frickinLasers wrote:
| They're saying people are assholes. For instance, in some
| American cultures (see my previous comment for which
| culture), it's a given that perhaps 40-60% of married
| individuals are cheating on their spouse. That's not love
| --that's doing what feels good, and then doing someone
| else that feels good.
|
| Where there's a power imbalance, it's easier to ban a
| class of abuses than to figure out the small percentage
| of cases where both parties are genuinely afflicted by
| mutual biological imperatives.
| whatshisface wrote:
| That's pretty easy to distinguish, I don't see any
| trouble at all in forming those categories.
| frickinLasers wrote:
| Okay, I got a little off track. Let's say you're an HR
| person (or whoever is at legal risk if an employee
| decides they've been taken advantage of), and someone
| come in with just such a complaint.
|
| How would you, an outside party, determine whether the
| superior was really [infatuated, in love, whatever], and
| not simply taking advantage of their situation? Or how
| would a judge determine that? Is it worth it to the
| company to work through that process every time it
| happens? What about the people who really were
| victimized, but the evidence is circumstantial and the
| court says otherwise? Isn't it easier to exclude the
| small pool of people that are subordinates and tell the
| supervisor to find romance anywhere else?
| zepto wrote:
| 'Doubled down' on what?
|
| I'm curious if you think any of his statements are
| untrue?
| afarrell wrote:
| Honestly, the people who jump to conclusions about him
| saying this need to read some Berne Brown.
| teddyh wrote:
| I've noticed a pattern where, wherever someone feels the
| need to write (after a disparaging story about someone
| else) " _They were not joking_ " or " _They really said
| this_ ", it's almost always false, and the disparaged
| person really _was_ joking, or really did _not_ say that,
| or it was taken wildly out of context, etc.
| alisonkisk wrote:
| What's the missing context here? That he used an
| important public speaking opportunity to actually mock
| the women he was assigned to help, and we misunderstood
| it as a serious appeal? That he was being fake offensive
| because it would be funny or enlightening? This wasn't an
| opportunistic pun. Humor is rooted in one's worldview.
| hnuser123456 wrote:
| He also implies that if you're too good of a student, at
| least around him, he might fall in love with you and
| become a worse teacher, and he's just being honest about
| it. You could mock men for being bad at suppressing
| feelings equally from the same comment.
| teddyh wrote:
| I don't know anything about this specific case. It might
| be the exception which proves the rule.
|
| In general, I try to follow the HN guideline: " _Please
| respond to the strongest plausible interpretation of what
| someone says, not a weaker one that 's easier to
| criticize. Assume good faith._"
| NovemberWhiskey wrote:
| I don't know any of the context here, but it seems to me
| that what the quote actually says is that the author
| admits that he is unable to function effectively with
| female colleagues, that there are other men like him, and
| that it's a problem that mustn't be allowed to hold women
| back from doing science.
| skystarman wrote:
| It's interesting that everyone defending this old coot is
| just memory-holing the clearly misogynistic comment "if
| you a criticize a woman in the lab, they cry" and are
| choosing to only mention the comments that are more
| easily defensible as a "joke"
| NovemberWhiskey wrote:
| Please give some credit to other posters.
|
| The way I read this; when he says "you", it's the generic
| you: he's talking about the "chauvinist monsters" like
| himself, and it's in the context of three particular ways
| in which his own failings prevent him working with women.
|
| He forms and encourages inappropriate emotional
| relationships with his female colleagues and it affects
| his ability to give criticism of their work effectively.
|
| Now you have the right to read that a different way; but
| please respect that others are not just "memory-holing"
| misogynistic comments for some reason or another.
|
| I'm not defending him: he definitely has a problem with
| his attitude to women, it sounds like it might absolutely
| create a hostile workplace, and it's almost certainly
| inappropriate to be talking about the subject in such a
| light-hearted manner - but this is fundamentally a _mea
| culpa_ rather than a criticism of women in science.
| skystarman wrote:
| "Now you have the right to read that a different way; but
| please respect that others are not just "memory-holing"
| misogynistic comments for some reason or another."
|
| Almost every single one of the posters defending this
| misogynistic behavior are just deliberately leaving out
| his most incendiary and misogynistic remark and instead
| focusing on comments he made that are less objectionable.
|
| Why do these people deserve "credit"? Am I suppose to
| believe only focusing on the least objectionable comments
| to paint him as some unfairly maligned martyr is just an
| accident?
| unishark wrote:
| Yes I imagined his idea of "criticizing" amounted to
| abusively yelling at people. From what I've seen, the way
| people win Nobel prizes is by working their lab 24/7 like
| a slavedriver. I actually have seen psychopathic
| professors yell at female lab members until they cry. I
| knew one guy who complained about it. He apparently got
| much more productivity out of yelling at the guys. Pretty
| dark humor if it's a joke.
| zozbot234 wrote:
| I mean, who could prove him wrong? Maybe his idea of
| "criticizing" his fellow professionals involves a lot of
| yelling and throwing chairs around. There's certainly
| _some_ people like that in the workplace.
| Veen wrote:
| > What's the missing context here?
|
| The missing context is that many women he worked with
| defended him. That he has consistently worked with,
| hired, supported, and promoted women throughout his
| career. That he has done far more to benefit women in
| science and humanity generally than the whining
| Twitterati who denounced him ever will.
| emn13 wrote:
| It sometimes surprises me how people don't seem to see
| that while all of this is part of a healthy (and
| seemingly normal) societal change, that it's unfortunate
| that not just are oldfashioned behaviors shunned, but
| that merely talking about struggles with them is so
| taboo. Is society really going to adapt better because
| people lash out so uncompromisingly?
| guywhocodes wrote:
| It's a self deprecating joke that is praising women.
|
| Go back to primary school and learn the basics of _human_
| social interactions.
| wpietri wrote:
| I definitely wonder if industry is any better here. At least in
| academia, the papers are public, so there's an opportunity for
| scrutiny. But I've heard tell of "AI" boondoggles in both large
| companies and small. E.g., the large corporate "AI" efforts
| burning millions without making any real improvements. And I
| wonder how many startups out there have standards that are in
| effect lower than academia, but instead of writing papers they
| are shipping products than harm people's lives when they go
| wrong.
| jerzyt wrote:
| It is rampant in consulting business. At least in an internal
| project it's possible to pull the plug when the results are
| not promising. In a consulting engagement, when the results
| are garbage, there's no revenue and no potential for an
| upsell. In effect the pressure to "find" significant results
| is enormous.
| Aperocky wrote:
| As a previous Data 'Scientist' I can tell you that it
| absolutely is.
|
| Garbage in garbage out is the norm, the models are long
| established, but they can't mine gold from dirt. But nobody
| except the engineers seem to understand this.
|
| I'm so glad I left to become a regular software engineer. My
| code does not depend on a blackbox that is fed with crap, and
| can be reliably tested.
| jollybean wrote:
| Imagine if this happened in private enterprise, like at Google.
| The press would be all over it 24/7.
| coldtea wrote:
| Or rather you'd never had known about it. Much worse things
| have happened in private companies, only a few of which ever
| make it out in the press.
| dcolkitt wrote:
| Of course a lot of abuse happens in private enterprise. But
| typically, senior management is much much more proactive
| about it than department chairs and provosts. There's very
| few Fortune 500 companies where you could publicly get away
| with the kind of abuse that's just part and parcel of being
| a grad student or post doc at a major research university.
|
| It might still happen either because it's well concealed.
| Or it might happen at smaller or poorly managed companies.
| But at the typical functional, large corporation, egregious
| abuse of your subordinates essentially guarantees that
| you'll be terminated if/when it's brought to senior
| management's attention.
|
| Academia is different because tenured professors are given
| far more independence and autonomy. By contrast middle
| managers are tightly monitored and controlled by their own
| line managers. The typical large corporation strives very
| hard to promulgate a homogenous corporate culture across
| the org. Whereas academia as a system encourages professors
| to be fiercely independent maverick. That has both pros and
| cons, but one of the major cons is that it tolerates a lot
| more abuse and dysfunctional management towards the non-
| tenured subordinates.
| Beldin wrote:
| Your rosy picture of industry is surprising to me. It
| could be right, and the few takes I've heard (eg.
| recently, google's AI ethics mess) being only one side or
| exceptions that prove the rule.
|
| But I have currently no reason to expect better from
| industry, and will want some proof before putting it on a
| pedestal.
| [deleted]
| cycomanic wrote:
| You mean the number of suicides at foxconn? The forced labour
| that is likely contributing to many of the large mobile phone
| company bottomlines. The blatant benchmark fraud that happens
| all the time by all the large GPU and CPU manufacturers, I
| could go on. But I don't see the press all over this 24/7 t
| all, maybe some small niche outlets sometimes. In comparison
| the scrutiny that the academic world is under (if we relate
| to the affected people and effects) is much, much larger
| jollybean wrote:
| Your example proves my point: Foxconn is 1/2 a world away,
| and news from that part of the world never penetrates pop
| culture.
|
| That _we have_ heard about it, implies a degree of
| scrutiny.
|
| When Google fires an AI researcher in a sensitive position,
| it's an international event.
|
| If Exxon executives committed suicide in the face of some
| kind of forced fraud/corruption, it would be a national
| story.
| barry-cotter wrote:
| > You mean the number of suicides at foxconn?
|
| There are fewer suicides at FoxConn than would be expected
| given the age profile and number of their employees. If you
| employ hundreds of thousands of people some of them will
| kill themselves for reasons entirely unrelated from work.
| If they live in work dormitories they'll do it at work.
| totalZero wrote:
| Yes, I agree. It seems to me that academia feels external to
| the power structure, so it rarely gets the same kind of
| attention as industry because it doesn't stir up resentment.
| jollybean wrote:
| They are part of the power structure, they're just
| 'protected'. Most publications don't want to be seen as
| promoting a narrative that goes 'against science' even
| though of course it's not. There's no room for nuance in
| populism.
| 3grdlurker wrote:
| Protected, or just less noticed.
| salawat wrote:
| It's more the fact that academia is granted a greater
| degree of leniency because it is fundamentally something
| that is societally required for innovations to take
| place. Genuine ingenuity requires room to flourish, so in
| the West at least, the theory is you leave tge Boffins
| alone to do their wizardry.
|
| The part I don't understand is how extorting the students
| to fund bloated Administrative processes, sport teams,
| stadiums, and executive staff ever became a thing,
| nevermind the incestuous relationship with academic
| publishing. Almost all great work I've seen came not out
| of gnashing of teeth and publish or perish, but out of a
| labor of love or an odd obsession with truly
| understanding something until you could practically get
| it across to a 5th grader.
|
| "I can explain how, don't ask me why too many times,
| still figuring that part out."
| unishark wrote:
| The faculty have very little say in most of those aspects
| of universities you're complaining about.
|
| The media is just a crazy sideshow that cherry picks a
| tiny subset of stories to run or people to destroy when
| it fits the right narratives. When it comes to real
| scrutiny, I'd say faculty are under vastly more than
| people in industry. Though yes, google as an entire
| entity will analyzed more than some random professor. But
| the rank-and-file professor is also probably in far more
| constant danger of being ruined than almost any
| individual in a comparable role in industry. Also more
| than most businesses that no one cares about.
| jollybean wrote:
| Academia is not protected because of some intellectual
| notion of 'innovation' so much as they are on the right
| side of the bias presented in most publications. Most
| writers and commentators I think have venerable views of
| academia and probably err towards supporting that
| narrative publicly.
|
| While it's true that Profs may be under excessive
| scrutiny in some ways, which frankly might make them
| skittish - they are obviously not under 'the most
| important' kind of scrutiny which relates to the material
| legitimacy of their work in terms of 1) reproducability
| 2) fudging results and borrowing ideas and 3)
| misappropriation of credit 'up the chain of power' and
| 'from other peers'.
|
| Hence this article, and some other issues of legitimacy
| within academia.
|
| I think almost everything boils down to the fact that the
| low-hanging fruit have been had in science, and though
| there are 20x more scientists alive now than just a few
| generations ago, in many ways we're getting diminishing
| marginal returns - and even worse - it's incredibly hard
| to know which teams to back, and which not to.
|
| In the fog of war for funding, it leaves more room for
| back-stabbing than in most other places, even in the
| corporate world where at least there is some degree of
| job security.
| aabaker99 wrote:
| I'm really happy to see this article here. I quit my PhD where I
| was working on AI partly because of the "mundane day-to-day
| fraud" that the author and I observed in the field. Once you've
| read enough papers, you can easily see a kind of recipe to an
| academic paper. Present a new method, compare against a baseline,
| discuss the differences in the results found. To me, some
| researchers just follow this recipe and do not think about the
| steps critically. You see figures that plot two categories of
| points that are purportedly following different distributions but
| the error bars (or even more of the distributions) overlap. But a
| p-value says they are significant. So that's good enough to
| publish! I felt a lot of anxiety about sharing my research in
| this environment and felt pressure to make my work seem like it
| was the best solution for every problem when in reality there are
| more nuances.
| oldnews193 wrote:
| > I felt a lot of anxiety about sharing my research in this
| environment and felt pressure to make my work seem like it was
| the best solution for every problem when in reality there are
| more nuances.
|
| I am curious what kind of pressure you are referring to. It's
| your work and your decision how to present it to the world.
|
| As far as other people's research is concerned, academics can
| express their opinions by participating in peer review and
| expressing their opinions vocally at program-committee meetings
| (for instance, proposing artifact evaluation as a part of the
| review process).
| aabaker99 wrote:
| It is true that as the author of a research paper it is my
| decision how to present it. However, if you're too far off
| the mark, you are just going to be rejected by peer review or
| they will ask for revisions. The fact is as a PhD student,
| you are trying to join the research community. You aren't in
| a position to change that community. You have to tailor your
| work and statements to fit the mold. Students who are
| outspoken voicing these concerns, especially if they rise to
| the level of abuse or research misconduct, must tread
| carefully. A case I was made aware of in the study of
| research ethics is of Anil Potti [1], [2]. From [2], the
| statement from the whistleblower, "In raising these concerns,
| I have nothing to gain and much to lose" is apt.
|
| For more mild cases, like those mentioned in the article, the
| stakes are lower but there is also more plausible
| deniability. If Duke University tried to bury even the
| blatant abuse, you can imagine how it is also hard to
| confront the article's so-called "mundane, day-to-day fraud".
|
| [1] https://en.wikipedia.org/wiki/Anil_Potti [2]
| https://www.sciencemag.org/news/2015/01/duke-university-
| offi...
| voakbasda wrote:
| I had the same experience as GP. I dropped out of a PhD
| program after witnessing widespread low-level fraud.
|
| I published a single paper with my advisor. I asked questions
| that a peer reviewer would have asked, and my concerns were
| basically ignored. It was more important to publish than be
| accurate and precise with wording (partly due to length
| limits imposed by the publication). Pushing back harder
| likely would have had a deleterious effect on my progress in
| the program. It would have been career suicide.
|
| Worse, the topic was nothing more than rehashed results from
| a paper he published years earlier. There was really nothing
| even worth publishing. This was but one example that showed
| me that academia is endemic with fraud.
|
| Eventually I killed that career path, because I could never
| participate in such a fundamentally corrupt system. There
| will be no reform here without Revolution.
| marsven_422 wrote:
| Government funding...what you expect?
| re-al wrote:
| The lies in science and academia are out of control. Whether its
| this example, the replication crisis, the funding crisis - the
| upshot is that we cannot be confident about the 'reality' that is
| being presented to us.
| gher-shyu3i wrote:
| How much of this fraud is committed by Chinese? I've known
| several people in academia who told me about blatant cheating and
| fraud committed by them, their culture does not frown upon this
| behavior, and now everyone is paying the price.
| weasel_words wrote:
| The actual "smoking gun" is three levels deep:
|
| 1) https://cacm.acm.org/magazines/2021/6/252840-collusion-
| rings... then 2) https://medium.com/@tnvijayk/potential-
| organized-fraud-in-ac... then 3)
| https://huixiangvoice.medium.com/evidence-put-doubts-on-the-...
|
| Not to try and dissuade this internet mob...but instead of doing
| the whole internet-mob-get-eyeballs-to-my-blog-by-rehashing-
| someone-elses-blog thing, get to basics and be precise like a
| scientist would be. Examples. Forensics. Details.
| tester756 wrote:
| I'd add to it: "Statistics done wrong"
___________________________________________________________________
(page generated 2021-05-30 23:01 UTC)