[HN Gopher] i sensed anxiety and frustration at NeurIPS 24
___________________________________________________________________
i sensed anxiety and frustration at NeurIPS 24
Author : wavelander
Score : 120 points
Date : 2024-12-22 05:48 UTC (2 days ago)
(HTM) web link (kyunghyuncho.me)
(TXT) w3m dump (kyunghyuncho.me)
| PaulHoule wrote:
| Trained as a physicist I became acutely aware of what mismatches
| in the academic job market look like and, particularly, how
| smoking hot fields can become burned out fields in just about
| _the time it takes to complete a PhD._
|
| (Some physicists told me about how quickly Ken Wilson's
| application of RG to phase transitions went from the the next big
| thing to old hat, for instance.)
|
| When I can bear to read editorials in CACM I see the CS
| profession has long been bothered by whipsawing demand for
| undergraduate CS degrees. I've never heard about serious
| employment problems for CS PhDs and maybe I never will because
| they have a path to industry that saves face better than the
| paths for physics.
|
| Maybe we will hear about a bust this time. As a cog in the social
| sciences department, I used to have a view of a baseball diamond
| out my office window but now there is a construction site for a
| new building to house the computer science, information science
| and "statistics and data science" departments which are bulging
| in undergraduate enrollment.
|
| Will there finally be a bust?
| NewJazz wrote:
| Universities are definitely trying to use data science as a
| draw for students in the face of declining enrollment. I know
| one R1 employee who is taking it upon themselves to enable the
| training of tens of thousands of students over the years.
| People are more excited about AI than coding.
| zitterbewegung wrote:
| AI has always had the concept of a "winter" or bust. [1] As in
| computer science in general we have had people shift
| concentration on what can make the most and that can occur
| during a PhD. The thing about the AI winter this time is that
| ML and Deep learning and LLMs advanced so quickly that a large
| bust in AI didn't really occur. Also, I have heard about people
| getting the AI jobs and then complaining it's all about
| cleaning data and nothing about applying new AI techniques to
| solve a big challenging problem. For CS in general and IT I can
| sort of see this as a business cycle and LLMs doing some
| replacement of jobs or it could just be a business cycle.
|
| [1] https://en.wikipedia.org/wiki/AI_winter?wprov=sfti1#
| janalsncm wrote:
| I think you two are talking about different things. The AI
| winter as I understand it refers to slowing technological
| advancement in "expert systems" for example. But
| technological development is moving faster than ever.
|
| This is different. It's a weird combination of huge amounts
| of capital chasing a very specific idea of next token
| prediction, and a slowdown in SWE hiring that may be related.
| It's the first time "AI" as an industry has eaten itself.
| est31 wrote:
| From what I read, AI "winters" have not just had less
| progress but also have had less research activity in
| general: during them, AI was seen as a dead end that didn't
| provide value, which translates to less funding, which
| translates to less people doing research on it (or
| introducing AI to the research they wanted to do already).
| throwup238 wrote:
| The AI winter refers to the collapse in government and
| industry funding. It came as a result of a lot of hype like
| expert systems not panning out but it was about the money
| that dried up. This time "it's different" but VCs are just
| as fickle as grant reviewers and executives - that funding
| could dry up at any moment if one of the big AI companies
| has a down round or goes bankrupt altogether. I don't think
| a second AI winter is likely because of how useful LLMs
| have proven this time around, but we're probably going to
| have a correction sooner or later.
|
| I don't think the hiring slowdown and AI are related. Some
| companies are using rhetoric about AI to save face but the
| collapse in the job market was due to an end to ZIRP.
| prpl wrote:
| Physics degrees are great for a successful career outside of
| physics.
| FredPret wrote:
| If you can do an undergrad in physics and also hold a
| conversation, your path is open to getting to the top 0.1% in
| terms of problem solving ability
| vslira wrote:
| It's kinda funny, the general dread regarding AI in tech circles
| is mostly due to fears that all jobs will be automated away,
| while the concerns expressed by the researchers according to the
| author are ironically a lot narrower in the sense that they
| haven't learned the currently cool topic.
| inerte wrote:
| Also, and not quite. I believe the author is mostly talking
| about how undergraduate and masters degree can already do what
| PhD students set out to do a few years ago, so they might
| complete their PhD but it will be them vs someone cheaper and
| younger with the same hot skills and knowledge. The extra PhD
| ooomph isn't providing more value.
| neom wrote:
| I always thought PhDs often get long because research takes a
| long time and things can change drastically over that period
| of time, be it the research doesn't go the direction they
| expected or the world around them changed? If I was to go
| down the PhD path I'd consider it might end up taking quite a
| long time, because of exactly what you mention in your
| comment.
| sdenton4 wrote:
| Big conferences suck in just about every area. Back when I was
| going to the joint math meetings (a couple few thousand math
| people), the dominant smell was adrenaline, as departments tried
| to streamline their hiring process into the conference setting...
| Phds seeking postdoc positions were therefore trying to talk up
| their work publicly and rushing around to interviews on the side.
| It sucks as a conference, though: you get lots of heat, and
| little light.
|
| I get the same feeling going to big ML conferences now. The
| incentives are all about job prospects and hype, rather than
| doing cool stuff.
|
| The sweet spot is the 200 person conferences, imo. You have a
| much better focus on the science, can literally meet everyone if
| you put your mind to it. And the spaces tend to be more about the
| community and the research directions, rather than job prospects.
| Ar-Curunir wrote:
| What does this comment have to do with the article?
| Aurornis wrote:
| The article is about NeurIPS, a very large conference. It's
| in the title.
| Animats wrote:
| _" There are much fewer opportunities if they do not want to work
| on productizing large-scale language models, and these positions
| are disappearing quickly."_
|
| Yes. In 1900, the thing to be was an expert electrician. I had a
| friend with a PhD in bio who ended up managing a coffee shop
| because she'd picked the wrong branch of bio.
|
| LLMs may be able to help with productizing LLMs, further reducing
| the need for people in that area.
| jvanderbot wrote:
| I appreciated the humility in this paragraph in particular
|
| > once the first generation of lucky PhD's (including me!) who
| were there not out of career prospects but mostly out of luck (or
| unluck), we started to have a series of much more brilliant and
| purpose-driven PhD's working on deep learning. because these
| people were extremely motivated and were selected not by luck but
| by their merits and zeal, they started to make a much faster and
| more visible progress. soon afterward, this progress started to
| show up as actual products.
|
| I have to say, though, that AI and robotics are going through
| similar transitions as were highlighted in TFA. Robotics has been
| basically defined by self driving cars for a long time, and we're
| starting to see the closure of some very big programs. Already
| the exorbitant salaries are much lower based on what I've seen,
| and demand is flat lining. My hope is that the senior level
| engineers with good PhD backgrounds move out into the broader
| field and bring their experience and research zeal with them as a
| force multiplier. I expect the diaspora of talent to reinvigorate
| industry innovation in robotics.
|
| So it will be with LLM-focused researchers in industry in the
| next phase after we pass peak hype. But the things those battle-
| scarred researchers will do for the adjacent fields that were not
| hype-cycled to death will probably be amazing.
|
| Unless they succeed in replacing themselves with general AI. Then
| all bets are off.
| jprete wrote:
| TFA?
| fph wrote:
| Slang for "the fine article". Comes from RTFA, "read the fine
| article". Or some other adjective of your preference that
| starts with F.
| swores wrote:
| (Where 99% of use refers to fuck not fine.)
| xandrius wrote:
| Dunno but RTFA was always presented to me in the less kind
| version, so it's pretty odd to see it when someone uses it
| in its positive spin.
| ska wrote:
| Totally different typical usage:
|
| RTFA == "you should have actually read the article
| instead of wasting everyone's time"
|
| TFA == "referring specifically to the [original, in this
| context] article"
| SoftTalker wrote:
| I always thought it was "The Featured Article" to be polite
| about it.
| AndrewKemendo wrote:
| This is probably the best example of what it feels like to be
| someone dealing with cognitive dissonance of Elite Overproduction
| in AI.
|
| https://en.m.wikipedia.org/wiki/Elite_overproduction
|
| Since 2012 (Alexnet winning Imagenet with DL) "AI" has been
| dominated by corporations for one simple and obvious reason that
| Suttom has pointed out over and over: The group with the best
| data wins
|
| That wasn't always true. It used to be the case that the
| government had better data or academia had better data, but it's
| not even close at this point to the point where both of those
| groups are nearly irrelevant in either applied AI or AI research.
|
| I've been applying "AI" at every possible chance I have since
| 1998 (unreal engine + A* + knn ...) and the field has
| foundationally changed three or four times since.
|
| When I started, people looked at those of us that stated out loud
| that we wanted to work on AGI as totally insane.
|
| Bengio spoke at the AGI 2014 conference, and at a lunch between
| myself, Ben G, Josha Bach and Bengio we all agreed DL was going
| to central to AGI - but it was unclear who would own the data.
| Someone argued that AGI would most likely come out of an open
| source project and my argument was there are no structured
| incentives to allow for the type of organization necessary at the
| open source level for that.
|
| My position was that DL isn't sufficient - we need a forward
| looking RL streaming reward system that is baked into the
| infrastructure between planning and actuation. I still think
| that's true and why AGI (robots doing things instead of people at
| greater than human level) is probably only a decade away.
|
| It's pretty wild that stating out loud that your goal is to build
| superhuman artificial intelligence is still some kind crazy idea
| - even though it's so obviously the trajectory we're on.
|
| As much as I dislike OpenAI and the rest of the corporate AI
| companies, I respect the fact that they've been very vocal about
| trying to build general intelligence and superhuman intelligence.
| nextos wrote:
| > My position was that DL isn't sufficient - we need a forward
| looking RL streaming reward system [...]
|
| I agree, but don't you think some degree of symbolism or
| abstraction building is necessary for AGI? Current systems seem
| to be too fragile and data-intensive.
| mjburgess wrote:
| I'm reading the author's PhD thesis at the moment, which he seems
| to have successfully applied to some modelling projects.
|
| I'd imagine the most significant aspect of this betrayal of PhD
| students, as far as it exists, is to polarise the domain along
| research lines that never very clearly mapped to the perennial
| objectives of modelling or mere curve-fitting. Had a thread of
| 'modelling science' been retained then it would always be useful.
|
| Businesses may, at the moment, appear to have been confused into
| the value of curve-fitting --- the generic skills of algorithm
| design, scientific modelling, and the like, will survive the next
| wave of business scam.
| antonvs wrote:
| I'll take "what are capital letters" for $500 Alex
| sparrish wrote:
| For real. Very distracting to read this with no upper-case
| letters at the beginning of sentences. Strangely, many acronyms
| have properly placed upper-case letters. I don't understand the
| decision to omit them for single letter words or at the
| beginning of a sentence.
| levitate wrote:
| Guessing this is a generational thing. I wasn't distracted by
| it because it's the same style that many of my friends text
| with.
|
| This is the first time I've seen it in an actual article,
| however.
| fngjdflmdflg wrote:
| Relevant quote:
|
| >this post will be more of less a stream of thoughts rather
| than a well-structured piece
| renjimen wrote:
| As a consultant data scientist who attended NeurIPS this year I
| was surprised at how few of the talks and posters covered
| innovations practically applicable outside of niche domains.
| Those that did (DL for tabular data, joint LLM training with RAG,
| time series foundational models) were swamped with attendees.
| It's not that there aren't jobs for innovators, it's just that
| there needs to be some amount of applicability of your research
| outside the theory of learning.
| mvkel wrote:
| Isn't this just the culture of "true" AI researchers? Until
| very recently (2019?) it was almost exclusively an academic
| pursuit
| currymj wrote:
| the physical conference itself is now an excuse for companies
| to throw lavish parties, but NeurIPS as a publication venue is
| still mainly for academics.
|
| for research that translates directly to an industry setting,
| look someplace like KDD. venues like that are where data
| science/ML teams who do some research but are mainly product
| focused tend to publish.
| bowsamic wrote:
| Absolutely no chance am I reading an article that doesn't even
| give the respect of capitalising sentences. Honestly despicable
| and I feel personally offended
| haustlauf wrote:
| It's an interesting piece if you can muster the strength to
| make it through the irksome writing style. Either way, I don't
| think your comment adds much. Why not suffer in silence?
| amelius wrote:
| The field is now owned by Big Capital.
|
| Democratization of AI was a joke.
| adityamwagh wrote:
| How about governments pass a law that requires all AI research
| & engineering efforts to provide all the data that they use,
| and all technical details must be open access and any AI stuff
| cannot be patented. Does that solve the democratisation issue?
| monocasa wrote:
| There's next to no chance of that happening though.
|
| It'd be like asking if a law was passed making all of the
| companies on the S&P 500 co-ops owned equally by all
| Americans would solve the democratization issue.
| pgodzin wrote:
| The author is certainly doing his part fighting against capital
| (letters)
| mattlondon wrote:
| "betrayed", "promised" - these are quite telling about the sense
| of entitlement these people have (rightly or wrongly).
|
| The world doesn't owe you anything, even if you did a PhD in AI.
|
| Sorry if you picked the wrong thing, but it's the same for
| anything at degree level.
| currymj wrote:
| i think PhD programs and advisors did make some implicit or
| explicit promises to recruit students to work for them, very
| hard, for years, at very low salaries.
|
| the PhD is not like a master's program, it's an apprenticeship
| relationship where you essentially work for a senior professor,
| and your work directly financially benefits the department as
| well.
|
| so there is a deal here, but if one end of it is collapsing,
| the students in some departments might very well feel betrayed
| -- not by society but by their advisors and programs.
| SoftTalker wrote:
| There's no reason to do a Ph.D. if you don't aspire to an
| academic position. If you want to be a practitioner, do a
| Masters.
| ska wrote:
| It's really not that simple. But it's fair to say that
| while a masters can often be a very straightforward
| training to get you ready for a particular job, you really
| shouldn't go into a PhD program without a good idea of a)
| why and b) where you are aiming for afterward. For STEM I
| would usually add c) someone else is paying for it
| mistrial9 wrote:
| the competitive investment world has gotten a flavor of
| ever-increasing requirements.. it fits with a scarcity
| model they thrive in
| currymj wrote:
| this is true overall, but for the last decade there have
| been many research-oriented positions at large tech
| companies for which the PhD degree is a formal job
| requirement. others don't make it a formal requirement, but
| demand research experience which can be gotten most easily
| as a PhD student.
| perching_aix wrote:
| Is there a point to saying something like this other than to be
| explicitly uncompassionate towards those affected?
|
| I could simply respond with "sorry to hear you upset yourself
| by considering these people entitled", and it would have about
| the same merit.
|
| It's like telling somebody after a close relative or some else
| dear to them died that what, did they expect that person would
| live forever? No, do you?
|
| Do you genuinely think their feelings of betrayal stem from an
| unreasonable notion of the world? Have you at all considered
| that the expectations they harbored were not 100% of their own
| creation?
| friendlyasparag wrote:
| I suspect many commentators would say the same thing to the
| English majors out there who were told that passion and a
| college degree would lead to economic mobility and not a load
| of debt. Weren't they also harboring unreasonable views that
| were not 100% of their own creation?
| perching_aix wrote:
| I'm unable to read minds, so if in your head that self-
| serving scenario played out such that they were, then they
| were. Does that justify being uncompassionate towards them
| by calling them entitled? Is entitled really the best fit
| for what they were being, rather than say, just plain
| unreasonable or misguided? Can you read minds that you know
| they were thinking the world owes them _and that 's how_
| they developed a passion and completed a college degree?
| friendlyasparag wrote:
| I'm glad you are willing to treat both groups the same. I
| suspect many on this site wouldn't and would take pains
| to explain how they're entirely different :)
| riku_iki wrote:
| > Is there a point to saying something like this other than
| to be explicitly uncompassionate towards those affected?
|
| those "affected" have the same little compassion towards many
| millions of others who are not lucky to work on AI in top
| school and visiting top AI conference. That's what makes them
| entitled.
| perching_aix wrote:
| How would you know that everyone who felt "betrayed" were
| also "have the same little compassion towards many millions
| of others who are not lucky to work on AI in top school and
| visiting top AI conference"?
|
| This is just blatantly painting high profile individuals
| with a negative brush because it personally appeals to your
| fantasies.
| riku_iki wrote:
| > everyone who felt "betrayed"
|
| feeling betrayed is my test criteria, since many others
| are "betrayed" much more, but those individuals focused
| on their feelings and not on systematic issues in
| general, hence this makes them "entitled".
|
| > This is just blatantly painting high profile
| individuals with a negative brush because it personally
| appeals to your fantasies.
|
| you have to check your language if you want to continue
| productive discussion.
| perching_aix wrote:
| I disagree with your test criteria. It doesn't adequately
| support that entitlement is present in or applies to
| these people. You can be swayed by your feelings for a
| myriad of things, on its own I do not consider that any
| sufficient. Even if they come from a privileged
| background or gained a privileged status by virtue of
| studying where they did.
|
| > you have to check your language if you want to continue
| productive discussion.
|
| I disagree that my use of language was unreasonable. And
| to clarify, I do not wish to continue this conversation,
| productively or otherwise.
| riku_iki wrote:
| > I disagree that my use of language was unreasonable.
| And to clarify, I do not wish to continue this
| conversation, productively or otherwise.
|
| bye then
| bartwr wrote:
| If someone believed they will earn 2-5x better than in academia,
| with full freedom to work on whatever interests them, and no need
| to deliver value to the employer... Well, let's say "ok", we have
| all been young and naive, but if their advisors have not adjusted
| their expectations, they are at fault, maybe even fraudulent.
|
| Even being in elite research groups at the most prestigious
| companies you are evaluated on product and company Impact, which
| has nothing to do with how groundbreaking your research is, how
| many awards it gets, or how many cite it. I had colleagues at
| Google Research bitter that I was getting promoted (doing
| research addressing product needs - and later publishing it,
| "systems" papers that are frowned upon by "true" researchers),
| while with their highly cited theoretical papers they would get a
| "meet expectations" type of perf eval and never a promotion.
| einpoklum wrote:
| > full freedom to work on whatever interests them, and no need
| to deliver > value to the employer...
|
| That was an exaggeration. No employee has full freedom, and I
| am sure it was expected that you do something which within some
| period of time, even if not immediately, has prospects for
| productization; or that when something becomes productizable,
| you would then divert some of your efforts towards that.
| arugulum wrote:
| I believe the above post was highlighting that as a
| misconception young people may have, not saying it is the
| case.
| SoftTalker wrote:
| Well there are a few. The Distinguished Scientists at
| Microsoft Research probably get to work on whatever interests
| them. But that is a completely different situation from a new
| Ph.D. joining a typical private company.
| TrackerFF wrote:
| Someone correct me if this is wrong, but wasn't that pretty
| much the premise of Institute for Advanced Study? Minus very
| high-paying salaries. Just total intellectual freedom, with
| zero other commitments and distractions.
|
| I know Feynman was somewhat critical to IAS, and stated that
| the lack of accountability and commitment could set up
| researchers to just follow their dreams forever, and eventually
| end up with some writers block that could take years to
| resolve.
| Aurornis wrote:
| > Minus very high-paying salaries.
|
| They very high salaries are central to the situation.
|
| If you remove high salary then you have a lot more freedom.
| The tradeoff is the entire point of discussion.
| dr_dshiv wrote:
| Come be a professor in the Netherlands! You can even run a
| company on the side. Freedom is real. You don't get paid
| well for it.
| nomad_horse wrote:
| Yet your Google Research colleagues still earned way more than
| in academia, even without the promo.
|
| Plus, there were quite a few places where a good publication
| stream did earn a promotion, without any company/business
| impact. FAIR, Google Brain, DM. Just not Google Research.
|
| DeepMind didn't have any product impact for God knows how many
| years, but I bet they did have promos happening:)
| bowsamic wrote:
| > with full freedom to work on whatever interests them, and no
| need to deliver value to the employer
|
| You know that in academia you constantly have to beg for money
| by trying to convince government agencies that you're bringing
| them value right?
| yodsanklai wrote:
| > you are evaluated on product and company Impact, which has
| nothing to do with how groundbreaking your research is,
|
| I wonder... There are some academics who are really big names
| in their fields, who publish like crazy in some FAANG. I assume
| that the company benefits from just having the company's name
| on their papers at top conferences.
| bundie wrote:
| Great read! Though it's a bit off putting how paragraphs start
| with a lower case letter
| fuzzfactor wrote:
| I guess I'm a little bit optimistic, but looking at the title I
| was thinking that the AI models had reached the point where
| _they_ were the ones expressing anxiety and frustration :)
|
| Never mind . . .
| patrickhogan1 wrote:
| This is the best time in history to research AI and get paid for
| it.
|
| I respect your perspective and acknowledge I might be missing key
| context since I didn't attend the conference. That said, one
| interpretation is that you're grappling with the rapid
| acceleration of innovation--a pace you've helped shape.
| Challenges that once seemed distant now feel imminent, creating a
| mix of excitement and unease, like the moment a rollercoaster
| drop starts to terrify.
| atorodius wrote:
| As someone who did a PhD ending 4y ago and joined a industry
| research lab, I can relate to this a lot, I think this is very
| much spot on
|
| > a lot of these PhD's hired back then were therefore asked to
| and free to do research; that is, they chose what they want to
| work on and they publish what they want to publish. it was just
| like an academic research position however with 2-5x better
| compensation as well as external visibility and without teaching
| duties,
|
| exactly!
|
| > such process standardization is however antithetical to
| scientific research. we do not need a constant and frequent
| stream of creative and disruptive innovations but incremental and
| stable improvements based on standardized processes.
|
| A lot of the early wave AI folks struggle with this. They wanna
| keep pushing wild research ideas but the industry needs slow
| incremental stuff, focused on serving
| white_beach wrote:
| (shoift)Tab Control, also make it work with the mouse like others
| LarsDu88 wrote:
| I could've swore I posted this article as well.
| whiplash451 wrote:
| > some of them probably feel betrayed, as the gap between what
| they were promised earlier and what they see now is growing
| rapidly.
|
| When you enter a PhD program, you are promised nothing else but
| hard work (hopefully within a great research group). I don't know
| what the author is referring to here.
| ipunchghosts wrote:
| There's an underlying sentiment through the article that there
| isn't more significant research to do to further improve ai and
| generate more revenue. This is a large gap and myopic.
|
| For example, the amount of compute needed to obtain state of the
| art on a benchmark is only obtainable by big labs. How can we
| improve the learning efficiency of deep learning so that one can
| obtain similar performance with 100x less compute.
| jfmc wrote:
| Wrong capitalization makes me feel really axious and frustrated.
| vvrm wrote:
| This story plays out so often that here should be a law about it:
| Supply lags demand, prices soar, everybody hears about it,
| everybody pours in, supply surges, demand normalizes, supply
| overshoots demand, prices collapse. Already happened to software
| engineering, data science. Keeps happening to hardware production
| every few years. Sounds like AI research is headed that way too.
| somethingsome wrote:
| In my lab, we try to focus on the science and less on applicative
| deep learning, of course it has its uses and we develop
| 'fashionable' deep learning with some students, but I think the
| value of a PhD remains in the path taken.
|
| It's all that you learn doing it, it is not just knowledge, it's
| the scientific method, social skills, learning to communicate
| science.
|
| A PhD is a personal path, an experience in life that changes it
| profoundly, I hope that those students will be able to handle any
| kind of future issues, not just a very niche applicative aspect
| of deep learning.
___________________________________________________________________
(page generated 2024-12-24 23:01 UTC)