[HN Gopher] Accelerating scientific breakthroughs with an AI co-...
___________________________________________________________________
Accelerating scientific breakthroughs with an AI co-scientist
Author : Jimmc414
Score : 231 points
Date : 2025-02-19 14:32 UTC (8 hours ago)
(HTM) web link (research.google)
(TXT) w3m dump (research.google)
| Jimmc414 wrote:
| paper:
| https://storage.googleapis.com/coscientist_paper/ai_coscient...
| mnky9800n wrote:
| Tbh I don't see why I would use this. I don't need an ai to
| connect across ideas or come up with new hypothesis. I need it to
| write lots of data pipeline code to take data that is organized
| by project, each in a unique way, each with its own set of
| multimodal data plus metadata all stored in long form documents
| with no regular formatting, and normalize it all into a giant
| database. I need it to write and test a data pipeline to detect
| events both in amplitude space and frequency space in acoustic
| data. I need it to test out front ends for these data analysis
| backends so i can play with the data. Like I think this is domain
| specific. Probably drug discovery requires testing tons of
| variables one by one iterating through the values available. But
| that's not true for my research. But not everything is for
| everybody and that's okay.
| not_kurt_godel wrote:
| Agreed - AI that could take care of this sort of cross-system
| complexity and automation in a reliable way would be actually
| useful. Unfortunately I've yet to use an AI that can reliably
| handle even moderately complex text parsing in a single file
| more easily than if I'd just done it myself from the start.
| mnky9800n wrote:
| Yes. It's very frustrating. Like there is a great need for a
| kind of data pipeline test suite where you can iterate
| through lots of different options and play around with
| different data manipulations so a single person can do it.
| Because it's not worth it to really build it if it doesn't
| work. There needs to be one of these
| astronomer/dagster/apache airflow/azure ml tools that are
| quick and dirty to try things out. Maybe I'm just naive and
| they exist and I've had my nose in Jupyter notebooks. But I
| really feel hindered these days in my ability to prototype
| complex data pipelines myself while also considering all of
| the other parts of the science.
| SubiculumCode wrote:
| The doing tends to be the hard part. Every scientist has 1000
| idea for every one they get a chance to pursue.
|
| That said, I requested early access.
| parineum wrote:
| > I don't need an ai to connect across ideas or come up with
| new hypothesis.
|
| This feels like hubris to me. The idea here isn't to assist you
| with menial tasks, the idea is to give you an AI generalist
| that might ne able to alert you to things outside of your field
| that may be related to your work. It's not going to reduce your
| workload, in fact, it'll probably increase it but the result
| should be better science.
|
| I have a lot more faith in this use of LLMs than I do for it to
| do actual work. This would just guide you to speak with another
| expert in a different field and then you take it from there.
|
| > In many fields, this presents a breadth and depth conundrum,
| since it is challenging to navigate the rapid growth in the
| rate of scientific publications while integrating insights from
| unfamiliar domains.
| ttpphd wrote:
| Are you a scientist?
| OutOfHere wrote:
| It is obvious that scientists are afraid for their job,
| that mere lab technicians will now be sufficient.
| ttpphd wrote:
| Witness the pure arrogance of tech bros
| goatlover wrote:
| Sure, as soon as managers replace software engineers by
| spending their working hours prompting LLMs.
| iak8god wrote:
| > the idea is to give you an AI generalist that might ne able
| to alert you to things outside of your field that may be
| related to your work
|
| That might be a good goal. It doesn't seem to be the goal of
| this project.
| coliveira wrote:
| > This feels like hubris to me.
|
| No, any scientist has hundreds of ideas they would like to
| test. It's just part of the job. The hard thing is to do the
| rigorous testing itself.
| the_snooze wrote:
| >The hard thing is to do the rigorous testing itself.
|
| This. Rigorous testing is _hard_ and it requires a high
| degree of intuition and intellectual humility. When I 'm
| evaluating something as part of my resaerch, I'm constantly
| asking: "Am I asking the right questions?" "Am I looking at
| the right metrics?" "Are the results noisy, to what extent,
| and how much does it matter?" and "Am I introducing
| confounding effects?" It's really hard to do this at scale
| and quickly. It necessarily requires slow measured thought,
| which computers really can't help with.
| mnky9800n wrote:
| I have a billion ideas, being able to automate the testing of
| those ideas in some kind of Star Trek talk to the computer
| and it just knows what you want way would be perfect. This is
| the promise of ai. This is the promise of a personal
| computer. It is a bicycle for your mind. It is not hubris to
| want to be able to iterate more quickly on your own ideas. It
| is a natural part of being a tool building species.
| anothermathbozo wrote:
| Imagine someone can do the things you can't do and needs help
| doing the things you can.
| tippytippytango wrote:
| Exactly, they want to automate the most rewarding part that we
| don't need help with... plus I don't believe they've solved the
| problem of LLMs generating trite ideas.
| trilobyte wrote:
| Sounds like the message artists were giving when generative
| AI started blowing up.
| eamag wrote:
| I think you're just not the target audience. If AI can come up
| with some _good_ ideas and then split it into tasks some of
| them an undergrad can do - it can speed up the global research
| speed by involving more people in useful science
| coliveira wrote:
| In science, having ideas is not the limiting factor. They're
| just automating the wrong thing. I want to have ideas and ask
| the machine to test for me, not the other way around.
| eamag wrote:
| The difference is the complexity of ideas. There are
| straightforward ideas anyone can test and improve, and
| there are ideas where only PhDs in CERN can test
| JW_00000 wrote:
| I don't think that's really right. E.g. what makes
| finding the Higgs boson difficult is that you need to
| build a really large collider, not coming up with the
| idea, which could be done 50 years earlier. Admittedly
| the Higgs boson is still a "complex idea", but the
| bottleneck still was the actual testing.
| crypto420 wrote:
| I'm not sure if people here even read the entirety of the
| article. From the article:
|
| > We applied the AI co-scientist to assist with the prediction of
| drug repurposing opportunities and, with our partners, validated
| predictions through computational biology, expert clinician
| feedback, and in vitro experiments.
|
| > Notably, the AI co-scientist proposed novel repurposing
| candidates for acute myeloid leukemia (AML). Subsequent
| experiments validated these proposals, confirming that the
| suggested drugs inhibit tumor viability at clinically relevant
| concentrations in multiple AML cell lines.
|
| and,
|
| > For this test, expert researchers instructed the AI co-
| scientist to explore a topic that had already been subject to
| novel discovery in their group, but had not yet been revealed in
| the public domain, namely, to explain how capsid-forming phage-
| inducible chromosomal islands (cf-PICIs) exist across multiple
| bacterial species. The AI co-scientist system independently
| proposed that cf-PICIs interact with diverse phage tails to
| expand their host range. This in silico discovery, which had been
| experimentally validated in the original novel laboratory
| experiments performed prior to use of the AI co-scientist system,
| are described in co-timed manuscripts (1, 2) with our
| collaborators at the Fleming Initiative and Imperial College
| London. This illustrates the value of the AI co-scientist system
| as an assistive technology, as it was able to leverage decades of
| research comprising all prior open access literature on this
| topic.
|
| The model was able to come up with new scientific hypotheses that
| were tested to be correct in the lab, which is quite significant.
| preston4tw wrote:
| This is one thing I've been wondering about AI: will its broad
| training enable it to uncover previously covered connections
| between areas the way multi-disciplinary people tend to, or
| will it still miss them because it's still limited to its
| training corpus and can't really infer.
|
| If it ends up being more the case that AI can help us discover
| new stuff, that's very optimistic.
| semi-extrinsic wrote:
| In some sense, AI should be the most capable at doing this
| within math. Literally the entire domain in its entirety can
| be tokenized. There are no experiments required to verify
| anything, just theorem-lemma-proof ad nauseam.
|
| Doing this like in this test, it's very tricky to rule out
| the hypothesis that the AI is just combining statements from
| the Discussion / Future Outlook sections of some previous
| work in the field.
| blacksmith_tb wrote:
| Not that I don't think there's a lot of potential in this
| approach, but the leukemia example seemed at least poorly-
| worded, "the suggested drugs inhibit tumor viability" reads
| oddly given that blood cancers don't form tumors?
| klipt wrote:
| Health professionals often refer to leukemia and lymphoma as
| "liquid tumors"
| drgo wrote:
| Lots of blood cancers form solid tumors (e.g., in lymph
| nodes)
| terminalbraid wrote:
| I expect it's going to be reasonably useful with the "stamp
| collecting" part of science and not so much with the rest.
| shpongled wrote:
| That a UPR inhibitor would inhibit viability of AML cell lines
| is not exactly a novel scientific hypothesis. They took a
| previously published inhibitor known to be active in other cell
| lines and tried it in a new one. It's a cool, undergrad-level
| experiment. I would be impressed if a sophomore in high school
| proposed it, but not a sophomore in college.
| klipt wrote:
| Only two years since chatGPT was released and AI at the level
| of "impressive high school sophomore" is already blase.
| thomastjeffery wrote:
| Sure, but is it more impressive than _books_?
| directevolve wrote:
| Most people here know little to nothing of biomedical
| research. Explaining clearly why this isn't a
| scientifically interesting result is helpful.
| rtkwe wrote:
| Suggesting "maybe try this known inhibitor in other cell
| lines" isn't exactly novel information though. It'd be more
| impressive and useful if it hadn't had any published
| information about working as a cancer inhibitor before.
| People are blase about it because it's not really beating
| the allegations that it's just a very fancy parrot when the
| highlight of it's achievements is to say try this known
| inhibitor with these other cell lines, decent odds that the
| future work sections of papers on the drug already
| suggested trying on other lines too...
| baq wrote:
| A couple years ago even suggesting that a computer could
| propose anything at all was sci-fi. Today a computer read
| the whole internet, suggested a place to look at and
| experiments to perform and... 'not impressive enough'.
| Oof.
| Workaccount2 wrote:
| People are facing existential dread that the knowledge
| they worked years for is possibly about to become worth a
| $20 monthly subscription. People will downplay it for
| years no matter what.
| Nevermark wrote:
| Especially when you consider the artificial impressive high
| school sophomore is capable of having impressive high
| school sophomore ideas across and between an incredibly
| broad spectrum of domains.
|
| And that their generation of impressive high school
| sophomore ideas is faster, more reliable, communicated
| better, and can continue 24/7 (given matching
| collaboration), relative to their bio high school sophomore
| counterparts.
|
| I don't believe any natural high school sophomore as
| impressive on those terms, has ever existed. Not close.
|
| We humans (I include myself) are awful at judging things or
| people accurately (in even a loose sense) across more than
| one or two dimensions.
|
| This is especially true when the mix of ability across
| several dimensions is novel.
|
| (I also think people under estimate the degree that we, as
| users and "commanders" of AI, bottleneck their potential. I
| don't suggest they are ready to operate without us. But
| that our relative lack of energy, persistence & focus all
| limit what we get from them in those dimensions, hiding
| significant value.
|
| We famously do this with each other, so not surprising. But
| worth keeping in mind when judging limits: whose limits are
| we really seeing.)
| howenterprisey wrote:
| I don't need high school level ideas, though. If people
| do, that's good for them, but I haven't met any. And if
| the quality of the ideas is going to improve in future
| years, that's good too, but also not demonstrated here.
| Nevermark wrote:
| I am going to argue that you do. Then I will be
| interested in your response, if you feel inclined.
|
| We all have our idiosyncratically distributed areas of
| high intuition, expertise and fluency.
|
| None of us need apprentice level help there, except to
| delegate something routine.
|
| Lower quality ideas there would just gum things up.
|
| And then we all have vast areas of increasingly lesser
| familiarity.
|
| I find, that the more we grow our strong areas, the more
| those areas benefit with as efficient contact as possible
| with as many more other areas as possible. In both
| trivial and deeper ways.
|
| The better developer I am, in terms of development skill,
| tool span, novel problem recognition and solution vision,
| the more often and valuable I find quick AI tutelage on
| other topics, trivial or non-trivial.
|
| If you know a bright high school student highly familiar
| with a domain that you are not, but have reason to think
| that area might be helpful, don't you think instant
| access to talk things over with that high schooler would
| be valuable?
|
| Instant non-trivial answers, perspective and suggestions?
| With your context and motivations taken into account?
|
| Multiplied by a million bright high school students over
| a million domains.
|
| --
|
| We can project the capability vector of these models onto
| one dimension, like "school level idea quality". But
| lower dimension projections are literally shadows of the
| whole.
|
| It if we use them in the direction of their total ability
| vector ( _and given they can iterate, it is actually a
| compounding eigenvector!_ ) and their value goes way
| beyond "a human high schooler with ideas".
|
| It does take time to get the most out of a differently
| calibrated tool.
| CaptainOfCoit wrote:
| > I would be impressed if a sophomore in high school proposed
| it
|
| That sounds good enough for a start, considering you can
| massively parallelize the AI co-scientist workflow, compared
| to the timescale and physical scale it would take to do the
| same thing with human high school sophomores.
|
| And every now and then, you get something exciting and really
| beneficial coming from even inexperienced people, so if you
| can increase the frequency of that, that sounds good too.
| shpongled wrote:
| We don't need an army of high school sophomores, unless
| they are in the lab pipetting. The expensive part of drug
| discovery is _not_ the ideation phase, it is the time and
| labor spent running experiments and synthesizing analogues.
| Teever wrote:
| It sounds like you're suggesting that we need machines
| that mass produce things like automated pipetting
| machines and the robots that glue those sorts of machines
| together.
| 11101010001100 wrote:
| This exists, but does not require AI, so there is no
| hype.
| shpongled wrote:
| They already exist, and we use them. They are not cheap
| though!
| Teever wrote:
| Any idea why they're they so expensive?
| shpongled wrote:
| There is a big range in both automation capabilities and
| prices.
|
| We have a couple automation systems that are semi-custom
| - the robot can handle operation of highly specific, non-
| standard instruments that 99.9% of labs aren't running.
| Systems have to handle very accurate pipetting of small
| volumes (microliters), moving plates to different
| stations, heating, shaking, tracking barcodes, dispensing
| and racking fresh pipette tips, etc. Different
| protocols/experiments and workflows can require vastly
| different setups.
|
| See something like:
|
| [1] https://www.hamiltoncompany.com/automated-liquid-
| handling/pl...
|
| [2] https://www.revvity.com/product/fontus-lh-
| standard-8-96-ruo-...
| dekhn wrote:
| I've built microscopes intended to be installed inside
| workcells similar to what companies like Transcriptic
| built (https://www.transcriptic.com/). So my scope could
| be automated by the workcell automation components (robot
| arms, motors, conveyors, etc).
|
| When I demo'd my scope (which is similar to a 3d printer,
| using low-cost steppers and other hobbyist-grade
| components) the CEO gave me feedback which was very
| educational. They couldn't build a system that used my
| style of components because a failure due to a component
| would bring the whole system down and require an
| expensive service call (along with expensive downtime for
| the user). Instead, their mech engineer would select
| extremely high quality components that had a very low
| probability of failure to minimize service calls and
| other expensive outages.
|
| Unfortunately, the cost curve for reliability not pretty,
| to reduce mechanical failures to close to zero costs
| close to infinity dollars.
|
| One of the reasons Google's book scanning was so scalable
| was their choice to build fairly simple, cheap, easy to
| maintain machines, and then build a lot of them, and
| train the scanning individuals to work with those
| machines quirks. Just like their clusters, they tolerate
| a much higher failure rate and build all sorts of
| engineering solutions where other groups would just buy 1
| expensive device with a service contract.
| kridsdale1 wrote:
| That's similar to how Google won in distributed systems.
| They used cheap PCs in shipping containers when everyone
| else was buying huge expensive SUN etc servers.
| dekhn wrote:
| yes, and that's the reason I went to work at google: to
| get access to their distributed systems and use ML to
| scale up biology. I never was able to join Google
| Research and do the work I wanted (but DeepMind went
| ahead and solved protein structure prediction, so, the
| job got done anyway).
| shpongled wrote:
| They really didn't solve it. AF works great for proteins
| that have a homologous protein with a crystal structure.
| It is absolutely useless for proteins with no published
| structure to use as a template - e.g. many of the
| undrugged cancer targets in existence.
| jiggawatts wrote:
| This sounds like it could be centralised, a bit like the
| clouds in the IT world. A low failure rate of 1-3% is
| comparable to servers in a rack, but if you have
| thousands of them, then this is just a statistic and not
| a servicing issue. Several hyperscalers simply leave
| failed nodes where they are, it's not worth the bother to
| service them!
|
| Maybe the next startup idea is biochemistry as a service,
| centralised to a large lab facility with hundreds of each
| device, maintained by a dedicated team of on-site
| professionals.
| dekhn wrote:
| None of the companies that proposed this concept have
| managed to demonstrate strong marketplace viability. A
| lot of discovery science remains extremely manual,
| artisinal, and vehemently opposed to automation.
| dekhn wrote:
| Replacing a skilled technician is remarkably challenging.
| Often times, when you automate this, you just end up
| wasting a ton of resources rather than accelerating
| discovery. Often, simply integrating devices from several
| vendors (or even one vendor) takes months.
| petra wrote:
| So pharmaceutical research is largely an engineering
| problem, of running experiments and synthesizing
| molecules as fast, cheap and accurate as possible ?
| shpongled wrote:
| I wouldn't say it's an engineering problem. Biology and
| pharmacology are very complex with lots of curveballs,
| and each experiment is often different and not done
| enough to warrant full engineering-scale optimization
| (although this is sometimes the case!).
| kridsdale1 wrote:
| It also seems to be a financial problem of getting VC
| funds to run trials to appease regulators. Even if you've
| already seen results in a lab or other country.
| devmor wrote:
| This is the general problem with nearly all of this era
| of generative AI and why the public dislike it so much.
|
| It is trained on human prose; human prose is primarily a
| representation of ideas; it synthesizes ideas.
|
| There are very few uses for a machine to create ideas. We
| have a wealth of ideas and people enjoy coming up with
| ideas. It's a solution built for a problem that does not
| exist.
| falcor84 wrote:
| As discussed elsewhere, Deepmind are also working on
| extending Alphafold to simulate biochemical pathways and
| then looking to tackle whole-cell simulation. It's not
| quite pipetting, but this sort of AI scientist would
| likely be paired with the simulation environment
| (essentially as function calling), to allow for very
| rapid iteration of in-silico research.
| dekhn wrote:
| (to be Shpongled is to be kippered, mashed, smashed,
| destroyed...completely geschtonkenflopped)
| hinkley wrote:
| I have a less generous recollection of the wisdom of
| sophomores.
| xbmcuser wrote:
| Similar stuff is being done for material sciences where AI
| suggest different combinations to find different properties. So
| when people say AI(machine learning, LLM) are just for show I
| am a bit shocked as AI's today have accelerated discoveries in
| many different fields of science and this is just the start.
| Anna archive probably will play a huge role in this as no human
| or even a group of humans will have all the knowledge of so
| many fields that an Ai will have.
|
| https://www.independent.co.uk/news/science/super-diamond-b26...
| fhd2 wrote:
| It's a matter of perspective and expectations.
|
| The automobile was a useful invention. I don't know if back
| then there was a lot of hype around how it can do anything a
| horse can do, but better. People might have complained about
| how it can't come to you when called, can't traverse stairs,
| or whatever.
|
| It could do _one_ thing a horse could do better: Pull stuff
| on a straight surface. Doing just one thing better is
| evidently valuable.
|
| I think AI is valuable from that perspective, you provide a
| good example there. I might well be disappointed if I would
| expect it to be better than humans at anything humans can do.
| It doesn't have to. But with wording like "co-scientist", I
| see where that comes from.
| bjarlsson wrote:
| What does this cited article have to do with AI? Unless I'm
| missing something the researchers devised a novel method to
| create a material that was known since 1967.
| hirenj wrote:
| I read the cf-PICI paper (abstract) and the hypothesis from the
| AI co-scientist. While the mechanism from the actual paper is
| pretty cool (if I'm understanding it correctly), I'm not
| particularly impressed with the hypothesis from the co-
| scientist.
|
| It's quite a natural next step to take to consider the tails
| and binding partners to them, so much so that it's probably
| what I would have done and I have a background of about 20
| minutes in this particular area. If the co-scientist had
| hypothesised the novel mechanism to start with, then I would be
| impressed at the intelligence of it. I would bet that there
| were enough hints towards these next steps in the discussion
| sections of the referenced papers anyway.
|
| What's a bit suspicious is in the Supplementary Information,
| around where the hypothesis is laid out, it says "In addition,
| our own preliminary data indicate that cf-PICI capsids can
| indeed interact with tails from multiple phage types, providing
| further impetus for this research direction." (Page 35). A bit
| weird that it uses "our own preliminary data".
| TrainedMonkey wrote:
| > A bit weird that it uses "our own preliminary data"
|
| I think potential of LLM based analysis is sky high given the
| amount of concurrent research happening and high context load
| required to understand the papers. However there is a lot of
| pressure to show how amazing AI is and we should be vigilant.
| So, my first thought was - could it be that training data /
| context / RAG having access to a file it should not have
| contaminated the result? This is indirect evidence that maybe
| something was leaked.
| hinkley wrote:
| > in silico discovery
|
| Oh I don't like that. I don't like that at all.
| j_timberlake wrote:
| Don't worry, it takes about 10 years for drugs to get
| approved, AIs will be superintelligent long before the
| government gives you permission to buy a dose of AI-developed
| drugs.
| dekhn wrote:
| So, I've been reading Google research papers for decades now
| and also worked there for a decade and wrote a few papers of my
| own.
|
| When google publishes papers, they tend to juice the results
| significance (google is not the only group that does this, but
| they are pretty egregious). You need to be skilled in the field
| of the paper to be able to pare away the exceptional claims. A
| really good example is https://spectrum.ieee.org/chip-design-
| controversy while I think Google did some interesting work
| there and it's true they included some of the results in their
| chip designs, their comparison claims are definitely over-hyped
| and they did not react well when they got called out on it.
| tsumnia wrote:
| Remember Google is a publicly traded company, so everything
| must be reviewed to "ensure shareholder value". Like dekhn
| said, its impressive, but marketing wants more than
| "impressive".
| dekhn wrote:
| This is true for public universities and private
| universities; you see the same thing happening in academic
| papers (and especially the university PR around the paper)
| hall0ween wrote:
| I would say anecdotal. This hasn't been my case across
| four universities and ten years.
| BeetleB wrote:
| The actual papers don't overhype. But the university PR's
| regarding those papers? They can _really_ overhype the
| results. And of course, the media then takes it up an
| extra order of magnitude.
| dekhn wrote:
| I've definitely seen many examples of papers where the
| conclusions went far beyond what the actual results
| warranted. Scientists are incentivized to claim their
| discovery generalizes as much as possible.
|
| But yes, it's normally: "science paper says an experiment
| in mice shows promising results in cancer treatment" then
| "University PR says a new treatment for cancer is around
| the corner" and "Media says cure for all cancer"
| ein0p wrote:
| That applies to absolutely everyone. Convenient results are
| highlighted, inconvenient are either not mentioned or de-
| emphasized. You do have to be well read in the field to see
| what the authors _aren't_ saying, that's one of the purposes
| of being well-read in the first place. That is also why 100%
| of science reporting is basically disinformation -
| journalists are not equipped with this level of nuanced
| understanding.
| dekhn wrote:
| yes, but google has a long history of being egregious, with
| the additional detail that their work is often
| irreproducible for technical reasons (rather than being
| irreproducible for missing methods). For example, we
| published an excellent paper but nobody could reproduce it
| because at the time, nobody else had a million spare cores
| to run MD simulations of proteins.
| ein0p wrote:
| It's hardly Google's problem that nobody else has a
| million cores, wouldn't you agree? Should they not
| publish the result at all if it's using more than a
| handful of cores so that anyone in academia can reproduce
| it? That'd be rather limiting.
| dekhn wrote:
| Well, a goal of most science is to be reproducible, and
| it couldn't be reproduced, merely for technical reasons
| (and so we shared as much data from the runs as possible
| so people could verify our results). This sort of thing
| comes up when CERN is the only place that can run an
| experiment and nobody can verify it.
| Workaccount2 wrote:
| Does this qualify as an answer to Dwarkesh's question?[1][2]
|
| [1]https://marginalrevolution.com/marginalrevolution/2025/02/dw
| ... [2]https://x.com/dwarkesh_sp/status/1888164523984470055
|
| I don't know his @ but I'm sure he is on here somewhere
| ACV001 wrote:
| Just as the invention of writing degraded human memory (before
| that they memorized whole stories, poems), with the advent of AI,
| humans will degrade their thinking skills and knowledge in
| general.
| azinman2 wrote:
| It seems in general we're heading toward's Minsky's society of
| minds concept. I know OpenAI is wanting to collapse all their
| models into a single omni model that can do it all, but I wonder
| if under the hood it'd just be about routing. It'd make sense to
| me for agents to specialize in certain tool calls, ways of
| thinking, etc that as a conceptual framework/scaffolding provides
| a useful direction.
| yjftsjthsd-h wrote:
| Isn't that kinda the idea of Mixture of Experts?
| mythrwy wrote:
| I wonder if OpenAI might be routing already based on speed of
| some "O1" responses I receive. It does make sense.
| willy_k wrote:
| Also, for some more complex questions I've noticed that it
| doesn't expose its reasoning. Specifically, yesterday I asked
| it to perform a search algorithm provided a picture of a
| grid, and it reasoned for 1-2 minutes but didn't show any of
| it (neither in real time nor afterwords), whereas for simpler
| questions I've asked it the reasoning is provided as well.
| Not sure what this means, but it suggests some type of
| different treatment based on complexity.
| ThouYS wrote:
| I guess we do live in the fast take off world
| celltalk wrote:
| "Drug repurposing for AML" lol
|
| As a person who is literally doing his PhD on AML by implementing
| molecular subtyping, and ex-vivo drug predictions. I find this
| super random.
|
| I would truly suggest our pipeline instead of random drug
| repurposing :)
|
| https://celvox.co/solutions/seAMLess
|
| edit: Btw we're looking for ways to fund/commercialize our
| pipeline. You could contact us through the site if you're
| interested!
| heyoni wrote:
| Can you explain what you mean by subtyping and if/how it
| negates the usefulness of repurposing (if that's what you meant
| to say). Wouldn't subtyping complement a drug repurposing
| screen by allowing the scientist to test compounds against a
| subset of a disease?
|
| And drug repurposing is also used for conditions with no known
| molecular basis like autism. You're not suggesting its
| usefulness is limited in those cases right?
| celltalk wrote:
| Sure. There are studies like BEAT-AML which tests selected
| drugs' responses on primary AML material. So, not on a cell-
| line but on true patient data. Combining this information
| with molecular measurements, you can actually say something
| about which drugs would be useful for a subset of the
| patients.
|
| However, this is still not how you treat a patient. There are
| standard practices in the clinic. Usually the first line
| treatment is induction chemo with hypomethylating agents
| (except elderly who might not be eligible for such a
| treatment). Otherwise the options are still very limited, the
| "best" drug in the field so far is a drug called Venetoclax,
| but more things are coming up such as immuno-therapy etc.
| It's a very complex domain, so drug repurposing on an AML
| cell line is not a wow moment for me.
| ttpphd wrote:
| It's almost like scientists are doing something more than a
| random search over language.
| celltalk wrote:
| I do hallucinate a better future as well.
| coherentpony wrote:
| It bothers me that the word 'hallucinate' is used to
| describe when the output of a machine learning model is
| wrong.
|
| In other fields, when models are wrong, the discussion is
| around 'errors'. How large the errors are, their structural
| nature, possible bounds, and so forth. But when it's AI
| it's a 'hallucination'. Almost as if the thing is feeling a
| bit poorly and just needs to rest and take some fever-
| reducer before being correct again.
|
| It bothers me. Probably more than it should, but it does.
| pertymcpert wrote:
| I think hallucinate is a good term because when an AI
| completely makes up facts or APIs etc it doesn't do so as
| a minor mistake of an otherwise correct reasoning step.
| throwawaymaths wrote:
| its more like conspiracy theory. when you're picking a
| token youre kinda like putting a gun to the LLM's head
| and demanding, "what you got next?"
| waynenilsen wrote:
| it seems that humans may become the hands of the AI before the
| robots are ready
|
| mechanical turk, but for biology
| akomtu wrote:
| That's the Quake version of the machine civilization: machines
| make the decisions, but use chunks of humans to improve their
| unholy machinery. The alternative Doom version is the opposite:
| humans make the decisions, but they are blended in an unholy
| way into the machines.
| tsumnia wrote:
| Now do the Warhammer 40k version :D
| quinnjh wrote:
| The market seems excited to charge in whatever direction the
| weathervane has last been pointing, regardless of the real
| outcomes of running in that direction. Hopefully I'm wrong, but
| it reminds me very much of this study (I'll quote a paraphrase)
|
| "A groundbreaking new study of over 1,000 scientists at a major
| U.S. materials science firm reveals a disturbing paradox: When
| paired with AI systems, top researchers become extraordinarily
| more productive - and extraordinarily less satisfied with their
| work. The numbers tell a stark story: AI assistance helped
| scientists discover 44% more materials and increased patent
| filings by 39%. But here's the twist: 82% of these same
| scientists reported feeling less fulfilled in their jobs."
|
| Quote from https://futureofbeinghuman.com/p/is-ai-poised-to-suck-
| the-so...
|
| Referencing this study
| https://aidantr.github.io/files/AI_innovation.pdf
| yodon wrote:
| As a dev, I have the same experience.
|
| AI chat is a massive productivity enhancer, but, when coding
| via prompts, I'm not able to hit the super satisfying developer
| flow state that I get into via normal coding.
|
| Copilot is less of a productivity boost, but also less of a
| flow state blocker.
| sanderjd wrote:
| Yep! I think these tools are incredibly useful, but I think
| they're basically changing all our jobs to be more like what
| product managers do, having ideas for what we want to
| achieve, but farming out a significant chunk of the work
| rather than doing it ourselves. And that's fine, I find it
| very hard to argue that it's a bad thing. But there's a
| reason that we aren't all product managers already.
| Programming is fun, and I do experience it as a loss to find
| myself doing less of it myself.
| pradn wrote:
| There is some queasy feeling of fake-ness when auto-
| completing so much code. It feels like you're doing something
| wrong. But these are all based on my experience coding for
| half my life. AI-native devs will probably feel differently.
| radioactivist wrote:
| I'm a bit skeptical of this study given how it is unpublished,
| from a (fairly junior) single author and all of the underlying
| details of the subject are redacted. Is there any information
| anywhere about what this company in the study was actually
| doing? (the description in the article are very vague --
| basically something to do with materials)
| BeetleB wrote:
| The feeling of dissatisfaction is something I can relate to. My
| story:
|
| I only recently started using aider[1].
|
| My experience with it can be described in 3 words.
|
| Wow!
|
| Oh wow!
|
| It was amazing. I was writing a throwaway script for one time
| use (not for work). It wrote it for me in under 15 minutes
| (this includes my time getting familiar with the tool!) No
| bugs.
|
| So I decided to see how far I could take it. I added command
| line arguments, logging, and a whole bunch of other things.
| After a full hour, I had a production ready script - complete
| with logs, etc. I had to debug code only once.
|
| I may write high quality code for work, but for personal
| throwaway scripts, I'm sloppy. I would not put a command line
| parser, nor any logging. This did it all for me for very cheap!
|
| There's no going back. For simple scripts like this, I will
| definitely use aider.
|
| And yeah, there was definitely no satisfaction one would derive
| from coding. It was truly _addictive_. I _want_ to use it more
| and more. And no matter how much I use it and like the results,
| it doesn 't scratch my programmer's itch. It's nowhere near the
| fun/satisfaction of SW development.
|
| [1] https://aider.chat/
| m3kw9 wrote:
| I really would like to see a genuine breakthru amongst all this
| talk about AI doing that
| stanford_labrat wrote:
| So I'm a biomedical scientist (in training I suppose...I'm in my
| 3rd year of a Genetics PhD) and I have seen this trend a couple
| of times now where AI developers tout that AI will accelerate
| biomedical discovery through a very specific argument that AI
| will be smarter and generate better hypotheses than humans.
|
| For example in this Google essay they make the claim that CRISPR
| was a transdisciplinary endeavor, "which combined expertise
| ranging from microbiology to genetics to molecular biology" and
| this is the basis of their argument that an AI co-scientist will
| be better able to integrate multiple fields at once to generate
| novel and better hypothesis. For one, what they fail to
| understand as computer scientists (I suspect due to not being
| intimately familiar with biomedical research) is that
| microbio/genetics/mol bio are closer linked than you may expect
| as a lay person. There is no large leap between microbiology and
| genetics that would slow down someone like Doudna or even myself
| - I use techniques from multiple domains in my daily work. These
| all fall under the general broad domain of what I'll call
| "cellular/micro biology". As another example, Dario Amodei from
| Claude also wrote something similar in his essay Machines of
| Loving Grace that the limiting factor in biomedical is a lack of
| "talented, creative researchers" for which AI could fill the
| gap[1].
|
| The problem with both of these ideas is that they misunderstand
| the rate-limiting factor in biomedical research. Which to them is
| a lack of good ideas. And this is very much not the case.
| Biologists have tons of good ideas. The rate limiting step is
| testing all these good ideas with sufficient rigor to either
| continue exploring that particular hypothesis or whether to
| abandon the project for something else. From my own work, the
| hypothesis driving my thesis I came up with over the course of a
| month or two. The actual amount of work prescribed by my thesis
| committee to fully explore whether or not it was correct? 3 years
| or so worth of work. Good ideas are cheap in this field.
|
| Overall I think these views stem from field specific nuances that
| don't necessarily translate. I'm not a computer scientist, but I
| imagine that in computer science the rate limiting factor is not
| actually testing out hypothesis but generating good ones. It's
| not like the code you write will take multiple months to run
| before you get an answer to your question (maybe it will? I'm not
| educated enough about this to make a hard claim. In biology, it
| is very common for one experiment to take multiple months before
| you know the answer to your question or even if the experiment
| failed and you have to do it again). But happy to hear from a CS
| PhD or researcher about this.
|
| All this being said I am a big fan of AI. I try and use ChatGPT
| all the time, I ask it research questions, ask it to search the
| literature and summarize findings, etc. I even used it literally
| yesterday to make a deep dive into a somewhat unfamiliar branch
| of developmental biology more easy (and I was very satisfied with
| the result). But for scientific design, hypothesis generation? At
| the moment, useless. AI and other LLMs at this point are a very
| powerful version of google and code writer. And it's not even
| correct 30% of the time to boot so you have to be extremely
| careful when using it. I do think that wasting less time
| exploring hypotheses that are incorrect or bad is a good thing.
| But the problem here is that we can pretty easily identify good
| and bad hypotheses already. We don't need AI for that, what takes
| time is the actual amount of testing of these hypotheses that
| slows down research. Oh and politics, which I doubt AI can magic
| away for us.
|
| [1] https://darioamodei.com/machines-of-loving-
| grace#1-biology-a...
| colingauvin wrote:
| It's pretty painful watching CS try to turn biology into an
| engineering problem.
|
| It's generally very easy to marginally move the needle in drug
| discovery. It's very hard to move the needle enough to justify
| the cost.
|
| What is challenging is culling ideas, and having enough SNR in
| your readouts to really trust them.
| bjarlsson wrote:
| This is marketing material from Google and people are accepting
| the premises uncritically.
| anothermathbozo wrote:
| Almost this entire thread is criticism
| writeslowly wrote:
| I recently ran across this toaster-in-dishwasher article [1]
| again and was disappointed that the LLMs I have access to could
| replicate the "hairdryer-in-aquarium" breakthrough (or the
| toaster-in-dishwasher scenario, although I haven't explored it as
| much), which has made me a bit skeptical of the ability of LLMs
| to do novel research. Maybe the new OpenAI research AI is smart
| enough to figure it out?
|
| [1] https://jdstillwater.blogspot.com/2012/05/i-put-toaster-
| in-d...
| BriggyDwiggs42 wrote:
| Do you mean they sided with the incorrect common wisdom all the
| people in the article were using?
| hinkley wrote:
| I am generally down on AI these days but I still remember using
| Eliza for the first time.
|
| I think I could accept an AI prompting _me_ instead of the other
| way around. Something to ask you a checklist of problems and how
| you will address them.
|
| I'd also love to have someone apply AI techniques to property
| based testing. The process of narrowing down from 2^32 inputs to
| six interesting ones works better if it's faster.
| insane_dreamer wrote:
| Seems like the primary value-add is to speed up the literature
| review process during the hypothesis formulation process.
___________________________________________________________________
(page generated 2025-02-19 23:00 UTC)