[HN Gopher] Ex-Meta scientists debut gigantic AI protein design ...
___________________________________________________________________
Ex-Meta scientists debut gigantic AI protein design model
Author : gmays
Score : 85 points
Date : 2024-07-12 17:22 UTC (5 hours ago)
(HTM) web link (www.nature.com)
(TXT) w3m dump (www.nature.com)
| trhway wrote:
| Tangential - the laws of nature discovered by our brain usually
| involve just few quantities, like f=ma. On one side it is a great
| ability of our brain for analytical reduction, on the other side
| it is just an inability to deal with complex multiparameter
| phenomena without such a reduction. I wonder if pumping more and
| more data into the NNs we'd be able to distill emerging
| multiparameter correlations which happen to be new laws of nature
| irreducible to more simpler ones.
| spywaregorilla wrote:
| Eh, not in this context at least. Nature is a big term. Physics
| just "is". This makes it a good place for simple mechanics to
| appear and be derived. You can argue around how universal that
| is all you want but proteins themselves are the result of an
| enormously large search of a hugely high dimensional space. It
| doesn't seem possible for them to ever be reduced.
| ben_w wrote:
| I disagree, our brains almost always deal with complex
| multiparameter phenomena.
|
| The part of us that can't cope with that is our System 2
| reasoning: our minds' rules based thinking is great at F=ma but
| cannot enumerate the rules necessary at the level of retina
| cell activation patterns to recognise a tiger; our System 1
| can, and always does, we just call it "intuition" or "common
| sense" or "a gut feeling".
|
| https://en.wikipedia.org/wiki/Dual_process_theory
| btbuildem wrote:
| I wonder if this will ever come full circle (or.. spiral) and the
| AI tools we've created will in turn lead the way to discovering /
| inventing new proteins / cells / life forms that eventually
| outsmart and outcompete us.
| szundi wrote:
| Isn't the Terminator good enough?
| warkdarrior wrote:
| As usual, life is stranger than fiction.
| Spacecosmonaut wrote:
| Very nice work. We need brighter fluorescent protein tags that
| are more compact, in particular in the far red spectrum. The size
| of current fluorescent protein coding DNA sequences is out of
| reach of prime editing and still relies on less efficient gene
| editing technology.
| alan-hn wrote:
| But they aren't brighter...
| changoplatanero wrote:
| its not more compact either, right?
| yeutterg wrote:
| Just want to congratulate Tom Hayes on the big launch!
| thebeardisred wrote:
| > For a smaller open-source version, certain sequences, such as
| those from viruses and a US government list of worrying pathogens
| and toxins, were excluded from training. Neither can ESM3-open --
| which scientists anywhere can download and run independently --
| be prompted to generate such proteins.
|
| That sounds like a glove being thrown down.
| trott wrote:
| > matching less than 60% of the sequence of the most closely
| related fluorescent protein
|
| > When the researchers made around 100 of the resulting designs,
| several were as bright as natural GFPs, which are still vastly
| dimmer than lab-engineered variants.
|
| So they didn't come up with better functionality, unlike what
| some commentators imply. They basically introduced a bunch of
| mutations while preserving the overall function.
|
| Relevant: https://en.wikipedia.org/wiki/Conservative_replacement
| civilized wrote:
| If the mutations were non-synonymous, resulting in different
| amino acids, the fact that they keep the natural function is
| still kinda cool. Very much a pure research result AFAICT, but
| worth a little something.
| flobosg wrote:
| This is already a well known fact: protein structure (and
| consequently function) is much more conserved than sequence,
| mostly due to biophysical constraints.
| civilized wrote:
| Forgive my ignorance - is that true even for non-synonymous
| mutations? They usually have a hard time disrupting the big
| structure?
| flobosg wrote:
| If non-synonymous mutations do not change the biophysical
| features of the amino acid residues, then the structure
| is usually kept. Alternatively, it can be the case that a
| disruptive mutation is compensated by another one that
| keeps the structure/function/phenotype. This is the basis
| for evolutionary coupling based structure prediction
| methods, such as Alphafold.
| throwup238 wrote:
| That's still a pretty significant result. Imagine how much more
| effective directed evolution could be if it weren't driven by
| random mutation, or if the random mutations were applied on top
| of already working variants.
| 0cf8612b2e1e wrote:
| That feels like deliberately cute wording. What was the
| similarity vs the starting molecule? I could probably hand pick
| a few point mutations substituting one small hydrophobic AA for
| another without impacting function.
| flobosg wrote:
| You'd be surprised by how many failures you get by such an
| approach. Hydrophobic cores can be easily disrupted by
| altering some side chain volumes.
| flobosg wrote:
| > However, its amino-acid sequence is vastly different, matching
| less than 60% of the sequence of the most closely related
| fluorescent protein in its training data set.
|
| Not to downplay this achievement, but 60% sequence identity is
| nowhere near "vastly different".
| trott wrote:
| The twilight zone is 20-35%:
| https://pubmed.ncbi.nlm.nih.gov/10195279/ (Incidentally, the
| author was on my thesis committee, but this isn't precisely my
| field of expertise.)
| flobosg wrote:
| Proteins from the same family (and thus the same fold) can
| share less than 10% identity and still keep the same
| functionality as shown by e.g. profile Hidden Markov Model
| comparisons.
| throwaway24124 wrote:
| Are there any good resources for understanding models like this?
| Specifically a "protein language model". I have a basic grasp on
| how LLMs tokenize and encode natural language, but what does a
| protein language actually look like? An LLM can produce results
| that look correct but are actually incorrect, how are proteins
| produced by this model validated? Are the outputs run through
| some other software to determine whether the proteins are valid?
| cottonseed wrote:
| Proteins are linear molecules consisting of sequences of
| (mostly) 20 amino acids. You can see the list of amino acids
| here: https://en.wikipedia.org/wiki/Amino_acid#Table_of_standar
| d_a.... There is a standard encoding of amino acids using
| single letters, A for alanine, etc. Earlier versions of ESM (I
| haven't read the ESM3 paper yet) uses one token per amino acid,
| plus a few control tokens (beginning of sequence, end of
| sequence, class token, mask, etc.) Earlier versions of ESM were
| BERT-style models focused on understanding, not GPT-style
| generative models.
| hn_throwaway_99 wrote:
| Agreed, would be interested if someone with more knowledge
| could comment.
|
| My layman's understanding of LLMs is that they are essentially
| "fancy autocomplete". That is, you take a whole corpus of text,
| then train the model to determine the statistical relationships
| between those words (more accurately, tokens), so that given a
| list of tokens of length N, the LLM will find the next most
| likely token for N + 1, and then to generate whole
| sentences/paragraphs, you just recursively repeat this process.
|
| I certainly understand encoding proteins as just a linear
| sequence of tokens representing their amino acids, but how does
| that then map to a human-language description of the function
| of those proteins?
| changoplatanero wrote:
| Most protein language models are not able to understand
| human-language descriptions of proteins. Mostly they just
| predict the next amino acid in a sequence and sometimes they
| can understand certain structured metadata tags.
| changoplatanero wrote:
| The way you would use a protein language model is different
| from how you would use a regular LLM like chatgpt. Normally,
| you aren't looking for one correct answer to your query but
| rather you would like thousands of ideas to try out in the lab.
| Biologists have techniques for trying out thousands or tens of
| thousands of proteins in a lab and filtering it down to a
| single candidate thats the best solution to whatever they are
| trying to achieve.
| brcmthrowaway wrote:
| Can protein models help cure prion diseases?
| Exoristos wrote:
| At present this is unclear. But on the other hand, think of all
| the potential new prion diseases!
___________________________________________________________________
(page generated 2024-07-12 23:01 UTC)