[HN Gopher] How to inject knowledge efficiently? Knowledge infus...
___________________________________________________________________
How to inject knowledge efficiently? Knowledge infusion scaling law
for LLMs
Author : PaulHoule
Score : 52 points
Date : 2025-10-04 17:18 UTC (5 hours ago)
(HTM) web link (arxiv.org)
(TXT) w3m dump (arxiv.org)
| gdiamos wrote:
| I wonder if this depends on what is inside the domain specific
| data.
|
| I'm happy to see ML papers on hacker news.
| adsharma wrote:
| I wish the authors calculated a plot of model size (number of
| params) vs number of triples it can hold before the memory
| collapse happens.
|
| It's hard to map the frequency of knowledge injection to a real
| world understanding of "how much knowledge" can a 4B param model
| hold?
| bconsta wrote:
| There is a study that gives a rule of thumb of ~2 bits per
| param for a model's memorization capacity:
| https://arxiv.org/abs/2404.05405
| dart_pink wrote:
| Seems they have replicated Gardner's work, without mentioning
| it, "Maximum Storage Capacity in Neural Networks" (1987),
| which established that the storage capacity of a neural
| network is about 2N (2 bits per parameter)
| daft_pink wrote:
| I'm really curious how much it costs to inject information like
| this into an LLM as people say training an LLM is very expensive,
| so if you want a domain specific LLM, how much does the
| additional training cost to get this?
| simonw wrote:
| It sounds like you're talking about fine-tuning an existing
| model. That's not what this paper did - they studied the effect
| of training small models entirely from scratch with varying
| amounts of domain knowledge.
|
| I still haven't seen strong evidence that fine-tuning to add
| extra knowledge is effective, but I'd be delighted to learn
| otherwise.
| hollerith wrote:
| Are there _any_ effective ways to add extra knowledge to an
| LLM, ways that are more than just demos or proofs of concept?
|
| For example, could there be a site like HN with ten thousand
| contributors where the contributions are changes to an LLM
| rather than posts and comments?
|
| One issue is that if contribution A contradicts contribution
| B, then on HN the contradiction presents no problem (i.e.,
| two HN comments can and often do contradict each other just
| fine) whereas AFAICT the LLM will need to resolve the
| contradiction somehow to give coherent answers on the subject
| matter of the contributions A and B. Then again I suppose the
| LLM's answer could take the form, "opinions on la la vary,
| with some maintaining that la la la whereas others claim that
| la la la".
| simonw wrote:
| This is a solved problem. The answer is to add extra
| relevant information to the context as part of answering
| the user's prompt.
|
| This is sometimes called RAG, for Retrieval Augmented
| Generation.
|
| These days the most convincing way to do this is via tool
| calls.
|
| Provide your LLM harness with a tool for running searches,
| and tell it to use that tool any time it needs additional
| information.
|
| A good "reasoning" LLM like GPT-5 or Claude 4 can even
| handle contradictory pieces of information - they can run
| additional searches if they get back confusing results and
| work towards a resolution, or present "both sides" to the
| user if they were unable to figure it out themselves.
| mtokarski wrote:
| Interesting work, but I think the interpretation may be a bit
| overstated. The authors claim that injecting too much factual
| "knowledge" during pretraining causes models to collapse --
| performance drops below the baseline once knowledge frequency
| crosses a threshold.
|
| The problem is how they inject it. Their "knowledge" isn't
| natural language; it's templated Wikidata triples like "X is the
| capital of Y." That's a super low-entropy, highly repetitive
| distribution. When you cram enough of that into a fixed token
| budget, you're not really teaching the model more facts -- you're
| just destroying linguistic diversity and skewing the token
| statistics.
|
| In real pretraining or domain adaptation scenarios, "knowledge"
| tends to appear in richer, more varied contexts. The practical
| takeaway isn't "don't add too much domain data," but rather
| "don't overrepresent any single format or narrow syntactic
| pattern" The issue seems more about representation homogeneity
| than about factual density itself.
___________________________________________________________________
(page generated 2025-10-04 23:00 UTC)