[HN Gopher] Karpathy on DeepSeek-OCR paper: Are pixels better in...
___________________________________________________________________
Karpathy on DeepSeek-OCR paper: Are pixels better inputs to LLMs
than text?
Author : JnBrymn
Score : 59 points
Date : 2025-10-21 17:43 UTC (1 days ago)
(HTM) web link (twitter.com)
(TXT) w3m dump (twitter.com)
| yunwal wrote:
| > The more interesting part for me (esp as a computer vision at
| heart who is temporarily masquerading as a natural language
| person) is whether pixels are better inputs to LLMs than text.
| Whether text tokens are wasteful and just terrible, at the input.
|
| > Maybe it makes more sense that all inputs to LLMs should only
| ever be images.
|
| So, what, every time I want to ask an LLM a question I paint a
| picture? I mean at that point why not just say "all input to LLMs
| should be embeddings"?
| smegma2 wrote:
| No? He's talking about rendered text
| rhdunn wrote:
| From the post he's referring to text input as well:
|
| > Maybe it makes more sense that all inputs to LLMs should
| only ever be images. _Even if you happen to have pure text
| input, maybe you 'd prefer to render it and then feed that
| in:_
|
| Italicized emphasis mine.
|
| So he's suggesting that/wondering if the vision model should
| be the only input to the LLM and have that read the text. So
| there would be a rasterization step on the text input to
| generate the image.
|
| Thus, you don't need to draw a picture but generate a raster
| of the text to feed it to the vision model.
| fspeech wrote:
| If you can read your input on your screen your computer
| apparently knows how to convert your texts to images.
| CuriouslyC wrote:
| All inputs being embeddings can work if you have embedding like
| Matryoshka, the hard part is adaptively selecting the embedding
| size for a given datum.
| dang wrote:
| Recent and related:
|
| _Getting DeepSeek-OCR working on an Nvidia Spark via brute force
| with Claude Code_ - https://news.ycombinator.com/item?id=45646559
| - Oct 2025 (43 comments)
|
| _DeepSeek OCR_ - https://news.ycombinator.com/item?id=45640594 -
| Oct 2025 (238 comments)
| sabareesh wrote:
| It might be that our current tokenization is inefficient compared
| to how well image pipeline does. Language already does lot of
| compression but there might be even better way to represent it in
| latent space
| ACCount37 wrote:
| People in the industry know that tokenizers suck and there's
| room to do better. But actually doing it better? At scale? Now
| that's hard.
| typpilol wrote:
| It will require like 20x the compute
| CuriouslyC wrote:
| Image models use "larger" tokens. You can get this effect with
| text tokens if you use a larger token dictionary and generate
| common n-gram tokens, but the current LLM architecture isn't
| friendly to large output distributions.
| hbarka wrote:
| Chinese writing is logographic. Could this be giving Chinese
| developers a better intuition for pixels as input rather than
| text?
| varispeed wrote:
| Text is linear, whereas image is parallel. I mean when people
| often read they don't scan text from left to right (or different
| direction, depending on language), but rather read the text all
| at once or non-linearly. Like first lock on keywords and then
| read adjacent words to get meaning, often even skipping some
| filler sentences unconsciously.
|
| Sequential reading of text is very inefficient.
___________________________________________________________________
(page generated 2025-10-22 23:00 UTC)