[HN Gopher] Probably pay attention to tokenizers
       ___________________________________________________________________
        
       Probably pay attention to tokenizers
        
       Author : ingve
       Score  : 148 points
       Date   : 2024-10-23 10:29 UTC (12 hours ago)
        
 (HTM) web link (cybernetist.com)
 (TXT) w3m dump (cybernetist.com)
        
       | Spivak wrote:
       | I think I take something different away from the article, yes
       | tokenizers are important but they're a means to get at something
       | much much bigger which is how to clean up and normalize
       | unstructured data. It's a current endeavor of mine at $dayjob for
       | how to do this in a way that can work reasonably well even for
       | badly mangled documents. I don't have any silver bullets, at
       | least nothing worthy of a blog-post yet, but since this is needed
       | when dealing with OCR documents so "post-ocr correction" turns up
       | quite a few different approaches.
       | 
       | And this is an aside, but I see folks using LLMs to do this
       | correction in the first place. I don't think using LLMs to do
       | correction in a multi-pass system is inherently bad but I haven't
       | been able to get good results out of "call/response" (i.e. a
       | prompt to clean up this text). The best results are when you're
       | running an LLM locally and cleaning incrementally by using token
       | probabilities to help guide you. You get some candidate words
       | from your wordlist based on the fuzzy match of the text you do
       | have, and candidate words predicted from the previous text and
       | when both align -- ding! It's (obviously) not the fastest method
       | however.
        
         | SEGyges wrote:
         | you might have better luck giving the LM the original document
         | and having it generate its own OCR independently, then asking
         | the llm to tiebreak between its own generation and the OCR
         | output while the image is still in the context window until it
         | is satisfied that it got things correct
        
         | 7thpower wrote:
         | This is interesting. What types of content are you using this
         | approach on and how does it handle semi structured data? For
         | instance, embedded tables.
        
       | kelseyfrog wrote:
       | Tokenizers aren't considered the "sexy" part of LLMs, but where
       | others see boring, I see opportunity. Papers like xVal[1], point
       | toward specialization strategies in tokenization. Spelling and
       | letter tasks are another problem that could benefit from
       | innovation on the tokenization.
       | 
       | LLMs are notoriously bad at counting letters in words or
       | performing simply oulipos of letter omission. GPT-4o, for
       | example, writes a small python program and executes it in order
       | to count letter instances. We all know that tokenization
       | effectively erases knowledge about letters in prompts and
       | directly negatively impacts performance at these tasks, yet we
       | haven't found a way to solve it.
       | 
       | 1. https://ar5iv.labs.arxiv.org/html/2310.02989
        
         | IncreasePosts wrote:
         | What's the issue with character-level tokenization(I assume
         | this would be much better at count-the-letter tasks)? The
         | article mentions it as an option but doesn't talk about why
         | subword tokenization is preferred by most of the big LLMs out
         | there.
        
           | SEGyges wrote:
           | tokens are on average four characters and the number of
           | residual streams (and therefore RAM) the LLM allocates to a
           | given sequence is proportionate to the number of units of
           | input. the flops is proportionate to their _square_ in the
           | attention calculation.
           | 
           | you can hypothetically try to ameliorate this by other means,
           | but if you just naively drop from tokenization to character
           | or byte level models this is what goes wrong
        
           | Centigonal wrote:
           | I think it has to do with both performance (smaller tokens
           | means more tokens per sentence read and more runs per
           | sentence generated) and with how embeddings work. You need a
           | token for "dog" and a token for "puppy" to represent the
           | relationship between the two as a dimension in latent space.
        
           | stephantul wrote:
           | Using subwords makes your sequences shorter, which makes them
           | cost less.
           | 
           | Besides that, for alphabetic languages, there exists almost
           | no relation between form and meaning. I.e.: "ring" and "wing"
           | differ by one letter but have no real common meaning. By
           | picking the character or byte as your choice of
           | representation, the model basically has to learn to
           | distinguish ring and wing in context. This is a lot of work!
           | 
           | So, while working on the character or byte level saves you
           | some embeddings and thus makes your model smaller, it puts
           | all of the work of distinguishing similar sequences with
           | divergent meanings on the model itself, which means you need
           | a larger model.
           | 
           | By having subwords, a part of this distinguishing work
           | already has been done by the vocabulary itself. As the
           | article points out, this sometimes fails.
        
             | bunderbunder wrote:
             | I suspect that the holy grail here is figuring out how to
             | break the input into a sequence of morphemes and non-
             | morpheme lexical units.
        
             | p1esk wrote:
             | Has anyone tried to combine a token embedding with some
             | representation of the characters in the (sub)word? For
             | example, use a 512 long vector to represent a token, and
             | reserve the last 12 values to spell out the word.
        
               | RicoElectrico wrote:
               | Well, fastText uses character n-grams to compute
               | embeddings for out-of-vocabulary words. This is pre-
               | transformers work BTW.
        
               | mattnewton wrote:
               | I'm not following - spell out the word how? Like put the
               | actual bytes as numerical input to the transformer layer?
        
             | sundarurfriend wrote:
             | > Besides that, for alphabetic languages, there exists
             | almost no relation between form and meaning.
             | 
             | Also true for Abugida-based languages, for eg. crm (saram =
             | string) vs mrm (maram = tree), and many more. I think your
             | intention with specifying "alphabetic languages" was to say
             | "non-logographic languages", right?
        
               | bunderbunder wrote:
               | I'll do you one more and say "non-Chinese languages".
               | Written Japanese - including the kanji portion of the
               | script - has the same characteristic.
               | 
               | And even in Chinese it's a fairly weak relationship. A
               | large portion of the meanings of individual characters
               | come from sound loan. For example the Ying  in Ying Xiong
               | means "hero", in Ying Yu  means "England", an in Jing
               | Ying  means "flower". The relationship there is simple
               | homophony.
               | 
               | On the other hand, one thing you do get with written
               | Chinese is that "1 character = 1 morpheme" very nearly
               | works. So mechanistically breaking a text into a sequence
               | of morphemes can be done pretty reliably without the aid
               | of a semantic model or exhaustive hard-coded mapping. I
               | think that for many other languages you can't even get
               | close using only syntactic analysis.
        
           | cma wrote:
           | Context length performance and memory scales N^2. Smaller
           | tokens mean worse scaling, up to a point.
        
         | kaycebasques wrote:
         | > but where others see boring, I see opportunity
         | 
         | I feel this way about embeddings
         | 
         | This line of thought seems related to the old wisdom of finding
         | innovative solutions by mucking around in the layer below
         | whatever the "tools of the trade" are for your domain
        
         | screye wrote:
         | Tokenizers face an odd compute issue.
         | 
         | Since they're part of the pre-processing pipeline, you can't
         | quickly test them out for effectiveness. You have to restart a
         | pretraining run to test downstream effectiveness.
         | 
         | Separately,
         | 
         | As much as an attention module can do universal nonlinear
         | transformations....I wonder if it makes sense to add specifuc
         | modules for some math primitives as well. I remember that the
         | executor paper [1] (slightly precursor to the attention is
         | allyou need paper) created self contained modules for
         | operations like less than, count, sum and then explicitly
         | orchestrated them in the decoder.
         | 
         | I'm surprised we haven't seen such solutions produce sota
         | results from math-ai or code-ai research communities.
         | 
         | [1] https://arxiv.org/abs/1705.03633
        
         | doctorpangloss wrote:
         | > LLMs are notoriously bad at counting letters in words or
         | performing simply oulipos of letter omission.
         | 
         | If it were so simple, why hasn't this already been dealt with?
         | 
         | Multimodal VQA models also have had a hard time generalizing
         | counting. Counting is not as simple as changing the tokenizer.
        
           | kelseyfrog wrote:
           | I'm saying the oulipo rule is simple, not the task given
           | current tokenization methods
        
         | Der_Einzige wrote:
         | I wrote a whole paper about this exact topic! (Syntactic,
         | phonetic, and related constraints)
         | 
         | https://aclanthology.org/2022.cai-1.2/
        
         | bunderbunder wrote:
         | This was ages ago, in the pre-transformer era, and I can't find
         | the link anymore. But once upon a time I read a great paper
         | that demonstrated that most of the performance differences
         | being reported among popular embedding models of the time were
         | better explained by text cleaning and tokenization than they
         | were by the embedding model itself.
         | 
         | In other words, if you train a model using word2vec's
         | preprocessing and GloVe's algorithm, the result looks more like
         | a "standard-issue" word2vec model than a "standard-issue" GloVe
         | model.
        
       | Joker_vD wrote:
       | > You need to understand [the input data] before you can do
       | anything meaningful with it.
       | 
       | IMHO that's the main reason people turn to any sort of automated
       | data-processing tools in the first place: they don't want to look
       | at the input data. They'd rather have "the computer" look at it
       | and maybe query them back with some additional info gathering
       | requests. But thinking on their own? Ugh.
       | 
       | So I boldly propose the new definition of AGI: it's the data-
       | processing entity that will (at last!) reliably liberate you from
       | having to look at your data before you start shoving this data
       | into that processing entity.
        
         | bunderbunder wrote:
         | Over the past year I've encountered so many situations where a
         | person's opinion of how well an LLM accomplishes a task
         | actually says more about that person's reading comprehension
         | skills than it does the LLM's performance. This applies to both
         | positive and negative opinions.
        
       | ratedgene wrote:
       | Can't someone expand on this
       | 
       | > Chunking is more or less a fixable problem with some clever
       | techniques: these are pretty well documented around the internet;
       | 
       | Curious about what chunking solutions are out there for different
       | sets of data/problems
        
         | pphysch wrote:
         | Most data has semantic boundaries: whether tokens, words,
         | lines, paragraphs, blocks, sections, articles, chapters,
         | versions, etc. and ideally the chunking algorithm will align
         | with those boundaries in the actual data. But there is a lot of
         | variety.
        
         | hansvm wrote:
         | It's only "solved" if you're okay with a 50-90% retrieval rate
         | or have particularly nice data. There's a lot of stuff like
         | "referencing the techniques from Chapter 2 we do <blah>" in the
         | wild, and any chunking solution is unlikely to correctly answer
         | queries involving both Chapter 2 and <blah>, at least not
         | without significant false positive rates.
         | 
         | That said, the chunking people are doing is worse than the
         | SOTA. The core thing you want to do is understand your data
         | well enough to ensure that any question, as best as possible,
         | has relevant data within a single chunk. Details vary (maybe
         | the details are what you're asking for?).
        
       | bcherry wrote:
       | It's kind of interesting because I think most people implementing
       | RAG aren't even thinking about tokenization at all. They're
       | thinking about embeddings:
       | 
       | 1. chunk the corpus of data (various strategies but they're all
       | somewhat intuitive)
       | 
       | 2. compute embedding for each chunk
       | 
       | 3. generate search query/queries
       | 
       | 4. compute embedding for each query
       | 
       | 5. rank corpus chunks by distance to query (vector search)
       | 
       | 6. construct return values (e.g chunk + surrounding context, or
       | whole doc, etc)
       | 
       | So this article really gets at the importance of a hidden,
       | relatively mundane-feeling, operation that occurs which can have
       | an outsized impact on the performance of the system. I do wish it
       | had more concrete recommendations in the last section and code
       | sample of a robust project with normalization, fine-tuning, and
       | eval.
        
       | yoelhacks wrote:
       | I used to work on an app that very heavily leaned on
       | Elasticsearch to do advanced text querying for similarities
       | between a 1-2 sentence input and a corpus of paragraph+ length
       | documents.
       | 
       | It was fascinating how much tokenization strategies could affect
       | a particular subset of queries. A really great example is a "W-4"
       | or "W4" Standard tokenization might split on the "-" or split on
       | letter / number boundaries. That input now becomes completely
       | unidentifiable in the index, when it otherwise would have been a
       | very rich factor in matching HR / salary / tax related content.
       | 
       | Different domain, but this doesn't shock me at all.
        
         | carom wrote:
         | The trained embedding vectors for the token equivalents of W4
         | and W-4 would be mapped to a similar space due to their
         | appearance in the same contexts.
        
           | dangerlibrary wrote:
           | The point of the GP post is that the "w-4" token had very
           | different results from ["w", "-4"] or similar algorithms
           | where the "w" and "4" wound up in separate tokens.
        
       | Xenoamorphous wrote:
       | > One of the things I noticed over the past year is how a lot of
       | developers who are used to developing in the traditional
       | (deterministic) space fail to change the way they should think
       | about problems in the statistical space which is ultimately what
       | LLM apps are.
       | 
       | I'm a developer and don't struggle with this, where I really
       | struggle is trying to explain this to users.
        
       | halyax7 wrote:
       | an issue I've seen in several RAG implementations is assuming
       | that the target documents, however cleverly they're chunked, will
       | be good search keys for incoming queries. Unless your incoming
       | search text looks semantically like the documents you're
       | searching over (not the case in general), you'll get bad hits. On
       | a recent project, we saw a big improvement in retrieval relevance
       | when we separated the search keys from the returned values
       | (chunked documents), and we used an LM to generate appropriate
       | keys which were then embedded. Appropriate in this case means
       | "sentences like what the user might input if theyre expecting
       | this chunk back"
        
         | marlott wrote:
         | Interesting! So you basically got a LM to rephrase the search
         | phrase/keys into the style of the target documents, then used
         | that in the RAG pipeline? Did you do an initial search first to
         | limit the documents?
        
           | NitpickLawyer wrote:
           | IIUC they're doing some sort of "q/a" for each chunk from
           | documents, where they ask an LLM to "play the user role and
           | ask a question that would be answered by this chunk". They
           | then embed those questions, and match live user queries with
           | those questions first, then maybe re-rank on the document
           | chunks retrieved.
        
       | quirkot wrote:
       | Is this true?
       | 
       | >> Do not panic! A lot of the large LLM vocabularies are pretty
       | huge (30k-300k tokens large)
       | 
       | Seems small by an order of magnitude (at least). English alone is
       | 1+ millions words
        
         | mmoskal wrote:
         | Tokens are often sub-word, all the way down to bytes (which are
         | implicitly understood as UTF8 but models will sometimes
         | generate invalid UTF8...).
        
         | macleginn wrote:
         | Most of these 1+ million words are almost never used, so 200k
         | is plenty for English. Optimistically, we hope that rarer words
         | would be longer and to some degree compositional (optim-ism,
         | optim-istic, etc.), but unfortunately this is not what
         | tokenisers arrive at (and you are more likely to get "opt-i-
         | mis-m" or something like that). People have tried to optimise
         | tokenisation and the main part of LLM training jointly, which
         | leads to more sensible results, but this is unworkable for
         | larger models, so we are stuck with inflated basic
         | vocabularies.
         | 
         | It is also probably possible now to go even for larger
         | vocabularies, in the 1-2 million range (by factorising the
         | embedding matrix, for example), but this does not lead to
         | noticeable improvements in performance, AFAIK.
        
       | HanClinto wrote:
       | I really appreciated this blog post, and in particular I
       | appreciated the segment talking about typos.
       | 
       | We were discussing this earlier this week -- I'm helping with a
       | RAG-like application for a project right now, and we're concerned
       | with how much small typos or formatting differences in users'
       | queries can throw off our embedding distances.
       | 
       | One thought was: Should we be augmenting our training data (or at
       | the very least, our pretraining data) with intentional typos /
       | substitutions / capitalizations, just to help it learn that "wrk"
       | and "work" are probably synonyms? I looked briefly around for
       | typo augmentation for (pre)training, and didn't see anything at
       | first blush, so I'm guessing that if this is a common practice,
       | that it's called something else.
        
         | tmikaeld wrote:
         | I work with full text search where this is common. Here is some
         | points.
         | 
         | Stemming: Reducing words to their base or root form (e.g.,
         | "working," "worked" becoming "work").
         | 
         | Lemmatization: Similar to stemming, but more sophisticated,
         | accounting for context (e.g., "better" lemmatizes to "good").
         | 
         | Token normalization: Standardizing tokens, such as converting
         | "wrk" to "work" through predefined rules (case folding,
         | character replacement).
         | 
         | Fuzzy matching: Allowing approximate matches based on edit
         | distance (e.g., "wrk" matches "work" due to minimal character
         | difference).
         | 
         | Phonetic matching: Matching words that sound similar, sometimes
         | used to match abbreviations or common misspellings.
         | 
         | Thesaurus-based search: Using a predefined list of synonyms or
         | alternative spellings to expand search queries.
         | 
         | Most of these are open and free lists you can use, check the
         | sources on manticore search for example.
        
           | soared wrote:
           | Porter stemming is currently widely used in adtech for
           | keywords.
        
         | andix wrote:
         | For queries there is an easy solution: give the question/search
         | term to a LLM and let it rephrase it. A lot of basic RAG
         | examples do that.
         | 
         | This might also work for indexing your data, but has the
         | potential to get really expensive quickly.
        
         | bongodongobob wrote:
         | I'm glad this is mentioned. I've suspected that using correct
         | grammar, punctuation and spelling greatly impacts response
         | quality. It's hard to objectify so I've just decided to write
         | my prompts in perfect English just to be sure. I have a friend
         | who prompts like he texts and I've always felt he was getting
         | lower quality responses. Not unusable, just a little worse, and
         | he needs to correct it more.
        
       | woolr wrote:
       | Can't repro some of the numbers in this blog post, for example:
       | from sentence_transformers import SentenceTransformer       from
       | sentence_transformers import util            model =
       | SentenceTransformer('all-MiniLM-L6-v2')            data_to_check
       | = [         "I have recieved wrong package",         "I hve
       | recieved wrong package"       ]       embeddings =
       | model.encode(data_to_check)       util.cos_sim(embeddings,
       | embeddings)
       | 
       | Outputs:                 tensor([[1.0000, 0.9749],
       | [0.9749, 1.0000]])
        
         | 1986 wrote:
         | Your data differs from theirs - they have "I have received
         | wrong package" vs "I hve received wrong pckage", you misspelled
         | "received" in both and didn't omit an "a" from "package" in the
         | "bad" data
        
       | andix wrote:
       | This is an awesome article, but I'm missing the part where
       | solutions for each of the problems were discussed.
       | 
       | Run a spell check before tokenizing? Maybe even tokenize the
       | misspelled word and the potential corrected word next to each
       | other like ,,misspld (misspelled)"?
       | 
       | For the issue with the brand names the tokenizer doesn't know, I
       | have no idea how to handle it. This problem is probably even
       | worse in less common languages, or in languages which use a lot
       | of compound words.
        
       ___________________________________________________________________
       (page generated 2024-10-23 23:00 UTC)