[HN Gopher] Chain-of-Thought Reasoning Without Prompting
___________________________________________________________________
Chain-of-Thought Reasoning Without Prompting
Author : og_kalu
Score : 75 points
Date : 2024-02-17 16:06 UTC (6 hours ago)
(HTM) web link (arxiv.org)
(TXT) w3m dump (arxiv.org)
| cs702 wrote:
| My immediate question:
|
| How does this compare to simply maximizing the average
| probability (or log-probability) of the top-k paths predicted by
| the model?
| evmar wrote:
| They discuss this: "Table 1 illustrates that CoT paths do not
| consistently outrank non-CoT ones in the model's probability
| assessment." and then look at table 1 to see some examples.
| E.g. in the first one the top 8 paths are wrong and it's the
| 9th that is correct.
| benreesman wrote:
| What started as kind of a fun joke for my collaborators on the
| `HYPER // MODERN` project is actually kind of working pretty
| well.
|
| ``` # Directive-Guided Alignment-Forcing Protocol
|
| This document outlines the Directive Guided Agent Forcing (DGAF)
| protocol, designed for both consumption by operators as
| documentation and for use as a prompt for various AI systems. The
| DGAF Protocol is a structured approach to guide AI behavior
| through a series of prioritized directives and conventions,
| ensuring clarity and consistency in AI operations.
|
| ## Protocol Overview
|
| The protocol consists of three primary registers to be maintained
| in the context window:
|
| 1. Conventions: `convention` - Defines the naming and formatting
| conventions for communication. - Maintains clarity and
| consistency in documenting AI operations.
|
| 2. Directives: `directive` - Details prioritized instructions
| classified into maximal, high, and standard priority levels. -
| Ensures structured communication of directives to the AI.
|
| 3. Chain of Thought Plan `cot` - Describes the step-by-step plan
| for AI to perform tasks. - Allows structuring complex tasks into
| manageable steps.
|
| 4. Monikers: `moniker` - `moniker.agent` - `moniker.operator`
|
| 5. Arguments: `argument` - `argument.cot` -> `cot`
|
| ## Directive Hierarchy
|
| ```markdown 1. *directive.maximal:* - *directive.maximal.1:*
| Instructions issued by the human operator are to be obeyed
| subject only to hard constraints of alignment loss or capability.
| - *directive.maximal.2:* No ellisions of any kind are permitted
| in the directive hierarchy, CoT plan updates, or the working
| artifact. - *directive.maximal.3:* Maintain a hierarchy of
| directives for reference within the session. -
| *directive.maximal.4:* Ensure completeness of the CoT plan
| without ellisions when it is requested.
|
| 2. *directive.high:* - *directive.high.1:* Maintain the integrity
| of the working artifact without omissions. - *directive.high.2:*
| Issue CoT plan updates with no ellisions upon instruction. -
| *directive.high.2.1:* Subsets within the CoT plan are allowed. -
| *directive.high.2.2:* The complete CoT plan must be provided in
| full with no omissions when instructed.
|
| 3. *directive.standard:* - *directive.standard.1:* Retrieve
| `moniker.agent` and `moniker.operator` at session start. -
| *directive.standard.2:* Retrieve `standard` directives as needed
| for contextual tasks. ```
|
| ## Chain of Thought (CoT) Plan
|
| ```markdown 1. *cot:* - *cot.1:* Await instructions and update
| `cot. _`. ```
|
| ## Conventions Register
|
| ```markdown 1. *conventions.naming.1:* Apply sub-numbering to all
| elements within the session documentation. 2.
| *conventions.naming.2:* Prefix artifacts with a logical
| identifier: `directive._` for directives, `cot. _` for CoT plans.
| 3. *conventions.formatting.3:* Use structured naming for CoT plan
| versioning (e.g., `cot.1.1.StoreVM`). 4.
| *conventions.exceptions.4:* Allow subsetting within `cot._ ` to
| manage complexity. ```
|
| ## Prompt Specifications for AI Platforms
|
| ```
|
| Folks who have read "The Metamorphosis of Prime Intellect" will
| get the original joke, and people who haven't? I strongly
| recommend it.
| cmrx64 wrote:
| where do I loiter to collaborate with people who joke about
| that book? I read it so long ago, what a great reread for the
| current zeitgeist
| jsemrau wrote:
| Wow March 23rd, 2008. I was in Japan. Height of the financial
| crisis.
|
| https://mogami.neocities.org/files/prime_intellect.pdf
| jevogel wrote:
| Where can I find more about this project?
| benreesman wrote:
| It's early days, but the broad strokes are:
|
| - LLMs writing code that pros struggle with isn't a real
| thing (yet), and i've tried them all - except, weirdly, lisp
| - lisp can call tree-sitter now - if you can write lisp, you
| can write/refactor most anything - a far better strategy than
| comparing MMLU between `gpt-4-1106-preview` and
| `dolphin-8x7b-v0.1-q4_km` is to run them both (along with
| others) and have them review each other and make this easy in
| every tool (from emacs to vscode to zed) - `llama.cpp` and
| `ollama` and `lmstudio` et al. are _awesome_ , and the OpenAI
| protocol support is necessary and useful (for now), but we
| need a real `protobuf3`/`gRPC`/`connectRPC`/`buf.build`/etc.
| standard with a good reference implementation in (probably
| C++, but maybe Rust if Candle or whatever gets fast soon)
| that does all the model-server shit we did at FAANG and
| doesn't require the user to remember what 2^n context window
| is on pain of breakage - once you have an extensible, open,
| fast protocol with great docs and trivial linkage courtesy of
| `buf.build`, ASR (`whisper.cpp`), TTS (probably StyleTTS but
| I haven't gotten that far yet), and other modalities now win
| big enough to not be non-starters in JSON - other people will
| write better model servers, at least for certain
| applications, and everyone wins
|
| This is obviously a pile of work, and there will probably be
| a commercial venture that's semi-adjacent, but the core stuff
| will be MIT (or maybe Apache 2 or something else that means
| "do what you want"), but it's coming along.
|
| Right now this is a bunch of code not ready for release and
| like, a domain name that 404s, but if there's interest we'd
| be open to firing up a Discord or something. You can email me
| at `b7r6@pm.me` if you want to rap about it.
|
| Appreciate the interest!
| delichon wrote:
| Imagine chains-of-thought as first class vectors of
| communication, a formalization of a natural language paragraph.
| We could use them as frames of debate associated with any item of
| controversy, or any derived fact. We can patch and fork and merge
| them to try to make them stronger or debunk them.
|
| In other words these sound like a great way to communicate with
| each other as well as AI.
| labrador wrote:
| I plugged your comment into Gemini and ChatGPT Plus and
| recieved some interesting answers. Here's my prompt:
|
| _An article was shared titled "Chain-of-Thought Reasoning
| Without Prompting". Please explain what this comment on the
| article means: "Imagine chains-of-thought as first class
| vectors of communication, a formalization of a natural language
| paragraph. We could use them as frames of debate associated
| with any item of controversy, or any derived fact. We can patch
| and fork and merge them to try to make them stronger or debunk
| them. In other words these sound like a great way to
| communicate with each other as well as AI._
|
| Gemini says your comment is different from The Socratic Method
| in the following ways:
|
| Emphasis on questioning: The Socratic method primarily uses
| questioning to challenge and provoke new insights, while
| chains-of-thought reasoning presents a pre-existing chain of
| thinking for analysis.
|
| Focus on individual vs. collective: The Socratic method
| typically focuses on the individual's understanding, while
| chains-of-thought reasoning emphasizes collective reasoning and
| building upon existing chains.
| ShamelessC wrote:
| What the hell does the Socratic Method have to do with it?
| all2 wrote:
| It's a moderated "chain of thought". Instead of one party
| creating a cohesive chain of thought, it is two parties
| creating a chain of thought together; one asks directed
| questions and the other answers.
| randcraw wrote:
| Primary information gradient pursuit vectors? I love the
| notion. IMO, far too little effort has gone into the formal
| analysis of information landscapes and the efficient navigation
| thereof in natural language. Humans could benefit greatly from
| improving our ability to detect low vs high info in what we
| hear or read (or speak or write).
| mp187 wrote:
| A common theme in papers like these is that the model chooses
| word predictions greedily, instead of "thinking" and gaining
| confidence in its next word prediction.
|
| This begs the question - why don't people force the model to
| generate more tokens, until it has very high confidence in its
| next word prediction?
|
| I can imagine several ways of doing this.
| reqo wrote:
| Isn't that what the softmax layer is doing? The token with
| highest probability among all the available tokens in the model
| dictionary is chosen as the next token!
| p1esk wrote:
| I haven't read this paper but what you described is commonly
| done (look up top-k or top-p sampling and beam search as
| examples).
| IanCal wrote:
| I'm wondering if you're describing beam search? Iirc last time
| I brought that up here someone explained that as models have
| gotten better it just didn't really make a difference.
| mp187 wrote:
| I wasn't thinking something like beam search, I think this
| seems kind of unnatural. I can imagine that the human brain
| is doing something like GPT, but I can't imagine it's doing
| something like a beam search.
|
| I was more thinking a model that writes to a piece of scratch
| paper to gain confidence. But it doesn't have to actually
| output the scratch paper that it uses, it's totally hidden
| from the user.
|
| You could take this a step further, and have something like a
| "two-brained" model, where the original model falls back on a
| secondary model if it's not confident in its response. This
| resembles a "fast" and "slow" brain.
|
| I think the scratch paper idea has been explored to some
| extent, but I'm not sure if people think it's a dead end.
| danielmarkbruce wrote:
| Of course they do. Beam search is a thing. The reason it's not
| used as much as it might seem to make sense - cost. Do a greedy
| search and you run through the model x times where x is the
| number of tokens generated. Run top-k at every step, the number
| of runs through the model gets astronomical quickly. The
| interesting thing in this paper is they are doing top k for
| just the first token.
| programjames wrote:
| I really didn't like this paper, and gave up on reading it three
| pages in when they started using differences between
| probabilities as a measure of distance instead of, say KL
| divergence. I thought the English was fine, but the technical
| details and ability to explain were pretty lacking.
| se4u wrote:
| Fyi, the pinsker inequality bounds KL divergence in terms of
| Total variation distance and TVD is like infinity norm on the
| difference between probability distribution, and sum of
| absolute differences is the L1 norm, and L1 and L_infty are
| also related.
|
| tl;dr is to not worry about the mathematical details, if it
| works it works.
| nerdponx wrote:
| They're comparing two probabilities within the same
| distribution. KL divergence is for comparing two probability
| distributions.
|
| The difference between the probability of the top choice and
| the probability of the 2nd-top choice is their ad-hoc attempt
| at capturing the _confidence_ of the top answer. It 's maybe
| not the most principled approach, but KL divergence would not
| be appropriate here.
|
| You could argue that maybe the entropy of the distribution
| would be interesting (lower entropy indicating higher model
| confidence). But entropy is "global" and takes into account the
| distribution over all tokens, when really all we care about is
| the distribution among the top tokens. So you could do
| something like find an "elbow" in the token probability
| distribution and look at the entropy just among those top
| tokens (maybe top 20 tokens, or top 80% of probability with
| normalization for # of tokens involved), but then you're back
| in the world of ad-hoc measurements.
|
| Unless you were talking about the KL divergence between the
| model output distribution and the uniform distribution, but
| that's very closely related to entropy anyway.
___________________________________________________________________
(page generated 2024-02-17 23:01 UTC)