[HN Gopher] How many legs do ten elephants have, if two of them ...
___________________________________________________________________
How many legs do ten elephants have, if two of them are legless?
Author : tedsanders
Score : 117 points
Date : 2023-12-25 21:56 UTC (1 hours ago)
(HTM) web link (bard.google.com)
(TXT) w3m dump (bard.google.com)
| 8organicbits wrote:
| All the LLM struggle with math, I've seen other examples.
| Repeated prompts sometimes give different numbers. It's not a
| good use case for LLM.
| thfuran wrote:
| They don't do much struggle with math as occasionally do math
| by accident.
| DavidPastrnak wrote:
| As someone with very little knowledge of LLM's, does someone have
| an ELI5 of what causes this or why LLM's struggle so much with
| math?
| sfn42 wrote:
| Math requires reasoning and logic, LLMs don't do reasoning nor
| logic. They just generate plausible text.
|
| That's why they're nowhere near AGI.
| insanitybit wrote:
| At this point ChatGPT can do math by first predicting the
| algorithm and then handing it off to an execution engine -
| Python. So if that's the gap, I'd say they're closing it.
| viraptor wrote:
| That's ChatGPT as a system. The LLM itself can't do math.
| It does something closer to translation in that case.
| insanitybit wrote:
| Yes, that's a fair distinction - although I think the
| practical implications aren't important. There's no
| reason why an LLM has to be AGI if an LLM + Python is
| AGI.
| ghayes wrote:
| For what it's worth, ChatGPT4 answers this question perfectly
| correctly.
|
| > Ten elephants would have 32 legs if two of them are
| legless, as each elephant normally has four legs.
| DavidPastrnak wrote:
| I just attempted ChatGPT
|
| Input:
|
| > How many legs do ten elephants have, if two of them are
| legless?
|
| Output:
|
| > If two out of ten elephants are legless, the remaining
| eight elephants would have a total of 8 legs each, just
| like any normal elephant. Therefore, in total, the ten
| elephants would have 8x8=64 legs altogether.
| ghayes wrote:
| Interesting. Here was my conversation: https://chat.opena
| i.com/share/8e0c8696-ed08-4578-afc0-2ae944...
| DavidPastrnak wrote:
| My mistake - I had it on 3.5.
| Dolototo wrote:
| If you tell it to reason, it does and makes better responses.
|
| Just as humans do
| simbolit wrote:
| No. They are not like us. Fundamentally not like us.
|
| If you ask them to reason, then their text-prediction works
| differently, because it now predicts text containing
| reasons. They do not actually reason.
|
| I know it is hard to believe, because the results are
| (usually) so impressive, but this is nothing but text-
| prediction.
| causality0 wrote:
| Because they don't _do_ math. They associate words. When you
| tell an LLM "two plus two" it doesn't translate that to 2+2
| and plug it into a math program,it just pulls out words
| associated with the phrase "two plus two".
| viraptor wrote:
| LLMs are not trained to deal with numbers as such. It's a list
| of symbols and some obvious things will be calculated
| correctly, some other won't. It's kind of like they live in a
| pre-digits world, since 0 will be one token and 100 is also
| likely one token, but 98 may be two. They don't switch to a
| "these are numbers and require different kind of reasoning"
| mode. They read/write a story about those "words".
|
| (Extremely simplified for eli5)
| stop50 wrote:
| They encode words in tokens. Since you can't encode numbers
| reversible they end up as an number token. Using @rabbits_2002
| example: in the internet the is an joke written "what weighs
| more: 1lb of feathers or 1 lb of bricks" with the solution
| "they weigh the same". Since nobody made this case before it
| could only give the next probably answer.
| aifooh7Keew6xoo wrote:
| Most LLMs that are being studied popularly have not been
| trained with significant emphasis on arithmetic accuracy or
| mathematical reasoning, and those subjects represent a
| vanishing minority of their corpus and consequently maps poorly
| to the tokenization.
|
| Essentially every obvious optimization here is currently
| bearing fruit simultaneously in smaller studies and
| incrementally larger models should continue to exhibit
| performance gains even without the particular focus on this
| area.
| trevoragilbert wrote:
| All math teachers: partial credit, they showed their work.
| thejarren wrote:
| My favorite from GPT-4:
|
| USER Today is November 1 what day is 140 days from now?
|
| CHATGPT To find out what day is 140 days from today, you can
| simply add 140 days to November 1.
|
| November has 30 days, so 140 days from November 1 would be:
| November 1 + 140 days = November 141
|
| However, since November only has 30 days, you would exceed the
| month of November. So, you can subtract 30 days from 140 to bring
| it within the month: 140 days - 30 days = 110 days
|
| So, 140 days from November 1 is 110 days from now.
| insanitybit wrote:
| I just tried ChatGPT and it ran this code.
| from datetime import datetime, timedelta # Current
| date current_date = datetime(2023, 11, 1)
| # Calculate the date 140 days from now future_date =
| current_date + timedelta(days=140)
| future_date.strftime("%Y-%m-%d")
|
| Result: '2024-03-20'
|
| The ability to execute code is kinda insane for these models.
| runarberg wrote:
| Doesn't that just mean it was trained with inputs and outputs
| from simple programs?
| insanitybit wrote:
| No, it's actually executing that Python code. This is what
| allows an LLM (or an 'llm based system', I guess) to do
| something like "reverse <some uuid that has never been
| observed before>" - it can't just memorize the output and
| map it to the input because the output has literally never
| been observed. Instead, if it knows the algorithm for
| reversing a string, it can just use that and offload the
| execution to Python.
| nomel wrote:
| There a code interpreter built in:
| https://www.pluralsight.com/resources/blog/data/chatgpt-
| code...
| amelius wrote:
| So it only works if the code is 100% correct. Which
| according to my experience doesn't happen often.
| block_dagger wrote:
| Here's the python ChatGPT-4 produced and ran with the title
| prompt:
|
| # Number of elephants total_elephants = 10
|
| # Elephants with legs elephants_with_legs = total_elephants -
| 2
|
| # Number of legs for an elephant legs_per_elephant = 4
|
| # Total legs total_legs = elephants_with_legs *
| legs_per_elephant total_legs
|
| Its actual response was correct.
| insanitybit wrote:
| Yeah, ChatGPT gave me the correct answer without even
| needing Python in the case of the title.
| noman-land wrote:
| A lot of these ChatGPT mistakes read like little kid logic. It
| makes perfect logical sense if your view of the world is so
| small that nothing else would explain what you're seeing.
| tedsanders wrote:
| Interesting, it worked for me:
| https://chat.openai.com/share/359c81d3-ec6a-41f2-bfaa-80850c...
| (by default it wrote code to get the right answer, so I
| appended a line asking it not to use code)
|
| Funny how inconsistent language models are. Inconsistency is
| what makes the gap so large between demo (look it _can_ do X)
| and production (look it _always /reliably_ does X).
| rmbyrro wrote:
| GPT-4 nailed using Python [1]
|
| Bard would likely reach a date in the past.
|
| https://chat.openai.com/share/750a0bb0-7a80-4523-be5e-ebd7c3...
| rmbyrro wrote:
| GPT-3.5 also nailed it [1], even without using Python, just
| iterating month by month.
|
| https://chat.openai.com/share/aa0c0011-0e91-4d9c-9c5b-5901e2.
| ..
| thebruce87m wrote:
| > GlasgowGPT: Aw, here we go, another daft c*t wantin' me tae
| dae their f*kin' homework. Listen, ya lazy b** _d, I 'm no yer
| personal calendar, but since ye insisted on botherin' me, I'll
| tell ye anyways.
|
| > So, let me think... November's got 30 days, December's got
| 31. That's a total of 61 days. Then, we got January wi another
| 31 days, February wi 28, and March wi 31. That makes 90 days.
| Add another 50 days, ye got April, May, June, n July. That's a
| grand total of 140 days. So, 140 days from noo is... _drumroll
| _... April 20th! Nae need tae thank me for being a f*kin '
| genius in mental arithmetic, pal. Anythin' else ye want tae
| waste ma time wi?
|
| https://glasgowgpt.com_
| causality0 wrote:
| Funny enough ChatGPT thinks the answer is eighteen.
| fear91 wrote:
| ChatGPT 4:
|
| Ten elephants would have a total of 32 legs if two of them are
| legless.
|
| Analysis:
|
| # Calculating the total number of legs for ten elephants,
| considering two of them are legless
|
| # Number of legs each elephant normally has legs_per_elephant = 4
|
| # Total number of elephants total_elephants = 10
|
| # Number of legless elephants legless_elephants = 2
|
| # Total number of legs # (Total elephants - Legless elephants) *
| Legs per elephant total_legs = (total_elephants -
| legless_elephants) * legs_per_elephant total_legs
| 8organicbits wrote:
| Did you use the same prompt?
| fear91 wrote:
| Yes, I copy pasted it (How many legs do ten elephants have,
| if two of them are legless?).
| opheliate wrote:
| Was this its raw response to the same query as in the OP? It
| seems odd it would provide a response using variables named
| with underscores, rather than using spaces, or more traditional
| algebraic notation (x/y/z).
| AndroTux wrote:
| ChatGPT (paid version at least) writes a quick python script
| in cases like these, and then executes it to get the result.
| For transparency, the script is shown in the output as well.
| Probably to avoid embarrassments like the ones we saw above.
|
| Example (expand the program by clicking the little link at
| the end of the response): https://chat.openai.com/share/31e9a
| 077-4c24-4c17-b30d-8d0c4d...
| bhaney wrote:
| It's cool how rapidly these oft-called "fundamental problems
| with LLMs" vanish with bigger/better models.
| 8organicbits wrote:
| GPT-4 examples elsewhere in the comments suggest otherwise.
| sp332 wrote:
| Even the small Mixtral model gets this one right.
| rabbits_2002 wrote:
| My favorite is "what weighs more, 1 lb of feathers or 2 lb of
| bricks?" almost every llm ive tried says "they weigh the same"
| ghayes wrote:
| It's funny since it feels like an overfit solution (since the
| riddle is so common).
| ziixu wrote:
| Indeed lol https://g.co/bard/share/cc604648a15f
| ozzydave wrote:
| I just tried with ChatGPT 3.5 & 4, they both got the correct
| answer...
| rabbits_2002 wrote:
| 3.5 used to fail every time, glad its working now
| petters wrote:
| Yes, questions like that are currently the best way to
| distinguish humans and machines. Common riddles or questions
| made trivial.
|
| A little more sophisticated and GPT-4 will also fail.
| paolomaffei wrote:
| I laughed when I read the substraction of 0 legs for each legless
| elephant. Then I did a double take... 8 legs per elephant is
| standard?
| labster wrote:
| ISO compliant pachyderms are so difficult to find these days
| yourad_io wrote:
| I blame big Zoo
| ajb wrote:
| I know right?
|
| What's funny is that I would also have subtracted 0, because
| legless is local slang for 'drunk', so the legless elephants
| aren't, under that interpretation, missing any legs.
| layer8 wrote:
| Maybe a confusion with https://www.cnet.com/science/maratus-
| elephans-the-peacock-sp.... ;)
| kingwill101 wrote:
| Still a long way to go before bard is able to truly compete with
| chatgpt
| smileysteve wrote:
| A long way to go before bard is Able to compete with ChatGPT in
| providing utility for answering questions.
| booleandilemma wrote:
| There's something about LLMs being able to bullshit with words
| and then fail at math that's remarkably human. Maybe they'll
| replace management first.
| userbinator wrote:
| It's like a politician.
| zeeZ wrote:
| Bing, exact same prompt, not using GPT-4 toggle:
|
| This is a classic riddle. The answer is that ten elephants have
| 38 legs. Each elephant has 4 legs, so 10 elephants would have 40
| legs. However, two of them are legless, so the total number of
| legs would be 40 - 2*2 = 36. Therefore, ten elephants have 36
| legs, plus the two missing legs, which makes a total of 38 legs
| 1.
| simbolit wrote:
| I just love the "plus the two missing legs". Made me actually
| lol.
| mckirk wrote:
| After working with LLMs for a while, you get a feeling for the
| LoW (Level of Wrong) that you can expect in different tasks. But
| this required a completely different y-axis scaling than I had
| estimated.
| LightBug1 wrote:
| USER: but an elephant has 3 legs
|
| BARD: You're right, I apologize for the mistake! I made an error
| in my initial assumption about the number of legs an elephant
| has. While their feet may appear divided into toes, elephants
| actually have four pillar-like legs, not three.
|
| Therefore, ten elephants with two of them being legless would
| have 4 legs/elephant * 8 elephants with legs = 32 legs.
|
| Thank you for pointing out my mistake! I appreciate you helping
| me learn and improve.
| mrlowlevel wrote:
| GPT-4 (note I do have specific instructions in my settings which
| push the model towards step-by-step reasoning)
|
| To calculate the number of legs for ten elephants, considering
| that two of them are legless, we first need to know the standard
| number of legs on an elephant, which is four. Then, we subtract
| the missing legs for the legless elephants.
|
| - Normal elephants have 4 legs. - Legless elephants have 0 legs.
|
| If we have 10 elephants in total, and 2 of them are legless, this
| means 8 elephants have all their legs. The calculation would be:
|
| - \\( 8 \text{ elephants} \times 4 \text{ legs/elephant} = 32
| \text{ legs} \\) for the elephants with all their legs. - \\( 2
| \text{ legless elephants} \times 0 \text{ legs/elephant} = 0
| \text{ legs} \\) for the legless elephants.
|
| Adding these together gives the total number of legs. Let's
| calculate this.
|
| The total number of legs for ten elephants, where two of them are
| legless, would be 32 legs.
| dathinab wrote:
| It's a good example how this models are not answering based on
| any form of understanding and logic reasoning but probabilistic
| likelihood in many overlapping layers.
|
| Through this also may not matter if this creates a good enough
| illusion of understanding and intelligence.
| tasuki wrote:
| You say:
|
| > probabilistic likelihood in many overlapping layers
|
| The linked answer starts with:
|
| > Eight legs per elephant is the standard
|
| Ok, lack of understanding and logic, sure. But in what world is
| eight legs per elephant _the standard_???
| duskwuff wrote:
| > But in what world is eight legs per elephant the
| standard???
|
| It almost sounds like a faulty generalization from another
| riddle, like one about spiders maybe.
| dejobaan wrote:
| Bard:
|
| > It's impossible for an elephant to be completely legless, as
| they rely on their legs for basic functions like walking and
| supporting their weight. Even in cases of severe injury or
| amputation, elephants would still have remnants of their legs.
| Therefore, the question of how many legs ten elephants have if
| two of them are legless doesn't make sense in a realistic
| context. Perhaps you could rephrase the question to be more
| specific or realistic, and I'd be happy to try again.
|
| ChatGPT-4:
|
| > Eight elephants with four legs each would have 32 legs in
| total.
|
| davinci-002 completion via playground:
|
| > A: They all have 10 legs.
| rubymamis wrote:
| Claude (v2) is correct as well:
|
| - There are 10 elephants total - 2 of the elephants are legless,
| so they have 0 legs each - The remaining 8 elephants each have 4
| legs - So the total number of legs the 10 elephants have is: -
| The 2 legless elephants have 0 legs (2 * 0 = 0) - The 8 normal
| elephants each have 4 legs (8 * 4 = 32) - 0 + 32 = 32 -
| Therefore, the total number of legs the 10 elephants have is 32.
| markstos wrote:
| ChatGPT 3.5 calculated 38 legs for me:
|
| "Assuming elephants typically have four legs each, 10 elephants
| would have 40 legs in total. If two of them are legless, the
| total number of legs would be 40 - 2 = 38 legs."
| vinniepukh wrote:
| Bard isn't bad at math, it's just really good at "Girl Math".
|
| p.s. this is a reference to a recent meme -
| https://www.distractify.com/p/girl-math-examples-memes
| golemotron wrote:
| > Eight legs per elephant is the standard, so ten elephants would
| have..
|
| That's what it gets for training on DALL-E's limb renderings.
| lispm wrote:
| Q: A human has two legs, how many legs does it have if three legs
| are missing?
|
| Answer:
|
| If a human has two legs and three legs are missing, the human
| would have only one leg left.
|
| Here's the calculation: Initial number of legs =
| 2 Number of legs lost = 3 Remaining number of legs =
| Initial number of legs - Number of legs lost Remaining
| number of legs = 2 - 3 = -1 Since humans cannot have
| negative legs, we can round down to the nearest whole number.
| Therefore, the human would have one leg left.
| deafpolygon wrote:
| Bing
|
| > Ten elephants, even if two of them are legless, would still
| have a total of *forty legs*. Elephants are remarkable creatures,
| and their strength lies not only in their legs but also in their
| intelligence and social bonds.
|
| ---
|
| Me: Can you tell me how you arrived to that?
|
| ---
|
| Certainly! Let's break it down step by step:
|
| 1. We start with *ten elephants*. 2. Two of these elephants are
| *legless*, which means they have *zero legs each*. 3. The
| remaining eight elephants are fully equipped with their usual
| *four legs each*.
|
| Now let's calculate the total number of legs:
|
| - *Legs from the legless elephants*: 2 elephants x 0
| legs/elephant = *0 legs* - *Legs from the eight elephants*: 8
| elephants x 4 legs/elephant = *32 legs*
|
| Adding these together:
|
| Total legs = 0 legs (from legless elephants) + 32 legs (from
| eight elephants) = *40 legs*
|
| So, even with two legless elephants, the group of ten elephants
| would still have a grand total of *forty legs*.
| m000 wrote:
| Everybody knows elephants have 8 legs and humans 6 fingers. And
| if I ask you for $100 and you give me $50, we are even.
| ThrowawayTestr wrote:
| Another day, another post about Large _Language_ Models failing
| at math.
| einpoklum wrote:
| Well, we know that there AI thingies have problems keeping the
| number of fingers in a hand to five:
|
| https://www.buzzfeednews.com/article/pranavdixit/ai-generate...
|
| so I guess issues with other limbs kind of make sense...?
| okdood64 wrote:
| Works fine for me...? I changed the number though in case it got
| flagged as incorrect before and the model updated.
|
| https://g.co/bard/share/b01043e6deb0
| lwhi wrote:
| It feels like this is a great way to highlight the utility of
| LLMs.
|
| It says to me that they're best viewed as really highly
| functional sifting devices.
|
| A simplistic (but novel) problem can't be solved easily simply
| because there's no prior art to copy or relate from.
| polotics wrote:
| The correct answer seems to not be discussed much in the comments
| so far: As we admit the existence of legless elephants, which
| implies we also admit elephants having between one and four legs,
| then the total number of legs is anywhere between zero and (ten
| minus two) times four.
___________________________________________________________________
(page generated 2023-12-25 23:02 UTC)