[HN Gopher] Mathematicians and the Selection Task (2004)
___________________________________________________________________
Mathematicians and the Selection Task (2004)
Author : tzs
Score : 44 points
Date : 2024-05-19 02:30 UTC (20 hours ago)
(HTM) web link (eric.ed.gov)
(TXT) w3m dump (eric.ed.gov)
| x3n0ph3n3 wrote:
| > less than a third of students and less than half of staff gave
| the correct answer.
|
| This is incredibly troubling. If universities cannot produce
| people that can consistently get these kinds of problems right,
| what the hell are they even good for?
| drsopp wrote:
| Not necessarily more troubling than being tricked by an optical
| illusion. Perhaps this problem is more like a logical illusion
| because of the presentation/wording.
| cjohnson318 wrote:
| I think the fact is that although predicate logic is a
| foundation of mathematics, it is not what mathematicians spend
| the majority of their time thinking deeply about. You might use
| English every day of your life, but still struggle to explain
| what a transitive verb is, or a gerund.
| bee_rider wrote:
| It would be interesting to run this experiment on CS or EE
| students. I hope they would do a little better...
| DeathArrow wrote:
| All people would do better if they are warned first it's a
| tricky question.
|
| Also, training in this kind of problems will help.
| clipsy wrote:
| I think you are presuming that the participants who failed were
| unable to solve the underlying logic problem, when it is
| entirely possible that they (eg) misread part of the problem
| setup.
|
| (Likewise the paper seems to infer a difference in logical
| thinking rather than considering a difference in
| processing/interpreting the problem.)
| wruza wrote:
| Maybe, but understanding a problem is half its solution.
| Misread and couldn't solve correctly are the same thing in
| problem solving.
| tzs wrote:
| An interesting thing about the Wason selection task is that
| people do a lot better when given a task that requires the exact
| same reasoning but involves social situations.
|
| For example if the cards have on one side either a picture of a
| mug of beer or a picture of a can of soda, and the other side
| have a number representing the age of a person drinking that
| drink, and the rule they are supposed to be checking is that if
| someone is drinking beer they have to be at least 21 then 75% of
| people correctly figure out that they need to check the other
| side of the cards showing beer and the cards showing an age under
| 21.
|
| Here's Wikipedia's article on the Wason selection task [1].
|
| [1] https://en.wikipedia.org/wiki/Wason_selection_task
| penteract wrote:
| I'm not sure it's true to say that that task requires the exact
| same reasoning - There are various logically equivalent ways of
| phrasing the rule such as "No card has a D on one side and a
| number other than 3 on the other" which make the problem
| easier.
|
| Since the rule "you can't drink alcohol if you're underage" is
| one people are familiar with, they aren't being asked to make
| the same logical deduction they do in the letters and numbers
| question. I'd go further and speculate that they aren't all
| reading the question carefully - if you replaced the rule by
| "if someone is over 21 they are drinking beer", how many people
| would get it wrong?
| etangent wrote:
| It seems to me that the selection task is tricky because it
| concerns interpretation of language. "Every card that has a D
| on one side has a 3 on the other" makes a claim: that there is
| a directional dependency "D => 3" but it makes no claim that "3
| => D". However the absence of the latter claim is not stated
| explicitly, it is supposed to be inferred from the original
| statement. The English language seems to lack a way to encode
| unambiguously the "A => B" relationship. So it should not be
| surprising that students used to looking out for language
| pitfalls when checking proofs also happen to be the students
| who do better on this task.
| two_handfuls wrote:
| > The English language seems to lack a way to encode
| unambiguously the "A => B"
|
| It doesn't: "If A then B" encodes it unambiguously.
|
| It's just that as you said, many people don't think hard
| about the difference between this and similar-but-different
| concepts like "B only if A".
|
| It's not the language itself, it's the way people use the
| language and think about what it says (or in this case,
| don't).
| BoiledCabbage wrote:
| > It doesn't: "If A then B" encodes it unambiguously.
|
| No, actually it doesn't
|
| > Some authors have argued that participants do not read
| "if... then..." as the material conditional, since the
| natural language conditional is not the material
| conditional.
|
| > It's just that as you said, many people don't think hard
| about the difference between this and similar-but-different
| concepts like "B only if A".
|
| While a nice simplistic answer it's likely not what's going
| on here. There is more here than "People just aren't good
| at thinking".
|
| If your statement were true, then you'd be forced to say
| that the following statement is also obviously true:
|
| "If the Nazis won World War II, then everybody would be
| happy"
|
| The fact that you can rightfully say that sentence is
| false, means that your comment above about implication and
| "if" statements is wrong.[1] Language is more complex then
| you're giving it credit.
|
| [1] - https://en.m.wikipedia.org/wiki/Paradoxes_of_material
| _implic...
| hardlianotion wrote:
| For what it's worth, I think that the problem lies in the
| solution feeling "obvious" and people being a little lazy
| - I don't think the problem lies with inability.
|
| I also think that if A then B is unambiguous, the counter
| that languages are different doesn't really fit with what
| I think I observe in the wild. For the full house, I also
| fail to see how that means I must accept that your
| example statement is obviously true.
| DeathArrow wrote:
| I just failed the test. :) The only thing that this teaches me
| is to pay more attention before attempting to solve a problem.
| uolmir wrote:
| This result reminds me of a paper I read last week via Andrew
| Gelman's blog [1]. It's a very thorough review of the, so called,
| bat and ball problem and is an up to date summary of something
| brought to many people's attention via Kahneman's Thinking Fast
| and Slow. As other commenters have suggested, the most reasonable
| explanation for the mathematicians to get this problem wrong is
| something more like carelessness than a lack of logical reasoning
| ability.
|
| [1] https://statmodeling.stat.columbia.edu/2024/04/21/now-
| heres-...
| klyrs wrote:
| One must ask if the investigators of this study selected the
| correct faculty for examination. (I kid, this is embarrassing)
|
| As a mathematician, I was primed with the knowledge that a large
| fraction of a mathematics department failed this test. I looked
| it up on Wikipedia, didn't spoil the answer, and thought damn
| hard before unfolding the "solution" section. I was relieved to
| see my answer therein. I do wonder if the students and staff were
| primed to think about this as a logic puzzle, or if they simply
| went with a gut answer. Because in my experience, that makes
| loads of difference in how people of all stripes, mathematicians
| included, respond to challenging questions.
|
| My gut response was to flip an extra card, for what that's worth.
| Secondary consideration took a couple of seconds, and I spent
| another thirty convincing myself that I was correct.
| firewolf34 wrote:
| I think we're looking at this wrong. I feel like this test is
| designed to investigate social biases not test for logical
| skills and if these people are failing it, it's not so much of
| a failure in their understanding of logic but rather a
| procedural impact of the way the question is framed, which is
| probably precisely why "reframing it in a social context"
| changes their result populations. I think this test is
| extremely sensitive to how you pose the question.
|
| Are we trying to test if the candidate can solve the logic
| problem, or are we trying to test how they handle an
| /intentionally-confusing/ situation and what (psychological)
| biases they jump to with their solution?
|
| If it's a test of their logic capabilities, then it seems like
| the numbers are artificially low, so maybe not so embarrassing
| as you say... Reason being, I think there are several
| confounding variables included in the results they'd need to
| control for if that was the point.
|
| An obvious one, if we were testing logic directly, then I
| wonder if they allowed the participants submission to "show
| their work" rather than just which final cards they chose.
| Doing so would eliminate the "carelessness" confounder in the
| result where they didn't thoroughly think through all of the
| logical cases of the cards or where they accidentally included
| an incorrect card but understood the nature of the required
| solution, ie. if they knew they needed to disprove rather than
| confirm but accidentally included a useless card for
| disproving, they still understood how to solve the problem and
| thus the logic. What percentage of their results fall into that
| bucket?
|
| There's also other confounding factors that are set up to
| "confuse" the participant here that could be removed if we
| wanted to truly test their /logical skills/ and /not/ some
| psychological/sociological property. For example, the question
| merely says: "test that if a card shows an even number". In
| English, "if" can mean both the inclusive or exclusive OR
| depending on context - it's needlessly vague, and additionally,
| I posit that in English, given the common usage of the phrase
| "test ... if", the phrase is /leading/ the participant to look
| for /positive confirmation of the rule/ rather than the
| negative. You can of course derive that the negative test is
| needed by studying the cards but why try to mislead them
| outright? Why not say "choose the set of cards that you'd need
| to flip to prove the rule is false"? This clearly demonstrates
| the task and doesn't send them on a goose chase.
|
| There's other things too. It doesn't mention if these cards are
| from a global set of cards or the rule is only meant to be
| proven on the 4 cards presented. It implies the latter but if
| you start thinking about "confirming if the rule is true for
| all cards", it sends you down another useless logical
| rabbithole, yet, /cards normally come from a deck in real life/
| and it is natural to expect there are more cards. Maybe if they
| wanted to be exact we shouldn't be using cards at all but
| rather wooden blocks or something.
|
| And I'm sure there are more "biases" that I'm not catching
| here. If your goal is to test people's likelihood of affected
| by certain biases psychologically, then all's well and good
| with the test, go right ahead. But if you're going to present
| the poor results as some sort of indicator of an population's
| skill at logic, maybe not the best test without some better
| testing procedures, imo.
| beyondCritics wrote:
| "Four cards were placed on a table: [D K 3 7] The participants
| were given the following instructions: Here is a rule: "every
| card that has a D on one side has a 3 on the other." Your task is
| to select all those cards, but only those cards, which you would
| have to turn over in order to discover whether or not the rule
| has been violated. The correct answer is to pick the D card and
| the 7 card,..."
|
| Wait a minute, this is wrong! You have to check the K card also,
| since it could have a D on the hidden side. Only if you were
| given the instruction, which is done later, that all cards have a
| number and a letter on one side, you know that the K card must
| have a letter on the hidden side.
| ccppurcell wrote:
| I think it's given that there is a letter on one side and a
| number on the other side if every card and implicitly, that
| that rule hasn't been broken, otherwise you have to check every
| card to see that there's no emoji there
| beyondCritics wrote:
| We have to make sure, that if there is a "D" on one side,
| there must be a "3" on the other side. If we see a "3", we
| are done, otherwise if there is not a "3" we have to check
| that there is no "D" on the hidden side. Hence we have to
| check exactly all cards facing a "3".
| Jtsummers wrote:
| No, because the framing of the problem includes a statement
| that if there is a letter on one side then there is a
| number on the other and vice versa. The K can't have a D on
| the other side unless they're lying in the problem
| statement (which would defeat the purpose of the
| experiment, so why would they and why would you assume they
| are?).
| beyondCritics wrote:
| I now how the experiment was conducted. That wasn't the
| question either. Read carefully what i have cited. There
| is nothing about what you have said. If you specify
| something, you ought to be very precise :-)
| Jtsummers wrote:
| > Participating subjects were shown a selection of cards,
| each of which had a letter on one side and a number on
| the other.
|
| You skipped that part just above your quote. There is no
| ambiguity. In both this experiment and the original
| experiment they were given that information before their
| selection.
| StevenXC wrote:
| As an math educator, I think there's a huge flaw in this study.
| The investigators failed to follow up to see why the mistake was
| made. They leap to assuming the player is trying to "deny the
| antecedent", but I think there's a much simpler explanation: the
| players aren't reading the instructions carefully.
|
| There's two reading errors I would expect someone to make given
| this experiment:
|
| 1. The instructions that each card has exactly one letter and
| exactly one number are before the big cards. I bet many players
| just skipped that instruction.
|
| 2. Mistaking the P-Q as P--Q smacks more of a reading
| comprehension error than a logical error.
|
| Disclosure: I made both mistakes. (-:
___________________________________________________________________
(page generated 2024-05-19 23:02 UTC)