[HN Gopher] What Is Bayesian/Frequentist Inference? (2012)
___________________________________________________________________
What Is Bayesian/Frequentist Inference? (2012)
Author : spekcular
Score : 63 points
Date : 2022-10-07 23:33 UTC (2 days ago)
(HTM) web link (normaldeviate.wordpress.com)
(TXT) w3m dump (normaldeviate.wordpress.com)
| lalaithion wrote:
| This falls apart in higher dimensions, but in the example given
| in the article the two answers only differ because they have
| different priors. If you repeat the bayesian analysis using the
| prior \theta ~ N(0, x), and let x go to infinity, then you
| approach the frequentist answer.
|
| In my opinion,
| https://stats.stackexchange.com/questions/2272/whats-the-dif...
| is a better explanation of the difference between confidence and
| credible intervals.
|
| Editing to add more commentary to my link:
|
| If you've read the link, one of the principal objections of the
| frequentist is "What if the jar is type B? Then your interval
| will be wrong 80% of the time, and only correct 20% of the time!"
|
| This is because, if you look at the original numbers, jar B has
| all types of cookies, and therefore any single draw from jar B
| can "look like" any other jar, and because other jars have more
| concentrated cookie types, they are "more likely" answers for
| each potential sample.
|
| This issue also comes up with the frequentist analysis! If you
| look at the confidence intervals, they all say "This jar could be
| jar B". Instead of being bad at detecting jar B, they are good at
| always considering jar B no matter the evidence - but it's the
| same uncertainty.
|
| The bayesian version of this objection is "when you pull a cookie
| with 3 chips your interval is only correct 41% of the time". This
| is because if we get jar A, we'll probably draw a 2-chip cookie,
| so it's outside of our confidence interval.
|
| But note that we probably don't really care about the confidence
| interval or credibility interval. It's basically a hack to take a
| probabilistic problem and turn our answer into black and white.
| To say "these hypotheses are valid and these are invalid".
|
| But this is statistics! If you just take the Bayesian approach,
| and throw out the need to create an arbitrary interval, you can
| just stop at the table titled P(Jar|Chips). That's all the
| information you need. If you draw a N-chip cookie, you can use
| that table to update your P(Chips_2) for a second draw, and
| you'll get a concrete probabilistic answer. Yes, you have to
| assume a prior. But frequentist statistics literally can't answer
| this question! Without a prior, there's no way to turn P(Chips |
| Jar) into a P(Jar | Chips) to update on, so you can't track your
| evidence to get better predictions. You just sit there saying
| "well, my interval meets the criterion even in the worst case".
| dekhn wrote:
| Whilst successful in my career and user of probability,
| statistics, and inference on a regular basis, I simply cannot
| understand what's being discussed here.
|
| I don't even want to understand it. Just like quantum, half the
| argument seems to be the a mismatch between mental models and
| actual reality.
| ordu wrote:
| _> half the argument seems to be the a mismatch between mental
| models and actual reality._
|
| Which half seems to be a mismatch to you? A bayesian half or a
| frequentist one?
| dekhn wrote:
| Every time I've tried to understand the entire argument it
| just raises more questions to me. For example as I was first
| introduced to it, frequentists simple count frequencies
| observed in nature and then compute stats on them, and then
| build inferential models using those stats without assuming
| any complex underlying distribution. While Bayesians count
| frequencies, apply a prior correction (say, adding a
| pseudocount of one for every unobserved possible event, or
| any other way of assuming the generative process has a
| distribution that we've previously estimated), some
| stats,then build models from that.
|
| however, after I was told that, I've seen several other
| arguments that quickly dive into: the distribution of the
| underlying events (I've heard that frequentists assume one
| type while bayesian assume another). Other folks just sort of
| give the example of the base rate fallacy.
|
| Throughout all of this I've realized: I don't understand
| stats at all. I came to the scientific world with a view much
| more like physics: there is a microscopic event system (a
| particle simulation, or whatever) that we are observing, but
| due to limitations, we can only make macroscopic
| observations, which represent biased aggregations of the
| underlying microscopic event system. We can figure out those
| biases and use the aggregate data to build predictive models
| of the underlying systems- without ever really knowing the
| true details of the microscopic model.
|
| From what I can tell, everything about what physicists do to
| model the world mentally is more Bayesian than Frequentist,
| if I understand what the hell people mean when they argue
| about it. However, as I said, every time I look at the
| arguments, I realize I don't understand stats, while I
| understand the physics approach which seems to be fairly
| obvious.
| sega_sai wrote:
| In the end I believe the Bayesian inference is more
| straightforward to implement and understand if you can afford
| computationally sampling of the posterior. So I think at least in
| physics there is a shift towards Bayesian approaches.
| clircle wrote:
| This blog is by Larry Wasserman, so i think his advice should be
| taken seriously. I agree that there are uses of both
| philosophies, and that statisticians should be pragmatic rather
| than dogmatic.
|
| My issue is that his advice is most useful for statisticians
| working in the abstract, but it doesn't really help people
| working with real data. Scientists and data analysts just want to
| know how to analyze their data, and this does not help them. I
| know that stats isn't a cookbook, but we could put some guard
| rails down that help practitioners with their problems.
| okennedy wrote:
| Guardrails for stats are something that I've put a lot of
| thought towards. The fundamental challenge is that statistics
| is operating in a world of incomplete information. Statistical
| measures are almost never monotone with respect to new
| information, and so any new piece of context might completely
| invalidate an analysis. Going beyond the abstract requires an
| intimate knowledge of the domain being analyzed, and the
| limitations of statistical methods as applied to that domain.
| "Guardrails for stats" have to be domain- and even dataset-
| specific.
| biomcgary wrote:
| >Scientists and data analysts just want to know how to analyze
| their data, and this does not help them.
|
| I'm a computational biologist that uses Bayesian and
| Frequentist approaches depending on what I'm trying to achieve.
| This article was very helpful for making explicit a distinction
| I had not recognized before. With most of science (in my field)
| being done by people with Doctorates of Philosophy, I think it
| is reasonable to expect them to understand the underlying
| concepts of the math they are using. But, I'm anachronistic in
| wanting science to have a bit more natural philosophy rather
| than just "shut up and calculate"
| (https://en.wikiquote.org/wiki/Shut_up_and_calculate).
| enaaem wrote:
| The difference between Bayesian and Frequentist is in the
| interpretation of randomness. In Bayesian statistics 'randomness'
| is not a property of nature but a description of our knowledge.
|
| What's randomness in a coin toss? If we had all the information
| we could perfectly predict the result of a toss. But if we know
| nothing then at most we can say is that both outcomes are equally
| probable.
|
| Another example, if you had no idea who the next presidential
| winner will be between two candidates, than saying it's 50-50 is
| an accurate description of your knowledge.
|
| If anyone is more interested I would refer to you to [1]. Here,
| probability theory is interpreted as an extension of logic. Very
| interesting stuff.
|
| [1]
| http://www.med.mcgill.ca/epidemiology/hanley/bios601/Gaussia...
| angrais wrote:
| >> What's randomness in a coin toss? If we had all the
| information we could perfectly predict the result of a toss.
| But if we know nothing then at most we can say is that both
| outcomes are equally probable.
|
| Perhaps I'm being dense or overall don't understand, but how is
| this possible? What is "all the information"? Isn't it at most
| likely they the outcome is 50/50?
| analog31 wrote:
| >>> What's randomness in a coin toss? If we had all the
| information we could perfectly predict the result of a toss.
| But if we know nothing then at most we can say is that both
| outcomes are equally probable.
|
| That's because you know it's a coin toss. If it was something
| else like whether a seed will germinate or not, I wouldn't
| assume equal probability.
|
| Admittedly, this is something that's always puzzled me about
| Bayesian statistics, though I'm not sure it's fundamental.
| wirrbel wrote:
| Dunno whether I agree to this. I agree that both are acceptable
| ways to do statistics. However
|
| 1. Bayesian stats is an approach that tends to make model
| assumptions fairly explicity, whereas in frequentist approaches,
| many assumptions are fairly implicit (Normal distribution of
| data, etc.) 2. I would consider myself a Bayesianist but I am
| sceptical about too much mention of esoteric terminology like
| "Belief". Bayesian probabilities are probabilities following the
| Kolmogorov axioms, which is also the foundation of Frequentist
| stats.
|
| For decades, Bayesian inference was impractical because we need
| to resort to sampling methods and (a) computational power was
| insuffient and (b) we didn't have algorithms like No U-Turn
| Sampler (NUTS).
|
| Both aspects are 'solved', so why is Bayesianism not universally
| adopted? Of course it still has a reputational disadvantage, but
| I think more importantly its because
|
| * frequentist methods are good enough for purposes of publishing
| research [ _]_ for some problems we really have a hard time
| assembling bayesian graphs * some inference methods (e..g. Kalman
| filter) can both be seen as frequentist or Bayesian
|
| As a bayesianist I am amazed at how well frequentism can work,
| even when the 'traditional' way of applying it contradicts the
| derivations of founding fathers like Fisher/Pearson. It's almost
| as if we have an evolutionary process at play.
|
| [*] That is, if you use p-Values as publication thresholds
| NohatCoder wrote:
| Scientific publishing has largely gone off the rails, thanks in
| no small part to the frequentist p-value obsession. It is not
| good enough, people just use it anyway.
|
| I think most people want to avoid the dance of picking a prior,
| that is why frequentism is still so widespread.
| dang wrote:
| Discussed at the time:
|
| _What Is Bayesian /Frequentist Inference?_ -
| https://news.ycombinator.com/item?id=4800449 - Nov 2012 (27
| comments)
___________________________________________________________________
(page generated 2022-10-10 23:01 UTC)