[HN Gopher] Bayes Rules - An Introduction to Applied Bayesian Mo...
___________________________________________________________________
Bayes Rules - An Introduction to Applied Bayesian Modeling (2021)
Author : RafelMri
Score : 126 points
Date : 2022-07-17 11:09 UTC (11 hours ago)
(HTM) web link (www.bayesrulesbook.com)
(TXT) w3m dump (www.bayesrulesbook.com)
| turbocon wrote:
| I've come across this format a couple times now, does anyone know
| how to export it to PDF?
|
| I honestly don't even mind paying for them but I want the flat
| file.
| t-3 wrote:
| Pandoc + wget
|
| wget -r https://www.bayesrulesbook.com/ && cd
| www.bayesrulesbook.com && pandoc --pdf-engine=xelatex
| foreword.html preface.html chapter-{1..19}.html references.html
| -o book.pdf
| mkl wrote:
| For me that repeats the table of contents many times, and
| leaves the maths as Latex code instead of formatting it as
| formulas.
| platz wrote:
| The hardest part of bayes is
|
| 1) understanding what 'likelihood' actually is or represents
|
| 2) understanding what a 'partition function' actually is or
| represents
|
| I always forget what these mean since I don't use bayes in anger
|
| Also the above reveal there is more elaborate structure in bayes
| than just the idea of 'updating your prior with new information',
| which is simply a platitude among STEM folk
| jltsiren wrote:
| The hard part is that the intuitive understanding of
| probability you learned as a kid is almost certainly wrong. You
| need to unlearn it and replace it with proper axiomatic
| treatment of probability based on measure theory. Once you have
| done that and built a new intuition, Bayesian thinking should
| feel pretty natural. Once you accept that probability is just
| the "size" of a set relative to the size of a superset and it's
| up to you to attach meaning to the sets, much of the confusion
| goes away.
| nerdponx wrote:
| I think you can get most of the way there with
| straightforward geometric intuition, treating the sample
| space as a rectangle of area 1 and proceeding from there into
| joint and conditional probability.
| oxff wrote:
| The 'hardest' part is realizing that basically everyone reasons
| this way unless taught otherwise. The books do a lot of hard
| work to cover this up.
| MontyCarloHall wrote:
| Likelihood: given a probabilistic model and its parameters,
| what is the probability of observing some data under that
| model?
|
| For example, given a coin with some probability _f_ of getting
| heads and thus 1- _f_ of getting tails (model parameter), a set
| of coin flips (data), and the assumption that each coin flip is
| independent of all others (model), the probability of seeing
| _an arbitrary_ sequence with _H_ total heads and _T_ total
| tails after _H_ + _T_ flips is
|
| p( _H_ , _T_ | _f_ ) = _f_ ^ _H_ (1-f)^ _T_ [0]
|
| Note that this likelihood is normalized (i.e. sums up to 1)
| with respect to all possible sequences of flips of length _H_ +
| _T_ , e.g. for length 2, there are 4 possible sequences of coin
| flips:
|
| p( _hh_ |f) + p( _ht_ |f) + p( _th_ |f) + p( _tt_ |f) = _f_ ^2
| + 2 _f_ (1- _f_ ) + (1- _f_ )^2 = 1
|
| (Also note that the likelihood is only a probability for
| discrete data; it's a _probability density_ for continuous
| data, since the underlying terms would no longer be
| probabilities but rather densities. In that case, replace the
| previous sum with an integral ranging over the entire domain of
| your continuous data.)
|
| Partition function (usually called a "marginal likelihood"):
| what if we instead normalize the likelihood with respect to the
| model parameters? Then it would no longer be a probability
| distribution with respect to the data, but rather a probability
| distribution with respect to the parameters. The marginal
| likelihood is just this normalizing constant. In the previous
| example,
|
| p( _f_ | _H_ , _T_ ) = p( _H_ , _T_ | _f_ )/<constant that
| would normalize p( _H_ , _T_ | _f_ ) to integrate to 1 over all
| possible values of _f_ >
|
| You can optionally weight this likelihood by a prior p( _f_ ),
| in which case the numerator would be p( _H_ , _T_ | _f_ )p( _f_
| ), with the partition function updated accordingly.
|
| >Also the above reveal there is more elaborate structure in
| bayes than just the idea of 'updating your prior with new
| information', which is simply a platitude among STEM folk
|
| I could not agree more. The core philosophical tenet of
| Bayesian inference is that this re-normalization of the
| likelihood returns something probabilistically meaningful. The
| debate over the validity of priors pales in comparison to the
| debate over whether the likelihood ought to be treated as
| proportional to a probability distribution over the model
| parameters.
|
| [0] Note that this is distinct from the probability of seeing
| _any_ sequence with _H_ heads and _T_ tails. That would require
| normalizing with respect to the number of total sequences with
| _H_ heads and _T_ tails ( _H_ + _T_ choose _H_ ), yielding the
| binomial distribution.
| lisper wrote:
| No, the hardest part is realizing that there are all kinds of
| tacit assumptions that we bring to bear merely by _formulating_
| a problem for Bayesian analysis.
|
| Take the example used in chapter 2 of the book. It assumes that
| news articles can be classified as "real" or "fake", and there
| is no middle ground. It assumes that the initial prior produced
| by experts is reliable. And most of all it assumes that certain
| features, like exclamation points in the title, are causally
| related to realness or fakeness.
|
| To illustrate this last point, consider analyzing the titles
| for a different features, the presence of the letter "z" rather
| than the presence of exclamation points. If it turned out that
| fake news in the corpus used to generate the priors just turned
| out to have more zees in their titles, would you then be
| justified in concluding that a news article about Zanzibar was
| more likely to be fake because it contained two zees?
|
| That example might seem contrived, but if you use a Bayesian
| spam filter it is actually plausible that the presence of the
| letter "v" is dignostic of spam because of the prevalence of
| spam concerning viagra. But again, there is a _causal_ model of
| this: viagra is a product that is often the subject of spam
| marketing, and the word "viagra" happens to have a v in it,
| which is a priori an uncommon letter in English.
|
| But all this can fall apart depending on the circumstances. If
| you one day joined an email discussion of Stradavarius violins,
| the v signal could suddenly fail. An even more dramatic
| example: suppose you are an academic who starts to do research
| on spam filters and you have a collaborator who starts to send
| you examples of hard-to-filter spam. Now you have a _very_
| strong signal, but no straightforward textual analysis will
| allow you to extract it.
|
| The hard part of Bayesian analysis is deciding what features to
| even look at. All the rest is borderline trivial by comparison.
| bumby wrote:
| > _the hardest part is realizing that there are all kinds of
| tacit assumptions that we bring to bear merely by formulating
| a problem for Bayesian analysis._
|
| I think the common argument is this is a strength of Bayesian
| analysis. Namely, that your priors make you explicitly state
| your assumptions. All models integrate assumptions, but not
| all of them make you explicitly quantify them like Bayesian
| analysis does.
| nerdponx wrote:
| This was my experience as well, and it's why I recommend
| learning Bayesian stats to any student of data analysis and
| statistics. I find that learning how to set up Bayesian
| models has a strong elucidating effect on model-building in
| general.
| nequo wrote:
| > would you then be justified in concluding that a news
| article about Zanzibar was more likely to be fake because it
| contained two zees?
|
| I completely agree with you about hidden assumptions. But I
| don't think that causal thinking is one of the problems here.
|
| Bayes' rule applies whether or not the relationship between X
| and Y is causal. Bayes' rule is a predictive model and
| prediction does not need causation.
|
| The problem in the Zanzibar example is that the model is
| misspecified. Or at least the model specification does not
| represent how you as a human being with cultural knowledge
| think about the problem. You look at Zanzibar and see an
| island in Tanzania. Your model looks at Zanzibar and sees two
| a's and two z's and a few other letters. So it is the wrong
| implicit assumption about the underlying model that's giving
| you "non-sensical" results.
| lisper wrote:
| > the model is misspecified... it does not represent how
| you as a human being with cultural knowledge think about
| the problem
|
| Yes, that is exactly my point. The way "you as a human with
| cultural knowledge think about the problem" is a major
| component of Bayesian analysis. The math is almost
| incidental. By the time you have chosen what features to do
| the math on, most of the heavy lifting has already been
| done.
|
| This matters because if you just crunch the Bayesian
| numbers on an arbitrary data set you will almost certainly
| find features that appear predictive but are not. The more
| numbers you crunch, the more likely this is to happen. A
| lot of people lose money in the stock market this way.
| nequo wrote:
| Much agreed. Data analysis needs a good theory.
| [deleted]
| analog31 wrote:
| >>> ... the idea of 'updating your prior with new information'
| ...
|
| ... dates back to antiquity. I'm not a statistician, but my
| impression is that Bayesian methods are a way of formalizing
| that approach. On the other hand, use of the term "prior" is
| somewhat misleading because the formulas do not specify a time
| sequence for acquiring information.
|
| I tend to have a lesser view of sprinkling Bayesian terminology
| into blogs about social issues. That's what I call _Bayes
| Theater_.
| DiogenesKynikos wrote:
| > understanding what 'likelihood' actually is or represents
|
| The probability of observing the data you observed, assuming
| that the model parameters take on a certain value.
|
| > understanding what a 'partition function' actually is or
| represents
|
| The probability of observing the data you observed, this time
| averaging over all possible parameter values (weighted by the
| priors).
| psi75 wrote:
| > The probability of observing the data you observed
|
| Yes, in discrete cases. In continuous cases, you have to work
| with a probability _density_. I think this is one of the
| hurdles people encounter when they 're first exposed to
| Bayesian stats. The probabilities, technically speaking, are
| zero.
|
| The important insight in Bayesian work is that it's often not
| the probabilities themselves that matter but the ratios
| thereof, since from those alone you can compute posteriors.
| DiogenesKynikos wrote:
| Indeed, but it's a bit tedious to always say,
| "probabilities (in the discrete case) or probability
| densities (in the continuous case)."
|
| In general, you have sums in the discrete case and
| integrals in the continuous case, but most formulas are
| otherwise the same.
| psi75 wrote:
| That's quite true. Also, one could argue that continuous
| probabilities in practice are discrete probabilities due
| to finite resolution--we just don't care to specify what
| the resolution is.
| analog31 wrote:
| I once had a TA job for an undergrad stats course. This
| was the "non calculus" course for the psych majors. I had
| also taken the "math" version of the same course, where
| we spent two semesters and proved everything. I honestly
| never came up with a satisfactory layman's explanation
| why continuous distributions are necessary, or what
| "continuous" is. I knew that we used calculus to derive
| the formulas that they were faced with memorizing, but
| that would have been irrelevant to them.
|
| The best explanation I can think of today is: Use the one
| that makes the math easier or more readable.
| melling wrote:
| I thought likelihood doesn't necessarily sum to one so it's
| not a probability.
| MontyCarloHall wrote:
| It will always sum (or integrate) to one with respect to
| the data. For example, given likelihood p(x1, x2, ...,
| x_N|params), summing (or integrating) over all possible
| values of x_1 ... x_N will indeed yield 1.
| master_yoda_1 wrote:
| I always get confused who the audience is for these kind of
| books?
|
| For advanced reader this is really shallow, for beginners its
| really advanced (who really want to read 543 page book as
| beginner).
| siddboots wrote:
| Simply put, there are many people who are neither "beginners"
| nor "advanced". There is a section of the book titled
| _Audience_ which reads
|
| > Bayes Rules! brings the power of Bayes to advanced
| undergraduate students and comparably trained practitioners.
| Accordingly, the book is neither written at the graduate level
| nor is it meant to be a first introduction to the field of
| statistics.
| lemursage wrote:
| The scope seems pretty standard but I really like strongly
| structured form and exercises. Is there a pdf version available
| perchance?
| [deleted]
| _plg_ wrote:
| Review by Christian Robert, a statistician:
| https://xianblog.wordpress.com/2022/07/05/bayes-rules-book-r...
| nerdponx wrote:
| This is a really useful book review, thank you for sharing it
| (and thank you to the author for writing it!).
___________________________________________________________________
(page generated 2022-07-17 23:01 UTC)