[HN Gopher] Information Theory: A Tutorial Introduction
       ___________________________________________________________________
        
       Information Theory: A Tutorial Introduction
        
       Author : teleforce
       Score  : 172 points
       Date   : 2021-06-26 15:52 UTC (7 hours ago)
        
 (HTM) web link (arxiv.org)
 (TXT) w3m dump (arxiv.org)
        
       | jostmey wrote:
       | I think of information theory as an entirely new branch of
       | mathematics because it represents a new way of quantitatively
       | thinking. Calculus provided a different way to think about
       | problems separate from geometry. Information theory let's me
       | think about problems _differently_.
        
       | phonebucket wrote:
       | Another fantastic (and free!) Information Theory resource is
       | David MacKay's book, which also addresses aspects of Bayesian
       | Statistics and Machine Learning:
       | http://www.inference.org.uk/mackay/itila/
       | 
       | It's among my favourite textbooks: you can feel his enthusiasm
       | and personality through the pages. The world is a poorer place
       | without him around.
        
         | vmilner wrote:
         | See also his video course:
         | 
         | https://www.youtube.com/watch?v=BCiZc0n6COY
        
       | elromulous wrote:
       | Written some 60 years ago, Information Theory and Coding by
       | Abramson[1] is an absolute gem for those looking to get into info
       | theory. The cover book [2] being the more complete resource (and
       | somewhat of a grad level defacto standard text).
       | 
       | [1] https://www.amazon.com/Information-Theory-Coding-Norman-
       | Abra...
       | 
       | [2] https://www.amazon.com/Elements-Information-Theory-
       | Telecommu...
        
       | mcguire wrote:
       | " _To mistake a binary digit for a bit is a category error. In
       | this case, the category error is not as bad as mistaking marzipan
       | for justice, but it is analogous to mistaking a pint-sized bottle
       | for a pint of milk._ "
       | 
       | That's well put.
        
       | chrispeel wrote:
       | Shannon's original paper "A Mathematical Theory of Communication"
       | which introduced information theory is so accessible as to be
       | useful as a tutorial itself:
       | 
       | http://people.math.harvard.edu/~ctm/home/text/others/shannon...
        
         | Swizec wrote:
         | +1 for the original paper. I used it to pass my information
         | theory class in college because it made more sense than the
         | official materials and my lecture notes.
         | 
         | It's a fantastic explanation of the theory.
        
         | m-hilgendorf wrote:
         | The edition with Weaver's introduction is also very good,
         | especially if you would like the context and ramifications of
         | Shannon's theory.
         | 
         | https://pure.mpg.de/rest/items/item_2383164/component/file_2...
        
         | haolez wrote:
         | Except that some of the notation has changed a little. But it's
         | insanely good and impactful for a piece that is only 8 pages
         | long, if I remember correctly.
        
           | anon_tor_12345 wrote:
           | it's 55 pages
        
             | haolez wrote:
             | His master's thesis was much smaller, if I remember
             | correctly. Gonna check with my ex teacher :)
        
               | duped wrote:
               | His master's thesis was on applying boolean algebra to
               | the design of switching circuits, which was similarly
               | groundbreaking (although discovered in parallel by
               | another).
        
       ___________________________________________________________________
       (page generated 2021-06-26 23:00 UTC)