[HN Gopher] The Shannon Limit (2010)
___________________________________________________________________
The Shannon Limit (2010)
Author : federicoponzi
Score : 60 points
Date : 2021-02-26 14:54 UTC (8 hours ago)
(HTM) web link (news.mit.edu)
(TXT) w3m dump (news.mit.edu)
| altcognito wrote:
| Previous discussion of the Shannon limit -- specifically the top
| end of analog lines:
|
| https://news.ycombinator.com/item?id=4344349
| dragontamer wrote:
| This MIT article is okay at an introduction to the subject.
|
| I found the following Youtube video to be better however:
| https://simons.berkeley.edu/events/theoretically-speaking-ma...
|
| As a ~1-hour talk, Dr. Wootters is able to dig more deeply into
| Reed Solomon codes (a very popular error-correction code for
| nearly 50 years), as well as applications into strange stuff:
| like using RS codes in "Test Pooling" to save money on Syphilis
| tests (and probably the same methodology being used for "Test
| Pooling" in today's COVID19 world).
|
| Dr. Wootters keeps things relatively dumbed down, never getting
| too into the weeds of the math (and indeed: only sticks with the
| GF(5) field, a prime field instead of talking about the more
| applicable extension fields). Still, extension fields follow
| mostly the same concepts, and GF(5) is sufficient to cover all
| the concepts.
| thisiscorrect wrote:
| I've never heard of Prof. Mary Wootters before. I wonder if
| there's any relation to Prof. William Wootters. Neither
| mentions that in their biography but it's not a terribly common
| name, and both seem to work at the intersection of physics and
| information theory.
| bob1029 wrote:
| Information theory is one of very few things out of academia that
| has really stuck with me and shaped how I look at problems in the
| world. The current applications for it are so vast and immediate
| that I wonder how many other places it could still be applied to.
|
| Being able say and practice things like "now let's consider what
| this might look like in the frequency domain..." can open up
| radical new approaches to solving problems. "Ohhhh that
| signal/channel has a hard roll-off starting at 20khz... I
| probably need to increase/decrease my sample rate accordingly" vs
| flying blind with traditional time domain analysis. In practical
| terms, frequency domain work is mandatory for things like high-
| efficiency video or audio codecs.
|
| Understanding what entropy vs information really means is a huge
| part of being able to competently build cryptographic primitives
| from first principles.
| amelius wrote:
| This sounds more like Linear Systems Theory than Information
| Theory, to be honest.
| teekert wrote:
| So how does this relate to Hamming codes? [0]
|
| [0] https://m.youtube.com/watch?v=X8jsijhllIA
| chrispeel wrote:
| Shannon's original article "A Mathematical Theory of
| Communication" is available online, for example at [1]; I find it
| very readable. The MIT article talks about information theory as
| applied to our digital world, but does not mention that it's now
| applied widely in theoretical physics; watch [2] for an intro.
| For example, the Hawking radiation from a black hole can function
| as an error correcting (erasure) code, allowing recovery of the
| information that fell into the black hole.
|
| [1]
| http://people.math.harvard.edu/~ctm/home/text/others/shannon...
|
| [2] https://www.youtube.com/watch?v=v5UbN0xx4X0
| thisiscorrect wrote:
| Information Theory is the ur-science. It's the theory of
| mapping observed data onto models. In other words, it's how to
| fuse inductive logic into deductive logic. In physics, this is
| especially salient, since many of the models are of low-
| information systems. See, e.g. Wootter's paper "Statistical
| Distance and Hilbert Space" [1], which gets pretty close to
| deriving the Dirac formulation fo quantum mechanics as the
| physics of low information systems in those explicit terms. (It
| doesn't seem to explain why it's a complex Hilbert space.)
|
| [1]
| https://journals.aps.org/prd/abstract/10.1103/PhysRevD.23.35...
| a1369209993 wrote:
| > It doesn't seem to explain why it's a complex Hilbert
| space.
|
| It's surprisingly difficult to motivate complex- (rather than
| real-) valued quantum mechanics. See eg
| https://www.scottaaronson.com/blog/?p=4021, for example.
|
| (As they point out, though, motivating "not quaternions" is
| pretty easy at least.)
| gnulinux wrote:
| > Information Theory is the ur-science.
|
| Eh, this can be said about a lot of foundational
| disciplines/studies/theories. Philosophy of science (or
| philosophy of mathematics), logic, category theory, model
| theory etc would all fit the description. I'm not saying
| you're wrong, it's just that information theory isn't unique
| in this respect.
| Fomite wrote:
| It's also widely applied in ecology and microbiome research -
| often people don't even realize its origins.
___________________________________________________________________
(page generated 2021-02-26 23:01 UTC)