[HN Gopher] Claude Shannon: Tinkerer, Prankster, and Father of I...
___________________________________________________________________
Claude Shannon: Tinkerer, Prankster, and Father of Information
Theory (2016)
Author : phonebucket
Score : 165 points
Date : 2021-05-24 05:30 UTC (17 hours ago)
(HTM) web link (spectrum.ieee.org)
(TXT) w3m dump (spectrum.ieee.org)
| tekstar wrote:
| A really good book is "The Idea Factory" by Jon Gertner. It
| describes Bell Labs in this period, and through the book
| basically tells the story of where modern life was born:
| everything from growing crystals to transistors to cell phones to
| radar was created at Bell Labs.
| krosaen wrote:
| An interesting tidbit from Hamming in The Art of Doing Science
| and Engineering [1][2] on the name "Information Theory"
|
| > Information Theory was created by C.E.Shannon in the late
| 1940s. The management of Bell Telephone Labs wanted him to call
| it "Communication Theory" as that is a far more accurate name,
| but for obvious publicity reasons "Information Theory" has a much
| greater impact--thus Shannon chose and so it is known to this
| day.
|
| [1] https://press.stripe.com/#the-art-of-doing-science-and-
| engin... [2] http://worrydream.com/refs/Hamming-
| TheArtOfDoingScienceAndEn...
| ansible wrote:
| Curiosity Stream has a recent video centering around a
| fictionalized interview with Claude Shannon:
|
| https://curiositystream.com/video/3831
|
| If you sign up through a YouTube promotional link, you also get
| access to Nebula for free:
|
| https://curiositystream.com/legaleagle/index.html
| Wistar wrote:
| Here's a very so-so promo site for The Bit Player. I saw it
| about a year ago (Curiosity Stream) and thought it was pretty
| decent. I somewhat sheepishly admit that, for a while, I
| thought that they had somehow interviewed the real Claude
| Shannon.
|
| https://thebitplayer.com/
|
| Although not devoted to Claude Shannon, Jim Al-Khalili's
| documentary "The Story of Information" does discuss Shannon's
| significant contributions and is how I first became aware of
| Claude Shannon.
|
| https://topdocumentaryfilms.com/story-information/
| sillysaurusx wrote:
| Shannon was cursed by being the inventor of information theory.
| It seems like this wonderful thing; who wouldn't want to invent
| information theory? But as an inventor, you'll never top it.
| You'll forever live in your own shadow. And for someone that
| enjoys inventing things, that's... not a happy thing.
| mycologos wrote:
| Nothing I've read about Shannon suggests that he felt "cursed"
| in his later life. Is there some reason to think otherwise, or
| is this just assuming that anybody who does something big early
| feels sad about it later?
| jecel wrote:
| In Shannon's case, given how fundamental his 1937 thesis was
| (you can use Boolean Algebra to design digital circuits),
| most other people might have been afraid of not being able to
| top that. My impression was that he was always looking
| forward to bigger things through his own work and through his
| students.
|
| https://en.wikipedia.org/wiki/A_Symbolic_Analysis_of_Relay_a.
| ..
| nathias wrote:
| 'shannonigans'
| Shatnerz wrote:
| I recommend reading "A Mind at Play", a biography about Shannon.
| I found it to be a decent read. Funnily enough, I knew about him
| more through the unicycling community than I did through computer
| science. It wasn't until a few years ago when I realised what
| sort of impact he had on the world.
| idatum wrote:
| I also enjoyed "A Mind at Play".
|
| I had forgotten (or never fully appreciated) what Shannon did
| and what other early pioneers did in computing. I have a better
| appreciation for all of them now.
|
| One other highlight was Shannon's early childhood and using
| electrified fencing to communicate with others in his farmland
| neighborhood.
| ShaneCurran wrote:
| Many know Shannon as the author of "A Mathematical Theory of
| Communication" and the founder of information theory, as well as
| the brain behind "Communication Theory of Secrecy Systems"
|
| Both papers derive from a technical report, "A Mathematical
| Theory of Cryptography"[0], written by Shannon in 1945 -- 3 years
| before his breakthrough papers in the Bell Labs Technical
| Journal.
|
| [0]: https://evervault.com/papers/shannon.pdf
| vieclamdailoan wrote:
| realy?
| legohead wrote:
| another recommended book: The Idea Factory: Bell Labs and the
| great age of american innovation.
|
| Includes a lot of interesting information, including Claude
| Shannon of course, who worked at Bell Labs. Bell Labs was a
| remarkable place.
| noneeeed wrote:
| Funny timing. Tim Harford's Cautionary Tales podcast had an
| episode about Shannon. He sounds like a fun guy to have been
| around.
|
| https://timharford.com/2021/05/cautionary-tales-fritterin-aw...
| phonebucket wrote:
| The timing is a little less funny if you know that I listened
| to the podcast shortly before finding the IEEE article.
|
| The podcast was great! I would think that many on HN would find
| it interesting.
| vaughandroid wrote:
| Cautionary Tales is my favourite podcast. Most episodes
| concern either interesting (mis)use of statistics or
| cognitive biases, both of which are topics I find inherently
| interesting, but Tim Harford and team do a brilliant job of
| the storytelling too.
|
| Tim Harford's other podcasts are great too - More or Less,
| and 50 Things Which Made the Modern Economy.
|
| Does anyone have any recommendations along similar lines?
| doublejay1999 wrote:
| There's a passable movie called "the bit player" currently on
| amazon and I guess elsewhere.
|
| I'd never heard of him, and that's embarrassing, but he
| probably prefers it that way :-).
| JoeDaDude wrote:
| Claude Shannon was also an early pioneer in artificial
| intelligence, the minimax algorithm applied to game trees was one
| of his inventions. He built several machines to play games and
| demonstrate his ideas, some of which can be seen in the article
| photos.
|
| I compiled a list of his game gadgets, most of which still exist
| in the care of the MIT museum.
|
| https://boardgamegeek.com/geeklist/143233/claude-shannon-man...
| op03 wrote:
| From what I understand, before Shannon showed up no one knew how
| to calculate the Channel Capacity of a medium and how to deal
| with noise in the medium.
|
| So people just yelled louder or yelled repeatedly to get things
| across with better reliability. And when channel capacity was
| exceeded with everyone yelling too much, no one knew it had
| exceeded and lot of energy and time was wasted with increasing
| errors in the system.
|
| Doesn't that feel like a repeat with whats happening in social
| media and news media these days? Or is it just different things.
| rusk wrote:
| I think it's a different thing but an interesting analog. In
| conventional information theory you can add another channel. I
| guess the social media metaphor is like crowded wifi or if I'm
| being clever alohanet... question is, how do we add another
| channel here?if you're talking about social discourse rather
| than a physical medium are we talking about the necessity of
| class hierarchy? Is this what the Tower of Babel was about?
| TeMPOraL wrote:
| > _Doesn 't that feel like a repeat with whats happening in
| social media and news media these days? Or is it just different
| things._
|
| Maybe. It can feel like this, but applying mathematical models
| to social phenomena is tricky. You have to map formal
| parameters to fuzzy, poorly understood factors. How do you
| define, precisely, what is a "channel" on a social network?
| What is "signal", and what is "noise"? People can and do prove
| anything by using slightly different (or inconsistent)
| definitions here, getting the numbers to line up just like they
| want them to.
|
| My understanding is that a better way to approach such mapping
| would be to shove in probability distributions in place of
| hard-to-map exact parameters - abstracting away choices and
| measurements lets you see the wider context here. This pushes
| the problem to defining the appropriate distributions, but I
| think that's more tamper-proof. Unfortunately, the results may
| come out next to useless - e.g. probability distributions so
| wide you could sail a carrier strike group through them.
|
| I'd love to know what's considered the correct, robust approach
| to such problems.
| sig-reduction wrote:
| I think OP has a good point. Although language if fuzzy, so
| are "binary" signals once you drill down to the physical
| layer. The fuzziness of language goes away if you allow
| yourself to assume an idealized language embedding model.
| Similar ideas are in an epsilon ball around the embedding of
| some concrete linguistic instantiation of the idea.
|
| An example of a binary signal: "It is unambiguously true that
| a lab leak did not occur in Wuhan." The spread of this binary
| signal could be measured using the technique described above.
| A BERT embedding may suffice.
|
| The channels are just tweets, DMs, and Vox articles.
| JoeDaDude wrote:
| Your analogy doesn't sound far off from what information theory
| describes. Still, we need to be cautious of applying
| information theory to areas in which it might be over extended
| and may not be suitable. Claude Shannon himself was aware of
| this trend and wrote his essay "The Bandwagon" [1] urging
| caution and diligence in applying information theory to
| different fields.
|
| [1].
| https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=1056774
| gugagore wrote:
| One key insight of Shannon's is that to study information as he
| did, you need to remove "meaning". You could measure the
| information content of anything and it doesn't matter what the
| symbols mean. Paradoxically, a random sequence of characters of
| the same length as this message would contain more information
| than this message that I am writing.
|
| ---
|
| The connection you draw to social media is cute, and I think it
| could help as an analogy in some cases. The Bit Player (2018
| movie) makes a similar cute analogy to robust communication in
| marriage, that rather than saying the same message louder
| (yelling) or repeating the message (nagging), you should find
| many _different_ ways to say the _same_ message. And you could
| add an error-correcting code like "I love you".
|
| I personally think it's a cute (not disparaging) way to think
| about communication among humans, but note that we still need
| to rely heavily on _meaning_ , to decide what it even means to
| say the same message with different words.
|
| What goes on with social media can be seen through lenses such
| as decision theory, and game theory, and graph theory, ..., and
| I think those fields have more to offer from the get-go than
| information theory. In any case, I personally would lean
| heavily on social science to try to get a better understanding
| of social situations, and to consider possible "interventions".
|
| In particular, I do not think the kinds of communication errors
| that occur between humans are the same as e.g. bit flips. And
| to go back to the original point, it's tempting to conflate
| "information" (as in information theory) with "knowledge", or
| "understanding", or "truth".
| cmehdy wrote:
| Now we need an analog of Shannon in social sciences to offer
| a Theory of Meaning. Then perhaps a century or so where there
| are still incompatibilities with the theory of information
| that prevent a bigger picture from emerging, and eventually
| an all-encompassing theory ushering humanity into an Era of
| Meaning.
|
| Ted Chiang could probably write a plausible future with that
| as a writing prompt.
| CaptArmchair wrote:
| > Now we need an analog of Shannon in social sciences to
| offer a Theory of Meaning.
|
| That's called semiotics and it was first developed in the
| late 19th century by Ferdinand de Saussure.
|
| > Saussure approaches theory of language from two different
| perspectives. On the one hand, language is a system of
| signs. That is, a semiotic system; or a semiological system
| as he himself calls it. On the other hand, a language is
| also a social phenomenon: a product of the language
| community.
|
| > One of Saussure's key contributions to semiotics lies in
| what he called semiology, the concept of the bilateral
| (two-sided) sign which consists of 'the signifier' (a
| linguistic form, e.g. a word) and 'the signified' (the
| meaning of the form). Saussure supported the argument for
| the arbitrariness of the sign although he did not deny the
| fact that some words are onomatopoeic, or claim that
| picture-like symbols are fully arbitrary. Saussure also did
| not consider the linguistic sign as random, but as
| historically cemented.[a] All in all, he did not invent the
| philosophy of arbitrariness, but made a very influential
| contribution to it.
|
| > After his death, structural and functional linguists
| applied Saussure's concept to the analysis of the
| linguistic form as motivated by meaning. The opposite
| direction of the linguistic expressions as giving rise to
| the conceptual system, on the other hand, became the
| foundation of the post-Second World War structuralists who
| adopted Saussure's concept of structural linguistics as the
| model for all human sciences as the study of how language
| shapes our concepts of the world. Thus, Saussure's model
| became important not only for linguistics, but for
| humanities and social sciences as a whole.
|
| https://en.wikipedia.org/wiki/Ferdinand_de_Saussure
| [deleted]
| swiley wrote:
| I think the social media issue has more to do with the heavy
| application of ML curation to feeds and ads creating feedback
| loops where people are really talking to themselves using a
| computer rather than other people who might confront them.
| steve76 wrote:
| Information, as in interaction of subatomic particles. No
| analogy to the bad habits and lack of culture to humans on a
| propped up bailed out market.
| lqet wrote:
| > In 1973, he recalled, he persuaded Shannon to give the first
| annual Shannon lecture at the International Information Theory
| Symposium, but Shannon almost backed out at the last minute. "I
| never saw a guy with so much stage fright," Berlekamp said. "In
| this crowd, he was viewed as a godlike figure, and I guess he was
| worried he wouldn't live up to his reputation."
|
| > In 1985 he made an unexpected appearance at the International
| Information Theory Symposium in Brighton, England. The meeting
| was proceeding smoothly, if uneventfully, when news raced through
| the halls and lecture rooms that the snowy-haired man with the
| shy grin who was wandering in and out of the sessions was none
| other than Claude Shannon. Some of those at the conference had
| not even known he was still alive.
|
| > At the banquet, the meeting's organizers somehow persuaded
| Shannon to address the audience. He spoke for a few minutes and
| then--fearing that he was boring his audience, he recalled later
| --pulled three balls out of his pockets and began juggling. The
| audience cheered and lined up for autographs.
| harveywi wrote:
| > At the banquet, the meeting's organizers somehow persuaded
| Shannon to address the audience. He spoke for a few minutes and
| then--fearing that he was boring his audience, he recalled
| later--pulled three balls out of his pockets and began
| juggling.
|
| Shannon was famous for his entropy.
| 1cvmask wrote:
| The alleged origins of the naming of entropy:
|
| When Shannon first derived his famous formula for
| information, he asked von Neumann what he should call it and
| von Neumann replied "You should call it entropy for two
| reasons: first because that is what the formula is in
| statistical mechanises but second and more important, as
| nobody knows what entropy is, whenever you use the term you
| will always be at an advantage!
|
| From:
|
| http://www.spatialcomplexity.info/what-von-neumann-said-
| to-s...
| 2snakes wrote:
| Yeah, I would characterize it as a mathematical theory of
| -symbols- and that information has different definitions.
| However, the argument can be made that symbolic
| representation is a principle of reality.
| ajarmst wrote:
| I don't think you can meaningly disentangle "information"
| and its representation without doing violence to the
| underlying theory. Nor is a discussion of "symbols"
| coherent without a reference to the information encoded.
| Grounding the idea of information in the logical and
| physical limitations of representation is part of the
| genius of the theory, not an unfortunate epiphenomenon.
| llaolleh wrote:
| Shannon was awesome. It seemed like after he wrote his
| groundbreaking information theory paper he did whatever the hell
| he wanted for the rest of his life in Cambridge. Made all kinds
| of contraptions and devices.
___________________________________________________________________
(page generated 2021-05-24 23:02 UTC)