INFORMATION THEORY
How Shannon Entropy Imposes Fundamental Limits on Communication
What’s a message, really? Claude Shannon recognized that the elemental ingredient is surprise.
Kevin Hartnett
September 6, 2022
Abstractions blogcomputer scienceexplainersinformation theorymathematicsAll topics
If someone tells you a fact you already know, they’ve essentially told you nothing at all. Whereas if they impart a secret, it’s fair to say something has really been communicated.How Shannon Entropy Imposes Fundamental Limits on Communication
How Claude Shannon’s Concept of Entropy Quantifies Information | Quanta Magazine