How Claude Shannon’s Concept of Entropy Quantifies Information | Kevin Hartnett in Quanta Magazine

INFORMATION THEORY

How Shannon Entropy Imposes Fundamental Limits on Communication

What’s a message, really? Claude Shannon recognized that the elemental ingredient is surprise.

Kevin Hartnett

Contributing Writer


September 6, 2022


VIEW PDF/PRINT MODE

Abstractions blogcomputer scienceexplainersinformation theorymathematicsAll topics

If someone tells you a fact you already know, they’ve essentially told you nothing at all. Whereas if they impart a secret, it’s fair to say something has really been communicated.How Shannon Entropy Imposes Fundamental Limits on Communication

How Claude Shannon’s Concept of Entropy Quantifies Information | Quanta Magazine