How Claude Shannon’s Concept of Entropy Quantifies Information | Kevin Hartnett in Quanta Magazine

INFORMATION THEORY

How Shannon Entropy Imposes Fundamental Limits on Communication

What’s a message, really? Claude Shannon recognized that the elemental ingredient is surprise.

Kevin Hartnett

Contributing Writer

September 6, 2022

VIEW PDF/PRINT MODE

Abstractions blog computer science explainers information theory mathematics All topics

If someone tells you a fact you already know, they’ve essentially told you nothing at all. Whereas if they impart a secret, it’s fair to say something has really been communicated.How Shannon Entropy Imposes Fundamental Limits on Communication
How Claude Shannon’s Concept of Entropy Quantifies Information | Quanta Magazine