How Shannon Entropy Imposes Fundamental Limits on Communication
What’s a message, really? Claude Shannon recognized that the elemental ingredient is surprise.
September 6, 2022
If someone tells you a fact you already know, they’ve essentially told you nothing at all. Whereas if they impart a secret, it’s fair to say something has really been communicated.How Shannon Entropy Imposes Fundamental Limits on CommunicationHow Claude Shannon’s Concept of Entropy Quantifies Information | Quanta Magazine