The Science of Information: From Language to Black Holes
Entropy and the Average Surprise
S1 E4:
Intuition says we measure information by looking at the length of a message. But Shannon's information theory starts with something more fundamental: how surprising is the message? Through illuminating examples, discover that entropy provides a measure of the average surprise.