Editing Entropy and Information (section)

== <span style="color: #FFFFFF;">Understanding</span> ==
Information and entropy are understood through '''Uncertainty''' and '''Predictability'''.

'''1. Information as "Surprise"''':
Shannon realized that if you already know what a message is going to say, it contains **zero** information.
* If I tell you "The sun will rise tomorrow," that is a 100% certain event. Surprise = 0, Information = 0 bits.
* If I flip a coin, there is a 50/50 chance. Surprise = 1, Information = 1 bit.
* Information is the "Resolution of Uncertainty."

'''2. Entropy (The Measure of Chaos)''':
Entropy (H) calculates the average "Surprise" of a whole system.
* If a coin is weighted and always lands on Heads, its entropy is 0.
* If a coin is fair, its entropy is at its maximum (1 bit).
* In language, some letters (like 'E' in English) are very common. They have low entropy. Rare letters (like 'Z' or 'Q') have high entropy because they are "Surprising."

'''3. The Absolute Limit''':
Shannon proved that Entropy is the "Speed Limit" of communication.
* You can never compress a file smaller than its entropy without losing data.
* You can never send data over a noisy wire faster than the "Channel Capacity."

'''The Bit''': Before Shannon, "Information" was a vague idea. He turned it into a physical substance—the bit—that could be measured, counted, and sold like electricity or water.
</div>

<div style="background-color: #8B0000; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">