Probability Theory

How to read this page: This article maps the topic from beginner to expert across six levels � Remembering, Understanding, Applying, Analyzing, Evaluating, and Creating. Scan the headings to see the full scope, then read from wherever your knowledge starts to feel uncertain. Learn more about how BloomWiki works ?

Probability Theory is the branch of mathematics concerned with the analysis of random phenomena. While the outcome of a single event (like a coin flip) might be unpredictable, probability theory reveals the hidden patterns that emerge when an event is repeated many times. It is the mathematical framework for Uncertainty. From the insurance industry predicting risk to physicists calculating the position of an electron and AI models predicting the next word in a sentence, probability is the tool we use to navigate a world that is not deterministic.

Remembering

• Probability — A measure of the likelihood that an event will occur, ranging from 0 (impossible) to 1 (certain).
• Sample Space ($S$) — The set of all possible outcomes of an experiment.
• Event ($E$) — A subset of the sample space (e.g., "Rolling an even number").
• Independent Events — Events where the outcome of one does not affect the other (e.g., two coin flips).
• Dependent Events — Events where the outcome of one affects the likelihood of the other (e.g., drawing cards without replacement).
• Conditional Probability — The probability of an event occurring given that another event has already occurred ($P(A|B)$).
• Bayes' Theorem — A formula that describes how to update the probability of a hypothesis as more evidence becomes available.
• Random Variable — A variable whose value is determined by the outcome of a random experiment.
• Mean (Expected Value) — The "Average" outcome if an experiment is repeated infinitely.
• Variance / Standard Deviation — Measures of how "spread out" the outcomes are from the mean.
• Normal Distribution (Bell Curve) — A common probability distribution where most outcomes cluster around the center.
• Law of Large Numbers — The principle that as a sample size grows, its mean gets closer to the average of the whole population.
• Central Limit Theorem — The amazing fact that the sum of many independent random variables tends toward a normal distribution, regardless of the original distribution.

Understanding

Probability is understood through Frequency and Belief.

1. The Frequentist View: Probability is what happens in the long run. If you flip a coin 1 million times, it will be heads 50% of the time. $P(Heads) = 0.5$.

2. The Bayesian View: Probability is a "Degree of Belief." If I say there is a 70% chance of rain, I am expressing my confidence based on the current data. As soon as I see a dark cloud, I update my belief.

3. Distributions:

• Binomial: Used for "Yes/No" outcomes (e.g., How many people will click this ad?).
• Poisson: Used for events happening over time (e.g., How many emails will I get in an hour?).
• Normal: Used for natural traits (e.g., Height, IQ, Measurement errors).

The Gambler's Fallacy: The mistaken belief that if something happens more frequently than normal during a given period, it will happen less frequently in the future (and vice versa). If you flip 5 heads in a row, the 6th flip is still 50/50. The "Universe" has no memory.

Applying

Modeling 'Bayesian Inference' (Spam Filtering): <syntaxhighlight lang="python"> def update_probability(prior_prob, prob_evidence_given_spam, prob_evidence_given_ham):

   """
   Bayes' Theorem: P(Spam|Word) = [P(Word|Spam) * P(Spam)] / P(Word)
   """
   # P(Word) = P(Word|Spam)P(Spam) + P(Word|Ham)P(Ham)
   p_ham = 1 - prior_prob
   p_word = (prob_evidence_given_spam * prior_prob) + (prob_evidence_given_ham * p_ham)
   
   posterior_prob = (prob_evidence_given_spam * prior_prob) / p_word
   return posterior_prob

1. Prior: 10% of emails are spam.
2. 'Buy Now' appears in 80% of spam but only 1% of ham.

print(f"Prob it is spam if it says 'Buy Now': {update_probability(0.1, 0.8, 0.01):.2f}")

1. This 'Learning' logic is how your email filter gets smarter.

</syntaxhighlight>

Probability Paradoxes

The Monty Hall Problem → Why you should always "switch doors" on a game show (it doubles your chances!).

The Birthday Paradox → In a room of just 23 people, there is a 50% chance two share a birthday.

Simpson's Paradox → A trend appears in several different groups of data but disappears or reverses when these groups are combined.

Analyzing

Combinations vs. Permutations

Feature	Permutation	Combination
Order Matters?	Yes (ABC != CBA)	No (ABC == CBA)
Example	Entering a safe code	Picking a team of 3 people
Formula	$n! / (n-r)!$	$n! / [r!(n-r)!]$
Complexity	Higher (More possibilities)	Lower (Fewer possibilities)

The Concept of "Expectation": Insurance companies don't care about one person's accident; they care about the Expected Value of 1 million people. If they charge $100 and the average payout is $80, they are guaranteed to make a profit due to the Law of Large Numbers. Analyzing these "Expected Values" is how casinos stay in business.

Evaluating

Evaluating a probability claim:

1. Sample Size: Is the "100% success rate" based on 2 people or 2,000?
2. Independence: Are you assuming events are independent when they are actually linked (e.g., multiple stock market crashes)?
3. Selection Bias: Did you only look at the data that proved your point?
4. Base Rate Fallacy: If a test is 99% accurate for a disease that only 1 in 10,000 people have, a "Positive" result is actually more likely to be a false alarm.

Creating

Future Frontiers:

1. Quantum Probability: Dealing with the "True" randomness of the universe at the subatomic level.
2. Algorithmic Information Theory: Using probability to define how much "Information" or "Complexity" is in a piece of data.
3. Stochastic AI: Building neural networks that don't just give one answer, but a "Probability Distribution" of possible answers.
4. The End of Randomness: The theoretical debate over whether "Randomness" actually exists, or if we just lack the data to predict it perfectly.