Motivation§

We want to develop a mathematical formulation of probability

Sample Space

A sample space: $\Omega,$ the set of all possible outcomes of a random experiment.

Sample Point

A point $\omega \in \Omega$ is called a sample point.

Events

A subset $S \subseteq \Omega$ are called events, and $\mathcal{F}$ is the set of all events of $\Omega,$ i.e. $\mathcal{F} = \mathcal{P}(\Omega).$

Thus, a set $S$ containing sample points $\omega$ of $\Omega$ is an event of the sample space.

Kolmogorov Axioms§

Probability Measure (Distribution)

A probability measure is a functor $P: \mathcal{F} \to \mathbb{R}^{+}$

The Kolmogorov axioms on this measure are:

  1. the measure of an event is in the interval $[0, 1]$, $$ \forall E \in \mathcal{F}, P(E) \in [0, 1]. $$

  2. the probability of at least one event of the sample space occuring is 1, $$ P(\Omega) = 1. $$ Also note that $P(\varnothing) = 0.$

  3. given a countable sequence of pairwise disjoint sets $a_1, a_2, \dots$, then the following is true: $$ P\left(\bigcup_{i=1}^{\infty}a_i\right) = \sum_{i=1}^{\infty}P(a_i). $$

Probability Space

A probability space is defined by the triple

$$ (\Omega, \mathcal{F}, P). $$

Pairwise Disjoint

Given $a_i, a_j \subset A$ and $i \neq j$, if $a_i \cap a_j = \varnothing$, then we say that $a_i$ and $a_j$ are pairwise disjoint.

Coin Flip

Assume we flipped a fair coin, we have that

$$ \Omega = \{H, T\} $$ $$ \mathcal{F} = \left\{\varnothing, \{H\}, \{T\}, \Omega\right\}. $$

Then, $P(\{H\}) = P(\{T\}) = \frac{1}{2}$. Note that $P(\{H, T\}) = 1$ due to axiom 2.

Note: We often omit the curly braces when writing the probability of events, e.g. $P(\{H\})$ would often be written as $P(H).$

Rolling a Die

Assuming a fair die, $\Omega = \{1, 2, 3, 4, 5, 6\}$ and $\mathcal{F} = \mathcal{P}(\Omega)$ (there are 64 elements in $\mathcal{F}$, we need not enumerate all cases).

Let $A$ be the set of outcomes where the digit is odd, i.e. $A = \{1, 3, 5\}.$ Each outcome of the die is equally likely, i.e. $P(1) = P(2) = \dots = P(6) = \frac{1}{6}.$

Then, $P(A) = P(1) + P(3) + P(5) = \frac{1}{2},$ due to axiom 3.

The probability measure $P$ contains assumptions and beliefs of the model.

Unfair Coins or Loaded Dice

Sps. we had a biased coin where $P(H) = 3P(T),$ we also know that $P(H) + P(T) = 1.$

  • Then, $P(H) = \frac{3}{4}$ and $P(T) = \frac{1}{4}.$

Sps. we had a loaded die such that $P(6)$ is twice as likely as any other number.

  • Then, $P(6) = \frac{2}{7}$ and $P(1) = P(2) = \dots = P(5) = \frac{1}{7}.$

Sps. an otherwise fair die but where one number is replaced with another, e.g. 5 replaced with 6.

  • We define a new probability measure $\tilde{P},$ where $$ \tilde{P}(1) = \dots = \tilde{P}(4) = \frac{1}{6}, $$

    $$ \tilde{P}(6) = \frac{1}{3}, $$ and $$\tilde{P}(5) = 0.$$

Stemming from the last example, we can obtain different results on the same probability space. To keep things clear, we use different notations if multiple measures exist, like $P$ vs $\tilde{P}.$