posted 28 Jul 2025

Basic Probability for Randomized Algorithms


§ Contents

§ Overview

  • a random variable is typically denoted as a capital letter XX
  • a sample space Ω\Omega is the set of all possible values of a random variable, and can be discrete or continuous, finite or infinite
  • the notation Pr[X=x]\Pr[X = x] represents the probability that the random variable XX has value xΩ.x \in \Omega.

Joint Distribution

Pr[X=x,Y=y] \Pr[X = x, Y = y]

The probability that random variables XX and YY have values xx and yy, respectively.

Conditional Distribution

Pr[X=xY=y]=Pr[X=x,Y=y]Pr[Y=y] \Pr[X = x\,|\,Y = y] = \frac{\Pr[X = x, Y = y]}{\Pr[Y = y]}

The probability that XX has value xx when YY has value yy

Marginal Distribution

Pr[X=x]=yΩYPr[X=xY=y] \Pr[X = x] = \sum_{y \in \Omega_Y} \Pr[X = x\,|\,Y = y]

Independence

Random variables XX and YY are independent if for all xΩXx \in \Omega_X and yΩYy \in \Omega_Y

Pr[X=x]=Pr[X=xY=y]\Pr[X = x] = \Pr[X = x\,|\,Y = y]

Boole's Inequality (Union Bound)

Let S1,S2,,SkS_1, S_2, \dots, S_k be a set of events occuring with probability p1,p2,,pk.p_1, p_2, \dots, p_k.

The probability that at least one of the events S1,S2,,SkS_1, S_2, \dots, S_k occurs is at most p1+p2++pk.p_1 + p_2 + \dots + p_k.

The probability that none of the events S1,S2,,SkS_1, S_2, \dots, S_k occur is at least 1(p1+p2++pk).1 - (p_1 + p_2 + \dots + p_k).

Then,

Pr[AB]=Pr[A]+Pr[B]Pr[AB]. \Pr[A \cup B] = \Pr[A] + \Pr[B] - \Pr[A \cap B].