Probability

2020-08-25

Introduction

Event: A subset of the sample space

Notation: A, B, C,

e.g. Noisy channel transmissions

A = {x : x>=10} : at least 10 transmissions required
B={2,4,6,…} : # of transmission needed are even

e.g. price two numbers at random [0, 1]

A= {(x,y) : 0<=x<=1, 0<=y<=1, |x-y|<=1/2}
The two numbers has a difference of less than or equal to 1/2

Set theory review

Let $A, B$ be two events

A U B : union of the two events (sets) consists of all events that are either in A or B or in both
A Intersect B (or AB): Intersection of two events consists of all events in both A and B

e.g. One coin flip: $A = {H}, B={T}$

$A \cup B = {H, T} = \Omega$
$A \cap B = \emptyset$ : null or empty set

e.g. two coin flips

$A = {HH, TT}$ : first toss is a H
$B = {HH, TH}$: 2nd toss a H
$A \cap B = {HH}$
$\bigcup\limits{n=1}^{\infty} A_n$
$\bigcap\limits{n=1}^{\infty} B_n$

Complements
e.g. one coin flip
$A = {H}$
$A^c = {T} = \Omega - {H}$

The relative complement of sets $A$ and $B$ has all elements in $A$ that are not in $B$
Notation: $A-B = A\cap B^c$

e.g. Recall noisy channel transmissions

Given $A = {x: x\ge10}$, $B = {2,4,6,8,10,...}$
$A \cup B = {2,4,6,8,10,12...}$
$A\cap B = {10, 12, 14, ...$
$A-B = {11, 13, 15,...}$

Probabilities
For each event $A \in \omega$, define a value $P(A)$ that satisfies the conditions

1. $0 \le P(A) \le 1$

2. $P(\Omega)=1$
3. For any sequence of events $A_1, A_2, A_3, ...$ that are mutually exclusive ($A_i \cap A_j =
\emptyset$ ), the value of $P(\bigcup\limits_{i}A_i) = \sum P(A_i)$

$P(A)$ is the probability of the event $A$

It turns out you can't assign probabilities to events of any arbitrary set of events.
The set of events $\mathscr{F}$ that you can assign must satisfy:

1. $\emptyset \in \mathscr{F}$

2. $A \in \mathscr{F} \rightarrow A^c \in \mathscr{F}$
3. if $A_1, A_2, ... \in \mathscr{F} \rightarrow \bigcup\limits_{i} A_i \in \mathscr{F}$ (any two
events, the union has to be in the field)

then $\mathscr{F}$ is called a sigma field and we study the tripley $(\Omega, P, \mathscr{F})$

$P$ is called the probability measure

Example: One coin flip
$\Omega = {H,T}$
$\mathscr{F} = {\emptyset, H, T,{H,T} }$
This is the power set of $\Omega$ (all the probabilities are there)

$P(\emptyset)=0, P(\Omega)=1$
$P(HT) = P(H\cup T) = P(\Omega) = 1$
$P(H) = P = 1/2 , P(T) = 1-p = 1/2$

ex. two coin flip, $\omega={HH, HT, TH, TT}$

$\Omega$ has 4 elements.
The power set of $\Omega$ = $2^4 = 16$

$\mathscr{F} = { \emptyset, HH, HT, TH, TT ,$

$ {HH, HT}, {HH,TH}, {HH, TT}{HT, TH}, {HT, TT}, {TH, TT}, {HH, HT, TH}, $

${HH, HT, TT}, {HT, TH, TT}, {HH, TH, TT}, \Omega }$
ex. A red, blue, and green ball is inside a jar. Pick one at random and note the color.
$\Omega_1 = {r,g,b}$
$\mathscr{F}1 = {\emptyset, r, g, b, rg, rb, gb, \Omega}$
Suppose one cannot tell the difference between blue and green; blue is seen both cases.
$\Omega_2 = {R, B}$, where $B$ refers to both $G$ and $B$ cases
$\mathscr{F}2 = {\emptyset , R, B, RB=\Omega } = {\emptyset, r, gb, rgb} \ne$ power set of
$\Omega_1$

Ex. Let's flip a coin $1,000,000$ times and count the number of Heads.
It will be close to $500,000$ and $P(H)$ is the frequency of occurance of $H$
$P(H) = \frac{num of heads}{\text{total num of coin tosses}$

where $1_{{A}} = 1$ if A is True; $0$ otherwise