21 views

Uploaded by jel220

- Probability
- Probability 05-05-13
- Statistics Problems2
- Chapter_8
- Random Variable - Probability
- Lecture 3 Prob if 2016
- L_1.pdf
- Course Outline - Probability and Statistics for Engineers (ES 202)
- Convergence
- Conditional Probability
- Lecturenotes9 10 Probability
- Library Management System -System Use Case Diagram
- Statistics Tutorial-Laws of Probability
- IEJME 346 Article 57594ecb348f9
- How Many Coin Flips on Average Does It Take to Get n Consecutive Heads
- Unit WISE 2 MARKS.pdf
- Walks and Branching
- Uncertainty-Measurements-and-Error-Analysis-PowerPoint-2015.pdf
- Bayesian Decision Theory
- ISYE 3232 Exam 1 solution

You are on page 1of 121

Tuesday, February 4, 14

What is Probability?

Tuesday, February 4, 14

What is Probability?

The actual science of logic is conversant at present only with things either certain, impossible, or entirely doubtful, none of which (fortunately) we have to reason on. Therefore the true logic for this world is the calculus of Probabilities, which takes account of the magnitude of the probability which is , or ought to be, in a reasonable man's mind James Clerck Maxwell (1850)

Tuesday, February 4, 14

CS Theory

Alg

ori

thm

Probability

ion t a y m or eor f In Th

N w et ks or

Tuesday, February 4, 14

Probabilistic Model

The sample space , which is the set of all possible outcomes of an experiment. The probability law, which assigns to a set A of possible outcomes (also called an event) a nonnegative number P(A) (called probability of A) that encodes our knowledge or belief about the collective likelihood of the elements A. The probability law must satisfy certain properties to be introduced shortly.

Tuesday, February 4, 14

Probabilistic Model

Probability law B

P( B )

A

P ( A)

Events

Tuesday, February 4, 14

Set Theory

Finite, Countable and Uncountable Sets. Set Operations. The Algebra of Sets.

Tuesday, February 4, 14

Sets

A set is a collection of objects, which are the elements of the set. xS x /S

Tuesday, February 4, 14

Specifying Sets

As a list of elements: S = {x1 , x2 , . . . , xn } With words: the set of even natural numbers Specify a rule or algorithm: S = {r Q : r < 2}

2

Tuesday, February 4, 14

Subsets

If every element of a set S is also an element of a set T , we say that S is a subset of T , and we write S T or T S If S T and T S the two sets are equal , and we write S=T The universal set, denoted by , contains all the objects of interest in a particular context.

Tuesday, February 4, 14

Set Operations

The complement of a set S , with respect to the universe is the set of elements in that do not belong to S : S = {x : x / S}

c

Tuesday, February 4, 14

Set Operations

The union of two sets S and T is the set of all elements that belong to S or T (or both): S T = {x : x S orx T } The intersection of two sets S and T is the set of all elements that belong to both S and T : S T = {x : x S andx T }

Tuesday, February 4, 14

Algebra of Sets

Tuesday, February 4, 14

De Morgans laws

Tuesday, February 4, 14

Functions

Domain

Range

Tuesday, February 4, 14

Cardinality

{Thorin,Balin, Bifur, Bofur, Bombur, Dori, Fili, Gloin, Kili} {Gruon, Mocoso, Tmido, Mudito, Dormiln, Felz, Sabio}

Tuesday, February 4, 14

Cardinality

Tuesday, February 4, 14

Cardinality

Tuesday, February 4, 14

Cardinality

Tuesday, February 4, 14

Cardinality

Tuesday, February 4, 14

Cardinality

Tuesday, February 4, 14

Cardinality

Tuesday, February 4, 14

Tuesday, February 4, 14

Probabilistic Model

The sample space , which is the set of all possible outcomes of an experiment. The probability law, which assigns to a set A of possible outcomes (also called an event) a nonnegative number P(A) (called probability of A) that encodes our knowledge or belief about the collective likelihood of the elements A. The probability law must satisfy certain properties to be introduced shortly.

Tuesday, February 4, 14

Probabilistic Model

Probability law B

P( B )

A

P ( A)

Events

Tuesday, February 4, 14

Sample Space

Different elements of the sample space should be distinct and mutually exclusive. The sample space must be collectively exhaustive.

Tuesday, February 4, 14

For a single toss coin the sample space consist of two points: = {H, T } (We exclude possibilities like the coin stands on edge, the coin disappears, etc.)

Tuesday, February 4, 14

For n tosses of a coin the sample space is = {w : w = (a1 , . . . , an ), ai = H or T } and the general number N () of outcomes is 2n .

Tuesday, February 4, 14

For n tosses of a coin the sample space is = {w : w = (a1 , . . . , an ), ai = H or T } and the general number N () of outcomes is 2n .

Tuesday, February 4, 14

ue Tre nc eia ba ld s es ed cr ip tio n

2nd roll

se q

Tuesday, February 4, 14

Probability Laws

1. (Non-negativity) P(A) 0, for every event A.

Tuesday, February 4, 14

Probability Laws

2. (Additivity) If A and B are two disjoint events, then the probability of their union satises P ( A B ) = P ( A) + P ( B ) More generally, if the sample space has an innite number of elements and A1 , A2 , . . . is a sequence of disjoint events then the probability of their union satises P ( A1 A2 ) = P ( A1 ) + P ( A2 ) +

Tuesday, February 4, 14

Probability Laws

3. (Normalization) The probability of the entire sample space is equal to 1, that is P() = 1

Tuesday, February 4, 14

Properties

Let A, B and C be events. 1. P() = 0. 2. If A B , then P(A) P(B ) 3. P(A B ) = P(A) + P(B ) P(A B ) 4. P(A B ) P(A) + P(B )

Tuesday, February 4, 14

If the sample space consists of a nite number of possible outcomes, then the probability law is specied by the probabilities of the events that consists of a single element. In particular, the probability of any event A = {s1 , s2 , . . . , sn } is the sum of the probabilities of its elements: P ( A) = P ( s 1 ) + P ( s 2 ) + + P ( s n )

Tuesday, February 4, 14

If the sample space consists of n possible outcomes which are equally likely (i.e. all single-element events have the same probability), then the probability of any event A is given by N ( A) P ( A) = n

Tuesday, February 4, 14

Sample space for pair of 4-sided rolls 4

2nd roll

3 2 1 1 2 3 1st roll 4

Tuesday, February 4, 14

Counting

Tuesday, February 4, 14

Cardinality

Tuesday, February 4, 14

Product Rule

Theorem (Pairs). With n elements a1 , a2 , . . . , an and m elements b1 , b2 , . . . , bm , it is possible to form nm pairs (ai , bj ) containing one element for each group.

Tuesday, February 4, 14

Theorem. Given n1 elements a1 , a2 , . . . , an1 and n2 elements b1 , b2 , . . . , bn2 , etc., up to nr elements x1 , x2 , . . . , xnr ; it is possible to form n1 n2 nr ordered r-tuples (ai1 , ai2 , . . . , air ) containing one element for each group.

Tuesday, February 4, 14

n1 n2 n3 n4

choices Stage 1

choices Stage 2

choices Stage 3

choices Stage 4

Tuesday, February 4, 14

Examples

Throwing a dice r times. No 1 in r throws? Display of ags. Suppose r ags of dierent colors are to be displayed on n poles in a row. In how many ways can this be done? Loops, Recursions, ...

Tuesday, February 4, 14

Ordered Samples

Consider a set or population of n elements a1 , . . . , an . Any ordered arrangement (ai1 , . . . , aik ) of k symbols is called an ordered sample of size k drawn from our population. Sampling with replacement (repetitions are allowed). Sampling without replacement (repetitions are not allowed).

Tuesday, February 4, 14

Ordered Samples

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

Tuesday, February 4, 14

Ordered Samples

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

Tuesday, February 4, 14

Ordered Samples

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

Tuesday, February 4, 14

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

Tuesday, February 4, 14

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

N () = nk

Tuesday, February 4, 14

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

N () = nk

Tuesday, February 4, 14

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

Tuesday, February 4, 14

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

Tuesday, February 4, 14

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

Tuesday, February 4, 14

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

Tuesday, February 4, 14

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

N () = (n)k

Tuesday, February 4, 14

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

N () = (n)k

Tuesday, February 4, 14

Ordered Samples

Theorem. For a population of n elements and a prek scribed sample size k , there exist n dierent samples with replacement and (n)k samples without replacement.

Tuesday, February 4, 14

Ordered Samples

When k = n that sample is called a permutation of the elements of the population and (n)n = n! Whenever we speak of random samples of xed size k , the adjective random is to imply that all samples have the same probability, namely, nk in sampling with replacement and 1/(n)k in sampling without replacement. If n is large and k is relatively small, the ratio (n)k /nk is near unity i.e. the two ways of sampling are practically equivalent.

Tuesday, February 4, 14

Probability an element is not included in the sample? If n balls are randomly placed into n cells, what is the probability every cell is occupied?

Tuesday, February 4, 14

Birthday Paradox

Throw a dice 6 times, what is the probability all six faces appear? Elevator (10 oors, 7 people)

Tuesday, February 4, 14

Theorem. If A1 , A2 , . . . , An are disjoint sets, then: | A1 A2 An | = | A1 | + | A2 | + + | An |

Tuesday, February 4, 14

Counting Passwords

On a certain computer system, a valid password is a sequence between six and eight symbols. The rst symbol must be a letter which can be upper case or lower case and the remaining symbols must be either letter or numbers.

Tuesday, February 4, 14

Subpopulations

Two populations are considered dierent only if one contains an element not contained in the other.

Tuesday, February 4, 14

Subpopulations

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

Tuesday, February 4, 14

Subpopulations

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

Tuesday, February 4, 14

Subpopulations

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

Tuesday, February 4, 14

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

Tuesday, February 4, 14

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

n N () = k

Tuesday, February 4, 14

We now consider examples involving the selection of k balls from an urn containing n distinguishable balls.

n N () = k

Tuesday, February 4, 14

Subpopulations

Theorem. A population of n elements possesses ferent sub-populations of size r n. n n = r nr n = 1, 0! = 1 0

n r

dif-

Tuesday, February 4, 14

Theorem. If f : A B is k -to-1, then: | A| = k | B | Unlike as it may seem, many counting problems are made much easier by initially counting every item multiple times and the correcting the answer using the division rule.

Tuesday, February 4, 14

Examples

Hands of poker. What is the probability that a hand of poker contains contains ve dierent face values? Occupancy problem. Probability that a specied cell contains exactly k balls?

Tuesday, February 4, 14

Tuesday, February 4, 14

Partitions

Theorem. Let k1 , . . . , kr be integers such that k1 + k2 + + kr = n, ki 0,

The number of ways in which a population of n elements can be divided into k ordered parts (partitioned into k subpopulations) of which the rst contains k1 elements, the second k2 elements, etc., is n! k1 ! k2 ! kr !

Tuesday, February 4, 14

How many sequences can be formed by permuting the letters in the 10-letter word BOOKKEEPER?

Tuesday, February 4, 14

Tuesday, February 4, 14

Theorem. For all n N and a, b R: ( a + b) =

n

k=0

n X

n n k k a b k

Tuesday, February 4, 14

Theorem. For all n N and zi R: X ( z1 + z2 + + zm ) n k1 k2 km z1 z2 zm k 1 , . . . km

n

k1 , . . . , k m N k1 + + km = n

Tuesday, February 4, 14

Combinatorial Proofs

Symmetry n n = k nk Pascals Identity n n1 n1 = + k k1 k 3n n = n X

r =0

n r

2n nr

Tuesday, February 4, 14

Conditional Probability

Tuesday, February 4, 14

Conditional Probability

Conditional Probability provides us with a way to reason about the outcome of an experiment, based on partial information. We seek to construct a new probability law that takes into account the new information: a probability law that form ant even A, species the conditional probability of A given B denoted by P ( A| B )

Tuesday, February 4, 14

Conditional Probability

P(A|B ) must constitute a legitimate probability law. The conditional probabilities must be consistent with our intuition in important special cases P( A B ) P ( A| B ) = P( B ) where we assume that P(B ) > 0.

Tuesday, February 4, 14

Conditional Probability

1. (Non-negativity) P(A|B ) 0, for every event A.

Tuesday, February 4, 14

Conditional Probability

2. (Additivity) If A1 and A2 are two disjoint events, then the probability of their union satises P ( A1 A2 | B ) = P ( A1 | B ) + P ( A2 | B ) More generally, if the sample space has an innite number of elements and A1 , A2 , . . . is a sequence of disjoint events then the probability of their union satises P ( A 1 A2 | B ) = P ( A1 | B ) + P ( A2 | B ) +

Tuesday, February 4, 14

Conditional Probability

3. (Normalization) The probability of the entire sample space is equal to 1, that is P( | B ) = 1

Tuesday, February 4, 14

Examples

We toss a coin three successive times. Compute the conditional probability P(A|B ) when A and B are the events: A = { more heads than tails come up} B = {1st toss is a head}

Tuesday, February 4, 14

When building models for experiments with sequential character, it is natural to rst specify conditional probabilities and the use them to determine unconditional probabilities: P ( A B ) = P ( B ) P ( A| B )

Tuesday, February 4, 14

Examples

If an aircraft is present in a certain area, a radar detects it and generates an alarm signal with probability 0.99. If an aircraft is not present, the radar generates a (false) alarm with probability 0.10. We assume that an aircraft is present with probability 0.005. What is the probability of no aircraft presence and a false alarm? What is the probability of aircraft presence and no detection?

Tuesday, February 4, 14

Examples

Aircraft Present

P(

P(

P

5 0 . 0

) A | (B

.99 0 =

= ) A

P( B c | A)

=0

.01

Missed Detection

Ac )=

0. 9

P

5

A (B |

c)

0 1 . 0

False Alarm

P( B c |A c )=

0.90

Tuesday, February 4, 14

Multiplication Rule

Assuming that all of the conditioning events have positive probability, we have:

n \ n 1 \ i=1

P(

i=1

Ai ) = P ( A1 ) P ( A2 | A1 ) P ( A3 | A1 A2 ) P ( An |

Ai )

Tuesday, February 4, 14

Example

Three cards are drawn from an ordinary 52-card deck without replacement. What is the probability that none of the three cards is a heart, assuming that all triplets are equally likely?

Tuesday, February 4, 14

1 2 3

Tuesday, February 4, 14

1 2 3

Tuesday, February 4, 14

1 2 3

Tuesday, February 4, 14

1 2 3

Tuesday, February 4, 14

D 1 1 2 3 O 2 3 3 2 P( D )

1 3 1 3 1 3 1 3

P( O | D ) p (1 p) 1 1

P ( D ) P( O | D ) 1 3p 1 3 (1 p)

1 3 1 3

Tuesday, February 4, 14

Global min cut. Given a connected, undirected graph G = (V, E ), nd a cut (A, B ) of minimum cardinality. Applications. Identify clusters, network reliability, TSP solvers.

Tuesday, February 4, 14

Contraction algorithm

- replace u and v by single new super-node w - keep parallel edges, but delete self-loops - preserve edges, updating endpoints of u and v to w

Repeat until graph has just two nodes v1 and v1. Return the cut (all nodes that were contracted to form v1).

b d e

u

f

contract u-v f

10

Tuesday, February 4, 14

1

Minimum Global Cut n the cut (all nodes that were contracted to form v ).

Tuesday, February 4, 14

Tuesday, February 4, 14

Let A1 , A2 , . . . , An be events that form a partition of the sample space , that is Ai Aj = for i 6= j and

n [

Ai =

i=1

and assume that P(Ai ) > 0, for all i. Then, for any event B , we have P ( B ) = P ( A1 ) P ( B | A1 ) + P ( An ) P ( B | An )

Tuesday, February 4, 14

A1 B A3

A4 A2

Tuesday, February 4, 14

Example

You enter a chess tournament where your probability of winning is 0.3 against half the players (call then type I), 0.4 against a quarter of the players (call the type II), and 0.5 against the remainder quarter of the players (call them type III). You play a game against a randomly chosen opponent. What is the probability of winning?

Tuesday, February 4, 14

Let A1 , A2 , . . . , An be disjoint events that form a partition of the sample space, and assume that P(Ai ) > 0, for all i, then, for any event B such that P(B ) > 0, we have P ( Ai | B ) = = P ( Ai ) P ( B | Ai ) P( B ) P ( Ai ) P ( B | Ai ) P ( A1 ) P ( B | A1 ) + + P ( An ) P ( B | An )

Tuesday, February 4, 14

Prior

P ( Ai ) P ( B | Ai ) P ( Ai | B ) = P( B )

Posterior

Tuesday, February 4, 14

Example

Probability that an aircraft is present? Suppose that you won. What is the probability you had an opponent of type I? The False-Positive Puzzle.

Tuesday, February 4, 14

Examples

If an aircraft is present in a certain area, a radar detects it and generates an alarm signal with probability 0.99. If an aircraft is not present, the radar generates a (false) alarm with probability 0.10. We assume that an aircraft is present with probability 0.005. What is the probability of aircraft presence given alarm went o ?

Tuesday, February 4, 14

Example

You enter a chess tournament where your probability of winning is 0.3 against half the players (call then type I), 0.4 against a quarter of the players (call the type II), and 0.5 against the remainder quarter of the players (call them type III). You play a game against a randomly chosen opponent. What is the probability of having an opponent of type I given you won?

Tuesday, February 4, 14

Independence

Tuesday, February 4, 14

Independent Events

When the occurrence of B does not alter the probability of A: P ( A| B ) = P ( A) we say that A is independent of B . Equivalently, P ( A B ) = P ( A) P ( B )

Tuesday, February 4, 14

Example

Consider an experiment involving two successive rolls of a 4-sided die in which all 16 possible outcomes are equally likely. Ai = {1st roll results in i}, A = {1st roll is a 1}, Bj = {2nd roll results in j } B = {sum is a 5}

A = {minimum is 2},

Tuesday, February 4, 14

B = {maximum is 2}

Conditional Independence

Given an even C , the events A and B are called conditionally independent if P ( A B | C ) = P ( A| C ) P ( B | C )

Tuesday, February 4, 14

Example

Tuesday, February 4, 14

We say that the events A1 , A2 , . . . , An are independent if Y \ P(Ai ), for every subset S of {1, 2, . . . , n} P( Ai ) =

i S i S

Tuesday, February 4, 14

Tuesday, February 4, 14

Example

Tuesday, February 4, 14

Tuesday, February 4, 14

Network Connectivity

0.8 0.9

E

0. 9

0.9

A

0. 7 5

0.85

0.95

Tuesday, February 4, 14

Network Connectivity

1

2 3 Parallel Connection

Series Connection

Tuesday, February 4, 14

If an experiment involves a sequence of independent but identical stages, we say we have a sequence of independent trials. In the special case where there are only two possible results at each stage, we say we have a sequence of independent Bernoulli trials.

Tuesday, February 4, 14

Contention resolution. Given n processes P1, , Pn, each competing for access to a shared database. If two or more processes access the database simultaneously, all processes are locked out. Devise protocol to ensure all processes get through on a regular basis. Restriction. Processes can't communicate. Challenge. Need symmetry-breaking paradigm.

P1

P2 . . . Pn

Tuesday, February 4, 14

4

- ProbabilityUploaded byAbhishek Mukherjee
- Probability 05-05-13Uploaded byKennedy Roallos
- Statistics Problems2Uploaded byZek Zarin
- Chapter_8Uploaded byNdomadu
- Random Variable - ProbabilityUploaded bySreekanth Madhavan
- Lecture 3 Prob if 2016Uploaded byMobasher Messi
- L_1.pdfUploaded byALIKNF
- Course Outline - Probability and Statistics for Engineers (ES 202)Uploaded byHassan Ali
- ConvergenceUploaded byEdmond Z
- Conditional ProbabilityUploaded byGautam D
- Lecturenotes9 10 ProbabilityUploaded byarjunvenugopalachary
- Library Management System -System Use Case DiagramUploaded bySolver
- Statistics Tutorial-Laws of ProbabilityUploaded byTeena Thapar
- IEJME 346 Article 57594ecb348f9Uploaded byWahyudiUkswSalatiga
- How Many Coin Flips on Average Does It Take to Get n Consecutive HeadsUploaded byasfax0
- Unit WISE 2 MARKS.pdfUploaded byInfi Coaching Center
- Walks and BranchingUploaded byxiaosheegwah
- Uncertainty-Measurements-and-Error-Analysis-PowerPoint-2015.pdfUploaded byMuhammad Qasim
- Bayesian Decision TheoryUploaded bysaba
- ISYE 3232 Exam 1 solutionUploaded byKaranPahawa
- From Alpha to Omega and Back AgainUploaded byAnthony Forwood
- Design for InstructionUploaded byNicholas Lacasse
- AnswerUploaded byUttam MN
- K-2Uploaded bydhaakchik
- FormUploaded bypankaj baviskar
- lecture_6.pdfUploaded bysdede
- On Probability - Georg-Henrik Von WrightUploaded byMarcela
- L2 Probability ReviewUploaded byMoein Razavi
- A Pr 2003 ClassicsUploaded byGanesh Kumar
- Problem Conversion GuideUploaded byRahul Dev

- chapter 10 statistics and probabilityUploaded byapi-235259942
- 47566148 PQT Question BankUploaded byPaul Jacob
- Irwin Miller, Marylees Miller-John E. Freund's Mathematical Statistics With Applications-Pearson (2014)Uploaded byClaudine Reid
- Probability CompressedUploaded byCratos_Poseidon
- Notes on standard continuous distributionsUploaded byChris Durell
- Statistics SPSS Homework Sample with SolutionsUploaded byHebrew Johnson
- 2104253 Assignment 4 Joint Probability Distribution.176656.1519666683.4243Uploaded byBeam JF Teddy
- Probability TestUploaded byJoseph
- BBS10 Ppt Mtb Ch04 ProbabiltyUploaded byAgenttZeeroOutsider
- Seminar Vasicek 08 2008Uploaded byViviana Istrate
- Estes’ Statistical LearningUploaded byjoaomartinelli
- Chihara Et Al-Mathematical Statistics With Re Sampling and R-2011-ToC_201Uploaded byuri20
- Sample ExamUploaded byNoha El-sheweikh
- Statistical InferenceUploaded bymegld
- Monckton AffidavitUploaded byKelly Lincoln
- 19 Probability Part 2 of 3Uploaded byMathematics
- AP and GP important questions for sscUploaded byAMIN BUHARI ABDUL KHADER
- Tutsheet5Uploaded byvishnu
- Lesswrong SeqUploaded byHareesh Kanchanepally
- 4IQM_Syllabus (1)Uploaded byHamis Mohamed
- madridnoteslarge.pdfUploaded byshobu_iuj
- Normal DistributionUploaded byadd313199
- Ian Hacking-Hume's Species of ProbabilityUploaded byjf_90
- Grd 5:6 DM & ProbabilityUploaded byElizabeth Mangos
- APStat_ProbabilityReviewUploaded byrohitrgt4u
- Quiz 6Uploaded byrrro0orrr
- QRM R CODEUploaded byandroidDownloadOnly
- 945_Lecture6_Generalized.pdfUploaded byYusrianti Hanike
- Odds ProbabilityUploaded byBella Fo
- syllabus2Uploaded byajee