You are on page 1of 92

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

FACULTY OF ENGINEERING
DEPARTM ENT OF ELECTRICAL AND COMPUTER ENGINEERING

E NE E 331

ENGINEERING PROB ABILITY & STATISTICS

LECTURE NOTES
BY Dr. WA EL HASHLAMOUN
SEPTEMBER, 2008

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

C HA P T E R I

F U N D A M E N T A L C O N C E PT S
O F P R O BA B I L I T Y

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

Basic Definitions:
We start our treatment of probability theory by introducing some basic definitions.

Experiment:
By an experiment, we mean any procedure that: 1- Can be repeated, theoretically, an infinite number of times. 2- Has a well-defined set of possible outcomes.

Sample Outcome:
Each of the potential eventualities of an experiment is referred to as a sample outcome(s).

Sample Space:
The totality of sample outcomes is called the sample space (S).

Event:
Any designated collection of sample outcomes, including individual outcomes, the entire sample space and the null space, constitute an event.

Occur:
An event is said to occur if the outcome of the experiment is one of the members of that event.
EXAMPLE (2-1):

Consider the experiment of flipping a coin three times. a- What is the sample space? b- Which sample outcomes make up the event: A : Majority of coins show heads.
SOLUTION:

a- Sample Space (S) = {HHH, HHT, HTH, THH, HTT, THT, TTH, TTT} b- A = {HHH, HHT, HTH, THH}

§ Algebra of Events:
Let A and B be two events defined over the sample space S, then: - The intersection of A and B, (A B), is the event whose outcome belongs to both A and B. - The union of A and B, (A U B), is the event whose outcome belongs to either A or B or both. - Events A and B are said to be Mutually Exclusive (or disjoint) if they have no outcomes in common, that is A B = Ø, where Ø is the null set (a set which contains no outcomes). - The complement of A (Ac or contained in A. ) is the event consisting of all outcomes in S other than those

- Venn Diagram is a graphical format often used to simplify the manipulation of complex events. -1-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

A

B

A

B A

A

B

S
B A

AUB B

S
A

Ac
B

S

A A B=Ø

S

(A

B)c

S

(A

Bc) U (B

A c)

S

§ De Morgan's Laws:
Use Venn diagrams to show that: 1- (A B)c = Ac U Bc 2- (A U B)c = Ac Bc
EXAMPLE (2-2):

An experiment has its sample space as: S = {1, 2, 3, , 48, 49, 50}. Define the events A : set of numbers divisible by 6 B : set of elements divisible by 8 C : set of numbers which satisfy the relation 2 n , n = 1, 2, 3, Find: 1- A, B, C 2- A U B U C
SOLUTION:

3- A

B

C

1- Events A, B, and C are: A = {6, 12, 18, 24, 30, 36, 42, 48} B = {8, 16, 24, 32, 40, 48} C = {2, 4, 8, 16, 32} 2- A U B U C = {6, 12, 18, 24, 30, 36, 42, 48, 8, 16, 32, 40, 2, 4} 3- A B C = { Ø }
EXAMPLE (2-3):

The sample space of an experiment is: S = { - 20 £ x £ 14 }. If A = { - 10 £ x £ 5 } and B = { - 7 £ x £ 0 } find. 1- A U B 2- A B A
SOLUTION:

1- A U B = { - 10 £ x £ 5 } 2- A B = { - 7 £ x £ 0 }

-20

-10 B -7

0 0

5

14

x x

-2-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

§ Definitions of Probability:
Four definitions of probability have evolved over the years: - Definition I: Classical (a priori) If the sample space S of an experiment consists of finitely many outcomes (points) that are equally likely, then the probability of event A, P(A) is: Number of outcomes in A P(A) = Number of outcomes in S Thus in particular, P(S) = 1 - Definition II: Relative Frequency (a posteriori) Let an experiment be repeated (n) times under identical conditions then, the relative frequency: f(A) Number of times A occurs P(A) = lim = n ®¥ n Number of trials f(A) is called the frequency of (A) f(A) Clearly 0 £ £1 n f(A) = 0 if (A) does not occur in the sequence of trials n f(A) = 1 if (A) occurs in each of the (n) trials n
EXAMPLE (2-4):

In digital data transmission, the bit error probability is (p). If 10,000 bits are transmitted over a noisy communication channel and 5 bits were found to be in error, find the bit error probability (p).
SOLUTION:

According to the relative frequency definition we can estimate (p) as: (p) =

5 10,000

- Definition III: Subjective Probability is defined as a person's measure of belief that some given event will occur. Example: What is the probability of establishing an independent Palestinian state in the next 2 years? Any number we might come up with would be our own personal (subjective) assessment of the situation. - Definition IV: Axiomatic Given a sample space (S), with each event (A) of (S) (subset of S) there is associated a number P(A), called the probability of (A), such that the following axioms of probability are satisfied: 1- P(A) ³ 0 ; Probability is nonnegative 2- P(S) = 1 ; Probability of the sample space is a certain 3- For the mutually exclusive events (A) and (B) (A B = Ø) P(A U B) = P(A) + P(B) ; (A B = Ø) 4- If (S) is infinite (has infinitely many points), axiom (3) is to be replaced by: P(A1 U A2 U A3 U ) = P(A1) + P(A2) + P(A3) + Where A1, A2, A3 are mutually exclusive events (A1 A2 = Ø A1 A 3 = Ø A2 A 3 = Ø ......)

-3-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

§ Basic Theorems for Probability:
1- P(Ac) = 1 P(A) Proof: S = A U Ac P(S) = P(A) + P(Ac) 1 = P(A) + P(Ac)

è

P(Ac) = 1 - P(A)

2- P(Ø) = 0 Proof: S = S U Sc S = S U Ø ; Sc = Ø P(S) = P(S) + P(Ø) è P(Ø) = 0 3- P(A U B) = P(A) + P(B) - P(A B) Proof: For events (A) and (B) in a sample space: {A U B} = {A Bc} U {A B} U {B Ac} 1 + 2 + 3

A
1 2 3

B

S
Where events (1) and (2) and (3) are mutu ally exclusive P(A U B) = P(1) + P(2) + P(3) P(A) = P(1) + P(2) P(B) = P(2) + P(3) è P(A U B) = {P(1) + P(2)} + {P(2) + P(3)} {P(2)} è P(A U B) = P(A) + P(B) P(A B) - Theorem: If A, B, and C are three events, then: P(A U B U C) = P(A) + P(B) + P(C)
EXAMPLE (2-5):

P(A

B)

P(A

C)

P(B

C) + P(A

B

C)

One integer is chosen at random from the numbers {1, 2, , 50}. What is the probability that the chosen number is divisible by 6? Assume all 50 outcomes are equally likely.
SOLUTION:

S = {1, 2, 3, , 50} A = {6, 12, 18, 24, 30, 36, 42, 48} Number of elements in A 8 P(A) = = Number of elements in S 50

-4-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

EXAMPLE (2-6):

If the probability of occurrence of an even number is twice as likely as that of an odd number in Example (2-5). Find P(A); A is defined above.
SOLUTION:

P(S) = P(even) + P(odd) = 1 ; Let (P) be the probability of occurrence of an odd number, then (2P) will be the probability of occurrence of an even number. (25)(2P) + (25)(P) = 1 1 (50 + 25)(P) = 1 è P = 75 16 P(A) = 8 ´ 2P = 75
EXAMPLE (2-7):

Suppose that a company has 100 employees who are classified according to their marital status and according to whether they are college graduates or not. It is known that 30% of the employees are married, and the percent of graduate employees is 80%. Moreover, 10 employees are neither married nor graduates. What proportion of married employees are graduates?
SOLUTION:

Let: M : set of married employees M G : set of graduate employees N(.) : number of members in any set (.) 10 20 è N(S) = 100 N(M) = 0.3 × 100 = 30 10 N(G) = 0.8 × 100 = 80 N(M U G)c = 10 è N(M U G) = 100 10 = 90 N(M U G) = N(M) + N(G) N(M G) 90 = 30 + 80 N(M G) N(M G) = 30 + 80 90 = 20 è Two third of the married employees in the company are graduates.
EXAMPLE (2-8):

G
60

S

An experiment has two possible outcomes; the first occurs with probability (P), the second with probability (P2), find (P).
SOLUTION:

P(S) = 1 P + P2 = 1 P2 + P 1 = 0 -1 + 5 P = ; 2

(only the positive root is taken)

-5-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

EXAMPLE (2-9):

A sample space S consists of the integers 1 to 6 inclusive. Each has an associated probability proportional to its magnitude. If one number is chosen at random, what is the probability that an even number appears?
SOLUTION:

Sample Space S = {1 , 2 , 3 , 4 , 5 , 6} Event (A) = {2 , 4 , 6} P(A) = P(2) + P(4) + P(6) P(S) = 1 =
6 6

å p(i) = å a(i) =
i =1 i =1

6(6 + 1) a =1 2
1 21

è The proportionality constant a =

P( A ) =

2 4 6 12 + + = 21 21 21 21

EXAMPLE (2-10):

Let (A) and (B) be any two events defined on (S). Suppose that P(A) = 0.4, P(B) = 0.5, and P(A B) = 0.1. Find the probability that: 1- (A) or (B) but not both occur. 2- None of the events (A) or (B) will occur. 3- At least one event will occur. 4- Both events occur.
SOLUTION:

P(A) = P[(A Bc) U (A B)] Using Venn diagram: P(A) only = 0.3 P(B) only = 0.4 1- P(A or B only) = 0.3 + 0.4 = 0.7 Note that: P(A U B) = P(A) + P(B) P(A U B) = 0.4 + 0.5 P(A B)

A
0.3 0.2 0.1 0.4

B

S

0.1 = 0.8

2- P(none) = P(AUB) = 0.2 3- P(at least one) = P(A U B) = 0.8 4- P(both) = P(A B) = 0.1

-6-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

§ Discrete Probability Functions:
If the sample space generated by an experiment contains either a finite or a countable infinite number of outcomes, then it is called a discrete sample space. Any probability assignment on that space such that: a- P(s i ) ³ 0 b- å P(s i ) = 1
si Î S

is said to be a discrete probability function. If (A) is an event defined on (S), then P(A) =
si Î A

å P(s )
i

For example, the sample space, S = {1 , 2 , 3 , 4 , 5 , 6} is countably finite, while the set of positive integers, S = {1 , 2 , 3 , ...} is countably infinite.
EXAMPLE (2-11):

The outcome of an experiment is either a success with probability (1/2) or a failure with probability (1/2). If the experiment is to be repeated until a success comes up for the first time. What is the probability of that happening on an odd-numbered?
SOLUTION:

Trial (i) 1 2 3 4 5
......

Outcome S FS FFS FFFS FFFFS
......

Probability P(i) P(1) =
1 2

æ1ö æ1ö P(2) = ç ÷ ç ÷ è2ø è2ø
æ1ö æ1ö P(3) = ç ÷ ç ÷ è2ø è 2ø
2

æ1ö æ1ö P(4) = ç ÷ ç ÷ è2ø è 2ø æ1ö æ1ö P(5) = ç ÷ ç ÷ è2ø è 2ø
......
k -1 k 4

3

k

F F F F ........ F S

æ1ö P(k)= ç ÷ è2ø

æ1ö æ1ö ç ÷ = ç ÷ ; 1£ k £ ¥ è 2ø è2ø

P(A) = P(1) + P(3) + P(5) +

æ1ö P(A) = å ç ÷ i= 0 è 2 ø ¥ 1 å xk = 1- x k=0

¥

2 i +1

1 ¥ æ1ö = åç ÷ 2 i =0 è 4 ø

i

è P(A) =

1 ´ 2

1 1 14

(Geometric series) è P(A) =

2 3

Note that, the sample space of this experiment is countably infinite.

-7-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

EXAMPLE (2-12):

The discrete probability function for the countably infinite sample space S = {1, 2, 3, C P(x) = 2 ; x = 1, 2, 3, x a- Find the constant C so that P(x) is valid discrete probability function. b- Find the probability that the outcome of the experiment is a number less than 4.
SOLUTION:

} is:

a. By Axiom 2, P(S) = 1 2 ¥ ¥ C 1 6 å x 2 = 1 è Cå x 2 = 1 è C 6 = 1 , è C = 2 x =1 x =1 b. The event A is A = {1 , 2 , 3} P(A) = P(1) + P(2) + P(3) 6 æ 1 1 1 ö 49 è P(A) = 2 ç 2 + 2 + 2 ÷ = 2 = 0.827 ç (1) (2) (3) ÷ 6 è ø

§ Continuous Probability Functions:
If the sample space associated with an experiment is an interval of real numbers, then (S) has an uncountable infinite number of points and (S) is said to be continuous. Let f (x) be a real-valued function defined on (S) such that: a- f ( x) ³ 0 bAll x

ò f ( x) dx = 1
xÎA

is called a continuous probability function. If (A) is an event defined on (S), then P(A) =

ò f ( x) dx

For example, the sample space S = { 1 £ x £ 2 } is uncountably infinite.

EXAMPLE (2-13):

Let the sample space of an experiment be: S = { 1 £ x £ 2 }. The probability function defined over S is: k f (x) = 2 , 1 £ x £ 2 . x a- Find (k) so that f (x) is a valid probability function. b- Find P (x £ 1.5)
SOLUTION:
2 2

a- P(S) = ò f (x) dx = 1 Þ
1

òx
1

k
2

dx = 1 Þ k = 2

1.5

b- P(x £ 1.5) =

òx
1

k
2

dx =

2 3

-8-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

EXAMPLE (2-14):

The length of a pin that is a part of a wheel assembly is supposed to be 6 cm. The machine that stamps out the parts makes them 6 + x cm long, where x varies from pin to pin according to the probability function: f(x) = k(x + x2) ; 0 £ x £ 2 where (k) is a constant. If a pin is longer than 7 cm, it is unusable. What proportion of pins produced by this machine will be unusable?
SOLUTION:

P( S ) = ò f ( x) dx = 1

k ò (x + x 2 ) dx = 1
0

2

é x2 x2 6 kê + =1 Þ k = 28 ë2 2 0
A cotter pin is not accepted if the error x ³ 1 cm. P( x ³ 1 ) =

2

ò k(x + x
1

2

2

) dx
2

é x2 x2 = kê + ë2 2

=
1

6 é 4 8 1 1 ù 23 + - - = 28 ê 2 3 2 3 ú 28 ë û

P( x ³ 1 ) = p(pin length ³ 7 cm) =

23 28

§ Conditional Probabilities and Statistical Independence:
- Definition: Given two events (A) and (B) with P(A) and P(B) > 0. We define the Conditional Probability of (A) given (B) has occurred as: P(A I B) (1) P(A/B) = P(B) and the probability of (B) given (A) has occurred as: P(A I B) (2) P(B/A) = P(A) In (2), (A) serves as a new (reduced) sample space and P(B/A) is the fraction of (A) which corresponds to (A B).

A

B

S

-9-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

EXAMPLE (2-15):

A sample space (S) consists of the integers 1 to n inclusive. Each has an associated probability proportional to its magnitude. One integer is chosen at random, what is the probability that number 1 is chosen given that the number selected is in the first (m) integers.
SOLUTION:

Let (A) be the event number 1 occurs (A) = {1} (B) the outcome belongs to the first m integers (B) = {1 , 2 , 3 , , m} n n n n (n + 1) 2 å Pi = å a i = 1 Þ aå i = 1 è a 2 = 1 Þ a = n(n + 1) i =1 i =1 i =1 P(A I B) P(1) 1 2 P(A/B) = = = m = m = è P(A/B) = m(m + 1) P(B) P(B) m(m + 1) å Pi å i 2 i =1 i =1 2 A priori probability: P(A) = n (n + 1) 2 A posteriori probability: P(A/B) = m(m + 1) Clearly P(A/B) > P(A) due to the additional information given by event (B). - Theorem: Multiplication Rule If (A) and (B) are events in a sample space (S) and P(A) P(A B) = P(A) P(B/A) = P(B) P(A/B) For three events A, B, and C: P(A B C) = P(A) P(B/A) P(C/B,A)
EXAMPLE (2-16):

0, P(B)

0, then:

A certain computer becomes inoperable if two components A and B both fail. The probability that A fails is 0.001 and the probability that B fails is 0.005. However, the probability that B fails increases by a factor of 4 if A has failed. Calculate the probability that: a- The computer becomes inoperable. b- A will fail if B has failed.
SOLUTION:

P(A) = 0.001 P(B) = 0.005 P(B/A) = 4 × 0.005 = 0.020 a- The system fails when both A and B fail, i.e., P(A I B) = P(A) P(B/A) P(A I B) = 0.001´ 0.020 = 0.00002 b- P(A I B) = P(A) P(B/A) = P(B) P(A/B) 0.001 ´ 0.020 P(A/B) = = 0.004 0.005

-10-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

EXAMPLE (2-17):

A box contains 20 non-defective (N) items and 5 defective (D) items. Three items are drawn without replacement. a. Find the probability that the sequence of objects obtained is (NND) in the given order. b. Find the probability that exactly one defective item is obtained.
SOLUTION:

a. P(NÇNÇD) = P(N)× P(N/N)× P(D/N,N) 20 20 - 1 5 20 19 5 P(NND) = ( )( )( ) = ( )( )( ) 25 25 - 1 25 - 2 25 24 23 b. One defective item is obtained, when any one of the following sequences is obtained: (NND), (NDN), (DNN) The probability of getting one defective item is the sum of the probabilities of these sequences and is given as: 20 19 5 20 5 19 5 20 19 20 19 5 ( )( )( ) + ( )( )( ) + ( )( )( ) = (3)( )( )( ) 25 24 23 25 24 23 25 24 23 25 24 23 Later in Chapter 2, we will see that Part (b) can be solved using the hyper-geometric distribution. Definition: Statistical Independence Two events (A) and (B) are said to be statistically independent if: P(A B) = P(A) P(B) From this definition we conclude that: P(A) P(B) P(A/B) = = P(A) Þ a posteriori probability = a priori probability P(B) P(A) P(B) P(B/A) = = P(B) P(A) This means that the probability of (A) does not depend on the occurrence or nonoccurrence of (B) and vice versa. Hence, the given information does not change our initial perception about the two given probabilities. Independence of Three Events: Events (A), (B) and (C) are independent if the following conditions are satisfied: P(A B) = P(A) P(B) P(A C) = P(A) P(C) P(B C) = P(B) P(C) P(A B C) = P(A) P(B) P(C)
EXAMPLE (2-18):

-

Let S = {1 , 2 , 3 , 4} ; Pi = 1 . A = {1 , 2} and B = {2 , 3}. Are (A) and (B) independent?
4

SOLUTION:

P(A) = 1 , P(B) = 1
2

2

è è

(A

B) = {2} , P(A

B) = 1

4

è P(A

B) = P(A) P(B)

Events are independent

-11-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

EXAMPLE (2-19):

Consider an experiment in which the sample space contains four outcomes {S1, S2, S3, S4} such that P(si) = 1 . Let events (A), (B) and (C) be defined as:
4

A = {S1, S2} , B = {S1, S3} , C = {S1, S4} Are these events independent?
SOLUTION:

P(A) = P(B) = P(C) = (A P(A P(A B) = {S1} ; (A B) = 1 4 B) =

1 2

C) = {S1} C) = 1 4

; ;

(B P(B

C) = {S1} ; (A C) = 1 4 ; P(A

B B

C) = {S1} C) = 1
4

; P(A

Check the conditions:

1 1 1 1 1 1 = P(A) P(B) = ´ ; P(A C) = = P(A) P(C) = ´ 4 2 2 4 2 2 1 1 1 P(B C) = = P(B) P(C) = ´ 4 2 2 1 P(A B C) = P(A) P(B) P(C) = 1 ´ 1 ´ 1 = 1 4 2 2 2 8 è Events are not independent (even though the pair wise conditions of independence are satisfied )
EXAMPLE (2-20): Reliability of a series system

Suppose that a system is made up of two components connected in series, each component has a probability (P) of working Reliability . What is the probability that the system works assuming that components work independently?
SOLUTION: P P

P(system works) = P(component 1 works P(system works) = P × P = P2

component 2 works)

* The probability that the system works is also known as the Reliability of the system.

EXAMPLE (2-21): Reliability of a parallel system

Suppose that a system is made up of two components connected in parallel. The system works if at least one component works properly. If each component has a probability (P) of working Reliability and components work independently, find the probability that the system works.
SOLUTION:

Reliability of the system = P(system works) P(system works) = P(C1 or C2 or both C1 and C2 works) = P(C1 U C2) = P(C1) + P(C2) =P+P P(C1 C2) P2

C1

P P
C2

(P × P) = 2P

* This system fails if both components fail. -12-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

EXERCISE:

A pressure control apparatus contains 4 electronic tubes. The apparatus will not work unless all tubes are operative. If the probability of failure of each tube is 0.03, what is the probability of failure of the apparatus assuming that all components work independently?

EXERCISE: Mixed system

Find the reliability of the shown mixed system, assuming that all components work independently, and P is the reliability (probability of working) of each component.
P P P P EXAMPLE (2-22):

A coin may be fair or it may have two heads. We toss it (n) times and it comes up heads on each occasion. If our initial judgment was that both options for the coin (fair or both sides heads) were equally likely (probable), what is our revised judgment in the light of the data?
SOLUTION:

Let A : event representing coin is fair B : event representing coin with two heads C : outcome of the experiment H H 4H 44H 14H2 H ...3
n times

A priori probabilities: P(A) = 1
2

,

P(B) = 1

2

è We need to find P(A/C) = ?
P(A/C) = P(A I C) P(A) P(C/A) = P(C) P(C)

P(A/C) =

P(A) P(H H H ... H / fair coin) P(A) P(H H H ... H / fair coin) + P(B) P(H H H ... H / coin with two heads)
1 æ1ö ç ÷ 2 è2ø
n n

æ1ö ç ÷ 1 2 P(A/C) = = è nø = n 1 + 2n 1 æ1ö 1 æ1ö ç ÷ + (1) ç ÷ + 1 2 è 2ø 2 è2ø

P(B/C) = 1 - P(A/C) = 1 -

1 2n = 1 + 2n 1 + 2n

-13-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

§ Theorem of Total Probability:
Let A1, A2, , An be a set of events defined over (S) such that: S = A1 U A2 U U An ; Ai Aj = Ø for i j, and P(Ai) > 0 for i = 1, 2, 3, For any event (B) defined on (S): P(B) = P(A1) P(B/A1) + P(A2) P(B/A2) + + P(An) P(B/An) Proof: For events (A) and (B) in a sample space: B = {A1 B} U {A2 B} U {A3 B} U {A4 B} Since these events are disjoint, then: P(B) = P(A1 B) + P(A2 B) + P(A3 B) + P(A4 B) But P(A B) = P(A) P(B/A) = P(B) P(A/B) n.

A1

B

A4
A4 A3 B B

A1

B B

A2

A2

A3

P(B) = P(A1) P(B/A1) + P(A2) P(B/A2) + P(A3) P(B/A3) + P(A4) P(B/A4)
P(A1) P(A2) P(B/A 1 ) P(B/A 2)

B
P(B/A n)

P(An)

EXAMPLE (2-23):

If men constitute 47% of the population and tell the truth 78% of the time, while women tell the truth 63% of the time. What is the probability that a person selected at random will answer a question truthfully?
SOLUTION:

A1 = Event representing the selected person is a man A2 = Event representing the selected person is a woman B = Event representing telling the truth P(A1) = 0.47 P(A2) = 0.53 B = (A1 B) U (A2 B) è P(B) = P(A1 B) + P(A2 B) P(B) = P(A1) P(B/A1) + P(A2) P(B/A2) P(B) = (0.47 × 0.7) + (0.53 × 0.63) P(B) = 0.7

A1 A1 B A2 B

A2

B

§ Baye's Theorem:
If A1, A2, A3, , An are disjoint events defined on (S), and (B) is another event defined on (S) (same conditions as above), then:

P(A j /B) =

P(A j ) P(B/A j )

å P(A ) P(B/A )
i =1 i i

n

=

P(A j I B) P(B)

-14-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

EXAMPLE (2-24):

Suppose that when a machine is adjusted properly, 50% of the items produced by it are of high quality and the other 50% are of medium quality. Suppose, however, that the machine is improperly adjusted during 10% of the time and that under these conditions 25% of the items produced by it are of high quality and 75% are of medium quality. a- Suppose that one item produced by the machine is selected at random, find the probability that it is of medium quality. b- If one item is selected at random, and found to be of medium quality, what is the probability that the machine was adjusted properly.
SOLUTION:

A1 = Event representing machine is properly adjusted A2 = Event representing machine is improperly adjusted H = Event representing item is of high quality M = Event representing item is of medium quality From the problem statement we have: P(A1) = 0.9 ; ; ; P(A2) = 0.1 P(H/A2) = 0.25 P(M/A2) = 0.75 A1 M a- P(M) = P(A1 M) + P(A2 M) P(M) = P(A1) P(M/A1) + P(A2) P(M/A2) P(M) = (0.9)(0.5) + (0.1)(0.75) = 0.525 A2 M A1 P(H/A1) = 0.5 P(M/A1) = 0.5

M A2

b- P(A 1 /M) =

P(A 1 Ç M) P(A 1 ) P(M/A 1 ) = P(M) P(M)
(0.9)(0.5) = 0.8571 (0.525)
0.5

P(A1 /M) =

A1 P(A1) = 0.9

H
0.25

0.5

A2 P(A2) = 0.1

M
0.75

-15-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

EXAMPLE (2-25):

Consider the problem of transmitting binary data over a noisy communication channel. Due to the presence of noise, a certain amount of transmission error is introduced. Suppose that the probability of transmitting a binary 0 is 0.7 (70% of transmitted digits are zeros) and there is a 0.8 probability that a given 0 or 1 being received properly. a- What is the probability of receiving a binary 1. b- If a 1 is received, what is the probability that a 0 was sent.
SOLUTION:

A0 = Event representing 0 is sent A1 = Event representing 1 is sent B0 = Event representing 0 is received B1 = Event representing 1 is received From the problem statement we have: P(A0) = 0.7 ; P(B0/A0) = 0.8 ; P(B1/A0) = 0.2 ; P(A1) = 0.3 P(B0/A1) = 0.2 P(B1/A1) = 0.8

A0 P(A0) = 0.7

0.8

B0
0.2

0.2

A1 P(A1) = 0.3

B1
0.8

a- P(B1) = P(A0) P(B1/A0) + P(A1) P(B1/A1) P(B1) = (0.7)(0.2) + (0.3)(0.8) = 0.38 P(B0) = 1 P(B1) = 0.62 b- P(A 0 /B1 ) =

P(A 0 Ç B1 ) P(A 0 ) P(B1 /A 0 ) = P(B1 ) P(B1 ) (0.7)(0.2) P(A0 /B 1 ) = = 0.3684 (0.38)

EXAMPLE (2-26):

In a factory, four machines produce the same product. Machine A1 produces 10% of the product, A2 20%, A3 30%, and A4 40%. The proportion of defective items produced by the machines follows: A1: 0.001 ; A2: 0.005 ; A3: 0.005 ; A4: 0.002

An item selected at random is found to be defective, what is the probability that the item was produced by machine A1?
SOLUTION:

Let D be the event: Selected item is defective P(D) = P(A1) P(D/A1) + P(A2) P(D/A2) + P(A3) P(D/A3) + P(A4) P(D/A4) P(D) = (0.1 × 0.001) + (0.2 × 0.005) + (0.3 × 0.005) + (0.4 × 0.002) P(D) = 0.0034 P(A1 ) P(D/A1 ) (0.1) (0.001) 0.0001 1 P(A 1 /D) = = = = P(D) (0.0034) 0.0034 34

-16-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

§ Counting techniques:
Here we introduce systematic counting of sample points in a sample space. This is necessary for computing the probability P(A) in experiments with a finite sample space (S) consisting of (n) equally likely outcomes. Then each outcome has probability æ 1 ö. ç ÷
ènø

m And if (A) consists of (m) outcomes, then P ( A ) = n

- Multiplication Rule: If operation A can be performed in n1 different ways and operation B in n2 different ways, then the sequence (operation A , operation B) can be performed in n1 x n2 different ways.
EXAMPLE (2-27):

There are two roads between A and B and four roads between B and C. How many different routes can one travel between A and C.
SOLUTION:
A B C

n=2x4=8

§ Permutation:
Consider an urn having (n) distinguishable objects (numbered 1 to n). We perform the following two experiments: 1- Sampling without replacement: An object is drawn; its number is recorded and then put aside, another object is drawn; its number is recorded and then put aside, the process is repeated (k) times. The total number of ordered sequences {x1, x2, ., xk} (repetition is not allowed) called permutation is: N = n (n 1) (n 2) (n k + 1) n! 1 .. (1) N= 2 ( n - k )! where n! = n (n 1) (n 2) (3) (2) (1) 3 2- Sampling with replacement:
n

If in the previous experiment, each drawn object is dropped back into the urn and the process is repeated (k) times. The number of possible sequences {x1, x2, ., xk} of length (k) that can be formed from the set of (n) distinct objects (repetition allowed): N = nk .. (2)
EXAMPLE (2-28):

How many different five-letter computer passwords can be formed: a- If a letter can be used more than once. b- If each word contains each letter no more than once.
SOLUTION:

a- N = (26)5 26! b- N = (26 - 5)!

-17-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

EXAMPLE (2-29):

An apartment building has eight floors (numbered 1 to 8). If seven people get on the elevator on the fist floor, what is the probability that: 8 ‚ a- All get off on different floors? 7 ‚ b- All get off on the same floor? 6 ‚
SOLUTION:

Number of points in the sample space: First person can get off at any of the 7 floors. Person (2) can get off at any of the 7 floors and so on. è The number of ways people can get off: (N) = 7 × 7 × 7 × 7 × 7 × 7× 7 = 7 7

5 4 3 2

‚ ‚ ‚ ‚

‚‚‚‚‚‚‚

a- Here the problem is to find the number of permutations of 7 objects taking 7 at a time. 7! P= 7 7 b- Here there are 7 ways whereby all seven persons get off on the same floor. 7 P= 7 7

EXAMPLE (2-30):

If the number of people getting on the elevator on the first floor is 3: a- Find the probability they get off the elevator on different floors. b- Find the probability they get off the elevator on the same floor.
SOLUTION:

Number of points in the sample space (N) = 7 × 7 × 7 = 73

7 ´ 6´ 5 73 7 b- P = 3 7
a- P =

EXAMPLE (2-31):

If the number of floors is 5 (numbered 1 to 5) and the number of people getting on the elevator is 8. Find the probability that exactly 2 people get off the elevator on each floor.
SOLUTION:

Number of points in the sample space (N) = 4 × 4 × 4 × 4 × 4 × 4 × 4 × 4 = 4 8 æ 8 öæ 6 öæ 4 öæ 2 ö ç ÷ç ÷ç ÷ç ÷ ç 2 ÷ç 2 ÷ç 2 ÷ç 2 ÷ P = è øè øè øè ø 48

-18-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

EXAMPLE (2-32):

To determine an "odd man out", (n) players each toss a fair coin. If one player's coin turns up differently from all the others, that person is declared the odd man out. Let (A) be the event that some one is declared an odd man out. a- Find P(A) b- Find the probability that the game is terminated with an odd man out after (k) trials
SOLUTION:

a- P(A) =

Number of outcomes in event (A) Number of possible sequences number of outcomes leading to an odd man out: (n 1) Heads and one Tail (n 1) Tails and one Head 2n n P(A) = n = n -1 2 2 with an odd man out, a success is obtained and the game is over.

n 64444744448 O O O O O ... O
H T H T H T H T H T H T

b- A second trial is needed when the experiment ends with a failure: è P(a second trial is needed) = 1 P(A) For (k) trials:

H H H H H ... T ü H H H H ... T H ï ï ï H H H ... T H H ý Þ (n) ï M ï T H H H H ... H ï þ T T T T T ... H ü T T T T ... H T ï ï ï T T T ... H T T ý Þ (n) ï M ï H T T T T ... T ï þ

P(F F F F4 F S) = P(F) k -1 P(S) 14 ... 2 3
k -1 Trials

P(F F F F4 F S) = [1 - P(A)]k-1 P(A) 14 ... 2 3
k -1 Trials

§ Combination:
In permutation, the order of the selected objects is essential. In contrast, a combination of a given objects means any selection of one or more objects without regard to order. The number of combinations of (n) different objects, taken (k) at a time, without repetition is the number of sets that can be made up from the (n) given objects, each set containing (k) different objects and no two sets containing exactly the same (k) objects. The number is: ænö n! ç ÷= ç k ÷ k ! ( n - k )! è ø Note that: æ Arrange (k) objects ö æ First select (k) ö æ Arrange the (k) ö ç ç selected from (n) ÷ is the same as ç objects from (n) ÷ and then ç selected objects ÷ ÷ ç ÷ ç ÷ è 44424443 ø è 44 2444ø è 44 244 3 1 1 4 3 1 4 4ø ænö N k! ç ÷ ç ÷ çk÷ è ø æ nö n! N = ç ÷ ´ k! where N = çk÷ (n - k )! è ø è ænö N n! ç ÷= = ç k ÷ k! k! (n - k )! è ø -19-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

EXAMPLE (2-33):

From four persons (set of elements), how many committees (subsets) of two members (elements) may be chosen?
SOLUTION:

Let the persons be identified by the initials A, B, C and D Subsets: (A , B) , (A , C) , (A , D) , (B , C) , (B , D) , (C , D) æ 4ö 4! N =ç ÷= ç 2 ÷ 2! (4 - 2)! = 6 è ø Missing sequences: (A , A) , (B , B) , (C , C) , (D , D) è (repetition is not allowed) Missing sequences: (B , A) , (C , A) , (D , A) (C , B) , (D , B) , (D , C) è (order is not important)
EXAMPLE (2-34):

Consider the rolling of a die twice, how many pairs of numbers can be formed for each case?
SOLUTION:

n = 6 and k = 2 Case I: Permutation a- With repetition N = nk = 62 = 36 b- Without repetition

D2 D1 1 2 3 4 5 6

1 (1,1) (2,1) (3,1) (4,1) (5,1) (6,1)

2 (1,2) (2,2) (3,2) (4,2) (5,2) (6,2)

3 (1,3) (2,3) (3,3) (4,3) (5,3) (6,3)

4 (1,4) (2,4) (3,4) (4,4) (5,4) (6,4)

5 (1,5) (2,5) (3,5) (4,5) (5,5) (6,5)

6 (1,6) (2,6) (3,6) (4,6) (5,6) (6,6)

N=

n! 6! = = 30 (n - k)! (6 - 2)!

Case I: Combination ænö n! 6! ç ÷= ç k ÷ k!(n - k)! = 2! (6 - 2)! = 15 è ø
EXAMPLE (2-35):

In how many ways can we arrange 5 balls numbered 1 to 5 in 10 baskets each of which can accommodate one ball?
SOLUTION:

The number of ways ( N) =
NOTE:

n! 10! 10! = = (n - k )! (10 - 5)! 5!

If we remove the numbers of the balls so that the balls are no longer distinguishable, then: ænö n! 10! 10! The number of ways ç ÷ = ç k ÷ k!(n - k )! = 5!(10 - 5)! = 5! 5! è ø This is because the permutation within the 5 balls is no longer needed. Arrangement of Elements of Two Distinct Types When a set contains only elements of two distinct types, type (1) consists of k elements and type (2) consists of (n-k) elements, then the number of different arrangements of all the -20-

FUNDAMENTAL CONCEPTS OF PROBABILITY

CHAPTER II

elements in the set is given by the binomial coefficient. Suppose, for example, that we have k ones and (n-k) zeros to be arranged in a row, then the number of binary numbers that can be ænö formed is ç ÷ . If n = 4 and k = 1, then the possible binary numbers are (0001, 0010, 0100, çk ÷ è ø 1000). Exercise: How many different binary numbers of five digits can be formed from the numbers 1, 0? List these numbers. Exercise: How many different binary numbers of five digits can be formed from the numbers 1, 0 such that each number contains two ones? List these numbers. Exercise: In how many ways can a group of five persons be seated in a row of 10 chairs? The Multinomial Coefficient: The number of ways to arrange n items of which n1 are of one type, n2 of a second type, n æ ö n! of a k'th type is given by N = ç ç n n . . n ÷ = n !n !...n ! ÷ è 1 2 k ø 1 2 k - Comments: Stirling's formula Computing n! can be approximated by: n! ~ 2p n n +1/2 e - n , nk

-21-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

C H A P T ER I I

SINGLE RANDOM VARIABLES AND P R O B A B I LI T Y D I S T R I B U T I O N S

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

§ Definition:
A real-valued function whose domain is the sample space is called a random variable (r.v). - The random variable is given an uppercase letter X, Y, Z, while the values assumed by this random variable are given lowercase letters x, y, z, - The whole idea behind the r.v is a one to one mapping from the sample space on the real line via the mapping function X(s). - Associated with each discrete r.v (X) is a Probability Mass Function P(X = x). This density function is the sum of all probabilities associated with the outcomes in the sample space that get mapped into (x) by the mapping function (random variable X). - Associated with each continuous r.v (X) is a Probability Density Function (pdf) fX(x).This fX(x) is not the probability that the random variable (X) takes on the value (x), rather fX(x) is a continuous curve having the property that:

P(a £ X £ b) = ò f X ( x ) dx
a

b

§ Definition:
The cumulative distribution function of a r.v (X) defined on a sample space (S) is given by: FX(x) = P{X £ x} - Properties of F X(x) 1- FX( ) = 0 2- FX( ) = 1 3- 0 £ FX(x) £ 1 4- FX(x1) £ FX(x2) if x1 £ x2 5- FX(x+) = FX(x) function is continuous from the right 6- P{x1 £ X £ x2} = FX(x2) FX(x1)
EXAMPLE (3-1):

A chance experiment has two possible outcomes, a success with probability 0.75 and a failure with probability 0.25. Mapping function (random variable X) is defined as: x = 1 if outcome is a success x = 0 if outcome is a failure
SOLUTION:

P(X < 0) = 0

;

P(X £ 0) = 0.25

;

P(X < 1) = 0.25 P(X £ x)

;

P(X £ 1) = 1

F
P(X = x)
0.25

S
0.75

1.0

0.75

0.25

Real Line
Probability Mass Function

0

1

x

0

1
Cumulative Distribution Function

x

-21-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

EXAMPLE (3-2):

Let the above experiment be conducted three times in a row. a- Find the sample space. b- Define a random variable (X) as X = number of successes in the three trials. c- Find the probability mass function P(X = x). d- Find the cumulative distribution function FX(x) = P{X £ x}
SOLUTION:

In the table below we show the possible outcomes and the mapping process: Sample Outcome F F F F F S S F F F S F S S F S F S F S S S S S P(si) (0.25)3 (0.75) (0.25)2 (0.75) (0.25)2 (0.75) (0.25)2 (0.75)2 (0.25) (0.75)2 (0.25) (0.75)2 (0.25) (0.75)3
FFS SFF FSF

x 0 1

P(X = x) (0.25)3 = 0.015625 3 x (0.75) (0.25)2 = 0.140625 3 x (0.75)2 (0.25) = 0.421875 (0.75)3 = 0.421875

2 3
FSS SFS SSF

FFF

SSS

Probability Mass Function
P(X = 1) 0.140625 P(X = 0) 0.015625

P(X = 2) 0.421875

P(X = 2) 0.421875

Real Line
1 2 3
P(0) +P(1)+(2) + P(3) = 1.0

0

x

FX(x) = P{X £ x}

Cumulative Distribution Function

P(0) + P(1) + P(2) = 0.578125

P(0) + P(1) = 0.15625 P(X = 0) = 0.015625

0

1

2

3

x

Binomial Distribution: (n) = number of trials ; (x) number of successes in (n) trials

-22-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

EXAMPLE (3-3):

Suppose that 5 people including you and your friend line up at random. Let (X) denote the number of people standing between you and your friend. Find the probability mass function for the random variable (X).
SOLUTION:

Number of different ways by which the 5 people can arrange themselves = 5! This is the total number of points in the sample space. Let (A) denote you, (B) denote your friend. The random variable (X) assumes four possible values 0, 1, 2, 3 as shown below:
A B O O Oü O A B O Oï ï ý Þ (X = 0) O O A B Oï O O O A Bï þ A O B O Oü ï O A O B Oý Þ (X = 1) O O A O Bï þ A O O B Oü ý Þ (X = 2) O A O O Bþ A O O O B} Þ (X = 3)

Any Sequence similar to what is shown can be done in:
4 ´ 2!´ 3! = 0.4 5! 3 ´ 2!´ 3! P(X = 1) = = 0.3 5! 2 ´ 2! ´ 3! P(X = 2) = = 0 .2 5! 1 ´ 2! ´ 3! P(X = 3) = = 0.1 5! P(X = 0) =

you and your friend

2! {

´ 3! {

the other people

fX(x)
0.4 0.3 0.2 0.1

0

1

2

3

x

Probability Mass Function

-23-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

§ Continuous Random Variables and Distribution:
- Definition: A random variable and its distribution are called of continuous type if the corresponding cumulative distribution function FX(x) can be given by an integral of the form:

FX (x) =

òf

x

X

(u) du

where fX(x) is the probability density function related to FX(x) by: d f X (x) = FX (x) dx - Properties of fX(x) 1- fX(x) ³ 0
¥

; nonnegative

2-

òf

X

(x) dx = 1
x2

3- P{x1 £ X £ x2}=
EXAMPLE (3-4):

x1

òf

X

(u) du ; Probability is the area under the fX(x) curve between x1 and x2.

Let (X) have the pdf:

fX(x) = 0.75 (1

x2) ; {-1 £ x £ 1}
fX(x)
0.75

1- Verify that fX(x) is indeed a valid pdf. 2- Find: a- FX(x) 1 1 b- P{ - £ X £ } 2 2
SOLUTION:
¥

1-

ò

f X (x) dx = 1 è 2ò 0.75 (1 - x 2 ) dx
0

1

-1

1

x

Probability Density Function FX(x)
1 0.5

= 2 ´ (0.75 u - 0.75
x

u 3

3 1

= 2(0.75 - 0.25) = 1.0
0.25 x 3

0

2-a) FX (x) = ò 0.75 (1 - u 2 ) du = 0.5 + 0.75x

2-b) P{ -

1 1 £X £ } = 2 2

1 2

ò 0.75 (1 - u
FX( -

1 2

2

) du
-1 1

x

1 = FX( ) 2
EXERCISE:

1 ) = 0.6875 2

Cumulative Distribution Function

Find x0 such that FX(x) = P{X £ x0} = 0.95
SOLUTION:

P{X £ x0} = 0.5 + 0.75x 0

0.25 x03 = 0.95 è 3 x0

x03 = 1.8 è x0 @ 0.73

-24-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

§ M ean and Variance of a Distribution:
- Definition: The mean value or expected value of a random variable (X) is defined as:
m X = E{X} = å x i P(X = x i )

if x is discrete if x is continuous

m X = E{X} = ò x f X (x) dx

¥

- Definition: The variance of a random variable (X) is defined as:
s 2 = E{(X - m x ) 2 } = å (X - m x ) 2 P(X = x i ) if x is discrete X
¥

s = E{(X - m x ) } = ò (x - m x ) 2 f X (x) dx
2 X 2 -¥

if x is continuous is the standard deviation

sX = s2 X

The variance is the measure of the spread of the distribution. - Definition: For any random variable (X) and any continuous function Y = g (X), the expected value of g(X) is defined as:
E{g(X)} = å g(x i ) P(X = x i )

if x is discrete if x is continuous

E{g(X)} = ò g ( x ) f X (x) dx

¥

- Theorem: Let (X) be a random variable with mean m X , then:

s 2 = E(X 2 ) - m 2 X X
Proof:
¥

s 2 = E{(X - m X ) 2 } = ò (x - m X ) 2 f X (x) dx X

s 2 = ò (x 2 - 2xm X + m 2 ) f X (x) dx X X
-¥ ¥

¥

s 2 = ò x 2 f X (x) dx - 2m X ò x f X (x) dx + m 2 ò f X (x) dx X X
-¥ -¥ -¥

¥

¥

s 2 = E(X 2 ) - 2m X m X + m 2 X X
2 X

= E(X 2 ) -

2 X

-25-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

- Illustration: 1. The center of mass for a system of particles of masses m1, m2 ..., mn placed at x1, x2 1 x cm = (x1m1 + x 2 m 2 + ...... x n m n ) å mi If we let m1 = p1, m2 = p2, , then: x cm = x1p1 + x 2p 2 + ...... x n p n (The mean of a discrete distribution) 2. If (x) is the density of a rigid body along the x-axis, then the center of mass is: 1 x cm = ò x (x) dx M Where M = ò (x) dx Again, if (x) is replaced by fX(x), the pdf function, then:

, xn is:

x cm = ò x f X (x) dx is the mean of continuous distribution.
Moment of Inertia: 3. If the particles in (1) above rotate with angular velocity (w), then the moment of inertia is evaluated as:
I = å m i x i2
i =1 n n

With mi replaced by pi , we have:
I = å p i x i2
i =1

4. If the rigid body in (2) rotates with angular velocity (w), then: I = ò x 2 (x) dx è E(x 2 ) = ò x 2 f X (x) dx 5. The variance E{(X - X ) 2 } parallels the moment of inertia about the center of mass. Recall the parallel axis theorem I = Icm + M h2 E(x 2 ) = 2 + E 2 (x)
EXAMPLE (3-5):

In the kinetic theory of gases, the distance (x) is described by the exponential function -x 1 f X (x) = e x > 0 a- Find the mean free path defined as the average distance between collisions.
SOLUTION:

Mean Free Path =
¥

X

= E{X} = ò x f X (x) dx
0

¥

1 æ1ö = ò x ç ÷ e dx = = 2 d 2 N/V è ø 0 Where (N/V) is the number of molecules per unit volume and (d) is the molecular diameter.
b- If the average speed of a molecule is Rate = m/s, what is the average collision rate.

-x

-26-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

EXAMPLE (3-6):

Maxwell s Distribution Law: The speed of gas molecules follows the distribution:

æ M ö2 2 2RT f (n ) = 4 ç e ³0 ÷ è2 R Tø Where v is the molecular speed T is the gas temperature in Kelvin R is the gas constant (8.31 J/mol.K) M is the molecular mass of the gas a- Find the average speed, b- Find the root mean square speed vrms c- Find the most probable speed
SOLUTION:

3

-M

2

f (v)

v

a-

= E(v) = ò v f (v) dv =
0
2 2 ¥

¥

8R T M

3R T 3R T è n rms = ; rms = E(n 2 ) M M 0 c- The most probable speed is the speed at which f (n ) attains its maximum value. Therefore, we differentiate f (n ) with respect to (v), set the derivative to zero and solve for the maximum. The result is:

b- E{n } = (n rms ) = òn 2 f ( v) dv =

Most probable speed = Root è Mean è E(.) Square è v2 è rms = E(n 2 )

2R T M

Exercise The radial probability density function for the ground state of the hydrogen atom (the pdf of the electron position from the atom) is given by 4 f ( r ) = 3 r 2 e -2 r / a for r > 0 a where a is the Bohr radius (a = 52.9 pm). a. What is the distance from the center of the atom that the electron is most likely to be found? b. Find the average value of r?, (the mean distance of the electron from the center of the atom). c. What is the probability that the electron will be found within a sphere of radius a centered at the origin? - Theorem: Let (X) be a random variable with mean X and variance s 2 . X Define Y = aX + b ; (a) and (b) are real constants, then: -27-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

mY = a mx + b
s =a s
2 Y 2 2 X

(a) (b)

Proof: a- Y = E{aX + b}

= ò (ax + b) f X (x) dx = a ò x f X (x) dx + b ò f X (x) dx è
-¥ -¥ -¥ ¥ ¥

¥

mY = a m x + b

b- s 2 = E{(Y - m Y ) 2 } Y = E{[(ax + b) - (am X + b)] 2 }
= a 2 E{(x - m X ) 2 }
EXAMPLE (3-7):

= E{[a(x - m X )]2 } 2 2 2 è Y =a X

Find the mean and the variance of the binomial distribution considered earlier (Example 3-2) with n = 3 and P(S) = 0.75
SOLUTION:

Mean = m X = E{X} = å x i P(X = x i ) x 0 1 2 3 P(X = x) 0.015625 0.140625 0.421875 0.421875 x . P(X = x) 0 0.140625 0.843750 1.265625 2.25

P(X = x i ) = 2.25 = 3 x 0.75 è E(X) = n p = number of trials x probability of a success
i

åx

Variance = s 2 = E(X 2 ) - [E(X)]2 X x 0 1 2 3
2 X

; x2 0 1 4 9

E{X 2 } = å x i2 P(X = x i )
P(X = x) 0.015625 0.140625 0.421875 0.421875 x2 . P(X = x) 0 0.140625 1.687500 3.796875 5.625

= 5.625 - (2.25)2 = 0.5625 = 3 x 0.75 x 0.25 = number of trials x probability of success x probability of Failure è

2 X

= n p (1 - p)

-28-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

EXAMPLE (3-8):

Find the mean and the variance of the uniform distribution shown in the figure.
SOLUTION:
¥

fX(x)

Mean = m X = E{X} = ò x f X (x) dx

1 a+b mX = ò x dx = b-a 2 a

b

1 b-a

x
a b

Var(X) = s 2 = E(X 2 ) - [E(X)]2 X
E{X 2 } = ò x 2
a b

1 b 3 - a 3 a 2 + ab + b 2 dx = = b-a 3(b - a) 3
2

a + ab + b 2 æ a + b ö (b - a ) 2 s = -ç ÷ = 3 12 è 2 ø
2 X 2

EXAMPLE (3-9):

Let Z =

X - mX (Standardized r.v.), show that the mean of (Z) is zero and the variance is 1. sX

SOLUTION:

X mX = aX + b sX sX 1 1 Mean = m Z = E{Z} = E{(X - m X )} = {E(X) - E(m X )} = 0 sX sX 1 Var(Z) = s 2 = 2 s 2 = 1 Z X sX

Z can be written as: Z =

- Some useful properties of expectation: - E{a} = a ; a is a constant - E{a g(X)} = a E{g(X)} ; a is a constant - E{ g 1 (X) + g 2 (X)} = E{ g 1 (X)} + E{g 2 (X)} - The median and the mode: - Definition: For a continuous random variable (X), the median of the distribution of (X) is defined to be a point (x0) such that: P(X < x 0 ) = P(X ³ x 0 ) - Definition: If a random variable (X) has a pdf fX(x), then the value of (x) for which fX(x) is maximum is called the mode of the distribution.

-29-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

§ Common Discrete Random Variables: I. The Binomial Distribution - Definition: A random experiment consisting of (n) repeated trials such that: a- The trials are independent. b- Each trial results in only two possible outcomes, a success and a failure. c- The probability of a success (p) on each trial remains constant Is called a binomial experiment. The r.v (X) that equals the number of trials that results in a success has a binomial distribution with parameters (n) and (p). The probability mass function of (X) is: ænö P(X = x ) = ç ÷ p x (1 - p) n - x ; x = 0, 1, 2, ...... , n çx÷ è ø Theorem: If (X) is a binomial r.v with parameters (n) and (p), then:

m X = E(X) = n p

s 2 = Var(X) = n p (1 - p) = n p q X
EXAMPLE (3-10):

Suppose that the probability that any particle emitted by a radioactive material will penetrate a certain shield is 0.02. If 10 particles are emitted. Find thee probability that: a- Exactly one particle will penetrate the shield. b- At least two particles will penetrate the shield.
SOLUTION:

ænö P(X = x) = ç ÷ p x (1 - p) n - x çx÷ è ø

; p = 0.02 ; n = 10

æ10 ö a- P(X = 1) = ç ÷ (0.02) 1 (1 - 0.02) 10 -1 ç1÷ è ø
10 10 æ ö b- P(X ³ 2) = å ç ÷ (0.02) x (1 - 0.02) 10 - x ç ÷ x =2 è x ø

Also: P(X = 0) + P(X = 1) + P(X ³ 2) = 1 è P(X ³ 2) = 1 [P(X = 0) + P(X = 1)]
éæ 10 ö ù æ10 ö P( X ³ 2) = 1 - êç ÷ (0.02) 0 (1 - 0.02) 10 -0 + ç ÷ (0.02)1 (1 - 0.02) 10 -1 ú ç 0÷ ç1÷ è ø ëè ø û

-30-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

EXAMPLE (3-11):

Consider the parallel system shown in the figure. The system fails if at least three of the five machines making up the system fail. Find the reliability of the system assuming that the probability of failure of each unit is 0.1 over a given period of time.
SOLUTION:

Let (X) be the number of machines in failure. (X) has a binomial distribution. P(system fails) = P(number of machines in failure ³ 3 ) = P(x ³ 3) æ 5ö æ5ö æ5ö = ç ÷ (p)3 (1 - p) 2 + ç ÷ (p)4 (1 - p) + ç ÷ (p)5 ç 3÷ ç 4÷ ç5÷ è ø è ø è ø P(system fails) = 0.00856 ; when p = 0.1 è Reliability = 1 P(Failure) = 0.99144

EXAMPLE (3-12):

The process of manufacturing screws is checked every hour by inspecting 10 screws selected at random from the hour s production. If one or more screws are found defective, the production process is halted and carefully examined. Otherwise the process continues. From past experience it is known that 1% of the screws produced are defective. Find the probability that the process is not halted.
SOLUTION:

Let (X) be the number of defective items in the sample. P(system is not halted) = P(X = 0) = P(number of defective items is zero) æ10 ö = ç ÷ (p)0 (1 - p)10 - 0 ç 0÷ è ø æ10 ö = ç ÷ (0.01) 0 (0.99)10 - 0 = (0.99)10 = 0.9043 ç 0÷ è ø

EXAMPLE (3-13):

Thirty students in a class compare birthdays. What is the probability that: a- 5 of the students have their birthday in January? b- 5 of the students have their birthday on January 1st? c- At least one student is born in January?
SOLUTION:

a- P(success) =

1 11 ; P(failure) = 12 12 Number of trials (n) = 30 Required number of successes (k) = 5

-31-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

ænö P(5 successes in 30 trials) = ç ÷ (p) k (1 - p) n - k çk÷ è ø 5 30 - 5 5 25 æ 30 ö æ 1 ö æ 11 ö æ 30 ö æ 1 ö æ 11 ö P(5 successes in 30 trials) = ç ÷ ç ÷ ç ÷ =ç ÷ç ÷ ç ÷ ç 5 ÷ 12 ç 5 ÷ 12 è ø è ø è 12 ø è ø è ø è 12 ø 1 364 b- P(success) = ; P(failure) = 365 365 Number of trials (n) = 30 Required number of successes (k) = 5 5 25 ænö æ 30 ö æ 1 ö æ 364 ö P(5 successes in 30 trials) = ç ÷ (p) k (1 - p) n - k = ç ÷ ç çk÷ ç 5 ÷ 365 ÷ ç 365 ÷ ø è ø è ø è øè 11 1 c- P(success) = ; P(failure) = 12 12 é æ 30 ö æ 1 ö 0 æ 11 ö 30 - 0 ù P(X ³ 1) = 1 - P(X = 0) = ê1 - ç ÷ ç ÷ ç ÷ ú = 1 - 0.0735 = 0.9265 ç ÷ ê è 0 ø è 12 ø è 12 ø ú ë û

EXAMPLE (3-14):

The captain of a navy gunboat orders a volley of 25 missiles to be fired at random along a 500-foot stretch of shoreline that he hopes to establish as a beach head. Dug into the beach is a 30-foot long bunker serving as the enemy's first line of defense. What is the probability that exactly three shells will hit the bunker?
SOLUTION:

500 ft 30 ft

30 P(success) = = 0.06 500
ænö P(3 successes in 25 shells) = ç ÷ (p) k (1 - p) n - k çk÷ è ø For p = 0.06 and n = 25

æ 25 ö æ 25 ö P(3 successes in 25 shells) = ç ÷ (0.06) 3 (1 - 0.06) 25 - 3 = ç ÷ (0.06) 3 (0.94) 22 ç3÷ ç3÷ è ø è ø II. The Geometric Distribution Let the outcome of an experiment be either a success with probability (p) or a failure with probability (1 p). Let (X) be the number of times the experiment is performed to the first occurrence of a success. Then (X) is a discrete random variable with integer values ranging from one to infinity. The probability mass function of (X) is:

P(X = x) = P( F F F F4. F S) = P(F) x - 1 P(S) 14 . .3 2
x -1

= (1 - Theorem:

p)x 1 (p) ; x = 1, 2, 3,

The mean and the variance of (X) are: -32-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

X 2 X

= E(X) =

1 p 1- p p2

= Var(X) =

EXAMPLE (3-15):

Let the probability of occurrence of a flood of magnitude greater than a critical magnitude in a given year be 0.02. Assuming that floods occur independently, determine the return period defined as the average number of years between floods.
SOLUTION:

(X) has a geometric distribution with p = 0.01 1 1 = = 50 years X = E(X) = p 0.02

-33-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

III. Hyper-geometric Distribution Consider the sampling without replacement of a lot of (N) items, (k) of which are of one type and (N k) of a second type. The probability of obtaining (x) items in a selection of (n) items without replacement obeys the hyper-geometric distribution: ækö æ N - kö ç ÷ç çx÷ çn - x ÷ ÷ Type I Type II ø P(X = x) = è ø è k (N k) æ Nö ç ÷ çn ÷ è ø (N objects) x=0,1,2, , min(n , k) NOTE: k p= is the ratio of items of type (I) to the total population N - Theorem: The mean and the variance of the hyper-geometric random variable are:
Type I x Type II (n x)

Sample of size (n)

m X = E(X) = n
s 2 = Var(X) = X

k =np N

n k (N - k) (N - n) æ k öæ N-k öæ N-n ö = nç ÷ç ÷ç ÷ 2 N (N - 1) è N ø è N ø è N -1 ø æ k öæ k öæ N-nö æ N-nö = n ç ÷ ç1 - ÷ ç ÷ = n p (1 - p) ç ÷ è N ø è N ø è N -1 ø è N -1 ø

EXAMPLE (3-16):

Fifty small electric motors are to be shipped. But before such a shipment is accepted, an inspector chooses 5 of the motors randomly and inspects them. If none of these tested motors are defective, the lot is accepted. If one or more are found to be defective, the entire shipment is inspected. Suppose that there are, in fact, three defective motors in the lot. What is the probability that the entire shipment is inspected?
SOLUTION:

Let (X) be the number of defective motors found, then (X) assumes the values (0 , 1 , 2 , 3). P(entire shipment is inspected) = P(X ³ 1) = 1 P(X = 0) æ 3 ö æ 47 ö ç ÷ç çx÷ ç5 - x÷ ÷ è øè ø P( X = x ) = æ 50 ö ç ÷ ç5 ÷ è ø æ 3 ö æ 47 ö ç ÷ç ÷ ç0÷ ç 5 ÷ P( X = 0) = è ø è ø = 0.72 (The lot is accepted) æ 50 ö ç ÷ ç5÷ è ø P(X ³ 1) = 1 0.72 = 0.28

-34-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

EXAMPLE (3-17):

A committee of seven members is to be formed at random from a class with 25 students of whom 15 are girls. Find the probability that: a- No girls are among the committee b- All committee members are girls c- The majority of the members are girls
SOLUTION:

Let (X) represents the number of girls in the committee. æ15 ö æ10 ö ç ÷ç ÷ ç 0÷ç 7 ÷ a- P(X = 0) = è ø è ø æ 25 ö ç ÷ ç7÷ è ø æ 15 ö æ 10 ö ç ÷ç ÷ ç 7 ÷ç 0 ÷ b- P(X = 7) = è ø è ø æ 25 ö ç ÷ ç7÷ è ø c- P(majority are girls) = P(X = 4) + P(X = 5) + P(X = 6) + P(X = 7) æ15 ö æ 10 ö ç ÷ç ÷ 7 ç x ÷ç 7- x÷ è øè ø =å æ 25 ö x =4 ç ÷ ç7÷ è ø - Theorem: For large (N), one can use the approximation: ænö k P ( X = x ) @ ç ÷ P x ( 1 - P ) n- x ; P = ç x÷ N è ø This approximation gives very good results if
P= n £ 0.1 , for the example above: N

æ5ö 3 = 0.06 è P( X = 0) @ ç ÷ (0.06)0 (1 - 0.06) 5-0 = 0.733 ç 0÷ 50 è ø IV. Poisson Distribution - Definition: A discrete random variable (X) is said to have a Poisson distribution if it has the following probability mass function: bx P ( X = x ) = e -b ; x=0,1,2, where (b) is a positive constant. x! - Theorem: If (X) is a Poisson r.v with parameter (b), then its mean and variance are: m X = E(X) = b
2 s X = Var(X) = b

-35-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

- Poisson Process: Consider a counting process in which events occur at a rate of (l) occurrence per unit time. Let X(t) be the number of occurrences recorded in the interval (0 , t), we define the Poisson process by the following assumptions: 1- X(0) = 0 , i.e., we begin the counting at time t = 0. 2- For non-overlapping time intervals (0 , t1) , (t2 , t3), the number of occurrences {X(t1) and {X(t 3) X(t2)} are independent. X(0)}

3- The probability distribution of the number of occurrences in any time interval depends only on the length of that interval. 4- The probability of an occurrence in a small time interval (Dt) is approximately (l Dt). X(t0) t=0 X(t1) t1 X(t2) t2 X(t3) t3

Using the above assumptions, one can show that the probability of exactly (x) occurrences in any time interval of length (T) follows the Poisson distribution and,

P ( X = x ) = e - lT
- Theorem:

( lT ) x x!

; x=0,1,2,3,

Let (b) be a fixed number and (n) any arbitrary positive integer. For each nonnegative integer (x): æn ö bx ; where p = b/n Lim ç ÷ (p) x (1 - p) n - x = e - b n ®¥ ç x ÷ x! è ø

EXAMPLE (3-18):

Messages arrive to a computer server according to a Poisson distribution with a mean rate of 10 messages/hour. a- What is the probability that 3 messages will arrive in one hour. b- What is the probability that 6 messages will arrive in 30 minutes.
SOLUTION:

a- l = 10 messages/hour è T = 1 hour (10 ´ 1) x (10) x ; x=0,1,2,3, P( X = x ) = e -10´1 = e -10 x! x! (10) 3 P( X = 3) = e -10 3! b- l = 10 messages/hour è T = 0.5 hour 1 x 1 (10 ´ ) x -10´ 2 = e -5 (5) P( X = x ) = e 2 ; x= 0, 1 ,2 ,3 , x! x! (5) 6 P ( X = 6) = e -5 6!

-36-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

EXAMPLE (3-19):

The number of cracks in a section of a highway that are significant enough to require repair is assumed to follow a Poisson distribution with a mean of two cracks per mile. a- What is the probability that there are no cracks in 5 miles of highway? b- What is the probability that at least one crack requires repair in ½ miles of highway? c- What is the probability that at least one crack in 5 miles of highway?
SOLUTION:

a- l = 2 cracks/mile è T = 5 miles ( 2 ´ 5) x (10) x ; x=0,1,2,3, P(X = x ) = e - 2´5 = e -10 x! x! P(X = 0) = e-10 b- l = 2 cracks/mile è T = 5 miles 1 x 1 (2 ´ ) x -1 - 2´ 2 = e -1 (1) = e ; x = 0 , 1 , 2 , 3 , P( X = x ) = e 2 x! x! x! -1 ¥ e P(X ³ 1) = å = [1 - P(X = 0)] = 1 - e -1 x =1 x! c- l = 2 cracks/mile è T = 5 miles ( 2 ´ 5) x (10) x ; x=0,1,2,3, P( X = x ) = e - 2´5 = e-10 x! x! ¥ e -10 (10) x P(X ³ 1) = å = [1 - P(X = 0)] = 1 - e -10 x! x =1
EXAMPLE (3-20):

Given 1000 transmitted bits, find the probability that exactly 10 will be in error. Assume that 1 the bit error probability is . 365
SOLUTION:

X: random variable representing number of bits in error. Exact solution: 1 P(bit error) = ; Number of trials (n) = 1000 365 Required number of bits in error (k) = 10 10 990 ænö æ1000 ö æ 1 ö æ 364 ö ÷ç P(X = 10) = ç ÷ (p) k (1 - p ) n - k = ç çk÷ ç 10 ÷ 365 ÷ ç 365 ÷ ø è ø è ø è øè Approximate solution: bx 1 1000 P( X = x ) = e -b ; b = n p = 1000 ´ = x! 365 365 b10 P( X = 10) = e - b 10 ! Exercise: Perform the computation and compare the difference

-37-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

§ Common Continuous Random Variables:
I. Exponential Distribution: It is said that a random variable (X) has an exponential distribution with a parameter l (l > 0) if (X) has a continuous distribution for which the pdf fX(x) is given as: ; x ³0 f X ( x ) = l e - lx The cumulative distribution function is: ; x ³0 F X ( x ) = 1 - e - lx fX(x) FX(x) - Definition:

x

x

The exponential distribution is often used in a practical problem to represent the distribution of the time that elapses before the occurrence of some event. It has been used to represent the such periods of time as the period for which a machine or an electronic component will operate without breaking down, the period required to take care of a customer at some service facility, and the period between the arrivals of two successive customers at a facility. If the event being considered occurs in accordance with a Poisson process, then both the waiting time until an event will occur and the period of time between any two successive events will have exponential distribution.
Occurrence of events Time

x - Theorem:

x

If the random variable (X) has an exponential distribution with parameter (l), then: 1 m X = E(X) = and

l

2 s X = Var(X) =

1

l2

Exercise The number of telephone calls that arrive at a certain office is modeled by a Poisson random variable. Assume that on the average there are five calls per hour. a. What is the average (mean) time between phone calls? b. What is the probability that at least 30 minutes will pass without receiving any phone call? c. What is the probability that there are exactly three calls in an observation interval of two consecutive hours? d. What is the probability that there is exactly one call in the first hour and exactly two calls in the second hour of a two-hour observation interval? -38-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

EXAMPLE (3-21):

Suppose that the depth of water, measured in meters, behind a dam is described by an exponential random variable with pdf: -x ì 1 13.5 e x >0 ï f X (x) = í13.5 ï 0 o.w î There is an emergency overflow at the top of the dam that prevents the depth from exceeding 40.6 m. There is a pipe placed 32.0 m below the overflow that feeds water to a hydroelectric generator (turbine). a- What is the probability that water is wasted though emergency overflow? b- What is the probability that water will be too low to produce power? c- Given that water is not wasted in overflow, what is the probability that the generator will have water to derive it?
SOLUTION:
¥

1 13.5 a- P(water wasted through emergency) = P(X ³ 40.6 m) = ò e dx = e -3 13.5 40.6
b- P(water too low to produce power) = P(x < 8.6 m)
= 1 - e -0.637 = 0.47

-x

(

)

c- P(generator has water to derive it / water is not wasted) = P(x > 8.6 / x < 40.6) -x 40.6 1 13.5 ò 13.5 e dx P(x > 8.6 I x < 40.6) P(8.6 < x < 40.6) 8.6 = = = 40.6 = 0.504 -x P(x < 40.6) P(x < 40.6) 1 13.5 -3 ò 13.5 e dx = e 0 II. Rayleigh Distribution: The Rayleigh density and distribution functions are: -x 2 f X (x) = x e b ; x ³0 b

FX (x) = 1 - e ; x ³0 The Rayleigh pdf describes the envelope of white noise when passed through a band pass filter. It is used in the analysis of errors in various measurement systems.
- Theorem:
X

-x 2 b

= E(X) =

b 4

and

2 X

= Var(X) =

b(4 - ) 4

III. Cauchy Random Variable: This random variable has: 1 1 a/p æ xö fX ( x )= 2 , F X ( x ) = + tan - 1 ç ÷ 2 2 p x +a èa ø

-39-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

IV. Gaussian (Normal) Distribution : - Definition: A random variable (X) with pdf:
f X (x) = 1 2ps
2 X -( x - m X )2

e

2 s2 X

- ¥ < x < ¥

has a normal distribution with parameters ( m X ) and ( s 2 ) where - ¥ < x < ¥ and s 2 ³ 0 . X X Furthermore: E( x ) = m X ; Var ( x ) = s 2 X Infinite number of normal distributions can be formed by different combination of parameters.

m1 m2 s1 = s2

m1 = m2 s1 s2

m1 s1

m2 s2

- Definition: A normal random variable with mean zero and variance one is called a standard normal random variable. A standard normal random variable is denoted as Z. fX(x)
1 2ps2 X 0 .607 2 p s2 X

fZ (z)

X

-

X

mX

X

+

X

x

0

z

z

- Definition: The function (z) = P{Z £ z} is used to denote the cumulative distribution function of a standard normal random variable:
e du 2p This function is tabulated for z ³ 0 For z < 0 ; (z) = 1 ( z)

(z) =

ò

z

1

-

u2 2

-40-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

X

-4

X

X

-3

X

X

-2

X

X

-

X

X

X

+

X

X

+2

X

X

+3

X

X

+4

x - Scale
X

z - Scale
-4 -3 -2 -1 0 1 2 3 4

fZ (z)

0

1

z

a

b

(1)

(b) -

(a)

1

-1

Area = 1 -

(1)

(-1) = 1 -

(1)

Area = =
-1 1

(1) (1)

(-1) [1 1 (1)]

= 2 (1)

-41-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

- Cumulative Distribution Function: P(X £ x) = FX ( x ) =

ò

x

1 2 p s2 X

- ( x - m X )2

e

2 s2 X

dx
è dx = s X du

æx- X Let u = ç ç è X

ö ÷ ÷ ø

è du =

dx sX

FX ( x ) =
z

x -m X sX

ò

1 2 ps 2 X
-u2 2

e

-u2 2

s X du

(z) =

ò

1 2p

e

du

æZ- X ö ÷ (z) = F ç ç ÷ è X ø Therefore, we conclude that: æx - X 1- P(X £ x0) = F ç 0 ç è X æx 2- P(x0 £ X £ x1) = F ç 1 ç è
EXAMPLE (3-22):

ö ÷ ÷ ø X

X

ö ÷ ÷ ø

æx Fç 0 ç è X

X

ö ÷ ÷ ø

Suppose the current measurements in a strip of wire are assumed to follow a normal distribution with a mean of 10 mA and variance 4 (mA)2. What is the probability that a measurement will exceed 13 mA?
SOLUTION:

X = current in mA
æXZ=ç ç X è
X

ö æ X - 10 ö ÷=ç ÷ è 2 ÷ ø ø

ì ü æ X - 10 ö æ 13 - 10 ö P(X > 13) = P í Z = ç ÷>ç ÷ = 1.5 ý è 2 ø è 2 ø î þ P(X > 13) = P {Z > 1.5 } = 1 - F (z ) è From tables: = 1 0.93319 = 0.06681
fX(x)

fZ(z)

10

13

x

0

1.5

z

-42-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

EXAMPLE (3-23):

The diameter of a shaft in an optical storage drive is normally distributed with mean 0.25 inch and standard deviation of 0.0005 inch. The specifications on the shaft are 0.25 ± 0.0015 inch. What proportion of shafts conforms to specifications?
SOLUTION:
fX(x)

P(0.2485 < X < 0 .2515) ìæ 0.2485 - 0.2500 ö æ 0.2515 - 0.2500 ö ü = P íç ÷<Z <ç ÷ý 0.0005 0.0005 ø è øþ îè = P {- 3 < Z < 3}
0.2485 0.25 0.2515

fZ(z)

= =2 =

(3) (3) (3)

( 3) 1 è From tables: 1 = 0.9973
-3 0 3

( 3) = (2 x 0.99865)

EXAMPLE (3-24):

Assume that the height of clouds above the ground at some location is a Gaussian random variable (X) with mean 1830 m and standard deviation 460 m. find the probability that clouds will be higher than 2750 m.
SOLUTION:
fX(x)

P(X > 2750) = 1 - P(X £ 2750)

ì æ 2750 - 1830 ö ü = 1 - PíZ £ ç ÷ý 460 è øþ î
= 1 - P( Z £ 2 . 0 )

x 1830 2750

fZ(z)

=1 =1

(2.0) è From tables: 0.9772
0 2.0

P(X > 2750) = 0 .0228

z

Exercise The tensile strength of paper is modeled by a normal distribution with a mean of 35 pounds per square inch and a standard deviation of 2 pounds per square inch. a. If the specifications require the tensile strength to exceed 33 lb/in2 , what is the probability that a given sample will pass the specification test? b. If 10 samples undergo the specification test, what is the probability that at least 9 will pass the test? c. If 20 samples undergo the test, what is the expected number of samples that pass the test? Exercise The rainfall over Ramallah district follows the normal distribution with a mean of 600 mm and a standard deviation of 80 mm. The rainfall is distributed over 500 km2 area. Find: -43-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

1. The probability of obtaining a rainwater volume less than 206 MCM (MCM = Million Cubic Meter) 2. Find the mean and the standard deviation of the volume (V) of rainfall in MCM. 3. Flooding condition will be considered if the rainfall is higher than 900 mm. Find the probability of flooding for any given year. - Remark: The area under the Gaussian curve within (k) standard deviations of the mean is given in the following table: k 1 2 3 4 Area P(m X - ks X £ X £ m X + ks X ) 0.6826 0.9544 0.9973 0.99994 è Total probability outside an interval of 4 standard deviations on each side of the mean is only 0.00006

§ Normal Approximation of the Binomial and Poisson Distribution:
- Theorem: De-Moiver-Laplace For large (n) the binomial distribution

ænö x 1 ç ÷ p (1 - p) n - x ~ e 2npq ( ~ : asymptotically equal) çx÷ 2pnpq è ø Which is a normal distribution with mean (n p) and variance (n p q). Therefore, if (X) is
a binomial r.v, then Z = ç
æX-n pö ÷ is approximately a standard normal r.v. ç npq÷ è ø

- ( x - np ) 2

The theorem gives better results when (n p > 5) and (n p q > 5) b ænö P(a £ X £ b ) = å ç ÷ p x (1 - p ) n - x ~ F (b ) - F ( a ) where: ç ÷ x=a è x ø
æ b-np b=ç ç npq è ö æ ÷ and a = ç a - n p ÷ ç npq ø è ö ÷ ÷ ø

EXAMPLE (3-25):

Consider a binomial experiment with n = 1000 and p = 0.2. if X is the number of successes, find the probability that X £ 240 .
SOLUTION:

æ 1000 ö ÷ (0.2 )x (1 - 0.2 )1000 - x ÷ x=0 è x ø Applying the Demoiver-Laplace theorem:

Exact solution: P(X £ 240) =

åç ç

240

æ 240 - 1000 ´ 0 . 2 ö ÷ = F (3 . 162 ) = 0 . 999216 P(X < 240 ) = F ç ç ÷ è 1000 ´ 0 . 2 ´ 0 . 8 ø

-44-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

- Theorem: If (X) is a Poisson r.v with E(X) = b and Var(X) = b, then:
- (x - b)2 2b

æX -b Z=ç ç b è

ö ÷ ÷ ø

is approximately a standard normal r.v. The approximation is good for (b > 5).

bx e ® x!
-b

1 e 2 b

EXAMPLE (3-26):

Assume the number of asbestos particles in a cm3 of dust follow a Poisson distribution with a mean of 1000. If a cm3 of dust is analyzed, what is the probability that less than 950 particles are found in 1 cm3?
SOLUTION:

Exact solution: P(X £ 950) = å e -1000
x =0

950

(1000 ) x x!

Approximate:

ì 950 - 1000 ü P(X < 950 ) = P í Z £ ý = P {Z £ - 1 .58} = 0 .057 1000 þ î

§ Transformation of Random Variables:
Let (X) be a random variable with a pdf fX(x). If Y = g(X) is a function of (X), then (Y) is a random variable. Its pdf is to be determined. The function g(X) is a single valued function of its argument. I. Discrete Case:
EXAMPLE (3-27):

Let (X) be a binomial r.v with parameters (n = 3) and (p = 0.75). Let Y = g(x) = 2X + 3 P(Y = y) = P(X = x)
SOLUTION:

The table below shows the (x) and (y) values and their probabilities. ænö x P(X = x ) = ç ÷ p x (1 - p) n - x Y = g(X) ç x÷ è ø
(0, 1, 2, 3)
y = 2x + 3
10 8 (y) 6 4 2 0 0 1 2 (x) 3 4

y
(3, 5, 7, 9)

-45-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

x 0 1 2 3

y 3 5 7 9

P(X = x) (1 p)3 3 p (1 p)2 3 p2 (1 p) p3

P(Y = y) (1 p)3 3 p (1 p)2 3 p2 (1 p) p3

EXAMPLE (3-28):

1 ; x = -3 , -2 , -1 , 0 , 1 , 2 6 Define Y = g(x) = X 2. Find the pdf of the random variable Y.

Let (X) has the distribution P{X = x} =

SOLUTION:

P(X=x) = 1/6

x 7 -2 -1 0 1 2

y 9 4 1 0 1 4

P(X = x) 1/6 1/6 1/6 1/6 1/6 1/6

P(Y = y) 1/6 1/6 1/6 1/6 1/6 1/6

-3

-2

-1

0

1

2

x

Probability Density Function fX(x)
P(Y=1) P(Y=0) P(Y=4) P(Y=9)

The distribution of Y is: P(Y = 0) = P(Y = 1) = P(Y = 4) = P(Y = 9) = 1/6 2/6 2/6 1/6
0 1 4 9

y

Probability Density Function fY(y)

II. Continuous Case: Let Y = g(X) be a monotonically increasing or decreasing function of (x).
P(x < X < x + D x ) = P {y ( x ) < Y < y ( x + D x )} P(x < X < x + D x ) = P {y < Y < y + D y}

y
y(x+Dx) y(x)

y + Dy y

fX(x) Dx = fY(y) Dy

Dx f X (x ) f X (x ) f Y ( y) = f X ( x ) = = Dy Dy dy Dx dx y1 < y < y2

x

x+Dx

x

EXAMPLE (3-29):

Let (X) be a Gaussian r.v with mean (0) variance (1). Let Y = X2. Find fY(y) -46-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

SOLUTION:

f Y ( y) = 2

f X ( x) dy dx

0£y£¥ ;

dy = 2x dx

2 f Y ( y) = 2x f Y ( y) = f Y ( y) = 1 y 1

1 2p 1 2p e

e e

- (x )2 2

,

x=

y

-y 2

-y 2

2 py

;

y³0

EXAMPLE (3-30):

Let (X) be a uniform r.v in the interval (-1 , 4). If Y = X2. Find fY(y)
SOLUTION:

For (-1 £ X £ 1) : è For (1 < X £ 4 ) : ì 1 ï ï5 y ï ï 1 f Y ( y) = í ï10 y ï ï 0 ï î è

f Y ( y) = 2
f Y ( y) =

f X (x ) 2 ´ 1 5 1 = = dy dx 2x 5 y

f X (x) 1 5 1 = = dy dx 2 x 10 y

y
16

0 < y £1 1 < y £ 16 Otherwise
-1 1 4

x

-47-

SINGLE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

CHAPTER III

EXAMPLE (3-31):

ìl e - lx x > 0 Let (X) be a r.v with the exponential pdf: f X ( x) = í x <0 î 0 Let Y = 2X + 3. Find fY(y) and the region over which it is defined.
SOLUTION:

f Y ( y) =

f X ( x) dy dx

;

dy =2 dx

f Y ( y) =

ï e ï f Y (y) = í 2 ï 0 ï î

f X (x ) y -3 , but x = 2 2 y -3 ì l -l ( 2 ) y - 3
>0 2 y-3 <0 2

Þ

æ y -3ö fX ç ÷ è 2 ø f Y ( y) = 2
ì l -l æ y -3 ö ç ÷ ï e è2 ø f Y (y) = í 2 ï 0 î y>3 y <3

è

NOTE: P(3 < Y £ 5) = P(0 < X £ 1) =

ò le
0

1

-lx

dx = 1 - e - l

EXAMPLE (3-32):

Let (X) be a Gaussian r.v with mean ( m X ) variance ( s 2 ) X Let (Y) = aX + b be any r.v. Find fY(y)
SOLUTION:

Y = aX + b

è

dy y-b = a and x = dx a
-(x - mX )2 2 X -(

1 f Y ( y) = a

1 2ps
1 2 p s2 Y

e

2 s2 X

=

1 2 p (a s X )
2

e

y-b - m X )2 a 2 s2 X

=
2

1 2 p (a s X )
2 X

- ( y - b -a m X ) 2 2

e

2 (a s X ) 2

but from previous results we have: m Y = a m x + b and s = a s hence,
2 Y

- ( y - m Y )2

f Y ( y) =

e

2 s2 Y

Therefore, Y is Gaussian with mean ( m Y = a m X ) and variance ( s 2 = a2 s 2 ) Y X SPECIAL CASE: If (X) is a Gaussian r.v with mean ( m X ) variance ( s 2 ), then: X the r.v Z = ç ç
è æXX X

ö 2 ÷ is a Gaussian r.v with mean ( m Z = 0) variance ( s Z = 1). ÷ ø

That is (Z) is a standard normal random variable. GENERAL RESULT: A linear transformation of a Gaussian random variable is also Gaussian.

-48-

PROBABILITY DISTRIBUTIONS FOR MORE THAN ONE R.V.

CHAPTER IV

C H A P T ER I I I
P R O B A B I LI T Y D I S T R I B U T I O N S F O R M O R E T H A N O NE R A N D O M V A R IA B L E

PROBABILITY DISTRIBUTIONS FOR MORE THAN ONE R.V.

CHAPTER IV

In certain experiments we may be interested in observing several quantities as they occur, such as carbon content (X) and hardness (Y) of steel; input (X) to a system and output (Y) at a given time too. - If we observe two quantities (X) and (Y), each trial gives a pair of values X = x and Y = y, (x,y) which represents a point (x,y) in the x-y plane. - The joint cumulative distribution function of two r.v X and Y is defined as: FXY(x,y) D P{X £ x , Y £ y} Event (A) = {X £ x} Event (B) = {Y £ y}

y
(x,y)

x
A B

R

R S
A B

I. Discrete Two Dimensional Distribution: A random variable (x,y) and its distribution are called discrete if (x,y) can assume only countably finite or at most countably infinite pairs of values (x1,y1), (x2,y2), The joint probability mass function of (X) and (Y) is: Pij = P{X = xi , Y = yj } such that FXY(x,y) = and
xi £ x y j £ y

å åP
ij i j

ij

ååP

=1

II. Continuous Two Dimensional Distribution: A random variable (x,y) and its distribution are called continuous if FXY(x,y) can be given by:
FXY (x, y) =
y x

-¥ - ¥

òòf

XY

( x , y) dx dy

where fXY(x,y) is the joint probability density function (f being continuous and nonnegative)

-47-

PROBABILITY DISTRIBUTIONS FOR MORE THAN ONE R.V.

CHAPTER IV

- Properties of the joint pdf: 1- fXY(x,y) ³ 0
¥ ¥

y y2

2-

-¥ - ¥

òò

f XY ( x , y) dx dy = 1
y2 x 2

R
y1

3- P(x 1 < X £ x 2 , y1 < Y £ y 2 ) = and in general:

y1 x 1

òò

f XY ( x, y) dx dy

x1 y

x1

x

P(x , y Î R) = òò f XY ( x , y) dx dy
R

R
x

§ Marginal Distributions of a Discrete Distribution:
P(X = xi) = P(X = xi , Y arbitrary) = å P(X = x i , Y = y j )
y

This is the probability that (X) may assume a value (x), while (Y) may assume any value which we ignore. Likewise: P(Y = yj) =

å P(X = x
x

i

, Y = yj)
y

§ Marginal Distributions of a Continuous Distribution:
For a continuous distribution we have: x æ¥ ö FX (x) = P(X £ x) = ò ç ò f XY (x , y) dy ÷dx ç ÷ -¥ è - ¥ ø d but f X (x) = FX (x) dx
¥

è f X (x) = è f Y (y) =

-¥ ¥

òf

XY

( x , y) dy ; Marginal pdf
x

x

òf

XY

( x , y ) dx ; Marginal pdf

§ Independence of Random Variable:
- Theorem: FXY(x,y) = FX(x) FY(y) holds for all (x,y), or equivalently: fXY(x,y) = fX(x) fY(y) Proof: FXY(x,y) = P{X £ x , Y £ y} Let: A: event {X £ x} B: event {Y £ y} A and B are independent if: P(A B) = P(A) P(B) P(X £ x ,Y £ y) = P(X £ x) P(Y £ y) -48-

Region such that: {X x}

Two random variables (X) and (Y) are said to be independent if:

PROBABILITY DISTRIBUTIONS FOR MORE THAN ONE R.V.

CHAPTER IV

§ Conditional Densities:
Let (X) and (Y) be discrete random variables. The conditional probability density function of (Y) given (X = x), that is the probability that (Y) takes on the value (y) given that (X = x), is given by: P(X = x , Y = y) f Y / X ( y) = P(Y = y / X = x) = P( X = x ) If (X) and (Y) are continuous, then the conditional pdf of (Y) given (X = x) is given by: f X , Y (x, y) f Y / X ( y) = f X (x)
EXAMPLE (4-1):

Let (X) and (Y) be continuous random variables with joint pdf: 1 f X , Y ( x , y ) = (6 - x - y ) ; 0 < x £ 2 , 2 < y £ 4 y 8 1- Find fX(x) and fY(y). 4 2- Find fY/X(y). 3- Find P(2<y £ 3) 3 4- Find P(2<y £ 3 / x=1)
2 1
XY

R
x
1 2

SOLUTION:
¥

1-

f X (x) =
=

ò 8 (6 - x - y) dy = 8 (6 - 2x )
2 ¥ -¥

-¥ 4

òf
1

( x , y) dy 1
; 0<x £ 2

0

f Y (y) =
2-

ò

f XY ( x, y) dx =
f X , Y ( x , y) f X (x)

ò 8 (6 - x - y) dx = 4 (5 - y)
0

2

1

1

; 2<y £ 4

f Y / X ( y) =

1 (6 - x - y ) (6 - x - y ) = 8 = ; 0<x £ 2 1 (6 - 2 x ) (6 - 2 x ) 8 3 3 1 5 3- P(2<Y £ 3) = ò f Y (y) dy = ò (5 - y) dy = 4 8 2 2 (5 - y ) 4- f Y / X ( y / x = 1) = ; 2<y£ 4 4 3 5- y 5 P(2<Y £ 3 / X=1) = ò dy = 4 8 2
Exercise: 1- Find P(2<Y £ 3 / 0 £ X £ 1) 2- Find m X , Y , 2 , and 2 X Y 3- Are X and Y independent? -49-

,

2<y £ 4

PROBABILITY DISTRIBUTIONS FOR MORE THAN ONE R.V.

CHAPTER IV

EXAMPLE (4-2):

Suppose that the random variable (X) can take only the values (1 , 2 , 3) and the random variable (Y) can take only the values (1 , 2 , 3 , 4). The joint pdf is shown in the table. Y X 1 2 3

1 0.1 0.3 0

2 0 0 0.2

3 0.1 0.1 0

4 0 0.2 0

f XY (x, y)
0.3
X

0.2 0.2 0.1

3 2 1
1 2 3

0.1 0.1

1- Find fX(x) and fY(y). 2- Find P(X ³ 2) 3- Are (X) and (Y) independent.
SOLUTION:

4

Y

1- P(X = 1) = 0.2 P(X = 2) = 0.6 P(X = 3) = 0.2

P(Y = 1) = 0.4 P(Y = 2) = 0.2 P(Y = 3) = 0.2 P(Y = 4) = 0.2

= 1.0 3- Check all pairs (x,y) for:

= 1.0

2- P(X ³ 2) = P(X = 2) + P(X = 3) = 0.6 + 0.2 = 0.8 P(X = x ,Y = y) = P(X = x) P(Y = y) P(X = 1 , Y = 1) = 0.1 (0.2 x 0.4) = 0.08 è we do not continue è X and Y are not independent Exercise: Find m X and s 2 X -50?

PROBABILITY DISTRIBUTIONS FOR MORE THAN ONE R.V.

CHAPTER IV

EXAMPLE (4-3):

Let (X) and (Y) have the joint pdf: ìk x y , 0 £ x £ 1, 0 £ y £ 1, f XY ( x , y) = í î 0 , otherwise abcdFind (k) so that fX(x) is a proper pdf. Find P(X > 0.5 , y > 0.5) Find fX(x) and fY(y) Are (X) and (Y) independent.

y
1

R
0.5

SOLUTION:

æ1 ö a- ò ò f XY ( x , y) dx dy = 1 è k ò yç ò x dx ÷ dy = 1 (0,0) ç ÷ -¥ - ¥ 0 è0 ø 1 æ 2 1 ö 1 2 1 ç x ÷ dy = k y dy = k y = k è \ k = 1 Þ k = 4 kò y ç 2 ÷ 2ò 2 2 0 4 4 0 è 0 0 ø
¥ ¥

1

x
0.5 1

b- P(X > 0.5 , y > 0.5) =

0.5 0.5

ò ò 4 x y dx dy

1

1

æ x2 4ç ç 2 è
c-

1

ö æ y2 ÷ç ÷ç 2 0.5 ø è
1 0

1

ö ÷ ÷ 0.5 ø

= (1 - 0.25) ´ (1 - 0.25) = 0.75 ´ 0.75 = 0.5625

f X (x) = ò f XY ( x , y ) dy

y2 f X (x) = ò 4 x y dy = 4 x 2 0

1

1

=2x
0 1

x2 2 0 Since fXY(x,y) = fX(x) fY(y)
f Y (y) = ò 4 x y dx = 4 y

1

=2y
0

è 4 x y = (2 x) (2 y) è (X) and (Y) are independent.

EXAMPLE (4-5):

For example (4-3), find P(X > Y).
SOLUTION:
1
1 x

y
1 1

P(Y < X) = òò 4 x y dx dy = ò
R

0 0

ò 4 x y dy dx = ò ò 4 x y dx dy
0 y

y=x

æ y2 = ò4x ç ç 2 0 è
1

1 1 2 ö ÷ dx = 4 x x dx = 2 x 3 dx ò 2 ò ÷ 0 0 0 ø æ x4 1 ö 1 ÷= P(Y < X) = 2 ç ç 4 ÷ 2 è 0ø x

x
1

-51-

PROBABILITY DISTRIBUTIONS FOR MORE THAN ONE R.V.

CHAPTER IV

EXAMPLE (4-4):

Two random variables (X) and (Y) have the joint pdf: ì5 2 ï x y , 0< y< x <2 f XY ( x , y) = í16 ï 0 , otherwise î a- Verify that fXY(x,y) is a valid pfd. b- Find the marginal density functions of X and Y. c- Are X and Y statistically independent? d- Find P{X<1} , P{Y<0.5} , P{XY<1}
SOLUTION:
2 x

y
2

R
x
2

a-

òò f
R

XY

( x , y) dx dy =

ò ò 16 x
0 0

5

2

y dy dx
2 4 5 ö ÷ dx = 5 x dx = 5 ´ 1 ´ x ÷ 16 ò 2 16 2 5 0 0 ø x
x

2 2 æ y2 æx ö 5 5 x 2 ç ò y dy ÷ dx = x2ç ÷ 16 ò ç 0 16 ò ç 2 0 0 è ø è
¥

2

=
0

5 1 32 ´ ´ =1 16 2 5

b-

f X (x) =

ò

f XY ( x , y) dy è f X (x) = ò

5 2 y2 5 2 x x y dy = 16 16 2 0
2

x

=
0

5 4 x 32
2

ì5 4 ï x , f X ( x ) = í 32 ï 0 , î
¥

0<x<2 otherwise

5 4 5 x5 è check ò x dx = 32 2 0 32
2

= 1 Þ OK
0

5 x3 5 y f Y (y) = ò f XY ( x , y) dx è f Y (y) = ò x 2 y dx = 16 3 16 -¥ y

2

=
y

5 y(8 - y 3 ) 48

2 ì5 3 2 5 5 æ 2 y5 ï y(8 - y ) , 0 < y < 2 4 ç 4y è ò f Y ( y) = í 48 (8y - y ) dy = = 1 Þ OK 48 48 ç 5 0 è 0 ï 0 , otherwise î c- Since fXY(x,y) fX(x) fY(y) è (X) and (Y) are not statistically independent.

d- P{X<1} =

ò
0

1

5 4 5 x5 f X (x) dx = ò x dx = 32 5 0 32
0.5

1

1

=
0

1 = 0.03125 32

P{Y<0.5} = =

òf
0

0 .5 Y

(y) dy =
4

ò 48 y(8 - y
0

5

3

) dy

y
2

5 æ 2 y ç 4y 48 ç 4 è

0.5

=
0

105 = 0.1025 1024

P{XY<1} = P{Y< P{Y<
1 }= X

1 } X
( x , y) dx dy
1 2

y=
1

1 x

òò f
R
1 x

XY

1 P{Y< } = X

x 5 2 5 x y dy dx + ò ò x 2 y dy dx ò ò 16 16 0 0 1 0

R
1 2

x

-52-

PROBABILITY DISTRIBUTIONS FOR MORE THAN ONE R.V.

CHAPTER IV

§ Addition of Means and Variances:
- Review: (Basic operations on a single random variable)
¥

- E(X) =

òx f
¥ -¥

X

(x) dx

- E{g(X)} = ò g(X) f X (x) dx - E{g1 (X) + g 2 (X)} = E{g1 (X)}+ E{g 2 (X)} - If Y = aX + b E(Y) = a E(X) + b - Definition: The expected value of a function g(x,y) of two random variables (X) and (Y) is: E{g(x,y)} = = è

Y

=a

X

+b

è

2 Y

= a2

2 X

å å g(x
xi yj
¥ ¥

i

, y j ) P(X = x i , Y = y j ) ; X and Y are discrete
XY

-¥ - ¥

ò ò g(x , y) f

( x , y) dx dy

; X and Y are continuous

Since summation and integration are linear processes, we have: E{a g1(x,y) + b g2(x,y)} = a E{g1(x,y)} + b E{g2(x,y)} - Theorem: Addition of Means
The mean or expected value of a sum of random variables is the sum of the expectations.

E(x1 + x2 +

. + xn) = E(x1) + E(x2) +

. + E(xn)

- Theorem: Multiplication of Means
The expected value of the product of independent r.v equals the product of the expected values.

E(x1 x2 Proof:

. Xn) = E(x1) E(x2)

. E(xn)

If (X) and (Y) are independent random variables, then fXY(x,y) = fX(x) fY(y), so:
¥ ¥

E(XY) =

-¥ - ¥

ò

ò x y f XY (x, y) dx dy = ò x f X (x ) dx

¥

¥

òy f

Y

( y) dy = E(X) E(Y)

And in general, if (X) and (Y) are independent, then: E{g1(X) g2(Y)} = E{g 1(X)} E{g2(Y)} - Theorem: Addition of Variances - Definition: The correlation coefficient between two random variables (X) and (Y) is:

E{(X XY

X X

)(Y Y

Y

)}

=
X

XY Y

where m XY is called the covariance and r XY is bounded between - 1 £ r XY £ 1 when r XY = 0 , (X) and (Y) are said to be uncorrelated. when r XY = ±1 , (X) and (Y) are said to be fully correlated. -53-

PROBABILITY DISTRIBUTIONS FOR MORE THAN ONE R.V.

CHAPTER IV

- Theorem: Let Y = a1X1 + a2X2 , then
2 Y 2 = a1 2 X1

+ a2 2

2 X2

+ 2 a 1a 2

X1

X2

X1X 2

Proof:
s 2 = E{(Y - m Y ) 2 } = E{(a 1 X 1 + a 2 X 2 - a 1m X1 - a 2 m X 2 ) 2 } Y = E{[a 1 (X 1 - m X1 ) + a 2 (X 2 - m X 2 )]2 }
2 = E{a 1 (X 1 - m X1 ) 2 } + E{a 2 (X 2 - m X 2 ) 2 } + 2 a 1 a 2 E{(X 1 - m X1 )(X 2 - m X 2 )} 2 2 = a 1 s 2 1 + a 2 s 2 2 + 2 a 1a 2 m XY X 2 X

m XY 2 \ 2 = a1 2 1 + a 2 Y X 2 sX sY - Theorem: Multiplication of Means

since r XY =

2 X2

+ 2 a1a 2

X1

X2

XY

If (X) and (Y) are independent random variables, then they are uncorrelated. Proof:
XY

= E{(X - X )(Y - Y )} = E{XY} - m Y E{X} - m X E{Y} + m X m Y = E{XY} - E{X}E{Y}

But since (X) and (Y) are independent, then E{XY} = E{X}E{Y} m è m XY = 0 è r XY = XY = 0 sX sY - Theorem: Let Y = a1X1 + a2X2 , and (X) and (Y) are independent random variables, then 2 s2 = a1 s2 1 + a 2s2 2 Y X 2 X This result follows immediately from the above two theorems.
The sum of independent random variables equals the sum of the variances of these variables.

§ Functions of Random Variables:
- Let (X) and (Y) be random variables with a joint pdf fXY(x,y) and let g(x,y) be any continuous function that is defined for all (x,y). then: Z = g(x,y) is a random variable. The objective is to find fZ(z). - When (X) and (Y) are discrete random variables, we may obtain the probability mass function P{Z = z} by summing all probabilities for which g(x,y) equals the value of (z) considered, thus:

P(Z = z ) =

g(x, y) =z

P(X = x i , Y = y j )

- In the case of continuous random variables (X) and (Y) we find FZ(z) first: FZ(z) = P{Z £ z} = òò f XY ( x , y) dx dy
g(x, y) £ z

Then we find: fZ(z) =

d Fz (z ) dz
-54-

PROBABILITY DISTRIBUTIONS FOR MORE THAN ONE R.V.

CHAPTER IV

- Theorem: Let Z = X + Y and let (X) and (Y) be independent random variables, then;
¥

f Z (z ) =
Proof:

òf

X

( x ) f Y (z - x ) dx

FZ (z ) = P(Z £ z ) = P (X + Y £ z ) FZ (z ) = P(Y £ z - X) = òò f XY ( x , y) dx dy
R
¥ z -x

=

-¥ - ¥

ò òf

XY

( x , y) dx dy

since (X) and (Y) be independent random variables, then fXY(x,y) = fX(x) fY(y), so:
Fz (z ) = Fz (z ) =
¥ z -x

-¥ - ¥ ¥

ò òf

X

(x ) f Y ( y) dx dy

æ z -x ö ò¥ç -ò¥ f Y ( y) dy ÷ f X ( x) dx ç ÷ - è ø
¥ -¥

Fz (z ) =
fZ(z) =

òf

X

(x ) FY (z - x ) dx

d Fz (z ) dz
¥

f Z (z ) =

òf

X

(x) f Y (z - x) dx

The Convolution Integral

y
z

y=z-x
z

x

-55-

PROBABILITY DISTRIBUTIONS FOR MORE THAN ONE R.V.

CHAPTER IV

EXAMPLE (4-6):

Consider the joint pdf shown in the table (considered before in example 4-1). Let Z = X + Y. 1- Find the probability mass function of (Z), P{Z = z}. 2- Find P(X = Y). 3- Find E{XY} Y X 1 2 3
SOLUTION:

1 0.1 0.3 0

2 0 0 0.2

3 0.1 0.1 0

4 0 0.2 0

1- Possible values of (Z) and their probabilities are shown as follows: Z 2 3 4 5 6 7 P(Z = z) P(X = 1 , Y = 1) = 0.1 P(X = 1 , Y = 2) + P(X = 2 , Y = 1) = 0 + 0.3 = 0.3 P(X = 1 , Y = 3) + P(X = 3 , Y = 1) + P(X = 2 , Y = 2) = 0.1 + 0 + 0 = 0.1 P(X = 1 , Y = 4) + P(X = 2 , Y = 3) + P(X = 3 , Y = 2) = 0 + 0.1 + 0.2 = 0.3 P(X = 2 , Y = 4) + P(X = 3 , Y = 3) = 0.2 + 0 = 0.2 P(X = 3 , Y = 4) = 0

2- P(Y = X) = summation of probabilities over all values for which x = y. = P(X=1 , Y=1) + P(X=2 , Y=2) + P(X=3 , Y=3) = 0.1 + 0 + 0 = 0.1 3- E{XY} =

å åx y
i xi yj

j

P(X = x i , Y = y j )

= (1)(1) P(X=1 , Y=1) + (1)(3) P(X=1 , Y=3) + (2)(1) P(X=2 , Y=1) + (2)(3) P(X=2 , Y=3) + (2)(4) P(X=2 , Y=4) + (3)(2) P(X=3 , Y=2) = (1)(1) (0.1) + (1)(3) (0.3) + (2)(1) (0.3) + (2)(3) (0.1) + (2)(4) (0.2) + (3)(2) (0.2) E{XY} Exercise: Let Z = |X Y| - Find the pmf of Z: P(Z = z) =5

-56-

PROBABILITY DISTRIBUTIONS FOR MORE THAN ONE R.V.

CHAPTER IV

EXAMPLE (4-7):

Let (X) and (Y) be two identical and independent random variables, such that: fX(x) ì a e - ax , ìa e - ay , y > 0 x>0 ; f Y ( y) = í f X (x) = í î 0 , otherwise î 0 , otherwise Let Z = X + Y. Find fZ(z ).
SOLUTION:

x

fY(x)
z -a (z - x)

f Z (z ) =

¥

-¥ z

ò

f X ( x) f Y (z - x ) dx = ò a e -ax . a e
0 - az + a x

dx

x fY(-x) x
fY(z x)
z

f Z (z ) = ò a 2 e -ax . e
0

dx

f Z (z ) = a 2 e
2

-az

z

ò dx
0

f Z (z ) = a ze

-az

x

EXAMPLE (4-8):

Let (X) and (Y) be two identical and independent random variables, such that: ì1 ì1 , 0<x<2 , 0< y<2 ï ï ; f Y ( y) = í 2 f X ( x) = í 2 fX(x) ï0 , otherwise ï0 , otherwise î î ½ Let Z = X + Y. Find fZ(z ).
SOLUTION:

Z=X+Y è 0<z<4 - For (z < 0) è fZ(z ) = 0 - For (0 < z < 2)
¥

fY(x)

2

x

fY(-x)

2

x

f Z (z ) =

f Z (z ) = ò
2

-¥ z

òf

X

(x) f Y (z - x) dx
-2

x

z 1 1 ´ dx = 2 2 4 0

fY(z ½x)
-2 +z z

- For (z = 2)

x

f Z (z ) = ò
0

1 1 x 1 ´ dx = = 2 2 40 2

2

- For (2 < z < 4)

fZ(z )

f Z (z ) =

- 2 +z

ò

2

x 1 1 ´ dx = 2 2 4

2

=
- 2 +z

1 [(2 - (-2 + z )] = 1 [4 - z 4 4

]

½
0 2 4 z

è Total area =

1 1 ´ 4 ´ = 1 è It is a pdf 2 2

-57-

PROBABILITY DISTRIBUTIONS FOR MORE THAN ONE R.V.

CHAPTER IV

EXAMPLE (4-9):

Let (X) and (Y) be two uniformly distributed and independent random variables, such that: ì1 ì1 fX(x) , 0<x<2 , 0< y<4 ï ï ; f Y ( y) = í 4 f X ( x) = í 2 ï0 , otherwise ï0 , otherwise î î x Let Z = X + Y. Find fZ(z ). 2 fY(x)
SOLUTION:

- For (z < 0)

è fZ(z ) = 0

- For ( 0 < z < 2 ) è

1 1 1 f Z (z ) = ò ´ dx = z 2 4 8 0
2

z

fY(-x)

4

x

1 1 1 1 - For ( 2 <z < 4 ) è f Z (z ) = ò ´ dx = ´ 2 = 2 4 8 4 0
- For ( 4 <z < 6 )

-4

x
fY(z x)
-4 +z z

x

f Z (z ) =

-4 +z

ò

2

1 1 1 ´ dx = x 2 4 8

2

-4 +z

1 = (6 - z ) 8

fZ(z )

ì 1 0 <z < 2 ï 8z ï ï 1 f Z (z ) = í 2 <z < 4 4 ï ï1 ï 8 (6 - z ) 4 < z < 6 î

1/4
Area = 1

2

4

6

z

-58-

ELEMENTARY STATISTICS

CHAPTER V

C H A PT E R I V
ELEMENTARY STATISTICS

ELEMENTARY STATISTICS

CHAPTER V

With an experiment in which we observe some quantity (number of defectives, noise, power, ), there is associated a random variable X whose probability distribution is given by: FX ( x ) = P{X £ x} - In statistics, we take a random sample (x1, x2, , xn) of size (n) by performing that experiment (n) times. The purpose is to draw conclusions from properties of sample about properties of the distribution of the corresponding X (the population). We do this by calculating point estimators or confidence intervals or by performing a test of parameters or by a test for distribution functions. - For populations we define numbers called parameters ( m X and s 2 in normal distribution, (p) X in binomial distribution, for the exponential distribution) that characterize important properties of the distributions. The unknown parameter (q) is estimated by some appropriate function of the observations q = f ( x 1 , x 2 , ........, x n ) The function is called statistics or an estimator. A particular value of the estimator is called an estimate of q. - The sample mean m X is defined as : 1 n mX = å xi n i =1 - The sample variance s 2 is defined as: X 1 n s2 = å (x i - m X ) 2 X n - 1 i =1
s X = s 2 is called the standard deviation of the sample. X

- A computationally simpler expression for s 2 is: X
æ n ö nå x i - ç å x i ÷ i =1 è i =1 ø s2 = X n (n - 1)
n 2 2

§ Regression Techniques:
Suppose in a certain experiment we take measurements in pairs, i.e. (x1,y1) , (x2,y2), (xn,yn). We suspect that the data can be fit in a straight line of the form y = ax + b . Suppose that the line is to be fitted to the (n) points and let ( Î ) denote the sum of the squares of the vertical distances at the (n) points, then

Î = å [y i - (a x i + b)]
i =1

n

2

The method of least squares specifies the values of a and b so that Î is minimized. n ¶Î = -2 å ( y i - a x i - b) x i = 0 y (xi, axi + b) ¶a i =1 n ¶Î = -2 å ( y i - a x i - b) = 0 ¶b i =1
nb + a å x i = å y i
i =1 i =1 n n

(1)
n

(xi,yi )

bå x i + a å x i2 = å x i y i
i =1 i =1 i =1

n

n

(2)

x

-59-

ELEMENTARY STATISTICS

CHAPTER V

In matrix form, these equations are: æ n å x i ö æ ö = æ å yi ö ç ÷ç ÷ ç ÷ ç x å i å x i2 ÷ ç ÷ ç å x i y i ÷ è ø è è ø ø These two equations are called the normal equations. Solving the above two equations, we get:

n (n - 1) s b = m Y - am X
where C XY =

a=

nå x i y i - å x i å y i
i =1 i =1 i =1 2 X

n

n

n

=

C XY s2 X
nå x i y i - å x i å y i
i =1 i =1 i =1 n n n

1 n å (x i - m X ) (y i - m Y ) = n - 1 i =1
n

n ( n - 1)

1 1 å x i and m Y = n å y i n i =1 i =1 A useful formula for a may also be taken as: mX =
a=

n

åx y
i =1 i n i =1

n

i 2 i

- n mX mY -nm
2 X

åx

(Curve passes through m X and m Y .)

§ Fitting a Polynomial by the Method of Least Squares:
Suppose now that instead of simply fitting a straight line to (n) plotted points, we wish to fit a polynomial of the form: y = b1 + b 2 x + b 3 x 2 The method of least squares specifies the constants b 1 , b 2 and b 3 so that the sum of the squares of errors Î is minimized.
Î = å y i - (b 1 + b 2 x i + b 3 x 2 ) i
i =1 n

[

]

2

Taking partial derivatives of
n n i =1 i =1

Î with respect to b1 , b 2 and b 3 .
n

b 1 n + b 2 å x i + b 3 å x i2 = å y i
i =1

(1)
n

b1 å x i + b 2 å x 2 + b 3 å x 3 = å x i y i i i
2 4 b 1 å x 1 + b 2 å x 3 + b 3 å x 1 = å x i2 y i i i =1 i =1 i =1 i =1

n

n

n

(2) (3)

i =1 n

i =1 n

i =1 n

i =1 n

In matrix form, these equations are: æ n å x i å x i2 ö æ b1 ö æ å y i ö ç ÷ç ÷ ç ÷ ç å x i å x i2 å x 3 ÷ ç b 2 ÷ = ç å x i y i ÷ i ç ÷ 2 3 4 ÷ç 2 ÷ ç è å x i å x i å x i ø èb3 ø è å x i yi ø Then these equations can be solved simultaneously.

-60-

ELEMENTARY STATISTICS

CHAPTER V

Fitting an Exponential by the Method of Least Squares:
Suppose that we suspect the data to fit an exponential equation of the form: (1) y = a e bx Taking the natural logarithm: ln( y) = ln(a) + ln(e bx ) ln( y) = ln(a) + b x (2) Let Y / = ln(y) ;
/

= ln(a) ;

/

=b

So, equation (2) now becomes Y / = / + / x Which is the case of the straight line treated first. For each yi take its natural logarithm to get Yi/ . The new pairs of the data are (x1, lny1), (x2, lny2), (xn, lnyn), the solution of which is known.
EXAMPLE (5-1):

Suppose that the polynomial to be fitted to a set of (n) points is y = b x. It can be shown that:
b=

åx y
i =1 n i

n

i

åx
i =1

2 i

EXAMPLE (5-2):

Let y = a xb. Taking the ln of both sides, then: ln y = ln a + b ln x y' = b' + a' x' (Linear regression) where: y' = ln y , b' = ln a , a' = b , x' = ln x

EXAMPLE (5-3):

If y = 1 - e Manipulation of this equation yields: æ 1 ö ln ln ç ç 1 - y ÷ = - ln a + b ln x ÷ è ø which is the standard form: y' = b' + a' x' (Linear regression)

-x b a

EXAMPLE (5-4):

L 1 + e a + bx This form reduces to: æ L- yö ln ç ç y ÷=a+bx ÷ è ø which is in the standard form: y' = b' + a' x' (Linear regression)

If y =

-61-

ELEMENTARY STATISTICS

CHAPTER V

§ Model followed:
- Data are the measured values of the random variables obtained from replicates of a random experiment. - The random variables X1, X2, , Xn have the same distribution and are assumed independent. - Definition: Independent random variables X1, X2, distribution are called a random sample. - Definition: A statistic is a function of a random sample. - Definition: A probability distribution of a statistic is called its sampling distribution. - Theorem: If X1, X2, , Xn are independent random variables with E( x i ) = m i and Var(x i ) = s i2 And the random variable Y is: Y = C1X1 + C2X2 + + CnXn then, E ( Y ) = C1 m 1 + C 2 m 2 + ......... + C n m n 2 2 s 2 = C 1 s 1 + C 2 s 2 + ......... + C 2 s 2 Y 2 2 n n Furthermore, If X1, X2, , Xn are independent normal r.v, then (Y) is a normal r.v. , Xn with the same Size (n) Population (one distribution) N

Random Sample X1, X2, , Xn

§ Sampling Distribution of the Sample Mean

X

:

- Suppose that the random sample of size (n) is taken from a normal population with mean m X and variance s 2 . Then X1, X2, , Xn are independent and identically distributed (iid) normal X random variables. Therefore X + X 2 + ......... + X n mX = 1 n m + m X + ......... + m X n m X E (m X ) = X = = mX n n s 2 + s 2 + ......... + s 2 n s 2 s 2 X X s2 = X = 2X = X X 2 n n n s2 è m X is normal random variable with E(m X ) = m X and Var (m X ) = s 2 = X X n - If we are sampling from a population that has an unknown probability distribution, the sampling distribution of the sample mean will still be approximately normal with:
s2 X if the sample size (n) is large. n This is one of the most useful theorems in statistics. It is called the central limit theorem.

mean = E(m X ) = m X and variance = Var (m X ) = s 2 = X

-62-

ELEMENTARY STATISTICS

CHAPTER V

§ Central Limit Theorem:
- If X1, X2, , Xn is a random sample of size (n) taken from a population with mean m X and variance s , and if m X is the sample mean, then the limiting form of the distribution of: m - mX Z= X as n ® ¥, is the standard normal distribution. sX / n
2 X

- In many cases of practical interest, if n ³ 30 , the normal approximation will be satisfactory regardless of the shape of the population. If n < 30, the central limit theorem will work well if the distribution of the population is not severely non-normal. The theorem works well for small samples n = 4 , 5 when the population has a continuous distribution.
EXAMPLE (5-5):

An electronic company manufactures resistors that have a mean resistance of 100 and a standard deviation of 10 . Find the probability that a random sample of n = 25 resistors will have an average resistance less than 95 .
SOLUTION:

m X is approximately normal with:

mean = E (m X ) = 100
Var (m X ) = s 2 = X

.

s 2 10 2 X = n 25

sX =

s2 10 2 X = =2 n 25

P{ m X < 95} = P{Z <

95 - 100 } 2

95

100

mX

= F ( -2.5) = 0.00621

EXAMPLE (5-6):

The lifetime of a special type of battery is a random variable with mean 40 hours and standard deviation 20 hours. A battery is used until it fails, then it is immediately replaced by a new one. Assume we have 25 such batteries, the lifetime of which are independent, approximate the probability that at least 1100 hours of use can be obtained.
SOLUTION:

Let X1, X2,

, X25 be the lifetimes of the batteries. + X25 be the overall lifetime of the system

Let Y = X1 + X2 +

Since Xi are independent, then Y will be approximately normal with mean and variance given -63-

ELEMENTARY STATISTICS

CHAPTER V

by: E{Y} =
s 2 10 2 Var (m X ) = s = X = n 25
2 X

sX =

s2 10 2 X = =2 n 25

P{ m X < 95} = P{Z <

95 - 100 } 2
95 100

mX

= F ( -2.5) = 0.00621

§ Estimation of Parameters:
- The field of statistical inference consists of those method used to make decisions or to draw conclusions about a population. These methods utilize the information contained in a sample from a population in drawing conclusions. - Statistical inference may be divided into two major areas: Parameter estimation and hypotheses testing. - By parameters we mean quantities appearing in distributions such as: (p in the binomial, m X and s 2 in normal, rate l in a Poisson process). X - We consider two types of estimators, point estimators and interval estimators.

-64-

ELEMENTARY STATISTICS

CHAPTER V

- Point Estimate: A point estimate of some population parameter (q) is a single numerical value (q) . - Interval Estimate: An interval estimate is an interval (confidence interval) obtained from a sample. Unknown Parameter (q) Statistic ( ) Remarks Used to estimate the mean regardless of whether the variance is known or unknown. Used to estimate the variance when the mean is unknown. Used to estimate the variance when the mean is known. Used to estimate the probability of a success in a binomial distribution. n : sample size x : number of successes in the sample Used to estimate the difference in the means of two populations. Used to estimate the difference in the proportions of two populations.

mX
s2 X s2 X

mX =
s2 = X

1 n å xi n i =1

1 n å (x i - m X ) 2 n - 1 i =1 1 n å (x i - m X ) 2 n i =1
x P= n

s2 = X

p

m X1 - m X 2
p1 p2

m X1 - m X2 = å
i =1

n

n x 1i x - å 2i n 1 i =1 n 2

P1 - P2 =

x1 x 2 n1 n 2

§ Desirable Properties of Point Estimators:
1- An estimator should be close to the true value of the unknown parameter. - Definition: A point estimator (q) is unbiased estimator of (q) if E (q) = q . If the estimator is biased, then E (q) - q = B is called the bias of the estimator (q) . 2- Let q1 , q 2 , ...... be unbiased estimators of (q). A logical principle of estimation when selecting among several estimators is to chooses the one that has the minimum variance.

Distribution Distribution of q 1

of q 2 - Definition: If we consider all unbiased estimators of (q), the one with the smallest variance is called the minimum variance unbiased estimator (MVUE). When ( Var (q1 ) < Var (q 2 ) , q 1 is called more efficient than q 2 )

q

The variance Var (q) = E{[q - E(q)] 2 } is a measure of the imprecision of the estimator. 3- the mean square error of an estimator (q) of the parameter (q) is defined as:
MSE (q) = E (q - q) 2 This measure of goodness takes into account both the bias and imprecision.

-65-

ELEMENTARY STATISTICS

CHAPTER V

MSE (q) can also be expressed as:

MSE (q) = E{[q - E(q) + E(q) - q] 2 } = E{[(q - E(q)) + ((E(q) - q))] 2 }
=B

MSE(q) = E(q - E(q)) + 2 B E{(q - E(q)} + B = Var (q) + B 2
2 2 =0

MSE (q) = Var(q) + B

2

- Definition: An estimator whose variance and bias goes to zero as the number of observations go to infinity is called consistent.
EXAMPLE (5-6):

Show that the sample variance s 2 is an unbiased estimator of the variance s 2 X X (mean unknown).
SOLUTION:

ì 1 n ü 1 ìn ü E{s 2 } = E í å ( x i - m X ) 2 ý = n - 1 E íå (x i2 + m 2X - 2 m X x i )ý X î n - 1 i =1 þ î i =1 þ n n 1 ì æn öü E{s 2 } = E íå x i2 + n m 2 - 2 m X ç å x i ÷ ý X X n - 1 î i =1 è n i =1 ø þ E{s 2 } = X E{s 2 } = X 1 ìn 2 ü Eíå x i + n m 2 - 2 n m 2 ý X X n - 1 î i =1 þ

1 ìn 2 ü 1 ìn 2 2 ü E íå x i - n m 2 ý = íå E{x i } - nE{ m X }ý X n - 1 î i =1 þ n - 1 î i =1 þ From previous results we know: E{x i2 } = m 2 + s 2 X X s2 E{m 2 } = m 2 + X X X n s2 ü 1 ìn 2 2 2 X E{s 2 } = )ý íå E (m X + s X ) - n (m X + X n - 1 î i =1 n þ
(n - 1) s 2 1 2 2 2 2 X E{s } = n mX + n sX - n mX - sX = = s2 X n -1 n -1 E{s 2 } = s 2 X X
2 X

{

}

-66-

ESTIMATION THEORY AND APPLICATIONS

CHAPTER VI

C H A P T ER V
E S T IM A T IO N T H E O R Y
A N D A PP L I C A T IO N S

ESTIMATION THEORY AND APPLICATIONS

CHAPTER VI

Method for Obtaining Point Estimators: The Maximum Likelihood (ML) Estimator.
- Let us take (n) statistically independent samples {X1, X2, , Xn} from a population. Let the parameter of the population to be estimated (q). - The joint pdf of the samples is (expressed in terms of q). L(q) = f{x1, x2, , xn ; q} = f{x1 ; q} . f{x2 ; q} f{xn ; q} è due to independent of xi. L(q) is called the likelihood function. The maximum likelihood technique looks for that value (q) of the parameter that maximizes the joint pdf of the samples. - A necessary condition for the maximum likelihood estimator of (q) is: d d L(q) = 0 or equivalently ln {L (q)} = 0 dq dq (The ln(*) is a monotonically increasing function of the argument)
EXAMPLE (6-1):

Given a random sample of size (n) taken from a Gaussian population with parameters m X and

s 2 . Use the ML technique to find estimators for the cases: X a- The mean m X when the variance s 2 is assumed known. X 2 b- The variance s X when the mean m X is assumed known. c- The mean m X and variance s 2 when both are assumed unknown. X
SOLUTION:

L=Õ
i =1

n

1 2 p s2 X

e

- ( x i -m X ) 2 2s 2 X

=e

-

å
i =1

n

(xi -mX )2 2s2 X n 2 2 X

(2 p s )

(x i - m X ) 2 n è ln( L) = - å - ln 2 p s 2 X 2 2 2s X i =1
n

(

)

a- Set
n

d ln L(m) = 0 è treating s 2 as a constant. X dm X
i

å (x
i =1

- mX )2 = 0 è mX =

1 n å xi n i =1

(1)

Unbiased Estimator

Thus the ML estimator of the mean is the sample average mentioned earlier. d b- Set ln L(s 2 ) = 0 è treating m X as a constant X ds 2 X 1 n The result is s 2 = å (x i - m X ) 2 (2) Unbiased Estimator X n i =1 Note that: the division is by (n) since we are using the known mean of the distribution c- Set

d d ln L(m X , s 2 ) = 0 and ln L(m X , s 2 ) = 0 X X dm X ds 2 X 1 n 1 n This results in: m X = å x i and s 2 = å ( x i - m X ) 2 X n i =1 n i =1 (n - 1) s 2 X s 2 is a biased estimator since E{s 2 } = X X n 1 n For this general case, the unbiased estimator of s 2 is: s 2 = å (x i - m X ) 2 X X n - 1 i =1 Which is the sample variance introduced earlier.

-66-

ESTIMATION THEORY AND APPLICATIONS

CHAPTER VI

§ Finding Interval Estimators for the Mean and Variance: - An interval estimate of an unknown parameter of (q) is an interval of the form q 1 £ q £ q 2 where the end points q1 and q 2 depend on the numerical value of the parameter to be estimated (q) for a particular sample. From the sampling distribution of (q) we will be able to determine values of q 1 and q 2 such that: P (q 1 £ q £ q 2 ) = 1 - a , a>0 ; 0 < a <1 where: q is the unknown parameter (1 - a ) is the confidence coefficient a is called the confidence level. q1 and q 2 are the lower and the upper confidence limits I. Confidence Interval on the Mean: (Variance Known) - Suppose that the population of interest has a Gaussian distribution with unknown mean m X and known variance s 2 . X s2 1 n The sampling distribution of m X = å x i is Gaussian with mean m X and variance X . n i =1 n m - mX Therefore, the distribution of the statistic Z = X is a standard normal distribution. sX / n

{ P{ m

P -z a £ Z £z a
2

X

-z a sX
2

ì ü m - mX ï ï è P í- z a £ X £z a ý =1- a 2 2ï 2 ï sX n î þ n £ mX £ mX + z a sX n =1 - a a/2
2

}= 1 - a

}

a/2 -z a/2 z z a/2 z

Confidence Interval Error

m X -z a s X

- Definition:

2

n

mx

mx

m X +z a s X
2

n

If m X is the sample mean of a random sample of size (n) from a population with known variance s 2 , a 100(1 a)% confidence interval on m X is given by: X

mX -z a sX
2
2

n £ mX £ mX + z a sX
2

n

where z a is the upper 100(a/2)% point of the standard normal. § Choice of the Sample Size: The definition above means that in using m X to estimate m X , the error E = | m X m X | is less than or equal to z a s X n with confidence 100(1 a). In situations where the sample size
2

can be controlled, we can choose (n) so that we are 100(1 estimating m X is less than a specified error (E). (n) is chosen such that E = z a s X
2

a)% confident that the error in

æz a s X n è n =ç 2 ç E è

ö ÷ . ÷ ø

2

-67-

ESTIMATION THEORY AND APPLICATIONS

CHAPTER VI

EXAMPLE (6-2):

The following samples are drawn from a population that is known to be Gaussian. 7.31 10.80 11.27 11.91 5.51 8.00 9.03 14.42 10.24 10.91

Find the confidence limits for a 95% confidence level if the variance of the population is 4.
SOLUTION:

From the sample we have: n = 10 1 n m X = å x i = 9.94 n i =1 z a = 1.96

P mX - z a sX
2

{

2

n £ mX £ mX + z a sX
2

n =1- a

}

ì 1.96 ´ 4 1.96 ´ 4 ü Pí9.94 £ m X £ 9.94 + ý = 0.95 10 10 þ î P{8.70 £ m X £ 11.1796} = 0.95
II. Confidence Interval on the Mean: (Variance Unknown) - Suppose that the population of interest has a normal distribution with unknown mean m X and unknown variance s 2 . X - Definition: , Xn be a random sample for a normal distribution with unknown mean m X and unknown variance s 2 . The quantity X mX - m fT(t) has a (t-distribution) with (n 1) degree of freedom. T= sX / n Let X1, X2,
æ ( k + 1) ö Gç ÷ 1 The t-pfd is: f ( t ) = è 2 ø T k +1 ækö p k Gç ÷ æ t 2 ö 2 è2ø ç ç k + 1÷ ÷ è ø -¥<t<¥

a/2 ta/2

a/2 -t a/2 t

(k) is the number of degrees of freedom. The mean of the t-distribution is zero and the variance

k . k-2 The t-distribution is symmetrically and unimodal, the maximum is reached when the mean is 0 (quite similar to normal distribution. As k ® ¥ , the t-distribution is the normal distribution).

P - t a 2 , n -1 £ T £ t a 2 , n -1 = 1 - a

{

}

T=

mX - m sX / n

is the t-distribution with (n

1) degree of freedom 1) degree of freedom

t

a 2 , n -1

is the upper 100(a/2)% point of the t- distribution with (n

-68-

ESTIMATION THEORY AND APPLICATIONS

CHAPTER VI

ì ü m - mX ï ï P í- t a 2 , n -1 £ X £ t a 2 , n -1ý = 1 - a ï ï sX n î þ
s s ü ì P ím X - t a 2 , n -1 X £ m X £ m X + t a 2 , n -1 X ý = 1 - a n nþ î - Definition: If m X and s X are the mean and standard deviation of a random sample from a normal distribution with unknown variance s 2 , the 100(1 a)% confidence interval on m X is: X s s ü ì P ím X - t a 2 , n -1 X £ m X £ m X + t a 2 , n -1 X ý = 1 - a n nþ î where

t

a 2 , n -1

is the upper 100(a/2)% point of the t-distribution with (n 1) degrees of freedom.

EXAMPLE (6-3):

For the following samples drawn from a normal population: 7.31 10.80 11.27 11.91 5.51 8.00 9.03 14.42 10.24 10.91

Find 95% confidence interval for the mean if the variance of the population is unknown.
SOLUTION:

From the sample we have: 1 n m X = å x i = 9.94 n i =1

s2 = X

1 n å ( x i - m X ) 2 = 6.51 n - 1 i =1
1 = 10 1=9=

From tables of t-distribution: Number of degrees of freedom = n
a = 0.05 ® a 2 = 0.025 ® t a 2 , 9 = 2.263

s s ü ì P ím X - t a 2 , n -1 X £ m X £ m X + t a 2 , n -1 X ý = 1 - a n nþ î

ì 6.51 6.51 ü P í9.94 - 2.263 £ m X £ 9.94 + 2.263 ý = 0.95 10 10 þ î
P{8.11 £ m X £ 11.77} = 0.95

-69-

ESTIMATION THEORY AND APPLICATIONS

CHAPTER VI

III. Confidence Interval on the Variance of a Normal Population: (Mean Known) - When the population is normal, the sampling distribution of:
n æ x - mX ö n s2 1 n ÷ ; s 2 = å (x i - m X ) 2 c = 2X = å ç i X ç s ÷ n i =1 sX i =1 è X ø is chi-square with (n) degrees of freedom. The confidence interval is developed as: 2 2 P{c 1- a 2 , n £ c 2 £ c a 2 , n } = 1 - a 2 2

ì 2 ü n s2 P íc 1- a 2 , n -1 £ 2 X £ c 2 2 , n -1 ý = 1 - a a sX î þ 2 2 ü ìns ns ï ï Pí 2 X £ s 2 £ 2 X ý = 1 - a X c 1-a 2 , n ï ïca 2, n î þ

a/2 -c
2

1
1-a/2 , n

a -c
2
a/2 , n

a/2

- Definition: If s 2 is the sample variance from a random sample of (n) observations from a normal X distribution with a known mean and an unknown variance s 2 , then a 100(1 a)% confidence X interval on s 2 is: X n s2 n s2 X £ s2 £ 2 X X 2 ca 2,n c 1-a 2 , n
2 2 where c a 2 , n and c 1-a 2 , n is the upper and lower 100(a/2)% point of the chi-square distribution

with (n) degrees of freedom, respectively.
EXAMPLE (6-4):

For the following samples drawn from a normal population: 7.31 10.80 11.27 11.91 5.51 8.00 9.03 14.42 10.24 10.91

Find 95% confidence interval for estimation of the variance if the mean of the population is known to be 10.
SOLUTION:

From the sample we have: 1 n 1 n m X = å x i = 9.94 and s 2 = å (x i - m X ) 2 = 5.866 X n i =1 n i =1 2 From tables of -distribution: Number of degree of freedom = n = 10 =
a = 0.05 ® a 2 = 0.025
ì n s2 n s2 ï 2 X Pí 2 £ sX £ 2 X c 1- a 2 , n ïca 2,n î ü ï ý =1- a ï þ

è è

2 c 0.025 , 10 = 20.483 and c 2.975 , 10 = 3.247 0

10 ´ 5.866 ü ì10 ´ 5.866 Pí £ s2 £ ý = 0.95 X 3.247 þ î 20.483

P 2.863 £ s 2 £ 18.065 = 0.95 X

{

}

-70-

ESTIMATION THEORY AND APPLICATIONS

CHAPTER VI

IV. Confidence Interval on the Variance of a Normal Population: (Mean Unknown) - When the population is normal, the sampling distribution of:

æ x - mX ö 1 n ÷ ; s2 = c = = åç i å (x i - m X ) 2 X 2 ç s ÷ n - 1 i =1 sX i =1 è X ø is chi-square with (n 1) degrees of freedom.
2

(n - 1) s 2 X

n

2

- Definition: If s 2 is the sample variance from a random sample of (n) observations from a normal X distribution with an unknown mean and an unknown variance s 2 , then a 100(1 a)% X confidence interval on s 2 is: X (n - 1) s 2 (n - 1) s 2 X X £ s2 £ 2 X 2 c a 2 , n -1 c 1-a 2 , n -1
2 2 where c a 2 , n -1 and c 1- a 2 , n -1 is the upper and lower 100(a/2)% point of the chi-square

distribution with (n
EXAMPLE (6-5):

1) degrees of freedom, respectively.

For the following samples drawn from a normal population: 7.31 10.80 11.27 11.91 5.51 8.00 9.03 14.42 10.24 10.91

Find 95% confidence interval for estimation of the variance if the mean of the population is unknown.
SOLUTION:

From the sample we have:

1 n å x i = 9.94 n i =1 1 n s2 = å ( x i - m X ) 2 = 6.51 X n - 1 i =1 mX =
From tables of
2

-distribution: 1=9=

Number of degree of freedom = n = 10
a = 0.05 ® a 2 = 0.025
2 c 0.025 , 9 = 19.023 and c 2.975 , 9 = 2.7 0

ì ( n - 1) s 2 (n - 1) s 2 ü ï X X ï Pí 2 £ s2 £ 2 ý =1- a X c 1- a 2 , n -1 ï ï c a 2 , n -1 î þ 9 ´ 6.51ü ì 9 ´ 6.51 Pí £ s2 £ ý = 0.95 X 2.7 þ î 19.023

P 3.0799 £ s 2 £ 21.7 = 0.95 X

{

}

-71-

ESTIMATION THEORY AND APPLICATIONS

CHAPTER VI

V. Confidence Interval on a Binomial Proportion: - Suppose that a random sample of size (n) has been taken from a large population and that X ; (x £ n) observations in this sample belong to a class of interest. Then P = x n is a point estimator of the proportion of the population (p) that belongs to this class. Here (n) and (p) are the parameters of a binomial distribution. (X) is binomial with mean (np) and variance np(1 p). Therefore, n p (1 - p) p (1 - p) P = x n has a mean (p) and variance = n n2 - As was mentioned earlier (limiting case of the binomial distribution to the normal distribution) p (1 - p) the sampling distribution P is approximately normal with mean (p) and variance . n (p is not too close to 0 or 1 and (n) is large; {n p > 5} and {n p (1 p) > 5}. - To find a 100(1 a)% confidence interval on the binomial proportion using the normal approximation we construct the statistic: X - np P-p è P -z a £ Z £z a =1- a Z= = 2 2 np (1 - p ) p (1 - p ) n

{

}

ì ï ï P í- z ï ï î

a

2

£

ü ï P-p ï £z a ý =1- a 2 p (1 - p ) ï ï n þ

ì p (1 - p) p(1 - p) ü P íP - z a £ p £ P +z a ý =1- a 2 2 n n þ î The last equation expresses the upper and lower limits of the confidence interval in terms of the unknown parameter. p (1 - p) - The solution is to replace (p) by P in so that: n ì P(1 - P) P(1 - P) ü ï ï P íP - z a £ p £ P +z a ý =1- a 2 2 n n ï ï î þ
EXAMPLE (6-6):

In a random sample of 85 automobile engine crankshafts bearings, 10 have a surface finish that is rougher than the specifications allow. A 95% confidence interval for (p) is: x 10 and P= = = 0.12 z a = z 0.025 = 1.96 2 n 85
ì P (1 - P ) P (1 - P ) ü ï ï PíP - z a £ p £ P +z a ý =1- a 2 2 n n ï ï î þ ì 0 . 12 (1 - 0 . 12 ) 0 .12 (1 - 0 . 12 ) ü P í 0 . 12 - 1 . 96 £ p £ 0 . 12 + 1 .96 ý = 0 . 95 85 85 î þ

P {0 . 05 £ p £ 0 . 19 } = 0 . 95

-72-

ENGINEERING DECISION

CHAPTER VII

C HA P T E R V I
E N G I N E ER I N G D EC I S I O N

ENGINEERING DECISION

CHAPTER VII

§ Hypothesis Testing: In the last chapter we illustrated how a parameter can be estimated (points or interval estimation) from sample data. However, many problems require that we decide whether to accept or reject a statement about some parameter. The statement is called a hypothesis, and the decision-making procedure is called Hypothesis Testing. Two types of error are possible in such a decision process: 1- We decide that the null hypothesis H0 is false when it is really correct This is called a type I error and its probability is denoted by a a is called the significance level or size of the test. 2- We decide that the null hypothesis H0 is correct when it is really false This is called a type II error and its probability is denoted by b - Definition: The power of the test (1 b) is the probability of accepting the alternative hypothesis when the alternative hypothesis is true. § One-Sided and Two-Sided Hypothesis: A test of hypothesis such as: H0 : q = q0 H1 : q q0 Reject H0 Accept H1 q q0 Rejection Region Accept H0 q = q0 Acceptance region Reject H0 Accept H1 q q0 Rejection Region

Is called a two-sided test. H0 : is known as the null hypothesis.

q

H1 : is known as the alternative hypothesis. Tests such as: H0 : H1 : H0 : H1 : q = q0 q > q0 q = q0 q < q0 Accept H0 q = q0 Reject H0 Accept H1 q > q0

Reject H0 Accept H1 q > q0

Accept H0 q = q0

Are called one-sided tests. - a is called the significance level or size of the test. - The power of the test plotted against the true parameter value is called the Operating Characteristic (OC) curve.

-73-

ENGINEERING DECISION

CHAPTER VII

§ Hypothesis Testing on the Mean: Variance Known - Suppose that we wish to test the hypothesis: H0 : H1 : m = m0 m m0

Where m0 is a specified constant. We have a random sample X1, X2, , Xn from the 2 population (assumed normal). m is normal with mean m 0 and variance s n when H0 is assumed true. m - m0 - We use the test statistic: Z = ; Z is N(0 , 1) when H0 is assumed true. s n - If the level of significance is (a), then the probability is (1 a) that the test statistic (Z) falls between - z a and z a .
2 2

N (0 , 1)
Critical region a/2
2 2

Acceptance region a/2 za/2 z

- Reject H0 if z > -z a or z < z a
2 2

-z a/2

Fail to reject H0 if - z a < z < z a

- In the terms of m , we reject H0 if: m > m 0 + z a s
2

n or m < m 0 - z a s
2

n

- Suppose that the null hypothesis is false and that the true value of the mean is m = m 0 + d , when H1 is true, Z is normal with mean: f(z / H0) E (m / H 1 ) - m 0 E( Z / H 1 ) = N (0 , 1) s n E( Z / H 1 ) = m0 + d - m0 s n = d s n
Acceptance region

f(z / H1)

N ( d n s , 1)

æd n ö ÷ and unit variance. Z : Nç ç s , 1÷ è ø

b
-z a/2 za/2

z

d n s
2 2

The probability of type II error is the probability that (Z) will fall between - z a and z a . This probability is: æ æ ö d nö ÷ - Fç - z a - d n ÷ b = F çz a ç 2 ç 2 s ÷ s ÷ è ø è ø Now if we want to test Z= m - m0 s n H0 : H1 : m = m0 m > m0
Acceptance region

N (0 , 1)
Critical region z

, Z is N(0 , 1) when H0 is assumed true.

If (a) is the level of significance, then H0 is rejected if z > z a and accepted if z < z a

a
za

0

-74-

ENGINEERING DECISION

CHAPTER VII

If H1 is true, that is m = m 0 + d , d > 0 , then The type II error is the probability that z falls between - ¥ and z a . This probability is:

æ d nö ÷ b = F çz a ç s ÷ è ø

N (0 , 1)
za

N ( d n s , 1)
0
z

d n s

EXAMPLE (7-1):

Aircrew space systems are powered by a solid propellant. The burning rate of this propellant is an important product characteristic. Specifications require that the mean burning rate must be 50 cm/s. We know that the standard deviation of burning rate is 2 cm/s. The experimenter decided to specify a type I error probability of significance level of a = 0.05. He selects a random sample of n = 25 and obtains a sample average burning rate of m = 51.3 cm/s. What conclusions should be drawn?
SOLUTION:

Test H0 : m = 50 cm/s H1 : m 50 cm/s

, a = 0.05

Rejected H0 if z > 1.96 or z < -1.96 For m = 51.3 cm/s and s = 2 cm/s, then Z= m - m0 s n = 51.3 - 50 2 25 = 3.25

Since 3.25 > 1.96 we reject H0 and we have strong evidence that the mean burning rate exceeds 50 cm/s.

-75-

ENGINEERING DECISION

CHAPTER VII

SUMMARY FOR HYPOTHESIS TESTING PROCEDURE
NULL HYPOTHESIS H0 : m = m0 s 2 known TEST STATISTIC Z= m - m0 ALTERNATIVE HYPOTHESIS H1 : m ¹ m 0 CRITERIA FOR REJECTION Z >z a
2

s n N(0 , 1)
m - m0

H1 : m > m 0 H1 : m < m 0
H1 : m ¹ m 0

Z >z a

Z < -z a
t > t a 2 , n -1

H0 : m = m0 s 2 unknown

s/ n student t-distribution with (n 1) degrees of freedom

T=

H1 : m > m 0 H1 : m < m 0
H1 : s 2 ¹ s 2 0

t > t a 2 , n -1
t < - t a 2 , n -1
2 c 2 > c 2 2 , n -1 or c 2 < c 1- a 2 , n -1 a

H0 : s2 = s2 0 m unknown

(n - 1) s 2 c2 = 2 s0 Chi-square distributions with (n 1) degrees of freedom ns s2 0 Chi-square distributions with (n) degrees of freedom c2 =
2

H1 : s 2 > s 2 0
2 H1 : s 2 < s 0

c 2 > c 2 , n -1 a
2 c 2 < c 1-a , n -1 2 2 c 2 > c a 2 , n or c 2 < c 1- a 2 , n 2 c2 > ca,n 2 c 2 < c 1-a , n

H0 : s2 = s2 0 m known

H1 : s 2 ¹ s 2 0 H1 : s 2 > s 2 0
2 H1 : s 2 < s 0

H0 :

p = p0
Z=

X - np 0 np 0 (1 - p 0 )

=

P - p0 p 0 (1 - p 0 ) n

H1 : p ¹ p 0 H1 : p > p 0

Z >z a
Z >z a

2

N(0 , 1)

H1 : p < p 0

Z < -z a

-76-

ENGINEERING DECISION

CHAPTER VII

§ Decision Making for Two Samples: - The previous chapter presented hypothesis tests and confidence intervals for a single population parameter (the mean m , the variance s 2 , or the proportion p). Here we extend those results to the case of two independent populations.
2 - Population (1) has mean m1 and variance s 1 , population (2) has mean m 2 and variance s 2 . 2 Inferences will be based on two random samples of sizes (n1) and (n2). That is X11, X12, , X1n1 is a random sample of (n1) observations from population 1, and X21, X22, , X2n2 is a random sample of (n2) observations from population 2.

2 s1

s2 2

m1 § Inferences for a Difference in Means: Variances Known - Assumptions: 1- X11, X12, 2- X21, X22, , X1n is a random sample from population 1. , X2n is a random sample from population 2.

m2

3- The two populations presented by X1 and X2 are independent. 4- Both populations are normal, or if they are not normal, the conditions for the central limit theorem apply. - The test statistic Z =
m 1 - m 2 - (m 1 - m 2 )
2 s1 s 2 + 2 n1 n 2

has an N(0 , 1) distribution.

§ Testing hypothesis on (m 1 Null hypothesis: Test statistic: Z = H0 : m1

m 2): Variances Known m2 = D 0

m 1 - m 2 - (D 0 )
2 s1 s 2 + 2 n1 n 2

Alternative Hypothesis

Criteria for Rejection
Z > z a or Z < -z a
2 2

H1 : m1 - m 2 ¹ D 0 H1 : m1 - m 2 > D 0
H1 : m1 - m 2 < D 0

Z >z a

Z < -z a

-77-

ENGINEERING DECISION

CHAPTER VII

- Definition: Confidence Interval on the Difference in Two Means: Variances Known. If m 1 and m 2 are the means of independent random samples of sizes (n1) and (n2) with known 2 variances s 1 and s 2 , then a 100%(1 a) confidence interval for (m1 m2) is: 2
2 2 ì s1 s 2 s1 s 2 ü ï ï + 2 £ m1 - m 2 £ m1 - m 2 + z a + 2ý ím 1 - m 2 - z a 2 2 n1 n 2 n1 n 2 ï ï î þ where z a is the upper a 2 % point of standard normal distribution.

2

§ Inferences for a Difference in Means of Two Normal Distributions: Variances Unknown - Hypothesis tests for the difference in means: CASE I:
2 1

=

2 2

=

2

- The pooled estimator of
2 p

2

denoted by s 2 is defined as: p

2 (n 1 - 1) s 1 + (n 2 - 1) s 2 2 s = n1 + n 2 - 2

- The statistic T =

m 1 - m 2 - (m 1 - m 2 ) sP 1 1 + n1 n 2

has a t-distribution with (n1 + n2 2) degrees of freedom

when H0 is true. - The Two-Sample Pooled t-test: Null hypothesis: H0 : m1 Test Statistic:
T=

m2 = D 0
1 1 + n1 n 2

m 1 - m 2 - (D 0 ) sP

Alternative Hypotheses H1 : m1 - m 2 ¹ D 0
H1 : m1 - m 2 > D 0

Criteria for Rejection t > t a 2 , n1 + n 2 - 2 or t < t a 2 , n1 + n 2 - 2
t > t a , , n1 + n 2 - 2

H1 : m1 - m 2 < D 0

t < - t a , , n1 + n 2 - 2

- Definition: Confidence Interval on the Difference in Means of Two Normal Distributions: Variances Unknown and Equal.
2 If m1 , m 2 , S1 , and S 2 are the means and variances of two random samples of sizes (n1) and 2 (n2) respectively from two independent normal populations with unknown but equal variances, then a 100%(1 a) confidence interval on the difference in means (m1 m2) is:

ì 1 1 1 1 ü + £ m1 - m 2 £ m1 - m 2 + ta 2 , n +n -2 s P + ím 1 - m 2 - t a 2 , n1 + n 2 - 2 s P ý 1 2 n1 n 2 n1 n 2 þ î

-78-

ENGINEERING DECISION

CHAPTER VII

CASE II: If H0 : m1

2 1

¹

2 2

m2 = D0 is true, then the test Statistic T * =

m 1 - m 2 - (D 0 )
2 s1 s2 + 2 n1 n 2

Is distributed approximately as t with degrees of freedom given by:
2 æ s1 s2 ö ç + 2 ÷ çn ÷ è 1 n2 ø 2 1 2

=

(s

n1 s2 n + 2 2 n1 + 1 n2 + 1

)

2

(

)

2

-2

if H0 is true.

- Definition: Confidence Interval on the Difference in Means of Two Normal Distributions: Variances Unknown and Unequal.
2 If m1 , m 2 , S1 , and S 2 are the means and variances of two random samples of sizes (n1) and 2 (n2) respectively from two independent normal populations with unknown and unequal variances, then an approximate 100%(1 a) confidence interval on the difference in means (m 1 m 2) is:

ì ï í ï î

1 -

2

- t 2,n

2 S1 S2 + 2 £ n1 n 2

1 -

2

£

1 -

2 + t 2,n

2 S1 S2 ü ï + 2ý n1 n 2 ï þ

§ Inferences on the variances of two normal populations: Next, we introduce tests and confidence intervals for two population variances. Both populations are assumed normal. - Definition: Let X11, X12, , X1n1 be a random sample from a normal population with mean m1 and 2 variance s 1 , and let X21, X22, , X2n2 be a random sample from a second normal population with mean m 2 and variance s 2 . Assume that both normal populations are 2 2 independent. Let S1 and S 2 be the sample variances, then the ratio: 2
2 2 S1 / s1 F= 2 2 S2 / s 2 has an F distribution with (n1 degrees of freedom.

1) numerator degrees of freedom and (n2

1) denominator

- Hypothesis testing procedure: A hypothesis testing procedure for the equality of two variances is based on the following: 2 Null hypothesis: H0 : s 1 = s 2 2 Test Statistic: F0 =
2 S1 S2 2

-79-

ENGINEERING DECISION

CHAPTER VII

Alternative Hypotheses 2 H1 : s1 ¹ s 2 2
2 H1 : s1 > s 2 2 2 H 1 : s1 < s 2 2

Rejection Criterion f 0 > f a 2 , n1 -1, n 2 -1 or f 0 < f1- a 2 , n1 -1, n 2 -1
f 0 > f a , n1 -1, n 2 -1 f 0 < f1- a , n1 -1, n 2 -1

- Definition: Confidence Interval on the Ratio of Variances of Two Normal Distributions.
2 If S1 and S 2 are the sample variances of random samples of sizes (n1) and (n2) respectively 2 2 from two independent normal populations with unknown variances s 1 and s 2 , then a 2 2 s 100%(1 a) confidence interval on the ratio 1 is: s2 2
2 S1 s 2 S2 f 1- a 2 , n1 -1,n 2 -1 £ 1 £ 1 f a 2 , n1 -1,n 2 -1 S2 s 2 S2 2 2 2 where f a 2 , n1 -1,n 2 -1 and f 1- a 2 , n1 -1,n 2 -1

are the upper and lower a/2% points of the F distribution with (n2 1) numerator degrees of freedom and (n1 1) denominator degrees of freedom respectively. 1 - Remark: f1- a 2 , u ,n = f a 2 , u ,n § Inferences on Two Population Proportions:

fX(x)

a/2 1 a

a/2

f1-

2 , u,

f

2 , u,

Now we consider the case where there are two binomial parameters of interest p1 and p 2 and we wish to draw inferences about these proportions. - Large Sample Test for H0: p 1 = p2 Suppose that the two independent random samples of sizes (n1) and (n2) are taken from two populations, and let X1 and X2 represent the number of observations that belong to the class of interest in the samples. Furthermore, suppose that the normal approximation is applied to each population so that the estimators of the population proportions: X X P1 = 1 and P2 = 2 have approximate normal distributions. n1 n2 - Hypothesis testing procedure: Null hypothesis: H0 : p 1 = p 2 Test Statistic:
Z0 = P1 - P2 æ 1 1 ö P (1 - P )ç çn + n ÷ ÷ è 1 2 ø

P=

X1 + X 2 n1 + n 2

-80-

ENGINEERING DECISION

CHAPTER VII

Alternative Hypotheses

H 1 : p1 ¹ p 2 H 1 : p1 > p 2 H 1 : p1 < p 2

Rejection Criterion Z 0 > z or Z 0 < - z
2

2

Z0 > z a Z 0 < -z a

- Confidence Interval for p 1

p 2:

The confidence interval for p1 p 2 can be found from the statistic: P1 - P2 - ( p 1 - p 2 ) Z= p 1 (1 - p 1 ) p 2 (1 - p 2 ) + n1 n2 which is a standard normal r.v. The 100%(1 a) confidence interval on p 1 p 2 is:

P1 - P2 - z a

2

p 1 (1 - p 1 ) p 2 (1 - p 2 ) p 1 (1 - p 1 ) p 2 (1 - p 2 ) + £ p 1 - p 2 £ P1 - P2 + z a + 2 n1 n2 n1 n2

-81-