Statistics and

Probability Theory
Random Variables and Their Properties
 Distributions of Random Variables
• Suppose we have a sample space “S”. The function that assigns a
value between 0 and 1 to any event inside “S” is called a random
function and the events in S are called random variables or
Stochastic Variables
• If the random variable takes on only discrete values, then it is called
Discrete Random Variable “DRV”. Example: Number of Columns
complying with shear-ductility requirements, Number of Failure
Cases.
• If the random variable takes on continuous values, then it is called
Continuous Random Variable “CRV”. Example: The crushing force
for a specimens of concrete cylinders

2
Discrete Random Variables
•  If X is a discrete random variables, then the function P(X = x) = f(x)
is called a probability density function “PDF” if:
• 1.0 ≥ f(x) ≥ 0
• for all X = x.
Example: A fair coin is tossed twice, this process is repeated many
times, Let X represent the possible values “x” for having Heads
Here, the possible sample space is S = { HH, HT, TH, TT}, for each
one, there is a value for each possible x as shown in the Table
P(X=0) = P(TT) = ¼, Sample Event HH HT TH TT

P(X=1) = P(HT U TH) = P(HT)+P(TH) = ¼+ ¼ X== x½ 2 1 1 0

P(X=2) =P(HH) = ¼ f(X=x) ¼ ½ ¼

3
Cumulative Distribution of DRV
• The function that gives the probability of having X ≤ x , i.e F(x)=P(X ≤ x)
is called a Cumulative Distribution Function “CDF”
• Simple: This function F will give the probability of having an event equals
x or less within all the possible values of X
• The function “F” must have certain properties:
• It must be ever increasing as X is increased. ( if y>x, then (Fy) > F(x) )
• F(x) must go to 1 if x goes to infinity , and F(x) goes to 0 as x goes to - infinity.
• In our previous example, we had X = number of heads per event in S.
• f(0) = 0.25, f(1) = 0.5, f(2) = 0.25. Then we have F as:
• F(x ≤ 0)=f(0) = 0.25
• F(x ≤ 1) = f(0)+f(1) = 0.75
• F(x ≤ 2) = f(0) + f(1) + f(2) = 1.0
• F(X<3) == F(X<2) = 1
4
Continuous Random Variables “CRV”
•  Most data have continuous nature. Weights, dimensions, masses,
capacities, strengths, forces…etc
• A probability distribution function “PDF” for a random variable generally
describes the “theoretical” relative frequency of having a certain value.
• The PDF has the following properties:
•

• = F(b) – F(a) , F(x) = , F = (01)

• One can easily see that the probability of having X = a is virtually zero
for a any continuous variable X.
• The cumulative distribution function is
 The Expected Value
•  The expected value , or the expectation of a random value { E(X) } is a
very important concept in all probability and statistics theory. It is defined as:
E(X) = “For a DRV”
“For a CRV”
• Note that if probabilities of all events are equal (uniform probability), then
f(x) = 1/n, then the expectation is simply the mean or average value of all X
• In our previous example, the expected value for X is
E(X) = 0*f(0) + 1*f(1)+2* f(2) =0*0.25+1*0.5+2*0.25 = 1
• The expected value is a generalization to the Arithmetic Mean , and it is
thus a measure of central tendency.

6
 The Variance and Standard Deviation
•  The variance for a random variable can defined as
{ DRV }
= E(X2) – { E(X) }2 { CRV }
• If the probabilities are all equal for all events of X then this definition
reduces to the known formula :

• The standard deviation is simply the square root of the variance

Coefficient of Variation = s / mean

7
Properties of the Expectation and the Variance
•   “c” and “b” are constants, then: E(c X + b) = cE(X) +b
If
• If X and Y are any random variables, then E(X+Y) = E(X)+E(Y)
• If X and Y are independent random variables, then E(XY) = E(X)E(Y)

or = +

• “c” and “b” are a constants, then: Var(cX + b ) = c2 Var(X)

• X, Y are random variables, then Var(X±Y) = Var(X)+Var(Y) or = +
• E{g(x)} = {DRV} , or E{g(x)} = {CRV}

8
 Joint Probability Function
• Most of times, experiments or data concern many random variables at the same
time, and these may or maynot be independent of each other, e.g. dimensions of
specimen, damage energy and peak frequency of an EQ.
• The probability density function that describes the simultaneous probability of
each variable is called a Joint Probability Density {or Mass} Function

  .0 {CRV}

  {CRV}

• If  X and Y are independent, then . The converse is true.

 Marginal and Conditional Probability
•  If we have a joint PDF for two variables X, Y, then we define the
marginal probability density function for a variable X as the sum of all
possible probabilities for the Y variable at a specific value of x.
• For Discrete Variables:
• For Continuous Variables:
• Recall that the conditional probability , we can use this as a basis for
defining the conditional probability density function of x given y as:

• Of course as usual, the probabilities are simply the Areas (for CRV) or
the Sums (for DRV) resulting from the density functions
The Expected Value and Variance, DRV
The Expected Value and Variance, CRV
 Example:
•   joint probability density function of two discrete random variables X and Y is given by
The
f(x, y)=c(2x+y), where x and y can assume all integers such that 0≤ x ≤ 2, 0 ≤ y ≤ 3, and f (x, y)= 0
otherwise:
1. Find the value of constant c. 2. Find P(X=2,Y=1), 3.Find the P(1 ≤ X, Y ≤ 2), 4. Find the Marginal
Probability Function for X. , 5. Find the mean value of X

1. The constant c must satisfy the basic property of PDF, i.e.: , thus doing the sum, we have:

2. P(X=2,Y=1) = (1/42)*(2*2+1)=5/42
3. P(1 ≤ X, Y ≤ 2)= (1/42)*
4. P(X=x)=
Function of One Random Variable
•  Most of the times, we use a function of parameters that are random,
e.g.: The pressure “p” behind a retaining wall is a function of soil density
“γ” and height of soil “h” and lateral pressure coefficient Ka . i.e. p = γ h
Ka
• Let’s say we have an invertible function W = G(x) , and let’s have the
function G(x) as monotonic
• Then logically, the probability of having a range of X is equivalent to
have a range of W. i.e. P(W=w) = P(X=x) = P(G –1(w))
• The probabilities are computed using the cumulative function as
For DRV :
For CRV:
, Hence:
Function of Two Random Variables
• 
• If W is a function of two random variables {X and Y}.

• Assume we know the Joint PDF for X and Y : i.e. fXY is given.

• We can identify the PDF for 4 cases of W:

1. W = X + Y  , note: w – y is in fact x.

2. W = X - Y 

3. W = X / Y 

4. W = XY 
 Example
• 
• The basic wind pressure is related to the wind velocity as p = cV 2 where
c is a constant. The PDF for the velocity V is fv = (a/v)*exp(-b/v), where a
and b are constant parameters. Find the PDF of the wind pressure p.

Here , and thus , and hence , by taking the positive derivative (because
the PDF must be positive)

E(p) = integral{-inf,+inf}( p*fp)

 Covariance

• Covariance is a measure of how linear the relationship is between the

random variables.
• If the covariance is negative, the relationship is probably inversely linear
between X and Y, if positive, then it is likely monotonically linear.
• If the relationship between the random variables is nonlinear, the covariance
might not be sensitive to the relationship.
• Notice that if X and Y are independent, then E(XY)=E(X)E(Y), thus, σXY = 0.
Examples on
Covariance
The Correlation Coefficient

• The correlation is some kind of a “normalized” covariance.

• Correlation also measures how “linear” the relationship is between
X and Y.
• Again, X and Y are independent if their correlation coefficient is
zero.
 Mean and Variance of a
Combination of Random Variables
•  Some times a random variable can be constructed using linear
combination of other random variables.
• For instance Take: Y = a1X1+a2X2+… where ai are constants, and Xi
are random variables.
• For such cases, due to linearity, the expected value and variance for the
variable Y is simply:
E(Y) = a1E(X1)+a2E(X2)+…
)
• If X1, X2,.. etc. are independent, then
Mean and Variance of a Nonlinear Function
of Random Variables
•  If the relation between the random variables is nonlinear, we can only
find approximations to the probabilities of their relation.
• If X1, X2 are independent and W = X1* X2 , then:
E(W) = E(X1)*E(X2)

• If W = G(, ,…, ), then ( as an approximation)

E(W) = G(, E(,…, )

• If Xi are not correlated then

Example:
• Let
  the random variables X1 and X2 denote the length and width,
respectively, of a beam. Assume E(X1)=40 centimeters with standard
deviation 2 centimeter and E(X2)=60 centimeters with standard deviation 4
centimeter. The covariance between X1and X2 is (– 0.3) cm. If both
variables are represented by a normal distribution, Determine the 95th
percentile of the perimeter of the beam.

The perimeter is Y = 2X1+2X2 and it is a random variable, which is also

normally distributed but with its own mean and variance. We have
E(Y) = 2*40+2*60 = 200 cm
Var(Y) = 4*(2)2+4*(4)2+2*2*2*(-0.3) = 77.6  cm
P95 = 200 + 1.64*8.8 = 214.4 cm