4.1.

1 Probability Density Function (PDF)

To determine the distribution of a discrete random variable we can either provide its PMF or
CDF. For continuous random variables, the CDF is well-defined so we can provide the CDF.
However, the PMF does not work for continuous random variables, because for a continuous
random variable P(X=x)=0P(X=x)=0 for all x∈ℝx∈R. Instead, we can usually define
the probability density function (PDF). The PDF is the density of probability rather than
the probability mass. The concept is very similar to mass density in physics: its unit is
probability per unit length. To get a feeling for PDF, consider a continuous random
variable XX and define the function fX(x)fX(x) as follows (wherever the limit exists):
fX(x)=limΔ→0+P(x<X≤x+Δ)Δ.fX(x)=limΔ→0+P(x<X≤x+Δ)Δ.
The function fX(x)fX(x) gives us the probability density at point xx. It is the limit of the
probability of the interval (x,x+Δ](x,x+Δ] divided by the length of the interval as the length of
the interval goes to 00. Remember that
P(x<X≤x+Δ)=FX(x+Δ)−FX(x).P(x<X≤x+Δ)=FX(x+Δ)−FX(x).
So, we conclude that
fX(x)=limΔ→0FX(x+Δ)−FX(x)ΔfX(x)=limΔ→0FX(x+Δ)−FX(x)Δ
=dFX(x)dx=F′X(x),if FX(x) is differentiable at x.=dFX(x)dx=FX′(x),if FX(x) is differentiable at x.

Thus, we have the following definition for the PDF of continuous random variables:
Definition
Consider a continuous random variable XX with an absolutely continuous CDF FX(x)FX(x).
The function fX(x)fX(x) defined by
fX(x)=dFX(x)dx=F′X(x),if FX(x) is differentiable at xfX(x)=dFX(x)dx=FX′(x),if FX(x) is
differentiable at x
is called the probability density function (PDF) of XX.

Let us find the PDF of the uniform random variable XX discussed in Example 4.1. This
random variable is said to have Uniform(a,b)Uniform(a,b) distribution. The CDF of XX is
given in Equation 4.1. By taking the derivative, we obtain
fX(x)={1b−a0a<x<bx<a or x>bfX(x)={1b−aa<x<b0x<a or x>b
Note that the CDF is not differentiable at points aa and bb. Nevertheless, as we will discuss
later on, this is not important. Figure 4.2 shows the PDF of XX. As we see, the value of the
PDF is constant in the interval from aa to bb. That is why we say XX is uniformly distributed
over [a,b][a,b].
Fig.4.2 - PDF for a continuous random variable uniformly distributed over [a,b][a,b].
The uniform distribution is the simplest continuous random variable you can imagine. For
other types of continuous random variables the PDF is non-uniform. Note that for small
values of δδ we can write
P(x<X≤x+δ)≈fX(x)δ.P(x<X≤x+δ)≈fX(x)δ.
Thus, if fX(x1)>fX(x2)fX(x1)>fX(x2), we can
say P(x1<X≤x1+δ)>P(x2<X≤x2+δ)P(x1<X≤x1+δ)>P(x2<X≤x2+δ), i.e., the value of XX is more
likely to be around x1x1 than x2x2. This is another way of interpreting the PDF.

Since the PDF is the derivative of the CDF, the CDF can be obtained from PDF by
integration (assuming absolute continuity):

FX(x)=∫x−∞fX(u)du.FX(x)=∫−∞xfX(u)du.
Also, we have
P(a<X≤b)=FX(b)−FX(a)=∫bafX(u)du.P(a<X≤b)=FX(b)−FX(a)=∫abfX(u)du.
In particular, if we integrate over the entire real line, we must get 11, i.e.,
∫∞−∞fX(u)du=1.∫−∞∞fX(u)du=1.
That is, the area under the PDF curve must be equal to one. We can see that this holds for the
uniform distribution since the area under the curve in Figure 4.2 is one. Note that fX(x)fX(x) is
density of probability, so it must be larger than or equal to zero, but it can be larger than 11.
Let us summarize the properties of the PDF.
Consider a continuous random variable XX with PDF fX(x)fX(x). We have
 1. fX(x)≥0fX(x)≥0 for all x∈ℝx∈R.
2. ∫∞−∞fX(u)du=1∫−∞∞fX(u)du=1.
3. P(a<X≤b)=FX(b)−FX(a)=∫bafX(u)duP(a<X≤b)=FX(b)−FX(a)=∫abfX(u)du.
4. More generally, for a set AA, P(X∈A)=∫AfX(u)duP(X∈A)=∫AfX(u)du.

In the last item above, the set AA must satisfy some mild conditions which are almost always
satisfied in practice. An example of set AA could be a union of some disjoint intervals. For
example, if you want to find P(X∈[0,1]∪[3,4])P(X∈[0,1]∪[3,4]), you can write
P(X∈[0,1]∪[3,4])=∫10fX(u)du+∫43fX(u)du.P(X∈[0,1]∪[3,4])=∫01fX(u)du+∫34fX(u)du.
Let us look at an example to practice the above concepts.