Preface

Probability Theory and Random Processes is an introductory text book, to provide

detailed concepts of probability, random variables, and random processes for BE and B-Tech
Electronics Engineering, Electronics and Communication Engineering Electronics and
Telecommunication Engineering graduate students, who have a strong mathematical background,
and if some of the more theoretical sections are excluded by the instructor. The concepts, which
are covered in this text book, are very useful for the advanced courses like, Digital
Communications and Advanced digital Signal Processing etc. This book has several features:
One of the main objectives of this book is to provide the students a clear and logical
presentation of the basic concepts in a simple and lucid language.
Many examples are included to illuminate subtle points that are often overlooked or not
fully explained in basic textbooks. Each chapter also contains homework problems.
Provides an organized and broad coverage of the theory as well as of illustration of the
applications thereof.
Several appendices provide background material on related topics in mathematics and
signals and systems.
The book has eight chapters, starting with basic mathematical concepts needed for
understanding Probability theory, followed by fundamentals of Random Variables and Random
Processes and their applications.
The probability concepts through the set theory are introduced in Chapter1. Chapter 1
also provides in detailed about various probability concepts like Sample space, Events,
Conditional probability, Total probability and the Bayes theorem. A clear understanding of this
chapter helps the students

to understand the concepts of random variables and random

processes.
Random variables are dened in Chapter 2, which includes a description of many wellknown (and not so well-known) continuous and discrete parametric families of distributions and
density functions. In this chapter, the important properties and applications of distribution and
density functions are also discussed.
i

Chapter 3 starts with the definition of expectation of a random variable and is extended to
functions of random variables. Moments, characteristic function, Conditional expectation and
several of their important properties are also discussed with proofs. Several techniques for
deriving the distributions of transformations of single random variable are also investigated.
Chapter 4 extends the single random variable theory to several random variables while
Chapter 5 defines operations with several random variables.
By extending the characterization of multiple random variables to be indexed by time, we
introduce random processes in Chapter 6. Various properties of a random process are covered,
such as independence, stationarity and ergodcity. Specic well-known random processes such as
the Poisson and Gaussian processes are developed and investigated.
The power spectral density of a random process is dened in Chapter 7, which is used to
describe the behavior of signals when processed (ltered) by a system and also provides various
properties of PSD.
Chapter 8 focuses on linear time-invariant systems, though some nonlinear processing is
mentioned. This Chapter also provides basic concepts of noise and its measurements.
Finally, the book closes with a number of appendixes that contain material helpful to the
student in the working examples.

ii

This book is dedicated to my late mother

Smt. Laxmi Mergu and to my family Members

Table of Contents

Chapter-I:- Introduction to probability

1.0 Introduction

1.1 Introduction to Set Theory

1.2 Important Theorems on Sets

6
iii

1.3 Operations on Sets

1.4 Venn Diagrams

1.5 Basics of Probability

11

1.6 Probability of an Event

15

1.7 Mathematical Model of Experiments

20

1.8 Joint and Conditional Probability

21

1.9 Statistical Independence

25

1.10 Total Probability and Bayes Theorem

25

Example Problems

31

Exercise Problems

44

Chapter-II:- The Random Variable

48

2.0 Introduction

48

2.1 The Random Variable

48

2.2 Cumulative Distribution Function(CDF)

49

2.3 Probability Density Function(PDF)

56

2.4 Some Important Distribution and Density Functions

62

2.5 Conditional Distribution and Density Functions

79

Example Problems

81

Exercise Problems

93

Chapter-III:- Operations on One Random Variable

95

3.0 Introduction

95

3.1 Expectation and Variance

96

3.2 Moments

101

3.3 The Markov and Chebyshev Inequalities

106

3.4 Moment Generating Function and Characteristic Function

110

3.5 Transformations of a Continuous Random Variable

115

3.6 Transformations of a Discrete Random Variable

120

Example Problems

121

Exercise Problems

133
iv

Chapter-IV:- Multiple Random Variable

135

4.0 Introduction

135

4.1 Vector Random Variables

135

4.2 Joint Distribution Function

136

4.3 Joint Density Function

139

4.4 Conditional Distribution and Density Function

143

4.5 Independence of Two Random Variables

145

4.6 Distribution and density of a sum of two or more Random Variables

147

4.7 Central Limit Theorem

152

Example Problems

155

Exercise Problems

168

Chapter-V:- Operations on Multiple Random Variables

170

5.0 Introduction

170

5.1 Expectation and Variance

170

5.2 Joint Moments

171

5.3 Joint Moments Generating Functions

175

5.4 Transformations of Multiple Random Variables

183

Example Problems

189

Exercise Problems

199

Chapter-VI:- Random Processes-Temporal Characteristics

202

6.0 Introduction

202

6.1 Random Process

202

6.2 Stationarity and Independence

205

6.3 Correlation and Covariance Function

211

6.4 Time Averages

223

6.5 Ergodicity

228

6.6 Gaussian and Poisson Random Processes

233

Example Problems

240

Exercise Problems

250
v

Chapter-VII:- Random Processes-Spectral Characteristics

252

7.0 Introduction

252

7.1 Power Spectrum Density and Its Properties

252

7.2 Relationship between Autocorrelation Function and PSD

258

7.3 Bandwidth of Power Spectrum Density

262

7.4 Cross Power Spectral Density

263

7.5 Relationship between Cross-correlation Function and PSD

267

7.6 Power Spectrum for Discrete-Time Processes and Sequences

270

Example Problems

273

Exercise Problems

387

Chapter-VIII:- Linear Systems with Random Inputs

289

8.0 Introduction

289

8.1 Linear Systems with Deterministic Inputs

289

8.2 Linear Systems with Random Inputs

292

8.3 Spectral Characteristics of System Response

296

8.4 Types of Random Processes

298

8.5 Modeling of Noise Sources

300

Example Problems

310

Exercise Problems

316

vi