Ch 1

New Environments
• Wireless communication networks provide voice and data
communications to mobile users in severe interference
environments.
• The vast majority of media signals, voice, audio, images, and
video are processed digitally.
• Huge Web server farms deliver vast amounts of highly
specific information to users.
New Dimensions
• Web information is created and posted at an accelerating rate; future search
applications must become more discerning to extract the required response
from a vast ocean of information.
• Information-age scoundrels hijack computers and exploit these for illicit
purposes, so methods are needed to identify and contain these threats.
• Machine learning systems must move beyond browsing and purchasing
applications to real-time monitoring of health and the environment.
• Massively distributed systems in the form of peer-to-peer and grid
computing communities have emerged and changed the nature of media
delivery, gaming, and social interaction; yet we do not understand or know
how to control and manage such systems.
Model
• A model is an approximate representation of a physical situation. A
model attempts to explain observed behavior using a set of simple
and understandable rules. These rules can be used to predict the
outcome of experiments involving the given physical situation. A
useful model explains all relevant aspects of a given situation. Such
models can be used instead of experiments to answer questions
regarding the given situation. Models therefore allow the engineer to
avoid the costs of experimentation, namely, labor, equipment, and
time.
Mathematical Model
• Mathematical models are used when the observational phenomenon
has measurable properties. A mathematical model consists of a set of
assumptions about how a system or physical process works. These
assumptions are stated in the form of mathematical relations
involving the important parameters and variables of the system. The
conditions under which an experiment involving the system is carried
out determine the “givens” in the mathematical relations, and the
solution of these relations allows us to predict the measurements
that would be obtained if the experiment were performed.
Deterministic Models
• In deterministic models the conditions under which
an experiment is carried out determine the exact
outcome of the experiment. In deterministic
mathematical models, the solution of a set of
mathematical equations specifies the exact outcome
of the experiment. Circuit theory is an example of a
deterministic mathematical model.
Probability Models
• Many systems of interest involve phenomena that exhibit unpredictable variation and
randomness. We define a random experiment to be an experiment in which the
outcome varies in an unpredictable fashion when the experiment is repeated under the
same conditions.
• Deterministic models are not appropriate for random experiments since they predict
the same outcome for each repetition of an experiment.
• Probability models are intended for random experiments. As an example of a random
experiment, suppose a ball is selected from an urn containing three identical balls,
labeled 0, 1, and 2. The urn is first shaken to randomize the position of the balls, and a
ball is then selected. The number of the ball is noted, and the ball is then returned to
the urn. The outcome of this experiment is a number from the set We call the set S={0,
1, 2} of all possible outcomes the sample space. It is clear that the outcome of this
experiment cannot consistently be predicted correctly.
Statistical Regularity
• In order to be useful, a model must enable us to make predictions
about the future behavior of a system, and in order to be predictable,
a phenomenon must exhibit regularity in its behavior. Many
probability models in engineering are based on the fact that averages
obtained in long sequences of repetitions (trials) of random
experiments consistently yield approximately the same value. This
property is called statistical regularity.
• Suppose we alter the above urn experiment by placing in the urn a
fourth identical ball with the number 0. The probability of the
outcome 0 is now 2/4 since two of the four balls in the urn have the
number 0. The probabilities of the outcomes 1 and 2 would be
reduced to 1/4 each. This demonstrates a key property of probability
models, namely, the conditions under which a random experiment is
performed determine the probabilities of the outcomes of an
experiment.
Properties of Relative Frequency
The Axiomatic Approach to a Theory of
Probability
• Equation (1.2) suggests that we define the probability of an event by its long-
term relative frequency. There are problems with using this definition of
probability to develop a mathematical theory of probability. First of all, it is not
clear when and in what mathematical sense the limit in Eq. (1.2) exists. Second,
we can never perform an experiment an infinite number of times, so we can
never know the probabilities exactly. Finally, the use of relative frequency to
define probability would rule out the applicability of probability theory to
situations in which an experiment cannot be repeated. Thus it makes practical
sense to develop a mathematical theory of probability that is not tied to any
particular application or to any particular notion of what probability means. On
the other hand, we must insist that, when appropriate, the theory should allow
us to use our intuition and interpret probability as relative frequency.
Axioms
Building a Probability Model
• Let us consider how we proceed from a real-world problem that
involves randomness to a probability model for the problem. The
theory requires that we identify the elements in the above axioms.
This involves
• (1) defining the random experiment inherent in the application,
• (2) specifying the set S of all possible outcomes and the events of interest,
and
• (3) specifying a probability assignment from which the probabilities of all
events of interest can be computed. The challenge is to develop the simplest
model that explains all the relevant aspects of the real-world problem.
Communication over Unreliable Channels
Communication over Unreliable Channels
• Many communication systems operate in the following way. Every T
seconds, the transmitter accepts a binary input, namely, a 0 or a 1, and
transmits a corresponding signal. At the end of the T seconds, the receiver
makes a decision as to what the input was, based on the signal it has
received. Most communications systems are unreliable in the sense that the
decision of the receiver is not always the same as the transmitter input.
Figure models systems in which transmission errors occur at random with
probability Ɛ. As indicated in the figure, the output is not equal to the input
with probability Ɛ Thus is the long-term proportion of bits delivered in error
by the receiver. In situations where this error rate is not acceptable, error-
control techniques are introduced to reduce the error rate in the delivered
information.
Communication over Unreliable Channels
Compression of Signals
• For example, suppose we are interested in compressing a music signal
S(t). This involves representing the signal by a sequence of bits.
Compression techniques provide efficient representations by using
prediction, where the next value of the signal is predicted using past
encoded values. Only the error in the prediction needs to be encoded
so the number of bits can be reduced.
Reliability of Systems
• Reliability is a major concern in the design of modern systems. A
prime example is the system of computers and communication
networks that support the electronic transfer of funds between
banks. It is of critical importance that this system continues operating
even in the face of subsystem failures. The key question is, How does
one build reliable systems from unreliable components? Probability
models provide us with the tools to address this question in a
quantitative way. The operation of a system requires the operation of
some or all of its components.
Resource-Sharing Systems
Internet Scale Systems