1 Intelligent Systems

1.1 INTRODUCTION
Advances in artificial intelligence (AI), soft computing, and related scientific fields
have brought new opportunities and challenges for researchers to deal with complex
and uncertain problems and systems, which could not be solved by traditional
methods. Many traditional approaches that have been developed for mathematically
well-defined problems with accurate models may lack in autonomy and decision-
making ability and hence cannot provide adequate solutions under uncertain and
fuzzy environments. Intelligent systems represent a new approach to addressing
those complex problems with uncertainties. Intelligent systems are defined with
such attributes as high degree of autonomy, reasoning under uncertainty, higher
performance in a goal seeking manner, high level of abstraction, data fusion from a
multitude of sensors, learning and adaptation in a heterogeneous environment, etc.
(Shoureshi and Wormley, 1990).
In the real world, we can find many problems and systems that are too complex
or uncertain to represent them in complete and accurate mathematical models. And
yet, we still have the need to design, optimize, or control the behavior of such
systems. As an example, humans can learn information through their cognitive
process, make decisions, and even adapt to the dynamic environment without using
mathematical models. The motivation for designing intelligent systems comes from the
efforts to mimic such a process without requiring precise mathematical models. There
are three major components for developing intelligent systems: knowledge acquisition
and representation, inferencing, and decision-making processes.
We can consider three possible types of knowledge or information representation
describing the input–output or cause–effect relationships of nonlinear systems. They
are quantitative analytical models, data, and heuristic rules as shown in Figure 1.1.
When none of these information types can provide the pertinent complete and
sufficient knowledge about the system behavior by itself alone, it will be necessary
to combine all three heterogeneous domains of information. Therefore, we will need
to consider different modeling strategies for capturing different types of information.
Neural networks (NNs) are suitable for representing the input–output relation-
ships of nonlinear systems. Due to its smoothing and global approximation capabil-
ities, if a sufficient amount of data is available, NNs can be trained to provide a
suitable mapping between input and output variables. On the other hand, heuristic
information can be captured by using fuzzy logic in the form of if-then rules. The
uncertainty or fuzziness associated with each linguistic description is handled by
using the so-called membership functions. By combining these fuzzy rules, we can
construct a fuzzy basis function network similar to NNs. This concept will be further
illustrated in Chapter 2.

ß 2008 by Taylor & Francis Group, LLC.

 Input Nonlinear Output
(cause) system (result)

Heuristic rules
d O d
I
o u o
n
m t m
p Empirical data
a p a
u
i u i
t
n t n
Analytical model

FIGURE 1.1 Three different knowledge representation schemes.

Examples of systems that would require and benefit from intelligent control and
optimization include electric power plants and their distributed networks, military
command and control systems, air traffic control systems, biological systems, and
manufacturing systems. The goal of this book is to cover essential elements of
knowledge and tools for developing intelligent systems and its application to com-
plex and uncertain problems and systems.
Activities in the development of intelligent systems are scattered over several
disciplines. In general, intelligent systems are based on expert systems, or soft
computing techniques such as fuzzy logic, NNs, and evolutionary strategies. Appli-
cations of expert system programming techniques to assist in a control design
process and in process monitoring and diagnostics are perhaps the most advanced
area in intelligent systems. There have been a number of success stories in applications
of expert systems including CRAFR, GEMS, GENAID, and TEXSYS (Shoureshi and
Wormley, 1990). However, current applications use rule-based algorithms that
are hand crafted, and typically lack the ability to add knowledge to their databases
in autonomous manner or adapt to the dynamic change of environment. On the
other hand, soft computing techniques provide potential for autonomous learning,
systematic reasoning, and adaptability, and hence offer a promise for the construction
of intelligent systems.
Since the mid-1980s, there has been a major explosion in the research related to
NNs, fuzzy logic, and evolutionary strategies. The most notable investigations are by
Werbos (1989) on back propagation, Widrow (1985) on formulation of Adaline, and
Narendra and Parthasarathy (1990) on NNs for system identification and control.
Another area that has potential for advancement of intelligent systems is fuzzy logic
and fuzzy control. In contrast to classical logical systems, fuzzy logic is aimed at a
formulation of models of reasoning that are approximate rather than exact. Fuzzy logic
provides tolerance for imprecision. This characteristic has attracted attentions of
researchers and engineers in various disciplines to design rule-based methodologies

ß 2008 by Taylor & Francis Group, LLC.

for uncertain systems (Zadeh, 1973; Kosko, 1992). Therefore, we briefly discuss the
basic concepts of these soft computing techniques. Evolutionary strategies provide
optimal point search techniques via evolution and adaptation and are useful for the
optimization of complex or ill-defined problems. The goal of this book is to cover
essential knowledge and methodologies for developing intelligent systems and its
application to design, optimization, and control of complex and uncertain problems
and systems.

1.2 INTRODUCTION OF SOFT COMPUTING TECHNIQUES

In recent years, various soft computing based techniques have emerged as useful
tools for solving various scientific and engineering problems that were not possible
or convenient to handle by traditional methods. The soft computing techniques
provide computationally efficient and yet effective means of modeling, analysis,
and decision making of complex phenomena. In Wikipedia, soft computing is
defined as ‘‘a collection of computational techniques in computer science, AI,
machine learning and some engineering disciplines, which attempt to study,
model, and analyze very complex phenomena: those for which more conventional
methods have not yielded low cost, analytic, and complete solutions.’’ The typical
techniques that belong to the soft computing arena include artificial neural networks
(ANNs), fuzzy sets and systems, evolutionary computation including evolutionary
strategies (ESs), swarm intelligence and harmony search, Bayesian network, chaos
theory, etc. Much of these soft computing techniques are inspired by biological
processes or are the results of attempts to emulate such processes.
Bäck (1996) provides further descriptions on soft computing:

Unlike hard computing schemes, which strive for exactness and full truth, soft com-
puting techniques exploit the given tolerance of imprecision, partial truth, and uncer-
tainty for a particular problem. In addition, soft computing offers computational
efficiency in inferencing compared with conventional techniques. Another common
contrast comes from the observation that inductive reasoning plays a larger role in soft
computing than in hard computing.

These attributes of computational efficiency and its ability to deal with imprecision
make soft computing very attractive techniques for dealing with complex, nonlinear,
and uncertain systems, which do not lend themselves to precise mathematical
expressions. In the next sections, the concepts of various commonly used soft
computing techniques are briefly described.

1.2.1 NEURAL NETWORKS

Artificial neural networks, often just called neural networks, are a generic method of
mapping or representing input and output relationships or patterns of nonlinear
functions or data through a single or multiple layers of an interconnected group of
artificial neurons (Figure 1.2). ANNs consist of processing nodes, called neurons,
which collect incoming signals, process them, and then generate output, and links

ß 2008 by Taylor & Francis Group, LLC.

 Hidden
Input
Output

FIGURE 1.2 Illustration of ANN.

providing connections among the layers of neurons. The neurons usually process the
incoming information via a certain activation function and generate output signal
when the strength of the incoming signals exceeds a specified threshold, which is
then passed to other neurons through connections. The neurons and connections
among the adjacent layers of neurons provide the ANNs computational ability to
approximate arbitrary–nonlinear relationships.
Artificial neural networks can be grouped into unsupervised and supervised
networks depending on their training methods. Unsupervised networks refer to
those NNs that do not require output data during learning or training such as adaptive
resonance networks. Supervised networks such as multilayer feedforward NNs
however require both input and output data during training of the networks. ANNs
can also be grouped into static and dynamic networks depending on their structures.
Static networks provide mapping between input and output data without any time-
dependent connections between layers or neurons in the network structure. On the
other hand, the dynamic networks have embedded feedback connections with delays
and hence can provide a more compact structure for representing the dynamic
relationships. For example, popularly used backpropagation NNs or radial basis
function NNs belong to the former, while recurrent NNs belong to the latter.

1.2.2 FUZZY LOGIC

Uncertainties or complexity of many physical systems or processes often lead to a
difficulty in developing accurate analytical models. Nonetheless, humans can still
describe the behavior of such systems through their cognitive processes, at least
qualitatively. An expert might be able to control a process based on his knowledge
and observation of the process even without any mathematical model. Fuzzy set
theory is ‘‘a body of concepts and techniques that give a form of mathematical
precision to human cognitive processes that are in many ways imprecise and ambiguous
by the standards of classical mathematics’’ (Kaufmann and Gupta, 1988). In effect, this
theory allows one to deal with fuzziness by grouping elements, which do not have clear
boundaries, into different classes. Fuzzy logic uses fuzzy set membership functions,
whose value range from 0 to 1, and allows for capturing linguistic representations of

ß 2008 by Taylor & Francis Group, LLC.

knowledge. The impreciseness of knowledge is handled by the membership functions
associated with each linguistic variable. In a narrow sense, fuzzy logic is a logical
system, which is a generalization of multivalued logic (Zadeh, 2007). Due to this
reason, the fuzzy theory has been applied to various engineering problems that are too
complex or ill-defined for the conventional mathematical analysis.
In a fuzzy system, knowledge is represented by if-then rules associated with
fuzzy variables. These rules along with the membership functions are processed
through the so-called compositional rule of inference. Unlike other reasoning pro-
cesses where qualitative reasoning is used with pure linguistic rules, fuzzy inference
logic involves numerical synthesis based on membership functions to form a fuzzy
decision table. Since the quantities synthesized in a fuzzy reasoning procedure are
generally fuzzy, the final decision is also fuzzy. Fuzzy inferencing is, therefore, usually
called approximate reasoning; that is, it matches process quantities with the rules in the
rule base to perform fuzzy inferencing by using the compositional rule of inference.

1.2.3 EVOLUTIONARY ALGORITHMS

Evolutionary algorithms (EAs) are computational algorithms that are inspired by the
nature’s evolutionary process. They emulate the principles of natural selection,
which favor the stronger species and guide further evolution such that they survive
in their environmental conditions. The primary focus of EAs is the search of global
optimal point. EAs or evolutionary computations are therefore suitable for optimiza-
tion problems with applications including, but not limited to, design optimization,
optimal parameter learning of ANNs or fuzzy basis function networks, or optimal
design of control parameters.
Evolutionary algorithms rely on the so-called subsymbolic, that is, numerical,
representation of knowledge. Contrary to the early days of AI that emphasized
the symbolic representations using predicate logic, semantic nets, or frames, EAs
emulate evolutionary processes in computational forms. Typical EAs include ESs,
evolutionary programming (EP), genetic algorithm (GA), and learning classifier
systems. Among them, the genetic algorithms are the most popular EAs and have
been actively used in various application areas. The three algorithms of ES, EP, and
GA are similar in nature since all of them are developed from the same idea of
applying the principle of organic evolution to optimization problems. However, they
are quite different in implementation procedures such as in the representation scheme
and self-adaptation abilities. For example, ESs use real vectors as coding represen-
tation, and primarily mutation and selection as search operators, while GAs use, at
least initially, binary string representations and EP finite state machines. Please refer
to Bäck (1996) for an extensive comparison of these three algorithms.
Despite the differences in specific approaches, EAs in general share some
common basic features (Menon, 2004):

. Population of individuals
. Notion of fitness
. Notion of population dynamics biased by fitness
. Notion of inheritance of properties from parent to child

ß 2008 by Taylor & Francis Group, LLC.

In the 1990s, interactions among the three different communities began to take place,
and consequently many attempts have been made in each community to supplement
its algorithm by adopting desirable components from other algorithms. Introduction
of floating variables instead of binary strings to GA is one of those examples
(Michalewicz, 1996). The adoption of uniform distribution for the mutation of
integer variables in ES seems to have borrowed its idea from GA (Bäck and Schtüz,
1995). The boundaries among the three algorithms are fading away and researchers
are adopting representation schemes and operators from various EAs, where they are
appropriate to their application in order to obtain the best possible results.
Contemporary derivatives of evolutionary computation include those such as
genetic programming, ES using a population of m parents and also recombination as
an additional operator (called (m=rþ, l)-ES), and compact GA, among many.
While not covered in this section, a notable evolutionary technique for optimiza-
tion also includes swarm intelligence such as ant colony optimization and particle
swarm optimization, which are known to work quite effectively for high-dimensional
problems.

1.3 SUMMARY
As described in this chapter, intelligent systems can be designed by various soft
computing techniques. Although it would be possible to adopt other techniques such
as expert systems and symbolic AI, the focus of this book is to construct various
intelligent system approaches based on soft computing techniques with applications
to design optimization, modeling, and control of complex systems and processes. In
the remainder of the book, readers will find the combined use of these soft computing
techniques for constructing intelligent systems approaches.

REFERENCES
Bäck, T., Evolutionary Algorithms in Theory and Practice, Oxford University Press,
New York, 1996.
Bäck, T. and Schtüz, M., Evolution strategies for mixed-integer optimization of optical
multilayer systems, Evolutionary Programming IV: Proceedings of the 4th Annual
Conference on Evolutionary Programming, pp. 33–51, 1995.
Kaufmann, A. and Gupta, M.M., Fuzzy Mathematical Models in Engineering and Manage-
ment Science, Elsevier Science, New York, 1988.
Kosko, B., Neural Networks and Fuzzy Systems—A Dynamical Systems Approach to Machine
Intelligence, Prentice Hall, Englewood Cliffs, NJ, 1992.
Menon, A., Frontiers of Evolutionary Computation, Kluwer Academic Publishers,
Boston=Dordrecht=London, 2004.
Michalewicz, Z., Genetic Algorithms þ Data Structures ¼ Evolution Programs, 3rd ed.,
Springer-Verlag, Reading, United Kingdom, New York, 1996.
Narendra, K. and Parthasarathy, K., Identification and control of dynamical systems using
neural networks, IEEE Transactions on Neural Networks, 1(1), 4–27, March 1990.
Shoureshi, R. and Wormley, D., Intelligent control systems, Final Report of NSF=EPRI
Workshop, October, 1990.
Werbos, P., Backpropagation and neural control: A review and prospectus, Proceedings of
International Joint Conference on Neural Networks (IJCNN), New York, June 1989.

ß 2008 by Taylor & Francis Group, LLC.

Widrow, B. and Steams, S.D., Adaptive Signal Processing, Prentice-Hall, Englewood Cliffs,
NJ, 1985.
Wikipedia, the free encyclopedia on web, http:==www.wikipedia.org=.
Zadeh, L.A., Outline of new approach to the analysis of complex structures and decision
processes, IEEE Transaction on Systems, Man and Cybernetics, SMC-3, 28–44, 1973.
Zadeh, L.A., From fuzzy logic to extended fuzzy logic—The concept of f-validity and the
impossibility principle, FUZZ-IEEE 2007, Imperial College, London, United Kingdom,
July 23, 2007.

ß 2008 by Taylor & Francis Group, LLC.

ß 2008 by Taylor & Francis Group, LLC.