STUDENT NAME: PATRICE SIKENA MUBITA.

STUDENT ID #: UB23404SEL31719.

MAJOR:

ELECTRICAL ENGINEERING

(SCHOOL OF SCIENCE AND ENGINEERING)

COURSE:

ARTIFICIAL INTELLIGENCE AND NEURAL NETWORKS

ASSIGNMENT TITLE:

MACHINE LEARNING & A.I (PART 1 OF 3)

ATLANTIC INTERNATIONAL UNIVERSITY

HONOLULU, HAWAII

15TH SEPTEMBER 2014.

 INTRODUCTION TO AI

From archaic times in memorial, man has always had a general tendency to re-
engineer his environment, tools and resources, with the sole aim of easing up his life. However,
the last 70 years has seen a drastic shift from the mere muscle power resolving component of
the science and engineering (i.e. levers, machines or robotics), to include brain power or
autonomous systemization. That is, we have experienced a paradigm shift from mere
mechanical efficiency concerns to include those of computational efficiency. After the coming
in of computers that essentially depended on continual updates, operators and even
programmers, the dream of autonomous super computers suddenly took to flight. Alongside
this dream, computer-controlled machines were created. These computers run on sets of
instructions embedded in them by programmers, since they weren't and still aren't fully
autonomous to runaffairs without human interventions. And in later years, following the
realization of thecomputer age, it became theoretically evident that integrating such super
computers withmachines would bring the machines to literal life (i.e. creating cyborgs and
other fullyautonomous robotic systems) intelligent embodiments that according to popular
belief,
would eventually enslave the human race (if not obliterate it). This is absurdity one may
conclude: but then, weren t there similar opinions before we took flight into the sky and then
into space? Only time will tell.
Questions such as who am I? Where am I? , etc. clearly exhibit intelligence; and would
somewhat suggest consciousness if the embodiment of intelligence were completely
autonomous (e.g. Humanity). Other equally autonomous and conscious beings, though less
intellectual, include dogs, ants, rats and monkeys among other. Intelligence that is equivalent
to or greater than that of a human is referred to as strong intelligence, whilst that which is less
is referred to as weak intelligence. Unlike the latter intelligence which can only solve simple
problems, the former is capable of solving complex problems.

The gestation period that characterized the focus on artificial intelligence (AI) as an
emerging way to find solutions began sometime in 1943 during the Second World War and
might have been probably inspired by the war itself. The military wings or warring men sought
cunning ways of eluding the enemy by using intelligent methods, animals and artifacts. In the
summer of 1956, John McCarthy first coined the phrase Artificial Intelligence to describe this
new science frontier of cunning arts, which is now commonly referred to by the letters AI. AI is
generally relevant to any performance involving intelligence (i.e. intelligent tasks), and is thus
perceived as a universal field - so I am compelled to think of AI as the science of general
intelligence. Nevertheless, the fields that typically constitute AI include economics, linguistics,
philosophy, mathematics, computers, psychology, control theory & cybernetics, and
neuroscience.

Figure 1 The AI network of related disciplines.

Basic questions the disciplines ask that make them useful to AI include;

1. Linguistics How does language relate to thought? (components and influences include;
generative grammar, computational linguistics and a book titled syntactic structure ,
written by Noam Chosky in 1950 his models were formal enough to program.)
2. Economics How should we make decisions in order to have optimized results?
(Decision theory, utility theory, Game theory- inspired by The theory of Games and
Economic behavior written by John von Neumann and Oskar Morgensken in 1944, and
operations research.)
3. Mathematics What are the formal rules for conclusive deductions? What can be
calculated? How do we resolve or reason with uncertain data? (Formal logic e.g.
Boolean algebra and incompleteness theorem, Algorithms, and probability theorem.)
4. Control theory & Cybernetics How can artifacts operate under their own control (i.e.
self-autonomy)? (Stable feedback (homeostatic) systems includes water clocks,
thermostats and steam engines, and cybernetics basically the science of
communications and automated control systems occurring in living things and
machines.)
5. Philosophy How can the mind arise from a physical brain? How can we draw valid
conclusions from formal rules? Where does knowledge come from and how can we use
 it to effect an action? (Empiricism induction, Logical positivism knowledge, Goal-
based analysis and Utilitarianism.)
6. Neuroscience How does the brain process information? (Basically the study of the
brain neurons.)
7. Psychology How do living things such as humans and animals act? (Behaviorism,
cognitive psychology and cognitive science.)
8. Computer Engineering (or computer science) How can we build a more efficient
computer? (Automatic calculus, Theory of artificial automata.) The first forms of
omputers were used to decipher communication data (language messages) among
warring factions during the Second World War.

AI is the art of creating machines that perform functions that require intelligence when
performed by people (Kurzweil, 1990).

Even though artificial intelligence cannot be defined with astute precision due to the
inability to categorically define what is meant by intelligence, all eight diverse disciplines that
comprise AI, indicate what AI is all about. Thus by knowing what is involved, we gain a broader
understanding of what AI is.

There is no standard definition of exactly what artificial intelligence is. If you ask five
computing professionals to define "AI", you are likely to get five different answers
(Munakata 2008).

The concise Oxford English dictionary (11th edition), defines intelligence as the ability
to acquire and apply knowledge and skills . The key phrase is ability to acquire and apply; so
that all the sub-abilities leading to this over-all objective of the catch-phrase are abilities
indicative of intelligence. But then what are these attributes or indications of intelligent
behavior that qualify an artifact to be considered as being intelligent? Attributes such as
reasoning, perception, communication, learning, interaction with a complex environment
(including other agents) are indicative of intelligent behavior whilst automated reasoning,
knowledge representation, natural language processing, machine learning, perception,
robotics, computer vision, etc. are all components of artificial intelligence (AI), drawn from the
various fields of AI. The various definitions of AI ultimately give four broad performance
perspectives to which AI generally inclines, based on the cognitive approach, as shown in the
table below;
THINKING LIKE A HUMAN: THINKING RATIONALLY:
"The exciting new effort to make computers "The study of mental faculties through the
think . . . machines with minds, in the full use of computational models"
and literal sense" (Haugeland, 1985) (Charniak and McDermott, 1985)

"[The automation of] activities that we
associate with human thinking, activities such
as decision-making, problem solving, learning
..."(Bellman, 1978)
ACTING LIKE A HUMAN:
"The art of creating machines that perform
functions that require intelligence when
performed by people" (Kurzweil, 1990)
"The study of the computations that make
it possible to perceive, reason, and act"
(Winston, 1992)
ACTING RATIONALLY:
"A field of study that seeks to explain and
emulate intelligent behavior in terms of
computational processes" (Schalkoff, 1 990)

"The study of how to make computers do "The branch of computer science that is
things at which, at the moment, people are concerned with the automation of intelligent
better" (Rich and Knight, 1 99 1 ) behavior" (Luger and Stubblefield, 1993)

Figure 2 the four perspective of AI

The four perspectives of AI are generally categorized into two main streams of
thought; the human approach and the rationalist approach.

Thinking like a human:

If we are going to say that a given program thinks like a human, we must have some
way of determining how humans think. We need to get inside the actual workings of human
minds. There are two ways to do this: through introspection trying to catch our own thoughts
as they go by or through psychological experiments. Once we have a sufficiently precise
theory of the mind, it becomes possible to express the theory as a computer program. If the
program's input/output and timing behavior matches human behavior, that is evidence that
some of the program's mechanisms may also be operating in humans. For example, Newell and
Simon, who developed GPS, the "General Problem Solver" (Newell and Simon, 1961), were not
content to have their program correctly solve problems. They were more concerned with
comparing the trace of its reasoning steps to traces of human subjects solving the same
problems. This is in contrast to other researchers of the same time (such as Wang (I960)), who
were concerned with getting the right answers regardless of how humans might do it. The
interdisciplinary field of cognitive science brings together computer models from AI and
experimental techniques from psychology to try to construct precise and testable theories of
the workings of the human mind. Although cognitive science is a fascinating field in itself, we
are not going to be discussing it all that much in this book. We will occasionally comment on
similarities or differences between AI techniques and human cognition. Real cognitive science,
however, is necessarily based on experimental investigation of actual humans or animals, and
we assume that the reader only has access to a computer for experimentation. We will simply
note that AI and cognitive science continue to fertilize each other, especially in the areas of
vision, natural language, and learning.

Acting like a human The Turing Test approach

Designed to provide a satisfactory operational definition of intelligence, the Turing
Test was proposed by Alan Turing in 1950. Turing defined intelligent behavior as the ability to
achieve human-level performance in all cognitive tasks, sufficient to fool an interrogator.
Roughly speaking, the test he proposed was that if a computer to be interrogated by a human
via a teletype, the computer would only pass the test of human-level performance ,if and only
if, the interrogator would not be able to tell whether there is a computer or a human at the
other end. Such a computer prototype would need to possess natural language processing
capabilities, and programming it would not be an easy task. Since physical simulation of a
person is not necessary for intelligence, Turing s test deliberately avoids direct physical
interaction between an interrogator and the computer.
Within AI, it is important to note that there has not been a big effort to try to pass the
Turing test. Acting like a human is an issue that comes up primarily when AI programs have to
interact with people (e.g. an expert system explaining its diagnosis or a natural language
processing system has a dialogue with a user). These programs must behave according to
certain normal conventions or laws of human interaction in order to make themselves
understood. The underlying representation and reasoning in such a system may or may not be
based on a human model.

Thinking rationally: The laws of thought approach

Aristotle (the Greek philosopher) was one of the first to attempt to codify "right
thinking," that is, irrefutable reasoning processes. It is said that his famous syllogisms provided
patterns for argument structures that always gave correct conclusions drawn from correct
premises. For example, "Socrates is a man; all men are mortal; therefore Socrates is mortal."
These laws of thought were supposed to govern the operation of the mind, and thus initiated
the field of logic. In the late nineteenth and early twentieth centuries, the development of
formal logic culminated into the provision of a set notation for statements about all kinds of
things in the world and the relations between them. Programs existed by 1965, that could,
given enough time and memory, take a description of a problem in logical notation and also
find the solution to the problem, in the event that one exists. The logicist perspective within
artificial intelligence hopes to build on programs such as this to create intelligent systems.
There are two main obstacles to this approach. First, there is a very big difference
between being able to solve a problem "in principle" and doing so in practice. Note that even
problems with just a few dozen facts can exhaust the computational resources of any computer
unless it has some guidance as to which reasoning steps to try first. Although both of these
obstacles apply to any attempt to build computational reasoning systems, they appeared first in
the logicist tradition because the power of the representation and reasoning systems are well-
defined and fairly well understood. Secondly, it is not easy to take informal knowledge and
state it in the formal terms required by logical notation, particularly when the knowledge is less
than 100% certain.

Acting rationally
In this approach, AI is viewed as the study and construction of rational agents (An agent
is just something that perceives and acts). When one acts so as to achieve one's goals, given
one's beliefs, the individual is said to be acting rationally. In the "laws of thought" approach to
AI, the whole emphasis was on the aspect of correct inferences. Being a rational agent entails
making correct inferences one way to act rationally is to reason logically to the conclusion that
a given action will achieve one's goals and then to act on that same conclusion. On the other
hand, correct inference is not all of rationality; because there are often situations where there
is no provably correct thing to do, yet something must still be done. Note also that there are
also ways of acting rationally that cannot be reasonably said to involve inference. For example,
pulling one's hand off of a hot stove is a reflex action that is more successful than a slower
action taken after careful deliberation. The cognitive skill set used for the Turing Test is
there to allow rational actions. Thus, we need the ability to represent knowledge and reason
with it because this enables us to reach or make good decisions in diverse of situations.
We need learning because having a better idea of how the world works actually enables
us to generate more effective strategies for dealing with it. We need visual perception in order
to get a better idea of what an action might achieve for example, being able to see a tasty
morsel helps one to move toward it.
The study of AI as rational agent design therefore has two advantages;
1. First, it is more general than the "laws of thought" approach, because correct inference
is only a useful mechanism for achieving rationality, and not a necessary one.
2. Secondly, it is more amenable to scientific

INTRODUCTION TO MACHINE LEARNING

Machine learning is the process by which a machine uses a sample training set to
learn and then to generalize the data that it receives based on experience. It must be noted that
such machines are not mere levers, but those integrated with logic circuits such as computers
designed to make decisions for the machines. Machine learning involves the adaption to new
circumstances and the detection and extrapolation of patterns. It is all about directly getting
computers to be smart and learn things by themselves so that human beings do not have to
reprogram them. Let us take handwriting analysis as an example. Machine learning would
involve the development of a computer algorithm to recognize and interpret a person's
handwriting based on a particular sample set. Although this can be done with relative ease in
the human brain, this form of artificial intelligence is very difficult to program in computers. In
order to fully understand and appreciate what is meant by machine learning, we need to first
understand what it means to learn. And by so doing, we will also better understand the
different methods in which an agent can learn and thus effectively improve its overall
performance.

HIERARCHICAL PERSPECTIVE AND FOUNDATIONS

The subject of machine learning is in a sense, as old as Artificial intelligence itself. In

about 1943, Warren McCulloch and Walter Pitts proposed a model of the neuron in the human
and animal brain (Forsyth & Naylor 1986). These abstract nerve cells actually provided the basis
for a formal calculus of brain and neural activity. Other workers, notably Norbert Wiener,
observed and elaborated these and similar ideas into the field that culminated into Cybernetics;
and it was from Cybernetics that AI emerged as a scientific discipline in the 1950s, as already
alluded to in the introduction of AI.

The study of Cybernetics is concerned with the mathematical properties of feed-back systems
and treats the human being as an automation, whereas AI is concerned with the cognitive
processes brought into play by the human being in order to perform what we regard as
intelligent tasks. (Bonnet, 1985)

During this period, the research interest was in building general purpose learning systems that
start with little or no initial structure or task-oriented knowledge.
According to Michalski, Carbonell & Mitchell (1984), Rosenblatt's Perceptron was an
elementary visual system which could be taught to recognize a limited class of patterns. It
consisted of a finite grid of light-sensitive cells. Experience with the perceptron spawned a new
discipline of pattern recognition and led to the development of the decision-theory approach to
the machine learning process. This approach equates learning with the acquisition of linear,
polynomial, or other related forms of discriminate functions from a given set of training
examples. One of the best known successful learning systems utilized such techniques was
Samuel's checkers program, which was able to acquire through learning a master level of
performance.
In the 1960s, AI was characterized by a Heuristic search. AI workers and researchers
abandoned their attempt to build artificial brains from the ground up. Instead they perceived
human thinking as a complex coordination of essentially simple symbol-manipulating tasks, and
this was research mainly coming from the work of psychologists and early AI researchers on
models of human learning. This paradigm utilized logic or graph structure representations
rather than numerical or statistical methods. The systems learned symbolic descriptions
representing higher level knowledge and made strong structural assumptions about the
concepts that were to be acquired (Michalski, Carbonell & Mitchell 1984). Work in this
paradigm include research on human concept acquisition various applied pattern recognition
systems. The most influential workers during this time were Allen Newell and Herbert Simon of
Carnegie-Mellon University. They worked on theorem-proving and computer chess, among
other things. Their masterwork was a program known the General Problem Solver (GPS).
The central idea behind GPS was that problem solving was a search through a space of probable
potential solutions. To make the search efficient, it had to be guided by heuristic rules that
directed it towards the desired destination. Thus, an automaton wandering around a maze
would have to use an exhaustive search technique if it knew nothing about the structure of that
maze; but if it had some way of telling when it was getting 'warm' it could normally reach its
goal state sooner. Heuristics are not guaranteed to work, and may occasionally lead it down a
blind alley; thus were not much good at real-life problems

Knowledge-base systems characterized AI research in the 1970s. Researchers

broadened their interest beyond learning isolated concepts from examples, and have begun
investigating a wide range of learning methods, mostly based upon knowledge-rich systems. A
team led by Edward Feigenbaum at Stanford University began to remedy the GPS defect of the
1960s. They focused on very narrow areas of expertise, rather than trying to computerize
general intelligence. And thus lead to the birth of the expert system. The first expert system
was DENDRAL, a mass-spectrogram interpreter built as early as 1967; the most influential has
proved to be MYCIN, from 1974 onwards (and was strongly influenced by DENDRAL). the
inventor of MYCIN, Shortliffe, who was a doctor, also devised a scheme based on certainty
evidence. The significant point, however, is that MYCIN and systems like it can arrive at correct
conclusions even with incomplete and partly incorrect information. The PROSPECTOR, one of
MYCIN'S successors, geological exploration system, has been widely quoted as helping to
discover a vast unknown molybdenum deposit in the Washington state of the USA.
By the 1980s, Machine learning took its recognized form, and during this time, expert
systems became a trend their key ingredient was knowledge. The scope and quality of an
expert system's knowledge base determines its success. The task of codifying a human expert's
skill can be a long and labor-intensive process. AI then moved on to concentrate on the
problem of machine learning - which is one way of synthesizing knowledge automatically.
Another expert program later developed was the EURISKO, a discovery program which
extended and improved its own body of heuristic rules automatically, by induction.
EURISKO is thought to be the first computer program holding a patent, though most of the
credit rightly belongs to its author, Doug Lenat. And since AI itself can be viewed as a leading
branch of computer science, it is the place to look for a peek into the future of computing.
Ironically enough, by concentrating once again on learning, AI has returned to its roots (learning
was seen as the key problem in the early cybernetic days).
Figure 3 History of Machine Learning

Objectives of Machine Learning:

Research in the subject of machine in artificial intelligence is currently organised

into three main research areas of foci;

1. Task-oriented studies This focus of machine learning is also referred to as the

Engineering approach. It involves the development and analysis of learning systems
designed to improve performance in a predetermined set of tasks.
2. Cognitive Simulation This focus involves the investigation and simulation of human
learning processes.
3. Theoretical Analysis The focus on theoretical exploration of the space of possible
learning methods and algorithms independent of the application domain.
 LEARNING

Learning is a multi-faceted phenomenon that implies thinking and purpose. An agent

is said to learn if it does so intentionally. That is why we wouldn t say that a vine has learned to
grow around a trellis in a vineyard we d say it has been trained. Learning without purpose is
regarded merely training. In learning the purpose is the learner s, whereas in training it is the
teacher s. Thus to decide whether something has actually learned, you need to see whether it
intended to and whether there was any purpose involved. *That makes the concept moot
(subject to debate), when applied to machines because it is unclear whether artifacts can
behave purposefully.

Learning Agents:
In artificial intelligence s machine learning, a learning agent is an intelligent agent, and is
defined as an entity capable of perception and action.

Figure 4 A learning agent.

AGENT SENSORS ACTUATORS

Human Agent Eyes, Ears, Tongue, etc. Legs, Hands, Mouth, etc.
Robot Agent Cameras, IR finders, etc. Motors
Software Agent File content, Key stroke, etc. Screen display, Disc write, File
write, etc.

Figure 5 Examples of Agents.

Learning Model of Conscious Level:
There are six major elements that constitute the conscious learning model; the
environment, the learning element, the knowledge base, the effectors, the receptors, and the
problem solver.

(i) The environment This is the input to the learning system in which the learner
currently finds himself/itself, and it is referred to as the problem domain in machine
learning.

(ii) The learning element This is the l learning pattern recognizer, and it is an interface
between the problem solver and the knowledge base. The manner that learning
element consults the knowledge base is known as the learning skills.
Many of the techniques adapted in Artificial Intelligence society are to simulate
those learning skills, e.g., rote learning, learning by advice taking, etc.. This will be
discussed later. There are a number of inference techniques applied by human
beings, i.e. induction, deduction, abduction and creation.
Induction:
This is the principle of reasoning to a conclusion about all members of a
classification resulting from the thorough examination of a sample of the class;
broadly, reasoning from the particular to the general (i.e. the inference of a general
law from particular instances).

Deduction:
This is a process of reasoning in which a conclusion is made from the stated
premises; reasoning from the general to specific (i.e. the inference of particular
instances from reference to a general law).

Abduction:
This is a form of deductive logic which provides only a 'plausible inference'.
Using statistics and probability theory, abduction may yield the most probable
inference among many possible likely or inference.

(iii) The knowledge base:

This harbors all stored knowledge, and it captures the expertise of problem
solving. According to Michalski (1984), there are several different ways of
representing knowledge include parameters in algebraic expressions, formal
grammars, production rules, decision trees, formal logic-based expressions, and
related formalisms, graphs and networks, frames and schemes, computer programs
and other procedural encodings, taxonomies, and multiple representations.
 (iv) The effectors:
The system output producing structures, whose output depend on the nature
of solutions generated from the problem solver. The interaction between system
response and the environment will generate new stimuli. Hence the learning cycle or
process.

(v) The receptors:

The receptors will decode received stimuli and transmit the resultant signal to
the problem solver.

(vi) The problem solver:

This integrates all signals collected from the sensors into a pattern decodable
by the learning element, which in turn finds the matching solution from the
knowledge base. If it is found, the solution is returned to the problem solver which
in turn generates a suitable output pattern (of the solution found). The solution is
then finally sent to effectors for producing system response.

Figure 6 conscious level model.

The combined three components comprising, the learning element, the problem
solver, and the knowledge base constitute the Central Nervous System (CNS).
Why conscious level?
This model is classified as conscious level owing to the usual application of symbolic
knowledge in the problem solving procedure. For example, let us imagine that we are to answer
a mathematics examination. We will in the first instance perceive the question with our eyes
(i.e. stimulus received by the receptors, which code into system signals, and then sends signals
to problem solver, then secondly, we will try to search our memory for the solutions (i.e.
problem solving). The receptors in this case are our eyes (in problem solving, the problem
solver sends the request to the learning element, ordering it to read or find the solution for the
question). If the learning element succeeds in finding the answer, feedback the problem solver
which in turn orders the eyes (receptors) and hands (effectors) to read and to write it down on
the answering paper simultaneously. The answer that has been written down will then become
a new stimulus for the system to validate. However, if the learning element fails to find the
solution for the question, the learning element may request the problem solver to go on trying
the next question (depends on an applying situation), or even ask for more input (i.e.
information).

ROT LEARNING

This is essentially learning by memorization or programming. No inference or

transformation of the knowledge is required on the learner s or agent s part. It is simply data
caching or storage in the raw format.
• Learning by memorization
The facts and data (information) is simply memorized with no inference made nor
any transformation made to the data i.e. learning by habitual repetition. An example
of this kind of learning is that performed by primitive database systems.
• Learning by being programmed
This kind of learning makes use of the intervention of an external entity, requiring
no effort from the part of the learner i.e. learning by implanting information. An
example is a software engineer programming a computer.

LEARNING BY ADVICE

Sometimes referred to as learning by direct instruction, learning by advice is simply

learning by being taught or rather acquiring knowledge directly from a teacher (or other
organized source). This new knowledge is integrated with already acquired information to
transform the knowledge for effective use (knowledge is transformed input language to an
internally usable language or representation). Thus the learner is required to perform some
inference, though a large amount of its success rests on the teacher s ability to effectively
organize the knowledge, so as to augment the student or learner s already existing knowledge.
Learner by instruction parallels or goes hand-in-hand with most formal education methods of
teaching. The machine learning task of getting an expert system that can accept, store and
effectively respond to instruction (of language, voice input or otherwise) has taken its toll in the
Artificial intelligence frontier.

LEARNING IN PROBLEM SOLVING

This is actually learning from our own experience, without the aid of an instructor or
advisor. And this type of learning does not involve an increase in knowledge; but rather, in the
methods of solving a problem, using our already existing knowledge. Thus our experiences
teach us new rules which in turn direct the problem-solving processes. In machine learning the
problem solver needs to store and consult these rules from time to time, which can be partially
overcome by a utility measure in order to keep track of how useful the learnt rules are, and
deleting them when necessary (i.e. when they outlive their usefulness). Learning in problem
solving can be sub-classified as learning by parametric adjustment, learning with macro-
operators, and learning by chunking.

(i) Learning by parametric adjustment Under this learning process, learning

programs rely on an evaluation procedure that combines information from several
sources into one single summary statistic. A good example is Samuel's checker
player. In this sub-class of learning, outcomes are used to adjust the weights for
factors in an evaluation function. Considerations include initial weights, increase or
decrease in certain weights, etc.

(ii) Learning with macro-operators This involves rote learning a sequence of

operations found to be useful during problem solving. Macro-operators were used in
the early problem-solving expert system known as STRIPS. After each problem-
solving episode, the learning component takes the computed plan and stores it in
the manner of macro-operators.

(iii) Learning by chunking: This learning classification is rote learning in the context of a
Production System (Chunking is a process similar in flavor to macro-operators). Rules
which are useful and always fired together are chunked to form one large
production. The idea of chunking comes was drawn from a psychological literature
on memory and problem solving, and its basis of computation is in production
 INDUCTIVE LEARNING
Also known as learning from examples, this type of learning is a form of supervised
learning that uses specific examples of particular instances to reach general conclusions (i.e.
applying the specific to the general). Here, the amount of inference by the learner is much
more than in that engaged in, when learning by instruction; as no general concepts are
provided by a teacher. Although learning by examples does not require as much inference as in
learning by analogy since there are no similar seed concepts, by which new concepts can be
created. This type of learning can be sub-classified according to the source of examples;

(i) The source is a teacher This is a source of examples generated by a teacher who
knows the concepts well and presents them in a way meant to be as helpful as
possible to the learner. If the teacher knows the learner s learning abilities well, he
can choose examples to suit the learner s convergence on the required concept.

(ii) The source is the learner itself Under this source the learner quite typically
understands its own knowledge state, but does not really know the required
concept to acquire.

(iii) The source is the external environment In this case, the example generation
process is operationally random, as the learner must learn from relatively
uncontrolled observations.

(iv) Only positive examples Although positive examples provide instances for the
required concept to be acquired, they do not provide information for preventing
over generalization of inferred concepts.

(v) Positive and negative examples for these kinds of examples, positive examples
force generalization while negative examples prevent overgeneralization. This is the
most typical form of learning from examples.

EXPLANATION BASED LEARNING

This type (EBL) of learning involves the extraction of the concept behind the
information contain within one example, and generalize to other instances. And it broadly
requires domain-specific knowledge. In general, the inputs to explanation based learning
programs are as follows:
1. A Goal Concept
2. A Domain Theory (or Knowledge Base)
3. A Training Example
4. An Operationally Criterion

LEARNING FROM OBSERVATION AND DISCOVERY

Also known as unsupervised learning, this type of learning is a very general form of
inductive learning that requires the learner to perform more inference than all the other
methods of learning discussed. It includes theory formation tasks, discovery systems, the
creation of classification criteria to form taxonomies and other similar tasks without the benefit
of an external teacher. Learning by observation tends to span several concepts that need to be
acquired rather than only one concept. Much like Learning in Problem Solving, this involves
gleaming information without the use of a teacher, and Focuses much more on extracting
knowledge, rather than strategies or even operations in problem solving. We may sub-classify
learning from observation according to the level of interaction with the environment. These
being:
1. Passive observation This is where the learner classifies and taxonomizes observations
of multiple aspects of the environment.
2. Active experimentation Where the learner perturbs the environment to observe the
results of its perturbations. Experimentations may be random and dynamically focused
according to how interesting they are, or according to theoretical constraints.

LEARNING BY ANALOGY

This is the determination of correspondence between two or more different

representations. It entails acquiring new knowledge or skill by way of transforming and
augmenting already existing knowledge bearing a similarity to the desired or required concept,
into a form effectively useful as an application, to the new situation (i.e. Use methods in
previously solved problems to derive methods of solving new problems). A good example is the
consideration of someone who drives small vehicles with relative expertise who has never
driven a truck. Such an individual can easily adjust to the truck driving task using his already
existing small-vehicle driving knowledge though perhaps imperfectly. Similarly, the learning-
by-analogy system can be used to convert an existing computer program, into one capable of
performing a related task or function different from that for which it was originally designed.
 INTRODUCTION TO FORMAL LEARNING THEORY
Learning theory standing alone merely suggests, as some cognitive philosophers
believe, the empirical study of human and animal learning (emanating from the behaviorist
paradigm in psychology).The phrase formal places a different paradigm on the implied
learning theory under consideration here, distinguishing it from the former.

Formal learning theory is a mathematical embodiment of normative epistemology

(i.e. the theory of knowledge visa vis methods validity and scope), which deals with the
phenomenon of how an agent should use observations about its environment in arriving at
correct and informative projections. Formal learning theory is also known as computational
learning theory, due to the vast development and applications drawn from computer science.
In computer science, many resultant developments in learning theory are inclined to valiant s
idea of learning generalizations that are Probably Approximately Correct (PAC learning). This
sub-branch of learning was developed by a group of philosophers (Putnam, Glymour and Kelly)
to serve as a normative framework for scientific reasoning and inductive inference in cognitive
science. Some philosophical terms for learning-theoretic epistemology include logical reliability
(Kelly 1996), and means-end epistemology (schulte 1999). Unlike other philosophical
approaches to inductive inference, formal learning theory pursues the context of a dependent
means-ends analysis; i.e. which implies that the best way of achieving goals, given an empirical
problem and a set of given goals.
 BOOKS:

1. Anderson, Dave; McNeill, George. (1992). Artificial Neural Networks technology. A DACS
State-of-the-Art Report. Kaman Sciences Corporation, Utica, New York 13502-4627.
2. Bellman, R. E. (1978). An Introduction to Artificial Intelligence: Can Computers Think?
Boyd &Fraser Publishing Company, San Francisco.
3. Charniak, E.; McDermott, D. (1985). Introduction to Artificial Intelligence. Addison-
Wesley, Reading, Massachusetts.
4. Haugeland, J. editor (1985). Artificial Intelligence: The Very Idea. MIT Press, Cambridge,
Massachusetts.
5. Haykin, Simon. (1999). Neural networks: A Comprehensive Foundation. 2nd edition.
Pearson Education, Inc. McMaster University. Ontario Canada.
6. Kelly, K. (1996). The Logic of Reliable Inquiry. Oxford University Press.
7. Luger, G. F. and Stubblefield, W. A. (1993). Artificial Intelligence: Structures and
Strategies for Complex Problem Solving. Benjamin/Cummings, Redwood City, California,
second edition.
8. Michalski, R.S; Carbonell, J; Michel, T. (1983).Machine learning: An Artificial Intelligence
Approach. Tioga Publishing Co., Palo Alto, California.
9. Michalski, Ryszard S; Carbonell,Jaime G; Mitchell, Tom M. (1984). (Symbolic
Computation) Machine Learning - An Artificial Intelligence Approach. Springer-Verlag,
Berlin Heidelberg, New York, Tokyo.
10. Munakata, Toshinori. (2008). Fundamentals of the New Artificial Intelligence. 2nd
Edition. Springer-Verlag London Limited.
11. Perlovsky; Leonid I. (2001). Neural Networks and Intellect: Using Model-Based Concepts.
New York. Oxford university press, Inc.
12. Kecxnan, Vojislav. (2001).Learning and soft computing: support vector machines, neural
networks, and fuzzy logic models. Massachusetts Institute of Technology (MIT),
Massachusetts (United States of America).
13. Kurzweil, Ray. (1990). The Age of Intelligent Machines. MIT Press, Cambridge,
Massachusetts.
14. Rich, E.; Knight, K. (1991). Artificial Intelligence. 2nd edition. McGraw-Hill, New York.
15. Russell, Stuart; Norvig, Peter. (1995). Artificial Intelligence: A Modern Approach. 1st
Edition. Prentice-Hall, Inc. A Simon & Schuster Company, Englewood Cliffs, New Jersey
07632.
16. Russell, Stuart; Norvig, Peter. (2010). Artificial Intelligence: A Modern Approach. 3rd
Edition. Pearson Education, Inc., Upper Saddle River, New Jersey 07458.
17. Schalkoff, R. I. (1990). Artificial Intelligence: An Engineering Approach. McGraw-Hill,
New York.
 18. Winston, Patrick H. (1992). Artificial Intelligence. 3rd Edition. Reading, MA: Addison-
Wellsley. ISBN: 0201533774.
19. Witten, Ian H; Frank, Eibe; Hall, Mark A. (2011). Data Mining; Practical Machine Learning
Tools and Techniques. 3rd Edition. Elsevier Inc.
.

JOURNALS:

1. Gifford, David K. (2005). Bioengineering Applications in Computer Science. Computer

Science and Artificial Intelligence Laboratory (CSAIL), MIT.
2. Hoffman, Guy; Lockerd, Andrea. Robot Task Learning and Collaboration. MIT Media
Laboratory.
3. Schulte, O. (1999). Means-Ends Epistemology. The British Journal for the Philosophy of
science.
4. Sotala, Kaj. (2012). Advantages of Artificial Intelligences, Uploads, and Digital Minds.
University of Helsinki, Singularity Institute Research Associate. (International Journal of
Machine Consciousness.)

WEBSITES:

1. Chakraborty, R C. (June 01, 2010). Fundamentals of Neural Networks.

http://www.myreaders.info/html/artificial_intelligence.html
2. Cybernetics: A Definition. (2004). PANGARO Incorporated.
http://www.pangaro.com/published/cyber-macmillan.html
3. Hong, Hui-ling. (1994). Machine learning and its applications in reliability analysis systems.
Durham theses, Durham University. Available at Durham E-Theses Online:
http://etheses.dur.ac.uk/5513/
4. Introduction to Artificial Intelligence. (2012). VELOXITI, Inc.
5. McIlraith, Sheila. (2011). Intro to Artificial Intelligence. University of Toronto.
http://www.cs.toronto.edu/~sheila/384/w11/
6. Stringer, Simon M. Oxford Foundation for Theoretical Neuroscience and Artificial
Intelligence. www.oftnai.org