23AIE231M Introduction to AI & Data Science

Minor: Artificial Intelligence & Data Science

October 2023

History & Foundations of AI & Data Science

Unit 01: L01-L03

Puja Dutta, PhD Amit Agarwal, PhD

Assistant Professor, Civil Engineering Professor, Electrical & Electronics Engineering | Cybersecurity
+91 97432 94057 +91 98679 10690
https://www.amrita.edu/faculty/dr-puja-dutta https://www.amrita.edu/faculty/amit-agarwal
https://www.linkedin.com/in/amit-agarwal-635a548
I. What is AI?
II. History of AI
III. Problem Set
 1. What is AI? [1/9]
Matches Intuition?
Completeness?
Consistency?

Counter Examples:
- Human flight
- Airline routing
- Career choice
- Chess
Hypotheses about what humans do, why humans do, how humans do
Hypotheses about what humans perceive as useful or displaying intelligence
3 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security
 1. What is AI? [2/9]
Do the quadrants on the last slide, suggest that humans are irrational?
T h i n k i n g H u m a n l y [ C o g n i t i v e S c i e n c e ] : To make the claim that a program behaves
like a human we need to understand the working on the human mind.
If the program’s input–output behavior matches corresponding human behavior, that is
evidence that some of the program’s mechanisms could also be operating in humans.
The field of cognitive science brings together computer models from AI and experimental
techniques from psychology to construct theories of the human mind.
Over time, the fields of AI and cognitive science have developed distinctly, though with
infusions into each other.

4 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security

 1. What is AI? [3/9]
A c t i n g H u m a n l y [ Tu r i n g Te s t ( 1 9 5 0 ) ] : conceptualized to provide an operational
definition of intelligence – “A computer passes the test if a human interrogator, after posing some
questions, cannot tell whether the written responses come from a person or from a computer.”
What do we need to conduct the test at a minimum? A computer with following capabilities:
1. Natural Language Processing: Understand what the human asks and reply in a manner that a
human is unable to distinguish whether the response is from a human or a machine
2. Knowledge Representation: Store what it hears or knows
3. Automated Reasoning: Use stored info to draw new inferences if necessary to answer questions
4. Machine Learning: Detect and extrapolate patterns
Intelligence is NOT (deemed to be) in the appearance. Therefore, the physical appearance of
the machine is not relevant.
However, the machine will need to sense (to receive what is being asked) and use an actuator to
control an end effector (in this case, a keyboard or a microphone) to pass the test.

5 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security

 1. What is AI? [4/9]
T h i n k i n g R a t i o n a l l y : Rationality rests on the notion that there exist patterns of
arguments that always yield correct conclusions when given correct premises. E.g. All planets
revolve around the Sun. The Earth is a planet. Therefore, the Earth revolves around the Sun.
Using this notion, by 1965, programs were developed that could solve any solvable problem
described in logical notation. Proof was mathematical. The logicist tradition in AI was a hope
to build intelligent systems using this approach. Two problems with this approach:
1. Hard to find atomic assertions, specially in the presence of uncertainty
2. Computational complexity
All traditions in AI suffer from these limitations however, it got highlighted in logicist traditions
first because of grand assertions of completeness raised early on in this field.

Then, ChatGPT???
➢ Its design assumes bounded-rationality on part of the user
➢ It has a gigantic parametric space and therefore, it can address many questions

6 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security

 1. What is AI? [5/9]
A c t i n g R a t i o n a l l y : A rational agent is one that acts so as to achieve the best outcome or,
the best expected outcome. Thus, we can see that acting rationally is not just about:
1. using the correct logic to draw inferences, but also, about 𝐼𝐹 𝐴𝑁𝐷 𝐴, 𝐵 𝑇𝐻𝐸𝑁 𝐶, …
2. which data points to act upon and 𝑆ℎ𝑜𝑢𝑙𝑑 𝐼 𝑓𝑜𝑐𝑢𝑠 𝑜𝑛 𝑡ℎ𝑒 𝑝ℎ𝑜𝑛𝑒 𝑡𝑎𝑙𝑘? 𝑈𝑛𝑡𝑖𝑙 𝑤ℎ𝑒𝑛?
3. what objective fn to use [ 𝛼1 + 𝛽1 𝐵𝑜𝑜𝑙𝑒𝑎𝑛𝑚𝑖𝑙𝑘−𝑠𝑝𝑖𝑙𝑙𝑒𝑑 + 𝛽2 𝑄𝑡𝑦𝑚𝑖𝑙𝑘−𝑠𝑝𝑖𝑙𝑙𝑒𝑑 +
Overlap

𝛾1 𝐶𝑜𝑛𝑣𝑒𝑟𝑠𝑎𝑡𝑖𝑜𝑛 𝑃𝑙𝑒𝑎𝑠𝑢𝑟𝑒 + 𝜃1 𝑃ℎ𝑜𝑛𝑒 𝑆𝑎𝑓𝑒𝑡𝑦 ]

Aforementioned example is for the case of speaking over the while the milk is boiling …
The ‘act rational’ agent approach is more general than the ‘law of thoughts’ approach, i.e., a
strong logic is neither a sufficient condition nor a dominant strategy for rationality.
Knowing a lot of facts may improve one’s ability to act rationally. But it may neither be
necessary nor sufficient and, in some cases, damage one’s ability to act rationally.
Wrong or anachronistic objectives can render any edge from logic or collection of data points
that one has entirely irrelevant to demonstration of superior rationality.
Problem: Discuss the case of using SM while walking on the street in the context of rationality.
7 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security
 1. What is AI? [6/9]
I f A I E m e r g e d o u t o f S i z e : People from the neuroscience community who have an
interest in replicating humanistic behaviors on machines hold a position that if intelligence is a
physical process then, observing intelligent behavior in machines should be possible once their
computing power approaches that of the human brain.

8 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security

 1. What is AI? [7/9]
Buzzwords with a material overlap with AI
➢ Neuroscience
➢ Cognitive Science
➢ Game Theory
➢ Operations Research
➢ Control Theory
➢ Machine Intelligence
➢ Computational Intelligence
➢ Data Science
➢ Decision Science

9 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security

 1. What is AI? [8/9]
What is AI through Buzzwords/Examples:
➢ Knowledge Representation and Reasoning ➢ Planning
➢ Search, Satisfiability, and Constraint Satisfaction ➢ Cognitive Modeling
➢ Optimizing the mix of petroleum fractions ➢ Probabilistic Reasoning
➢ Logic Programming and Theorem Proving ➢ Machine Learning
➢ Neural Networks and Genetic Algorithms ➢ Data Mining
➢ Automated Fault Diagnosis for Artillery ➢ Multi-Agent Systems
➢ Optimizing Chip Design Process ➢ Robotics and Perception
➢ IC chip verification ➢ Building Environment Control
➢ Radar software for multi-target tracking ➢ Games with Computer Opponents

10 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security

 1. What is AI? [9/9]
AI Epistemology:
1. Philosophy 3. Computer Science
➢ Can formal rules lead to valid conclusions? ➢ Knowledge representation
➢ How does the mind arise from the brain? ➢ Logical Reasoning
➢ Where does knowledge come from? ➢ Algorithms What cutting edge AI&ML firms assume
you know …
➢ How does knowledge lead to action? ➢ Data structures
➢ Algorithm complexity [memory, bandwidth, time, scalability]
2. Math
➢ Statistics 5. Tools
➢ Probability theory What cutting edge AI&ML firms ➢ Visualization
➢ Calculus assume you know … ➢ Design
What many employers expect you know …
➢ Linear algebra ➢ Optimization

4. Hardware ➢ Computing
E.g. Ad tools, News writing tools, Microsoft
➢ Applications
➢ Sensors Tools, Graphics design tools …
What some employers ask for …
➢ Actuators

11 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security

I. What is AI?
II. History of AI
III. Problem Set
 2. History of AI [1/6]

Warren McCulloch & Walter Pitts (1943-55): AI Marvin Minsky (1950): built the first neural
founded in physiology, neuron function & computer.
propositional logic.
In 1957, his PhD thesis reported mathematical
Theory of how of neurons are stimulated by analysis of neural networks.
sufficient number of neurons in the
John McCarthy, Marvin Minsky, et. al. (1956):
neighborhood firing.
organized a 2-month summer workshop for
The state of a neuron was conceived of as researchers in automata theory and neural
“equivalent to a proposition which proposed its networks - . It is here, where for the first time,
adequate stimulus.” the term ‘artificial intelligence’ was used.
They showed that: i) any computable function
could be computed by a network of connected
neurons (CNs) & ii) all the logical gates can be
implemented by CNs &, suggested that iii)
networks could learn.

13 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security

 2. History of AI [2/6]
Donald Hebb (1949): A rule to update the
connection strengths between neurons.
If two connected neurons are ‘ON’
simultaneously then, the weight of their
mutual connection must increase. This is an
example of ‘learning by association’.
Rosenblatt (1968): laid out the math behind
the Hebbian learning rule. He called the
Perceptron Learning Illustration
learning agent ‘perceptron’.
Teuvo Kohonen (1982): Self Organizing Map
(SOM) learning rule for unsupervised
learning. It is based on the notion of
distance between feature vectors.

Self Organizing Map

14 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security

 2. History of AI [3/6]
John McCarthy (1958): invented LISP in 1958
and also wrote a program that showed how
new axioms can be accepted in the normal
course of operation without being
reprogrammed. They used generating driving
plans to the airport as an illustrative example.
In 1963 James Slagle’s program could solve
closed-form calculus integration problems
typical for 1st year students.
In 1968 Tom Evan’s program could solve
geometric analogy problems in IQ tests.
15 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security
In 1967 Daniel Bobrow’s program could solve
algebra problems such as: “If the number of
customers Tom gets is twice the square of 20% of
the number of advertisements he runs, and the
number of advertisements he runs is 45, what is
the number of customers Tom gets?”
In 1967 Allen Newell and Herbert Simon
expressed that (paraphrased) any (human or
machine) system exhibiting intelligence must
operate by manipulating data structures
composed of symbols.
 2. History of AI [4/6]
The 2 decades … (1952-69): As both computers & programming tools were primitive, they could
only perform basic operations. Where they could demonstrate some intelligence, their success
reports were exaggerated..
Challenge 1: These early programs operated with syntactic manipulation – they did not have any
domain knowledge.
In the wake of Sputnik launch in 1957, the US generously funded AI efforts to speed up
translation of Russian scientific papers. It was thought that syntactic transformations based on
the two grammars and word replacement from an electronic dictionary would suffice to preserve
the exact meanings of sentences.
The fact that accurate translation requires background knowledge to resolve ambiguity was lost.
The famous retranslation of “the spirit is willing, but the flesh is weak” as водка хорошая, но
мясо тухлое (the vodka is good but the meat is rotten) instead of дух желает, но плоть слаба
illustrates this.

16 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security

 2. History of AI [5/6]
Challenge 2: That a program can find a solution in principle does not mean that it can find the
solution in practice.
As an illustration of Challenge 2, while Machine Evolution introduced by Friedberg (1958) were
based on the sound view that by making an appropriate series of small mutations to a machine-
code program, one can generate a program with good performance for any particular task, despite
spending thousands of hours of CPU time, almost no progress was demonstrated.
In the late 1960s the USA ended most funding, except at a few universities. The same happened
in the UK in 1973.
Challenge 3: Single layer perceptron was known to be unable to solve non-linear problems such
as the XOR problem. This limitation does not apply to multi-layer perceptron (Arthur Earl Bryson,
Yu-Chi Ho, 1969) but this was discovered at a time of declining interest in AI. Thus, it had to wait
for resurgence in interests in AI in mid-80s.
Another known weakness of perceptron was a poor ability to represent anything. This spurred the
development of Knowledge-based Systems (KBS) 1969-79.

17 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security

 2. History of AI [6/6]
Rise of AI as an Industry (1980s-): DEC used the first AI-system in 1982 to help configure
computer orders. This reportedly saved it $40mn annually by 1986. By late 1980s, most major
US firm owned AI systems. The market grew 1000x to billions of dollars in just 10 years.

And then came the AI Winter as AI failed to meet extravagant promises.

Neural Networks on a Comeback (1986-): As mentioned earlier, multi-layer perceptron was

reinvented. It was widely disseminated in the collection Parallel Distributed Processing
(Rumelhart and McClelland, 1986). The ‘connectionist’ model came back as a competitor to
‘symbolic’ representation’.

The debate as to which model is superior has been not settled but cast aside in preference for
‘what works’.

AI Adopts Scientific Methods (1987-): Math came to the centerstage with rapid incorporation of
Bayesian reasoning, probability theory and statistics. This & hardware advancements paved way
for the rise of ‘AI Agents’.

18 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security

I. What is AI?
II. History of AI
III. Problem Set
 3. Problem Set [1/5]
Q1: Supermarket bar code scanners can scan lakhs of different products and provide several
bits of information about those products with very high accuracy (99.99%+). They are scalable
in the sense that they can do the same for hundreds of crores of different objects without
losing their accuracy. Internet routing algorithms respond dynamically to the state of the
network to route packets between source and destination. A good routing algorithm will
provide the necessary bandwidth throughout the route, minimize the latency and minimize
packet drops that can arise due to packet collisions or packet corruption.
Internet routing algorithms neither have the scalability of supermarket bar codes nor its
accuracy. Which one would you deem more intelligent, and why?
Q2: Write the objective function you seek to maximize over the next 5 years after graduation
with respect to your career (only).
Q3: Write the objective function you seek to maximize over the next 5 years after graduation
with respect to your personal life (only).
Q4: Make changes to the outcome of Q2 to simultaneously maximize career and personal life.
Briefly comment on the changes in the objective function.
20 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security
 3. Problem Set [2/5]
Q5A: Write 5 parameters of a crude oil refinery that you will maximize.
Q5B: Write constraints on maximization of each of those parameters.
Q5C: Combine the two & write the utility function.
Q5D: Where is intelligence required to achieve the utility function maximization? Justify.

21 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security

 3. Problem Set [3/5]
Q6: Computational models of cognitive vision involve complex mathematical operations. Most
humans never learn the kind of complex math at all. Yet, we all can process very large amounts
of visual data (or eye resolution is 5-15 megapixel when there is no relative motion between the
scene and our eye, vs, up to, 576 megapixel with relative motion) without feeling tired or
mathematically challenged. The two images show satellite data obtained at different times.
i) Identify change in infrastructure between the two images.
ii) Mark those changes in the 2nd image.
iii) Write steps in bullet points of the
process you followed.
iv) Mention the steps where you think
a machine could do ‘better’ than you
were able to and, justify your opinion.
v) Mention the metadata you are
using to make your assessments faster.

22 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security

 3. Problem Set [4/5]
Q7: Statement1: All planets revolve around the Sun. {𝑃} revolves around 𝑆
Statement2: The Earth is a planet. 𝐸 ∈ 𝑃
Conclusion: Therefore Earth revolves around the Sun. 𝑆1 & 𝑆2 ⇒ 𝐸 revolves around 𝑆
Study the example above.
1. Rewrite the italicized assertions and conclusion in symbolic form (the colored parts) and
then answer if the conclusion logically follows from the assertion.
2. Justify your claim in 1 or 2 bullet points.
All planets revolve around the Sun. The Earth revolves around the Sun.  the Earth is a planet.
Q8: In the context of Slide 15 where we noted a logic
system that can absorb new axioms, consider this:
Assertion 01: Classroom A has 4 students.
Assertion 02: Circle A has 3 points.
We wish to introduce Assertion 03: Classroom A has 5 students. and Assertion 04: Circle A has 5 points.,
to the logic system. Write the steps checks the logic system checks performs prior to
incorporating the new information.
23 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security
 3. Problem Set [5/5]
Q9: Estimate how much is the total computer memory in the world? I am interested in your
reasoning as opposed to the actual number.
Q10: Recall Hebbian Learning Rule from Slide 13.,
Give examples from real-life where this learning rule
fails. Justify in bullet points.

24 Puja Dutta, CIE, Amit Agarwal, EEE | TIFAC-CORE in Cyber security