Decision-Making With

Big Data and AI

Suri Gurumurthi, Ph.D.
November 2019
Virtual & Augmented Reality
• Virtual Reality
• “The computer-generated simulation of a 3D
image or environment that a person can be
interact with in a seemingly real or physical
way by a person using special electronic
devices…”
• Augmented Reality
• “Superimposition of a computer-generated
image on a person's view of the real world,
providing a composite and altered
perspective...”
 Decision Making in VR and AR
• US Army trains its soldiers on high pressure
combat environments using VR
• De Beers allows potential customers to virtually
try on jewelry
• Entice and attract potential buyers
• Personalize the experience
• IKEA allows customers to virtually place the
catalog items in your home using mobile apps
• Visualize before purchase
• Reduce regret and returns!
 VR is a 3D Simulates space
and higher Simulates time
dimensional Simulates interactions!
simulation Can be used to model multi-agent decision-making

Differences
Between VR
and AR
Augmented Marries personal experiences with related imagery and
simulations
reality is
Adds value to the real world experience
more
Enables superior decision-making with additional
personal superimposed data
and Looks for and provides the missing elements that can
complex accelerate decision-making!
 Big Data are high-volume, high-velocity, and/or high-variety

“Big Data” Defined…

information assets that require new forms of processing to
enable enhanced decision making, insight discovery and
process optimization (Gartner, 2012)

2.5 quintillion bytes (10^18 bytes) of data

10,000
is created every day as of 2018)…

• More than 90% is unstructured data UNSTRUCTURED DATA

• Approx. 500 quadrillion files

• Quantity doubles every 2 years
• Most unstructured data is neither stored nor
analyzed!
GB of Data
(IN BILLIONS)

STRUCTURED DATA

2005 2010 2015

 Opportunities for Decision Makers…

European
As of 2017, administrators Poor data
53% of Forbes 40% projected
could save across Poor data can
surveyed Bad data or growth in
more than businesses cost
companies poor data global data
€100 billion and the businesses
adopted Big quality costs generated per
($149 billion) government 20%–35% of
Data US businesses year vs. 5%
in operational costs the U.S. their
Analytics. $600 billion growth in
efficiency economy $3.1 operating
annually. global IT
Up From 15% improvements trillion dollars revenue.
spending.
in 2015 alone by using a year.
big data
 Unstructured The key need in today’s digital
Data environment is to:
1. Reduce the dimensionality of big data
(aggregate)
Structured
Data 2. Transform and synthesize
3. Extract intelligence (usable
information)
Useful
Information 4. Analyze (understand relationships)
5. Value-added decision-making

Analytics

Decision-
Making
Big Data = Big Nothing, Unless…
Descriptive Analytics: Inference
Identify Draw from
goals sample
Start Population Sample

Draw Collect raw data

conclusions and summarize

Make inferences
about population
Population Sample Statistics
parameters

Source: Bennett, Briggs, and Triola (2014), Statistical Reasoning for Everyday Life.
9
 Inference: Causality Versus Correlation
No. of Eggs

No. of Hens

10
Correlation Coefficients… Correlation = 0

Dependent variable
• Correlation 30
25
• If two events (features or 20
measurement variables) 15
tend to occur together, they 10
5
are said to be correlated 0
• Example: Lighting and 0 2 4 6 8 10 12
thunder, weight and height Independent variable
• Correlation coefficient:
Measuring the strength of • Outliers
linear relationship of • An outlier that is consistent with the trend of
measurement variables the rest of the data can inflate the correlation
• An outlier that is not consistent with the rest
• r è +1: Positive correlation of the data can deflate the correlation
• r è -1: Negative correlation
• Simpson’s paradox
• r è 0: No correlation (?) • Groups combined inappropriately may mask
relationships
• The missing link (effect of a third variable) 11
Example: Death rate vs. speed limit

• Correlation between death rate and speed limit is

r = 0.55 (Rivkin, 1986)
• If Italy is removed, correlation drops to 0.098
• If Britain is also removed, correlation jumps to 0.70
12
Simpson paradox example

13
 Lurking vs. confounding variables
• Confounding occurs in a study
A
when you are unable to
B Causation
distinguish among the effects of
two or more explanatory
variables
• Potential problem
A B
• Example: Testing the effectiveness
Common response:
of a new drug
A B C = Lurking variable • Design of experiments: Give
C placebo to patients in clinic A and
real drug to patients in clinic B
Correlation • Even if we find difference between
exists patients in clinics A and B, what
kind of conclusion we can draw?
A ? B • The difference is due to the new
C= Confounding drug or clinics?
variable
C

14
Why should we care about correlation?
• How to confirm causation?
• Use of randomized experiments
(treatment-control study)
• Use of observational (case-control)
•
•
Easily observable
No proof needed
study if experiments are not possible or
ethical
• Evidence of a possible causal
connection:
• Not easily observable • There is a reasonable explanation of
• Require proof cause and effect
• The connection happens under varying
conditions
• Potential confounding variables are
ruled out
• Has research potential
• Most useful to us • Example: Developing a new medicine
15
Where Big Data and AI meet for a movie…

https://www.youtube.com/watch?v=ImpV70uLxyw
 Driverless Cars

https://www.youtube.com/watch?v=ftouPdU1-Bo
Credit Fraud
Detection

https://www.youtube.com/watch?v=AuwayMjvuT0
Autonomous Storage and Handling
https://www.youtube.com/watch?v=UtBa9yVZBJM
 Defining AI
• For e.g. Science which
makes machines Systems mimicking Rational thinking
perform real-time tasks human reasoning systems
that require higher
intelligence,
independent of human
intervention and
decision-making
• AI is the study of formal
AI
structures and problem
solving methods, that
aims to equip
computing machines Systems mimicking Rational acting
with capabilities to
solve problems, that
are comparable to
human behavior systems
human beings
 Using formal rules to draw valid conclusions
Philosophy What does knowledge truly mean?
How can we convert knowledge into action?
Creating formal rules to draw valid conclusions
Measures and computation
Mathematics
Probability and inference
Fuzzy logic

Operations Utility theory and decision theory

Research and Game Theory
Economics Probabilistic Decision Making & Control

Fundamental Neuroscience Information processing in the brain

Structures in AI
Behavioral conditioning
Psychology Cognitive psychology
Cognitive science

Computer Efficient algorithms

Science Autonomous agents

Natural language processing

Linguistics
Knowledge representation
 Systems That Think Like Humans
• Cognitive Science
• Computational Models integrated with
psychology
• Understands Emotional States
• Robots that understand jokes (not just
tell them)
• Robots that understand human moods
• Conversation with Alexa
• Me: Alexa, tell me all about the Beatles Song
“Imagine”
• Alexa: John Lennon composed “Imagine”…
• Me: Play “it”
• Alexa: Playing “Imagine” by John Lennon
• Develop second order associations
Systems Mimicking Human
Behavior (Act like humans)
• I propose to consider the question, 'Can machines think?
• To pass the Turing Test a computer must have the
following skills:
• Natural language processing
• Knowledge representation
• Automated reasoning
• Machine Learning
• Code-Breaking Algorithms
• Decoding message & identifying who sent the message!
• British and US Navy destroyed the German Navy in
WW-II
System of logical rules
• All men are mortal;
Systems that
• à Socrates is a man; Socrates is mortal Think
• Developing first order associations… Rationally
Software Programs & Algorithms
• Optimization of Routes
• Medical diagnoses (process of
elimination)
• Decision-making under uncertainty
(maximizing likelihood)
Systems that Act Rationally
• Autonomous Agents
• Achieves the best outcome
• Under uncertainty, the best expected outcome
• May act differently from humans
 1 2 3
Formal (Structured) Tasks Unstructured (Mundane) Expert tasks (structured and
• Games Tasks mundane)
• Mathematical Problems • Perception (visual vs speech) • Design
• Natural language processing • Scientific Analysis
• Judgement • Medical decision-making
• Control • Financial analysis and stock-
picking

Common Tasks in AI