How Do Machines

Learn from Data?

Prof. Wei-Guang Teng
Dept. of Engineering Science, NCKU
What is AI?

■ In this 2001 American movie, it says

that “his love is real, but he is not”
– See the trailer here

■ A more general definition:

“An artificial creation of human-like
intelligence that can learn, reason,
plan, perceive or process natural
language”

https://upload.wikimedia.org/wikipedia/en/e/e6/AI_Poster.jpg
2
Advertisement:
Machine Learning for Modern Artificial
Intelligence @ICS-2020 (Dec. 17~19)
https://ics2020.ncku.edu.tw/

3
Why Data Science?
For Data-Enabled AI

“Businesses,
governments and society
are only starting to tap its
vast potential”
– The Economist, 2010/2/27

4
Outline

■ Buzzwords? Data Mining vs. Machine Learning

■ A General Flow to Discover Knowledge
■ Classical Data Mining Tasks
– Classification
– Prediction
– Clustering
– Association
■ Concluding Remarks

5
Many Relevant But Confusing Topics

https://i2.wp.com/thedatascientist.com/wp-content/uploads/2018/12/data-science2.jpeg?w=925&ssl=1

6
DM & ML: Brotherhood!

■ A mixture of theoretical and practical tools

– Theory-oriented: derive everything deeply for solid
understanding, but less interesting
– Technique-oriented: flash over the techniques broadly for
shiny coverage, but difficult to be applied in practice

■ We’d like to tell stories to illustrate key theories, core

techniques, and usage in practice!
(Thanks for “Machine Learning Foundations” by Prof. Hsuan-Tien Lin)

7
From Learning to Machine Learning

■ Learning: acquiring new skill with experience

accumulated from observations
observations learning skill

■ Machine Learning: acquiring new skill with experience

computed/mined from data
data ML/DM techniques skill

8
What is Skill?

■ To improve some performance measure (e.g., prediction accuracy)

■ An application in computational finance:

stock data ML/DM techniques more $

9
An Application:
Recognizing Animals
■ What are these animals?

Source: http://dev.digsrome.org/wp-content/uploads/2013/07/duck-duck-goose-swan-660x220.jpg

■ Possible solutions:
– To define each of them and hand-program: difficult
– To learn from data (observations) and recognize: a two-year-old kid can do so

10
My kid @2Y2M (2016/1/26)

11
Key Essence of DM/ML:
Help Decide Whether to Use DM/ML

■ DM/ML: To improve some performance measure with experience

computed from data
improved
data ML/DM techniques performance
measure
1) There are some underlying pattern to be learned
So performance measure can be improved
2) But no programmable (easy) definition
So DM/ML is needed
3) Somehow there are data about the pattern
So DM/ML has some input to learn from

12
Quiz

■ Which of the following is best suited for DM/ML?

1) Predicting whether a baby cries in the next 5 minutes

2) Determining whether a given graph contains a cycle

3) Deciding whether to approve credit card to some customer

4) Guessing whether the earth will be destroyed by the misuse of nuclear

power in the next ten years

13
Outline

■ Buzzwords? Data Mining vs. Machine Learning

■ A General Flow to Discover Knowledge
■ Classical Data Mining Tasks
– Classification
– Prediction
– Clustering
– Association
■ Concluding Remarks

14
Data Overloading

■ Vast quantities of data are collected / stored out of fear

that important info will be missed
■ Data volume grows so fast that old data are never
analyzed
■ Database systems do not support queries like
– “Who is likely to buy product X”
– “List all reports of problems similar to this one”
– “Flag all fraudulent transactions”
■ But these may be the most important questions!

15
Too Much Data

■ “We are drowning in data, but starving for knowledge!”

https://c8.alamy.com/comp/X2PNH2/beach-boy-thirsty-water-
bottle-drinking-portrait-side-view-people-children-blond-summer-
holidays-holidays-sunny-heat-hot-thirst-drink-bottle-water-sea-
mineral-water-X2PNH2.jpg

16
An Ideal DM Workflow

https://miro.medium.com/proxy/1*KzmIUYPmxgEHhXX7SlbP4w.jpeg

17
A More Practical DM Workflow

18
Outline

■ Buzzwords? Data Mining vs. Machine Learning

■ A General Flow to Discover Knowledge
■ Classical Data Mining Tasks
– Classification
– Prediction
– Clustering
– Association
■ Concluding Remarks

19
Prediction: To Estimate a Value

https://www.researchgate.net/profile/Hieu_Tran
33/publication/333457161/figure/fig3/AS:7639
59762247682@1559153609649/Linear-
Regression-model-sample-illustration.ppm

https://i.stack.imgur.com/0BLlF.png
20
Classification: To Decide a Category
“Which characteristics distinguish the two groups?”

Honest
Tridas Vickie Mike

Crooked

Wally Waldo Barney

SOURCE: WELGE & REINCKE, NCSA

21
Learned Rules in the Model

Tridas Vickie Mike

Honest = has round eyes and a smile

SOURCE: WELGE & REINCKE, NCSA

22
A More Practical Example
Devise a predictive rule to classify a new person as A or B

height hair eyes class

short blond blue A
tall blond brown B
tall red blue A
short dark blue B
tall dark blue B
tall blond blue A
tall dark brown B
short blond brown B
SOURCE: WELGE & REINCKE, NCSA

23
To Build a Decision Tree
hair
dark blond
red

short, blue = B short, blue = A

{tall, blue = A}
tall, blue = B tall, brown = B
tall, brown= B tall, blue = A
short, brown = B

Completely classifies dark-haired Does not completely classify

and red-haired people blonde-haired people.
More work is required!

24
To Build a Decision Tree
hair
dark blond
red
short, blue = B short, blue = A
{tall, blue = A}
tall, blue = B tall, brown = B
tall, brown= B tall, blue = A
short, brown = B
eye
Decision tree is complete because blue brown
1. All 8 cases appear at nodes
2. At each node, all cases are in short = A tall = B
the same class (A or B) tall = A short = B
25
Final Decision Tree and Learned Rules

hair

dark blond
red

B A eyes
blue brown

A B

26
A Worse Decision Tree

■ Good news: a decision tree can always be built from training data
– Any variable can be used at any level of the tree
■ Bad news: every data point may wind up at a leaf (tree has not compressed
the data)
height
short tall

eyes eyes blue

blue brown brown

hair B B hair
blonde blonde dark
dark red

A B B A B
8 cases, 7 nodes. This tree has not summarized the data effectively

27
Clustering

■ Cluster: groups of records with similar characteristics

■ Searches for characteristics that most frequently occur in
common, and groups the related records accordingly
■ Example: Some specific products are most frequently
bought by shoppers who enter the supermarket around
5pm on Fridays
– Beers and diapers in Osco drug stores
– To get more knowledge with association rule techniques

28
An Example of Clustering

■ Are there natural clusters in the data (36,10), (12,8), (38,42),

(13,6), (36,38), (16,9), (40,36), (35,19), (37,7), (39,8)?

29
Supervised vs. Unsupervised Learning

■ Supervised learning (classification)

– The training data (observations, measurements, etc.) are
accompanied by labels indicating the class of the observations
– New data are classified based on the training set
■ Unsupervised learning (clustering)
– The class labels of training data are unknown
– Given a set of measurements, observations, etc. with the aim of
establishing the existence of classes or clusters in the data

30
https://i2.wp.com/thedatascientist.com/wp-content/uploads/2018/06/machine_learning_flowchart_scikit_learn.png?resize=1024%2C638&ssl=1
31
There are more …

32
Outline

■ Buzzwords? Data Mining vs. Machine Learning

■ A General Flow to Discover Knowledge
■ Classical Data Mining Tasks
– Classification
– Prediction
– Clustering
– Association
■ Concluding Remarks

33
 AI: Now and Next

By Prof. Hsuan-Tien Lin

34
https://miro.medium.com/max/1248/1*_t6GJPXvVrQEMqLZfz7CvA.png

35
 How Target Figured
Concluding Remarks Out A Teen Girl Was
Pregnant Before Her
Father Did (!!)

■ Problem definition is usually the most important &

challenging step!

https://www.english-to-go.info/2016/wp-content/uploads/2020/01/confusion.jpg 36
References

■ What is Data Mining, https://www.youtube.com/watch?v=grRwJ5jZBog

■ Data Mining: How You're Revealing More Than You Think,
https://www.youtube.com/watch?v=EH3bp5335IU
■ Machine Learning Foundations (機器學習基石),
https://www.youtube.com/playlist?list=PLXVfgk9fNX2I7tB6oIINGBmW50rr
mFTqf
■ scikit-learn Flowchart,
http://scikit-learn.org/stable/tutorial/machine_learning_map/index.html

37
Thank You!

Q&A
38