Professional Documents
Culture Documents
Summer School
AI and IP
10 July 2020
Gyudong HAN
Director of Information Management Division
KIPO
Why do we need to know about AI?
Deepfake AI vs Bias
* deepfake(1:11) : https://www.youtube.com/watch?v=cQ54GDm1eL0
Machine Learning(ML)
Deep Learning(DL)
4
01 What is AI?
What is the first thing that comes to mind
when you hear the word “AI”?
AI In
daily life
* http://www.bbc.com/news/technology-35785875
* Google search 5
That’s partially because the
AI, difficult to define notion has changed over time
* “인공지능과 딥러닝”, 마쓰오 유타카, 동아 엠엔비 [Book: “AI and deep learning”, Yutaka Matsuo, Donga M&B] 8
Areas of AI
What’s required for a machine to be intelligent?
* http://futurearchitectureplatform.org/news/28/ai-architecture-intelligence/
Predictive analytics : statistical techniques from predictive modelling, machine learning, and data mining
that analyze current and historical facts to make predictions about future or otherwise unknown events
* https://en.wikipedia.org/wiki/Predictive_analytics 9
Relations between AI, ML and DL
Search, Inference,
Expert system etc
* http://blogs.nvidia.co.kr/2016/08/03/difference_ai_learning_machinelearning/
10
AI, a marketing strategy?
Traditional Program
Program Output
Input
(Rule)
Machine Learning
Input (Training ML Trained Model
Answer
Data)
Model (Rule)
Input A Answer X
* https://www.slideshare.net/JinwonLee9/ss-70446412
* Andrew Ng , Coursera Machine Learning 강의(https://www.coursera.org/learn/machine-learning/), Lecture 1
* “구글에서 배우는 딥러닝” 28쪽, 닛케이 빅 데이터 著, 서재원 옮김, 영진닷컴(Book: Deep learning from Google] 14
Estimating
ML vs Traditional Program House price
15
* https://medium.com/@ageitgey/machine-learning-is-fun-80ea3ec3c471 (Modified)
Areas of Machine Learning
➢ Supervised Learning
➢ Training with (input, label) pair
➢ Classification, Regression
➢ (Model) SVM, Decision Tree, kNN, Regression
➢ Unsupervised Learning
➢ Training only with input (w/o label)
➢ Clustering, Compression
➢ (Model) K-means clustering
➢ Reinforcement Learning
➢ Reward instead label
➢ Action Selection, Policy Learning
➢ (Model) MDP(Markov Decision Process)
* https://www.slideshare.net/JinwonLee9/ss-70446412
* Andrew Ng, Coursera Machine Learning(https://www.coursera.org/learn/machine-learning/), Lecture 1, 2
* 인공지능, 머신러닝, 딥러닝 입문, 김의중, 위키북스(Book: Introduction to AI, ML, DL) 16
Supervised Learning
Example of ML – Linear regression
(Core algorithm)
for iter = 1:num_iters
predictions = X * theta;
theta_0 = theta(1) - alpha / m *
sum(predictions - y);
theta_1 = theta(2) - alpha / m *
(X(:,2)' * (predictions - y));
theta = [theta_0; theta_1];
end
* Andrew Ng, Coursera Machine Learning lecture(https://www.coursera.org/learn/machine-learning/), Programming Exercise 1, Lecture 2 18
Supervised Learning
Example of ML – Linear regression
https://www.youtube.com/watch?v=4PHI11lX11I
Linear Regression using Gradient Descent in Python - Machine Learning Basics(00:09)
19
Unsupervised Learning
Example of ML – Clustering
20
* https://www.youtube.com/watch?v=bHi6DacI3os, Machine Learning: What is k-Means clustering?(01:46)
With
Human teaching
Example of ML – Reinforcement Learning
Table Tennis
https://www.youtube.com/watch?v=SH3bADiB7uQ
Towards Learning Robot Table Tennis(02:37), 2012. 5. 24.
21
Without
Human teaching
Example of ML – Reinforcement Learning
https://www.youtube.com/watch?v=V1eYniJ0Rnk
Google DeepMind's Deep Q-learning
playing Atari Breakout(01:42), 2015.3.7.
22
03 Artificial Neural Network(ANN)
ANN is based on a collection of
connected units or nodes called
artificial neurons, which loosely
model the neurons in a biological
brain. (Wikipedia)
Artificial neuron:
neuron in the brain Perceptron(Rosenblatt, 1958)
* http://solarisailab.com/archives/1206
* https://en.wikipedia.org/wiki/Biological_neuron_model
*https://www.yumpu.com/en/document/view/33756198/axon-dendrite-cell-body-neuron-nerve-cell-anatomy-synapse 23
ANN, Multi-Layer Perceptron(MLP)
* http://solarisailab.com/archives/1206 24
Example of ANN – Autonomous Vehicle
* 2017년 특허청 심사관 신기술 교육(제4차 산업혁명과 인공지능) 교재[Book: New technology textbook for Patent examiner] 25
Example of ANN – Autonomous Vehicle
➢ ALVINN: Autonomous Land Vehicle In a Neural Network
▪ First developed in 1986 by Carnegie Mellon University’s NavLab
▪ Architecture: Single hidden layer back-propagation network,
two retinas and a single intensity feedback unit
* https://papers.nips.cc/paper/95-alvinn-an-autonomous-land-vehicle-in-a-neural-network.pdf
* https://www.youtube.com/watch?v=5-acCtyKf7E 26
04 Deep Learning(DL)
Deep learning is part of a broader
family of machine learning methods
based on artificial neural networks
with representation learning(feature
learning). (Wikipedia)
* http://solarisailab.com/archives/1206
* https://www.analyticsvidhya.com/blog/2017/04/comparison-between-deep-learning-machine-learning/
27
Event 1 of Deep Learning
SuperVision team
(AlexNet) deep convolutional
Alex Krizhevsky, Ilya Sutskever, neural network
Geoffrey Hinton
University of Toronto
* https://www.slideshare.net/JinwonLee9/ss-70446412
32
DL, Self feature engineering
Deep learning allows computational models to learn
representations of data with multiple levels of
abstraction.
Deep-learning methods are representation-learning
methods with multiple levels of representation,
obtained by composing simple but non-linear modules
that each transform the representation at one level
(starting with the raw input) into a representation at
a higher, slightly more abstract level. […] The key
aspect of deep learning is that these layers of
features are not designed by human engineers: they
are learned from data using a general-purpose
learning procedure.
* https://machinelearningmastery.com/what-is-deep-learning/
* Deep learning(Yann LeCun, Yoshua Bengio, Geoffrey Hinton), Nature 521, 436-444(28 May 2015), 10.1038/nature14539 33
Feature extraction
Convolution
Pooling
* http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/
40
* https://medium.com/@ageitgey/machine-learning-is-fun-part-5-language-translation-with-deep-learning-and-the-magic-of-sequences-2ace0acca0aa
CNN + RNN for image description
* http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/ 41
* https://medium.com/@ageitgey/machine-learning-is-fun-part-5-language-translation-with-deep-learning-and-the-magic-of-sequences-2ace0acca0aa
DL needs Training/Retraining
Forget
everything?
42
DL needs Training/Retraining
Classification using DL
Bike
What if,
1. To add a new class?
(ex) Want to classify bike
→ Need to train whole
model again
2. Quality decline?
(ex) The model cannot
classify new car models
→ Train new models only
or whole model
* Andrew Ng, Coursera Machine Learning lec(https://www.coursera.org/learn/machine-learning/), Lecture 8
* Bike image : https://kr.clipart.me/transportation/bike-bicycle-clip-art-44237 43
Retraining vs Continuous Learning
* https://docs.aws.amazon.com/machine-learning/latest/dg/retraining-models-on-new-data.html
* https://mlinproduction.com/model-retraining/ 44
Strength of Deep Learning
* https://www.quora.com/What-is-the-difference-between-Deep-Learning-Machine-learning-and-Artificial-Intelligence-Is-Deep-learning-related-to-data-science
45
Godfathers of Deep Learning
https://www.wipo.int/wipo_magazine/en/2018/05/article_0001.html
Robot image: https://publicdomainvectors.org/ko/tag/%EB%A1%9C%EB%B4%87/2 47
Automation issue
AI and IP automation
https://www.wipo.int/wipo_magazine/en/2018/05/article_0001.html
48
Automation issue
* https://www.bbc.com/news/technology-49191645 50
Patent policy issue
53
Patent policy issue
* https://www.fiveipoffices.org/material/ai_roundtable_2018_report/ai_roundtable_2018_report/ai_roundtable_2018_report 54
Copyright policy issue
AI Arts
Authorship/ownership
Only works created by human can be protected
59
The more you know about AI,
the better you can do with it.
Thank you.