AI & Maintenance

Lecture 5: Unsupervised Learning

Recap: Last week

• Model evaluation

• Different cases  different metrics

• Use your data wisely

• “Not all errors are created equal”

Unsupervised Learning
Unsupervised Machine Learning
Supervised Learning
Machine Learning Unsupervised Learning

Planning Reinforcement Learning

Expert Systems
Sentiment Analysis
Natural Language Processing Topic Modeling
Artificial Intelligence
Chatbots
Knowledge Representation

Computer Vision

Robotics

Speech
Unsupervised Machine Learning
Supervised Learning
Machine Learning Unsupervised Learning

Planning Reinforcement Learning

Expert Systems
Sentiment Analysis
Natural Language Processing Topic Modeling
Artificial Intelligence
Chatbots
Knowledge Representation

Computer Vision

Robotics

Speech
Unsupervised Machine Learning
Classification
Supervised
Learning
Regression

Clustering
Unsupervised
Machine Learning Association
Learning
Dimensionality reduction

Game AI
Reinforcement
Learning
Robot navigation
Unsupervised Machine Learning
Classification
Supervised
Learning
Regression

Clustering
Unsupervised
Machine Learning Association
Learning
Dimensionality reduction

Game AI
Reinforcement
Learning
Robot navigation
Unsupervised Machine Learning
Customer Segmentation
Clustering
Anomaly Detection

Recommender Systems
Machine Learning Association
Association Rule Learning

Data Visualisation
Dimensionality
Reduction
Signal separation
Unsupervised Machine Learning

• No labels available

• Clustering

• Association

• Dimensionality reduction
Clustering
Clustering

• Find similar items

• Describe groups

• K-means

• Hierarchical clustering

• Anomaly detection
K-means

1. Pick K points
K-means

1. Pick K points

2. Assign closest points

K-means

1. Pick K points

2. Assign closest points

3. Compute mean of cluster

K-means

1. Pick K points

2. Assign closest points

3. Compute mean of cluster

4. Shift points to means

K-means

1. Pick K points

2. Assign closest points

3. Compute mean of cluster

4. Shift points to means

5. Back to step 2
K-means

1. Pick K points

2. Assign closest points

3. Compute mean of cluster

4. Shift points to means

5. Back to step 2
K-means

1. Pick K points

2. Assign closest points

3. Compute mean of cluster

4. Shift points to means

5. Back to step 2
K-means

1. Pick K points

2. Assign closest points

3. Compute mean of cluster

4. Shift points to means

5. Back to step 2
K-means

• When to stop?

• Max iterations

• Variation threshold
K-means

Pros Cons

• Explainable • Need to know # of clusters

• Easy to train • Depends on random start

• Assumes spherical clusters

Hierarchical clustering

• Build a similarity tree (dendrogram)

• Top down (Divisive)

• Bottom up (Agglomerative)
Agglomerative hierarchical clustering

• Items start in own cluster

• Iteratively aggregate

• Stop when all in one cluster

Agglomerative hierarchical clustering
Hierarchical clustering

Pros Cons

• Deterministic • Complexity (O(n²))

• Flexible • Does not scale to big datasets

• Can take some time to compute

• No assumed cluster shape

• Works categorical data

• Provides multiple clusterings

Evaluation

• Unsupervised machine learning

• No correct labels

• How to compute scores?

Evaluation

• Compare items

• Davies-Bouldin index

• Silhouette Score
Davies-Bouldin index

• Distances to center

• Distance to other centers

Davies-Bouldin index

• Distances to center

• Distance to other centers

Silhouette score

• Distances to points within cluster

• Distances to points outside cluster

Silhouette score

• Distances to points within cluster

• Distances to points outside cluster

Assocation
Association Rule Mining

• Market basket analysis

• Recommender systems

• If you liked X you might also like Y

• “Recommended for you”

Association Rule Mining

• Market basket analysis

• Recommender systems

• If you liked X you might also like Y

• “Recommended for you”

Association Rule Mining

• Collaborative Filtering

• Content-based Filtering
Collaborative Filtering

• Compare across users A B C D

• Similarity  
   
  
 ? 
Collaborative Filtering

• Compare across users A B C D

• Similarity  
   
  
 ? 
Collaborative Filtering

• Compare across users A B C D

• Similarity  
   
  
 ? 
Collaborative Filtering

• Compare across users A B C D

• Similarity  
   
  
 ? 
Collaborative Filtering

• Compare user scores A B C D

• User Similarity  
   
  
 ? 
Content-based Filtering

• Compare across items    ?

• Content similarity     
• “Rabbit hole”     
    
    
Content-based Filtering

• Compare across items    ?

• Content similarity     
• “Rabbit hole”     
    
    
Content-based Filtering

• Compare across items    

• Content similarity     
• “Rabbit hole”     
    
    
Association Rule Mining

• Cold start problem

• Scaling is hard

• Sparsity
Dimensionality Reduction
Dimensionality Reduction

• Visualisation

• Feature learning / signal separation

Principal Component Analysis

• Linear Algebra

• Greatest variance

• Number of components
Principal Component Analysis

• Linear Algebra

• Greatest variance

• Number of components
Principal Component Analysis

• Linear Algebra

• Greatest variance

• Number of components
Auto-encoders

• No labels?
• Use the input as the labels!

• Neural network

• Middle hidden layer < input layer

Auto-encoders

Input Hidden Output

layer layers layer
Auto-encoders

Pros Cons

• Very powerful • Black box

• Training time / complexity

Recap

• Unsupervised Machine Learning

• Clustering

• Association

• Dimensionality reduction
Next week

• Guest Lecturer

• Real world applications of machine learning