You are on page 1of 73

Jan Mulkens

Practical
SQL Server
Machine Learning Services
Thank you Sponsors

You are Community


Who am I

Jan Mulkens
Microsoft BI Consultant
Competence Lead: Microsoft Advanced Analytics

@JanMulkens
www.janmulkens.be
www.globalpowerbi.com

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
My employer

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Activities
Blog:
- www.janmulkens.be
- www.globalpowerbi.com

Organiser:
- Conference (www.powerbidays.com)
- User groups
- Microsoft Advanced Analytics (bit.ly/msaaug)
- Power BI Belgium (bit.ly/FlemishPowerBI)
- Virtual Power BI events (www.globalpowerbi.com)
- Internal Ordina events (BI Bake Off, Exam Preps, Fast Tracks, ...)

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Power BI Days
By & for the community
September 22 | 2018 Mechelen, Belgium
September 23 | 2018 Virtual

www.powerbidays.com
Activities

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Activities
Speaker & Mentor:
- Conferences, User Groups, Webinars

- Guest lectures, Workshops, Customer Training


- Internship mentor
- www.speakingmentors.com
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Agenda

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Side effects do NOT include
• Becoming an expert
• Full depth and breadth of SQL Server ML Services
• Gaining statistical knowledge

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Side effects DO include
• Get started with Machine Learning
• Good practices

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
The usual ML talk
Not so classic
Introduction to
Machine Learning

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
What is Machine Learning
Machine learning explores the study and construction of
algorithms that can learn from and make predictions on
data. Such algorithms operate by building a model from
example inputs in order to make data-driven predictions
or decisions, rather than following strictly static program
instructinos.
Machine learning is closely related to and often overlaps
with computational statistics; a discipline that also
specializes in prediction-making. It has strong ties to
mathematical optimization, which delivers methods,
theory and application domains to the field. @JanMulkens
https://en.wikipedia.org/wiki/Machine_learning
Un événement #SQLSatParis – SQLSaturday Paris 2017
What is Machine Learning

@JanMulkens
http://dilbert.com/strip/2013-02-02
Un événement #SQLSatParis – SQLSaturday Paris 2017
Machine Learning
Traditional Programming: Data and program is run on the
computer to produce the output.

Machine Learning: Data and output is run on the computer to


create a program. This program can be used in traditional
programming.

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Types of algorithms

https://en.wikipedia.org/wiki/Outline_of_machine_learning

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Types of algorithms

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Types of algorithms

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Applications Machine Learning Types

Supervised Unsupervised Semi-supervised Reinforcement


Learning Learning Learning Learning

Continuous Categorical Target Variable


Target Variable Target Variable not available

Regression Classification Clustering Association

Housing Price Medical Customer Market Basket


Prediction Imaging Segmentation Analysis

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Applications
• Spam filters • Player matching

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Applications • Recommendations

• Chat bots

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Standard demo
SQL Server SQL Server
2016 2017

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Standard demo
• Install and enable R Services
• sp_execute_external_script: Hello world
• Basic output of data with R Services
• Basic output of data – with a twist (column names! Wow!)
• Basic statistics operation and output
• Create stored procedure and save data

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
What you should
actually know
Revolution Analytics

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
In-database

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Machine Learning Services
Data Analyst / Scientist T-SQL Developer

RevoScaleR package Embed R code in stored procedure

Primary benefits Primary benefits


Performance Complex calculations with R code
Bringing compute to the data Results can be called from applications
Performance & flexibility
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Basic workflow Import Data

Preprocess

Pick ML
Split Data
Algorithm

Train Model

Score Model
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Import data

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Basic workflow Import Data

Preprocess

Pick ML
Split Data
Algorithm

Train Model

Score Model
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Preprocess data

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Basic workflow Import Data

Preprocess

Pick ML
Split Data
Algorithm

Train Model

Score Model
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Split data
Original dataset

60% - 80% 40% - 20%


Training Testing

Training | Validation Testing

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Basic workflow Import Data

Preprocess

Pick ML
Split Data
Algorithm

Train Model

Score Model
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Pick an algorithm

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Basic workflow Import Data

Preprocess

Pick ML
Split Data
Algorithm

Train Model

Score Model
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Basic workflow Import Data

Preprocess

Pick ML
Split Data
Algorithm

Train Model

Score Model
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Evaluation!
https://towardsdatascience.com/metrics-to-evaluate-
your-machine-learning-algorithm-f10ba6e38234

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Basic workflow

Predictive
Experiment

Training experiment

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Demo

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
How to
Get Started
Microsoft Team Data Science Process
• Data science lifecycle definition
• Standardized project structure
• Infrastructure and resources for data science
projects
• Tools and utilities for project execution

@JanMulkens
http://aka.ms/tdsp
Un événement #SQLSatParis – SQLSaturday Paris 2017
Microsoft Team Data Science Process

@JanMulkens
http://aka.ms/tdsp
Un événement #SQLSatParis – SQLSaturday Paris 2017
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Business Understanding

@JanMulkens
http://aka.ms/tdsp
Un événement #SQLSatParis – SQLSaturday Paris 2017
Business Understanding
Project Lead Project Manager

Create Template Create Project


Repository Charter

Project Charter

@JanMulkens
http://aka.ms/tdsp
Un événement #SQLSatParis – SQLSaturday Paris 2017
Data Acquisition & Understanding

@JanMulkens
http://aka.ms/tdsp
Un événement #SQLSatParis – SQLSaturday Paris 2017
Data Acquisition & Understanding
Solution Architect Project Lead Data Scientist

Design Solution Provision Data Provision Compute Data Ingest &


Architecture Infrastructure Assets Explore

Solution
Data Summary
Architecture
Report
Diagram

@JanMulkens
http://aka.ms/tdsp
Un événement #SQLSatParis – SQLSaturday Paris 2017
Modeling

@JanMulkens
http://aka.ms/tdsp
Un événement #SQLSatParis – SQLSaturday Paris 2017
Modeling Data Scientist

Feature Model
Engineering Development

Model Report
Solution Architect

Develop Data
Pipeline

@JanMulkens
http://aka.ms/tdsp
Un événement #SQLSatParis – SQLSaturday Paris 2017
Deployment

@JanMulkens
http://aka.ms/tdsp
Un événement #SQLSatParis – SQLSaturday Paris 2017
Deployment Data Scientist

Deploy Scoring Monitor Health &


Process Metrics

Dashboard
Solution Architect

Deploy Pipeline

@JanMulkens
http://aka.ms/tdsp
Un événement #SQLSatParis – SQLSaturday Paris 2017
Acceptance

@JanMulkens
http://aka.ms/tdsp
Un événement #SQLSatParis – SQLSaturday Paris 2017
Acceptance
Project Lead Project Manager

Transition to
Checkpoint Project Finalize
Production Support
Documentation

Data Scientist

Project Final Report


Check in Final Decomission
Artifacts Compute Assets

@JanMulkens
http://aka.ms/tdsp
Un événement #SQLSatParis – SQLSaturday Paris 2017
How to advance
What’s your ML test score?
A rubric for ML production systems
• https://ai.google/research/pubs/pub45742

Lessons learned turning machine learning models into real


products and services
• https://www.oreilly.com/ideas/lessons-learned-turning-
machine-learning-models-into-real-products-and-
services @JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Start now
Get into Data Science in only 25 minutes
(includes transcripts so you could be even faster!)
https://docs.microsoft.com/en-us/azure/machine-
learning/studio/data-science-for-beginners-the-5-
questions-data-science-answers

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Start now
Let the Data Science Process be your guide
https://aka.ms/tdsp

Start your deep dive in an awesome course


https://academy.microsoft.com

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Start now
Get the project templates
https://github.com/Azure/Azure-TDSP-ProjectTemplate

Get easy to use utilities to do the heavy lifting


https://github.com/Azure/Azure-TDSP-Utilities
IDEAR – Interactive Data Exploration, Analysis, and Reporting
AMAR – Automated Modeling and Reporting

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Start now
Start learning R
• https://www.datacamp.com/courses/free-
introduction-to-r
Or learn Python
• https://www.edx.org/course/introduction-python-
absolute-beginner-microsoft-dev236x
• https://www.edx.org/course/introduction-python-
fundamentals-microsoft-dev274x
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Reading Material

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Reading material

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Blogs to read
• Ginger Grant (@DesertIsleSQL)
• Stairway to Machine Learning Services
http://www.sqlservercentral.com/stairway/171757/

• Niels Berglund (@nielsberglund)


• A lot & great content on SQL Server ML Services
http://www.nielsberglund.com
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Books to read
• Leila Etaati (@leila_etaati)
• (December 2018)
Machine Learning with Microsoft Technologies:
Selecting the Right Architecture and Tools for Your
Project

@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Viewing Material (old to new)
Machine Learning with R and SQL Server 2016
• https://www.youtube.com/watch?v=LvQYiVj4TNQ

In database Machine Learning with Python in SQL Server


• https://www.youtube.com/watch?v=gSA_pSQod-s

How to build machine learning applications using R and Python in SQL Server 2017
• https://www.youtube.com/watch?v=b9Xhvyfx6qk

Performance patterns for machine learning services in SQL Server


• https://www.youtube.com/watch?v=QTxGzspOc5o

Machine Learning Services in SQL Server


• https://www.youtube.com/watch?v=33CNYhS8ByQ

How to use R, Python, and machine learning with Microsoft SQL Server 2017
• https://www.youtube.com/watch?v=6K9IY_4ghn0
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
@JanMulkens
Un événement #SQLSatParis – SQLSaturday Paris 2017
Contact Jan

Jan@JanMulkens .be

Un événement
Speaker help (FREE!) www.SpeakingMentors.com @JanMulkens
#SQLSatParis – SQLSaturday Paris 2017
Thank you Sponsors

You are Community

You might also like