Predictive Maintenance

From Wine Cellars To 40,000ft

Paul-Louis Pröve – Data Scientist
Rotkäppchen-Mumm
Freyburg, Sachsen-Anhalt
288 Fermentation Tanks with
analog Thermometers and
Barometers
The Idea
Using a computer vision enabled app to read the instruments

Steps

1. Employee takes a photo of the instrument

2. Computer Vision algorithm reads the temperature
5. and pressure
1.
3. Data is send to the server
4. Data is saved to a database
5. Employees can access the data
2. 4.
3.

meetup.ai - Predictive Maintenance

Thermometer Algorithm
1. QR detector reads the tank ID
2. A median filter smoothens the image
while keeping edges sharp
3. The Canny Edge Detector finds and
extracts instrument edges
4. Using the blue color channel we can
detect the fluid level
5. Based on these values we can roughly
estimate the temperature (+/- 4°)
6. Search for the first thick line above the
blue fluid
7. Decide whether it‘s the 20° oder 30°
mark
8. Calculate the temperature based on the
offset (0.6 degrees MAE)

meetup.ai - Predictive Maintenance

Thermometer Algorithm
Examples for perspective challenges and hard to read images

meetup.ai - Predictive Maintenance

Barometer Algorithm
1. QR detector reads the tank ID
2. A median filter smoothens the image
while keeping edges sharp
3. Circle Hough Transform (CHT) finds the
largest circle in the image
4. Another CHT finds the exact center of
the instrument
5. Transformation to polar coordinates
6. The darkest row indicates the position
of the pointer
7. The row with the highest level of red
indicates the upper calibration mark
8. Based on the angle we can calculate
the pressure (0.13 bar MAE)

meetup.ai - Predictive Maintenance

Barometer Algorithm
Examples for blurs, reflections and sideways shifted images

meetup.ai - Predictive Maintenance

What does broken sound like?

meetup.ai - Predictive Maintenance

Audio Anomaly Detection
Learn what a healthy machine sounds like in order to know when it doesn‘t

Raw Audio Feature Model Anomaly

Signal Transformation Training Predictions

meetup.ai - Predictive Maintenance

Feature Extraction
We extracted tabular features as well as mel-spectrograms for convolutional models

Audio Features

§ Binned Frequencies
§ Mel Spec Coefficients
§ Spectral Spread
§ Chroma
§ Tonnetz

à 284 Features

meetup.ai - Predictive Maintenance

Robust Covariance for Anomaly Detection
What‘s wrong with using the Euclidean distance?

§ We could define an anomaly as a point that is far

away from the mean of our data
§ However, the Eucildean distance doesn‘t take into
account how the data is distributed
§ All 4 points on the right have the same Euclidean
distance from the mean of the data
§ The Mahalanobis distance is the Euclidean
distance scaled by the covariance matrix
§ !"#$#%#&'()* + = + − . ∑01(+ − .)
§ The distance is measured in standard deviations
from the data distributions
§ Simple and intuitive

meetup.ai - Predictive Maintenance

Isolation Forest for Anomaly Detection
How long does it take to isolate a sample from the training data?

1. Build a binary tree by random partitioning 4. At inference, sort new samples into the trees
2. Stop when every data point is isolated a. Many splits à normal sample
3. Do this multiple times: tree à forest b. Few splits à abnormal sample

meetup.ai - Predictive Maintenance

Autoencoders for Anomaly Detection
What happens if the decoder gets a latent vector from a different distribution?

§ Train a convolutional autoencoder to

process normal audio spectrograms
§ Abnormal inputs will have a far
larger reconstruction loss than the
normal inputs

§ Sounds great in theory

§ Didn‘t work well in practice

meetup.ai - Predictive Maintenance

GANs for Anomaly Detection
Can we turn anomaly detection into a binary classification by adverserial example?

• Generator is trained to generate

normal data samples Features/
Data
Embedding
• Discriminator is trained to
distinguish real normal data x from
generated normal data G(z) z G G(z)
• After the training the discriminator
will predict abnormal samples as D P(y)
generated
x

• Very slow to train

• Good inference speed
G: Generator
• Good performance
D: Discriminator

*Zenati, Houssam, et al., arXiv preprint arXiv:1802.06222 (2018)

meetup.ai - Predictive Maintenance

Thank You
Paul-Louis Pröve – Data Scientist
paul-louis.proeve@lhind.dlh.de