SEMINAR REPORT ON

Machine Learning in Chemical Engineering

BACHELOR OF TECHNOLOGY
(CHEMICAL ENGINEERING)

Tushar Goel
2018chb1061

Department of Chemical Engineering

Indian Institute of Technology Ropar

1|Page
 ABSTRACT

This report aims at introducing the concept of machine learning to the readers and
building a sufficient level of understanding of the subject so that they can easily
continue their learning on the same. The report starts with a brief introduction to
machine learning explaining its significance especially for Chemical Engineering
people. Some basic and frequently used machine learning algorithms are also
discussed, the mathematical concepts of these algorithms are not covered in this
report because the objective is only to provide enough explanation so that one can
apply the algorithm in his/her project or any application without going deep into the
details of the mathematics involved in these. The intended audience for this report is
the Chemical Engineering folks or any other segment of population who is interested
in machine learning and is just a beginner in this field. To show the magic of
machine learning, some chemical engineering related industrial applications are also
discussed, this will help in better understanding of machine learning and its
importance in industries and various other areas.

2|Page
 TABLE OF CONTENTS

ABSTRACT ---------------------------------------------------------------------------------------2

LIST OF FIGURES -------------------------------------------------------------------------------4

LIST OF EQUATIONS --------------------------------------------------------------------------4

1. Introduction -------------------------------------------------------------------------------------5

1.1 What is Machine Learning? -------------------------------------5

2. Machine Learning and Deep Learning Algorithms ----------------------------------------6

2.1 Linear and Non-Linear Regression -----------------------------7

2.2 Support Vector Machines (SVMs) ------------------------------9
2.3 Neural Networks --------------------------------------------------10

3. Machine Learning in Chemical Engineering and Industrial Applications ---------------11

3.1 Support Vector Machines for Quality Monitoring in IMP --11

3.2 Dissolved Gas Analysis ------------------------------------------13
3.3 Big Data in Oil and Gas Pipelines ------------------------------15

4. Conclusion ----------------------------------------------------------------------------------------16

REFRENCES ----------------------------------------------------------------------------------------17

3|Page
 LIST OF FIGURES

Figure 2.1 Linear Regression Example ------------------------------------------------8

Figure 2.2 Non-Linear Regression Example ------------------------------------------8

Figure 2.3 Support Vector Machines Decision Hyperplane ------------------------9

Figure 2.4 Structure of a Neural Network --------------------------------------------10

Figure 3.1 Injection Molding Machine -----------------------------------------------12

Figure 3.2 Possible Defects in Injection Molding Process -------------------------13

Figure 3.3 Concentration of gases emerged during Transformer Operation -----14

Figure 3.4 Magnetic Flux Leakage ----------------------------------------------------16

LIST OF EQUATIONS

Equation (1) Linear Regression prediction function ----------------------------------7

Equation (2) Cost Function for Linear Regression ------------------------------------7

Equation (3) Polynomial Regression 2nd Degree Prediction Function -------------8

4|Page
1. Introduction

Machine Learning (ML) is one of the hottest topics in today's technology driven
world. ML is basically a prediction tool which can automatically learn from the data
and based on its learnings make predictions for the unseen data. Thus, the input being
mainly, the data. As almost every process is undergoing digitization these days, the
data generated in these production processes is ever increasing. This calls for the
effective use of the huge amount of data that is being generated in most of the
services and production sector. ML steps in here as it can very efficiently harness the
power of data and use it to solve various unsolved problems in many sectors.

The purpose of this report is to introduce the idea of machine learning and the basic
and the most frequently used ML algorithms in various practical problems to mainly
Non-Computer Science background folks especially Chemical Engineering
undergraduates and graduates. Sometimes, the Non-Computer Science background
academicians miss out certain golden opportunities due to the lack of knowledge in
ML. This report is meant for these types of people to give them a head start in this
vast field so that, the next time an opportunity knocks their door, they are able to
identify it and take its full advantage. ML can be applied to solve or optimize almost
all the problems which have some meaningful data associated with it. So, in order to
exploit its full potential, one needs to understand the key concepts involved in
machine learning. This report also includes some practical applications of ML which
are used in various industries to solve many different kinds of problems. The purpose
of these applications is to inspire the readers and to convey the idea that ML is not
limited to one single domain but has diversified applications and it is very much
important for all types of technology folks to have a basic understanding of ML.

1.1 What is Machine Learning?

Machine Learning is a highly sophisticated technique which can make

predictions for some unknown data when fed with some known data, the
process behind this involves certain mathematical models and algorithms.
The only input required in this technique is data, if one has some
meaningful data then he/she can obtain the desired results through ML.
This is the main reason why data is being given so much importance these
days, and in most of the production and service areas data generation and
handling is done with utmost care. Before the inception of machine
learning, it was very difficult to analyze the large datasets and to identify
the hidden patterns in the data as all these were done manually. With the
development of ML and increasing computational resources, all these tasks
became fairly easy as the data patterns could now be learned automatically
in very less time and the results were also highly accurate as compared to
5|Page
before. Machine learning has found its applications in several areas like
speech recognition, computer vision and recommendation systems etc. and
is continuing to solve the day to day problems faced in various sectors and
making the existing systems more efficient.

2. Machine Learning and Deep Learning

Algorithms

Machine learning algorithms can be classified on the basis of various grounds

but here we will see only two bases based on which these are classified.
1. On the basis of the type of data or dataset: In this classification, the
machine learning algorithms can be classified into two categories which are
supervised and unsupervised. In supervised algorithms, the data includes both
the feature variables as well as the corresponding labels or target variables. For
instance, we have gender detection problem in which we are given a dataset
which includes two feature variables: - height and weight, and also an output
variable i.e. gender (male or female), based on this data we have to predict the
gender of some unseen data. On the other hand, unsupervised algorithms only
have feature variables with no labels. For instance, the same gender detection
problem but this time the data only has two feature variables (height and
weight) but no corresponding label(gender).
2. On the basis of the output variable type: This is a sub classification under
the class of supervised algorithms in which we have an output variable. In this
categorization, the machine learning algorithms are divided into two categories:
- Regression and Classification algorithms. In regression, the output variable is
a real, continuous value. For example, the prediction of house price based on
certain features of the house (like the number of bedrooms, bathrooms etc), in
this case, the price variable belongs to a real and continuous number. On the
other hand, in classification problems the output variable can only assume some
fixed number of values which are called classes. For example, the same gender
detection problem stated above as the gender variable can only have two values
(male or female).
In this report, we will be concentrating only on the supervised machine learning
algorithms because in most of the chemical industries and other chemical
engineering related applications, the labels are also extracted along with the
other data.
Now we will see some machine learning algorithms which are widely used in
6|Page
 solving the chemical engineering machine learning problems.

2.1 Linear and Non-Linear Regression:

Linear Regression is the simplest machine learning algorithms of all. In this
algorithm our objective is to find an approximate function which could map the
features or the input variables to the output variable. This process of finding the
function based on some data is known as training or fitting to the data. In linear
regression, this function is a linear combination of weights and feature variables
and there is also a bias term. Weights and bias are the parameters of this model
which need to be determined in order to know the function completely. For
example, we have a problem in which ‘n’ training examples are given with each
example having ‘m’ features or dimensions. The feature variables are
represented as X 1 , X 2 , … , X k ,… , X m where X k(K is any number between 1 to m) is
the K th dimension or feature and the output label which is the actual value of the
target variable is denoted by ‘Y’ while the predictions of the output label made
by the model are denoted by ‘Ỹ’. Then weights are represented as
W 1 , W 2 ,… , W k , … , W m where W k is the weight corresponding to the K th dimension
while ‘b’ is the bias also known as the intercept. The mathematical function
used to predict the output label is the following Eq (1).

Ỹ =¿ b+W 1 X 1+W 2 X 2+ …+W m X m (1)

i=n
J ( W , b )=∑ (Y −Ỹ )2 (2)
i=1

Where, J (W , b) is the cost function and W is the column vector consisting of all the
weights.
The function in Eq. (1) will be used to make the predictions, but before that we
have to determine the unknown parameters (weights and bias). In this process, we
need to define another function called cost function Eq. (2) which is a function of
the weights and bias and is a measure of the error between the predictions and the
actual value of the output variable. So, the lower the error the better will be the
predictions. Therefore, we need to find the minima of the cost function and the
corresponding weights and bias and those will be the desired value of our model
parameters. This is done through an algorithm called ‘Gradient Descent
Algorithm’. But we will not go through the details of the algorithm, as our
objective is only to understand their implementation, and the linear regression can
be easily implemented through pre-defined libraries in python with just a few lines
of code.
7|Page
Similarly, a non-linear function like a polynomial function instead of a linear
function can also be fitted or trained on the data if the linear function is not able to
produce reasonable predictions with low error. This non-linear model is called Non

Figure 2.1 Linear Regression Example: As we can see in the red line represents the predictions while the blue points
represent the actual value.

Linear Regression. For example, a polynomial function of degree 2 is to be

fitted and we have only two feature variables ( X 1 ∧X 2 ¿ and we have weights and
bias similar to those of linear regression, then our mathematical model takes the
form Eq(3).
2 2
Ỹ =b+W 1 X 1 +W 2 X 2 +W 3 X 1 +W 4 X 2+W 5 X 1 X 2 (3)

It can also be easily implemented through sklearn library in python.

Figure 2.2 Non-Linear Regression: As we see the red dotted line is the non-linear fitted function which represents the
predictions while the points represent the actual value

2.2 Support Vector Machines (SVMs)

8|Page
It is the most widely used classification algorithm in many industrial problems
where a class has to be predicted based on some sensor produced data in
industries.

Figure 2.3 Support Vector Machine

As we can see in Fig. 2.3, there are two classes represented in red and blue. In
every classification algorithm, the goal is to find a decision boundary which
can efficiently separate the classes but this decision boundary can have infinite
orientations in space as we rotate it about different axis (see Fig. 2.3). This
problem is solved by support vector machine as it finds an optimal decision
boundary from among all the sub-optimal solutions. Instead of a thin line type
decision boundary, it finds a thick decision boundary (in case of 2D data while
a thick hyperplane in higher dimensions) which has margin on its both sides.
The objective of the algorithm is to maximize the width of this margin, thus
finds an optimal hyperplane (as there will be only one hyperplane with
maximum length of margin). In Fig. 2.3, the smooth line is called the center of
the optimal hyperplane while the dotted lines are the margin boundaries of the
hyperplane. The algorithm classifies all the points which lie below the center of
the hyperplane as one class while all the points which lie above as the other
class.
It can be implemented through the ‘svm’ class of the ‘sklearn’ library. But we
need to specify the values of two parameters: -

1. C: The parameter ‘C’ acts as a regularization parameter which helps in

keeping a balance between overfitting and underfitting. Overfitting occurs
when the model does not generalize well to the unseen data while underfitting
9|Page
happens when the model is unable to fit the training data well to the required
accuracy.

2. Kernel Function: This function is mainly used when we are dealing with a
data which is not linearly separable and requires non-linear decision boundary.
Some of the kernel functions are linear, polynomial and radial basis function.

2.3 Neural Networks

Fig 2.4 Structure of a Neural Network

The latest innovation in the field of machine learning is Deep Learning which is a
sub set of machine learning and provides new and more efficient methods of
prediction. Before, the inception of Deep Learning, machine learning techniques
were limited in terms of power of prediction as these models could not take the
advantage of the humungous data generated in most of the sectors. Therefore, more
sophisticated models were required to handle the large amount of data and make
more accurate predictions. This was made possible with the development of Deep
Learning.

Deep Learning is all about Neural Networks. Neural Networks are very similar to the
machine learning models, the difference arises due to their structure, in case of
machine learning there is an input layer which consists of the feature variables and an
output layer which constitutes of the final predictions of the output variable. On the
other hand, in Neural Networks in addition to the input and output layers, there is one
10 | P a g e
more type of layers which appear between the input and output layers, these are
called hidden layers (see Fig 2.4). The number of hidden layers can vary from 0 to
any finite number depending upon the complexity of the prediction problem. The
circle in the hidden layer is called a node and each node, just like the output variable
make predictions based on the features of the previous layer and these predictions
(values in the nodes of the hidden layer) of the current layer are further used to
predict the values for the nodes in the following layer and in this way the process of
prediction continues until it reaches the output layer. Each hidden layer and the
output layer have some parameters (weight and bias matrices) associated with them
which needs to be learned or trained which is the objective of the model.

Neural Networks can solve regression as well as classification problems. These are
largely used when huge amount of data is available as they are capable of learning
any pattern or relationship between variables with high accuracy, the only condition
being large data. This is the reason why neural networks are gaining so much
popularity and everyone is shifting to neural networks in order to exploit the huge
amount of data. Neural Networks can be easily implemented through TensorFlow or
Keras library in python.

3. Machine Learning in Chemical Engineering and

Industrial Applications

This section will reveal how the various machine learning algorithms are adapted in a
way so as to fit in the numerous chemical engineering and industrial applications.
The problems faced in the industries and various other research areas which were
previously practically unsolvable due to time and cost constraints are thoroughly
discussed and how the integration of machine learning in these fields has led to the
solution of these problems by increasing the efficiency in terms of time and cost.
This is just a small representation of the huge world of possibilities and opportunities
that the machine learning can generate if proper research and knowledge in the field
of machine learning is integrated with the various other areas.

3.1 Support Vector Machines for Quality Monitoring in a Plastic Injection

Molding Process

Injection Molding Process (IMP) is one of the most important plastic manufacturing
techniques and widely used extrusion processes. In the IMP, a thermoplastic, in the
form of granules, passes from a feed hopper into a barrel where it is heated and
11 | P a g e
melted [1]. It is then injected into a cold mold that is clamped tightly closed. When
enough time has elapsed, the plastic returns to a solid state, the mold opens and the
produced part is extracted. All the mold details are, by this way, reproduced in the
finished part [1].

From the description, this process might sound quite simple but in practice, at the
manufacturing scale, loads of problems are faced in the process of maintaining the

Figure 3.1 Injection Molding Machine

required final product quality. There are a large number of process parameters like
temperature, pressure, flow rate and many more, which controls the quality of
molded parts and even a slight variation in one of these parameters can result in
considerable variation in the quality. The relationship between the parameters and the
product quality is highly non-linear and complex and is therefore, very challenging to
discover. Moreover, this relationship is not consistent and also there are fluctuations
in parameters like temperature, pressure fluctuations which makes it even more
difficult to control process. In order to detect these quality variations, a system is
required which will help in predicting and suggesting the process parameters for the
target product quality.

Currently, this problem of monitoring product quality is mainly solved by statistical

analysis of the related data (parameters’ data and the corresponding parts quality
data). The various parameters are monitored by an operator who based on intuition
controls the set points of various parameters manually. This hit and trial procedure
leads to erroneous results in terms of product quality. Thus, a more reliable method is
required which could provide a better control over the product quality.

A more sophisticated solution can be achieved through the machine learning model
Support Vector Machines (SVMs), which can better analyze the data and predicts the
defects in the product based on the parameters which in turn helps in determining the
appropriate set points for the various parameters. The data employed in the model is
generated through the various sensors fitted at different locations. This sensor data
12 | P a g e
includes twenty-six process parameters but for the model input features only six are
selected which include cycle time, metering time, injection time, barrel temperature
before nozzle, cushion, and injection velocity. The output of the model is a measure
of product quality which is represented through six quality variables namely streaks,
stains, burn marks, edges, unfilled parts and warpage (see Fig 3.2) which have been
found to be representative of the range of defects that can occur so far in the plant
[1]. There is one logical assumption that only one defect will occur out of the six
defects

Figure 3.2 Variety of possible defects in Injection Molding Process

for a single run through the injection molding process. Thus, this has become a
Multiclass Classification Problem which can be solved through SVMs with very high
classification accuracy (see Section 2.2).

3.2 Dissolved Gas Analysis

Dissolved Gas Analysis (DGA) is a technique for the assessment of electrical

transformers which is employed in various industries for over more than 30 years.
All the transformers during normal operation generate some amount of gases like
hydrogen, methane, ethane, ethylene, acetylene, carbon monoxide, and carbon
dioxide which gets dissolved in transformer oil. But when the transformers become
faulty, these gas concentrations increase in comparison to their normal counterparts.
DGA is involved in measuring these gas concentrations and analyzing them in order
to detect some anomaly in the transformer's functioning. However, DGA has been in
deployment for several decades, there is no unique and universally accepted method
of analyzing and interpreting the DGA concentrations. This is because, the amount of
gas produced not only depends on the presence of faults but also has a strong and
13 | P a g e
complex dependence on various other factors like age, loading, thermal history etc.

Fig 3.3 Concentration of gases emerged during transformer operation

The most promising method till now are the supervised machine learning models
which uses the labelled DGA data to predict the faults in the transformer. So, there
will be seven feature variables in our machine learning model which are the seven
gas concentrations mentioned above. Before, using these gas concentrations as
feature variables, these are first preprocessed by log transformation and
standardization (mean=0 and standard deviation=1). The time is also noted for each
measurement but not used as a feature variable. This machine learning problem can
be formulated in two ways which are classification and regression.

In the classification formulation, it is assumed that the transformer will be either in

"normal" condition or "faulty" condition. For the normal case, the label y is assigned
the value "1" while in the case of faulty, it is assigned "0". The convention used in
labelling the transformer as normal or faulty is that, if the DGA measurement X i was
taken at least five years prior to failure then the corresponding label is normal and
faulty otherwise.

In the regression formulation, in addition to the time stamps for each DGA
measurement we also have the information about time of failure. We used this
information to obtain more informative labels Y i  [0,1], where Y i=0 would mean
“bound to fail”, Y i=1 would mean “should not fail in the foreseeable future”, and
values between those two extremes would quantify the risk of failure [2]. For
calculating the values of labels, first, the date at which the measurement was
recorded is subtracted from the date at which the transformer has failed. The result
we get is called "Time to Failure" (TTF). The readings which were taken when the
transformer has already failed are given the label "0", while those readings in which
the transformer has not failed yet, the labels are assigned value close to 1. These TTF
values are then sorted in order to compute the empirical cumulated distribution
function (CFD). TTFs of zero would correspond to a CDF of zero, while very long
TTFs would asymptotically converge to a CDF of one. Thus, the CDF value
corresponding to each TTF value becomes our label for the regression model.

Thus, we have formulated our problem of detecting faults in the transformer through
14 | P a g e
classification as well as regression frameworks. The classification problem can be
solved using K-Nearest Neighbors (KNN) or Support Vector Machines (SVMs)
while the regression can be solved by using Neural Networks.

3.3 Big Data in Oil and Gas Pipelines

The natural gas and crude oil produced in industries is transported through long
metallic pipes. Due to the extreme weather conditions, these pipes are prone to many
different types of defects like corrosion, cracks, dents etc. These pipeline defects can
result in huge financial losses, damage to the environment, and loss of life [3].
Therefore, it is very important to devise an efficient and effective method which
would locate these defects and also identify the defect type (corrosion, cracks or
dents etc.).
One such method which is largely employed in most of the industries is the use of
Magnetic Flux Leakage (MFL) signals. In this method, magnetic field is produced
and allowed to pass through the pipelines, in this process, some of the magnetic flux
leaks out and gets detected by the MFL sensors which are installed along the length
of the pipeline at equal distances. When a defect is encountered along the pipe, the
magnetic flux leakage increases and thus, the MFL sensors record a high amplitude
at these locations which indicates the presence of defects (see Fig 3.3). Also,
different types of defects will have certain characteristic patterns in the graph plotted
between the Magnetic Flux Leakage components ( Bx , B y and Bz ) and the length of the
pipe. Thus, studying these graphs can help in identifying the type of defect present.
These two problems (defect detection and defect type identification) can be solved
through statistical techniques and machine learning models as well.

But there is one more problem of defect depth estimation which is quite difficult to
solve manually. The MFL signal increases in magnitude when the defect depth
increases. But this relationship is very complex and cannot be determined
analytically. Therefore, to capture this relationship, machine learning models are
used. The training data for the model is generated such that for every training
example, the set of MFL sensor readings represent only one particular measurement
of the defect depth. These MFL sensor recordings will serve as the input features for
the model. But as the number of sensors is very large, the dimensionality of the
features is reduced first through dimensionality reduction techniques. The output
variable is the defect depth which is a real number. Thus, we have collected the
features and the output variable for training our model. So, now we can easily use
Neural Networks to solve this prediction problem.

15 | P a g e
 Fig 3.4 Manetic Flux Leakage in the presence of defects like corrosion

4. Conclusion

We have explored quite a few applications of machine learning in chemical

engineering related industries which have substantially reduced the human effort
and helped save a lot of time and money and made possible and feasible which was
considered impossible and impractical before. These examples (see section 3) were
chosen in such a way so that these serve as an eye-opener and inspire the chemical
engineering people to at least learn the basics of machine learning so that whenever
there is a possibility of machine learning easing their task and making their project
more efficient, they can detect this possibility and then, can take the appropriate
measures towards integrating their project with ML. We have also learned a few
machine learning algorithms like Linear and Non-Linear Regression, Support
Vector Machines and Neural Networks, and all these can be easily applied with the
help of pre-defined libraries in python with just a few lines of code. The most
important thing in ML is data as this the only input required in machine learning.
Therefore, the first and the most important step in applying ML is to clean the
available data and extract the useful features from it that will be fed into the ML
model as the input.

These applications of ML in chemical engineering are a mere beginning, there’s a

lot more to come, only if we, chemical engineers instead of adopting a
conventional thinking, starts to incorporate machine learning in our daily
curriculum. The integration of chemical engineering and machine learning is a
revolutionary step towards achieving the unimaginable targets and solutions to
unsolvable problems in chemical engineering.

16 | P a g e
 REFERENCES

[1] Bernardete Ribeiro, Member, IEEE, “Support Vector Machines for Quality
Monitoring in a Plastic Injection Molding Process,” vol. 35, No. 3, AUGUST 2005

[2] Piotr Mirowski, Member, IEEE, and Yann LeCun, “Statistical Machine Learning
and Dissolved Gas Analysis: A Review,” vol.27, No.4, OCTOBER 2012

[3] Abduljalil Mohamed, Mohamed Salah Hamdi Information Systems Department

ABMMC Doha, Qatar and Sofiene Tahar Electrical and Computer Engineering
Department line Concordia University Montreal, Quebec, Canada, “A Machine
Learning Approach for Big Data in Oil and Gas Pipelines,” 2015 3rd International
Conference on Future Internet of Things and Cloud

[4] Pedro Larrañaga David Atienza Javier Diaz-Rozo Alberto Ogbechie Carlos
Puerto-Santana Concha Bielza, “Industrial Applications of Machine Learning,”
Chapman & Hall/CRC Data Mining and Knowledge Series

[5] Pankaj Mehtaa, Marin Bukovb, Ching-Hao Wanga, Alexandre G.R. Daya, Clint
Richardsona, Charles K. Fisherc, David J. Schwabd, “A high-bias, low-variance
introduction to Machine Learning for physicists,” Physics Report (810) 2019

17 | P a g e