materials

Article
A Deep Neural Network for Accurate and Robust
Prediction of the Glass Transition Temperature of
Polyhydroxyalkanoate Homo- and Copolymers
Zhuoying Jiang 1,† , Jiajie Hu 2,† , Babetta L. Marrone 3 , Ghanshyam Pilania 4
and Xiong (Bill) Yu 1,2, *
1 Department of Civil and Environmental Engineering, Case Western Reserve University,
Cleveland, OH 44106, USA; zxj45@case.edu
2 Departments of Computer and Data Sciences, and Electrical, Computer, and Systems Engineering,
Case Western Reserve University, 2104 Adelbert Road, Cleveland, OH 44106, USA; jxj919@case.edu
3 Bioscience Division, Los Alamos National Laboratory, Los Alamos, NM 87545, USA; blm@lanl.gov
4 Materials Science and Technology Division, Los Alamos National Laboratory, Los Alamos, NM 87545, USA;
gpilania@lanl.gov
* Correspondence: xxy21@case.edu
† These two authors contributed equally to this work.




Received: 15 November 2020; Accepted: 9 December 2020; Published: 14 December 2020


Abstract: The purpose of this study was to develop a data-driven machine learning model to predict the
performance properties of polyhydroxyalkanoates (PHAs), a group of biosourced polyesters featuring
excellent performance, to guide future design and synthesis experiments. A deep neural network (DNN)
machine learning model was built for predicting the glass transition temperature, Tg , of PHA homo-
and copolymers. Molecular fingerprints were used to capture the structural and atomic information of
PHA monomers. The other input variables included the molecular weight, the polydispersity index,
and the percentage of each monomer in the homo- and copolymers. The results indicate that the DNN
model achieves high accuracy in estimation of the glass transition temperature of PHAs. In addition,
the symmetry of the DNN model is ensured by incorporating symmetry data in the training process.
The DNN model achieved better performance than the support vector machine (SVD), a nonlinear ML
model and least absolute shrinkage and selection operator (LASSO), a sparse linear regression model.
The relative importance of factors affecting the DNN model prediction were analyzed. Sensitivity of
the DNN model, including strategies to deal with missing data, were also investigated. Compared
with commonly used machine learning models incorporating quantitative structure–property (QSPR)
relationships, it does not require an explicit descriptor selection step but shows a comparable performance.
The machine learning model framework can be readily extended to predict other properties.

Keywords: molecular fingerprint; deep neural network; glass transition temperature; copolymers;
quantitative structure–property relationship (QSPR)

1. Introduction
With the rapid development of civilization, the demand for plastic products has increased
significantly. However, plastic waste has become a worldwide environmental issue because plastic
materials are durable and difficult to degrade [1–3]. As a result, bioplastics, which are made from
biological substances, have attracted significant interest as an alternative for conventional plastics made
from petroleum [4,5]. Polyhydroxyalkanoate (PHA) polymers are a type of green bioplastics that are
produced naturally by bacteria and utilized by the cells for energy and carbon storage [6]. PHAs have
been used commercially for a wide range of applications due to their excellent versatility, biocompatibility,

Materials 2020, 13, 5701; doi:10.3390/ma13245701 www.mdpi.com/journal/materials

Materials 2020, 13, 5701 2 of 15

and biodegradability Their application areas range from food packaging and agricultural films to
biomedical fields including use as drug carriers, tissue engineering scaffolds, etc. [7–9].
To fulfill the requirements for various applications, fabrication of PHAs with a variety of properties,
such as physical, thermal, and mechanical properties, becomes a new challenge. PHAs with different
properties are achieved by tuning the monomers and configurations [10–12]. It is known that
approximately 150 types of PHA monomers can be used to constitute different PHA copolymers [13].
These combinations of single monomers can provide PHA polymers with diverse and flexible properties.
In addition, the polymer properties can also be tuned by modifying chemical configurations, i.e., the size,
shape, and branching of polymers [14,15]. Therefore, the design and synthesis of novel PHAs typically
need trial and error experimentation to obtain the structure–property–performance relationships.
This traditional approach requires enormous lab and labor investments, and progress is typically slow.
As data science has rapidly developed in the last decade, machine learning (ML) became a
promising tool for data analytics and predictions. ML is part of data science and it is a technology for
processing a large amount of data. It learns from the existing data, finds data patterns, and provides
solutions [16]. It is faster, cheaper, and more flexible than experimentation. A neural network is an
algorithm of ML. Inspired by the neural network of the human brain, an artificial neural network can
learn from past data and generate a response [17]. The structure of a typical artificial neural network is
usually composed of three essential types of layers, i.e., the input layer, hidden layer(s), and output
layer. They are used for receiving, processing, and exporting information, respectively. Each neuron
is connected with an assigned weight and the weight sum is calculated by a transfer function [18].
Deep learning is a class of ML that allows multiple hidden layers for data processing [19]. A deep
neural network (DNN) combines an artificial neural network with deep learning and is capable of
providing a better solution to problems in cognitive learning such as speech and image recognition [20].
So far, DNN models have been successfully applied to learn and predict a range of properties of diverse
types of materials, including metals, ceramics, and macromolecular materials [21–23].
Due to the increasing demand for plastic materials and rising awareness of the plastic waste
crisis, PHAs have been intensively explored since their first discovery in 1926 [24]. With tremendous
historical experimental efforts focused on PHA synthesis and characterization, a large amount of data
is accessible from published sources. Therefore, this allows data to be extracted from the literature and
used to build a data-based ML model to investigate the connections between structures and properties
of PHAs and to extract design rules and useful chemical trends.
This study is a follow-up to [25], in which an ML model incorporating descriptors extracted
from the quantitative structure–property relationship (QSPR) for glass transition temperature (Tg )
prediction was used. It describes the development of a DNN-based model for Tg predictions of PHA
homo- and copolymers. We used Tg as the predicted property of this model because Tg is an important
thermal property for polymers turning from a rigid state to a rubbery state. Tg has been extensively
studied and its value depends on the structure of polymers, e.g., molecular weight and branching
network [26,27]. The database consists of 133 data points obtained from published experiments [25].
The DNN-based model was used to investigate the connections between the inputs, i.e., structure of
PHA polymers, and the output, i.e., Tg . The trained model can provide guidance for material design of
PHA polymers before conducting experiments, which can significantly reduce experimental trials,
and, therefore, save time and expense.
The novelty of this work is the prediction of the glass transition temperature using a DNN model
that is simple to implement and still achieves accurate results. Compared with prior ML model
approach (i.e., [25]), this model does not require the assistance of an expert to obtain the chemical
descriptors from QSPR; instead, it uses standard molecular fingerprints to convert the polymers into
machine-readable language. This is not only much easy to encode the copolymers but also avoids
possible cognitive bias in QSPR by different professionals. This ML model framework can also be
easily extended to a broader range of polymers beyond PHA copolymers.
Materials 2020, 13, x FOR PEER REVIEW 3 of 16

Materials 2020, 13, 5701 3 of 15

2. Materials and Methods

2.1.Materials
2. Data Preparation
and Methods
The experimental measurements of Tg of a variety of polyhydroxyalkanoate-based homo- and
2.1. Data Preparation
copolymers were obtained from the work of Pilania et al. by one of the coauthors [25]. The
experimental databasemeasurements
The experimental of the PHAofTTg g values of a varietyis cooling-rate-dependent
of polyhydroxyalkanoate-based and based homo- on and
the
metastable/dynamic
copolymers were obtainedbehaviorfromofthe amorphous polymer
work of Pilania et al.systems.
by one of Thetheexperimental
coauthors [25]. measurements
The experimental were
made using
database of the differential
PHA Tg values scanning calorimetry (DSC) technique
is cooling-rate-dependent and basedwith relatively slow cooling rates
on the metastable/dynamic
(~10–30 K/min).
behavior The Tg data
of amorphous set is provided
polymer systems. in theexperimental
The Supplementary Materials in Table
measurements were S1. Theusing
made database
the
included 133
differential data points
scanning and each
calorimetry datatechnique
(DSC) source contained five variables,
with relatively slow coolingi.e., chemical identities
rates (~10–30 of
K/min).
monomer A and monomer B, the fractional composition of monomer
The Tg data set is provided in the Supplementary Materials in Table S1. The database included A, the polymer molecular
weight
133 data(MW)
points with
andaeach
unit data
of kgsource
mol−1, and polydispersity
contained index i.e.,
five variables, (PDI). They were
chemical set as the
identities inputs of
of monomer
theand
A DNN model B,
monomer and thethe output was
fractional Tg. To convert
composition the chemical
of monomer A, the expressions
polymer molecular of monomer weight A (MW)
and B
into machine-readable −1 information, the monomer names were
with a unit of kg mol , and polydispersity index (PDI). They were set as the inputs of the DNN first transformed into two-
dimensional
model and the SMILES
outputstrings
was Tg[28].. ToSMILES
convert represents
the chemical theexpressions
simplified molecular
of monomer input
A and lineBentry
into
system; it describesinformation,
machine-readable the unique structure
the monomer and chemical
names species
were firstof atransformed
chemical compound. The chemical
into two-dimensional
identifierstrings
SMILES resolver [28].(https://cactus.nci.nih.gov)
SMILES represents the simplified was used for the conversion.
molecular input line entry The system;
second step was to
it describes
transform
the uniquethe SMILES
structure and strings
chemical intospecies
machine-readable
of a chemical data. Molecular
compound. Thefingerprinting is an approach
chemical identifier resolver
to encode a chemical into a binary digit vector, e.g., {0 0 1 … 1 1 0},
(https://cactus.nci.nih.gov) was used for the conversion. The second step was to transform the SMILESwhich represents molecules in
digital representation
strings into machine-readable readabledata. by a Molecular
computer. fingerprinting
The length of the is an vector is adjustable,
approach to encodewhich could
a chemical
significantly
into a binary affect the reliability
digit vector, e.g., {0of . . .DNN
0 1the model.
1 1 0}, whichTherefore,
representsa molecules
model optimization
in digital was conducted
representation
to select the
readable by best value of The
a computer. the vector
lengthlength
of the as discussed
vector in detail which
is adjustable, in the next
could subsection.
significantlyIn this work,
affect the
the RDKitofpackage
reliability the DNN (http://www.rdkit.org)
model. Therefore, a model was used to convert
optimization wasSMILES
conductedinto molecular
to select the fingerprints
best value
[29].
of theAn example
vector lengthof aasmonomer,
discussedi.e., 3-hydroxy-5-phenyl-pentanoate,
in detail in the next subsection. In this is illustrated
work, thefor this two-step
RDKit package
conversion in Figure 1. For a homopolymer, monomer B was set to an
(http://www.rdkit.org) was used to convert SMILES into molecular fingerprints [29]. An example of a all zero array and the fractional
composition
monomer, i.e.,of3-hydroxy-5-phenyl-pentanoate,
A in the polymer was 100%. All the monomersfor
is illustrated werethissubjected
two-stepto the encoding
conversion process
in Figure 1.
as illustrated in Figure 1 to produce the corresponding digital
For a homopolymer, monomer B was set to an all zero array and the fractional composition of A in the representation via molecular
fingerprints.
polymer was 100%. All the monomers were subjected to the encoding process as illustrated in Figure 1
to produce the corresponding digital representation via molecular fingerprints.

Figure 1.1.An
Figure Anexample
exampleillustration of steps
illustration in encoding
of steps a chemical
in encoding with digital
a chemical with representation to generate
digital representation to
agenerate
molecular fingerprint.
a molecular fingerprint.
2.2. DNN Model Construction
2.2. DNN Model Construction
A DNN model was built to learn and predict the relationship between the input variables and an
output. Five input variables included the molecular fingerprints of monomer A and B, the fractional
Materials 2020, 13, x FOR PEER REVIEW 4 of 16

Materials 2020, 13, 5701 4 of 15

A DNN model was built to learn and predict the relationship between the input variables and
an output. Five input variables included the molecular fingerprints of monomer A and B, the
fractional
composition composition
of monomer of monomer
A, molecularA, molecular
weight, andweight,
PDI.andThePDI. The output
output was Tgwas Tg. Figure
. Figure 2 shows
2 shows the
the schematic
schematic structure
structure of theof the
DNN DNN model.
model. The
The monomersAAand
monomers andBBwere
wereconverted
convertedto to molecular
molecular
fingerprintsbefore
fingerprints beforebeing
beingentered
entered into
into thethe model.
model. TheThe optimal
optimal length
length of molecular
of the the molecular fingerprints
fingerprints was
was determined using a Bayesian optimization scheme. After conversion
determined using a Bayesian optimization scheme. After conversion to the molecular fingerprints, to the molecular
fingerprints,
monomers A monomers
and B are eachA and B are each by
represented represented by aisvector
a vector that thatthe
fed into is fed into the corresponding
corresponding number of
neurons, i.e., as the number of neurons to accept the molecular fingerprint inputs is equalistoequal
number of neurons, i.e., as the number of neurons to accept the molecular fingerprint inputs the
to the of
length length of the molecular
the molecular fingerprintfingerprint
employed. employed. Each
Each of the of the
other other
three three numerical
numerical inputs, i.e.,inputs, i.e.,
fractional
fractional composition
composition of monomer of monomer
A, molecular A, molecular
weight, andweight, and PDI, one
PDI, occupied occupied
neuron.oneTherefore,
neuron. Therefore,
the total
number of neurons in the input layer equals the 2*N+3, where N is the length the
the total number of neurons in the input layer equals the 2*N+3, where N is length
of the of the
molecular
molecular fingerprints.
fingerprints. Rectified
Rectified linear linear unit
unit (ReLU) was (ReLU)
used as was used astransfer
a nonlinear a nonlinear transfer
function function
and the and the
loss function
loss function used was the mean square error (MSE) of the measured glass transition
used was the mean square error (MSE) of the measured glass transition temperature Tg . The learning temperature Tg.
The learning rate was chosen to be a cosine annealing learning rate, which reduces
rate was chosen to be a cosine annealing learning rate, which reduces the learning rate according to a the learning rate
according
half to a half
cosine curve. Thecosine curve. The
optimization optimization
function used wasfunction
Adam [30].usedThewas Adamtechnique
dropout [30]. Thewas dropout
also
technique was also used to avoid overfitting
used to avoid overfitting of the training data. of the training data.

Figure 2. Illustration of the deep neural network (DNN) model structure.

Figure 2. Illustration of the deep neural network (DNN) model structure.
To obtain a reliable DNN model structure, the hyperparameters of the model (i.e., the number of
To obtain
neurons and thea number
reliable DNN model
of layers) were structure,
optimized thewith
hyperparameters of the model
a Bayesian optimization (i.e., theBayesian
scheme. number
of neurons and the number of layers) were optimized with a Bayesian optimization
optimization works by constructing a posterior distribution of functions, i.e., a Gaussian process, scheme. Bayesian
optimization
that works
best describes theby constructing
target function.aWith
posterior distribution
the increase in the of functions,
number i.e., a Gaussiancombinations
of hyperparameter process, that
best describes the target function. With the increase in the number of hyperparameter
observed, the posterior distribution is improved, and the algorithm becomes more certain of which combinations
observed, the posterior
hyperparameter distribution
combination shouldisbeimproved,
picked forand the the
nextalgorithm becomes
observation. more certain
This process of which
is designed to
hyperparameter combination should be picked for the next observation. This
minimize the number of steps required to find a combination of hyperparameters that are close to the process is designed to
minimize the number of steps required to find a combination of hyperparameters
optimal combination [31]. From these, the length of the molecular fingerprints of the two monomers that are close to the
optimal
and combination
the structure of the[31]. From
hidden these,
layer, i.e.,thethelength
number of the molecular
of layers fingerprints
and neurons of the
in each two
layer, monomers
and dropout
rate, i.e., the percentage of neurons to be ignored during training, were determined. Tolayer,
and the structure of the hidden layer, i.e., the number of layers and neurons in each start and
the
dropout rate, i.e.,optimization
hyperparameter the percentage of neurons
process, the rangesto be ignored during training, were
of the hyperparameters were determined.
first defined.To start
It was
the hyperparameter
then subjected to the optimization process, the
Bayesian optimization ranges
process of automatically
that the hyperparameters
identifieswerethefirst defined.
optimal set ofIt
was then subjected to the Bayesian optimization process that automatically
hyperparameters. The hyperparameters of the DNN are optimized after this process. The resultant identifies the optimal set
of hyperparameters. The hyperparameters of the DNN are optimized after
optimal hyperparameters lead to the length of the molecular fingerprints as 128, the number of hiddenthis process. The resultant
optimal
layers sethyperparameters
as 7, the number of lead to theinlength
neurons of the layer
each hidden molecular
set as fingerprints as 128, the
512, and the dropout ratenumber
set as 0.2.of
hidden layers set as 7, the number of neurons in each hidden layer
To augment the limited data set, the DNN model was trained and tested via a fivefold set as 512, and the dropout rate
set as 0.2.
cross-validation method. This method first randomly partitioned the data into five nearly equal-sized
Materials 2020, 13, x FOR PEER REVIEW 5 of 16

Materials 2020, 13, 5701 5 of 15

To augment the limited data set, the DNN model was trained and tested via a fivefold cross-
validation method. This method first randomly partitioned the data into five nearly equal-sized
subgroups. Of Ofthe
thefive
fivesubgroups,
subgroups, a single
a singlesubgroup was
subgroup wasretained for for
retained testing, and and
testing, the remaining four
the remaining
subgroups
four werewere
subgroups usedusedfor for
training. ThisThis
training. process was
process wasthen repeated
then repeatedfive
fivetimes,
times,with
witheach
each of
of the
subgroups used
used exactly
exactlyonceoncefor
formodel
modeltesting. Such
testing. a process
Such a processeffectively augmented
effectively augmentedthe data by five
the data by
times.
five The prediction
times. results
The prediction of each
results testing
of each subgroup
testing werewere
subgroup obtained, and and
obtained, the overall accuracy
the overall was
accuracy
calculated
was by averaging
calculated the sum
by averaging the of theofevaluating
sum scores.scores.
the evaluating The training epoch epoch
The training was setwas
as 500
set to
asensure
500 to
convergence.
ensure convergence.

3. Results and Discussion

3.1. Performance of
3.1. Performance of the
the DNN
DNN Model
Model
The
The scatter
scatterplot of the
plot of predicted vs. experimental
the predicted Tg data isTshown
vs. experimental in Figure 3. The homopolymers
g data is shown in Figure 3. The
are labeled in orange
homopolymers and thein
are labeled copolymers
orange and arethelabeled in blue. The
copolymers accuracyinand
are labeled reliability
blue. of the DNN
The accuracy and
model was evaluated by the coefficient of determination (R 2 ), mean absolute error (MAE), and root
reliability of the DNN model was evaluated by the coefficient of determination (R ), mean absolute 2

mean square error

error (MAE), (RMSE).
and root meanThe R2 , MAE,
square and RMSE
error (RMSE). TheofR2each
, MAE, subgroup
and RMSE and of
their
each averaged
subgroup scores
and
2
their averaged scores using the fivefold cross-validation method are listed in Table 1. From TableR1,,
using the fivefold cross-validation method are listed in Table 1. From Table 1, it is seen that the
MAE,
it is seenandthatRMSE
the Rvalues
2, MAE, are
andclose
RMSE amongvalueseacharecross-validation
close among each subgroup, which indicates
cross-validation subgroup, that the
which
performance
indicates thatof the
the DNN modelofisthe
performance reliable
DNNand model robust againstand
is reliable small fluctuations
robust in thefluctuations
against small training data. in
The overall DNN model had an R 2 of 0.869, MAE of 4.010, and RMSE of 5.339 K. The performance
the training data. The overall DNN model had an R of 0.869, MAE of 4.010, and RMSE of 5.339 K.
2

of
The the DNN model
performance of is
thecomparable
DNN model toisthe results of to
comparable thetheML model
results based
of the MLon QSPR
model by Pilania
based on QSPR et al.,
by
which achieved an RMSE of 4.8K using the same data set [25]. A
Pilania et al., which achieved an RMSE of 4.8K using the same data set [25]. A major advantage ofmajor advantage of this DNN
model
this DNN developed in this study
model developed is its
in this simplicity.
study In the previous
is its simplicity. study, the
In the previous initial
study, thefeatures of PHAs
initial features of
were first described by a set of QSPR-based features, and then this feature
PHAs were first described by a set of QSPR-based features, and then this feature set was further set was further analyzed,
compacted, and trainedand
analyzed, compacted, by trained
a random by forest
a randomML model.
forest ML Thismodel.
process requires
This professional
process experience
requires professional
and significant domain knowledge in the polymer chemistry.
experience and significant domain knowledge in the polymer chemistry. Comparatively, the Comparatively, the DNN learning
DNN
model
learning developed here, which
model developed combines
here, which the binary the
combines fingerprinting with the DNN
binary fingerprinting withmodel,
the DNN is much
model,moreis
straightforward, easier to implement, and requires minimal expert
much more straightforward, easier to implement, and requires minimal expert judgement and judgement and therefore is free
from preference.
therefore In addition,
is free from preference.theInmodel can also
addition, be easily
the model can extended
also be easily to aextended
broader range of polymers
to a broader range
beyond PHA copolymers.
of polymers beyond PHA copolymers.

Figure
Figure 3. The scatter
3. The scatter plot
plot of
of the
the predicted
predicted glass
glass transition
transition temperature, Tgg by
temperature, T by the
the DNN
DNN model
model vs.
vs.
experimental measurement results, Tg on the test data.
experimental measurement results, Tg on the test data.
Materials 2020, 13, 5701 6 of 15

Table 1. The R2 , MAE, and RMSE of the cross-validated subgroups and their averaged scores.
Materials 2020, 13, x FOR PEER REVIEW 6 of 16
Subgroup Subgroup 1 Subgroup 2 Subgroup 3 Subgroup 4 Subgroup 5 Average
R2 0.799 0.883 and RMSE 0.857
Table 1. The R2, MAE, 0.824subgroups and
of the cross-validated 0.914 0.869
their averaged scores.
MAE 3.792
Subgroup 3.970 1
Subgroup 4.976
Subgroup 2 Subgroup 3.745
3 Subgroup 43.541
Subgroup 54.010
Average
RMSE R2
5.147 0.799
5.356 0.883
6.522 0.857 4.775 0.824 4.643 0.914 5.3390.869
MAE 3.792 3.970 4.976 3.745 3.541 4.010
RMSE 5.147 5.356 6.522 4.775 4.643 5.339
As there is no preference in labeling one of monomers as A and another as B, i.e., the model
should contain symmetry,
As there which is not captured
is no preference in theone
in labeling dataofset used in the
monomers as DNN
A andmodel
anothertraining shown
as B, i.e., the model
in Figure 2. For example, regarding monomer A (3H5PhP), monomer B (3H7PhHp) with
should contain symmetry, which is not captured in the data set used in the DNN model training fractional
composition of A at 23%
shown is equivalent
in Figure to monomer
2. For example, A (3H7PhHp),
regarding monomer with monomermonomer
A (3H5PhP), B (3H5PhP) having a with
B (3H7PhHp)
fractional composition
fractional composition of A at 100%of−A23% = 77%.
at 23% is equivalent to the
In principle, monomer
machine A learning
(3H7PhHp), withshould
model monomer B
capture such (3H5PhP)
symmetry.having a fractional
To evaluate whethercomposition
the DNN modelof A atcan
100% − 23%the
capture = 77%.
symmetryIn principle,
in inputthe machine
data,
a new symmetry data set is generated from the original data set by exchanging the monomer A and B the
learning model should capture such symmetry. To evaluate whether the DNN model can capture
and updatingsymmetry in input
the fractional data, a of
composition new symmetry
A by data set
100% fraction is generated
composition of A.from
Thethe originalweight,
molecule data set by
exchanging the monomer A and B and updating the fractional composition
PDI, and glass transition temperature remain the same. From physics, the symmetry data set contains of A by 100% fraction
composition of A. The molecule weight, PDI, and glass transition temperature remain the same. From
the same information of the original data set.
physics, the symmetry data set contains the same information of the original data set.
The comparison of predicted Tg by the DNN model versus the measured Tg on the symmetric
The comparison of predicted Tg by the DNN model versus the measured Tg on the symmetric
data set is shown in Figure 4. Compared with the results shown in Figure 3, the prediction results are
data set is shown in Figure 4. Compared with the results shown in Figure 3, the prediction results are
much worse. muchThis indicates
worse. Thisthat the trained
indicates DNN
that the model
trained DNN didmodel
not capture
did not the symmetry
capture and therefore
the symmetry and therefore
is sensitive toisthe order of monomers. This will lead to prediction bias in its application
sensitive to the order of monomers. This will lead to prediction bias in its application that needs
that to
needs to
be improved.be improved.

Figure 4. TheFigure
scatter4.plot
The of
scatter plot of thevs.
the predicted predicted vs. experimental
experimental Tg on theTgsymmetry
on the symmetry
data setdata set by the
by the
pretrained
pretrained DNN model. DNN model.

To solve
To solve this issue, we this issue,the
retrained wemodel
retrained
withthe model with
a combined a combined
initial initial
training data settraining data set and
and symmetry
data set, usingsymmetry
the samedata set, using
five-fold the same five-fold
cross-validation method. cross-validation method. The
The overall retrained DNN overall
modelretrained
had an DNN
model
R2 of 0.897, MAE ofhad an R
3.474,
2 of 0.897, MAE of 3.474, and RMSE of 4.748 K. Compared with the initial DNN model,
and RMSE of 4.748 K. Compared with the initial DNN model, the retrained
the retrained DNN model achieved higher prediction accuracy. This shows that training the DNN
DNN model achieved higher prediction accuracy. This shows that training the DNN model with a
model with a symmetry data set can further improve the model performance. The scatter plots of the
symmetry data set can further improve the model performance. The scatter plots of the predicted Tg
predicted Tg under different data sets are shown in Figure 5. The retrained model achieved similar
under different data sets are shown in Figure 5. The retrained model achieved similar performance on
performance on the initial training data set and the symmetry data set, which proves that
the initial training data set the
incorporating andsymmetry
the symmetry
data setdata set,
into which
the proves
training datathat incorporating
set can the symmetry
enable the model to describe the
data set into the trainingand
symmetry databe set can enable
insensitive theorder
to the model of to describeAthe
monomer and symmetry and be
B designation. insensitivecompared
Meanwhile, to
the order of monomer A and B designation. Meanwhile, compared with Figure 4, the performance
with Figure 4, the performance of the retrained DNN model with additional symmetry data set is of
the retrainedmuch
DNNbetter
model with
than thatadditional
trained onlysymmetry data set
with the initial is much better than that trained only
model.
with the initial model.
Materials 2020, 13, 5701 7 of 15
Materials 2020, 13, x FOR PEER REVIEW 7 of 16
Materials 2020, 13, x FOR PEER REVIEW 7 of 16

Figure 5.5. The

The scatter
scatter plot
plotof
ofthe
thepredicted
predictedvs.
vs. experimentalTTgg after
after retraining the
the DNN model
model with
Figure
Figure 5. The scatter plot of the predicted vs.experimental
experimental Tg afterretraining
retraining theDNN
DNN model with
with
the combined
the combinedinitial
initialand
and symmetry
symmetry data
data set;
set; (a)
(a) results
results with
with initial
initial data
data set;
set; (b)
(b) results
results with
with symmetry
symmetry
the combined initial and symmetry data set; (a) results with initial data set; (b) results with symmetry
data set.
data set.
data set.
3.2.
3.2. Importance
Importance Factors of the Input Variables on Tg Prediction
3.2. ImportanceFactorsFactorsofofthetheInput
InputVariables
Variableson onTTg gPrediction
Prediction
The
The interpretability of the DNN model was also studied, since understanding why aa model
Theinterpretability
interpretabilityofofthe theDNN DNNmodel modelwas wasalso also studied,
studied, sincesince understanding
understanding why why a modelmodel
makes
makes a certain prediction can be as crucial as the prediction’s accuracy for chemical applications.
makesa acertain
certainprediction
predictioncan canbebeasascrucial
crucialas asthetheprediction’s
prediction’s accuracy
accuracy for for chemical
chemical applications.
applications.
The
The DNN model belongs to the category of black-box models. For the interpretability of the DNN
The DNN model belongs to the category of black-box models. For the interpretability of
DNN model belongs to the category of black-box models. For the interpretability of the DNN
the DNN
model,
model, the
the SHapley
SHapley Additive
Additive exPlanations
exPlanations (SHAP)
(SHAP) method
method was
was used
used to to measure
measure the
the importance
importance of
model, the SHapley Additive exPlanations (SHAP) method was used to measure the importance of
of input
input features to the DNN model. SHAP is a unified framework for interpreting predictions,
inputfeatures
featurestotothe theDNN DNNmodel. model.SHAP SHAP isis aa unified unified framework
framework for for interpreting
interpreting predictions,
predictions,
especially
especially for
forcomplex
complex black-box
black-box models.
models. It assigns
It to each
assigns to feature
each an importance
feature an value for value
importance a particular
for aa
especially for complex black-box models. It assigns to each feature an importance value for
prediction
particular indicating
prediction how much how
indicating a modelmuch relies
a on each
model of the
relies on features—in
each of the other words,other
features—in howwords,
much
particular prediction indicating how much a model relies on each of the features—in other words,
each
how feature
much contributes
each to the prediction. This helps usThis to understand the impact ofthe input variables on
how much eachfeature
featurecontributes
contributestotothe theprediction.
prediction. Thishelps helpsus usto tounderstand
understand the impact impact of input
of input
the prediction
variables of T [32]. Figure 6 shows the importance factors of the five variables. It shows that
variablesononthe theprediction
g
prediction ofofTT g g[32].
[32].Figure
Figure66shows showsthe theimportance
importance factors factors of of the
the five variables. It
the
showsmonomer
showsthat thatthe A and
themonomer B both
monomerAAand had high
andBBboth importance
bothhad hadhigh values
highimportance and that
importance values they
values and were
and that of similar
that they
they were importance.
of similar
This is reasonable
importance.
importance. This since the
Thisisisreasonable types
reasonablesince of monomer
sincethe thetypes are the
typesofofmonomer most
monomerare crucial
arethe in
themost determining
mostcrucial
crucial in the T ;
in determining
g
determining in addition,
the Tgg;
they
inin are
addition,interchangeable
they are as input
interchangeable symmetry
as input as discussed
symmetry earlier.
as
addition, they are interchangeable as input symmetry as discussed earlier. The importance The
discussed importance
earlier. The factors of
importance monomer
factors
Aofand B were much
ofmonomer
monomer AAand higher
and BBwere than
were muchthe other
much higher
higherthreethanvariables.
than the
the other
otherThis indicates
three
three that the
variables.
variables. Thischemistry
This indicates
indicates of that
the twothe
monomers
chemistry has
chemistryofofthe the predominant
thetwo twomonomers
monomershaseffect in
hasthe determining
thepredominant the
predominantgeffect T of the
effect in PHA polymers.
in determining
determining the the Tgg of the inPHA
The next one the
importance
polymers.
polymers. factor
TheThenext isone
next the fractional
oneininthe composition
theimportance
importance factorof
factor A.
theThe
isisthe relatively
fractional
fractional low SHAP
composition
composition of values
of A.
A. The of molecular
The relatively low
weight and
SHAPvalues
SHAP PDI imply
valuesofofmolecular that the
molecularweight polymer
weightand chain
andPDI size
PDIimply and
implythat dispersity
thatthethe polymerhave
polymer chainrelatively
chain size less
size and influence on
and dispersity have T g of
the PHA polymers
relativelyless
relatively as
lessinfluence compared
influenceon onTT g gof
to
ofthethe
thePHA chemistry
PHApolymers
polymersasof the monomers
ascompared
compared to and
to the the fractional
the chemistry composition
chemistry of the monomers of
the polymer.
and the fractional composition
and the fractional composition of the polymer. of the polymer.

Figure 6. The bar plot ofthe

the importancefactors
factors ofeach
each inputvariable
variable tothethe DNNmodel.
model.
Figure 6. The
Figure 6. The bar
bar plot
plot of
of the importance
importance factors ofof each input
input variable to
to the DNN
DNN model.
Materials 2020, 13, 5701 8 of 15

3.3. Comparison with DNN Model with Other ML Methods

Materials 2020, 13, x FOR PEER REVIEW 8 of 16
The performance of the DNN ML model is compared with other nonlinear ML method such as
the support vector
3.3. Comparison machine
with (SVM)
DNN Model and
with LASSO,
Other a sparse linear regression model. The results of SVM
ML Methods
and LASSO methods on the Tg prediction have been compared. Fivefold cross-validation was used for
The performance of the DNN ML model is compared with other nonlinear ML method such as
all the models. The results are shown in the Table 2 and Figure 7. The Table 2 concludes the R2 , MAE,
the support vector machine (SVM) and LASSO, a sparse linear regression model. The results of SVM
and RMSE of three methods, i.e., SVM, LASSO, and the DNN model. The DNN model achieved the
and LASSO methods on the Tg prediction have been compared. Fivefold cross-validation was used
best performance among these ML models.
for all the models. The results are shown in the Table 2 and Figure 7. The Table 2 concludes the R2,
MAE, and RMSE of three methods, i.e., SVM, LASSO, and the DNN model. The DNN model achieved
Table 2. Comparison of the results by the support vector machine (SVM), least absolute shrinkage and
the best performance
selection among these
operator (LASSO) model,ML andmodels.
DNN model.

Table 2.Subgroup
Model Comparison of the results
Subgroup 1 by the support
Subgroup 2 vector machine
Subgroup 3 (SVM), 4leastSubgroup
Subgroup absolute shrinkage
5 Average
and selection
R2 operator (LASSO)
0.684 model, and
0.828 DNN model.
0.766 0.791 0.778 0.783
SVM MAE 3.981 4.075 2 6.285 3 3.937 4 4.436 5 4.548
Model Subgroup Subgroup 1 Subgroup Subgroup Subgroup Subgroup Average
R2
RMSE 0.684
6.403 0.828
6.488 0.766
8.361 0.791
5.206 0.778
7.449 0.783
6.871
SVM MAE 3.981 4.075 6.285 3.937 4.436 4.548
R2 0.607 0.802 0.768 0.648 0.610 0.714
LASSO RMSE 6.403 6.488 8.361 5.206 7.449 6.871
MAE 4.820 4.258 5.988 4.986 6.270 5.259
R2 0.607 0.802 0.768 0.648 0.610 0.714
LASSO RMSE
MAE 7.142
4.820 6.963
4.258 8.322
5.988 6.763
4.986 9.879
6.270 7.892
5.259
RMSE
R2 7.142
0.799 6.963
0.883 8.322
0.857 6.763
0.824 9.879
0.914 7.892
0.869
DNN R2
MAE 0.799
3.792 0.883
3.970 0.857
4.976 0.824
3.745 0.914
3.541 0.869
4.010
DNN MAE 3.792 3.970 4.976 3.745 3.541 4.010
RMSE 5.147 5.356 6.522 4.775 4.643 5.339
RMSE 5.147 5.356 6.522 4.775 4.643 5.339

Figure7.7.The
Figure Thepredicted
predictedvs.
vs. experimental
experimental T
Tgg using
using (a)
(a) SVM
SVMand
and(b)
(b)LASSO
LASSO(results
(resultsby
byDNN
DNNare
areshown
shown
ininFigure
Figure5).
5).

3.4.
3.4.DNN
DNNModel
ModelPrediction
PredictionPerformance
Performance with
with Inaccurate or Missing Information
Information
Although
Althoughextensive
extensiveefforts
effortshave
have been
been devoted
devoted to the fabrication
fabrication ofof PHA
PHA polymers
polymersandandthere
thereare
are
TTg gdata
dataononnumerous
numerouspolymers
polymers exhibiting
exhibiting a diverse set of chemistries
chemistries inin the
theliterature,
literature,some
somepolymer
polymer
information
informationmay maybebemissing
missingor orinconsistent
inconsistent for
for a certain type of
of PHA.
PHA.For Forexample,
example,thethemeasurements
measurements
ofofmolecular
molecularweight
weightand
andPDIPDI are
are complicated
complicated and are prone to error. It It is
is of
of interest
interestto
tosee
seehow
howsuch
such
inaccuracyor
inaccuracy ormissing
missinginformation
information wouldwould affect the performance
performance of of the
the DNN
DNN modelmodelprediction
predictionandand
developproper
develop propercountermeasures
countermeasures to to improve
improve the robustness and accuracy
accuracy of of the
the DNN
DNNmodel
modelunder
under
suchcircumstances.
such circumstances. ToTo evaluate
evaluate the the performance
performance of thethe developed
developed DNN DNNmodelmodelto tomissing
missingvalues,
values,
weassumed
we assumedone oneor
ortwo
twoinputs
inputs were
were missing
missing for
for the
the DNN model, e.g.,e.g., fractional
fractional composition
compositionofofAA
only,molecular
only, molecularweight
weightonly,
only,PDIPDIonly,
only,andandmolecular
molecularweight
weight and
and PDI both ignored.
ignored. TheTheperformance
performance
of different approaches, i.e., directly removing
of different approaches, i.e., directly removing input input nodes of the missing
missing values, andreplacing
values, and replacingthethe
missingdata
missing datawith
withthe
themean,
mean,maximum,
maximum, and and minimum
minimum values,
values, were
were evaluated
evaluated on onhow
howthey
theyaffect
affectthe
the
model performance.
model performance.
Materials 2020, 13, 5701 9 of 15

3.4.1. The Effects of Removing Input Nodes Corresponding to the Missing Values
The direct method to deal with a variable with missing data is to remove the corresponding input
nodes to the DNN model. Four sets of DNN models with some input node removed (i.e., without
fractional composition of A, without molecular weight, without PDI, and without both the molecular
weight and PDI in the feature set) were trained. The prediction results are shown in Figure 8 and the
evaluation scores of model accuracy are listed in Table 3. For the case without fractional composition of
A, the R2 of the model dramatically decreased to about 0.5, which indicates that fractional composition
of A is a significant factor for Tg prediction. Indeed, this observation is quite intuitive and can be
understood on a physical basis. As the fractional composition determines the chemical composition of
the copolymer chemistry, in the absence of this critical feature, all copolymer chemistries formed by
combining any two given homopolymers in different compositions essentially become indistinguishable
to the model and, as a result, the predictive performance degrades drastically.

Figure 8 8. The scatter plot of the predicted vs. experimental Tg after different inputs were removed
Figure
from the DNN model. (a) Fractional composition of A; (b) molecular weight; (c) polydispersity index
(PDI); (d) both molecular weight and PDI.

Table 3. The R2 , MAE, and RMSE with different ignored inputs.

Ignored Complete Fractional

MW PDI MW and PDI
Features Inputs Composition of A
R2 0.897 0.578 0.861 0.859 0.846
MAE 3.592 7.269 4.086 4.206 4.520
RMSE 4.812 9.586 5.511 5.543 5.800

Deleting the input node of molecular weight or PDI alone did not significantly compromise the
model performance compared with the full-input model. Even when further deleting both molecular
Materials 2020, 13, 5701 10 of 15

weight and PDI, the R2 was still comparable to the full-input model. These findings indicate
Materials 2020, 13, x FOR PEER REVIEW
that
10 of 16
molecular weight and PDI have limited contributions to the final prediction and they are much
less important
important thanthan the chemistry
the chemistry of theofmonomers
the monomers and
and the the fractional
fractional compositions
compositions of monomer
of monomer A. TheseA.
These are consistent with the results of importance factors of
are consistent with the results of importance factors of each input. each input.

3.4.2.
3.4.2.The
TheEffects
Effectsof
ofReplacing
Replacing the
the Missing
Missing Values
Values with
with the
the Mean,
Mean, Minimum, or Maximum
Minimum, or Maximum Values
Values
AnAnalternative
alternativeapproach
approach to to
deal with
deal missing
with datadata
missing is to replace the missing
is to replace values with
the missing thewith
values statistics
the
ofstatistics
trainingofdata such as the mean value, the minimum value, or the maximum value.
training data such as the mean value, the minimum value, or the maximum value. In this In this way,
the
way,models are notare
the models required to be retrained;
not required instead, instead,
to be retrained; the missingthe data are replaced
missing with the estimated
data are replaced with the
values
estimated values obtained from the training data. Figure 9 shows the statistical distribution of data.
obtained from the training data. Figure 9 shows the statistical distribution of the input the
The
inputdescriptive
data. Thestatistics of the
descriptive input of
statistics data
theare listed
input inare
data Table 4. in Table 4.
listed

Figure9.9.The
Figure Thestatistical
statistical distribution
distribution of the input data. (a)
(a) Fractional
Fractional composition
compositionof
ofA;
A;(b)
(b)molecular
molecular
weight;(c)
weight; (c)PDI.
PDI.

Table 4. The descriptive statistics of the input data.

Table 4. The descriptive statistics of the input data.
Statistics
Statistics Mean
Mean SD*
SD* Max
Max Min Median
Min Median
Fractional composition
Fractional of A of A 67.167.1
composition 31.5
31.5 100
100 00 74.574.5
MWMW 1159.8 1313.86
1159.8 1313.86 5240.4
5240.4 4747 505505
PDI 2.6 1.1 6.9 1.1 2.5
PDI 2.6 1.1 6.9 1.1 2.5
SD* is the acronym standard deviation.
SD* is the acronym standard deviation.

AAcommon
common method
method to to deal
deal with
with missing
missing data is to replace
replace the
the missing
missing values
valueswith
withthe
themean
mean
value.
value. Table
Table 55shows
shows thethe DNN
DNN model
model prediction
prediction performance using this
performance using this strategy
strategyfor
formissing
missingfour
four
types
typesofofinput
inputdata,
data, i.e.,
i.e., the
the fractional
fractional composition molecular weight,
composition of A, the molecular weight, the
thePDI,
PDI,and
andboth
both
molecular
molecularweight
weight and
and PDI.
PDI. The
The scatter
scatter plots
plots of the
the predictions are shown
shown in in Figure
Figure 10.
10. Compared
Compared
with
withthe
theDNN
DNNmodel
modelprediction
prediction with
with full
full original
original inputs, replacing
replacing the
the individual
individualinputs
inputswith
withthe
the
corresponding mean input for the molecular weight, or PDI, or both, exhibited nearly the same model
performance, with the differences of R2 less than 0.6%. However, an exception is observed for the
fractional composition of A, where the DNN model performance with mean value replacing the
Materials 2020, 13, 5701 11 of 15

corresponding mean input for the molecular weight, or PDI, or both, exhibited nearly the same model
performance, with the differences of R2 less than 0.6%. However, an exception is observed for the
fractional composition of A, where the DNN model performance with mean value replacing the missing
value is worse than simply deleting the input node for the fractional composition A. The observations
further validated that the most significant factors for Tg prediction were the chemistry of the monomers
and their fractional compositions in the copolymers. The DNN model still achieved a high degree
of accuracy in Tg prediction without information on the measurement of molecular weight, or PDI,
or both of molecular weight and PDI. Such missing information could be estimated with the mean
values typical of such data and still achieve decent accuracy.

Table 5. The R2 , MAE, and RMSE of the DNN model prediction when different input variables are
estimated with their corresponding mean values.

Fractional
Input Features Complete Inputs MW PDI MW and PDI
Composition of A
R2 0.897 0.423 0.866 0.865 0.864
MAE 3.592 8.819 4.055 4.086 4.083
RMSE 4.812 11.215 5.410 5.416 5.448
Figure 8

10 10. The scatter plot of the DNN model predicted vs. experimental Tg with different input
Figure
Figure
factors estimated with the mean value. (a) Fractional composition of A; (b) molecular weight; (c) PDI;
(d) molecular weight and PDI. 1

To further investigate the sensitivity of the DNN model to the estimation of missing input values,
we also analyzed its performance when the missing values are estimated with the minimum or
maximum values of the corresponding type of data from the data set. Compared with estimating the
missing data with the mean values, estimating the input with the minimum or maximum values can also
give a sense of how the model performance is affected by the data quality, since errors might inevitably
Materials 2020, 13, 5701 12 of 15

occur during measurement. Tables 6 and 7 summarize the model prediction performances when
the corresponding input variable is replaced with its minimum or maximum values. Four different
conditions were analyzed, i.e., the fractional composition of A, the molecular weight, PDI, and both
molecular weight and PDI. The scatter plots of the predictions are shown in Figures 11 and 12.

Table 6. The R2 , MAE, and RMSE with DNN model prediction when different input variables are
estimated with their minimum values.

Fractional
Input Features Complete Inputs MW PDI MW and PDI
Composition of A
R2 0.897 −1.488 0.857 0.835 0.807
MAE 3.592 20.146 4.302 4.590 5.204
RMSE 4.812 23.290 5.575 5.994 6.494

Table 7. The R2 , MAE, and RMSE with DNN model prediction when different input variables are
estimated with their maximum values.

Fractional
Input Features Complete Inputs MW PDI MW and PDI
Composition of A
R2 0.897 0.053 0.711 0.688 0.210
MAE 3.592 10.512 6.383 6.937 11.474
RMSE 4.812 14.370 7.942 8.247 13.123

11 11. Scatter plots of predicted vs. experimental Tg with different input factors estimated via their
Figure
Figure
minimal values. (a) Fractional composition of A; (b) molecular weight; (c) PDI; (d) molecular weight
and PDI.

Compared with the method of replacing the input variable with the mean value, estimating
the input variables with its minimum or maximum values showed a reduced prediction accuracy.
Materials 2020, 13, 5701 13 of 15

In general, the use of minimum values as the estimation led to predictions shifting toward the left of
the diagonals or resulted in lower predicted Tg than measured Tg , while the use of maximum values
showed the opposite effect (Figure 12). In terms of the polymer fractional composition of A, as shown
in Figure 9a, the minimum value is far away from where the rest of the data are concentrated, compared
with the maximum value. This caused the model prediction with the minimal value to be worse than
that with the maximum value. In contrast, for MW and PDI, as shown in Figure 9b,c, the maximum
value is far away from where the rest of the data are concentrated compared with the minimum value,
causing the model prediction with the maximum value to be worse than that with the minimum
value. The prediction accuracy by filling missing data with minimal values is only slightly lower
than with the mean values for input variables, including the molecular weight, PDI, and molecular
weight and PDI, whereas the prediction accuracy by filling missing data with the maximum values
drops dramatically. This is possibly due to the fact that the data values follow a skewed statistical
distribution (i.e., Figure 9). In such cases, the mean is larger than the median and the maximum value
is further away from the mode than the minimum value. Therefore, estimating the missing data with
its maximum value leads to a worse prediction than estimating with its minimum value. The results
show that the features of missing data are important to proper estimation of missing values to ensure
DNN model
Figure 11 prediction accuracy.

Figure 12 12. Scatter plots of the predicted vs. experimental Tg with different input factors estimated with
Figure
their maximum values. (a) Fractional composition of A; (b) molecular weight; (c) PDI; (d) molecular
weight and PDI.

4. Conclusions 2

A DNN model was developed for the prediction of Tg of PHA-based homo- and copolymers.
The information of monomers is encoded by its digital fingerprints and subsequently inputs to the
DNN model together with other factors affecting the Tg . The hyperparameters of the DNN model are
optimized with Bayesian optimization schema. The results indicate that the Tg values predicted by the
DNN model are in good agreement with experimental results. The symmetry of the DNN model is
Materials 2020, 13, 5701 14 of 15

achieved by incorporating the symmetry data set into the training data set, which further improves the
model prediction performance. The interpretability analyses based on the importance factors revealed
that the types of the two monomers are most important in determining the Tg values of the copolymers,
while the size and dispersity of the polymers did not significantly affect the Tg values. Furthermore,
different strategies to cope with missing data were evaluated, including removal of the input node,
as well as estimation of the missing data with the mean, minimum, or maximum values of the data set.
The results indicate that the DNN model is robust in achieving high accuracy even with missing data.
Understanding the characteristics of the data distribution is important to making sound estimations of
the missing data to ensure high performance of the DNN model. Compared with a commonly used
ML model incorporating QSPR, the DNN model does not require an explicit descriptor selection step
but shows a comparable performance. It is also noted that while the Tg is the targeted property in
this study, the DNN model can be readily adapted to predict other properties of PHA homopolymers
and copolymers.

Supplementary Materials: Available Statement: The following are available online at http://www.mdpi.com/1996-
1944/13/24/5701/s1, Table S1: the polymer training dataset employed for the Tg prediction model development.
Author Contributions: The individual contributions of each coauthor is as follows: conceptualization, X.Y. and
B.L.M.; methodology, X.Y.; software, J.H.; validation, J.H. and Z.J.; formal analysis, X.Y. and B.L.M.; investigation,
X.Y., J.H., and G.P.; resources, X.Y.; data curation, Z.J. and G.P.; writing—original draft preparation, Z.J. and J.H.;
writing—review and editing, X.Y. and B.L.M.; visualization, J.H.; supervision, X.Y.; project administration, X.Y.;
funding acquisition, X.Y. and B.L.M. All authors have read and agreed to the published version of the manuscript.
Funding: This work was partially supported by the U.S. National Science Foundation with an award number:
1563238. B.L.M. and G.P. would like to acknowledge support from the Los Alamos National Laboratory’s Laboratory
Directed Research and Development (LDRD) program’s Directed Research (DR) project titled Bio-Manufacturing
with Intelligent Adaptive Control (BioManIAC) #20190001DR. Los Alamos National Laboratory is operated by
Triad National Security, LLC, for the National Nuclear Security Administration of U.S. Department of Energy
(Contract No. 89233218CNA000001).
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Jambeck, J.R.; Geyer, R.; Wilcox, C.; Siegler, T.R.; Perryman, M.; Andrady, A.; Narayan, R.; Law, K.L. Plastic
waste inputs from land into the ocean. Science 2015, 347, 768–771. [CrossRef] [PubMed]
2. Chamas, A.; Moon, H.; Zheng, J.; Qiu, Y.; Tabassum, T.; Jang, J.; Abu-Omar, M.M.; Scott, S.; Suh, S. Degradation
rates of plastics in the environment. ACS Sustain. Chem. Eng. 2020, 8, 3494–3511. [CrossRef]
3. Lebreton, L.; Andrady, A. Future scenarios of global plastic waste generation and disposal. Palgrave Commun.
2019, 5, 1–11. [CrossRef]
4. Arikan, E.B.; Ozsoy, H.D. A Review: Investigation of bioplastics. J. Civ. Eng. Arch. 2015, 9, 188–192. [CrossRef]
5. Bátori, V.; Åkesson, D.; Zamani, A.; Taherzadeh, M.J.; Horváth, I.S. Anaerobic degradation of bioplastics:
A review. Waste Manag. 2018, 80, 406–413. [CrossRef] [PubMed]
6. Keshavarz, T.; Roy, I. Polyhydroxyalkanoates: Bioplastics with a green agenda. Curr. Opin. Microbiol. 2010,
13, 321–326. [CrossRef] [PubMed]
7. Philip, S.; Keshavarz, T.; Roy, I. Polyhydroxyalkanoates: Biodegradable polymers with a range of applications.
J. Chem. Technol. Biotechnol. 2007, 82, 233–247. [CrossRef]
8. Wu, Q.; Wang, Y.; Chen, G.-Q. Medical application of microbial biopolyesters polyhydroxyalkanoates.
Artif. Cells Blood Substit. Biotechnol. 2009, 37, 1–12. [CrossRef]
9. Raza, Z.A.; Abid, S.; Banat, I.M. Polyhydroxyalkanoates: Characteristics, production, recent developments
and applications. Int. Biodeterior. Biodegrad. 2018, 126, 45–56. [CrossRef]
10. Visakh, P.M. Polyhydroxyalkanoates (PHAs), Their Blends, Composites and Nanocomposites; Royal Society of
Chemistry: London, UK, 2014; pp. 1–17. [CrossRef]
11. Li, Z.; Yang, J.; Loh, X.J. Polyhydroxyalkanoates: Opening doors for a sustainable future. NPG Asia Mater.
2016, 8, e265. [CrossRef]
12. Mitra, R.; Xu, T.; Xiang, H.; Han, J. Current developments on polyhydroxyalkanoates synthesis by using
halophiles as a promising cell factory. Microb. Cell Factories 2020, 19, 1–30. [CrossRef] [PubMed]
Materials 2020, 13, 5701 15 of 15

13. Shrivastav, A.; Kim, H.-Y.; Kim, Y.-R. Advances in the applications of polyhydroxyalkanoate nanoparticles
for novel drug delivery system. BioMed Res. Int. 2013, 2013, 581684. [CrossRef] [PubMed]
14. Muneer, F.; Rasul, I.; Azeem, F.; Siddique, M.H.; Zubair, M.; Nadeem, H. Microbial polyhydroxyalkanoates
(PHAs): Efficient replacement of synthetic polymers. J. Polym. Environ. 2020, 28, 2301–2323. [CrossRef]
15. Sifri, R.J.; Padilla-Vélez, O.; Coates, G.W.; Fors, B.P. Controlling the shape of molecular weight distributions
in coordination polymerization and its impact on physical properties. J. Am. Chem. Soc. 2020, 142, 1443–1448.
[CrossRef]
16. Al-Jarrah, O.Y.; Yoo, P.D.; Muhaidat, S.; Karagiannidis, G.K.; Taha, K. Efficient machine learning for big data:
A review. Big Data Res. 2015, 2, 87–93. [CrossRef]
17. Goncalves, V.; Maria, K.; Da Silva, A.B.F. Applications of artificial neural networks in chemical problems.
In Artificial Neural Networks—Architectures and Applications; IntechOpen: London, UK, 2013; pp. 203–223.
18. Basheer, I.A.; Hajmeer, M. Artificial neural networks: Fundamentals, computing, design, and application.
J. Microbiol. Methods 2000, 43, 3–31. [CrossRef]
19. LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [CrossRef]
20. Alam, M.; Samad, M.; Vidyaratne, L.; Glandon, A.; Iftekharuddin, K. Survey on deep neural networks in
speech and vision systems. Neurocomputing 2020, 417, 302–321. [CrossRef]
21. Feng, S.; Zhou, H.; Dong, H.B. Using deep neural network with small dataset to predict material defects.
Mater. Des. 2019, 162, 300–310. [CrossRef]
22. Kondo, R.; Yamakawa, S.; Masuoka, Y.; Tajima, S.; Asahi, R. Microstructure recognition using convolutional
neural networks for prediction of ionic conductivity in ceramics. Acta Mater. 2017, 141, 29–38. [CrossRef]
23. Zhong, S.; Hu, J.; Fan, X.; Yu, X.; Zhang, H. A deep neural network combined with molecular fingerprints
(DNN-MF) to develop predictive models for hydroxyl radical rate constants of water contaminants.
J. Hazard. Mater. 2020, 383, 121141. [CrossRef] [PubMed]
24. Tan, G.-Y.A.; Chen, C.-L.; Li, L.; Ge, L.; Wang, L.; Razaad, I.M.N.; Li, Y.; Zhao, L.; Mo, Y.; Wang, J.-Y. Start a
research on biopolymer polyhydroxyalkanoate (PHA): A review. Polymers 2014, 6, 706–754. [CrossRef]
25. Pilania, G.; Iverson, C.N.; Xue, D.; Marrone, B.L. Machine-learning-based predictive modeling of glass
transition temperatures: A case of polyhydroxyalkanoate homopolymers and copolymers. J. Chem. Inf. Model.
2019, 59, 5013–5025. [CrossRef] [PubMed]
26. Khalyavina, A.; Häußler, L.; Lederer, A. Effect of the degree of branching on the glass transition temperature
of polyesters. Polymers 2012, 53, 1049–1053. [CrossRef]
27. Avaltroni, F.; Bouquerand, P.; Normand, V. Maltodextrin molecular weight distribution influence on the glass
transition temperature and viscosity in aqueous solutions. Carbohydr. Polym. 2004, 58, 323–334. [CrossRef]
28. Weininger, D. SMILES, a chemical language and information system. 1. Introduction to methodology and
encoding rules. J. Chem. Inf. Model. 1988, 28, 31–36. [CrossRef]
29. RDKit. Open Source Toolkit for Cheminformatics. Available online: http://www.rdkit.org (accessed on
10 December 2020).
30. Kingma, D.P.; Ba, J. A method for stochastic optimization. arXiv 2014, arXiv:1412.6980.
31. Snoek, J.; Larochelle, H.; Adams, R.P. Practical bayesian optimization of machine learning algorithms.
In Proceedings of the 26th Annual Conference on Neural Information Processing Systems 2012, Lake Tahoe,
NV, USA, 3–8 December 2012; MIT Press; pp. 2951–2959.
32. Lundberg, S.M.; Lee, S.I. A unified approach to interpreting model predictions. In Proceedings of the 31st
Annual Conference on Neural Information Processing Systems, Long Beach, CA, USA, 4–9 December 2017;
MIT Press; pp. 4765–4774.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional
affiliations.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).