Integration of AI & ML in Mechanical Engineering (V20MEJ03)

Unit-I : AI & ML in design of mechanical materials

Introduction:

Design of mechanical materials is the process of tailoring the composition and structure
of materials to achieve desired mechanical properties, which are of great importance to advanced
materials.

Materials are of significant importance as building blocks of the tools to develop our
civilization. Numerous effective methods to find new materials have been invented due to the
discovery of the intimate connection between the structure of materials and their properties. The
mechanical properties are of interest which influences mechanical failure. Synthetic composites
are formed by varying the composition of each constituent material and their structure.

The bio-inspired materials show excellent mechanical and biological properties, difficult
to design due to hierarchical and heterogeneous structure of bio-materials. Meta materials exhibit
excellent properties, attracted great attention due to the development of experimental techniques
and computer aided optimization tools to design complex material structures. Architected
materials have demonstrated superior mechanical properties such as high stiffness/strength to
weight ratios, recoverability under suppression and damage resistance.

The complexity of compositional and topological structures of advanced materials, the

material design problem to optimize mechanical behavior involves large design data that exceed
the computational limit of conventional methods.

Artificial intelligence (AI) especially Machine learning (ML) & Deep learning (DL) is
becoming an important tool in the field of materials and mechanical engineering due to its ability
to predict material properties, design novel materials and discover new mechanisms. The ML
models trained from large material datasets can relate structure, properties and function at
multiple hierarchical levels. Recent advances in ML techniques solved long standing mechanical
problems and developed improved material design strategies. The performance of a ML based
materials design approach depends on the collection or generation of a large dataset that is
properly pre-processed using the domain knowledge of material science and suitable selection of
ML model.

Schematic of a typical work flow for design of mechanical materials using ML

A typical work flow for combining ML and materials research consists of 3 key components

1. A well organized material database either collected from literature and existing databases
or generated from experiments and simulations.
2. A ML model that is capable to learn
3. A well defined research problem of mechanical materials tht has not been addressed by
conventional methods.
Summary of ML models:

The AI techniques has the potential to address the challenges in material design. The
most promising one is ML, which can discover the mapping from high throughput input data to
output that is used to make decisions. In ML algorithms, the features extracted manually are used
for representation of input data, where as DL learn the representation of input data and can
represent it into multiple levels useful for complex neural network structures.

General ML approaches cn be classified into 3 categories.

1. Supervised learning: A task driven approach to map inputs to outputs with data being
labeled during training.
2. Unsupervised learning: A data driven approach trained with unlabelled data to search for
undetected patterns of the given dataset.

Both supervised and unsupervised learning evaluates the model performance by minimizing
loss/objective function.

Reinforcement learning focuses on interaction between agents with the environment. The
objective of reinforcement learning is to maximize the cumulative reward.

Semi-supervised learning is another category of ML approaches. It lies between

supervised and unsupervised learning and uses both labeled and unlabeled data (most
generally unlabeled) during training.

In the current field of mechanical materials design, supervised learning approaches are
most widely used as supervised tools are more accurate and mature to implement compared
to the tools in other categories.

ML approaches such as Linear regression (LIR), Support vector regression (SVR), Feed
forward neural networks (FFNNs), Multi layer perceptron (MLP), Convolutional neural
networks (CNNs) and Recurrent neural networks (RNNs) are generally used for supervised
learning. Typical approaches to unsupervised learning include K-means clustering, Auto
encoder and Generative adversarial networks (GANs).
 LIR is one of the simplest algorithms aimed to find a linear relationship between the
input features and continuous output. An extension to LIR is Polynomial regression (PR)
which includes polynomial terms in finding linear solutions. SVR and Random forest (RF)
are used to support non-linearity. These non-linear models usually handle outliers better and
show higher accuracy than linear models.

Apart from regression, the other major category of ML tasks is classification, used to
classify input into predefined categories instead of predicting specific values. Logistic
regression (LOR) is a classification algorithm with a loss function in logistic form. Decision
tree (DT) and Gradient boosting are other classical ML algorithms which can handle both
regression classification problems.

Beyond classical ML techniques, Artificial neural networks (ANNs) are developed,

loosely inspired by the inter connected neurons in brain for deep data mining, to learn non
linear relation between input and output. DL models are formed by increasing the depth of
layer-to-layer networks.

Feed forward networks (FFNNs) or multi layer perceptron (MLP) are simple DL models,
the information passes through network in a uni-directional manner. Each layer computes
output to next layer based on input from previous layer. The weights used for calculation of
each neuron are optimized to minimize the loss function. Back propagation (BP) is widely
used technique in ANN training, is implemented together with gradient descent algorithm
(GD) for minimum of the loss function.

Two types of DL architectures namely CNN and RNN are popular. CNNs are used in
face recognition, image classification and object detection, first introduced in 1980. CNNs
are image based Dl architecture by calculating mathematical operation ‘convolution’ to
extract features of images. In CNN, simple features will be gradually assembled to intact and
complicated ones by stacking the convolutional layers. This capacity of capturing the
features at different hierarchical levels makes CNNs well suited to describe the properties of
materials like bio-materials in material design problems.
 RNNs gain popularity due to the capacity of dealing with sequential data with the output
being dependent on previous and later sequence and utilize memories in determining output
of each layer or state. For RNNs with large depth, the gradient calculated by BP easily
vanishes.

Generative models have been established to generate new data points based on the
distribution of existing data, GANs. It consists of two neural networks, the generator and the
discriminator. The generator proposes new data instances and the discriminator compares the
generated data with the real data. GANs reach convergence when the generator and
discriminator are at Nash equilibrium.

Vaariational encoder (VAE) is another type of generative model that uses one neural
network which first encodes the input data into an in-explainable code named as latent code
and then decodes the latent code to reconstruct the output.

Gaussian process regression (GPR) based on Bayesian learning can provide uncertainty
measurement of prediction, potentially useful for mechanical material design problems as
they are suitable for relatively small datasets and work well without prior knowledge of
model forms.

In RL, the training process is aimed at finding a balance between exploration (unknown
territory) and exploitation (of current knowledge), find its potential applications to materials
problems such as interactive materials design.

Graph neural networks (GNNs), operate on graphs consisting of nodes connected by

edges without natural orders. Graph convolutional networks (GCNs) have the capability of
GNNs to learn graph embeddings through message passing between the nodes, potentially
applicable to many materials and mechanical problems that inherently consist of graph
structures.
Data collection:

Sufficient amount of data is pre-requisite for better models, and high quality data enable
the models to run efficiently. The data relate the mechanics problems to applied ML models,
it is crucial for ML based design of mechanical materials to know the amount of sufficient
data, data collection, quality and improvement of data. Data can be collected from literature
or existing materials data bases such as AFlow, Materials project (MP), MATDAT, Matweb
etc.,

These database consist of mechanical properties like elastic constants,

tensile/flexural/shear/fatigue strength, fracture toughness, hardness etc., MP database consist
of more than one hundred vanadium oxide materials along with various unique compositions.
Labeled datasets of copper alloys with different tensile strengths and electrical conductivities
can be obtained from literature.

Data generation:

Data can be generated by experiments or simulations, have more freedom to design the
features and control the size of datasets. The major challenge is to balance the expense of
data generation and the resulting performance of the applied ML model. The datasets
generated by using domain knowledge in material science and solid mechanics results in
more relevant data for better results from the applied models.

Computational methods can be used to simulate materials of interest and relate the
mechanical properties to the representative structures of the materials at different scale from
continuum to atomistic levels. Finite element methods (FEM) was used to generate datasets
of 3D micro structures of high contrast composites and meta materials.

Bench mark databases such as MNIST are particularly useful for comparing the accuracy
and efficiency of various ML techniques on specific tasks. The mechanical MNIST, labeled
by different forms of mechanical responses calculated from FEM simulations, can be used to
evaluate the performance of meta models of heterogeneous materials under large
deformation.
 Performing experiments to create sufficient large datasets for training DL models is
currently difficult due to the extremely high cost. Recently an autonomous research system
has been built to enable not only automated experimentation but also the selection of
subsequent experiments under a frame work of Bayesian optimization, which can be utilized
to mechanical materials design problems such as optimization of additive manufacturing
structures.

Data pre-processing:

In case the collected databases of images or texts are understandable for human but un-
interpretable for machines, the raw data required to be pre-processed before fed into the ML
model in acceptable data structure (ie., images, texts, grsphs). During pre-processing
augmentation techniques can be implemented to enlarge the datasets, and irrelevant data points
that would deteriorate the performance of the model should be removed. Image processing
techniques such as scaling and cropping were utilized to augment the initial dataset that might be
in-sufficiently large to train a DL model.

Less effort on pre-processing are required to design features for DL than conventional
ML methods due to the ability of DL models to parse the representation from simple to abstract
features through the training process. The techniques used to develop data driven solvers might
also inspire efficient methods to process sparse and noisy data of materials responses.

For the prediction of fracture patterns in brittle materials, the discrete atoms in a
triangular lattice, which is adopted from the MD simulations to generate the datasets of crack
patterns, were mapped into ordered pixels in an image that cannot only be treated as input to the
first convolutional layer of the applied LSTM models but also eliminate the irrelevant
information in the atomic structure other than the spatial features of the crack.
Applications:

The ML models, trained on datasets containing material information will give fast and
ccurate predictions of target mechanical properties or behaviours, or to discover compositions or
structures. Materials with complex and dis ordered micro structures such as glasses have large
data bases obtained from experiments or simulations focusing on composition-property
relationships. The concentrations of components are feature vectors for predicting the property of
these materials. For predicting the youngs modulus of silicate glasses, ML algorithms namely
PR, LASSO, RF and MLP are used.

FFNNs were trained to design 8 essential properties of oxide glasses like young’s
modulus, shear modulus and hardness using datasets. Design systems were developed using
neural networks for copper alloys that provide composition of new copper alloys with a target
ultimate tensile strength and electrical conductivity. For designing strong and conductive copper
alloys, ML models namely LIR, SVR, regression tree and GPR; dimensionality reduction
techniques such as principal component analysis, correlation based and genetic algorithm are
used.

For gradient non-structured metals, Gaussian process based active learning sarrogate
models were developed to study the structural gradient effects on strength and deformation
mechanisms. New super hard materials were proposed with the aid of ML techniques such as
SVR, evolutionary algorithms and GNNS. High entropy alloys predicted applied ML models
were synthesized, showing higher hardness values than any other sample in the training dataset.
ML model can be trained to capture the relationship between salient structural features and
mechanical properties.

The prediction of fracture toughness of poly crystalline silicon specimens is achieved

using two ML algorithms, RFs and FFNNs. Gradient boosting regressor is used to predict
mechanical properties of zeolite frame works using atomistic properties such as local properties
(bond length, angle and dihedrals), global properties (density or ring sizes distribution) and
porosity related properties. Stochastic gradient descent, k-nearest neighbours, SVR, DT and
ANN are used to predict mechanical properties using system temperature, strain rate, vacancy
defect and chirality.
 ANNs are used to establish relationship between structural or topological descriptors and
mechanical stability of thousands of MOF materials to predict bulk modulus. The elastic
deformation fields and effective elastic properties of high contrast two phase composites were
predicted using 3D CNN and datasets of 3D volume elements with different micro structures.
Convolutional networks with different architectures were used to predict the mechanical
properties of polymer nano-composites based on micro structure images, thermo and mechanical
properties of uni directional composites and stress fields in cantilever structures.

Ridge regression, gradient boosting and CNN can predict the effective yield strength of
additive manufactured metals. CNN are capable of learning higher level features directly from
image data and reducing the efforts on pre-processing and feature extraction when 3D images of
the micro structures represented by crystal orientation are given, perform well compared to
models fed with micro structural features. cGAN model is used to address the inverse eleasticity
problem of calculating elastic modulus distribution from observed displacement or strain fields
using non-destructive evaluation techniques. Deep learning approach is used to predict complex
strain or stress fields of hierarchical composites directly from geometric information.

For fracture problems, CNN based methodology is used to predict the micro structure
sensitive propagation of a 3D fatigue crack in a poly crystalline alloy based on the past crack
surface. ML based approach combining convolutional layers and LSTM used for predicting
fracture patterns, fracture toughness, crack length in crystalline solids based on atomistic
molecular simulations.

RNNs learn history dependent behaviours of heterogeneous RVEs loaded along different
deformation paths used for prediction of plasticity constitutive laws in an efficient and accurate
manner without adopting the widely used assumptions in existing plasticity theories. Deep
residual network is used to predict crystal plasticity using high throughput discrete dislocation
simulations. RNN based on GRU used to predict the stress-strain evolutions of elasto-plastic
composite RVEs subjected to random loading paths. ANNs are used to construct constitutive
laws for isotropic hardening elasto plastic materials with complex micro structures. DL models
are used to find the inverse solution to collision load conditions with the post-collision plastic
deformation of shell structures given.
 CNNs are used to design tessellate composites with optimized strength and fracture
toughness. CNNs integrated with GA approach is used, CNN applied to extract local patterns of
composite around crack tip in frame work, GA to accelerate the search process using ML
prediction as fitness function for optimization.

Encoder and decoder frame works based on convolutional layers can be employed to
accelerate the process of topography optimization of mechanical structures. Deep neural
networks are used to provide optimal topologies for desired properties of spinodoid meta
materials. Bayesian ML methods are useful for data driven designs of super compressible and
recoverable metal materials made of brittle polymeric base materials. Deep material network is
used for structure property predictions of heterogeneous materials under the effects of non linear,
failure and interfacial behaviours.
Perspectives:
A straight forward benefit ML brings to materials and mechanics researches is promoting
the efficiency of materials designs using experiments and simulations. ML based design
approaches can incorporate materials and mechanical features during the pre-processing of input
data, learn the relationship between material structures and mechanical behaviours during
training and provide targeted designs using the trained models.
The performance of ML methods is elevated by encoding scientific knowledge in data
pre-processing and in neural network architectures. The physics informed neural networks
(PINNs) in which governing equations in the form of PDEs are incorporated into loss function
offer new perspectives for integration of ML and mechanical materials design. Symbolic
regression is trained to learn symbolic expression of physical laws to predict well known physics
concepts like lagrangian. Recent ML work using GNN are capable to discover new analytical
solutions for dark matter mass distribution by deriving governing equations in a unique way to
offer potential new direction for understanding the mechanisms and mechanical behaviours of
various materials.
Spike neural networks (SNNs) transmit sparse and asynchronous binary signals between
neurons which incorporate time into deep learning networks. They exhibit favourable properties
including low power consumption, fast inference and event driven information processing.
The potential of using ML in design of mechanical materials has not been fully exploited
yet with opportunities and challenges lying ahead to be explored and over come.