Professional Documents
Culture Documents
011 Machine Learning Approaches in Reliability - Shaomin Wu (2020)
011 Machine Learning Approaches in Reliability - Shaomin Wu (2020)
Abstract—Reliability and maintenance (R&M) engineering is relationship /function between x and y. It can be further
conventionally notorious for a lack of sufficient failure data to categorised to regression modelling and classification
develop robust statistical models. The increasing miniaturization modelling. The main difference between the regression
of data collection devices such as wireless sensors has provided
a promising infrastructure for gathering information about and classification modelling techniques is the levels of
parameters of the physical systems, which enable practitioners measurement of y: the supervised learning is regression
and researchers to apply machine learning (ML) algorithms to modelling if y is a continuous variable; the supervised
improve the efficiency of R&M analysis. The number of published learning is a task of classification modelling if y is a
papers on ML in R&M is enormous, this paper will therefore discrete variable.
categorizes those papers that were published between 2017 to
16/May/2020, that are written in English, that have received a Some modelling techniques are specifically designed for
top 5% number of citations in the year published, and that use these two tasks, respectively. For example, many machine
support vector methods, random forests, and cluster analysis. learning text books include linear regression modelling
Index Terms—Machine learning; Reliability estimation; Con- and logistic regression modelling methods, which are typ-
dition monitoring; Fault diagnosis; Maintenance policy ical examples of regression and classification modelling
methods, respectively. Nevertheless, some ML algorithms
I. I NTRODUCTION can also be used for both regression and classification
Machine learning (ML) is a discipline of computer science modelling, for example, support vector methods and
that designs algorithms for extracting useful information from decision trees (e.g., random forests).
data. It has been a thematic research area in the last two Unsupervised Learning In case the output y in the input-
decades and found applications in many industrial sectors output pair x and y is unknown, unsupervised learning
and academic disciplines. ML algorithms have therefore been may be applied. Examples of unsupervised learning tasks
used in many disciplines, including reliability and maintenance include cluster analysis and others.
(R&M), on which this paper gives a brief review of its state- Reinforcement Learning Reinforcement learning is de-
of-the-art development. signed to learn actions in an uncertain, potentially com-
The advance of data collection technologies makes it pos- plex environment to maximise reward.
sible to monitor the conditions of physical systems. As a
The reader is also referred to [1]–[3] for machine learning
result, large volumes of reliability and maintenance related
textbooks, [4]–[7] for a review papers on the applications of
data can be collected. ML algorithms have therefore gradually
machine learning in R&M.
been adapted in reliability analysis and maintenance policy
The number of publications on the application of ML
optimisation.
algorithms in R&M is vast. It is unlikely for us to be able
In the literature, two other terms data mining and artifi-
to review all of the existing publications on the subject topic.
cial intelligence have similar meanings and coverage similar
Nevertheless, based on the evaluation of 179 classification
research areas as machine learning and they are commonly
algorithms which are tested on 121 datasets, [8] concludes
exchangeably used.
that, within the 179 algorithms, SVM and random forests out-
ML chiefly covers three subdisciplines: supervised learning,
perform the others, whereas nearest neighbours and Bayesian
unsupervised learning and reinforcement learning.
underperform. To gain a rough impression of the numbers of
Supervised Learning Supervised learning is performed the papers published, we run six queries in www.scopus.com,
based on the observations {xi , yi }ni=1 of the input-output with the following conditions:
pair x and y, based on which it aims to learn a
(a) indexed by www.scopus.com;
The first author would like to acknowledge support from the ESRC Grant (b) published between 2017 and 16/May/2020;
with No. ES/L011859/1, UK. (c) written in English;
(d) published in journals;
(e) included one of the following keyword combinations: A. Support vector methods
(i) “support vector” AND (reliability OR maintenance); Support vector methods, including support vector regression
(ii) “random forest” AND (reliability OR maintenance); (SVR) and support vector machine (SVM), are a supervised
(iii) “cluster analysis” AND (reliability OR maintenance). ML technique. SVR is a modelling algorithm for regression
We search http://www.scopus.com and found the and SVM is for classification. The main difference between
numbers of publication shown in Table I. For statistical methods (such as multiple linear regression and
example, the last row was obtained by using the logistic regression) and the support vector methods lie in the
query string ((TITLE-ABS-KEY(”cluster analysis”)) use of the loss function: Unlike the multiple linear regression
AND (TITLE-ABS-KEY(reliability) OR TITLE-ABS- in which the least squares function is used as the loss function
KEY(maintenance)) AND (LIMIT-TO(PUBYEAR,2020) and the logistic regression whose loss function is the softmax
OR LIMIT-TO(PUBYEAR,2019) OR LIMIT- function, support vector methods use the -intensive loss
TO(PUBYEAR,2018) OR LIMIT-TO(PUBYEAR,2017)) function.
AND (LIMIT-TO(LANGUAGE,”English”)) AND (LIMIT- We have reviewed 25 papers, each of which has received
TO(SRCTYPE,”j”))) and we have some papers in other a good number of citations and listed their applications,
research fields such as earth science. methods, and sources of the dataset used in the paper in Table
II.
TABLE I
T HE NUMBERS OF PAPERS WITH SOME OF THE ABOVE - MENTIONED TABLE II
CONDITIONS . S UPPORT VECTOR METHODS IN R&M.
Search condition Number of papers Applications Methods Dataset
(a) AND (e).(i) 937 Assessment of health state SVR [14] Real-world [14]
(a) AND (e).(ii) 146 Estimation of remaining SVR [15]–[18] Turbofan [15],
(a) AND (e).(iii) 275 useful life Real-world [16],
(a) AND (b) AND (c) AND (d) AND (e).(i) 473 [17], Simulation
(a) AND (b) AND (c) AND (d) AND (e).(ii) 100 [18]
(a) AND (b) AND (c) AND (d) AND (e).(iii) 85 Fault diagnosis / detection one-class SVM [19], Simulation [21],
[20], binary tree SVM Real-world [19]–
[21], SVM [22]–[25], [27]
Table I shows that the number of application of ML relating SVR [26], [27]
to R&M is vast. Th existing literature also shows that the main Information fusion SVR [28], [29] Real-world [28],
[29]
application of ML relating to R&M are in reliability assess- Reliability assessment / Adaptive SVM [30], Simulation [30],
ment, condition monitoring, fault diagnosis, and maintenance fault prediction SVR [31]–[34], [32], [34], [35],
scheduling. As such, this paper gives a brief review on the wavelet SVM [35], Real-world [31],
least squares SVM [33], [36]
applications of ML in these four areas (despite the fact that [36]
in some cases those four areas may not have clear boarders) Vibration trend prediction least square SVM Real-world [37],
based on the papers satisfying conditions (a–d) and the two [37], SVR [38] [38]
following conditions.
(f) received a top 5% number of citations in the year pub- B. Random forests
lished; and
(g) focused on R&M of physical systems. Decision trees are a popular supervised ML technique. It
learns a tree-like structured graph from a training dataset. A
The remainder of this paper is structured as follows. Section
random forest is an ensemble of trees that are learnt from the
II categorises the literature on the applications of support
data sampled (without replacement) from the training dataset.
vector methods, random forests, and cluster analysis in R&M,
As [8] concludes that, within the 179 algorithms, random
respectively. Section III closes the paper and suggests further
forests have outstanding performance. However, it seems it is
research.
not so popular as the support vector methods in R&M. Table
II. T HREE ML TECHNIQUES IN R&M III lists its applications in R&M.
Authorized licensed use limited to: Lulea University of Technology. Downloaded on November 01,2020 at 15:25:41 UTC from IEEE Xplore. Restrictions apply.
2020 Asia-Pacific International Symposium on Advanced Reliability and Maintenance Modeling (APARM)
Authorized licensed use limited to: Lulea University of Technology. Downloaded on November 01,2020 at 15:25:41 UTC from IEEE Xplore. Restrictions apply.
2020 Asia-Pacific International Symposium on Advanced Reliability and Maintenance Modeling (APARM)
[18] L.-L. Li, Z.-F. Liu, M.-L. Tseng, and A. S. Chiu, “Enhancing the support vector regression improved by optimization algorithms,” Natural
lithium-ion battery life predictability using a hybrid method,” Applied Resources Research, pp. 1–24, 2019.
Soft Computing, vol. 74, pp. 110–121, 2019. [39] D. Wu, C. Jennings, J. Terpenny, R. X. Gao, and S. Kumara, “A com-
[19] K. Yan, Z. Ji, and W. Shen, “Online fault detection methods for parative study on machine learning algorithms for smart manufacturing:
chillers combining extended kalman filter and recursive one-class SVM,” tool wear prediction using random forests,” Journal of Manufacturing
Neurocomputing, vol. 228, pp. 205–212, 2017. Science and Engineering, vol. 139, no. 7, 2017.
[20] F. Harrou, A. Dairi, B. Taghezouit, and Y. Sun, “An unsupervised [40] A. Peimankar, S. J. Weddell, T. Jalal, and A. C. Lapthorn, “Evolutionary
monitoring procedure for detecting anomalies in photovoltaic systems multi-objective fault diagnosis of power transformers,” Swarm and
using a one-class support vector machine,” Solar Energy, vol. 179, pp. Evolutionary Computation, vol. 36, pp. 62–75, 2017.
48–58, 2019. [41] D. Zhang, L. Qian, B. Mao, C. Huang, B. Huang, and Y. Si, “A data-
[21] Y. Li, G. Li, Y. Yang, X. Liang, and M. Xu, “A fault diagnosis scheme driven design for fault detection of wind turbines using random forests
for planetary gearboxes using adaptive multi-scale morphology filter and xgboost,” IEEE Access, vol. 6, pp. 21 020–21 031, 2018.
and modified hierarchical permutation entropy,” Mechanical Systems and [42] Z. Chen, F. Han, L. Wu, J. Yu, S. Cheng, P. Lin, and H. Chen, “Random
Signal Processing, vol. 105, pp. 319–337, 2018. forest based intelligent fault diagnosis for pv arrays using array voltage
[22] S. E. Pandarakone, Y. Mizuno, and H. Nakamura, “Distinct fault analysis and string currents,” Energy conversion and management, vol. 178, pp.
of induction motor bearing using frequency spectrum determination and 250–264, 2018.
support vector machine,” IEEE Transactions on Industry Applications, [43] M. Li, D. Yu, Z. Chen, K. Xiahou, T. Ji, and Q. Wu, “A data-driven
vol. 53, no. 3, pp. 3049–3056, 2016. residual-based method for fault diagnosis and isolation in wind turbines,”
[23] M. M. Islam and J.-M. Kim, “Reliable multiple combined fault diagnosis IEEE Transactions on Sustainable Energy, vol. 10, no. 2, pp. 895–904,
of bearings using heterogeneous feature models and multiclass support 2018.
vector machines,” Reliability Engineering & System Safety, vol. 184, pp. [44] P. Marcelino, M. de Lurdes Antunes, E. Fortunato, and M. C. Gomes,
55–66, 2019. “Transfer learning for pavement performance prediction,” International
[24] M. Dey, S. P. Rana, and S. Dudley, “Smart building creation in Journal of Pavement Research and Technology, vol. 13, no. 2, pp. 154–
large scale hvac environments through automated fault detection and 167, 2020.
diagnosis,” Future Generation Computer Systems, 2018. [45] P. Hundi and R. Shahsavari, “Comparative studies among machine
[25] A. Kouadri, M. Hajji, M.-F. Harkat, K. Abodayeh, M. Mansouri, learning models for performance estimation and health monitoring of
H. Nounou, and M. Nounou, “Hidden markov model based principal thermal power plants,” Applied Energy, vol. 265, p. 114775, 2020.
component analysis for intelligent fault diagnosis of wind energy con- [46] R. Accorsi, R. Manzini, P. Pascarella, M. Patella, and S. Sassi, “Data
verter systems,” Renewable Energy, 2020. mining and machine learning for condition-based maintenance,” Proce-
[26] S. Jiang, M. Lian, C. Lu, S. Ruan, Z. Wang, and B. Chen, “SVM-DS dia Manufacturing, vol. 11, pp. 1153–1161, 2017.
fusion based soft fault detection and diagnosis in solar water heaters,” [47] K. Van Tricht, A. Gobin, S. Gilliams, and I. Piccard, “Synergistic use
Energy Exploration & Exploitation, vol. 37, no. 3, pp. 1125–1146, 2019. of radar sentinel-1 and optical sentinel-2 imagery for crop mapping: A
[27] F. Wang, Z. Chen, and G. Song, “Monitoring of multi-bolt connection case study for belgium,” Remote Sensing, vol. 10, no. 10, p. 1642, 2018.
looseness using entropy-based active sensing and genetic algorithm- [48] H. Gong, Y. Sun, X. Shu, and B. Huang, “Use of random forests
based least square support vector machine,” Mechanical Systems and regression for predicting iri of asphalt pavements,” Construction and
Signal Processing, vol. 136, p. 106507, 2020. Building Materials, vol. 189, pp. 890–897, 2018.
[28] J. Wang, J. Xie, R. Zhao, L. Zhang, and L. Duan, “Multisensory fusion [49] S. Sharma, Y. Cui, Q. He, R. Mohammadi, and Z. Li, “Data-driven
based virtual tool wear sensing for ubiquitous manufacturing,” Robotics optimization of railway maintenance for track geometry,” Transportation
and Computer-Integrated Manufacturing, vol. 45, pp. 47–58, 2017. Research Part C: Emerging Technologies, vol. 90, pp. 34–58, 2018.
[29] X. Zhang, S. Mahadevan, N. Lau, and M. B. Weinger, “Multi-source [50] J. Zhou, E. Li, H. Wei, C. Li, Q. Qiao, and D. J. Armaghani, “Random
information fusion to assess control room operator performance,” Reli- forests and cubist algorithms for predicting shear strengths of rockfill
ability Engineering & System Safety, vol. 194, p. 106287, 2020. materials,” Applied Sciences, vol. 9, no. 8, p. 1621, 2019.
[30] Q. Pan and D. Dias, “An efficient reliability method combining adaptive [51] Z. A. Bukhsh, A. Saeed, I. Stipanovic, and A. G. Doree, “Predictive
support vector machine and monte carlo simulation,” Structural Safety, maintenance using tree-based classification techniques: A case of railway
vol. 67, pp. 85–95, 2017. switches,” Transportation Research Part C: Emerging Technologies, vol.
[31] M. Baptista, S. Sankararaman, I. P. de Medeiros, C. Nascimento Jr, 101, pp. 35–54, 2019.
H. Prendinger, and E. M. Henriques, “Forecasting fault events for pre- [52] M. Diakhate, E. Bastidas-Arteaga, R. M. Pitti, and F. Schoefs, “Cluster
dictive maintenance using data-driven techniques and arma modeling,” analysis of acoustic emission activity within wood material: Towards
Computers & Industrial Engineering, vol. 115, pp. 41–53, 2018. a real-time monitoring of crack tip propagation,” Engineering Fracture
[32] J. Feng, L. Liu, D. Wu, G. Li, M. Beer, and W. Gao, “Dynamic Mechanics, vol. 180, pp. 254–267, 2017.
reliability analysis using the extended support vector regression (x-svr),” [53] J. Zhang, N. Jiang, H. Li, and N. Li, “Online health assessment of wind
Mechanical Systems and Signal Processing, vol. 126, pp. 368–391, 2019. turbine based on operational condition recognition,” Transactions of the
[33] A. Roy, R. Manna, and S. Chakraborty, “Support vector regression Institute of Measurement and Control, vol. 41, no. 10, pp. 2970–2981,
based metamodeling for structural reliability analysis,” Probabilistic 2019.
Engineering Mechanics, vol. 55, pp. 78–89, 2019. [54] R. R. de la Hermosa González et al., “Wind farm monitoring using
[34] Q. Wang, Q. Li, D. Wu, Y. Yu, F. Tin-Loi, J. Ma, and W. Gao, “Machine mahalanobis distance and fuzzy clustering,” Renewable energy, vol. 123,
learning aided static structural reliability analysis for functionally graded pp. 526–540, 2018.
frame structures,” Applied Mathematical Modelling, vol. 78, pp. 792– [55] Y. Peng, J.-R. Lin, J.-P. Zhang, and Z.-Z. Hu, “A hybrid data mining
815, 2020. approach on bim-based building operation and maintenance,” Building
[35] H. Dai and Z. Cao, “A wavelet support vector machine-based neural and environment, vol. 126, pp. 483–495, 2017.
network metamodel for structural reliability assessment,” Computer- [56] X. Wang, J. Li, and X. Shang, “The state subdivision of public
Aided Civil and Infrastructure Engineering, vol. 32, no. 4, pp. 344–357, traffic vehicles based on k-means algorithm,” International Journal of
2017. Manufacturing Technology and Management, vol. 33, no. 3-4, pp. 133–
[36] D. Liu, Y. Song, L. Li, H. Liao, and Y. Peng, “On-line life cycle 149, 2019.
health assessment for lithium-ion battery in electric vehicles,” Journal [57] W. Wang, B. Lou, X. Li, X. Lou, N. Jin, and K. Yan, “Intelligent
of Cleaner Production, vol. 199, pp. 1050–1065, 2018. maintenance frameworks of large-scale grid using genetic algorithm and
[37] W. Fu, K. Wang, C. Li, X. Li, Y. Li, and H. Zhong, “Vibration trend k-mediods clustering methods,” World Wide Web, vol. 23, no. 2, pp.
measurement for a hydropower generator based on optimal variational 1177–1195, 2020.
mode decomposition and an lssvm improved with chaotic sine cosine [58] S. K. Chelliah, P. Parameswaran, S. Ramasamy, A. Vellayaraj, and
algorithm optimization,” Measurement Science and Technology, vol. 30, S. Subramanian, “Optimization of acoustic emission parameters to
no. 1, p. 015012, 2018. discriminate failure modes in glass–epoxy composite laminates using
[38] H. Yang, H. N. Rad, M. Hasanipanah, H. B. Amnieh, and A. Nekouie, pattern recognition,” Structural Health Monitoring, vol. 18, no. 4, pp.
“Prediction of vibration velocity generated in mine blasting using 1253–1267, 2019.
Authorized licensed use limited to: Lulea University of Technology. Downloaded on November 01,2020 at 15:25:41 UTC from IEEE Xplore. Restrictions apply.
2020 Asia-Pacific International Symposium on Advanced Reliability and Maintenance Modeling (APARM)
Authorized licensed use limited to: Lulea University of Technology. Downloaded on November 01,2020 at 15:25:41 UTC from IEEE Xplore. Restrictions apply.