Proceedings of the 2022 14th International Pipeline Conference
IPC2022
NOT ALL DATA IS GOOD DATA: THE CHALLENGES OF USING MACHINE LEARNING
WITH ILI.
Adrian Belanger Dane Burden
T.D. Williamson T.D. Williamson
Houston, TX, USA Salt Lake City, UT, USA
ABSTRACT
Machine learning (ML) has grown extensively in most
industries, with learning models driving state of the art
performance in a variety of tasks. The in-line inspection (ILI)
industry is no exception: applications of machine learning
techniques have provided promising results for a wide range of
needs. Metal loss anomaly sizing, fitting classification, and
identification of interacting threats have all benefited from
different forms of learning models. The success of any learning
model requires detailed attention at all stages of the process, as
small nuances often manifest misleading results.
The ability of an ILI data analyst to accurately identify
pipeline anomalies, most importantly anomalies that affect the
integrity of a pipeline, is based on experience. Experience can
be sub-divided into two categories, observations and truth,
which allow a data analyst to identify patterns and make
predictions. Supervised learning models mimic this process by
using a mapped set of inputs (observations) to outputs (truth) to
develop a mathematical function that can be applied to new
examples. The input parameters, referred to as training data, are
a vector of engineered features relating to the desired output.
Like a data analyst’s experience, the quantity, quality and
representation of the training data directly influence the
performance of predictions made by the model.
Data curation, a process that includes the collection,
analysis and labeling of data used to train the mathematical
model, is critical and time-consuming. When curating data for
the development of a supervised model, two main obstacles must
be overcome. First, data from two measurement methods, for
example, magnetic flux leakage (MFL) and in-the-ditch non-
destructive examination (NDE), must be precisely correlated.
Second, the training data must represent the population of future
predictions and minimize coverage error by containing enough
appropriate examples. Metal loss anomalies examined by NDE
September 26 – 30, 2022, Calgary, Alberta, Canada
IPC2022-86934
Paul Dalfonso
T.D. Williamson
Salt Lake City, UT, USA
are correlated to the predicted geometries of interacting ILI
signatures that do not always align one-to-one. The estimated
position on the pipe is made from onboard ILI tool instruments
and is subject to measurement accuracy. The small errors
contained in each measurement method can make precisely
matching the NDE measurements to the ILI measurements a
tedious effort and minimizing the compounding errors requires
diligence.
The one-to-many and direct-to-indirect measurement
relationships between NDE and ILI make generalizing input
features and labeling training data a challenging task. In
addition to overcoming the difficulties associated with
correlating and labeling training data, significant attention must
also be given to the distribution of features being represented.
This includes metal loss geometry as well as the engineered
model input features. With dozens of dimensions included in the
input vector of supervised learning models, representing the
possible permutations can be overwhelming. Using NDE
external laser scans to maximize training data and
dimensionality reduction techniques can help, but do not remove
all underlying contributions.
After curating, developing, and training a model, it is
possible to assess the associated performance, but this should be
approached with cautious optimism. To accomplish this, a
general machine learning practice is to hold out a portion of the
training data to test model predictions while an iterative
development cycle refines the engineered features, model
architecture and training process. Since no two pipelines or
inspections are identical, validation against blind data is crucial.
Although the standard training hold-out practice may make data
appear to be blind, it often includes highly correlated features
and the iterative development cycle may inadvertently lead to
bias. Attention to the distribution of blind hold-out data helps
1 © 2022 by ASME
verify the generalization of a model and more accurately
represents future performance.
This paper describes the development of an ILI machine
learning model at each stage of the process and contrasts the
performance of correctly utilizing data in training and testing to
that of incorrectly utilized data.
NOMENCLATURE
AI Artificial Intelligence
AUT Automated Ultrasonic Technology
DEF Internal Geometry or Deformation
Measurements
DVP Dig Verification Program
EMAT Electromagnetic Acoustic Testing
GPS Global Positioning System
GPU Graphical Processing Unit
ILI In-line Inspection
KNN K-Nearest Neighbors Classification Model
LFM Low Field Magnetic Flux Leakage
MDS Multiple Dataset
MFL Magnetic Flux Leakage
NDE Non-Destructive Examination
PAUT Phased Array Ultrasonic Technology
SMFL Spiral Magnetic Flux Leakage
UT Ultrasonic Testing
1. INTRODUCTION
Inspection of steel pipelines with magnetic flux leakage
(MFL) tools has been around since the 1960s. At first, the data
from these tools was analog with coarse circumferential and
axial resolutions measured by induction coil sensors. The sizing
algorithms used were mostly based on the amplitude of the
recorded signals with a lot of analyst interpretation. Since then
things have changed considerably. During the 1990’s two
revolutions occurred in the in-line inspection (ILI) community:
advances in measurement technologies with the application of
hall sensors and ultrasonic testing (UT), and the computer
revolution that ushered in powerful desktop digital processing.
The latter continued into the 21st century with data processing
power growing exponentially and breakthroughs in artificial
intelligence (AI) including machine learning and deep learning.
Now, in the 2020s, we are seeing the impact of these
advancements over a range of ILI technologies used in pipeline
integrity [1]‒[4].
Machine learning tools are powerful but “with great power
there must also come great responsibility.” [5] There are two
regimes that must be addressed in building a reliable machine
learning algorithm: the type of machine learning methods to use
and the proper curation of truth data.
2. A CURSORY WALK-THROUGH OF LEARNING
MODELS
In early days of ILI, depth prediction algorithms were
mainly some combination of regression models. Manufactured
anomalies were put into pipe with very controlled dimensions,
which allowed for the construction of regression curves based on
the standard reported features of depth, length and width.
Regression models have been used as far back as Gauss in his
modeling of the heavenly bodies [6] and are a staple of
empirically derived engineering models [7].
As computational speeds and memory advanced, so did the
methodologies that used them. AI got its real start when
mathematician and computer scientist Alan Turning asked the
question, “Can machines think?” [8] Symbolic AI was the first
attempt at modeling how to think using predefined rules, much
as humans do with logic, mathematics and language.
Symbolic AI developed considerably over the course of the
1960s, but due to the a priori knowledge that had to be
programmed into these systems they began to be supplanted by
neural networks. Neural networks are a form of AI called
machine learning designed to learn from data that is presented to
it. There is no need for machine learning to be programmed with
explicit sets of instructions and symbols.
Neural networks were used on many ILI problems from
classifying features to determining the severity of mechanical
damage [9].Neural networks derive their name from the original
idea to mimic the connections of neurons in the brain. However,
even today these methods are grossly oversimplified when
compared to the human brain. Neural networks comprise layers,
each one containing a node (see Figure 1). The node is basically
a function that takes in inputs, processes them and then sends out
outputs that may be the inputs to another layer. These nodes
have numerical coefficients associated with them and they
“learn” by adjusting these coefficients to match their outputs to
the expected outputs they are being trained on. The data set used
for this learning step is called the training set. After being
trained, the model outputs are compared to data it has not been
trained on, called the validation set.
FIGURE 1: SCHEMATIC OF NEURAL NETWORK
2 © 2022 by ASME
 There have been many adaptations and variations on the generalize well to new examples. This is generally overcome by
regression fitting of nodes, including trees, forest, boosting. and, introducing additional training examples that more
most recently, deep learning. comprehensively represent the prediction space. The quantity of
Trees and forests use nodes but instead of assembling them training data may determine how sophisticated the model can be.
in layers they structure them in branches and leaves. Boosting A small data set may be more prone to overfitting when using
guides the learning process by combing the outputs of many deep learning techniques, whereas it may be more appropriate to
weak learners to obtain an overall strong model. Deep learning, apply random forest or gradient boosting [10].
the next generation of neural nets, exploits the ability to mine
large amounts of data using the computational power of
specifically engineered processing units including graphical
processing units (GPU). Deep learning models have enormous
amounts of nodes in many layers densely connected to each
other.
Another technique worth mentioning are support vector
machines, which belong to a category of pattern recognition
algorithms called kernels. This technique takes in an array of
features for each data point and operates on the distances
between these data points, treating the feature array as a
coordinate in a higher dimensional space. Support vectors are
(a) PIPELINE (A) METAL LOSS (b) PIPELINE (A)
very good at grouping similar data points and separating groups
DEPTH DISTRIBUTION RESIDUAL ERRORS
into classes.

2.1 Representing the Prediction Space

When choosing an machine learning model, it is very
important to understand the nature of the data that will provide
the inputs into the model and what the model is expected to
predict. Are you trying to predict a feature type such as a mill
anomaly from an internal pit? Are you trying to estimate the
dimensions of a corrosion pit? Or are you more interested in the
estimated severity of a mechanical damage event?
It is important to define the prediction space, that is, the set
of all expected predictions of the model, because overly
generalizing the space can miss critical exceptions and trying to
model every possible prediction can make the problem
intractable.
The type of prediction is also important. There are three
basic types of predictions: classification, clustering and
regression. Classification matches the data to known classes
based on how well those features match data known to belong to
that class. Clustering matches how well data points match each
other and makes distinctions between different populations.
Regression makes quantitative predictions based on the
relationships of previously measured data matched to known
predictions.
In addition, there are supervised and unsupervised learning.
If the training data has known properties, then the model can be
trained to predict known data; this is called supervised learning.
Unsupervised learning is a method of finding patterns and
features in data that do not have known properties.
Choosing an machine learning model can be dictated by the
data available for training and the complexity of the problem to
solve. In addition to the type of predictions expected by the
model, the quantity and quality of data points used in training
and validating are critical factors. If the problem is very complex
or there is high correlation between the sets of data, then the
prediction space will be too narrow, and a model will not
(d) PIPELINE (B) METAL
(c) COMPARISON OF
PIPELINE (A) & (B) LOSS ERRORS USING
METAL LOSS DEPTH MODEL TRAINED ON
DISTRIBUTION PIPELINE (A)
FIGURE 2: COMPARISON OF A MACHINE LEARNING MODEL
TRAINED ON A SPECIFIC PIPELINE APPLIED TO ANOTHER.
The choice of model is important, but the true strength of
machine learning is the data used to train the model. The
prediction space needs to effectively represent the expected
predicted values that the model may return. Using large amounts
of data over a small subset of the whole prediction space will not
produce accurate results for data outside of that training set.
Training data can also have unknown biases that do not show up
until the model is presented with new data that has different
parameters. Figure 2 shows results of training a model using only
data from one pipeline and validating it on a completely different
pipeline. The training set used to build the model (figure 2a)
shows excellent residual errors (figure 2b) with 80% of the
populations within ±0.05t. However, when this model is applied
to another pipeline with different conditions, such as grade of
pipe and distribution of corrosion types, it performs sub-
optimally with a bias to overcall depths (figure 2d). The solution
to this is either to train the model to be pipeline independent or

3 © 2022 by ASME
to detect and tune to pipeline differences by training it on data a digital gage to the pipe with a magnetic bridging bar can ensure
across all the cases to be inspected. a more level mounting on the pipe surface, reducing errors due
to rocking or tilting of the gage. Both gages are limited, however,
3. DATA CORRELATION by the width of the point extending into the metal loss.
ILI uses various technologies to gather data on the condition Because measuring only the deepest point can be difficult
of a pipeline while traveling internally through it. Technologies and it ignores other zones in the area, aligning the appropriate
include magnetic flux leakage (MFL), ultrasonic testing (UT), ILI signal associated with the actual deepest metal loss region
electromagnetic acoustic testing (EMAT) and internal geometry may not be feasible. Since areas of corrosion often manifest
measurements (DEF). Inspections can be done with separate multiple ILI signal responses there can be a one-to-many
tools; however, their results must be aligned for comprehensive relationship of NDE-to-ILI. This can lead to correlation errors
integrity analysis. Aligning data requires matching the position between ILI and NDE by assigning an ILI training sample
and orientation of each inspection and accounting for the errors (signal) to the incorrect target NDE value (measured depth).
associated with each tools’ method of measuring axial and
circumferential orientation. Pattern matching techniques of the
datasets can be used to refine alignment, but measurement
uncertainty will always be present, which complicates utilizing
separate inspections. The error resulting from matching ILI to
ILI or NDE to ILI will be referred to as correlation error. One
way to minimize ILI to ILI correlation error is to run multiple
technologies on a single inspection tool.
When an ILI is completed using multiple datasets, a more (a) MANUAL GAGE (b) DIGITAL GAGE
comprehensive view of what is happening on the pipeline can be FIGURE 3: EXTERNAL PIT GAGES
visualized.[11] This allows for the development of more
comprehensive machine learning model inputs. By capturing
NDE data collected at precisely known locations on the pipe, an
alignment can also be made with each of the ILI datasets,
building up the prediction space and reducing the error
associated with correlation.

4. USING NDE VALIDATION DATA

In the early days of ILI, NDE validation digs were done to
assess the performance of ILI tools and recalibrate an ILI run if
the results were not within the tool stated specifications. Since
then, the accuracy of both ILI tools and NDE examination
techniques has continued to improve, generally through
technology enhancements. However, improvements to ILI
predictions also rely heavily on the accuracy of NDE
measurements, as these are the target data for model training and
testing. Although some NDE technologies are extremely
accurate, the contributions from in-the-ditch environments,
human factors, time constraints and feature correlations present
opportunities for additional error [12] to be included in the model
development. Therefore, careful data curation is a crucial step
when developing any predictive model. Curation is the process
of aligning ILI and NDE data sets and evaluating the quality of
both to determine the applicability of labeling target data.
4.1 Manual Tools
Analog and digital pit gages (Figures 3a & 3b) are
commonly used in a standard excavation dig to measure metal
loss due to external corrosion. Typically, only the deepest point
in an area of general corrosion will be measured or . Drawbacks
to using either type of pit gage exist [13]. For example, analog
pit gages rest on the outer surface of the pipe, so if corrosion is
extensive, the gage may not sit level with the nominal surface of
the pipe and will not accurately measure the pit depth. Securing
FIGURE 4: MANUAL UT PROBE
Internal measurements have similar limitations as a hand-
held UT probe (Figure 4) is used to find and size the anomaly
and is subject to errors in positioning and coupling to the surface
of the pipe. In addition, probe measurements are typically taken
on a grid and are limited by the grid resolution. In a conventional
UT scan the probe used is a straight (or normal) beam which
reflects off the bottom surface of internal corrosion, returning a
measurement of the remaining pipe wall. This method can miss
pinhole pitting, narrow mill anomalies or even the peak depth
that may be detected by MFL. Generally, only the maximum
measured depths are recorded in the dig reports, with little else
about the morphologies or sizes of the anomalies. This limits the
available data from which to develop length and width models.
4.2 Automated Tools
Since the late 1990s, automated systems have been used to
map pipe surfaces. Laser and coherent light profilometry is used

4 © 2022 by ASME
to map the external surface [13] and automated ultrasonic
technology (AUT) [14] maps the internal surface.
Figure 5 shows the use of a handheld profilometry unit in
the ditch and the associated digital output of the scanned area.
The pipe is prepared by removing a small area of coating and
placing some magnetic markers for reference calibration. The
unit is manually positioned and the software automatically
calculates the surface profile of the pipe at resolutions much finer
than can be achieved with pit gages. The data is stored in digital
formats that can easily be converted into usable formats by ILI
providers and integrity consultants. Figure 6 shows an AUT
setup that can provide resolution and digital outputs similar to
that of laser profilometry.
FIGURE 5: OPTICAL PROFILOMETRY
FIGURE 6: PROFILE OF INTERNAL CORROSION USING AUT
Figure 7 shows an alignment using high resolution
profilometry data (Creaform HandySCAN) and the
corresponding MFL dataset from a T.D. Williamson (TDW)
multiple dataset (MDS) tool in a joint of test pipe. Standard ILI
and NDE reports use boxes to identify and correlate metal loss
features, with NDE generally only providing information on the
deepest feature in the area. This is a poor utilization of the true
power of using profilometry data to validate ILI results, as
evident in Figure 7, where far more comparisons could be made
if signal to signal matching occurred. In cases where corrosion
is complex, for example, matching boxes can lead to
misalignments as overlapping areas can include pits and signals
not used in the calculation of the boxes. Using actual depth
measurements at each axial and circumferential position allows
metal loss to be mapped directly to the ILI response location.
The features from the ILI data can then be mapped directly to the
NDE data, minimizing the correlation error, maximizing the
number of correlated points and creating a larger model
prediction space to be developed.
FIGURE 7: ALIGMENT OF ILI DATA AND PROFILOMETRY
5. The Dig Verification Program
Dig verification is a process where an operator exposes the
pipe and performs NDE to validate the calls made using an ILI
tool.
Traditionally, the dig verification process has been
executed in the manner shown in Figure 8. The vendor performs
ILI runs using information collected from a pipeline
questionnaire completed by the operator that includes details
about navigability, product and pressures used during the run.
The vendor then issues a preliminary report providing
information the operator needs before performing verification
digs. The verification digs from the preliminary report can be
used to assess the performance of the inspection and determine
5 © 2022 by ASME
if any analysis adjustments may be warranted. Any applicable
adjustments can then be implemented into the final report. More
verification digs can be done after the final report, and the cycle
of assessing performance and implementing applicable
adjustments and refinements can be repeated until a stable set of
accurate and actionable information is achieved.
planning of the Plan step. This transition from Act to Plan is a
decision process made by operators, vendors and third-party
consultants. Planning for NDE at this stage will direct future
attention of ILI data analysis toward an operator’s most pressing
integrity threats, assist in the selection of appropriate NDE
methods and facilitate the logistics of selecting dig sites. The
Check section is modified so as not to explicitly state digs
because the integrity planning is done in the Plan stage.
Use of the continuous improvement cycle of providing
successive feedback allows digs to be augmented with historical
and previous ILI data. The key in this process is the input from
Act into Plan, which provides a means to leveraging historical
data and continually improving high-resolution ILI. This
minimizes future reliance on validation digs and provides greater
confidence in using ILI data in integrity programs.

6. MODEL ENGINEERING USING NDE DATA

NDE dig feedback data is generally provided in the form of
a PDF or spreadsheet report. In a manual examination using
either a pit gauge or a handheld probe, the report will contain
pertinent information such as distance to a reference feature, e.g.
distance to an upstream weld, the orientation of the observed
metal loss deemed associated with the reported ILI feature and
FIGURE 8: THE DIG VERIFICATION PROCESS its measured depth. The report may also contain other
information, such as the feature length and width and pictures of
5.1 Plan, Do, Check, Act the area of corrosion as found in the ditch. Occasionally,
The Dig Verification Program (DVP) takes validation a step measurements of adjacent pitting may be included, although
further. In accordance with the API 1160 “Plan, Do, Check, Act” verification and repair digs usually target a specific feature and
(PDCA) [15] process, a system of continuous improvement is much of the surrounding area is not included.
outlined as shown in Figure 9. The flow is similar to Figure 8 An automatic NDE survey, such as laser profilometry or
but there are some subtle differences. AUT, will contain more data. This usually includes a C-scan of
Integrity Planning the area including dimensions of each of the individual
Integrity Feedback anomalies contained within the area. This provides a much more
comprehensive view of the investigated area, allowing for a more
Internally and to Plan accurate ILI correlation.
providers ILI Run Figure 10a provides an example of the C-scan from laser
profilometry with individually identified features with their size
Figure classifications. Figures 10b – 10d show the corresponding
signals of three components of the MFL from an ILI tool using
Remediation Act axial magnetization. A qualitative examination reveals the areas
& Repairs Analysis of the metal loss that have good correlation with MFL signals
Do and the areas where the technology is not as sensitive. Any
model inputs engineered from the MFL signals should have a
correlation to the physical dimensions of the geometry measured
Reassessment Preliminary by the NDE. Model inputs engineered from isolated machined
/ Regrade anomalies oversimplify the geometries of real-world corrosion
Check and thus have fundamental modeling limits. While C-scan
Validation Assessment images from profilometry reports are helpful in correlating NDE
data with ILI, they do not provide actual axial-circumferential
Final Report array of depth values. These values can easily be exported from
the NDE software and are invaluable in advancing the
FIGURE 9: PDCA CYCLE
development of sizing models.
Customer questionnaire data is replaced by the integrity
management data provided from the integrity feedback in the Act
step of the PDCA which is then provided to the integrity

6 © 2022 by ASME

(a) LASER PROFILOMETRY (b) MFL - AXIAL
(c) MFL - RADIAL (d) MFL - CIRCUMFERENTIAL
FIGURE 10: COMPARISON OF MFL RESPONSE TO
MEASURED PROFILE OF GENERAL CORROSION.
As the resolution of NDE data and ILI data increases, cross-
correlation issues that had traditionally been intractable can now
be addressed. Metal loss clustering, for example, is based on
anomaly dimensions and the distance between adjacent
anomalies. Since ILI and NDE are based on different
measurement methods, these calculations often do not match.
However, they can be reconciled if the raw data from both
surveys is compared and correlated [16].
FIGURE 11: COMPARISON OF LASER PROFILOMETRY
DATA (TOP) WITH ILI MFL DATA (BOTTOM) AND THE TWO
OVERLAIN(MIDDLE). RED BOXES ARE ANOMALY
DIMENSIONS CALLED BY ILI AND YELLOW CONTOURS
ARE DIMENSIONS CALLED BY NDE.
Figure 11 shows an example of discrepancies between ILI
and NDE metal loss calls. The top image is the C-scan of the
profilometry depth data and the yellow contour lines are the pit
dimensions called by the NDE. The bottom picture is the ILI
MFL data, and the red squares are the pit dimensions called by
the ILI. Looking at the axial-circumferential data displayed in
the image, the MFL signals can be matched to the corresponding
measurements in the NDE data. The discrepancy in cluster
sizing is due to the implementation of interaction criteria.
Aligning both data sets enables a more accurate correlation of
ILI and NDE.
7. SIZING MODEL EXAMPLE
This section will demonstrate the relationship between the
quality of the training data with its derived features and the type
of machine learning model used.
The examples shown in Figures 12 and 13 are k-nearest
neighbor (KNN) models. A KNN model is a regression model
that makes predictions based on the closest trained examples
from the feature space of the k-nearest neighbors.[17] machine
learning models typically use splits of the training data, using
some percentage of data for the model training and the remaining
data to test (validate). Splits are created by partitioning the data
differently for each split based on statistical methods and then
synthesizing the results in the final model allowing all training
data to be used. It also minimizes overfitting to the data and
enables estimation of the model performance on data the model
was not trained on. The schemes used for training and test data
splits are extremely important as test data that reflects an unseen
example best represents future prediction performance.
Often a randomly sampled 20% is withheld from the training
dataset. This provides enough data (80%) to train the model
while leaving a remaining dataset against which to assess how
well the model predicts. However, it is imperative to carefully
consider if the examples in the test dataset represent the
population of future examples. If the test dataset does not
represent future examples and is too similar to the training
dataset, then the model will learn how to perform well
specifically on the test dataset and not generalize to future
examples. Any performance metrics derived from this scenario
will be inflated as performance on new examples that differ from
the test dataset will be worse. The following examples utilize a
unique test dataset sampling scheme designed to significantly
improve the test dataset’s representation of future examples.
The following is three examples of how training data and
model selection affect performance. The validation data is the
same for all three scenarios and is the curated field measurements
for five different test splits.
Figure 12 shows a KNN model that was trained on machined
defects with simple well-defined geometries and its performance
on the validation data. As can be seen, the model performs
poorly and has a wide error distribution of predicted depths to
actual measured values. This is mostly due to the differences in
signals from machined anomalies to field data. Though
machined anomalies are a good foundation for analytical models,
machine learning models need to be trained on the data that has
features similar to the ones they are expected to predict.
7 © 2022 by ASME

FIGURE 12: RESULTS OF TRAINING A K NEAREST
NEIGHBOR (KNN) MODEL ON MACHINED ANOMALIES AND
PREDICTING DEPTHS OF FIELD CORROSION.
FIGURE 13: RESULTS OF TRAINING A K NEAREST
NEIGHBOR (KNN) MODEL ON FIELD DATA AND
PREDICTING THEIR DEPTHS.
Figure 13 shows the KNN model retrained using field data
instead of machine pull test features. As can be seen there is a
dramatic improvement in performance, with a tighter error
distribution. However, the field data contains a population
imbalance that has far more samples in the moderate depth range.
This introduces a bias into the model as the majority of neighbors
are in the moderate depth range and highlights the importance of
having representation over the full range of geometries to be
sized.
For the KNN model, using curated field data results
improves accuracy by relying on nearest neighbors. In densely
populated regions it performs well but struggles in regions more
sparsely populated with feedback. Switching to a gradient boost
(XGB) model allows for a more adaptive fitting using an
ensemble of many small decision trees that each minimize the
residual prediction errors [18]. Figure 14 shows the result of
using the XGB model. The linearity of the unity line has been
captured and the mean error is lowered.
A special note should be made regarding Figure 14. Split 1
shows a subpopulation that significantly differs from the other
points in the validation data (blue box). This subpopulation is
clearly an outlier from the rest of the population and is not
represented well in the training data. By identifying this, a
detailed review of those specific anomalies can be done and new
model inputs can be engineered or changes in the modeling
architecture can be made to improve performance.
FIGURE 14: RESULTS OF TRAINING A XGB MODEL ON
FIELD DATA AND PREDICTING THEIR DEPTHS. AN OUTLIER
SUBPOPULATION IS HIGHLIGHTED IN BLUE.
8 © 2022 by ASME
8. CONCLUSION
As demonstrated in this paper, machine learning is a
powerful tool, but only when carefully applied. Data used in
model development requires diligent curation, precise
correlation and accurate measurements. Subject matter experts
play a vital role in selecting a model, engineering its inputs and
checking the prediction space is well represented. Operators can
aid in the development by utilizing NDE techniques that
maximize the number of measurements per dig and provide
detailed information on anomaly geometry.
ACKNOWLEDGEMENTS
Tod Barker, Jeremy Clark, Josh Yunik, Robert Coleman,
Chris Goller, Jed Ludlow, James Simek, Andrew Moore, Callie
Lamb and Eric South.
REFERENCES
[1] Burden, D., Dalfonso, P. and Belanger, A., “The Current
Progeny of In-Line Inspection Machine Learning,”
Proceedings of the 2020 Pipeline Pigging and Integrity
Management Conference, Houston, TX, USA, February 17-
21, 2020, No. 54.
[2] Conrad, B., Kissel, M., Tse, V., Barlow, J., Chen, J., and
Kania, R., “Considerations for validating an ILI technology:
controlled implementation of a new MFL product,”
Proceedings of the 2020 Pipeline Pigging Management
Conference, Houston, TX, USA, February 17-21, 2020, No.
19
[3] Chakraborty, I. and Vyvial, B., “Using deep learning to
identify the severity of pipeline dents,” Proceedings of the
2020 Pipeline Pigging and Integrity Management
Conference, Houston, TX, USA, February 17-21, 2020, No.
70
[4] Hurd, G., Elliott, J., Farnie, S., and Sutherland, J.,
“Advancing In-line Inspection technology and pipeline risk
management through advanced analytics of big data,”
Proceedings of the 2020 Pipeline Pigging and Integrity
Management Conference, Houston, TX, USA, February 17-
21, 2020, No. 13
[5] Lee, S., 1962, Amazing Fantasy, #15, Marvel.
[6] Gauss, Carl F., 1809, Theory of the Motion of the Heavenly
Bodies Moving about the Sun in Conic Sections, Little,
Brown and Company, Boston, Doi: 10.5962/bhl.title.19023
[7] Box, G., and Draper, N., 1987, Empirical Model-Building
and Response Surfaces, Wiley, New York.
[8] Turing, A., 1950, “Computing Machinery and Intelligence,”
MIND a Quarterly Review of Psychology and Philosophy,
pp 433-460
[9] Bubenik, T. A., Nestleroth, J.B., Davis, R. J., Crouch, A.,
Upda, S., and Afzal, A. K., 2000, “In-line Inspection
Technologies for Mechanical Damage and SCC in
Pipelines: Final Report,” US DOT, OPS, Report No.
DTRS56-96-C-0010.
[10] Weiss, Gary M., and Foster Provost. "Learning when
training data are costly: the effect of class distribution on
tree induction." Journal of Artificial Intelligence
Research (2003): 315–354.
[11] Kirkwood, M., Burden, D. and Maldonado, M., “The Art of
Looking: An In-Line Inspection Perspective,” Proceedings
of the 2019 Pipeline Pigging and Integrity Management
Conference, Houston, TX, USA, February 18-22, 2019, No.
10.
[12] McNealy, R., McCann, R., Van Hook, M., Stiff, A. and
Kania, R., “In-Line Inspection Performance III, Effect of In-
Ditch Errors in Determining ILI Performance,” Proceedings
of the 8th International Pipeline Conference, IPC2010-
31269, pp. 469-473, Calgary, Alberta, Sept 27 – Oct 1, 2010,
https://doi.org/10.1115/IPC2010-31269
[13] Tomar, M., Fingerhut, M., and Yu, D., “Qualification of ILI
Performance in Accordance with API 1163 and the Potential
Impact for Management of Pipeline Integrity,” Proceedings
of the IPC2008, 7th International Pipeline Conference,
IPC2008-64469, pp. 693-703, Calgary, Alberta, Sept 29-
Oct3, 2008, https://doi.org/10.1115/IPC200864469
[14] Feole, R. A., “Automation of Fast Ultrasonic Technique for
Pipeline Weld Examinations,” PRCI, 1998, PR-250-9612,
Cat. No. L51777.
[15] American Petroleum Institute. (April 2019). Managing
System Integrity for Hazardous Liquid Pipelines. API
Standard 1160, Section 4.1
[16] Lamb, C. and Dalfonso, P., “How ILI Feature Grouping Can
Impact Apparent Accuracy from NDE,” Proceedings of the
2022 Pipeline Pigging and Integrity Management
Conference, Houston, TX, USA, Feuruary 2-40 2022, No.
43.
[17] Goldberger, J., Rowels, S., Hinton, G. and Salakhutdnov,
R., “Neighbourhood Components Analysis,” Advances in
Neural Information Processing Systems, Vol. 17, May 2005,
pp 513-520.
[18] Friedman, J. H. 1999, "Greedy Function Approximation: A
Gradient Boosting Machine" (PDF), IMS 1999 Reitz
Lecture.
9 © 2022 by ASME