TWI-2008-Reliability of Manually Applied Phased Array Ultrasonic Inspection For Detection and Sizing of Flaws PDF

17546/6-1/08
September 2008
Reliability of Manually Applied Phased Array

Ultrasonic Inspection for Detection and Sizing of
Flaws
Revised Final Report
For: A Group of Sponsors

Reliability of Manually Applied Phased Array Ultrasonic Inspection
for Detection and Sizing of Flaws
Revised Final Report
TWI Report No 17546/6-1/08

September 2008
Prepared for: A Group of Sponsors
Authors: C R Bird, C R A Schneider and D Caravaca
TWI Endorsement
The report has been reviewed in accordance with TWI policy.
Project Leade~ .. : TGM/Reviewer

.A--:- feJ--/'-J ~
(Signature) (Signature)
Print name: Aamir Kh id
Approved by : ~
Commercial.M.<~!Jef-----':::::"
.
(Signature)
Print name: Peter Oakley Print name: Andrew Carey
Secretary/Administrator ...
(Signature)
ai.~
Print name: Cathy Holder
orb Copyright C TWI Ltd 2008

Contents
Executive Summary
Background
Objectives
Work Carried Out
Conclusions
Recommendations
1 Introduction 1
2 Objectives 1
3 Approach 1
3.1 Introduction 1
3.2 Desig n of trial and statistical treatment of results 2
3.3 Design of test blocks 2
3.4 Inspection companies, data collection equipment and inspection procedure 3
3.5 Data collection operators 4
3.6 TOFD 5
3.7 Flaw sectioning 5
4 Inspection Data 6
5 Analysis of Phased Array Data 6

5.1 Approach to statistical analysis of phased array data 7
5.1.1 Ultrasonic data 7
5.1.2 Potential factors affecting sizing error 8
5.1.3 Analytical approach 9
5.2 Results of statistical analysis of phased array data 9
5.2.1 Before removal of 'outliers' 9
5.2.1.1 Main effects 9
5.2.1.2 Possible reasons for variations between operators 10
5.2.1.3 Interactions 11
5.2.1.4 Non-linearity with respect to continuous covariates 12
5.2.1.5 Homogeneity of the variance of the error term 12
5.2.1.6 Normality of the error term 12
5.2.2 After removal of 'outliers' 13
5.2.2.1 Combined data from all plates 13
5.2.2.2 Plate thickness greater than 6mm 13
5.2.2.3 Plate thickness equal to 6mm 15
5.3 Summary of strict statistical analysis 16
6 Analysis of Manual Inspection Results 16
7 Phased Array Results Compared with Manual Results 17
8 Discussion 19
8.1 Data collection and analysis 19
17546/6-1/08 TWILtd
8.2 Discussion of inspection procedure 19
8.3 Discussion of flaw sizing with respect to flaw type 20
8.4 Discussion of. flaw sizing with respect to through-wall flaw size and test block
thickness 20
8.5 Discussion of equipment performance 21
8.6 Discussion of operator performance 21
8.6.1 Identification of errors in flaw sizing and detection 21
8.6.2 Operator training and qualification 22
9 Conclusions 23
10 Recommendations 24
11 References 24
Tables 1-7
Figures 1-43
Appendix A: Phased array procedures

Appendix B: TOFD and manual UT procedures
Appendix C: Test block drawings
Appendix D: Detailed inspection results
Appendix E: Test block certificates
17546/6-1108 TWI Ltd

Executive Summary
Background
Phased array ultrasonic technology and instruments for non-destructive testing (NOT) have entered
the second stage of their development. They are now widely available in the form of affordable!
portable instruments from a number of manufacturers, as well as in the form of laboratory based, high
performance systems. The cost of the portable equipment has reduced to the level where the
technique can be considered for much more widespread high quality applications. Projects on a wide
range of components suggest that phased array ultrasonic technology can under controlled conditions
provide better flaw sizing than conventional manual ultrasonic inspection. Furthermore, the reliability
of inspection of automated systems in general has been investigated. The results of this project aim to
provide industry with the objective technical information and the confidence to make wider use of
phased array ultrasonic inspection, with all the attendant benefits that this will bring. To provide a
base line for the reliability results, TWI has also subjected the test blocks to conventional manual
ultrasonic inspection.
Objectives
• Provide an independent assessment of the flaw sizing accuracy of portable phased array
ultrasonic equipment.
• Compare the results obtained from different operators.
• Compare the performance of different instruments.
• Compare the results obtained from at least two inspection procedures.
• Explore the variation in flaw sizing accuracy due to flaw type and location.
Work Carried Out

This project studies the reliability of critical defect sizing using phased array inspection. A range of test
blocks, between 6 and 50mm thickness, were manufactured, which contained a total of 40 flaws
according to a specific experimental design. The flaws included lack of fusion, cracks and volumetric
flaws.
Fifteen operators were used in the project, ten phased array operators and five conventional manual
operators, each operator inspecting all the test blocks. The Sponsors were not approached for
volunteer phased array operators; instead, equipment manufacturers were approached. The
equipment manufacturers were told and will receive only their own results back and will not receive
any results from the analysis of the data except for the differences between the measured and actual
flaw sizes.
As stated in the sponsor meetings and in the project specification, the identity of the companies and
operators is to remain anonymous, but variations in sizing accuracy due to technical differences are
reported.
All the results for each operator were analysed statistically to establish the factors governing the
reliability of defect sizing.
This analysis examined the following factors: plate thickness, flaw type, flaw size, operator
qualifications and operator experience.
Furthermore, these results were compared with those for the manual operators.
Conclusions
1 Objective 1 of this project was to assess the sizing errors associated with manually applied
phased array inspection. The detailed conclusions below break down the errors according to
operator, plate thickness, flaw size and flaw type. A single figure for the flaw sizing error would
over-simplify the results; the individual trends are illustrated in the body of the report.
17546/6-1/08 TWI Ltd

2 Objective 2 of this project has been met by showing that there is a large variation in the ability of
phased array operators to size flaws. There is also strong evidence that this sizing capability is
linked with training and experience.
3 Objectives 3 and 4 of this project were to compare the performance of different phased array
instruments and procedures respectively. In practice, it was not possible to rank the performances
of the different equipment, because any such variability was swamped by the operator variability
and the fact that each operator only used one set of equipment. In particular, there was no
evidence from this project of the 16-channel instruments performing any worse than the 32-
channel instruments.
4 Objective 5 of this project was met by identifying a statistically significant (at the 5% level) link
between the flaw type and the random sizing error for phased array. In particular, rough flaws
were identified as having a significantly lower sizing error than smooth non-tilted flaws for plates
with a thickness greater than 6mm.
As well as meeting the broad project Objectives, this project resulted in the following, more detailed
conclusions:
5 Both the mean phased array sizing error and the scatter about this mean is greater for the thicker
plates than for the 6mm plate.
6 In general, the following factors have a significant effect (at the 5% level) on the mean
'systematic') phased array sizing error:
• Flaw height
• Operator
• Plate thickness.
7 The overall trend is for the phased array systematic sizing error to tend towards greater
undersizing with increasing flaw height, in that the systematic error changes from a small
oversizing error to an increasing amount of undersizing as the flaw size increases. This trend has
also been observed in the manual ultrasonic inspection results.
8 For a given flaw height, the systematic phased array sizing errors are slightly larger in the thicker
blocks than in the thinner blocks.
9 The phased array random sizing errors tend to increase with flaw height.
10 The phased array random errors are -20% smaller for those operators who hold a critical defect
sizing CSWIP compared to those who do not (in the plates thicker than 6mm).
11 The best phased array operators generated random and systematic errors of smaller magnitudes
than those of the best manual flaw detector operators.
12 Each flaw within the blocks was correctly sized «3mm error) by at least two operators and each
flaw was badly (>3mm error) sized by at least one operator demonstrating that all flaws could be
correctly sentenced.
13 Every phased array operator badly sized at least three flaws demonstrating a need for improved
training.
14 Flaw characterisation appears to be a generic problem amongst the operators and it is concluded
that improved training and examination is required if the NDT industry is to provide accurate flaw
reporting. The test blocks were appropriately designed to test the ability of the systems, including
the operators. Despite providing good visualisation of flaws, the results show that flaw
characterisation by swept beam phased array techniques remains technically challenging and this
should be addressed with improved quality assurance.
1754616-1108 TWI Ltd

Recommendations
• Improve the quality of data interpretation it is recommended that phased array operators are
trained to use crack tip diffraction signals and gain experience of large (>beam width) smooth
planar flaws.
• Provide a better phased array service to industry, it is recommended that examination of
operators include sizing of flaws greater than 6mm through wall and that the examination
certificate states that the operators have been examined for defect sizing.
17546/6-1108 lWlltd
1 Introduction
Phased array ultrasonic technology and instruments for non-destructive testing (NDT) have
entered the second stage of their development. They are now widely available in the form
of affordable/portable instruments from a number of manufacturers, as well as in the form of
laboratory based, high performance systems. The cost of the portable equipment has
reduced to the level where the technique can be considered for much more widespread
high quality applications. Projects, (Ship et ai, 2002), on a wide range of components
suggest that phased array ultrasonic technology can under controlled conditions provide
better flaw sizing than conventional manual ultrasonic inspection (simply referred to as
manual UT in this report). Furthermore, the reliability of inspection of automated systems in
general has been investigated. The results of this project aim to provide industry with the
objective technical information and the confidence to make wider use of phased array
ultrasonic inspection, with all the attendant benefits that this will bring.
To provide a base line for the phased array results, TWI subjected the test blocks to manual
UT, as well as phased array inspection; all these trials were blind trials. Following the blind
examination of the blocks, they were subjected to time-of-flight-diffraction (TOFD)
inspection and the results used to select the flaws for sectioning.
2 Objectives
• Provide an independent assessment of the flaw sizing accuracy of portable phased
array ultrasonic equipment.
• Explore the variation in flaw sizing accuracy due to flaw type and location.
3 Approach
3.1 Introduction
The project was designed to assess manually applied phased array inspection for flaw
sizing. Manually applied techniques were deliberately chosen to enable a number of
companies to take part without the expense of purpose-built manipulators. Although the
techniques were manually applied, all of the data was recorded by the phased array
systems, except in the case of the GE Phasor, which cannot record data.
A variety of companies and operators were used in the project. The Sponsors were not
approached for volunteer phased array operators; instead, equipment manufacturers were
approached. The equipment manufacturers were told and will only receive their own results
back and will not receive any results from the analysis of the data except for the differences
between the measured and actual flaw sizes.
As stated at the sponsor meetings and in the project specification, the identity of the
companies and operators is to remain anonymous. But variations in sizing accuracy due to
technical differences are reported.
The project was designed to demonstrate phased array technology and its capability for
through-wall sizing of flaws; it was not designed to assess the probability of detection. Once
the project commenced, it was clear that the number and quality of experienced phased
array operators was very limited and the results of the project may be poorer than will be
achievable in a few years time when the number of experienced phased array operators
increases. In addition to the phased array inspections, manual UT and TOFD were also
applied to the test blocks. The manual UT operators were used as a control set rather than
as a study of the reliability of manual UT per se. The TOFD was simply used as an aid to
assess which flaws to section.
17546/6-1108 TWI Ltd

3.2 Design of trial and statistical treatment of results
The Sponsors were requested to state their preference for flaw type and TWI has aimed to
include all the Sponsors' requests in the flaw and test block selection.
This resulted in the following flaw criteria.
Flaw types:
• Smooth planar, to simulate lack of side wall fusion.

• Rough planar, to simulate cracks.
• Volumetric, to simulate porosity or slag.
Flaw location and orientation:
• Near-normal incidence to inspection beams, ie fusion face flaws (referred to in this

project as tilted).
• Oblique incidence to inspection beam, ie centre-line flaws (referred to in this project as
non-tilted).
• Surface-breaking and sub-surface flaws.
To assist the statistical treatment of the results, a balanced design with respect to flaw type,
size and location within each test block size range was desirable. This was a complex
process, which necessitated a few compromises. The challenge was to provide a
reasonably balanced design without making the distribution of flaws obvious to the
operators. From the results of the analysis it is believed that this was achieved successfully.
A phased array procedure for inspection of the test blocks was developed by TWI and is
provided in Appendix A.
The phased array procedure was supplied to all operators in this trial. All operators with
suitable equipment followed the procedure. The GE equipment has 16 pulser receiver
channels and was therefore not compatible with this procedure. Similarly Olympus supplied
two operators, one of whom had a 16-channel instrument. Because of the equipment
characteristics, the procedure was modified by some of the operators. In particular, the time
corrected gain (TCG) could not be fully implemented with the AGR and Harfang instruments
and was not implemented by one of the Omniscan operators.
In addition to the phased array inspection the test blocks were subject to blind manual UT
trials by TWI and four other operators. This procedure is detailed in Appendix B.
Once all the blocks were inspected by all the operators, they were also inspected by TOFD
to confirm the through-wall size of the flaws. This TOFD procedure is detailed in Appendix
B. TOFD is subject to its own inspection errors; the TOFD inspection procedure was
optimised to minimise the sizing errors, based on the knOWledge of the location and
expected size of the flaws. Using the combined knOWledge of the design sizes of the flaws
and this TOFD inspection, together with an initial analysis of the phased array data, flaws
were selected for sectioning.
Eight flaws were chosen for metallurgical sectioning and photography once the inspection
programme was completed. The flaws to be sectioned were discussed during the 6-month
meeting after collection of all data from all operators.
3.3 Design of test blocks

To cover the full range of flaw categories, plate thicknesses and flaw orientations, a
statistically balanced selection of flaws were implanted into the test blocks.
The conclusion from the launch meeting was that the project should concentrate on butt
welds in plates with a thickness range from 6 to 50mm. Based on these requirements, eight
17546/6-1/08 2 TWI Ltd

test plates containing a total of 40 flaws were designed. To avoid any patterns in the
incidence of flaws being noticed by the NOT operators, each plate contained a different
number of flaws.
A fairly uniform spread of flaw sizes and types is contained in each plate thickness; of
course it was not possible to have large flaws in small plates, but the design is statistically
balanced. Because plates 1 and 2 are very thin, their analysis was treated separately.
The flaws were manufactured by Sonaspection and most of the flaws were coupons
implanted into the weld. For the main body of the weld, the welding process used manual
metal arc (MMA) with tungsten inert gas (TIG) for the root.
The following test blocks were manufactured for the project and are detailed in Appendix C.
No. of test Weld Plate Plate

blocks eeomelry thickness, mm Test block type leneth, mm
2 V 6 Plate 400
1 V Asymmetric Asymmetric weld 400
15/20
1 V 20 Svmmetric weld 400
2 V,J 35 Welded olate 400
2 J,X 50 Welded plate 400
The welds include the following flaw types: smooth planar, rough and VOlumetric.
For each flaw type there is a selection of flaws tilted and not tilted with respect to the
through-wall direction. Further, there was a uniform spread of surface flaws and embedded
flaws.
Flaw manufacture is difficult and has an associated manufacturing tolerance. Sonaspection

are experienced flaw manufacturers and state that their through-wall manufacturing
tolerances are ±0.75mm. Further, TWI has prior experience of the magnitude of errors
associated with the Sonaspection test blocks, based on many ultrasonic trials. However, to
confirm all the flaw sizes TOFO was applied; any flaws for which the evidence from either
the phased array or TOFO data appeared to be inconsistent with the as-built sizes were
sectioned.
The Sonaspection statement of as-built flaw size together with the metallurgical section
sizes are given in Table 1. Table 2 provides the TOFO inspection sizes of the implanted
flaws together with the as-manufactured sizes. The TOFO results, as for any other
inspection technique, will have associated sizing errors. The sectioning results provided
confidence in the stated manufactured size, position, and flaw type. The manufactured size
or, where available, the sectioned size were used for the analysis of the flaw sizing errors.
3.4 Inspection companies, data collection equipment and inspection procedure

The following companies volunteered inspection equipment and inspectors for this project:
Olympus using the Omniscan 32/128, 16/128 with one operator for each
instrument.
Sonatest using the Harfang X-32.
GE using the Phasor XS (a 16-channel instrument) with two operators
working separately.
M2M using the Multi 2000.
AGR using the TO Focus Scan.
British Energy/Oceaneering volunteered four manual UT operators and TWI provided a

further manual UT operator. The manual UT proVided a benchmark for the phased array
data.
17546/6-1/08 3 TWI Ltd

TWI generated an optimum procedure for the examination of the flaws in each welded test
block for the phased array ultrasonic results. This procedure was written to be applied by a
32-channel phased array instrument, using 32 active channel sector scanning at a constant
standoff. This procedure has been used by most of the operators. It was named Procedure
1 and is presented in Appendix A. In all cases except for the Phasor, the ultrasonic probes
as detailed in the procedure were used by each operator. The procedures operated by the
Phasor were clearly different in that there were fewer active elements and the data had to
be analysed 'live' as per conventional ultrasonic equipment. The other instrument used in
16-channel mode was the Omniscan.
The procedure was modified for the 16-channel instruments. In particular, the Phasor did
not record data. Further, the time corrected gain (TCG) was not used by all operators. The
TCG is applied in a different way by each instrument, as discussed in Section 3.2. One of
the operators used equipment with 3D data analysis soflware to aid the analysis process.
The following table presents the equipment specifications used in the project. The
equipment number given in the table is used and quoted in the results given in Appendix D.
Equipment Specification
1 32-channels
NoTCG
2 16-channels
No TCG. Data not
recorded
3 32-channels
TCG applied,
3D Analysis soflware
4 32 channels
TCG
5 16-channels
TCG
6 Manual
Each operator was required to report the following details for each flaw detected and to
comply as closely as possible with the procedure provided in Appendix A (previously issued
at the six month sponsor group meeting):
• Length
• Through-wall height
• Flaw type
• Start and end position
• Amplitude with respect to the reference amplitude.
The results were reported by the operators on the form reproduced in Table 4.
3.5 Data collection operators

The operators were instructed that the size of the flaw could be anything from very small to
very large and that these plates were not comparable to normal PCN or CSWIP
examinations. Further, they were warned to expect flaws at unusual orientations. This
instruction was specifically given to warn the operators to keep an open mind during the
data analysis and that some flaws could be unusual.
A total of ten phased array operators took part in the blind data collection and data analysis.
It is important to mention that none of the operators had any pre-knowledge of the flaws
(nature, dimension, location or distribution within the test block). The manual UT was
17546/6-1/08 4 TWI Ltd

performed in accordance with BS EN 1714, in all cases applying maximum amplitude
and/or 6dB drop flaw sizing techniques.
The ten phased array operators included two operators from TWI while the five manual UT
operators included one operator from TWI.
3.6 TOFD
A TOFD inspection was performed with pre-knowledge of the stated as-manufactured flaw
sizes. This inspection was undertaken to confirm the as-manufactured flaw sizes and to
help select the flaws to be sectioned. TOFD could not be applied fully to the 6mm weld due
to the presence of the weld cap.
The TOFD data collection procedure is detailed in Appendix B. The TOFD data was not
used in the analysis of the sizing accuracy of the other techniques.
The TOFD results are detailed in Table 2. The differences between the TOFD through-wall
sizes and the stated as-manufactured sizes were analysed as follows. The standard
deviation of the difference between the as-manufactured size estimate and the TOFD size
is 1.31mm. The anticipated standard deviation based upon TWl's knowledge of TOFD
sizing errors is between 1 and 2mm. Further, by studying the sectioned measurement and
comparing with the TOFD measurement, in some cases the TOFD size was larger than the
as-manufactured size and sometimes it was smaller. Sonaspection estimate that the sizing
error on manufacture is up to 0.75mm. It should be noted that, despite only sectioning the
flaws with reported sizes that were inconsistent with the as-manufactured sizes, the
sectioning results suggest that the stated as-manufactured sizes were essentially met.
From this analysis, it was concluded that all the statistical analysis should be based upon
the estimated as-manufactured sizes and, where available, the sectioning sizes.
It should be stated that flaw 20 gave confusing TOFD results. It showed some energy
transmission through the flaw. This would normally indicate that the flaw was either partially
fused in the middle or subject to very high compressive loads, but sectioning confirmed that
it was a continuous flaw.
Further, flaw 37 was 'salami' sectioned, partly because the TOFD results varied by up to
1mm in through-wall height along the length of the flaw. The height quoted in the results
table (Table 2) is the maximum TOFD measurement. Based on the sectioning data, there
actually appeared to be little variation in through-wall height. The maximum height on
sectioning was 5.73 and the minimum 5.32mm. Thus, the sectioning provided very little
insight into the variation in the TOFD results.
3.7 Flaw sectioning

After collating all the results, it was shown that some flaws presented more sizing difficulties
than others. The results were discussed with the Sponsors during the 6-month meeting and
eight flaws were chosen for sectioning. Those chosen for sectioning were flaws 11, 20, 25,
29, 31, 34, 35 and 37, as detailed in Table 2. The TOFD results were used to confirm the
axial position for sectioning.
The comparison between the as-manufactured and sectioned sizes is given in Tables 2 and
5. Flaw 37 was salami sliced because the results from the various techniques/operators
were very different in terms of flaw height and there was up to 1mm variation in the TOFD
data (see above). Macrographs of the flaws are presented in Figures 1 to 14.
The sectioning results could not be used to establish the definitive start and end positions of
each flaw; this project was not designed and did not attempt to assess flaw length. But, as
detailed earlier, all the flaws were measured for length and could, in principle, be analysed
in the future for length sizing errors.
17546/6-1/08 5 TWI Ltd

Flaw 20 was generally undersized by the operators. Representing the results of this flaw in
the results tables has been difficult because it could be clearly seen from the results that
the errors from a number of the operators were due to mistaken data interpretation rather
than measurement errors. It is clear that the basic mistake was in interpreting this flaw as
two separate flaws: one large one (apprOXimately 15mm through-wall) and the other a
thread-like flaw, ie a flaw with a height less than -1mm. This difficulty was further
highlighted by the TOFD inspection, which also misinterpreted the flaw despite the operator
having the pre-knowledge that it was just one flaw. The macrograph (Figure 2) shows that
the flaw is continuous, but no finite gape can be detected in the middle of the flaw.
Furthermore, both the TOFD and the pulse echo phased array data indicate that the flaw is
at least partly transparent to ultrasound. Structurally this would be treated as a continuous
flaw and for this reason the statistical analysis includes this flaw. Excluding this flaw from
the data set would clearly improve the apparent sizing errors but it is difficult to justify this
exclusion without being accused of bias in the analysis. Moreover, some operators
analysed the flaw correctly.
4 Inspection Data
Appendix D provides the results for the manual UT and phased array results. The TOFD
results are given in Table 2.
The results in AppendiX D are arranged in separate tables for each operator number. Each
table states the test block number, the as-manufactured flaw size and type, the ultrasonic
measurements of the flaw size, and the difference between the as-manufactured (and/or
sectioned) flaw size and measured flaw size.
The results of the phased array and manual UT inspections show that some flaws were
missed and that some operators recorded false calls. In particular, the false call results are
highlighted in blue and the missed flaws are highlighted in red.
The number of missed flaws and false calls are given for each operator in Table 6. From
examination of the false calls there appeared to be many reasons for these calls but the
reasons for many of them are speculative as the size reported was quite large and could
not be thought of as a point indication as mentioned in the Sonaspection report. Some
appeared to be misplotting of flaws that did exist, some misinterpretation of weld cap
signals, and some double reporting of a single flaw.
Every flaw was detected and correctly sized by at least one operator.
5 Analysis of Phased Array Data

The analysis of the inspection results is the most important part of this project and is the
most complex. Moreover, the set of results obtained in this project has become larger than
planned at the outset due to the generous contributions volunteered by the equipment
suppliers. As a result of the increased sample size, the results are more statistically
significant than originally anticipated.
At the outset of the project, it was agreed that the primary purpose of the project was to
assess the sizing accuracy of phased array inspection. However, due to the full
complement of Sponsors and the generous contribution of volunteer companies, it was also
possible to accommodate five manual UT operators and a TOFD inspection.
The emphasis of the data analysis is on the phased array results and a relatively simple
comparison is made with the manual results. Section 4 deals with the statistical analysis of
the phased array data and Section 5 deals with the manual results.
17546/6-1/08 6 TWI Ltd

5.1 Approach to statistical analysis of phased array data
The results of the phased array examination of the welded test blocks were analysed
statistically to determine the sizing accuracy of the procedures used for the different flaw
types and sizes.
The overall aims of the statistical analysis were to:
• Provide an independent assessment of the performance of the flaw sizing accuracy of

portable phased array ultrasonic equipment.
• Explore the variation in flaw sizing accuracy due to flaw type and location (ie embedded
flaws versus surface flaws).
Our chosen methodology was based on analysis of variance, which is described in detail in
the Minitab (1998) manual and more generally by Cooper (1969).
The test block exercise was designed to be 'balanced' with respect to the main factors of
interest. An experimental design that is 'balanced' with respect to operator, flaw type and
block thickness, for instance, is one that yields the same number of observations for each
combination of operator, flaw type and block thickness. This ideal was not qUite achieved,
in the event, because not all flaws were detected by every operator. Because of this, the
statistical analysis made use of the 'general linear model (GLM)' procedure rather than the
'balanced ANOVA' procedure provided by the Minitab software. Nevertheless, the overall
frequency of flaw detection during the trials was over 90% (see Table 6), so it is judged that
the results are similar to those that would have been obtained had the experimental design
been perfectly balanced.
The main aim of the analysis was to identify which of the following factors have a
statistically significant effect on sizing performance:
• Procedure
• Equipment
• Operator
• Operator experience
• Operator qualifications (ie whether or not the operator has a critical defect sizing
CSWIP in phased array)
• Wall thickness (including any asymmetry in the block)
• Weld shape (ie V, J or X)
• Flaw characteristics (eg size, type, location).
5.1.1 Ultrasonic data

The ultrasonic measurements were transcribed from the operators' forms into Excel and
subsequently imported into the software package Minitab for statistical analysis. The
operators' forms are held in the TWI project file.
During the data transfer, any false calls or missed flaws were excluded to avoid any
potential incompatibility with subsequent data.
17546/1>-1/08 7 TWI Ltd

At the outset it was decided to treat the 6mm plate separately from the other block
thicknesses; it was judged that the sizing errors pertaining to the 6mm plate might follow
different trends to those for the other plates, because:
• Inspection of the 6mm plate is limited to higher beam angles due to the presence of the
weld cap. No 45' beams, for instance, were used on the 6mm plate (unlike for the other
plates).
• Echoes from the weld cap and root generally hamper the sizing of flaws in the 6mm
plate to a much greater extent than for the other plate thicknesses.
Figure 15 shows the observed sizing errors for the 6mm plate versus those from the other
plates. It can be seen from Figure 15 that both the mean (or 'systematic') sizing error and
the scatter about this mean (or 'random' error) is greater for the thicker plates than for the
6mm plate. This finding was further supported by the results of analysing the two sets of
data as a single data set (see Section 4.2.2). Both here and elsewhere in Section 5,
systematic errors are considered to have a sign associated with them, so that a slight
undersizing error is treated as being 'larger' than a gross undersizing error (because sizing
errors for undersizing are assigned negative values).
Two sets of data have been accordingly created as separate files in Minitab; this was done
to avoid software limitations that can sometimes result from working with large data files.
5.1.2 Potential factors affecting sizing error

Operator
This is one of the main factors of interest. However, this factor is closely related to (and
difficult to distinguish from) both the equipment and the procedure.
Under this factor, a qualification in critical defect sizing has been examined as an extra
identifiable factor, in addition to that of experience. In all cases where CSWIP is quoted in
the text, it assumes critical defect sizing CSWIP. Further, it must be noted that this was not
a standard CSWIP critical defect sizing qualification. The details of the training and
examination are provided in Section 7.4.
Procedure
As detailed in AppendiX D, three procedures were used for the inspection.
Flaw height
In most cases, the actual flaw height is taken to be the 'as-built' dimension advised by the
test block manufacturer. However, for the flaws that were subsequently sectioned, it is
taken to be the height determined by sectioning.
Wall thickness of the block

It was discussed and agreed with the Sponsors that the blocks would be designed with
thicknesses in the range 6 to 50mm. Five different thicknesses were chosen including one
asymmetric weld. The thicknesses of the blocks were as follows:
• 6mm
• 15/20mm (asymmetric block)
• 20mm
• 35mm
• 50mm.
The asymmetric block was treated as 15mm thickness for the purposes of the statistical
analysis, but pair-wise comparisons were also made between this block and each of the
other wall thicknesses.
Weld shape (V, J or X
1754616-1/08 8 TWI Ltd

The design of the weld shapes can be seen in Appendix C Figures C1 to C8.
Equipment
A total of five different sets of equipment were used for this inspection. The equipment was
identified by the letters 1 to 5 as shown in Section 2.4. However it is important to note that
not the same number of operators used each set of equipment.
5.1.3 Analytical approach

The statistical models considered by TWI are all 'general linear models' of the form
(Minitab, 1998):
Y=A+X.B+E [1]
where Y is best estimate measurement error (the response variable),

A is a regression coefficient ('intercept') estimated from the data,
X is a vector of factors (including continuous 'covariates' such as actual flaw
height, as well as 'categorical' variables such as the operator that falls into one of
a number of discreet 'categories'),
B is a vector of regression coefficients, again estimated from the data, which
quantify how strongly the best estimate error depends on each factor,
E represents the error in the model, which is assumed to be a random variable,
independent of the factors X, and normally distributed with zero mean and
constant variance E (Which is again estimated from the data).
The model parameters are effectively estimated by the maximum likelihood method (Draper
and Smith, 1981). In a quite fundamental sense, the maximum likelihood method provides
parameter estimates that maximise the likelihood L (A, B, E) of obtain ing the observed
data. In this sense, the resulting estimates are those that agree most closely with the
observed data (Hosmer and Lemeshow, 1989).
The strategy followed, as usually adopted for model building in linear regression (Draper et
ai, 1981; Hosmer, 1989), is to initially assume a model that is linear in each of the factors of
Section 5.1.2 and identify those that are statistically significant (the so-called 'main effects').
Once the main effects have been established, it is then separately considered whether to
include extra factors in the model to describe interactions between the main effects and/or
non-linearity with respect to any continuous covariates.
Interactions arise when the effect of one factor changes according to the level of another
factor. For example, an interaction between operator and flaw height allows the effect of the
flaw height on the sizing error to vary from one operator to another, ie it allows the slope of
the relationship between the sizing error and the flaw height to vary between different
operators.
5.2 Results of statistical analysis of phased array data

5.2.1 Before removal of 'outliers'
5.2.1.1 Main effects
Plate thickness greater than 6mm
The 'main effects' were identified by successively adding each potential factor of Section
5.1.2 to the model and testing whether its coefficient was statistically significant (at the 5%
level). On this basis, the operator and the flaw height were successively identified as having
a significant effect on the sizing error and accordingly included as main effects in the linear
model Equation [1]. The variation of the observed sizing errors with these two factors is
illustrated in Figure 16. This includes separate trend lines fitted to each operator's results
(by ordinary linear regression) to assist in assessing the data by eye. The overall trend is
for the mean (systematic) sizing error to decrease with flaw height. One obvious
17546/6-1/08 9 TWI Ltd

explanation for this is that there is less scope for gross undersizing of small flaws as
compared with larger flaws.
Having included these two factors in the model, there is no evidence (at the 5% level) of
further variations in the sizing error with any of the other factors of interest, ie wall
thickness, flaw type, flaw length, flaw location or weld shape. The evidence for variation due
to the procedure or equipment (which is largely indistinguishable from that between
operators) is discussed further in Section 5.2.1.2.
Plate thickness equal to 6mm

The 'main effects' were identified using the same methodology as above for plate thickness
greater than 6mm. On this basis, the operator and flaw height again appear to have a
significant effect on the sizing error.
Figure 17 is the analogous plot to Figure 16 for the sizing errors observed in the 6mm plate.
As in Figure 16, the overall trend is for the mean (systematic) sizing error to decrease with
flaw height. The trend is less obvious than in Figure 16, but this may simply be because the
range of flaw heights in the 6mm plate is smaller.
5.2.1.2 Possible reasons for variations between operators

The apparent variations between operators could potentially be due to differences in:
• Procedure
• Equipment
• Experience
• Training/certification.
Variations due to these factors were further investigated by treating the operator as a factor
that is 'nested' within each of the above factors in turn. This yielded no evidence (at the 5%
level) of variations in the mean (systematic) sizing error according to whether the operator
had a phased array CSWIP qualification.
Additional to this strict statistical analysis, a simple ranking of the absolute mean and
standard deviation of errors for each operator has been undertaken. The results of this
ranking are discussed in Section 5.
There appears to be a variation, which is just significant at the 5% level, between the group
of operators with more than five years experience (operators 4, 5, 7, 10 and 15 in Figure
16) and the other operators; the mean (systematic) sizing error for the more experienced
operators appears to be -O.6mm larger (ie a slightly greater tendency towards oversizing)
than for the less experienced operators. However, there is much more significant difference
(over and above this) between the individual operators in each group, as illustrated by
Figure 16.
There also appear to be significant variations between the mean (systematic) sizing errors
for the groups of operators according to the procedure and equipment used. The reasons
for these variations are unclear. They might reflect genuine differences in methodology, eg
due to the different scanning patterns used. However, it appears just as plausible that these
variations reflect variations in the skill/experience of the individual operators.
In summary, it was very difficult to distinguish operator skill from procedure/equipment as

there is no evidence that apparent variations with procedure/equipment are, in fact, due to
differences in the experience or qualifications of the operators or simply the innate skill of
the operator. It was shown that variations with the operator were closely linked to apparent
variations with the equipment. This is to be expected given that each operator used only
one set of equipment and two of the sets of equipment were used by only one operator, ie
17546/6-1/08 10 TWI Ltd

there was a close correspondence between the equipment used and the operator who used
it.
To simplify the analysis, it was therefore decided to keep operator as a main effect. This
serves to encompass possible variations with all of the above factors.

As above, for plate thickness greater than 6mm, the apparent variations between operators
could potentially be due to differences in:
• Procedure
• Equipment
• Experience
• Training/certification.
In this case, whether or not the operator has a CSWIP in phased array seems to have a
high significance at the 5% level. However, as above, it is difficult to distinguish such effects
from the effect of the procedure or the innate skill of the operator, so it was decided to keep
the operator as the main factor. Note that, in Figures 16 and 17, it is operators 1, 2 and 10
that have a CSWIP in phased array.
5.2.1.3 Interactions
The interaction between flaw height and operator appears to be significant (at the 5%
significance level). This indicates that there is a statistically significant difference between
the slopes of the trend lines shown in Figure 16. The corresponding interaction term was
therefore included in the model.
Having included this interaction term, the tests of whether there are significant variations
with the factors discussed in Section 4.2.1.2 were repeated. This yielded no evidence (at
the 5% level) of residual variations in mean sizing error according to operator experience or
whether the operator had a phased array CSWIP qualification, ie inclusion of the interaction
term provides an alternative explanation of the apparent variation with operator experience
discussed in Section 4.2.1.2.
There appear to be significant interactions (at the 5% level) between flaw height and the
groups of operators using different procedures/equipment. As in Section 4.2.1.2, there is no
obvious physical reason why this should be. Also there remains a significant difference over
and above this between the individual operators in each group. As before, it seems
plausible that these apparent variations simply reflect variations in individual operators. It
was therefore decided to retain operator as a main effect rather than specifically including
procedure or equipment in the model. As before, this serves to encompass possible
variations with all of the above factors.

As above, for plate thickness greater than 6mm, the interaction between flaw height and
operator appears to be significant (at the 5% significance level), which suggests that there
is a statistically significant difference between the slopes of the trend lines in Figure 17.
This interaction was therefore also included in the model for the 6mm plate.
The same methodology as above (for plate thicknesses greater than 6mm) was followed;
the same conclusions apply to the 6mm plate.
17546/6-1/08 11 TWlltd
5.2.1.4 Non-linearity with respect to continuous covariates
Non-linearity over flaw height h can be tested by adding the term h.logh to the regression
model (Hosmer and Lemeshow 1989; Box and Tidewell, 1962). If the coefficient of this new
variable is statistically significant, this constitutes evidence of non-linearity in the model with
respect to h. This test showed that the linear model was adequate (at the 5% level). This
result was confirmed using a similar test based on an h' term to the model, and again
showing that its coefficient was insignificant (at the 5% level). Hence, there is at present no
need to include any non-linear terms in the model.

As above, for plate thickness greater than 6mm, non-linearity over flaw height h was tested
by adding the term h.logh to the regression model. This test showed that the linear model
was adequate (at the 5% level).
5.2.1.5 Homogeneity of the variance of the error term

One of the underlying assumptions of the above methodology (ie ANOVA) is that the
observed 'random' errors (or 'residuals') in the fitted model have constant variance E.
Figure 18 plots the variance of the residuals for each flaw height. This approach is based
on that of Draper and Smith (1981, p115), where the variance of the residuals is called the
'pure error mean square'. Although there is considerable scatter in these variances, there is
clearly a tendency for the variances (and hence the random sizing errors) to increase with
flaw height. This is supported by the finding that the slope of the trend line in Figure 18
(estimated by ordinary regression) appears to be significant (at the 5% level). This casts
some doubt on the validity of the ANOVA analysis above. This theoretical difficulty can be
overcome by the use of weighted regression, which is discussed further in Section 4.3.1.1.

Figure 19 is the equivalent plot to Figure 18 for the 6mm plate. The trend line again
suggests that the variances of the residuals increase with flaw height. In this case,
however, the slope of the trend line is not significant (at the 5% level); this may simply be
because the range of data in Figure 19 is much smaller than that in Figure 18.
5.2.1.6 Normality of the error term

Figure 20 shows a normal probability plot of the residuals for the fitted model. If the
residuals were normally distributed (as assumed in Eq.[1 n,
the data points on this plot
would lie close to the central straight line and would mostly lie within the plotted 95%
confidence limits shown on either side of this. The fact that they do not illustrates that the
residuals are not normally distributed and this finding casts further doubt on the validity of
the ANOVA analysis above. In particular, it is doubtful whether the largest of the observed
residuals in Figure 20 belong to the same distribution as the rest of the data. These may be
statistical outliers, possibly due to 'blunders' (eg misinterpretation of a large flaw as two
smaller flaws or gross errors in calibration or transcription), rather than true measurement
errors in locating the source of the ultrasonic signals from the extremities of the flaw.
In the circumstances, TWI decided to redo the analysis, after the exclusion of the results
that seemed most likely to be due to such blunders. Figure 21 shows a normal probability
plot of the raw sizing errors. Those sizing errors with an absolute value larger than -10mm
are more extreme than would generally be expected for ultrasonic measurement errors,
which are usually assumed to follow a normal distribution (eg as in Chapman, 1993); it is
assumed that these extreme sizing errors are, in fact, 'blunders' and do not follow the same
distribution as the rest of the data. The largest of these sizing errors were successively
removed, re-plotting the probability plots at each stage until the data remaining exhibited a
17546/6-1/08 12 TWI Ltd

reasonably good fit to a normal distribution. This procedure eventually resulted in discarding
21 of the observed sizing errors, all of which had an absolute value larger than 9.6mm. The
distribution of the remaining sizing errors is illustrated in Figure 22; it can be seen that this
distribution is approximately normal.

Figure 23 is the equivalent piotto Figure 20 for the 6mm plate. In this case, it can be seen
that the residuals exhibit a good fit to a normal distribution; there are now no obvious
outliers to the distribution.
5.2.2 After removal of 'outliers'

5.2.2.1 Combined data from all plates
Figure 24 shows the raw sizing errors from all the plates on the same probability plot, after
removal of the 21 outliers identified in Section 4.2.1.6. It can be seen that, for the combined
data, the most extreme sizing errors do not approximate well to a normal distribution. This,
in itself, suggests that the data for the 6mm plate follows a different distribution from the
data for the other plates. Nevertheless, the same approach as in Section 4.2.1.6 was
pursued, successively removing the largest of the sizing errors and re·plotting the
probability plots at each stage until the data remaining exhibited a reasonably good fit to a
normal distribution. This process resulted in discarding a further nine of the observed sizing
errors, all of which had an absolute value larger than 7.6mm. The distribution of the
remaining sizing errors, illustrated in Figure 25, is again approximately normal.
Once the data from the 6mm plate is included, the total amount of data becomes very large.
The derived model is also more complex; extra terms need to be included to account both
for variations with the weld shape and interactions with the weld shape, which (unlike
previously) now appear to be statistically significant (at the 5% level). A column was added
to test whether there were significant differences between the results from the small (6mm)
plate and the larger plates. Blocks with a thickness greater than 6mm were represented by
a '+' whereas the 6mm block was represented by a '.'. Then analysis of variance was used
to show that there was a highly significant difference (at the 5% level) between the mean
sizing error for the 6mm block and that for the thicker blocks; as noted in Section 5.1.1, the
mean (or 'systematic') sizing error is greater for the thicker plates than for the 6mm plate
(see also Figure 15). There was, moreover, a highly significant difference (at the 5% level)
between the standard deviation of the residuals for the 6mm block as compared to that for
the thicker blocks (in fact, the standard deviation of the residuals for the 6mm block was
about half that for the thicker blocks). In addition to the reasons set out in Section 5.1.1, this
finding makes physical sense, as only a small sizing error can be made in small plate; in
particular, the sizing error in a 6mm plate cannot be greater than 6mm. This justifies the
decision to treat the results from the 6mm plate separately from the other block
thicknesses, even though the model fitted to each set of data includes essentially the same
factors.
5.2.2.2 Plate thickness greater than 6mm

Unweighted regression
As explained above, the largest sizing errors were removed from the data for the plates
thicker than 6mm to obtain an approximately normal distribution (as expected for a
distribution of true measurement errors and as assumed in the statistical analysis). A total
of 21 outliers were removed which corresponds to about 10% of this data set.
The remaining data was then subjected to essentially the same analysis as that of Section
4.2.1. The same factors were found to be statistically significant with one exception. As
before, the operator and the flaw height were identified as main effects. However, in
contrast to Section 4.2.1, the block thickness was also found to be a main effect (at the 5%
significance level).
17546/6-1/08 13 TWI Ltd

The variation of the observed sizing errors versus the operator and the flaw height is
illustrated in Figure 26. This plot shows similar trends as Figure 16 did (before the outliers
were removed). However (not surprisingly) the relative performance of some of the
operators changes once the 'outliers' are removed.
The variation of the observed sizing errors versus block thickness and flaw height is
illustrated in Figure 27. It can be seen that, for a given flaw height, the mean sizing errors
are slightly larger (ie a greater tendency towards oversizing) in the thicker blocks than in the
thinner blocks. One possible explanation for this is that there is a little more scope for gross
oversizing of small flaws in thick blocks as compared with thinner blocks. Note, however,
that the effect of block thickness is much less pronounced than that of the operator or the
flaw height. In particular, pair-wise comparisons indicate that there is no significant
difference (at the 5% level) between the asymmetric block and the 20mm block.
The only significant interaction (at the 5% level) between the three main effects was that
between flaw height and the operator (as in Section 4.2.1.3). The linearity of the model with
respect to both flaw height and wall thickness was confirmed using the same approach as
in Section 4.2.1.4.
Weighted regression
The 6mm plate was treated separately from the other block thicknesses because it was
judged that the systematic sizing errors pertaining to the 6mm plate might follow different
trends to those for the other plates. It is assumed, however, that the random errors for both
data sets follow a common trend with flaw height. The two data sets were therefore
combined again (after removal of the 21 outliers identified in the thicker plates) and the
above model was re-fitted with the sole purpose of re-estimating this common trend in the
residual variances V (as illustrated in Figures 18 and 19 before removal of the outliers). The
resulting trend is shown in Figure 28, which suggests the relationship:
V = 3.271 + 0.3272 h [2]
The theoretical difficulty of inhomogeneous variances (ct. Section 4.2.1.5) can then be
mitigated by the use of 'weighted' regression (Draper and Smith, 1981, pp108-115). This
involves weighting each observation by a weight w(h) equal to the reciprocal of the
expected variance V indicated by Equation [2].
The use of weighted regression gives similar results as before, ie it confirms the operator,
the block thickness and the flaw height as main effects, with a significant interaction
between the operator and flaw height (at the 5% significance level). Pair-wise comparisons
again confirm that there is no significant difference (at the 5% level) between the
asymmetric block and the 20mm block.
Factors affecting random sizing errors

We can now finally test whether the standard deviations of the residuals from the above
model (ie the 'random' sizing errors) vary with the other factors of interest. In order to do
this, we first have to 'standardise' the residuals by mUltiplying each of them by VW(h); this
ensures that the standardised residuals have a homogeneous variance with respect to flaw
height (Draper and Smith, 1981, p115). It also means that the standardised residuals are
dimensionless.
The Minitab (1998) software provides two tests for equality of standard deviations: Bartlett's
test assumes that the underlying distribution is normal (Cooper, 1969), whereas Levene's
test applies to any continuous distribution. These tests do not indicate that there are
significant differences (at the 5% level) when the operators are treated as ten individuals.
However, Figure 29 indicates that, when the operators are grouped according to whether
they hold a phased array CSWIP, the standard deviation for those with the CSWIP is
slightly lower (by -20%) than for those without the CSWIP. The blue dots in the top half of
Figure 29 illustrate the standard deviations of the random sizing errors for the two groups of
17546/6-1/08 14 TWILtd
operators, with two-sided 95% confidence intervals given by the horizontal error bars. The
Minitab (1998) manual notes that the 95% confidence level applies to the family of intervals
and that the intervals are asymmetric because of the skewness of the chi-square
distribution, upon which they are based. The results of the F-test (a special case of
Bartlett's test for the comparison of just two sample variances) and Levene's test results are
also in the top half of Figure 29. Comparing the calculated P-value of 4.6% for the F-test
with a significance level of 5% indicates that the difference between the two standard
deviations is marginally significant.
The bottom half of Figure 29 is a boxplot (also called a box-and-whisker plot), which further
illustrates the spread in the distribution of random errors with reference to its quartiles. The
grey box lies between the first quartile (0,) and third quartile (0 3 ) of the distribution and the
line in the centre of the box indicates the median. The 'whiskers' are the horizontal lines on
either side of the box, which extend outwards as far as the highest and lowest observations
that are still inside the region defined by the following limits (Minitab, 1998):
• Lower limit = 0, 2 1.5 (03 2 0 , )

• Upper limit = 0 3 + 1.5 (03 2 0 , )
Observations outside these limits are plotted with asterisks.
Note that the plots in Figures 3D, 31, 35, 37 and 39 follow a similar format to those in Figure
29.
Similarly, it was found that there were significant differences (at the 5% level) between the
standard deviations of the random errors for the different flaw types, as illustrated in Figure
30. In particular, smooth non-tilted flaws appear to result in larger random errors than rough
flaws, perhaps because sizing of the former flaws typically depends on the correct
identification of low-amplitude edge-diffracted echoes.
There was no further evidence of significant variations (at the 5% level) in the standard
deviations of the standardised residuals with any of the other factors of interest, ie
procedure, equipment, block thickness, flaw type, flaw length, flaw location or weld shape.
5.2.2.3 Plate thickness equal to 6mm

There were no outliers identified in the data for the 6mm plate, so the case of unweighted
regression has already been covered in Section 4.2.1. This Section covers the case of
weighted regression, using the weights given in Equation [2].
The use of weighted regression for the data from the 6mm plate gives similar results as in
Section 4.2.1, ie it confirms that the operator and the flaw height as main effects, with a
significant interaction between the operator and flaw height (at the 5% significance level).
Factors affecting random sizing errors

Variations in the standard deviations of the standardised residuals were tested for in the
same way as in Section 4.2.2.2 (weighted regression) for plate thicknesses greater than
6mm. In this case, there are significant differences (at the 5% level) between the ten
operators, as illustrated in Figure 31.
As for the main effects (cf. Section 4.2.1.2), it is difficult to distinguish operator skill from
possible effects of the procedure/equipment. Figure 32 shows that, when the operators are
grouped according to experience, the standard deviation for those with more than five years
experience is lower (by -40%) than for those with less experience. Both Bartlett's and
Levene's tests suggest that this difference in the random errors is significant at the 5%
level. Note, however, that the standard deviations are very small in absolute terms. This is
presumably because the data set is small and there is less scope for large random errors in
the 6mm block.
17546/6-1108 15 TWlltd
There also appear to be significant differences between different groups of operators when
they are grouped by procedure or equipment. As in Section 4.2.1.2, it is unclear whether
these variations reflect genuine differences in methodology or variations in the
skill/experience of the individual operators.
For the 6mm plate (in contrast to the findings of Section 4.2.2.2 for the thicker plates), there
was no evidence of significant variations (at the 5% level) in the standard deviations of the
standardised residuals with flaw type or according to whether or not the operators held a
phased array CSWIP. Also, there was no evidence of significant variations (at the 5% level)
in these standard deviations with any of the other factors of interest, ie flaw length, flaw
location or weld shape.
5.3 Summary of strict statistical analysis

1 Both the mean ('systematic') sizing error and the scatter about this mean (the 'random'
sizing error) is greater for the thicker plates than for the 6mm plate (see Figure 16). For
this reason, these two cases were treated separately in the analysis.
2 About 10% of the sizing errors in the plates thicker than 6mm appear not to belong to
the same distribution (all larger than -10mm in magnitude) as the rest of the data.
These 'outliers' were removed from the data during the subsequent analysis. The
distribution of the remaining sizing errors is approximately normal.
('systematic') sizing error:
• Flaw height
• Operator
• Plate thickness
4 The overall trend is for the systematic sizing error to decrease with flaw height, ie there
is a greater tendency to undersize large flaws than small flaws. However, the trend is
significantly different (at the 5% level) for different operators, eg some operators have a
greater tendency to undersize large flaws than others (as illustrated in Figure 26).
5 For a given flaw height, the systematic sizing errors are slightly larger (ie greater
tendency towards oversizing) in the thicker blocks than in the thinner blocks. However,
the effect of block thickness is much less pronounced than that of the operator or the
flaw height (as illustrated in Figure 27). Also, there is no significant difference (at the
5% level) between the asymmetric block and the 20mm block.
6 In general, the random sizing errors also tend to increase with flaw height. A linear
trend was fitted to the variance of these random errors (see Figure 28).
7 The random sizing errors in the plates thicker than 6mm show significant variations (at
the 5% level) with flaw type. In partiCUlar, smooth non-tilted flaws appear to result in
larger random errors than rough flaws (see Figure 30). Also the random errors are
-20% smaller for those operators who hold a phased array CSWIP compared to those
who do not (see Figure 29). This latter effect is marginally significant at the 5% level.
8 The random sizing errors in the 6mm plate show significant variations (at the 5% level)
between different operators (see Figure 31). It is unclear whether these variations
reflect differences in procedure, equipment or the skill/experience of individual
operators. Note, however, that the random errors are -40% smaller for those operators
with more than 5 years experience, as compared to those with less experience (see
Figure 32).
6 Analysis of Manual Inspection Results

Many results from the manual UT were reported as '<3mm'. A more correct reporting of the
flaw would have been 'Pattern l' as stated in BS EN 1714, which indicates that the
17546/6-1108 16 TWI Ltd

indication was too small to size in this dimension. However, it has become common
practice to report such flaws as '<3mm'. It is difficult to work with inexact data such as this
in a rigorous statistical way. It was instead necessary to assign an unambiguous number to
the reported result of '<3mm'; for this we used both Omm and 3mm in tum. We are aware
that '<3mm' does not necessarily mean that the flaw is actually less than 3mm in size, ie
some of the manual results reported as '<3mm' are likely to actually be greater than 3mm in
size. However, it could be argued that the size will usually lie somewhere between 0 and
3mm. In conclusion, we have used both 3 and Omm in scatter plots of the data and we have
used exactly 3mm in a simplified statistical treatment of the data.
No detailed statistical analysis of the manual results has been undertaken but based upon
the conclusions of the phased array analysis a scatter plot showing sizing error versus flaw
size has been produced and is presented in Figure 33. This plot has not excluded any
outlying data points or any obvious outliers, eg due to misinterpretation of the ultrasonic
data. The plot provides trend lines for 'upper bound' and 'lower bound' cases. The term
'upper bound' is applied where the assigned size is 3mm and 'lower bound' where Omm has
been assigned. The plot shows that the larger the flaw size the greater the undersizing.
Furthermore, the larger the flaw size the larger the random error. This result is the same as
that for the phased array data, as discussed in Section 5. The scatter plot also shows that
the overall trend towards undersizing is greater for the manual UT than that of the phased
array inspection. It can also be seen that a large number of flaws were sized as '<3mm';
one operator even assigned '<3mm' to a 21 mm high flaw. No statistical tests for the
significance of these differences were undertaken; this lies outside the scope of the project.
To supplement this analysis, two simple metrics of sizing accuracy have been studied:
• Mean absolute sizing error

• Standard deviation of the sizing error
Outlying data points have not been removed from the data used in this analysis. These
metrics are presented in Table 7.
It can be seen that the manual operator without the critical flaw sizing qualification tends to
have both a larger mean absolute error and a larger standard deviation than the other
manual operators. Note that it is not possible to formally assess the statistical significance
of this result, because analysis of trends with respect to other factors (such as those in
Section 5) lies outside the scope of this project. Furthermore with only one manual operator
without a critical defect sizing qualification, it is not possible to claim a high degree of
confidence in this result. But it was observed that the operators with the critical defect sizing
qualification had reasonably consistent results and the operator without had an absolute
mean error that was almost twice as large. The results speak for themselves and can be
used for engineering guidance.
Closer examination of the manual operators' qualifications and experience revealed that the
operators with the critical flaw sizing qualification also have many other NDT certificates
and experience. Furthermore, they have all worked in the nuclear industry for many years
under strict audit conditions. In conclusion, they are probably some of the most experienced
flaw sizing operators available.
7 Phased Array Results Compared with Manual Results

The analysis of the phased array results has been discussed statistically in Section 4. The
analysis in this section compares the data for phased array and for manual UT without
taking into account outliers and other factors such as through-wall size of the flaw. Rather it
takes an overview of the full set of data without attempting to fit a formal statistical model.
As outlined in Section 5, it was necessary to find a way of simply but accurately comparing
the data from the manual UT with the phased array results. Section 4 established that
holding a critical defect sizing CSWIP in phased array UT resulted in significantly smaller
17546/6-1108 17 TWI Ltd

random errors (at the 5% level) for plates thicker than 6mm. Using this as a starting point,
we performed a simple test as detailed in Section 5 which identified that the four manual
operators who possessed a PCN with critical defect sizing also performed considerably
better than the manual operator without this qualification. But, as noted earlier, it is not
possible to claim a high degree of confidence in this finding, given that there was only one
manual operator without this qualification. To aid the comparison of manual UT with phased
array, we needed to create a like-for-like comparison of operators having comparable
ability/qualifications.
As described in Section 5, we also compared the mean absolute errors and standard
deviations of the errors for each manual UT operator with those for the phased array
operators; these results are presented in Table 7.
In Table 7, the operators are ranked by mean absolute error; where the mean absolute
errors are equal to one another, the operators are ranked by the standard deviation of the
errors instead. Note that the mean absolute error is not a conventional way of
parameterising a normal distribution; this statistic effectively reflects the magnitUde of both
the mean (systematic) error and random error. Furthermore, it does not take into account
any of the variables discussed in detail in Section 5 of this report. However, Table 7 does
indicate that the four manual operators with critical defect sizing certificates suffered smaller
flaw sizing errors than the other manual operator. This is seen in both the mean absolute
error and standard deviation.
To aid this comparative analysis, the key factors in the quality of the data interpretation
were identified, as indicated in Section 5. Operators who possess a CSWIP or PCN in
critical defect sizing (either manual or phased array) tend to give a flaw sizing performance
that is better than that of other operators. Some scalier graphs have been generated to
support this finding, using the Minitab (1998) software.
The first comparison is between the phased array operators who possess CSWIP critical
defect sizing ability and the remaining phased array operators. This is shown in Figures 34
and 35. The data shown in Figures 34 and 35 does not include the data from the 6mm plate
which did not show a clear trend (see Section 5.2.2).
Figure 35 presents a plot of the standard deviation of each group about the trend lines
illustrated in Figure 34, together with the associated 95% confidence intervals (assuming
the residuals are normal distributed). It can be seen that the CSWIP phased array operators
have a substantially smaller flaw sizing standard deviation. Further, Figure 35 shows the
spread and median for each group and again illustrates the improvement in the random
errors for the CSWIP group. As a secondary check, a similar analysis of the results has
been performed using site experience of phased array application. It should be noted at this
point that, among the ten phased array operators, only four had any substantial site
experience of flaw sizing, whereas the remaining six operators have generally been
involved in trials and development work. This is a rather SUbjective separation of the
operators and generally coincides with that of the CSWIP qualification. Figures 36 and 37
present the data for this comparison. From this analysis, again it can be seen that site
experience of application appears to be an important factor, but many of the operators who
were classified as having no site experience have worked with phased array for many
years. In conclusion, from taken in conjunction with the rigorous statistical analysis
discussed in Section 4, we believe that the influence of the CSWIP certification is clear and
well-grounded. This is discussed further in Section 7.
Having established that the CSWIP qualification was an important factor, it was decided
that the comparison between manual and phased array results would be more meaningful if
two groups of operators being compared each had a compatible level of qualification.
The plots given in Figures 38 and 39 accordingly compare manual and phased array
performance where both sets of operators have the appropriate critical defect sizing
qualifications. It can be seen from Figure 39 that the sizing errors for the phased array
17546/6-1/08 18 TWI Ltd

operators have a smaller standard deviation than that for the manual results. Thus, there
appears to be a distinction between the manual and phased array results.
The simple ranking of the absolute mean and standard deviation of errors (Table 7)
revealed that the best phased array operators performed better than that of the best manual
operators. Further, there was a very large spread of sizing ability in the phased array
operators compared with that of the manual operators.
8 Discussion
As previously pointed out, the primary aim of this project was to investigate the flaw sizing
capability of manually applied phased array ultrasonic inspection. However, following
further instructions from the sponsor group, a control inspection using manual UT was
undertaken.
8.1 Data collection and analysis

Both the manual and phased array trials were performed under blind conditions with only
one person (C R Bird of TWI) knowing the designed and manufactured flaw sizes. Each
operator was given one working week (40 hours) for data collection and analysis. The time
allowed for data collection included creation of the setups. No analysis was undertaken off
site (away from TWI) and all results were handed in to TWI after the full five days at TWI.
In general the full 40 hours was required for the data collection and analysis, and some of
the operators complained that there was insufficient time. The majority of operators
expected to complete in four days but were surprised that five days were required.
The manual inspection also required five days. Similarly, the operators were surprised by
the amount of work required.
As a conclusion the manual and phased array inspection required the same amount of man
hours. It is believed that the operators were surprised by the time required because of the
relatively large number of flaws included in the trial. Further, all the operators were mentally
tired by the end of the trial due to the large amount of detailed data analysis.
8.2 Discussion of inspection procedure

TWI supplied a procedure which was believed to provide the optimum flaw sizing ability for
the set of test blocks. This procedure did not include any linear scanning. Current industry
practice is to use linear scanning with more than one beam angle for phased array
inspection, the beam angle being chosen to be at near-normal incidence to the weld fusion
face. This tends to improve flaw detection, but it limits the through-wall sizing to the use of
the 6dB drop technique. Beam angles further away from normal incidence enable the
procedure to use the maximum amplitude sizing technique, make use echoes diffracted
from the edges of the flaws. It was anticipated that, by choosing an azimuthal scan rather
than a normal incidence linear scanning method, the flaw detection rate might be reduced.
Further, the inspection sensitivity needs to be higher to cope with the oblique angles of
incidence. Volunteer companies both followed the TWI procedure and provided additional
inspection procedures. In particular, there were two clearly different procedures applied in
addition to that of the TWI procedure. Procedure 2 used both linear and azimuthal scans
together with a 16-channel instrument. The flaw detection rate of 97.5% for this procedure
was comparable to that achieved by the better phased array operators using procedure 1.
Procedure 3 was manually applied phased array, where the operator could perform a
mixture of manual linear and phased array azimuthal scans. The detection frequency for
these scans was also 97.5%. The project scope did not include any analysis of probability
of detection (POD), but it is reasonably clear that there was no obvious variation in
detection performance among the three procedures applied.
17546/6-1/08 19 TWI Ltd

8.3 Discussion of flaw sizing with respect to flaw type
The data has been tested for differences in sizing accuracy for the following five cases, as
discussed in detail in Section 4.
• Smooth tilted
• Smooth non-tilted
• Rough tilted
• Rough non-tilted
• Volumetric
Section 4 also investigated possible variations between surface and embedded flaws.
The only significant variation (at the 5% level) in the sizing errors with flaw type was that
smooth non-tilted flaws appeared to result in larger random errors than rough flaws (for
plates >6mm thick). From a practical viewpoint, this is not unexpected. As can be seen from
the analysis, the results from these thicker plates have been dominated by quite large
sizing errors. In particular, the operators tend to misinterpret large flaws. It cannot be said
that errors greater than 6mm are due to the technique or that the flaws were inherently
misleading, but it is believed that these are essentially data analysis errors. Furthermore all
the flaws have now been analysed with knowledge of the flaw size and the operator errors.
This analysis has revealed that all the flaws can be sized correctly in the through wall
direction using the maximum amplitude technique and/or identification of the flaw tip
diffraction signal. Analysis has shown that the majority of the misinterpretation errors are
more likely where the signals from the flaw are weak and/or only tip signals are present.
This is consistent with the statistical analysis, which revealed that smooth flaws gave
significantly higher random errors. Smooth flaws lying on the fusion face can be
misinterpreted if the ultrasonic beams are not at perfectly normal incidence. Operators who
have the critical defect sizing certificate have been taught to size from tip echoes; this may
be why this set of operators achieve better overall sizing accuracy. The training of the
operators with the critical defect sizing qualification is discussed further in Section 7.5.
Further inspection of Figure 30 shows that the magnitude of the random errors for
volumetric flaws lies between that for smooth and rough flaws. In engineering terms, the
implanted flaws have mechanical features which are both smooth and rough in nature.
Hence it is not unexpected that the random errors for volumetric flaws lie between those for
the smooth and rough flaws.
8.4 Discussion of flaw sizing with respect to through-wall flaw size and test block
thickness
The trend in the mean sizing error with flaw height discussed in Section 4 initially seemed
surprising. After considering the large number flaws that were subject to misinterpretation, it
is not surprising that the larger the flaw the greater the potential for gross undersizing.
There was, of course, also potential for gross oversizing of flaws, but it can be seen that, in
practice, the trend was towards undersizing rather than oversizing. The wall thickness of
the blocks had a relatively minor effect on the sizing errors and small flaws were, on
average, not greatly oversized in the larger blocks. The only explanation of this result is that
operators are naturally disinclined to report large flaws. By detailed examination of the flaw
sizing trend lines (Figure 30) it may be observed that the operators with CSWIP critical
defect sizing errors had relatively flat trend lines. TWl's experience suggests that the most
serious errors that are made during inspection frequently arise from large flaws because the
operator's own experience often leads him to believe that a flaw is very unlikely to be as
large as that which he is observing. This was still unexpected, because TWI supplied a very
specific briefing where we told the operators to expect the unusual and unexpected. It was
noticeable that one of the operators who performed very well had previously undertaken
many controlled POD and flaw sizing trials.
1754616-1108 20 TWI Ltd

One of the aims of the project was to test whether or not asymmetric wall thicknesses made
a significant difference to through-wall flaw sizing. This was discussed in Section 4, which
concluded that there was no significant change in flaw sizing accuracy in asymmetric welds.
Nevertheless, it is foreseeable that flaw sizing would be more problematic with very difficult
weld geometries or where there is bad mismatch in the weld root due to an asymmetric
weld.
8.5 Discussion of equipment performance

The very concept of this project was such that it was going to be difficult to identify
equipment and procedure variations. In particular, once an operator had inspected the
blocks with one procedure and one set of equipment, that first inspection would influence
the results of the second inspection. As an obvious example, if the operator detected 39 out
of 40 flaws in the first test, he would be very unlikely to report less than 39 flaws the next
time he inspected the blocks. Further, there are very few, if any, operators who are equally
familiar with different sets of equipment. As shown above, there is a very large variation
between operators. To try to provide a conclusive comparison between equipment types, it
is considered that a minimum of 10 operators per equipment type would be required to
provide a statistically meaningful test of the effect of equipment type.
It was difficult to separate the effects of the equipment from those of the operator in this
study. However, it was clear that there was a great deal of operator variation and we saw
no obvious correlation between equipment and flaw sizing performance. From the results, it
can be seen that in some of the cases there was only one operator using a particular
instrument, but in other cases there were five different operators using a particular
instrument. In the latter group of five operators there was a large variation in performance
and this performance was similar to that of the variation between all the operators. Section
5 discussed the difference between manual and phased array results and quantified the
variation in operator performance.
By detailed analysis, it can be seen that both the 16 and 32 channel instruments provided
examples of both the worst and best operator sizing error results. This analysis emphasised
that the operator was the source of the dominant variation in sizing errors and not the
equipment. Further, the defect detection and false call rate appeared to be dominated by
the operator and not the procedure.
From the analysis, no clearly significant difference could be detected in equipment

performance. Further to the statistical analysis, no empirical trends could be identified.
8.6 Discussion of operator performance

8.6.1 Identification of errors in flaw sizing and detection
From a detailed analysis of the data versus operator error, it appears that there is a range
of basic data analysis errors. Particular operators appear to perform well on different types
of flaw. This project does not intend to identify operators individually, but examples of errors
and the associated flaw types are given below.
Figure 40 illustrates the phased array data from flaw 20. This shows very clear tip diffraction
signals, which provide a very accurate flaw size. Using this data, some operators sized the
flaw very accurately; others ignored the mode converted signal and classified it as two
smaller flaws. The TOFD also classified this as two flaws, but based on pre-knowledge of
the real shape of the flaw, was able to accurately size it. Figure 41 shows flaw 20 from a
near normal incidence direction. It also shows a tip echo that is detached from the main
echo.
Figure 42 illustrates the phased array data from flaw 12 (test block 3). The majority of
operators sized this flaw correctly, but some oversized it. The oversizing is assumed to be
because it is near normal incidence with a large echo and either the operator did not use
17546/6-1/08 21 TWILld
the non-saturated data to size the flaw or the operators did not use the tip signals from the
flaw.
Figure 43 illustrates the phased array data from flaw 35 (block 7). This flaw was sized very
accurately by most operators because it was a rough crack. However, some operators
badly undersized the flaw.
Analysis of probability of detection (POD) was not an objective of this project but the flaw
detection rate for each operator was recorded along with the false call rate.
As identified in Table 6, operators 6 and 7 had the highest missed defect rate (>25%); these
two operators also had the highest absolute mean sizing errors and were among the highest
three standard deviations of sizing error (see Table 7). Interviewing them revealed that both
operators had very little experience in the use of phased array equipment. If the results of
these are excluded from the data set the phased array flaw detection rate was 96%; by
contrast, if the poorest manual operator is excluded from the results the manual flaw
detection rate was 93%. If operators 6 and 7 are excluded from the missed defect statistics
the mean standard deviation for the remaining operators was 3.5%. The two poorest
operators had a detection rate considerably less than two standard deviation below that of
the mean result. Using the same treatment the mean detection rate after excluding the
poorest manual operator also showed that this operator's detection rate was more than two
standard deviations below that of the mean rate. It may be concluded that operators 6 and 7
achieved a particularly poor data analysis performance, which has an undue weight on this
project's overall results of detection rate and mean sizing error.
8.6.2 Operator training and qualification

Throughout this report, phased array CSWIP critical defect sizing has been identified as a
factor affecting flaw detection rate and sizing errors. The CSWIP that this report refers to
was a specially designed course and exam where the operators were trained to critically
size flaws using crack tip sizing including the use of crack tip diffraction signals. The exam
included a strict flaw sizing requirement. For this course the maximum flaw sizes examined
were 7mm through wall.
Conventional level 2 training and certification (phased array and conventional) does not
include defect sizing using crack tip sizing and the examination does not provide a rigorous
test of through wall sizing. Further, as far as TWI is aware, the level 2 courses do not
include training in the detection and sizing of large through wall flaws.
The majority of operators experience the inspection of very few large through wall flaws, but
the flaws that are of most concern to industry are large.
Based on analysis of the results, the following operator trends have been identified:
• The larger the flaw the greater the undersizing.

• The larger the flaw the greater the random sizing error.
• Rough flaws are sized more accurately than smooth.
• Critical defect sizing training improves operator performance.
The reason for the undersizing is not known but is believed to be psychological rather than
due to the inspection procedure. Follow-up interviews with the operators might provide
further insight into this and could potentially be included in an extension to the project. This
would involve a discussion of the results with individual operators with a view to
understanding why mistakes in interpretation were made. Such an exercise could also
provide useful insights into how to ensure more effective training provision.
From examination of the data it is clear that rough flaws provide data that is inherently
easier to analyse, leading to a reduction in gross sizing errors.
17546/6-1/08 22 lWlLtd
It is clear from this project that training and certification of operators needs to include
training in the detection and sizing of large flaws. Further, it is believed that, with
appropriate training, the operators' disinclination to report large flaws could be overcome,
In highly controlled environments like the nuclear industry, there are documents providing
detailed analysis methodologies. TWI suggests that a guidance document on phased array
data analysis should be generated and incorporated into a CEN standard.
Assessment of the statistical significance of the differences between the manual and
phased array results lies outside the scope of this project. The manual operators were
highly experienced, whereas the two phased array operators who performed best had only
2 years of ultrasonic experience. If this exercise is repeated in a few years time when there
is a larger pool of phased array operators, the distinction between phased array and
manual inspection quality may be clearer.
A further statement must be made about personnel qualification, TWI is aware that a
number of companies including TWI, BAE, ESKOM, Shell and Magnox have undertaken
extensive training and certification programs in phased array inspection and in these cases
the detection and sizing errors are substantially lower.
9 Conclusions
1 Objective 1 of this project was to assess the sizing errors associated with manually
applied phased array inspection. The detailed conclusions below break down the errors
according to operator, plate thickness, flaw size and flaw type. A single figure for the
flaw sizing error would over-simplify the results; the individual trends are illustrated in
the body of the report.
2 Objective 2 of this project has been met by showing that there is a large variation in the
ability of phased array operators to size flaws. There is also strong evidence that this
sizing capability is linked with training and experience.
3 Objectives 3 and 4 of this project were to compare the performance of different phased
array instruments and procedures respectively. In practice, it was not possible to rank
the performances of the different equipment, because any such variability was
swamped by the operator variability and the fact that each operator only used one set
of equipment. In particular, there was no evidence from this project of the 16-channel
instruments performing any worse than the 32-channel instruments.
4 Objective 5 of this project was met by identifying a statistically significant (at the 5%
level) link between the flaw type and the random sizing error for phased array. In
particular, rough flaws were identified as having a significantly lower sizing error than
smooth non-tilted flaws for plates with a thickness greater than 6mm.
As well as meeting the broad project objectives, this project resulted in the follOWing, re
detailed conclusions:
5 Both the mean phased array sizing error and the scatter about this mean is greater for
the thicker plates than for the 6mm plate.
'systematic') phased array sizing error:
• Flaw height
• Operator
• Plate thickness.
7 The overall trend is for the phased array systematic sizing error to tend towards greater
undersizing with increasing flaw height, in that the systematic error changes from a
small oversizing error to an increasing amount of undersizing as the flaw size
17546/6-1108 23 TWI Ltd

increases. This trend has also been observed in the manual ultrasonic inspection
results.
8 For a given flaw height, the systematic phased array sizing errors are slightly larger in
the thicker blocks than in the thinner blocks.
9 The phased array random sizing errors tend to increase with flaw height.
10 The phased array random errors are -20% smaller for those operators who hold a
critical defect sizing CSWIP compared to those who do not (in the plates thicker than
6mm).
11 The best phased array operators generated random and systematic errors of smaller
magnitudes than those of the best manual flaw detector operators.
12 Each flaw within the blocks was correctly sized «3mm error) by at least two operators
and each flaw was badly (>3mm error) sized by at least one operator demonstrating
that all flaws could be correctly sentenced.
13 Every phased array operator badly sized at least three flaws demonstrating a need for
improved training.
14 Flaw characterisation appears to be a generic problem amongst the operators and it is

concluded that improved training and examination is required if the NDT industry is to
provide accurate flaw reporting. The test blocks were appropriately designed to test the
ability of the systems, including the operators. Despite providing good visualisation of
flaws, the results show that flaw characterisation by swept beam phased array
techniques remains technically challenging and this should be addressed with improved
quality assurance.
10 Recommendations
• Improve the quality of data interpretation it is recommended that phased array
operators are trained to use crack tip diffraction signals and gain experience of large
(>beam width) smooth planar flaws.
• Povide a better phased array service to industry, it is recommended that examination of
operators include sizing of flaws greater than 6mm through wall and that the
examination certificate states that the operators have been examined for defect sizing.
11 References
Box G E P and Tidewell P W, 1962: 'Transformation of the independent variables'.
Technometrics 4, pp531-550.
Chapman R K, 1993: 'Guidance document on the assessment of flaw measurement errors

in the ultrasonic NDT of welds'. Nuclear Electric report TIGT/REP/0031/93 Issue 2, August.
Cooper B E, 1969: 'Statistics for experimentalists'. Pergamon, Oxford. ISBN 008 0126006.
Draper N R and Smith H, 1981: 'Applied regression analysis'. John Wiley, New York. ISBN
0-71-02995-5.
Hosmer D Wand Lemeshow S, 1989: 'Applied logistic regression'. John Wiley & Sons, New
York.
Minitab, 1998: 'Minitab reference manual - Release 12 for Windows'. Minitab Inc. (USA),
February.
1754616-1108 24 TWIUd
Shipp R, Schneider C R A, Bird C R and Wood 0 A, 2002: 'Independent qualification of
phased array inspection of fillet welds'. Proceedings of 2002 BINOT Conference.
17546/6-1/08 25 TWI Ltd

Table 1 Test block thicknesses, defect dimensions and defect geometries (RNT = rough non-tilted;
RT = rough tilted; 5NT = smooth non-tilted; 5T = smooth tilted; V = volumetric)
E
:dE ofu,gE 1ii• :5•E •••<
<
ot; • "f
gE• -,u - "•
E < E
E
E
E
E
.
.!! , u
< ~ g -' ..
• •• .!! ~
&of
... "
",. '0
Q
..
• 2"D.
• •
u
• E < E
E 4!!';= .2- •N '"C>< ~~
g'iij
•
" <
• .
...'"
Lack of '" 2" -' •
1 • V 6
rool fusion
SNT 28 40
lack of
2 - V 6
sidewall fusion
ST 3 25 88
Rough
3 • V 6
root crack
RNT 3.5 30 143
1
Lack of interrun
4 - V 6 cleaning V 2 35 208
leaving slag
Rough haz
5 • V 6
crack
RT 2.5 29 267
6 • V 6 Root concavity V 1.5 25 326
lack of side
7 - V 6
wall fusion
5T 2.5 30 88
2
Lack of side
8 - V 6
wall fusion
5T 3 40 326
9 • V 15 lack of fusion
centre Ime
5T 9 34 17.5
10 - V 15 RNT 6.4 16 69
""
Smooth vertical
11 • V 15
cracl<
SNT 10.4 10.9 34.5 130
3 12 - V 15 Lack of fusion 5T 5.1 31.8 210
13 - V 15 51. V 2.75 52.5 235
14 - V 15 Root crack RNT 7 22.5 288
Rough verieel
15 • V 15
crack
RT 5 41.1 327.5
Lack of interrun
16 V 20 V 4 20 16
fusion
4 17 - V 20 Side wall crack RT 5 18 51
Lack of
18 • V 20
root oenetration
ST 1 30 132
19 - V 20 Rouoh crack RT 5 41 266
20 • V 35 lack of fusiOn ST 24.3 23.3 31.5 44
21 - V 35 Lack of fusion 5T 7.1 31 100.5
Lack of root
5 22 • V 35
fusion
5NT 3 19 163
23 - V 35 51a line V 4.6 63.6 252
Rough toe
24 • V 35
crack
RNT 4.2 19.3 349
Lack of side
25 - V 35
waH fusion
SNT 6 7.2 31 39
Rough toe
6
26 • V 35
crack
RNT 14.1 57.2 66
Rough haz
27 V 35 RNT 8 35 208
crack
28 V 35 Rou h flaw V 6 28 271
29 - V 35 Sma or void V 7 6.24 24 329
Rough centre
35 J 50 RNT 18 32 50
line crack
36 - J 50 Gross DOrositv V 6 23 111
39 - J 50 Crack RNT 24 39.7 166
7 lad<. of side
37 - J 50
waUfusion
SNT 4.6 5.51 34 237
Lack of root
38 • J 50
fusion
SNT 2.4 45 256
40 - J 50 Lack. of fusion SNT 20 21 34 324
Lack 01 side waD
31 - X 50
fusion
ST 13 12.7 35 34
Lack of root
32 - X 50
oenetration
V 3 29 100
8 33 • X 50 Rou h crack RT 20.2 36 175
34 • X 50 Lack~:~Wau ST 5.3 5.32 15 264
Rough haz
30 X 50 RT 11 32 323
crack
17546/6-1108 TWI Ltd

Table 2 TOFD inspection results versus design size and sectioned results
c
E
~
"E
.c
E
E
"0
"-E E " "
Q>
~"O:
.~
z 1; E eE
.;I E E
"E E
""~
85 .-~~c .oC:lE
_ c.
..
",-
'"u0 "§c. ;:~

"0 "
" il :g z=> Designed DefectType .E:E 'm'" ~E co:BmE
" - "c
al u =>
ffi 'Q)'" " I" " u u "
~ mE
O.Q) ~
ffi 0
'"u .&
"2 u. " ~
£ "
0
::;"
>-
0"
>- 0 ~
0 ..
"0
c
V 6 1 Lack of root fusion 2 NA

V 6 2 Lack of sidewall fusion 3 NA
1 16
V 6 3 Rough root crack 3.5 NA
V 6 4 Siao 2 NA <1
V 6 5 Rouoh haz crack 2.5 2.75 0.25
V 6 6 Root concavity 1.5 2.52 1.02
2 16
V 6 7 Lack of side wall fusion 2.5 NA
V 6 8 Lack of side wall fusion 3 NA
V 15 9 Lack of fusion 9 10.5 1.47
V 15 10 Centre line crack 6.4 7.44 1.04
V 15 11 Smooth vertical crack 10.4 10.9 11.4 1 0.5
3 16 V 15 12 Lack of fusion 5.1 NA
V 15 13 Slag 2.75 2.67 -0.1
V 15 14 Root crack 7 7.17 0.17
V 15 15 Rough verical crack 5 5.5 0.5
V 20 16 Lack of interrun fusion 4 3.42 -0.6
V 20 17 Side wall crack 5 5.28 0.28
4 16
V 20 18 Lack of root penetration 1 1.5 0.5
V 20 19 Rough crack 5 6.2 1.2
V 35 20 Lack of fusion 24.3 23.3 21.5 -2.8 -1.8
V 35 21 Lack of fusion 7.1 7.5 0.4
5 16 V 35 22 Lack of root fusion 3 3.7 0.7
V 35 23 Slac line 4.6 4.7 0.1
V 35 24 Rough toe crack 4.2 5.5 1.3
V 35 25 Lack of side wall fusion 6 7.2 6.8 0.8 -0.4
V 35 26 Rouch toe crack 14.1 15 0.9
6 16 V 35 27 Rouoh haz crack 8 9.3 1.3
V 35 28 Rouch flaw 6 7.1 1.1
V 35 29 Slag or void 7 6.24 5.1 -1.9 -1.1
J 50 35 ROUQh centre line crack 18 19.3 1.3
J 50 36 Gross porosity 6 7.1 1.1
J 50 39 Crack 24 23.5 -0.5
7 16
J 50 37 Lack of side wall fusion 4.6 5.51 6.2 1.6 -0.7
J 50 38 Lack of root fusion 2.4 1.8 -0.6
J 50 40 Lack of fusion 20 21 19.4 -0.6 -1.6
X 50 31 Lack of side wall fusion 13 8.2 -4.8
X 50 32 Lack of root penetration 3 3.1 0.1
8 16 X 50 33 Rough crack 20.2 20.1 -0.1
X 50 34 Lack of side wall fusion 5.3 5.6 0.3
X 50 30 Rough haz crack 11 10.3 -0.7
17546/6-1/08 TWlltd
Table 3 Details of operator experience and qualifications
Manual
IPhased
Operators array Exoerience Qualifications TyPe of Exoerience
ooerator 1 PA <5 alS PA CSWIP. level II critical defect sizl Develo ment and a lication
oceraior 2 PA <5 alS PA CSWIP, level II critical defect siD Develo ment and a lieatien
operator 3 Manual >20 years peN UT level II Technician
operator 4 PA 5<10 vears Manual UT level III Development and sales
operator 5 PA 5<10 vears no Qualification, PA development Development
operator 6 PA >10 vears Manual UT level II & TOFD Development and sales
operator 7 PA >20 vears UT Level II, attended PA peN course Development & ADolication
ooerator 8 PA < 5vears no ualification, PA develo ment Develo ment
aceratcr 9 PA <5 alS peN UT level II Develo menta lication & Sales
ooerator 10 PA >20 alS PA C$W1P level II critical defect sizi . peN tevellll Technician
ooerator 11 Manual >20 alS PCN UT level I! & BE ualifications Technician
o erator 12 Manual >20 alS PCN UT level II & BE ualifications Technician
o erator 13 Manual >20 vears PCN UT level I! & BE Qualifications Technician
o erator 14 Manual >20 vears peN UT level II & BE Qualifications Technician
o erator 15 PA 10 vears no Qualification, PA development Develoomemt
17546/&-1/08 TWI Ltd

Phased Array Ultrasonic Testing Report
Provide a separate page describing the experience and

Date of Inspection Operator qualification qualifications of U1e operator (could be as a CV formal)
Operator Name Testblock No
Company Name Testblock description
Distance from Distance from weld centre Depth from top surface Through Wall
Length Max Amplitude
Flaw No. Flaw typo datum line to top of flaw to top of flaw Height
(mm) (dB)
(mm) (mm) (mm) (mm)
Table 5 Results of sectioning
Flaw Design through wall Section through

number measurement, mm wall measurement, mm
11 10.4 10.88
20 24.3 23.34
25 6 7.20
29 7 6.24
31 13 12.69
34 5.3 5.32
37 4.6 5.51 •
40 20 20.98
Table 6 Detailed results of missed and false calls
uelecllon rale
excluding
operators 6 & 7
Missed calls False calls Detection % %
operator 1 4 2 90 90
operator 2 1 1 97.5 97.5
operator 4 1 4 97.5 97.5
>-
'"
~
~
operator 15 1 1 97.5 97.5
'" operator 5 0 3 100 100
'0
Q) operator 6 10 2 75 NA
'"
;;;;
CL
operator 7 13 3 67.5 NA
operator 8 0 6 100 100
operator 9 1 2 97.5 97.5
operator 10 3 4 92.5 92.5
PAAverage 3.4 2.8 91.5 96.6
Std deviation 11.3 3.5
operator 11 2 0 95 95
I-
:::J operator 12 7 6 82.5 NA
'iii operator 13 5 2 87.5 87.5
::>
c: operator 14 2 0 95 95
'"
::E operator 3 1 0 97.5 97.5
Manual Average 3.4 1.6 91.5 93.8
Std deviation 6.3 4.3
17546/6-1/08 TWILtd
Table 7 Analysis of absolute mean error versus operator experience
I"A or Mean
Manual Error STDDev
Operator Inspection mm mm Qualifications and Exoerience
1 P 1.8 2.9 MA. TOFD. CSWIP PA Critical defect sizino. Site exoerience VT
2 P 1.9 2.3 MA. TOFD. CSWIP PA Critical defect sizino. Site experience VT
4 P 2.3 2.5 Level III many subjects. Years of Site exoerience with PA
10 PP 2.4 3.9 Level III. TOFD, CSWIP PA Critical defect sizinQ, Site experience
5 P 2.8 4.4 MAt Develooment of PA svstems
8 P 3.1 3.9 MA, 4 vear PA development experience
9 P 3.1 4.3 MSc. PCN II PA, TOFT, 4 vears experience PA development
15 PP 3.3 5.8 HNC. No formal UT Qual, No site exoerience, 10 years PA develooment
6 PP 3.6 5.1 SSe, PCN II PA TraininQ, No VT Qualifications. 4 vears PA Development
7 PP 4.0 6.4 ASNT Level1A Level 211, No site exoerience
3 Manual 2.4 3.7 UT Level 2 >10 year exoerience. Manual Critical defect sizinQ, AUS
11 Manual 2.4 4.4 UT Level 2 >10 vear ex eriance, Manual Critical defect sizina. AUS
13 Manual 2.4 6.1 UT Lel/el2 >10 vear exoerience, Manual Critical defect sizina, AUS
14 Manual 2.5 3.9 UT Level 2 >10 vear exoerience, Manual Critical defect sizina, AUS
12 Manual 4.4 6.1 UT Level 2 >10 year experience,
17546/6-1/08 TWI Ltd

Negative NO.0804A00128
Figure 1 Plate 3, flaw 11 cut at 147mm from datum.

Smooth vertical crack.

lack of fusion.
1154616-1108 lWl Ltd

Negative No.0804A00152

Lack of side wall fusion.
Figure 4 Plate 6, flaw 29 cut at 341 mm from datum.

Slag or void.
1754616-1108 TWI Ltd

Figure 5 Plate 6, flaw 29 cut at 341 mm from datum.

Slag or void.

17546/6-1108 TWlltd
Figure 7 Plate 7. flaw 37 cut at 242mm from datum


1754616-1/08 TWI Ltd



17546/6-1/08 TWllld
Figure 11 Plate 7. flaw 37 cut at 262mm from datum.


Lack of fusion.
1754616-1108 TWI Ltd

Negative No. 0804AOO 148


17546/6-1/08 TWIltd
20
• Block
• thICkness
•• • >6nm
• • X 6nm
~ 10 •
~ • II • •
..
......
0
,1Ilill"
It, r II
'!I •••
••
•• ••
•
I • ..,••
. ...':' I I
Qj
0 I·•
.~
. I• I: #I ,
••
.tt
~
I
•
I. •• • , ••
-10 •• •
• . •
•
-20 ••
•
0 5 10 15 20 25
Raw height (rnn)
Figure 15 Observed phased array sizing errors for 6mm plate versus other plate thicknesses.
-----.
30, I I Operator
1
~ 2
. -+ ' 4
----
20 -A- S
I....... *
.
6
.
0 10
4*
t' •
. If
+
~
--+--
7
B
9
10
Qj
CIl
4
~. W.tJ t ~ +" • - *' 15
••
.tt
CIl
0
....
~
.5Z'
:! -10
I .... +
-
-J
+
,.,
4
I I I I I
0 5 10 15 20 25
Raw height (rnn)
Figure 16 Observed phased array sizing errors versus flaw height and operator for plates thicker than
6mm.
1754B16-1/0B '!WI Ltd

3 • ~~
--+-
___
Opentor
21
2 _.a----
_..... -- •. · +-
..... -
4
5
I..
.... 1
~
• --.-
6
7
-
8
f!
~
C\
0 .•
~
-" _
-,
- ~ ....... _
. .-
.-:;;;;;;;;
x
&- .. _-- ~.- ~
-----~- --..., ... .
--+-- 9
10
15
.Ii:
-...'"":: ...~---"'- ... ---
.tI
!
III
~
.lZ'
-
-2
1
...
.... -- _-- - -...
....
..... - ....... -
....
- --
-3
...._~- ... ....... ....
'- .
+
-41
" I
I I I I I
1.5 2.0 2.5 3.0 3.5
Raw height (mn)
Figure 17 Observed phased array sizing errors versus flaw height and operator for 6mm plate
100 •
,.... •
N
< 80
....~ •
.
JQ
~
60
:5l
...".
III
0 40
"~ • ••
..>.
.!lI
20
•
• • • •
•• • •
0
L,
0 5 10 15 20 25
Raw height (mn)
Figure 18 Variance of residuals from model of Section 4.2.1 for phased array sizing errors versus flaw
height for plates thicker than 6mm
1754616-1108 TWI Ltd

0.8
•
~
N
0.7
<
~
•
0.6
.
~
J/l
::J
:lZ
.....
III
QI
0.5
0
QI
l!! 0.4
..
.1lI
> 0.3
•
•
0.2
1.5 2.0 2.5 3.0 3.5
Raw height (rrm)
Figure 19 Variance of residuals from model of Section 4.2.1 for phased array sizing errors versus flaw
height for 6mm plate.
Normal Probability Plot with 95% Confidence Interval

99.9
•
99 • •
95
90
80
....I: 70
..
~
8
60
50
40
30
20
10
5
0.1
-15 -10 -5 o 5 10 15 20 25
Residual (rrm)
Figure 20 Normal probability plot of residuals from model of Section 4.2.1 for phased array sizing
errors in plates thicker than 6mm.
17546/6-1108 TWI Ltd

99.9
•
99 •
9S
90
IlO
...c 70
60
~
...u SO
40
~ 30
20
10
,
S
0.1
•
-20 -10 o 10 20 30
Height sizing error (mm)
Figure 21 Normal probability plot of phased array sizing errors for plates thicker than 6mm.

99.9
99
9S
90
IlO
...c 70
60
fl... SO
40
~ 30
20
10
S
0.1
-10 -5 0 5 10
Height sizing error (mm)
Figure 22 Normal probability plot of phased array sizing errors from Figure 21 after removal of the 21
'outliers' identified in Section 4.2.1.6.
17546/6-1108 TWI Ltd

99.9
99
9S
90
III
OJ 'll
c:
..
8 so
~
60
40
30
20
10
S
O.I-'--..."'--~"----f:"'-_--r----.-------r----,r--C---.--J
-3 -2 -1 0 1 2 3
Residual (1Tf11)
Figure 23 Normal probability plot of residuals from model of Section 4.2.1 for phased array sizing
errors in 6mm plate.

99.9
99
9S
90
III
OJ 711
c:
.
8 so
~
60
40
30
20
10
S
•
O.I-'---..."--''-'':....---~-------r--------r-------r--'
-10 -5 o 5 10
Height sizing error (1Tf11)
Figure 24 Normal probability plot of combined phased array sizing errors for all plates after removal of
the 21 'outliers' identified in Section 4.2.1.6.
1754616-1/08 TWI ltd

99.9
99
95
90
..
cQJ
80
70
..
&
U
60
50
40
30
20
10
S
0.1.J.,--...L...L...L...----,------,----'----,---------r'
-10 -5 o 5 10
Height sizing error (nwn)
Figure 25 Normal probability plot of data in Figure 21 after removal of nine further phased array sizing
errors with absolute value larger than 7.6mm.
10
5
...
+.
• •
I.
•
• -----..
-6-
Operator
1
2
4
S
I...... • l!
-+--
6
7
0
...
QJ
Ol 0
• +
..
...•
- ---.
1 •
•
•
--.-
--+.--
8
9
10
15
...
.EO It
.tI
.~- ><--
.I:
.Il' ... ... ~...
lr+".-.... ...... ~
! -5 ~ ... ....,
~~ ...
1( -+-
•
-10 •
0 5 10 15 20 25
Raw height (nwn)
Figure 26 Observed phased array sizing errors versus flaw height and operator for plates thicker than
6mm. after removal of 21 'outliers'.
1754616-1/08 "!WI Ltd

10 Block
tt1ickness
• •
--
(1Ml)
• -+- 15
20
5
I.. •• ••
•
-4-
35
50
'-'
..
0 •• •• ••
III
01 0 I
.• •
*---- ~---
• ••
j • . •
.. ,. :
.6
.tl
"~
...
III
t #
.:
•• • I.
• ••
•5l'
! -5
• I • ••
••
•
-10
0 5 10 15 20 25
Raw height (1Tf11)
Figure 27 Observed phased array sizing errors versus flaw height and block thickness for blocks
thicker than 6mm. after removal of 21 'outliers'.
v = 3.271 + 0.3272 h
20 r - - - - - - - - - - - - - - - - - - - - - ,
5 4.59871
• R-Sq
R-Sq(adj)
19.6%
16.8%
~
N
<
•
~ 15
'-' •
>
JQ •
£I
~ 10
III
•••
f
'0
g •
..>.
5
.il!
0 •• •
....
•
0 5 10 15 20 25
Raw height h (1Tf11)
Figure 28 Variance of residuals from model of Section 4.2.2.1 versus flaw height for combined
phased array data (as used to derive Equation [2]).
17546/6-1/08 TWI Ltd

Test for Equal Variances for Standardised residuals
F-Test
Test 5tillislic 0.68
0;
CSWIP I • I P·Value 0.0<16
> Levene's Test
~ Test StiltJsIjC 2.10
5 P-Value 0.148
NO CSWIP I • I
0.8 0.9 1.0 J.1 1.2 1.3

95 % Bonferronf Confidence Intervals for Standard Deviations
0;
CSWIP
CD ""
...
~
5
NO CSWIP .. CD .. ..
-4 -3 -2 ·1 0 I 2 3
Standardised residuals
Figure 29 Comparison of standard deviations of the (dimensionless) standardised residuals for the
model of Section 4.2.2.2 for phased array sizing errors versus phased array qualifications for blocks
thicker than 6mm.
Bartlett's Test
RNT • Test 5tabstic 18.49
P·Value 0.001
Levene's Test:
..
~
~
RT • Test Stabstic
P-Value
3.24
0.013
c
~ SNT •
.5l'
In
2l ST •
v •
0.50 0.75 1.00 1.25 1.50 1.75 2.00
9S% Bonferroni Confidence Intervals for Standard Deviations
model of Section 4.2.2.2 for phased array sizing errors versus flaw type for blocks thicker than 6mm
(RNT = rough non-lilted; RT = rough tilted; SNT = smooth non-tilted; ST = smooth lilted; V =
volumetric).
17546/6-1/08 nYl LId

1 • Bartlett's Test
Test Stab5IIC 25.63
2 I • P-Value 0.002
levene's Test
4 • Test Stab5IIC 3.31
...
5 I • P·Value 0.002
........
0
6 •
! 7 •
8 •
9 ,e
10 •
15 Ie
0.0 0.5 1.0 1.5 2.0 2.5

95% Bonferroni ConFidence Intervals for Standard Deviations
model of Section 4.2.2.3 for phased array sizing errors versus operator for the 6mm block.

F-Test
Test StabStlC 2.66
. I • I P·Value 0.004
~ levene's Test
.!!
~
.K
~
+ I • I
Test StabStlC
P-Value
6.19
0.015
0.2 0.3 0.4 0.5 0.6 0.7

95~ 8onferroni Confidence Intervals for Standard Deviations
. I
~
.!!
~
K
.:l
+ I
-1.0 -0.5 0.0 0.5 1.0

Standardised residuals
model of Section 4.2.2.3 for phased array sizing errors versus operator experience for the 6mm block.
1754616-1/08 TWt Ltd

Flaw sizing error v Flaw Height
Manual and Phased Array Data
20
15
~ • - Lower bound
E 10 •
Manual UT
-...
E
0
5
0
• • Upper bound
Manual UT
41 -5
Cl
.5 -10 Phased array
•
~ -15 •
-20
-25
o 5 10 15 20 25 30
Flaw height (mm)
Figure 33 Comparison of sizing error for manual and phased array inspections with respect to flaw
size.
Scatterplot of Height sizing error (mm) vs Flaw height (mm)
~
30
20
• •
• --
1 __
UT Level
CSWIP
NOCSWIP
.
~
••
• •
..
2 10
CII
•
•
.~
.."'
.tl
r.
.lZ'
0
---
! -10
••
•
-20
•
0 5 10 15 20 25
Raw height (1TfTI)
Figure 34 Scatter plot of phased array data comparing the operators holding CSWIP with those
without.
1754616-1108 TWlltd
Comparison of Variances for Residuals
CSWIP I • I
~
....'"
5 NO CSWIP 1
• I
3.0 3.5 4.0 4.5 5.0 5.5

9S% Bonferroni Conrodence Intervals for Standard Deviations (mm)
Gi
CSWIP .. ...... ...
~
...J
!5 NO CSWIP .. .. .. I .. ....... ..
-20 -10 0 10 20 30
Residuals (mm)
Figure 35 Standard boxplot comparing the standard deviations of the phased array sizing errors
about the trend lines illustrated in Figure 34 for CSWIP operators versus non-CSWIP operators.
Scatterplot of height sizing error vs flaw height
~
30
20
•
• --... Site experience
No SIte expenence
Site exper.ence
....~
...
......
0 10 •
.IiO
'"
CI
..
.1:1
s:.
.~
III
0
! -10
~
-20
•
0 5 10 15 20 25
Raw height (nm)
Figure 36 Scatter plot of phased array data comparing the operators with site experience and those
without.
1754616-1/08 lWlltd
COl11lClrison of Variances for Residuals
2l
......
.~ No site experience I • I
...
)(
:t!
5le experience I • I
III
3.0 3.5 4.0 4.5 5.0 5.5 6.0

9S% Bonferroni Conrodenc:e Intervals for Standard Deviations (mm)
2l
.~
.... No site experience .. .. .. I ... '" .. ..
..
~
:t!
III
5le experience .. "" OJ .... ..
-20 -10 0 10 20 30
Residuals (mm)
Figure 37 Standard boxplot comparing the standard deviations of the phased array sizing errors
about the trend lines illustrated in Figure 36 for the operators with site experience versus those
without site experience.
Upper bound sog error vs flaw height for operators with flaw sizing quaiflCiltion
Techmque
15 • ___ Manual UT
~
•
......~ 10 • ... PA
• ••
..
0
l:lI
5 •
.S;
•
..
.!l
VI
..r.;.
0 •
• ....
.r.
. -5
•• • •• • • ••
'0
,.
l: ••
-10
• •
.8..
8- -15
•
B-
-20
•
0 5 10 15 20 25
Raw height (nm)
Figure 38 Scatter plot comparing phased array and manual sizing errors where both sels of operators
have critical defect sizing qualifications.
1754616-1/08 TWI Ltd

Comparison of Variances of Random Errors
.::s
Manual UT I • I
IZ
e:
.c:
~ PA I
• I
2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0

9S% Bonferroni ConFidence Intervals for Standard Deviations (mm)
.::s
.2'
Manual UT· • .. " • IT] """ -It... • • •
]
e:
PA· • ... ." I " ..

-15 -10 -5 0 5 10 15
Random errors (mm)
Figure 39 Comparison of random sizing errors for phased array (PA) and manual UT where both sets
of operators have critical defect sizing qualifications.
Figure 40 Data showing flaw 20 with the tip echoes clearly shown.
1754616-1/08 IWlltd
j:"",
•
....
_.
--- --
• ..
Figure 41 Flaw 20 at near normal incidence.
.... _... - ------ ..
Figure 42 Data showing flaw 12 test block 3.
1754616-1/08 TWI Lid

a
Figure 43 Data showing flaw 35 test block 7 of rough centreline crack.
17546/6-1/08 TWI Ltd

TWI-2008-Reliability of Manually Applied Phased Array Ultrasonic Inspection For Detection and Sizing of Flaws PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

TWI-2008-Reliability of Manually Applied Phased Array Ultrasonic Inspection For Detection and Sizing of Flaws PDF

Uploaded by

Copyright:

Available Formats

17546/6-1/08

Reliability of Manually Applied Phased Array

Revised Final Report

For: A Group of Sponsors

Revised Final Report

TWI Report No 17546/6-1/08

Prepared for: A Group of Sponsors

Authors: C R Bird, C R A Schneider and D Caravaca

The report has been reviewed in accordance with TWI policy.

Project Leade~ .. : TGM/Reviewer

Print name: Aamir Kh id

Print name: Peter Oakley Print name: Andrew Carey

orb Copyright C TWI Ltd 2008

5 Analysis of Phased Array Data 6

6 Analysis of Manual Inspection Results 16

7 Phased Array Results Compared with Manual Results 17

Appendix A: Phased array procedures

17546/6-1108 TWI Ltd

Work Carried Out

17546/6-1/08 TWI Ltd

1754616-1108 TWI Ltd

17546/6-1108 TWI Ltd

This resulted in the following flaw criteria.

• Smooth planar, to simulate lack of side wall fusion.

Flaw location and orientation:

• Near-normal incidence to inspection beams, ie fusion face flaws (referred to in this

3.3 Design of test blocks

17546/6-1/08 2 TWI Ltd

No. of test Weld Plate Plate

Flaw manufacture is difficult and has an associated manufacturing tolerance. Sonaspection

3.4 Inspection companies, data collection equipment and inspection procedure

British Energy/Oceaneering volunteered four manual UT operators and TWI provided a

17546/6-1/08 3 TWI Ltd

3.5 Data collection operators

17546/6-1/08 4 TWI Ltd

3.7 Flaw sectioning

17546/6-1/08 5 TWI Ltd

5 Analysis of Phased Array Data

17546/6-1/08 6 TWI Ltd

The overall aims of the statistical analysis were to:

• Provide an independent assessment of the performance of the flaw sizing accuracy of

5.1.1 Ultrasonic data

17546/1>-1/08 7 TWI Ltd

5.1.2 Potential factors affecting sizing error

Wall thickness of the block

1754616-1/08 8 TWI Ltd

5.1.3 Analytical approach

where Y is best estimate measurement error (the response variable),

5.2 Results of statistical analysis of phased array data

17546/6-1/08 9 TWI Ltd

Plate thickness equal to 6mm

5.2.1.2 Possible reasons for variations between operators

In summary, it was very difficult to distinguish operator skill from procedure/equipment as

17546/6-1/08 10 TWI Ltd

Plate thickness equal to 6mm

Plate thickness equal to 6mm

Plate thickness equal to 6mm

5.2.1.5 Homogeneity of the variance of the error term

Plate thickness equal to 6mm

5.2.1.6 Normality of the error term

17546/6-1/08 12 TWI Ltd

Plate thickness equal to 6mm

5.2.2 After removal of 'outliers'