j o u r n a l o f m a t e r i a l s p r o c e s s i n g t e c h n o l o g y 2 0 2 ( 2 0 0 8 ) 240–245

journal homepage: www.elsevier.com/locate/jmatprotec

Selection of sampling points for accurate evaluation of

flatness error using coordinate measuring machine

R. Raghunandan, P. Venkateswara Rao ∗

Department of Mechanical Engineering, Indian Institute of Technology, Delhi, Hauz Khas, New Delhi 110016, India

a r t i c l e i n f o a b s t r a c t

Article history: Dimensional inspection, in integrated manufacturing environments, requires accurate

Received 7 August 2006 inspection while minimizing the cost and time of inspection. The selection of sampling
Received in revised form plan—sample size and sample point locations, the method of evaluating the form error
19 August 2007 and the nature of the manufactured surfaces will play an important role in deciding the
Accepted 4 September 2007 best inspection strategy to be adopted. This paper deals with the strategy for evaluation
of flatness error which is one of the most commonly used form tolerances for control of
manufactured surfaces. Investigations have been carried out to ascertain the influence of
Keywords: surface quality (surface roughness) in determining the sampling strategy for accurate deter-
Coordinate measuring machine mination of flatness error while inspecting on a coordinate measuring machine (CMM). The
(CMM) sampling plan utilizes the Hammersley sequence for point location and the flatness error is
Flatness error evaluated using the minimum zone method (MZM) based on computational geometry tech-
Surface roughness niques. Results indicate that the surface roughness influences the accuracy of inspection
Sampling strategy and can be used as a parameter for determining an initial sample size for the determination
Minimum zone method of flatness error.
© 2007 Elsevier B.V. All rights reserved.

1. Introduction of CMMs it becomes important to have an efficient inspection

A present day inspection system should ideally be able to mea-
sure the dimensional characteristics of randomly presented
parts of virtually any configuration or complexity and provide
real time feedback to the manufacturing and design process.
In a typical batch manufacturing environment the coordinate
measuring machine (CMM) has emerged as a key device that
has enabled the inspection to be performed accurately even
at rapid processing speeds thereby reducing cycle times con-
siderably. Modern CMMs are designed to operate in the harsh
shop floor environments thereby providing flexibility to per-
form near instant component and process verification. CMMs
have also served to reduce the need for complex gauging fix-
tures thereby reducing not only inspection setup times but
also costs. In order to realize better productivity in the use
planning process. Traditionally inspection planning has relied
on the experience of inspection planners, which tends to lead
to inconsistencies in the inspection plan. Hence it is impor-
tant to have a systematic inspection plan that can help in the
gathering of meaningful data in order to analyse the process
and equipment conditions (Hwang et al., 2002). Automating
the task of inspection planning would greatly enable to reduce
the inconsistencies and result in more robust plans.
Automating the task of inspection planning in the context
of use of a CMM will involve several tasks such as integration
with the CAD system to extract inspection specific informa-
tion, generating part setups and probe orientation, creation
of a sampling plan and probe path planning. In recent years
the quality and functional requirements of products have
undergone tremendous changes which have necessitated the

∗
Corresponding author. Fax: +91 11 26582053.
E-mail address: pvrao@mech.iitd.ernet.in (P. Venkateswara Rao).
0924-0136/$ – see front matter © 2007 Elsevier B.V. All rights reserved.
doi:10.1016/j.jmatprotec.2007.09.066
 j o u r n a l o f m a t e r i a l s p r o c e s s i n g t e c h n o l o g y 2 0 2 ( 2 0 0 8 ) 240–245
shift towards tighter dimensional and geometric tolerances.
A typical component would have several dimensions and sur-
faces with tolerances specified by the designer and hence an
accurate and efficient measurement processes needs to be
developed. The sampling plan forms the core of the inspec-
tion process, since it brings out all the necessary activities
and details for inspecting a component for conformance to
specifications. While using the CMM, the inspection of dimen-
sional errors is relatively easy and reliable, where as the form
errors are more difficult to measure and quantify. Of the sev-
eral types of form tolerances, the flatness tolerance finds a
large instance of usage by designers and hence is taken up
for study in this work. The sampling strategy that needs to be
adopted for evaluating the flatness error forms the deliverable
of this work.
2. Sampling strategy
The sampling strategy selected plays an important role since
it enables to make valid inferences about dimensions and
forms of a workpiece. A knowledge of the entire surface is
needed to be able to ascertain the form errors with good accu-
racy. Sampling strategies while using CMM have to be selected
properly since the discrete sample that a CMM uses to mea-
sure should be able to provide adequate information about
the surface to determine the form error and at the same time
reduce the inspection cost and time. The sampling strategy
comprises of the optimum sample size and the locations of
the sampling points. The determination of the sample size is
a complicated process since it is affected by many factors such
as the manufacturing process used, tolerance specifications,
error evaluation method and confidence level of measured
results (Zhang et al., 1996). Since the sample should be a
good representative of the entire surface, the location of the
points should be such that a maximum amount of informa-
tion is obtained. These locations could be determined either
using statistical procedures or from the knowledge of manu-
facturing processes or a combination of both. Manufacturing
processes exhibit variations which result in different feature
surfaces on a workpiece having different variations in their
dimensional accuracy and surface finish. The variations can
differ considerably from one surface to another even though
those surfaces may share the same features (Lee et al., 1997).
Identification of the characteristics of a surface resulting
from a manufacturing process can be very complicated, which
may require extensive modeling of the manufacturing process
and with a large number of manufacturing processes avail-
able it is a challenging task to obtain proven models. In the
absence of such models an approach using the surface fin-
ish as a parameter to determine the sampling size could be
adopted as the surface finish can adequately represent a man-
ufacturing process.
2.1. Sampling size
The CMM inspection is based on a discrete measurement
strategy, which is inherently an approximation process—if the
sample size is infinite, the error in approximation approaches
zero and whenever the sample size is finite the error must
241
be non-zero (Woo and Liang, 1993). The sample size is also
proportional to the time and cost and savings in time can be
achieved through a reduction in sample size (Kim and Raman,
2000). One of the earliest work done in this direction was
by Yau and Menq (1992) who proposed a statistical approach
to determine the sample size in relation to process capabil-
ity. Hurt (1980) has also reported that a larger sample size
improves the accuracy if the least square method of evaluation
is employed. Lin and Chen (1997) identified pattern elements
in a feature based solid model and the sample size was a func-
tion of the pattern element arising after modeling. The sample
size data has also been referred from the British Standard BS
7172 which recommends that the sample size can vary from 5
to 15 points for various individual shapes and features (Zhang
et al., 2000; Beg and Shunmugam, 2002). Also, Fan and Leu
(1998) have considered the sample size based on the shape of
the planar face and the ratio between the length and breadth
of the area being measured. These reports have not discussed
the measurement uncertainty arising while considering the
sampling size. Zhang et al. (1996) have used a neural network
approach to determine the sample size for the inspection of
holes using the process type, size of hole and tolerance band
as factors. Hwang et al. (2002) have used a hybrid neuro-fuzzy
approach considering the tolerance and geometry features as
factors. It becomes apparent that arriving at the sample size
relies a lot on the manufacturing process used, but as men-
tioned earlier, problems can arise to establish concrete results
due to the variety in manufacturing processes that can be used
to produce the same kind of surface and also the variability
of a manufacturing process itself. In order to account for the
influence of the manufacturing process in determination of
the sample size, in this work the surface roughness is taken
as a basis since it can represent the manufacturing process
and also provide information as regard to variations obtained
on the surface at different locations.
2.2. Sample point locations
The quantity of information obtained in a sample depends on
the number of points sampled and the amount of variation
in the data. This latter factor can be controlled by selecting
suitable locations of the points on a surface. The location of
points should be so selected so as to provide the necessary
precision in terms of a bound on the error of estimation for
a minimum cost (Lee et al., 1997). Roth established a lower
bound in the discrepancy D for a finite set of N points, in d
dimensions (Woo and Liang, 1993):
D∼
= O(N−1 (log N(d−1)/2 ) (1)
where O is a constant in terms of the dimension d.
It can be observed that the lower bound is expressed only
in terms of N and d. It is possible to now determine whether a
finite set of samples of size N, giving rise to a discrepancy D,
meets Roth’s bound and hence giving the least amount of error
possible (Woo and Liang, 1993). From Eq. (1) it can be observed
that the sample size is inversely proportional to error for d = 1
and as the dimensionality increases the discrepancy is also
directly proportional to some power of the logarithm of N.
242 j o u r n a l o f m a t e r i a l s p r o c e s s i n g t e c h n o l o g y 2 0 2 ( 2 0 0 8 ) 240–245
Some of the commonly used methods for identifying the
location of the sample points are the uniform, random and
stratified sampling (Fan and Leu, 1998). Hammersley sequence
sampling and Harlton Zaremba sampling are two other meth-
ods that have been experimented with by researchers. Woo
and Liang (1993) introduced the application of Hammers-
ley sequence for surface measurement which is found to
approach Roth’s bound and observed that it leads to a
quadratic reduction in the number of sample points for a given
accuracy. This has been further verified by Lee et al. (1997)
who integrated the Hammersley sequence and stratified sam-
pling to improve on the results for turned components. Kim
and Raman (2000) conducted an exclusive study on the use of
sampling methods for flatness measurement and concluded
that both the Harlton Zaremba sequence and the Hammers-
ley sequence are good strategies for efficiency and accuracy.
The Harlton Zaremba sequence however suffers from a lim-
itation that the number of sample points has to be a power
of 2. Badar et al. (2005) have identified the location of sample
points based on the surface error pattern and optimization
search heuristics in end and face milling. This approach relies
on the availability of a proven model for surface error pattern,
which may not be present for all types of processes.
It can be seen that the Hammersley sequence offers a good
solution with flexibility to be adopted for locating the sample
points and hence is selected as the approach in this work to
locate the sample points.
In order to locate the points using the Hammersley
sequence, the surface is assumed to be a unit square address-
able by coordinates (s, t) where s, t ∈ [0, 1] and the coordinates
(si , ti ) of a Hammersley point in 2D are (Lee et al., 1997):
i The main objective of this work was to focus on obtaining a
si =
N sampling strategy for the evaluation of flatness error using
k−1 
  
i keeping in mind the following observations: (a) the surface
ti = Mod 2 × 2−j−1
2j
j=0
where i ∈ [0, N − 1], N is the sample size and k = [log2 N].
In this effort the Hammersley sequence for different sam-
ple sizes has been calculated through a programme written in
MATLAB6p5, using the above Eqs. (2) and (3).
3. Evaluation of flatness error
The assessment of flatness error is typically done by computer
based algorithms which should be robust, efficient and based
on proven mathematical principles. Also, these programmes
should follow the guidelines laid down in the standards. The
ANSI Dimensioning and Tolerance Standard Y14.5 defines,
evaluation of form tolerances on a component with reference
to an ideal geometric form. For evaluation of form, ISO stan-
dards specify that the ideal feature must be established from
the actual measurements, such that the deviation between it
and the actual feature concerned is the least possible value
(Samuel and Shunmugam, 1999). The CMM data is analysed
using techniques based on curve fitting. The two most com-
mon fitting techniques are the least square method (LSM) and
the minimum zone method (MZM). The LSM is superior to
MZM from the viewpoint of simplicity and calculating time.
LSM generally overestimates a form error and hence can lead
to rejection of good parts where as MZM tends to underesti-
mate the form error and is very sensitive to asperities which if
undetected can lead to poor results (Samuel and Shunmugam,
1999; Badar et al., 2003). The LSM does not follow the stan-
dards intently and may not guarantee the minimum zone
solution specified in the standards (Samuel and Shunmugam,
1999). With the availability of good high speed computers,
the speed of evaluation of errors using MZM is not a critical
issue especially if the sample sizes are not unusually large
(>500 points). Considering these factors the MZM approach
has been adopted in this work for assessment of flatness
errors.
Several numerical methods exist to obtain the minimum
zone solution: (a) methods based on Monte Carlo, simplex and
spiral search; (b) Chebyshev approximations; (c) non-linear
optimization approach and also conversion from non-linear
to a linear optimization problem. These numerical methods
require a large number of trials to satisfy the convergence cri-
teria and the computational time required is longer (Samuel
and Shunmugam, 1999). Another approach is the computa-
tional geometry based technique which has shown greater
promise for solving the minimum zone problem (Samuel and
Shunmugam, 1999; Lee, 1997; Barber et al., 1996). These tech-
niques are based on the determination of a convex hull for
the given finite set of CMM data and then determining the
minimum zone solution from analysis of the hull.
4. Experimental method
(2)
the CMM. The experimental method adopted was designed
(3)
finish is a parameter to determine the sample size; (b) the use
of Hammersley sequence for locating the sample points; (c)
the MZ method would be used to calculate the flatness error
and its behaviour for different sample sizes.
In order to meet the above requirements, three different
surfaces (different surface roughness) needed to be created
with an adequate area to accommodate varying sample sizes
with a maximum sample size of 64 points. The machining pro-
cesses were selected so as to obtain a suitably large range of
surface roughness values to be able to adequately capture the
variation in the flatness error. The approach even though being
arbitrary would suffice to bring out a relationship between the
surface roughness and the flatness error. Three workpieces
of size 60 mm × 60 mm were machined using shaping, face
milling and surface grinding to obtain surfaces of differing
roughness. Table 1 shows the workpiece specifications that
were obtained.
The effective inspection area was taken as approx.
50 mm × 50 mm to avoid any excessive deviations present in
the corners and edges due to the machining processes. In
order to capture the behaviour of the flatness error with
change in the sample size for a given surface nine sample
sizes were selected at random—10, 16, 20, 26, 32, 40, 48, 58
and 64 points. As mentioned in the earlier sections, the loca-
 j o u r n a l o f m a t e r i a l s p r o c e s s i n g t e c h n o l o g y 2 0 2 ( 2 0 0 8 ) 240–245 243

Table 1 – Workpiece specifications

Specimen No. Machining process Surface
roughness (␮m)

1 Shaping ∼9.0
2 Face milling using end ∼3.5
mill cutter
3 Surface grinding ∼0.2

tion of sample points for each sample was determined using

the Hammersley sequence.
The measurement was carried out on a CARL ZEISS CMM
with a single probe. For a given sample, the (x, y) coordinates
of the sample point locations were provided to the CMM and
for each point the corresponding z coordinate was measured
from which the deviation with respect to the ideal surface was Fig. 2 – Variation of flatness error w.r.t. sample size for Ra
computed. This data was then exported to MATLAB where a 3.5 ␮m.
programme was written to compute the flatness error using
the MZ method. This approach was adopted since the CMM
data analysis software did not have the functionality to deter-
mine the flatness error using the MZ method. The technique
used for evaluating the flatness error was to first create the
three-dimensional convex hull based on the approach pro-
posed by Barber et al. (1996). This was selected due to its
portability in the MATLAB software. After construction of the
convex hull, the antipodal pairs of face and vertex were deter-
mined by taking the face and the farthest vertex from that face
of the hull. Then the parallel planes resting on each face and
passing through the other vertex were determined. The pair
of planes that gives the minimum distance yields the value
of minimum zone flatness error (Samuel and Shunmugam,
1999). The procedure was repeated for the next two workpieces
to calculate the flatness.
Figs. 1–3 show the variation of flatness error with respect
to the sample size for each specimen. The first observation
from Figs. 1–3, is that the flatness error shows considerable
variation with sample size in the case of the rougher speci- Fig. 3 – Variation of flatness error w.r.t. sample size for Ra
men (Ra 9.0 ␮m) and the variation reduces as the roughness 0.2 ␮m.
value improves. The flatness error also does not converge to

stable values unless the sample size is large (>50) which has
also been reported in Badar et al. (2003). It is also seen that in
the case of a smoother surface the convergence of the error
will be at a smaller sample size than a rougher surface which
gives a preliminary indication that the sample size for accu-
rate inspection can be related to the surface finish.
The next step was to determine the discrepancy of mea-
surement, for which it was necessary to ascertain the flatness
error for a sufficiently large sample size to take it as a basis for
calculating the discrepancy. It is assumed that this value of
flatness error would be of high accuracy and more certainty
and further increase in sample size would have negligible
effect on the flatness error. The literature available does not
mention any criteria that can be used to determine this value
of a large sample size. In an earlier work carried out by Kim and
Raman (2000), a sample size of 300 points was considered for a
Fig. 1 – Variation of flatness error w.r.t. sample size for Ra specimen of 76.2 mm × 76.2 mm area which has been reported
9.0 ␮m. as an arbitrary assumption.
244 j o u r n a l o f m a t e r i a l s p r o c e s s i n g t e c h n o l o g y 2 0 2 ( 2 0 0 8 ) 240–245

Table 2 – Details of the curve fitting for flatness error

Specimen Curve fit details Y = A × log(x) + C Goodness of fit (R2 )

A C

With Ra 9.0 ␮m 0.0045 0.01 0.827

With Ra 3.5 ␮m 0.0047 0.002 0.837
With Ra 0.2 ␮m 0.0009 0.002 0.942

Table 3 – Values of flatness error from the trend lines

Specimen Flatness error at
1000 points (mm)

With Ra 9.0 ␮m 0.041

With Ra 3.5 ␮m 0.034
With Ra 0.2 ␮m 0.008

will enable to ascertain for example the sample size at which

the inspection accuracy will be 80%. The discrepancy rate
(r) was calculated based on this maximum achievable accu-
racy of measurement. The formula used for calculation is
“r = ((a − b)/a) × 100, where a is the flatness error at 1000 points
and b is the data value obtained from actual measurement.
The discrepancy rates were calculated for each of the speci-
Fig. 4 – Comparison of flatness error trends for the three mens and the results obtained are shown in Fig. 5.
specimens. Using this discrepancy data, a curve fit procedure was
adopted to ascertain the trend of the discrepancy. The fitting
was achieved through a rational fit given by f(x) = p/(x + q) which
In this work a sample of 1000 points was considered, as gave reasonable results.
this would give one point for every 2.5 mm2 area of workpiece It can be seen from Fig. 5 for a given sample size that as
thereby conveying an adequate amount of information about the surface roughness increases the percentage discrepancy
the surface. The value of the error at 1000 points was calcu- also increases, the increase being more pronounced at lower
lated from the trend curve obtained using the data from the values of surface roughness. This is because of the reduced
nine samples for the workpiece. The maximum sample size for variability of a surface of good quality across its area and so a
which flatness error was computed was 64 and this was inad- smaller sample can be a good representative of the population.
equate to fit a trend curve which would truly represent the Rougher surfaces will require more points since the variabil-
behaviour of the flatness error of the surface w.r.t. the sample ity of the surface is more. Also, inconsistency in measurement
size. To overcome this, the flatness error for 314 points was can arise from the fact that the probe has fixed diameter and
calculated which is obtained from the cumulative data of the the point of contact during measurement of rough surfaces
nine samples that were measured. This data was included in
the curve fitting procedure along with the nine other sample
data.
The curve fitting procedure was carried out using the curve
fit tool box in MATLAB. Several types of fit were evaluated and
it was found that the fit using the equation f(x) = A × log(x) + C
gave promising results. Table 2 gives the details of the fit and
Fig. 4 shows the trend obtained for the three specimens.
From the trend lines in Fig. 4, it can be seen that it may
be prudent to take a sample size of at least 1000 points to
assess the flatness error with high accuracy. Using the analysis
feature (curve equation) in the toolbox, the value of flatness
error at 1000 points was estimated for the different specimens.
Table 3 shows the result of the analysis of the trends at 1000
points for the specimens.
This data can now be used to determine the accuracy of
the measurement of a given sample size by determining the
discrepancy of measurement for each sample size of a given
specimen. The knowledge of the discrepancy of measurement Fig. 5 – Comparison of discrepancy for the three specimens.
 j o u r n a l o f m a t e r i a l s p r o c e s s i n g t e c h n o l o g y 2 0 2 ( 2 0 0 8 ) 240–245
Table 4 – Number of inspection points required for the
given surface quality
Specimen Sample size (pts)
Ra 0.2 ∼137
Ra 3.5 ∼208
Ra 9.0 ∼264
may not be exactly at the bottom tip of the probe. This shows
that surface finish does influence the accuracy of measure-
ment. It can also be observed that for a given accuracy of
measurement the sample size for a surface of good quality is
lesser than that of a surface with poorer quality. As an exam-
ple the results of the calculated samples sizes for 80% accurate
inspection, which are tabulated in Table 4, show that there
could be a difference of more than 100 points in the sample
size for a good quality surface and a poor quality surface.
This shows that the surface finish can be used as a criteria
to determine the sample size for flatness error measure-
ment. An observation that emerges from the study is that
the accuracy of measurement is low at smaller sample sizes
and increases with increase in sample size and beyond a cer-
tain sample size the change in accuracy is little affected by
a change in sample size. This trend is independent of the
type of surface. Also, it can be seen that the sample size is
relatively large (>100) for all specimens for 80% accuracy of
measurement, and hence, it may not be feasible to measure
all successive parts using the same sample size in batch or
mass production environments. In such cases, the first part
can be subjected to inspection on the lines of the above pro-
posed method and this information can then be taken as a
starting point to capture the errors at an initial stage which
would help in reducing the sample sizes for the succeeding
parts.
5. Conclusions
This study establishes that surface finish does play an
important role in determining the sample size for accurate
inspection of flatness errors and can now be included as a
parameter along with other parameters such as part geom-
etry, tolerance band and surface area to help in determining
the best sample size. The sample size increases with increase
in the Ra value of the surface and hence poor quality sur-
faces will require more number of points to be sampled to
obtain accurate value of flatness error. So it would be possible
to obtain the required number of inspection points for flatness
error measurement on the basis of surface quality with a good
245
degree of accuracy instead of arbitrarily taking a sample size
for measurement.
references
Badar, M.A., Raman, S., Pulat, P.S., 2003. Intelligent search-based
selection of sample points for straightness and flatness
estimation. J. Manuf. Sci. Eng. 125, 263–271.
Badar, M.A., Raman, S., Pulat, P.S., 2005. Experimental verification
of manufacturing error pattern and its utilization in form
tolerance sampling. Int. J. Mach. Tools Manuf. 45,
63–73.
Barber, C.B., Dobkin, D.P., Huhdanpaa, H.T., 1996. The Quickhull
algorithm for convex Hulls. ACM Trans. Math. Software 22 (4),
469–483.
Beg, J., Shunmugam, M.S., 2002. An object oriented planner for
inspection of prismatic parts OOPIPP. Int. J. Adv. Manuf.
Technol. 19, 905–916.
Fan, K.C., Leu, M.C., 1998. Intelligent planning of CAD-directed
inspection for coordinate measuring machines. Comput.
Integr. Manuf. Syst. 11 (1–2), 43–51.
Hurt, J.J., 1980. A comparison of several plane fit algorithm. CIRP
Ann. 29 (1), 381–384.
Hwang, I., Lee, H., Ha, S., 2002. Hybrid neuro-fuzzy approach of
measuring points for knowledge based inspection planning.
Int. J. Prod. Res. 40 (11), 2507–2520.
Kim, W.-S., Raman, S., 2000. On the selection of flatness
measurement points in coordinate measuring machine
inspection. Int. J. Mach. Tools Manuf. 40 (3), 427–443.
Lee, M.K., 1997. A new convex—hull based approach to evaluating
flatness tolerance. Comput. Aided Des. 29 (12), 861–868.
Lee, G., Mou, J., Shen, Y., 1997. Sampling strategy design for
dimensional measurement of geometric features using
coordinate measuring machine. Int. J. Mach. Tools Manuf. 37,
917–934.
Lin, Z.C., Chen, C.C., 1997. Study of the automatic planning of
measurement points with basic element features. Int. J. Prod.
Res. 35 (11), 3157–3178.
Samuel, G.L., Shunmugam, M.S., 1999. Evaluation of straightness
and flatness error using computational geometric techniques.
Comput. Aided Des. 31, 829–843.
Woo, T.C., Liang, R., 1993. Dimensional measurement of surfaces
and their sampling. Comput. Aided Des. 25 (4), 233–239.
Yau, H.-T., Menq, C.H., 1992. An automated dimensional
inspection environment for manufactured parts using
coordinate measuring machines. Int. J. Prod. Res. 30 (7),
1517–1536.
Zhang, Y.F., Nee, A.Y.C., Fuh, J.Y.H., Neo, K.S., Loy, H.K., 1996. A
neutral network approach to determining optimal inspection
sampling size for CMM. Comput. Integr. Manuf. Syst. 9,
161–169.
Zhang, S.G., Ajmal, A., Wootton, J., Chisholm, A., 2000. A Feature
based inspection process planning system for co-ordinate
measuring machine (CMM). J. Mater. Process. Technol. 107,
111–118.