Accred Qual Assur

DOI 10.1007/s00769-016-1245-5

GENERAL PAPER

An enhanced bootstrap method to detect possible fraudulent
behavior in testing facilities
Rosembergue P. de Souza1 • Luiz F. R. C. Carmo1 • Luci Pirmez1

Received: 12 March 2016 / Accepted: 12 November 2016
Ó Springer-Verlag Berlin Heidelberg 2017

Abstract The testing process determines the specified Introduction
characteristics of an object. Testing is commonly used to
verify the quality, safety, reliability, efficiency, and envi- The word fraud describes the actions that are surrepti-
ronmental sustainability of products and services. In Brazil, tiously undertaken by one party to obtain an unjust
the National Institute of Metrology, Quality and Technol- advantage over another [1]. Undertaking fraudulent
ogy (Inmetro) is responsible for accrediting the facilities behavior when testing processes can result in products that
that undertake testing and calibration activities. Typical are dangerous to the ordinary citizen and to the environ-
accredited testing services include: water quality, clinical ment. Further, fraudulent behavior produces unfair
tests, emissions tests, and vehicle safety tests. Undertaking competition between concurrent facilities. In Brazil, the
fraudulent behavior when testing processes can result in National Institute of Metrology, Quality and Technology
products that are dangerous to the ordinary citizen and to (Inmetro) is responsible for accrediting the facilities that
the environment. Further, fraudulent behavior produces undertake testing and calibration activities.
unfair competition between concurrent organizations. This Inmetro evaluates these accredited organizations annu-
paper presents a method to detect possible fraudulent ally to verify their compliance with technical requirements.
behavior in accredited testing facilities. The proposed Inmetro also maintains a list of trained and qualified
technique combines the bootstrap method and the Demp- experts. Additionally, Inmetro undertakes some surprise
ster–Shafer theory of evidence, to investigate several visits to its accredited organizations. Those in-loco
fraudulent aspects at once. To validate the proposed inspections are to verify whether the accredited organiza-
method, we use a Brazilian real-world dataset with real tions are continually following the technical regulations.
cases of fraud. The proposed technique successfully detects However, the high inspection costs mean that it is not
the accredited testing facilities with fraudulent behavior. financially viable to inspect all of the accredited testing
facilities. To enhance the efficiency of such surveillance
Keywords Testing  Fraud  Bootstrap  activities, we propose a statistical method that combines
Dempster–Shafer theory the bootstrap method and Dempster–Shafer theory. The
objective is to identify the facilities with suspicious
behavior; that is, to identify the potential candidates for
Inmetro inspections. On identifying the suspicious facili-
ties, Inmetro could undertake detailed investigations on the
tests performed by these organizations, using the tech-
niques presented in [2, 3].
& Rosembergue P. de Souza Bootstrapping is suitable to identify suspicious testing
rosembergue.souza@ppgi.ufrj.br facilities, because it can be used to test hypotheses
1 involving comparisons between two or more groups. The
Av. Athos da Silveira Ramos, 274, Prédio do CCMN -
Cidade Universitária, Ilha do Fundão, technique does not require any distribution assumptions,
Rio de Janeiro 21941-916, Brazil and it is relatively easy to account for the specifics of each

123

after repeating this procedure M times. lead to rejection of the null hypothesis H0 with the given While these proposed fraud detection methods are data. to combine evidence from different sources and yields a the first step is setting up the null hypothesis H0. S. the model of the approach used in this work. hypothesis. extreme. and the significance level a. If the fraud behavior when using bootstrapping. examine the suitability of existing methods the sample evidence suggests that H0 is false. using the significance level. neighborhood. is the same time. there is room for improvements. in both an informal we continue to believe in the plausibility of the null graphical mode and as a bootstrap method. followed by the general Finally. or (3) the form of an entire probability aspect at a time. including bootstrapping. They propose corresponding test statistics and critical area of the distribution falls under both distribu- carry out bootstrap tests [6]. commit fraud. tion tails. this cutoff is the significance level a. value. than this value in the empirical tistical hypothesis is a claim or assertion about (1) the distribution of S [4. and it can be concluded that the alternative hypothesis is more plausible. The of the squared differences between the correlations in p value is the smallest level of significance that would each site and the correlations in all of the data sites [7]. then we theory to merge several aspects of fraudulent behavior in reject H0 [11]. The two-sided test decides whether a parameter study the application of central statistical monitoring is either greater or less than some value. If the for fraud detection. In that case. or equal to some angular. They identify and explore parameter is less than. one degree of anomaly. or (2) the values of several Traditionally. A two-sided hypothesis test is undertaken when the tering. evidence [8]. To make a proper decision about whether or not to somewhat satisfactory. Wu and Carlsson critical area of the distribution falls under only one of the discuss the potential for using statistical clustering distribution tails. details [5]. then identify fake questionnaire data. less than 1000 for a test at the 5 % level of significance. denoted by H0. Combining multiple tests could make it more difficult to and M cannot be less than 5000 for a test at the 1 % level. then we fail to reject We use the bootstrap method and Dempster–Shafer H0. bootstrapping allows for the use of alternative hypothesis. The value s0 of S for the proposed model with real-world data. the significance level a is the probability of ical trials. Correspondingly. The one-sided test decides whether a methods in fraud detection. M cannot be these falsified data are likely to fail other tests [9]. The data are then resampled through replacement. one has to determine a cutoff probability for the For example. Subsequently. In testing a methods. is the assertion nonstandard test statistics [4]. A sta. To evaluate more than one aspect at the distribution. 12–15]. greater than. and the final the observed data is compared with the distribution of section concludes. resampling with replacement can be decided. The two possible conclusions from a technique exploits the underlying correlation structure of hypothesis testing analysis are (1) reject H0 and (2) fail a questionnaire and the difficulty in fabricating such to reject H0 [10]. Finally. Some studies use the that is contradictory to H0. We then check empirical distribution is built for S. because Then. 123 . in data from real clin. For example. value of a single parameter. Otherwise. the degree of belief that takes into account all of the available alternative hypothesis Ha. then the Background null hypotheses is accepted. considering that fraudulent data A one-sided hypothesis test is undertaken when the cases have certain clustering structures. rejected in favor of the alternative hypothesis only if Taylor et al. if the p value is less than a. and repeated measurements clus. bootstrapping is based on the sum rejecting the null hypothesis H0 when it is true. The proposed hypothesis. The significance level of s0 A hypothesis test decides which of two contradictory is the proportion or percentage of values that are as claims about a statistical distribution is correct. the number M of when data are falsified to pass certain statistical tests. The null hypothesis. Further. reject H0. denoted by Ha. if s0 is unusually large. such studies only consider one aspect of p value. a fusion technique—such as the Dempster– claim that is initially assumed to be true. if s0 looks like a typical value from the empirical distribution of S. Accred Qual Assur situation. p value is greater than or equal to a. in a one-sided test to decide whether a parameter is greater than some value. and the The remainder of this paper is structured as follows: We statistic of interest S is calculated for the resampled data. then the data are unlikely to have arisen if the null Hypothesis testing with bootstrapping model is true. or more extreme. first present some basic concepts. as in other hypothesis tests. The null hypothesis will be bootstrap method as a tool in the fraud detection process. a bootstrap method is used to evaluate one parameters. This method is particularly useful. They propose a new technique to sample evidence does not strongly contradict H0. The Shafer theory—is more appropriate. Kirkwood et al. Dempster–Shafer theory allows us In the bootstrap method.

The power set is 2X = {. and the alternative is the degree of anomaly S. we need to model the basic belief assignment states is warranted. it is using the Dempster–Shafer theory of evidence. we cal- support given to a subset in X. event in X is assigned a value. Then. then this site is marked as suspicious. D. We then perform the enhanced bootstrap method on the In Brazil. which is denoted by the symbol 2X. This method of obtaining degrees of belief for one question based on the combines several aspects of fraudulent behavior in one subjective probabilities for a related question. called the basic belief assignment. One piece of evidence is related to a single X = {A. Within Dempster– ing the degree of anomaly S for the selected site. ensuring that the sum of all The enhanced bootstrap procedure starts with calculat- of the possible outcomes sums to unity. Sub- C\D¼Z6¼. calculated or of a conjunction of possible states. when the sites are compared. The set of all of the subsets of X is known as the power set. in the empirical distribution of S. Some of all of the suspicious sites. which is denoted by m: 2X ? [0. which is Ha: The testing facility is anomalous represented by X. Therefore. example: 123 . A belief form a pseudo-site by sampling with replacement n sub- measure is given by the function B: 2X ? [0. Dempster’s rule can be used culate the degree of anomaly S for the created pseudo-site. 1]. N. to combine two independent sources of evidence. proposed enhanced bootstrap method are: The set of all of the possible states of the system con- H0: The testing facility is normal sidered is called the frame of discernment. or to a set of possible states [16]. Proposed procedure This study proposes a technique that combines the boot. and the denominator sum native hypothesis Ha is more plausible for the selected site. The first step is to select the sites.. the alter- supports the possible state Z. This list can be used to identify this correspondence is from the customers and the the sites that should be inspected to check for fraudulent employees of the relevant accredited organization. we can calculate degree Thus. we Shafer theory. and are mutually exclusive. Pieces of evidence facility are anomalous A or normal (not anomalous) could be symptoms or events that occurred or may occur N. than the anomaly degree of the selected site. A. anomaly degree to that site. where possible state. The second degree of anomaly. In others words. In the case required that all of the possible states are unique. We want to test whether some sites are idea is Dempster’s rule for combining the degrees of belief anomalous compared to others. 1]. N}. ð1Þ sequently. we count the 1  C\D¼. and . Moreover. we list fraudulent behavior in accredited testing facilities. The belief jects from the entire study. where M is P defined according to the desired significance level. which hypothesis H0 for the selected site. Then. The function. The objective is to detect accredited testing facilities that have possible Complaints about fraud in testing facilities fraudulent behavior. mðCÞ mðDÞ anomaly degree values that are as extreme. function for all of the evidence available that assign an This function assigns an evidential weight to a subset in X. V represents the state of the subset {either A or N}. mðCÞ mðDÞ mðZÞ ¼ P . Inmetro receives complaints and tip-offs about selected sites. When the test rejects the null hypothesis for a site. each possible of anomaly S using B(A) = S. the numerator represents counter is larger than aM. We repeat this procedure of sampling with replacement and sed by the following equation: anomaly degree calculation for M times. where n is the number of function can be interpreted as the total amount of justified elements in the dataset of the selected site. are not of anomaly detection. the frame of discernment for this context is within a system. analogous to the case of probabilities. the possible states for the testing overlapping. is the represents the empty set. If the m(C) and m(D). Illustrative case strap method with Dempster–Shafer theory. Next. Our hypotheses for the when they are based on independent items of evidence [8]. or more with C. these values denote the belief. quantifies the amount of conflict between the two sets [16]. The first is the notion to propose an enhanced bootstrap method. expres. Otherwise. V}. To obtain the degree of anomaly degree of belief that a possible state or a set of possible of a site. Then. then we accept the null the accumulated evidence for the sets C and D. For behavior. We perform the enhanced bootstrap procedure for all of the sites being investigated. Z  X and the basic belief assignment functions extreme.Accred Qual Assur Dempster–Shafer theory Enhanced bootstrap method The Dempster–Shafer theory is a mathematical theory of We combine bootstrapping with Dempster–Shafer theory evidence that it is based on two ideas. A The statistic to discriminate between the null hypothesis subset of those 2X sets may consist of a single possible state.

the vari- with invented results. model. 18]. All of the Evidence 1: Variability data were collected in 2014. hence. In this case. and whether the testing facility was detected as fraudulent. Brazil. the weight of the basic conditions in vehicles. Brazil. we expect a high vari. We use 6 174 Yes the Euclidean distance to compare the distribution of the 123 . the tendency to use some digits over others when falsifying test results. Table 1 outlines the dataset used in this study. the second evidence of potential fraud is digit Dataset preferences in the test results. the vehicle deviation of the data. we need to determine the of the patterns that could reveal fraud are: digit preference. Inmetro undertakes an in loco evaluation to poor random number generators. accredited vehicle testing facilities in Rio de Janeiro. i. Therefore. This basic belief assignment function has to map the First. 9 m1 ð AÞ ¼ w1 > = m1 ð N Þ ¼ 0 ð2Þ Table 1 Sideslip measurements and detection of fraud in six > . Moreover. The first symptom of potential Complaint 3: ‘‘The testing facility tampered with its fraud is in the variability of the test results. as vehicles age and accumulate mileage. A sideslip tester is used to verify the safety assignment functions. The data were Using the variability and the digit preference as evi- extracted from the sideslip tester measures from each dence factors.. 1]. Humans are facility.’’ variability. we can now provide the basic belief organization. we must identify the evidence that can indicate values of the standard deviation r in the range [0. Numerous factors affect the results estimation for the standard deviation domain is [0. checking for fraudulent data is looking for digit preference [9]. we use the standard Modeling basic belief assignment function deviation. leading to variable results. in 2014 m1 ðV Þ ¼ 1  m1 ð AÞ Site Sideslip measurements Fraudulent? where 1 137 No 18  r w1 ¼ : 2 142 No 18 3 237 Yes Evidence 2: Digit preference 4 117 No 5 130 No Digit preference patterns can reveal possible fraud. too few or too many outliers. There. and data that are can never exceed 60 % of the range [17]. strange peaks. For simplicity. across different vehicles. it undertakes a surprise visit to that testing editing the test reports with invented results. another way of verify the facility’s compliance with the law. Two of these testing facilities were belief assignment function can be interpreted as the degree detected as fraudulent in unexpected visits performed by of anomaly revealed by that evidence.e. Therefore. This measures the To use the Dempster–Shafer theory. Accred Qual Assur Complaint 1: ‘‘The testing facility exchanges a substan. The lower this equipment to falsify the test results. In this case. testing facilities in Rio de Janeiro. The higher the The dataset used in this study is from six accredited vehicle digit preference. the higher the suspicion of fraud. Inmetro. the fraudster aims for a Complaint 2: ‘‘The testing facility edits the test reports determined range of safety limit values. results. they are to measure the anomaly degree based on the standard more likely to fail the safety tests. including quantitative sideslip measurements. the too skewed [9]. Considering the sideslip measures. too least three numbers in the dataset. Some complete our linear model. we need to model the anomaly degree based on the standard deviation of the basic belief assignment functions from the set of evidence. Some of the complaints about fraud in accredited testing When Inmetro receives complaints about fraudulent facilities indicate that there may be some mechanism of behavior. the standard deviation little or too much variance. ?15 m/km. and the misuse of a vehicle can also increase the likelihood of a dard object with one with no defects to obtain legitimate vehicle failing the safety tests. the degree to which the owners maintain their vehicles. a good ability in the test results. To measure the variability of the data. In an accredited testing facility.’’ ability of the results is affected. To fraudulent behavior in accredited inspection bodies. range of the sideslip tester is defined by -15 m/km to In the case of vehicle safety tests. domain range of the standard deviation. Where there are at round number preference. the higher the suspicion of fraud. data.’’ when falsifying the test results. we use a linear function to model our basic belief assignment function. For Equation 2 defines the basic belief assignment function example.

Table 2 gives the results. þ m1 ðV Þ m2 ðV Þ portions. 3. 3 calculates the digit preference distance dDP. is 2.017. 1]. To complete the linear model. . using these two vectors of pro.374] use the following equations: 123 . 2 [2. the Site r range dDP range basic belief assignments of the zero value for all m(N) mean that those evidence give a zero degree of not 1 [2.263] To combine two independent sources of evidence.013.065. The per- m2 ð N Þ ¼ 0 ð4Þ > .333] [0. 0. we Bð V Þ ¼ K calculate the proportions of the leading digits from all of the where other sites together. In an extreme situation. The cumulative distribution ence distance of the data. Then. we fail to reject dDP w2 ¼ pffiffiffi : 2 Table 2 Standard deviation range and digit preference distance after M = 5000 iterations In each of the previous basic belief assignments.190] [0. the higher the digit preference. We can compute B(A) by combining any evidence pair and then combine that result With an index to compare the digit preference between with the remaining bits of evidence [18]. 0. the Results higher the digit preference distance. This functions models are valid to compare the testing facilities. This calculation uses the ratio between the leading digit frequency and the number of observations K ¼ m1 ð AÞ m2 ð AÞ þ m1 ð AÞ m2 ðV Þ þ m1 ðV Þ m2 ð AÞ in all of the remaining sites P0 . We do this þ m1 ðN Þ m2 ðN Þ þ m1 ðN Þ m2 ðV Þ þ m1 ðV Þ m2 ðN Þ for the digits 0–9. Finally.018. First. one site could have a that lie in the intervals of our basic belief assignment func- digit preference for only one digit. Table 2 shows that each of the six pseudo-sites created we need to determine the domain range of the digit prefer. The remaining sites could tion models. 0.017. 3. This section empirically investigates the proposed proce- For simplicity. 0.334] [0.213] 5 [2. Therefore. . We use a dataset of six real-world sites that contain basic belief assignment function. . qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Finally. safety inspection bodies in the study. the proposed basic belief assignment have digit preferences for only the other leading digits. 3. 2].064. we can 6 [2. we also use a linear function to model our dure. P2 .285] [0. This m1 ðN Þ m2 ðN Þ þ m1 ðN Þ m2 ðV Þ þ m1 ðV Þ m2 ðN Þ calculation uses the ratio between the leading digit fre. the higher the anomaly degree of the site. Eq. distance in the range [0. p2 .181] anomalous behavior about the inspection activities [18]. sites.074. centage of values that are as extreme or more extreme than m2 ðV Þ ¼ 1  m2 ð AÞ the anomaly degree of site 1 in the empirical distribution is greater than the chosen level of significance a = 1 %. We use a = 1 % level of significance. In where this case. P9 . Therefore.Accred Qual Assur leading (first) digits used from each site with the distribution m1 ð AÞ m2 ð AÞ þ m1 ð AÞ m2 ðV Þ þ m1 ðV Þ m2 ð AÞ from all of the other sites together. a good estimation pffiffiffi for the domain function for the anomaly degree of the six accredited vehicle of the digit preference distance is [0.124] [0. P1 . Bð N Þ ¼ K quency and the number of observations on that site m1 ðV Þ m2 ðV Þ p0 .280] [0. has standard deviation and digit preference distance values ence distance. .017. in the analysis.015.833. We use MATLABÒ degree based on the digit preference distance of the data. The results shown are Equation 4 defines the basic belief assignment function after M = 5000 sampling with data replacement using the to measure the anomaly degree based on the digit prefer. 0.173.127] Dempster–Shafer rule of combination 4 [2. consider combining multiple pieces of evidence dDP ¼ ðp0  P0 Þ2 þ    þ ðp9  P9 Þ2 ð3Þ from different properties of A...913. . 3. p9 . 3. Remember. We do this for the digits 0–9. and This function has to map the values of the digit preference M = 5000 iterations. the higher the suspicion of fraud. Next. 3. we calculate the Bð AÞ ¼ K proportions of the leading digits in the selected site. p1 . 0. the anomaly degree is S1 = 0. Therefore. as Figure 1 shows the empirical cumulative distribution defined here.051. the p value is 100 %. we can now provide a basic belief assignment function for this evidence. This measures the anomaly two real cases of fraud (sites 3 and 6). enhanced bootstrap method.187] 3 [1. function returns the probability of the values in the sample 9 being less than (or equal to) some value in the sample [19]. . implies that thepffiffimaximum ffi digit preference distance. m2 ð AÞ ¼ w1 > = For site 1.

The Institute of Metrology. the performed by the testing facilities. Accred Qual Assur Fig. On the other hand. The results show that sites 3 and 6 are anomalous com- Following.48 %. the anomaly degree is S5 = 0. Bennamoun M.917. This technique allows degree of site 3 in the empirical distribution is less than the us to map the accredited testing facilities with anomalous chosen level of significance a = 1 %. a = 1 %. Barone L. than the anomaly degree of site 5 in the empirical distri- bution is greater than the chosen level of significance Compliance with ethical standards a = 1 %. S5. we reject the null hypothesis. IEEE Trans and site 6 is marked as suspicious. Note that the percentage of values that are as extreme or more extreme than the anomaly degree of site 6 in the empirical distribution is less than the References chosen level of significance a = 1 %. we fail to reject the null hypothesis. S3. Therefore. The percentage of values successfully identifies the sites that presented fraudulent that are as extreme or more extreme than the anomaly data.871.02 %.879. rejecting the null hypothesis for site 3 and site 6. p value is 0.72 %. Behdad M. Therefore. we use a real dataset from and site 3 is marked as suspicious. S4. the p values of sites 3 and 6 extreme than the anomaly degree of site 2 in the empirical indicate that those anomaly degrees rarely appear on their distribution is greater than the chosen level of significance empirical distributions. for site 5.88 %. Carmo LFRC. Therefore. Acknowledgements The authors would like to thank the National lowing. 3. and site 4 is not marked as suspicious. we reject the null hypothesis. for site 3. the anomaly degree is S3 = 0.926. To test the technique. the anomaly interest. for site 6. 123 . In this case. Although all of the sites have a The percentage of values that are as extreme or more similar anomaly degree. and S6 are the observed anomaly degrees for the sites 1. the accredited testing facilities in Brazil. In this case. Finally. 2. the behavior.02 %. we fail to reject the null hypothesis. S2. the anomaly degree is S2 = 0. we fail to reject the null hypothesis. French T (2012) Nature- inspired techniques in the context of fraud detection. Pirmez L (2014) Detecção de Dados ulent sites. In this case. 1 Empirical cumulative distribution function and anomaly degree for the six accredited testing facilities after M = 5000 iterations using the enhanced bootstrap method. In this case. 1. Souza RP. and site 1 is not marked as suspicious. p value is 60. for site 4. Therefore. and site 5 is not Conflict of interest The authors declare that they have no conflict of marked as suspicious. degree is S6 = 0. 5. and 6. Therefore. pared with the other sites. 2. for site 2.868. the p value is 0. The percentage of values We present a technique to identify unusual test result pat- that are as extreme or more extreme than the anomaly terns in accredited testing facilities. respectively the null hypothesis. Quality and Technology of Brazil for its percentage of values that are as extreme or more extreme support. 4. the p value is 30. Syst Man Cybern C 42:1273–1290 The proposed method successfully identifies the fraud. Fol. the p value is 75. The proposed method anomaly degree is S4 = 0. where S1. Future studies could investigate methods to auto- degree of site 4 in the empirical distribution is greater than matically analyze the video records of the services the chosen level of significance a = 1 %. Next. Suspeitos de Fraude em Organismos de Inspeção Acreditados. and site 2 is not Conclusion marked as suspicious. considering several fraudulent aspects at once. In this case.

Boston. USA. In: Sixth international conference mining. Devore J (2012) Probability and statistics for engineering and the sciences. New Generations. UK. 57. USA. Massachusetts. Pharm Stat 10:257–264 its applications to system safety and reliability modelling. SIAM Rev 21:460–480 emissions tests performed by an accredited inspection body. Massachusetts.ufmg. Ge Y. Springer. Cox T. ISBN-10: detection and treatment of fraud in clinical trials.dcc. Teach Stat 26:54–55 8. Drug Inf J 36:115–125 978-1-4757-2348-9 6. Shatz SM. Green Tea Press. Manly BFJ (2006) Randomization. ISBN questionnaire data. Carlsson M (2011) Detecting data fabrication in clinical 16. 13. Dong F. Murray G. Clin 17. Cengage Learning. Runger GC (2010) Applied statistics and Sistemas Computacionais. Efron B. USA 14. Kirkwood AA.pdf. Buydens LMC (2000) The bootstrap: a ISBN 978-0-412-36710-6 tutorial. 2009. Evans S. Accessed 6 Nov 2014 978-0-470-05304-1 3. Chapman and Hall. Taylor RN. Wehrens R.Accred Qual Assur XIV Simpósio Brasileiro em Segurança da Informação e de 11. Tibshirani RJ (1993) An introduction to the bootstrap. Wiley. de Souza RP. In: IEEE 11th international conference on data using Dempster–Shafer theory. Chemometr Intell Lab 54:35–52 5.1109/ICDM. Efron B (1979) Computers and the theory of statistics: thinking detect suspected patterns of fraudulent behavior in vehicle the unthinkable. Liu C. bootstrap and Monte Carlo Chapman & Hall. Vol. http://www. programmers. 4. 9.br/ probability for engineers. Downey AB (2014) Think stats: probability and statistics for P. Zhou Z (2011) A taxi driving fraud 18. Good P (1994) Permutation tests: a practical guide to resampling techniques to detect fraud and other data irregularities in clinical methods for testing hypotheses. USA. Colton T. Lachenbruch 19.1109/ITNG.28 Lesaffre E. Kay RU (2007) Fundamentals of the Dempster–Shafer theory and trials from cluster analysis perspective. ISBN-10: 0412042312 methods in biology. ISBN-10: 0-538- 73352-7 123 . Verma BL (1999) The role of biostatistics in the prevention. Pirmez L (2016) A procedure to 12.1007/s00769-016-1231-y Monographs on statistics and applied probability.sbseg2014. McEntegart DJ. Edler L. Hackshaw A (2013) Application of Theory Appl 3:173–185 methods for central statistical monitoring in clinical trials. Stat Med 1449307116 18:3435–3451 10. Hutton J. Wu X.2011. Ranstam J. doi:10. London. Montgomery DC. Buyse M. Xu H (2009) Inference of online auction shills detection system.18 on information technology. ISBN–13: files/anais. Croucher JS (2004) An upper bound on the value of the standard Trials 10:783–806 deviation. Accred Qual Assur 21:323. Geller NL. New York. doi:10. Stillman EC (2002) Statistical 15. Putter H. New Zealand. Xiong H. Reliab 7. George SL. Scherrer B. Carmo LFRC. doi:10.