0 Up votes0 Down votes

3 views45 pagesJan 23, 2016

© © All Rights Reserved

PDF, TXT or read online from Scribd

© All Rights Reserved

3 views

© All Rights Reserved

- Neuromancer
- The E-Myth Revisited: Why Most Small Businesses Don't Work and
- How Not to Be Wrong: The Power of Mathematical Thinking
- Drive: The Surprising Truth About What Motivates Us
- Chaos: Making a New Science
- The Joy of x: A Guided Tour of Math, from One to Infinity
- How to Read a Person Like a Book
- Moonwalking with Einstein: The Art and Science of Remembering Everything
- The Wright Brothers
- The Other Einstein: A Novel
- The 6th Extinction
- The Housekeeper and the Professor: A Novel
- The Power of Discipline: 7 Ways it Can Change Your Life
- The 10X Rule: The Only Difference Between Success and Failure
- A Short History of Nearly Everything
- The Kiss Quotient: A Novel
- The End of Average: How We Succeed in a World That Values Sameness
- Made to Stick: Why Some Ideas Survive and Others Die
- Algorithms to Live By: The Computer Science of Human Decisions
- The Universe in a Nutshell

You are on page 1of 45

Questions Addressed

What

two or more system configurations?

What is hypothesis testing?

How do common random numbers work?

9-2

Purpose

Purpose:

Approach: discuss a few of many statistical

methods that can be used to compare two or more

system designs.

differences are due to:

Differences in design, or

The random fluctuation inherent in the models.

3

Outline

For two-system

comparisons:

Independent sampling.

Correlated sampling (common random numbers).

For multiple

system comparisons:

Bonferroni approach: confidence-interval

estimation, screening, and selecting the best.

attributable to actual differences in

performance and not to statistical variation.

is required.

9-5

Hypothesis Testing

Used

precision is required.

A hypothesis is first formulated (e.g. that

methods A and B both result in the same throughput)

results of the simulation lead us to reject

the hypothesis.

9-6

fail to reject a hypothesis?

1. Our hypothesis is correct

OR

2. The variance in the observed outcomes

are too high given the number of

replications to be conclusive (run more

replications or use variance reduction techniques).

9-7

Types of Errors

Type

Type II error: Accept False Hypothesis

True

Accept

Reject

False

II

I

9-8

[------|------]

(A)

(B)

(C)

[------|------]

Fail to

Reject H0

Reject H0

[------|------]

Reject H0

U1-U2=0

[------|------] denotes confidence Interval

9-9

maximize the throughput of the system.

We offer two methods based on the Confidence Interval

approach.

We seek to discover if the mean throughput of the system

due to Strategy 1 and Strategy 2 are significantly different.

We begin by estimating the mean performance of the two

proposed strategies by simulating each strategy for 16 days

past the warm-up period.

The simulation experiment was replicated 10 times for

each strategy.

The throughput achieved by each strategy is shown next.

9-10

(A)

Replication

(C)

(B)

Strategy 2

Strategy 1

Throughput Throughput

54.48

56.01

57.36

54.08

54.81

52.14

56.20

53.49

54.83

55.49

57.69

55.00

58.33

54.88

57.19

54.47

56.84

54.93

10

55.29

55.84

mean

Standard Deviation

56.30

1.37

54.63

1.17

Variance

1.89

1.36

9-11

Requires

from each simulated system be

normally distributed and independent

within a population and between

populations.

9-12

9-13

17.5

= T.INV.2T(0.05,17.5) = 2.10

9-14

9-15

9-16

Like

method requires that the observations drawing

from each population be normally distributed

and independent within a population.

However, the paired-t CI method does not

require that the observations between

populations be independent.

9-17

This

Numbers to force a positive correlation between

the two populations in order to reduce the halfwidth.

Finally, like the Welch method, the paired-t CI

method does not require that the population have

equal variances.

Given n observations (n1=n2=n), we pair the

observations from each population (x1 and x2) to

define a new random variable: x(1-2)j = x1j - x2j

9-18

9-19

Paired-t Comparison

(A)

Replication (j)

(B)

Strategy 1

Throughput x1j

(C)

Strategy 2

Throughput

x2j

(D)

Difference (B C)

X(1-2)j = x1j - x2j

54.48

56.01

-1.53

57.36

54.08

3.28

54.81

52.14

2.67

56.20

53.49

2.71

54.83

55.49

-0.66

57.69

55.00

2.69

58.33

54.88

3.45

57.19

54.47

2.72

56.84

54.93

1.91

10

55.29

55.84

-0.55

mean

Standard Dev.

Variance

1.67

1.85

3.42

9-20

9-21

Bonferroni

approach

ANOVA

Factorial

design and

optimization experiments

9-22

9-23

9-24

9-25

9-26

Factorial Design

Tests system response(s) when

multiple factors are being manipulated.

Input

Output measures are the responses

9-27

Define a high and low level setting for each factor.

Try every combination of factor settings.

Run detailed studies for factors that have the

greatest impact.

9-28

Fractional-factorial Design

Strategically "screen out" factors that have little or no

impact on system performance.

On remaining factors, run a full-factorial experiment.

Run detailed studies for factors that have the greatest

impact.

9-29

Variance Reduction

Common

Evaluates each system under the exact same

circumstances.

Helps ensure that observed differences of

system designs are due to the differences in

the designs and not to differences in

experimental conditions.

Antithetic Random Numbers (ARN)

9-30

common random numbers.

CRN Continued

(A)

Replication

1

2

3

4

5

6

7

8

9

10

(B)

Strategy 1

Throughput

79.05

54.96

(C)

Strategy 2

Throughput

75.09

51.09

(D)

Difference (B-C)

3.96

3.87

51.23

49.09

2.14

88.74

56.43

70.42

88.01

53.34

67.54

0.73

3.09

2.88

35.71

34.87

0.84

58.12

57.77

45.08

54.24

55.03

42.55

3.88

2.74

2.53

X Difference = 2.67

s Difference = 1.16

9-31

System Designs

Goal: compare two possible system configurations, e.g.:

two possible ordering policies in a supply-chain system,

two possible scheduling rules in a job shop.

output data.

qi (i = 1, 2) to obtain point and interval estimates for the

difference in mean performance, namely q1 q2.

32

Vehicle-safety inspection example:

The station performs 3 jobs: (1) brake check, (2) headlight check, and (3)

steering check.

Vehicles arrival: Possion with rate = 9.5/hour.

Present system:

Three stalls in parallel (one attendant makes all 3 inspections at each stall).

Service times for the 3 jobs: normally distributed with means 6.5, 6.0 and 5.5

minutes, respectively.

Alternative system:

Each attendant specializes in a single task, each vehicle will pass through

three work stations in series

Mean service times for each job decreases by 10% (5.85, 5.4, and 4.95

minutes).

Performance measure: mean response time per vehicle (total time from

33

Assuming that the estimators Yir are (at least approximately) unbiased:

q1 = E(Y1r ), r = 1, , R1;

q2 = E(Y2r ), r = 1, , R2

If c.i. is totally to the left of 0, strong evidence for the hypothesis that q1 q2 < 0 (q1 < q2 ).

If c.i. is totally to the right of 0, strong evidence for the hypothesis that q1 q2 > 0 (q1 > q2 ).

If c.i. contains 0, no strong statistical evidence that one system is better than the other

If enough additional data were collected (i.e., Ri increased), the c.i. would

most likely shift, and definitely shrink in length, until conclusion of q1 < q2 or

q1 > q2 would be drawn.

34

A two-sided 100(1-)% c.i. for q1 q2 always takes the

form of:

where Y.i is the sample mean performance measure for system i over all replications,

and is the degress of freedom.

two statistical techniques.

Both techniques assume that the basic data Yir are

approximately normally distributed.

We will discuss these two methods next.

35

Statistically significant versus practically significant

Statistical significance: is the observed difference Y.1 Y.2

Practical significance: is the true difference q1 q2 large

Confidence intervals do not answer the question of

difference within a range.

36

[Comparison of 2 systems]

used to simulate the two systems

All observations of simulated system 1 are statistically

independent of all the observations of simulated system 2.

The variance of the sample mean, Y.i , is:

V Y.i i2

V Y.i

,

Ri

Ri

i 1,2

12

R1

22

R2

37

[Comparison of 2 systems]

If it is reasonable to assume that 21 22 (approximately) or if R1 = R2,

a two-sample-t confidence-interval approach can be used:

The point estimate of the mean performance difference is: Y.1 Y.2

The sample variance for system i is:

The pooled estimate of 2 is:

( R1 1) S12 ( R2 1) S 22

S

,

R1 R2 2

2

p

i

1

2

Si

Yri Y.i

Ri 1 r 1

i

1

Yri 2 RiY.i 2

Ri 1 r 1

1

2

Standard error:

1

1

R1 R2

38

[Comparison of 2 systems]

approximate 100(1-)% C.I. for can be computed as:

S12 S 22

R1 R2

/ R1 S 22 / R2

S 2 / R 2 / R 1 S 2 / R

1

1 1

2 2

2

1

/ R

2

, round to an interger

39

[Comparison of 2 systems]

For each replication, the same random numbers are used to simulate

both systems.

For each replication r, the two estimates, Yr1 and Yr2, are correlated.

However, independent streams of random numbers are used on different

reduce variance in the point estimator of Y.1 Y.2 .

12

R

22

R

2 12 1 2

R

12 is positive

Variance of Y.1 Y.2 arising from CRN is less than that of independent

40

[Comparison of 2 systems]

shorter confidence interval for the difference.

S D2

Dr D

R 1 r 1

where Dr Yr1-Yr 2

Standard error:

D Y.1 Y.2 :

1

2

2

Dr RD

R 1 r 1

1

and D

R

D ,

r

r 1

SD

R

41

[Comparison of 2 systems]

random-number generator(s):

The random numbers must be synchronized: each

random number used in one model for some purpose

should be used for the same purpose in the other

model.

e.g., if the ith random number is used to generate a

service time at work station 2 for the 5th arrival in

model 1, the ith random number should be used for

the very same purpose in model 2.

42

To compare K alternative system designs based on some specific

performance measure, qi, of system i , for i = 1, 2, , K.

Procedures are classified as:

Fixed-sample-size procedures: predetermined sample size is used to

draw inferences via hypothesis tests of confidence intervals.

Sequential sampling (multistage): more and more data are collected

until an estimator with a pre-specified precision is achieved or until one

of several alternative hypotheses is selected.

Estimation of each parameter q.

Comparison of each performance measure qi, to control q1.

All pairwise comparisons, qi - qj, for all i not equal to j

Selection of the best qi.

43

Bonferroni Approach

[Multiple Comparisons]

(where all statements are true simultaneously).

Bonferroni inequality:

C

j 1

probability of a false conclusion

The smaller j is, the wider the jth confidence interval will be.

independent sampling or CRN

Major disadvantage: width of each individual interval increases

as the number of comparisons increases.

44

Bonferroni Approach

[Multiple Comparisons]

Practical upper limit: about 10 comparisons

3 possible applications:

Individual c.i.s: Construct a 100(1- j)% c.i. for parameter qi,

where # of comparisons = K.

Comparison to an existing system: Construct a 100(1- j)%

c.i. for parameter qi- q1 (i = 2,3, K), where # of comparisons

= K 1.

All pairwise: For any 2 different system designs, construct a

100(1- j)% c.i. for parameter qi- qj. Hence, total # of

comparisons = K(K 1)/2.

45

- Phase II of Golden RatioUploaded byAlejandra Zepeda
- 2009-07-16_030945_ex_parUploaded byHàMềm
- Paracetamol Toxicidad 0 MinUploaded byAntoineRiemann
- QNT 275 All ParticipationsUploaded byacc4922
- Comm291 Practice MidtermUploaded byal3ung
- Toilet TrainingUploaded byuji_faiz_adi_cu73
- AGS, Agriculture, 58-1, 2018Uploaded bySlovenian study references
- kelsie dee stat finalUploaded byapi-253997467
- Assignment 4 Rev3Uploaded byTummala KC
- Sample MBA Sem3 Summer 2015Uploaded byAmar Goel
- 3. Quiz 2 - 4Uploaded byhimanshubahmani
- Models and Estimators Linking Individual-based and Sample-based Rarefaction, Extrapolation and Comparison of AssUploaded byMateo Parra
- AkobengUploaded byCat Ngan
- 10.1056@NEJMoa1715474Uploaded byRoger Benito Ruiz Agurto
- sw4810Uploaded byapi-242944449
- Quantitative method problem setUploaded byConstance
- confidence interval mini-project pdfUploaded byapi-268986158
- Thursday PresentationUploaded byMichael Czahor
- confidence interval.pdfUploaded byMaiah Dinglasan
- Statistics Lecture Part 2Uploaded byKyla Fernandez
- Introduction to StatisticsUploaded bySifat Mahmud
- Chapter 16Uploaded byVee Lan
- F 2105 – 01Uploaded byFrancisco Guerra
- CHAPTER6 EstimationUploaded bySekut Tawar
- Effects of Snowfalls on Motor Vehicle Collisions.pdfUploaded byNgocTraiNguyen
- notes.pdfUploaded byLibyaFlower
- Hp 49g+_user's Guide_with BookmarksUploaded byBea Floyd
- final project reportUploaded byapi-289046270
- ΣΤΑΤΙΣΤΙΚΗ ΛΥΜΕΝΕΣ ΑΣΚΗΣΕΙΣUploaded byGwgw Veliou
- 1-s2.0-S0022391313603789-mainUploaded byAlina Vede

- Operations ManagementUploaded byrockon605
- fulltext (1)Uploaded byLibyaFlower
- tv_22_2015_6_1447_1456.pdfUploaded byMuhammad Izzamir Firdaus
- Introduction_methodology_2013.pdfUploaded byLibyaFlower
- Numerical Analysis BookUploaded byAniekan Esenam
- manual-PK-TAC-feb102.pdfUploaded byLibyaFlower
- Introduction to Clinical Research for Residents (16.9.14) Hani Tamim (FC1)Uploaded byLibyaFlower
- PIIS0169500215001622.pdfUploaded byLibyaFlower
- Stanford STATA BasicsUploaded byJason Pan
- jcph464.pdfUploaded byLibyaFlower
- yhyUploaded byLibyaFlower
- 16 Lovell March 2014 (1)Uploaded byRoland Duncombe
- Ielts Reading 2016 by Ngoc Bach_part 1Uploaded byNguyễn Yolo
- dining-application-2016.pdfUploaded byLibyaFlower
- art%3A10.1007%2Fs11222-005-3393-z.pdfUploaded byLibyaFlower
- art%3A10.1007%2Fs10182-013-0221-1.pdfUploaded byLibyaFlower
- A0250109.pdfUploaded byLibyaFlower
- Accurate ARL Calculation for EWMA Control Charts Monitoring Simultaneously Normal Mean and VarianceUploaded byHafiedz Pradana
- ewmacharts.pdfUploaded byLibyaFlower
- Categorical-Stata.pdfUploaded byLibyaFlower
- svy.pdfUploaded byLibyaFlower
- Operation NorthwoodsUploaded byandroline
- Operation WindigoUploaded byFuckencioPerez
- GE Six SigmaUploaded byimran27pk
- Six Sigma TutorialUploaded byarnabmoitra11
- 8-1.xlsxUploaded byLibyaFlower
- 251128537-Applied-Regression-Analysis-Draper (1) (1) (1).pdfUploaded byLibyaFlower
- Applied Multivariate Statistical Analysis 6th EditionUploaded byAbheera Hazra

- Placement Brochure RevisedUploaded bydibinkc
- Iso Tracking SheetUploaded bysasi10000
- Exams ManualUploaded byPaulo M Espinosa
- CV-English.docUploaded bySUGAN TUBE
- admin resumeUploaded byapi-458036870
- income.pptUploaded byJia-SiangShen
- DoD Inflation HandbookUploaded byA21Joker
- SME Growth Cycle - Presentation by Klaus SchnurrUploaded byNETParkNet
- G.R. No. 172948 _ Philippine Associated Smelting and Refining CorpUploaded byLien Patrick
- A.R Cornelius – the Islamic CatholicUploaded byhati1
- Hegerty vs. CAUploaded byMichelle Ann Orendain
- UMFA Sample ReportUploaded byutglib
- reflective writing assignment 2 - corry martin pidp 3420Uploaded byapi-318587758
- HEBREW is GREEK - The Work of Joseph YahudaUploaded byRonFisher
- Business Communication Module 2Uploaded byMohan Bn
- etheric_body.pdfUploaded byFrida Bendersky
- Nefropatía por Fabry.pdfUploaded byLuchoBofill
- Mob JusticeUploaded byJordanTabor
- Manfred SymphonyUploaded byBearer2008
- How to Conduct Security AuditUploaded byPeter George
- Call for Paper UDSMUploaded byRashid Bumarwa
- Human SexualityUploaded byMegurine Luka
- chapter 5Uploaded bylyj1017
- Matt Heinzelman Software Quality Assurance PresentationUploaded byVish Yrdy
- Nunley v. Berkshire Life Insurance Company of America - Document No. 5Uploaded byJustia.com
- Presentation_RBM_in_practice_and_programme_logic_for_NFPs_and_POs.pptUploaded byAmina
- iem pptUploaded byPRATIK SAWANT
- Hawaiian GodsUploaded byBlanche Althea
- Proc LogisticUploaded byNguyễn Thanh Trà
- Types of ForceUploaded byiskenderbey

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.