31 views

Uploaded by Teng Ker Sheng

Statistical

- 1-s2.0-S183695531930044X-main.pdf
- Signifcance Tests in Climate Science.pdf
- (New) Qnt 351 Qnt351 Final Exam Entire Answers With Questions Correct 100%
- 14 Chapter 5
- Perception of Female Chauffeurs in India
- Descriptive Statistics
- p Values Ruo Correlation Coefficient
- mkm042
- lec6
- Testing Hypothesis.college
- AN_ASSESSMENT_ON_THE_IMPACT_OF_INDUTRIALIZATION_ON_ECONOMIC_GROWTH_IN_NIGERIA.pdf
- Chi Square Practice Problems Key
- 1723-2014
- Hw4
- To p or Not to p - Tese
- 11k Exer1&2 Sanchez
- ttest1
- Strategy
- FD
- case study

You are on page 1of 22

Will G Hopkins

Auckland University of Technology

Auckland, NZ

Other titles:

probability

Statistical vs clinical,

practical, or mechanistic significance.

beneficial

A more meaningful

way to make inferences from a sample.

trivial

Statistical

significance

smallest

clinicallyis unethical; clinical significance isnt.

harmful

harmful

value your finding is beneficial or harmful?

What are the chances

Publishing without hypotheses and statistical significance.

value of effect statistic

Non-significant effect? No problem!

Summary

Background

Misinterpretation of data

Making inferences

Sample population

Statistical significance

Confidence limits

Precision of estimation

Smallest worthwhile effect

How to use possible, likely, very likely, almost certain

Examples

Background

Most researchers and students misinterpret statistical

significance and non-significance.

Few people know the meaning of the P value that defines

statistical significance.

Reviewers and editors reject some papers with statistically

non-significant effects that should be published.

Use of confidence limits instead of a P value is only a partial

solution to these problems.

Were trying to make inferences about a population from a

sample.

What's missing is some way to make inferences about the

clinical or practical significance of an effect.

We study a sample to get an observed value of a statistic

representing an interesting effect, such as the relationship

between physical activity and health or performance.

But we want the true (= population) value of the statistic.

The observed value and the variability in the sample allow us to

make an inference about the true value.

Use of the P value and statistical significance is one approach to

making such inferences.

Its use-by date was December 31, 1999.

There are better ways to make inferences.

Therefore, we need something to disprove.

Let's assume the true effect is zero: the null hypothesis.

If the value of the observed effect is unlikely under this

assumption, we reject (disprove) the null hypothesis.

"Unlikely" is related to (but not equal to) a probability or P value.

P < 0.05 is regarded as unlikely enough to reject the null

hypothesis (i.e., to conclude the effect is not zero).

We say the effect is statistically significant at the 0.05 or 5% level.

Some folks also say "there is a real effect".

We say the effect is statistically non-significant.

Some folks accept the null and say "there is no effect".

Failure to reject the null doesn't mean we have to accept the null.

In any case, true effects in real life are never zero. Never.

So, THE NULL HYPOTHESIS IS ALWAYS FALSE!

Therefore, to assume that effects are zero until disproved is

illogical, and sometimes impractical or even unethical.

0.05 is arbitrary.

uncertainties of real life:

More emphasis on (im)precision of estimation, through use of

confidence limits for the true value

Better types of P value, representing probabilities of clinical or

practical benefit and harm

P/2 is the probability that the true value is negative.

Example: P = 0.24

probability

(P value)/2

= 0.12

probability distribution

of true value given the

observed value

observed

value

negative 0 positive

value of effect statistic

Easier to understand, and avoids statistical significance, but

Problem: having to halve the P value is awkward, although we

could use one-tailed P values directly.

Problem: focus is still on zero or null value of the effect.

These define a range within which the true value is likely to fall.

"Likely" is usually a probability of 0.95 (defining 95% limits).

probability

Area = 0.95

lower likely limit

probability distribution

of true value given the

observed value

observed value

upper likely limit

negative 0 positive

value of effect statistic

Problem: 0.95 is arbitrary and gives an impression of imprecision.

0.90 or less would be better.

Problem: still have to assess the upper and lower limits and the

observed value in relation to clinically important values.

Clinical Significance

Statistical significance focuses on the null value of the effect.

More important is clinical significance defined by the

smallest clinically beneficial and harmful values of the effect.

These values are usually equal and opposite in sign.

Example:

smallest clinically

smallest clinically

harmful value

beneficial value

observed value

negative 0 positive

value of effect statistic

a statement about clinical significance.

probabilities that the true effect could be clinically beneficial,

trivial, or harmful (Pbeneficial, Ptrivial, Pharmful).

smallest clinically

These Ps make an effect

beneficial value

probability

easier to assess and

Pbeneficial

Ptrivial

(hopefully) to publish.

= 0.80

= 0.15

Warning: these Ps are

smallestP clinically

observed

harmful

NOT the proportions of harmful value

value

= 0.05

+ ive, non- and - ive

responders in the population.

negative 0 positive

The calculations are easy.

value of effect statistic

Put the observed value, smallest beneficial/harmful value, and

P value into the confidence-limits spreadsheet at newstats.org.

value, interpreting the probabilities, and publishing the work.

If you can't meet this challenge, quit the field.

For performance in many sports, ~0.5% increases a top

athlete's chances of winning.

The default for most other populations is Cohen's set of

smallest worthwhile effect sizes.

This approach applies to the smallest clinically, practically

and/or mechanistically important effects.

Correlations: 0.10

Relative risks: ~1.2, depending on prevalence of the disease

or other condition.

Changes or differences in the mean: 0.20 between-subject

standard deviations.

Trivial effect (0.1x SD):

females

females

males

males

intelligence

intelligence

You must also use the between-subject standard deviation when analyzing the change in

the mean in an experiment.

Many meta-analysts wrongly use the SD of the change score.

You should describe outcomes in plain language in your paper.

Therefore you need to describe the probabilities that the effect is

beneficial, trivial, and/or harmful.

Suggested schema:

Probability

<0.01

0.010.05

0.050.25

0.250.75

0.750.95

0.950.99

>0.99

Chances

<1%

15%

525%

2575%

7595%

9599%

>99%

Odds

<1:99

1:991:19

1:191:3

1:33:1

3:119:1

19:199:1

>99:1

is not, is almost certainly not

is very unlikely to be

is unlikely to be, is probably not

is possibly (not), may (not) be

is likely to be, is probably

is very likely to be

is, is almost certainly

Example:

TABLE 2. Differences in improvements in kayaking performance

between the slow, explosive and control training groups,

and chances that the differences are substantial (greater than the

smallest worthwhile change of 0.5%) for a top kayaker.

Mean improvement

Chances (% and qualitative)

(%) and 90%

Compared groups confidence limits of substantial improvementa

Slow - control

3.1; 1.6

99.6; almost certain

Explosive - control

2.0; 1.2

98; very likely

Slow - explosive

1.1; 1.4

74; possible

a

importance of p=0.20

p value

0.03

0.20

threshold values

value of

Conf. deg. of Confidence limits for clinical chances

statistic level (%) freedom lower

upper

positive negative

1.5

90

18

0.4

2.6

-1

2.4

90

18

-0.7

5.5

-1

clinically positive

prob (%)

78

odds

3:1

likely, probable

78

3:1

likely, probable

clinically trivial

prob (%)

22

odds

1:3

clinically negative

prob (%)

0

odds

1:2071

19

1:4

1:25

very unlikely

Summary

When you report your research

Show the observed magnitude of the effect.

Attend to precision of estimation by showing 90% confidence

limits of the true value.

Show the P value if you must, but do not test a null hypothesis

and do not mention statistical significance.

Attend to clinical, practical or mechanistic significance by stating

the smallest worthwhile value then showing the probabilities that

the true effect is beneficial, trivial, and/or harmful

(or substantially positive, trivial, and/or negative).

Make a qualitative statement about the clinical or practical

significance of the effect, using unlikely, very likely, and so on.

Supplementary slides:

Original meaning of P value

More examples of clinical significance

Example: P = 0.20 for an observed positive value of a statistic

If the true value is zero, there is a probability of 0.20 of

observing a more extreme positive or negative value.

probability

P value =

0.1 + 0.1

probability distribution

of observed value

if true value = 0

observed

value

negative 0 positive

value of effect statistic

Problem: everything that's wrong with statistical significance.

Examples for a minimum worthwhile change of 2.0 units.

Example 1clinically beneficial, statistically non-significant

(inappropriately rejected by editors):

The observed effect of the treatment was 6.0 units

(90% likely limits 1.8 to 14 units; P = 0.20).

The chances that the true effect is practically

beneficial/trivial/harmful are 80/15/5%.

(no problem with publishing):

The observed effect of the treatment was 3.3 units

(90% likely limits 1.3 to 5.3 units; P = 0.007).

The chances that the true effect is practically

beneficial/trivial/harmful are 87/13/0%.

(the worst kind of outcome, due to small sample or large error

of measurement; usually rejected, but could/should be

published to contribute to a future meta-analysis):

The observed effect of the treatment was 2.7 units

(90% likely limits 5.9 to 11 units; P = 0.60).

The chances that the true effect is practically

beneficial/trivial/harmful are 55/26/18%.

(good publishable study; true effect is on the borderline of

beneficial):

The observed effect of the treatment was 1.9 units

(90% likely limits 0.4 to 3.4 units; P = 0.04).

The chances that the true effect is practically

beneficial/trivial/harmful are 46/54/0%.

(publishable rare outcome that can arise from a large sample

size; usually misinterpreted as a worthwhile effect):

The observed effect of the treatment was 1.1 units

(90% likely limits 0.4 to 1.8 units; P = 0.007).

The chances that the true effect is practically

beneficial/trivial/harmful are 1/99/0%.

(publishable, but sometimes not submitted or accepted):

The observed effect of the treatment was 0.3 units

(90% likely limits 1.7 to 2.3 units; P = 0.80).

The chances that the true effect is practically

beneficial/trivial/harmful are 8/89/3%.

- 1-s2.0-S183695531930044X-main.pdfUploaded bySimple Mendhe
- Signifcance Tests in Climate Science.pdfUploaded byMaria Morins
- (New) Qnt 351 Qnt351 Final Exam Entire Answers With Questions Correct 100%Uploaded bytriskilie
- 14 Chapter 5Uploaded byAnuradha Nagarajan
- Perception of Female Chauffeurs in IndiaUploaded bybspkumar
- Descriptive StatisticsUploaded byCykeen Bofill Dublon
- p Values Ruo Correlation CoefficientUploaded bySanjeeb Sinha
- mkm042Uploaded byYacine Tarik Aizel
- lec6Uploaded byRajeevSangam
- Testing Hypothesis.collegeUploaded byKm de Leon
- AN_ASSESSMENT_ON_THE_IMPACT_OF_INDUTRIALIZATION_ON_ECONOMIC_GROWTH_IN_NIGERIA.pdfUploaded bySebastian Grove
- Chi Square Practice Problems KeyUploaded byJuan
- 1723-2014Uploaded byanchals_20
- Hw4Uploaded byCatherine Han
- To p or Not to p - TeseUploaded byVinicius
- 11k Exer1&2 SanchezUploaded byGustav Sanchez
- ttest1Uploaded byHumphrey Misiri
- StrategyUploaded byjaaridriv
- FDUploaded byOvidiu Bucsoiu
- case studyUploaded byVarun Soni
- dwp36Uploaded byArun Xavier
- BAYESIANS copiadoUploaded byDiego Retamozo Castillo
- What ZUploaded byHàMềm
- Learning Team Week 4 QNT351 University of PhoenixUploaded byLS
- The Financial DistressUploaded byShafqat Bukhari
- 119450_EEM1026assignmentUploaded byThaya Idhyomatic
- Lecture_6_Difference_of_Means.docUploaded byAhmed Kadem Arab
- ap statistic final project 2017 1Uploaded byapi-399832630
- RM,,,,,,Uploaded byFarhanAwaisi
- A Study on Impact of Designation & Employment Role Consumption of Multi-Functional Printers (Mfp) In Hyderabad.Uploaded byinventionjournals

- balancing chemical equationsUploaded byapi-256236481
- CSO Gaddis Java Chapter07Uploaded byqwert11j
- SSC Algebra Board Questions 2013Uploaded bySingh Sudip
- Teletraffic Engineering and Network PlanningUploaded byMuhammad Ridlon
- Tunnel Structures Seismic DesignUploaded byIbis Tello
- Chapter_06_Lecture_06_to_07_w08_431_supply_chain.pptUploaded byUlyviatrisna
- Particle Size Analyzer-Analysette22MicroTecXTUploaded byXue Han
- Pipe FrictionUploaded byPrince Israel Eboigbe
- Olympiad Combinatorics (1)Uploaded bybus9
- 4 pageUploaded bySHANKAR PRINTING
- Munger's Worldly WisdomUploaded byRaunakonkar
- ThinShells_11-04-07_ApprovedVersionUploaded byamirreza3853
- portfolio content.docxUploaded byMark Arthur Parina
- Generative Shape DesignUploaded byChethan Sk
- UntitledUploaded bySURESH V. DASARADDI
- Influence of Electrical Process ParametersUploaded bysiddy_777
- 1 Artificial Intelligence Based Dynamic Simulation of Induction Motor DrivesUploaded byDrPrashant M. Menghal
- DpSpeedupUploaded byTruc Khuyen Duong Dang
- Design, Optimization and Development of X-Band Microstrip Patch Antenna Array for High GainUploaded bymoomoer
- rprpcgenUploaded byFabian Muñoz
- ModulesUploaded byVaruni Mehrotra
- STEP 7 - Statement List for S7-300 and S7-400Uploaded byTetsusaiga
- Kaplan University MM 255 Homework HelpUploaded byjustquestionanswer
- 03 Digital Image ProcessingUploaded byNabilah Aziz
- Introduction to the Quantum Theory (2nd Ed) - David ParkUploaded bymach20_aardvark8064
- ON THE LINKING UP BETWEEN BINGHAM FLUID AND PLUGGED FLOW(1)Uploaded byIoana Ion
- LCR Hi Tester 3522-50Uploaded byDany Rassam
- Moments QuestionsUploaded bymuxadey
- Location Aided Routing ProtocolUploaded byRobert Horn
- Hansen, Van Der Stede - 2004 - Multiple Facets of Budgeting an Exploratory AnalysisUploaded byDaniele Bernd