47 views

Original Title: Statistical Significance

Uploaded by Lightcavalier

- BusinessStatisticsL3 MA202
- Walpole Chapter 10
- An Example of a Hypothesis
- Teste de Hipoteses Artigo
- Exam MATH2199 2012 (With Solutions)(1)
- PSY 315 TUTORS Learn by Doing- Psy315tutors.com
- Business Research Methods
- Hypothesis Testing
- Data Analysis
- Les4e Ppt Ch07
- Ch3 Slides
- Assignment: Silk Road An Evaluation
- Midterm Exam With Answers
- Testing of Hypothesis-AGBS
- An Analysis of Survey Data on Students from the University Park Campus at Penn State
- Stats 3 Review
- wp0704
- Hypothesis Testing Examples and Exercises
- Estadística - Ritchey Ch09
- Inferentials Stastistical Analysis

You are on page 1of 4

ing at least as extreme results given that the null hy-

pothesis is true.

[1][2][3][4][5][6][7]

It is an integral part of

statistical hypothesis testing where it helps investigators

to decide if a null hypothesis can be rejected.

[8][9]

In any

experiment or observation that involves drawing a sample

from a population, there is always the possibility that an

observed eect would have occurred due to sampling er-

ror alone.

[10][11]

But if the probability of obtaining at least

as extreme result (large dierence between two or more

sample means), given the null hypothesis is true, is less

than a pre-determined threshold (e.g. 5% chance), then

an investigator can conclude that the observed eect ac-

tually reects the characteristics of the population rather

than just sampling error.

[8]

The present-day concept of statistical signicance orig-

inated from Ronald Fisher when he developed statisti-

cal hypothesis testing in the early 20th century.

[2][12][13]

These tests are used to determine whether the outcome

of a study would lead to a rejection of the null hypothesis

based on a pre-specied low probability threshold called

p-values, which can help an investigator to decide if a re-

sult contains sucient information to cast doubt on the

null hypothesis.

[14]

P-values are often coupled to a signicance or alpha ()

level, which is also set ahead of time, usually at 0.05

(5%).

[14]

Thus, if a p-value was found to be less than 0.05,

then the result would be considered statistically signi-

cant and the null hypothesis would be rejected.

[15]

Other

signicance levels, such as 0.1 or 0.01, are also used, de-

pending on the eld of study.

In statistics, statistical signicance is not the same as re-

search, theoretical, or practical signicance.

[8][9][16]

1 History

Main article: History of statistics

The concept of statistical signicance was originated by

Ronald Fisher when he developed statistical hypothesis

testing, which he described as tests of signicance,

in his 1925 publication, Statistical Methods for Research

Workers.

[2][12][13]

Fisher suggested a probability of one in

twenty (0.05) as a convenient cutolevel to reject the null

hypothesis.

[17]

In their 1933 paper, Jerzy Neyman and

Egon Pearson recommended that the signicance level

(e.g. 0.05), which they called , be set ahead of time,

prior to any data collection.

[17][18]

Despite his initial suggestion of 0.05 as a signicance

level, Fisher did not intend this cuto value to be xed,

and in his 1956 publication Statistical methods and scien-

tic inference he recommended that signicant levels be

set according to specic circumstances.

[17]

2 Role in statistical hypothesis test-

ing

Main articles: Statistical hypothesis testing, Null hypoth-

esis, p-value and Type I and type II errors

Statistical signicance plays a pivotal role in statistical

In a two-tailed test, the rejection region or level is partitioned

to both ends of the sampling distribution and make up only 5%

of the area under the curve.

hypothesis testing, where it is used to determine if a null

hypothesis should be rejected or retained. A null hypoth-

esis is the general or default statement that nothing hap-

pened or changed.

[19]

For a null hypothesis to be rejected

as false, the result has to be identied as being statisti-

cally signicant, i.e. unlikely to have occurred by chance

alone.

To determine if a result is statistically signicant, a re-

searcher would have to calculate a p-value, which is the

probability of observing an eect given that the null hy-

pothesis is true.

[7]

The null hypothesis is rejected if the p-

value is less than the signicance or level. The level is

the probability of rejecting the null hypothesis given that

it is true (type I error) and is most often set at 0.05 (5%).

If the level is 0.05, then the conditional probability of a

type I error, given that the null hypothesis is true, is 5%.

[20]

Then a statistically signicant result is one in which the

1

2 6 REFERENCES

observed p-value is less than 5%, which is formally writ-

ten as p < 0.05.

[20]

If an observed p-value is not lower than the signicance

level, then rather than simply accepting the null hypoth-

esis, where feasible it would often be appropriate to in-

crease the sample size of the study, and see if the signif-

icance level is reached.

[21]

If the level is set at 0.05, it means that the rejection re-

gion comprises 5% of the sampling distribution.

[22]

This

5% can be allocated to one side of the sampling distri-

bution as in a one-tailed test or partitioned to both sides

of the distribution as in a two-tailed test, with each tail

(or rejection region) containing 2.5% of the distribution.

One-tailed tests are more powerful than two-tailed tests,

as a null hypothesis can be rejected with a less extreme

result.

3 Dening signicance in terms of

sigma ()

Main articles: Standard deviation and Normal distribu-

tion

In specic elds such as particle physics and

manufacturing, statistical signicance is often ex-

pressed in multiples of the standard deviation or sigma

() of a normal distribution, with signicance thresholds

set at a much stricter level (e.g. 5).

[23][24]

For instance,

the certainty of the Higgs boson particles existence was

based on the 5 criterion, which corresponds to a p-value

of about 1 in 3.5 million.

[24][25]

4 Eect size

Main article: Eect size

Researchers focusing solely on whether their results are

statistically signicant might report ndings that are not

necessarily substantive.

[26]

To gauge the research signif-

icance of their result, researchers are also encouraged to

report the eect size along with p-values (in cases where

the eect being tested for is dened in terms of an eect

size): the eect size quanties the strength of an eect,

such as the distance between two means or the correlation

between two variables.

[27]

5 See also

A/B testing

ABX test

Condence level, the complement of the signi-

cance level

Eect size

Fishers method for combining independent tests of

signicance

Look-elsewhere eect

Texas sharpshooter fallacy (gives examples of tests

where the signicance level was set too high)

Reasonable doubt

Statistical hypothesis testing

6 References

[1] Redmond, Carol; Colton, Theodore (2001). Clinical sig-

nicance versus statistical signicance. Biostatistics in

Clinical Trials. Wiley Reference Series in Biostatistics

(3rd ed.). West Sussex, United Kingdom: John Wiley &

Sons Ltd. pp. 3536. ISBN 0-471-82211-6.

[2] Cumming, Geo (2012). Understanding The New Statis-

tics: Eect Sizes, Condence Intervals, and Meta-Analysis.

New York, USA: Routledge. pp. 2728.

[3] Krzywinski, Martin; Altman, Naomi (30 October 2013).

Points of signicance: Signicance, P values and t-

tests. Nature Methods (Nature Publishing Group) 10

(11): 10411042. doi:10.1038/nmeth.2698. Retrieved

3 July 2014.

[4] Sham, Pak C.; Purcell, Shaun M (17 April 2014).

Statistical power and signicance testing in large-scale

genetic studies. Nature Reviews Genetics (Nature Pub-

lishing Group) 15 (5): 335346. doi:10.1038/nrg3706.

Retrieved 3 July 2014.

[5] Johnson, Valen E. (October 9, 2013). Revised stan-

dards for statistical evidence. Proceedings of the National

Academy of Sciences (National Academies of Science).

doi:10.1073/pnas.1313476110. Retrieved 3 July 2014.

[6] Altman, Douglas G. (1999). Practical Statistics for Med-

ical Research. New York, USA: Chapman & Hall/CRC.

p. 167. ISBN 978-0412276309.

[7] Devore, Jay L. (2011). Probability and Statistics for Engi-

neering and the Sciences (8th ed.). Boston, MA: Cengage

Learning. pp. 300344. ISBN 0-538-73352-7.

[8] Sirkin, R. Mark (2005). Two-sample t tests. Statistics

for the Social Sciences (3rd ed.). Thousand Oaks, CA:

SAGE Publications, Inc. pp. 271316. ISBN 1-412-

90546-X.

[9] Borror, Connie M. (2009). Statistical decision making.

The Certied Quality Engineer Handbook (3rd ed.). Mil-

waukee, WI: ASQ Quality Press. pp. 418472. ISBN

0-873-89745-5.

3

[10] Babbie, Earl R. (2013). The logic of sampling. The

Practice of Social Research (13th ed.). Belmont, CA: Cen-

gage Learning. pp. 185226. ISBN 1-133-04979-6.

[11] Faherty, Vincent (2008). Probability and statistical sig-

nicance. Compassionate Statistics: Applied Quantitative

Analysis for Social Services (With exercises and instruc-

tions in SPSS) (1st ed.). Thousand Oaks, CA: SAGE Pub-

lications, Inc. pp. 127138. ISBN 1-412-93982-8.

[12] Poletiek, Fenna H. (2001). Formal theories of testing.

Hypothesis-testing Behaviour. Essays in Cognitive Psy-

chology (1st ed.). East Sussex, United Kingdom: Psy-

chology Press. pp. 2948. ISBN 1-841-69159-3.

[13] Fisher, Ronald A. (1925). Statistical Methods for Research

Workers. Edinburgh, UK: Oliver and Boyd. p. 43. ISBN

0-050-02170-2.

[14] Schlotzhauer, Sandra (2007). Elementary Statistics Using

JMP (SAS Press) (PAP/CDR ed.). Cary, NC: SAS Insti-

tute. pp. 166169. ISBN 1-599-94375-1.

[15] McKillup, Steve (2006). Probability helps you make a

decision about your results. Statistics Explained: An In-

troductory Guide for Life Scientists (1st ed.). Cambridge,

United Kingdom: Cambridge University Press. pp. 44

56. ISBN 0-521-54316-9.

[16] Myers, Jerome L.; Well, Arnold D.; Lorch Jr, Robert F.

(2010). The t distribution and its applications. Research

Design and Statistical Analysis: Third Edition (3rd ed.).

New York, NY: Routledge. pp. 124153. ISBN 0-805-

86431-8.

[17] Quinn, Georey R.; Keough, Michael J. (2002). Experi-

mental Design and Data Analysis for Biologists (1st ed.).

Cambridge, UK: Cambridge University Press. pp. 4669.

ISBN 0-521-00976-6.

[18] Neyman, J.; Pearson, E.S. (1933). The testing of statisti-

cal hypotheses in relation to probabilities a priori. Math-

ematical Proceedings of the Cambridge Philosophical So-

ciety 29: 492510. doi:10.1017/S030500410001152X.

[19] Meier, Kenneth J.; Brudney, Jerey L.; Bohte, John

(2011). Applied Statistics for Public and Nonprot Admin-

istration (3rd ed.). Boston, MA: Cengage Learning. pp.

189209. ISBN 1-111-34280-6.

[20] Healy, Joseph F. (2009). The Essentials of Statistics: A

Tool for Social Research (2nd ed.). Belmont, CA: Cen-

gage Learning. pp. 177205. ISBN 0-495-60143-8.

[21] Cohen, Barry H. (2008). Explaining Psychological Statis-

tics (3rd ed.). Hoboken, NJ: John Wiley and Sons. pp.

4683. ISBN 0-470-00718-4.

[22] Health, David (1995). An Introduction To Experimental

Design And Statistics For Biology (1st ed.). Boston, MA:

CRC press. pp. 123154. ISBN 1-857-28132-2.

[23] Vaughan, Simon (2013). Scientic Inference: Learning

from Data (1st ed.). Cambridge, UK: Cambridge Uni-

versity Press. pp. 146152. ISBN 1-107-02482-X.

[24] Bracken, Michael B. (2013). Risk, Chance, and Causa-

tion: Investigating the Origins and Treatment of Disease

(1st ed.). New Haven, CT: Yale University Press. pp.

260276. ISBN 0-300-18884-6.

[25] Franklin, Allan (2013). Prologue: The rise of the sig-

mas. Shifting Standards: Experiments in Particle Physics

in the Twentieth Century (1st ed.). Pittsburgh, PA: Univer-

sity of Pittsburgh Press. pp. IiIii. ISBN 0-822-94430-8.

[26] Carver, Ronald P. (1978). The Case Against Statistical

Signicance Testing. Harvard Educational Review 48:

378399.

[27] Pedhazur, Elazar J.; Schmelkin, Liora P. (1991). Mea-

surement, Design, and Analysis: An Integrated Approach

(Student ed.). New York, NY: Psychology Press. pp.

180210. ISBN 0-805-81063-3.

7 Further reading

Ziliak, Stephen, and McCloskey, Deirdre, (2008).

The Cult of Statistical Signicance: How the Stan-

dard Error Costs Us Jobs, Justice, and Lives. Ann

Arbor, University of Michigan Press, 2009.

Thompson, Bruce, (2004). The signicance cri-

sis in psychology and education. Journal of Socio-

Economics, 33, pp. 607613.

Chow, Siu L., (1996). Statistical Signicance: Ra-

tionale, Validity and Utility, Volume 1 of series In-

troducing Statistical Methods, Sage Publications Ltd,

ISBN 978-0-7619-5205-3 argues that statistical

signicance is useful in certain circumstances.

Kline, Rex, (2004). Beyond Signicance Testing:

Reforming Data Analysis Methods in Behavioral Re-

search Washington, DC: American Psychological

Association.

8 External links

The article "Earliest Known Uses of Some of the

Words of Mathematics (S)" contains an entry on Sig-

nicance that provides some historical information.

"The Concept of Statistical Signicance Testing"

(February 1994): article by Bruce Thompon hosted

by the ERIC Clearinghouse on Assessment and

Evaluation, Washington, D.C.

"What does it mean for a result to be statistically

signicant"?" (no date): an article from the Statis-

tical Assessment Service at George Mason Univer-

sity, Washington, D.C.

4 9 TEXT AND IMAGE SOURCES, CONTRIBUTORS, AND LICENSES

9 Text and image sources, contributors, and licenses

9.1 Text

Statistical signicance Source: http://en.wikipedia.org/wiki/Statistical_significance?oldid=629745192 Contributors: Bryan Derksen, The

Anome, William Avery, Michael Hardy, Kku, Gabbe, Dcljr, Ellywa, Nichtich, Den fjttrade ankan, Nerd, Cherkash, Topbanana, Paranoid,

Gak, Henrygb, Giftlite, BrendanH, Pgan002, Antandrus, L353a1, DanielCD, Rich Farmbrough, Yknott, Kndiaye, Slb, Cretog8, Arcadian,

Andrewpmk, John Quiggin, Seans Potato Business, Alkarex, Woohookitty, Btyner, Rjwilmsi, Smoe, Thomas Arelatensis, Thisismikesother,

ElKevbo, Cjpun, EvanSeeds, Lborelli, Mathbot, Riki, Preslethe, Vonkje, Chobot, YurikBot, Wavelength, Gaius Cornelius, ENeville,

Nephron, DRosenbach, Jon Olav Vik, Doc pune, Lt-wiki-bot, Davril2020, Badgettrg, Darrel francis, SmackBot, McGeddon, Jtneill, Rob-

fuller, Ohnoitsjamie, Josefec, Nbarth, Danielkueh, Richard001, G716, Arodb, Euchiasmus, Tim bates, Nijdam, Tommyzee, Mmiller0712,

Mdgross50, Grapplequip, DwightKingsbury, Joseph Solis in Australia, Abeg92, Tawkerbot4, LarryQ, Thijs!bot, Tallred, Wildthing61476,

Erxnmedia, Fetchcomms, Magioladitis, Torchiest, Inhumandecency, MartinBot, ChemNerd, Lilac Soul, Coppertwig, Yym1997, Kenneth

M Burke, Spellcast, Philip Trueman, Don Quixote de la Mancha, MuanN, Seraphim, Sprasad.ee, SQL, Wangerin, Lavers, Jasondet, The-

G-Unit-Boss, Melcombe, Wjmummert, Martarius, ClueBot, Binksternet, Srudes2, Winsteps, Pwestfall, Lot49a, Qwfp, Staticshakedown,

Dthomsen8, SilvonenBot, Mifter, Aam aadmi, ZooFari, Jmkim dot com, Addbot, Eric Drexler, DOI bot, Fgnievinski, Bulletproofman19,

MrOllie, Palmerabollo, Numbo3-bot, Ehrenkater, Zorrobot, Luckas-bot, AnomieBOT, ChristopheS, Materialscientist, SvartMan, Xqbot,

Bbarkley, Sylwia Ufnalska, M12107, Constructive editor, FrescoBot, Sawomir Biay, Pinethicket, Edderso, Georg Hurtig, RedBot, Gjsis,

Cerebis, Animalparty, Indicedigini, Raylyons, Billare, Sir Arthur Williams, Rgmooney C109, GoingBatty, Schwa dk, HiW-Bot, Kostya 888,

Muditjai, Mysticyx, L Kensington, Mikhail Ryazanov, ClueBot NG, Mathstat, Michael D. Stephens, Helpful Pixie Bot, BG19bot, Wikstar7,

Lilingxi, Matthieu Vergne, Manoguru, Minsbot, MathewTownsend, BattyBot, HankW512, ChrisGualtieri, Eggingerik, BetseyTrotwood,

NicenFriendlyPerson, Sa publishers, Soranoch, Thewikiguru1, Rgiordan, EmilKarlsson, 1980na and Anonymous: 152

9.2 Images

File:Fisher_iris_versicolor_sepalwidth.svg Source: http://upload.wikimedia.org/wikipedia/commons/4/40/Fisher_iris_versicolor_

sepalwidth.svg License: CC-BY-SA-3.0 Contributors: en:Image:Fisher iris versicolor sepalwidth.png Original artist: en:User:Qwfp (origi-

nal); Pbroks13 (talk) (redraw)

File:NormalDist1.96.png Source: http://upload.wikimedia.org/wikipedia/en/b/bf/NormalDist1.96.png License: ? Contributors:

self-made

Original artist:

Qwfp (talk)

File:Wikiversity-logo.svg Source: http://upload.wikimedia.org/wikipedia/commons/9/91/Wikiversity-logo.svg License: ? Contributors:

Snorky (optimized and cleaned up by verdy_p) Original artist: Snorky (optimized and cleaned up by verdy_p)

9.3 Content license

Creative Commons Attribution-Share Alike 3.0

- BusinessStatisticsL3 MA202Uploaded bySarmela Damotheran
- Walpole Chapter 10Uploaded byMehmet Polat
- An Example of a HypothesisUploaded byAjibade Taofik
- Teste de Hipoteses ArtigoUploaded byFeolikelly
- Exam MATH2199 2012 (With Solutions)(1)Uploaded byJoe Di Napoli
- PSY 315 TUTORS Learn by Doing- Psy315tutors.comUploaded byleonardjonh5
- Business Research MethodsUploaded byDilawar
- Hypothesis TestingUploaded byKasun Geethaka
- Data AnalysisUploaded bygeniusMAHI
- Les4e Ppt Ch07Uploaded bybshakir
- Ch3 SlidesUploaded byRossy Dinda Pratiwi
- Assignment: Silk Road An EvaluationUploaded byGrace Raymond
- Midterm Exam With AnswersUploaded byShanna Basallo Alenton
- Testing of Hypothesis-AGBSUploaded bySonam Goel
- An Analysis of Survey Data on Students from the University Park Campus at Penn StateUploaded byJon Fivelsdal
- Stats 3 ReviewUploaded byAnonymous Zv32C1pt
- wp0704Uploaded byp anupam
- Hypothesis Testing Examples and ExercisesUploaded byWess Sklas
- Estadística - Ritchey Ch09Uploaded byVictoria Liendo
- Inferentials Stastistical AnalysisUploaded bychatfieldlohr
- Quiz 1 review questions (solutions).docxUploaded bySteven Nguyen
- Jfe September 80Uploaded byArif Kusanagi
- QuesUploaded byankitahanda
- Final ReportUploaded byPiyush Sharma
- IntroductionUploaded byajaydhameliya
- 6 Hypothesis TestingUploaded byMuhaini Mmj
- BSE 10 Market Capitalization folows Sentiment of Investors or Technical Methods- An analytical studyUploaded byAnonymous CwJeBCAXp
- math studies iaUploaded byapi-285168683
- The Statistical Significance of Randomized Controlled Trial Results is Frequently Fragile - A Case for a Fragility IndexUploaded byArlette Vassallo
- Ringkasan UAM ModellingUploaded byIotalaseria Putu

- Impact of Six-Sigma DMAIC Approach OnUploaded byVarun
- Bab III Integral GandaUploaded byMessy Cool
- 2010_JoP_Position PaperUploaded byJohn Richardson
- Application of Regression Models for Area, Production and Productivity Trends of Maize (Zea mays) Crop for Panchmahal Region of Gujarat State, IndiaUploaded byEditor IJRITCC
- Changing Everyday Memory Behaviour in AmnesticUploaded bydaniloc9430
- The Impact of InterestUploaded byCristie Sumbilla
- Intro_Lec_3.pdfUploaded bySohail Saleem Jutt
- Davim, J. Paulo -- Statistical and Computational Techniques in Manufacturing __ Finite Element Modeling of Chip Formation in Orthogonal MachiningUploaded byYanuar Pratama
- Sample Risk Management PlanUploaded byEdwin Okoampa Boadu
- Tikona PresentationUploaded bySoumya Chopra
- As NZS 2433-1994 Plastics - Method for Exposure to Ultraviolet LampsUploaded bySAI Global - APAC
- Oviatt and McDougall 1994 JIBS Toward a Theory of International New Ventures(1)Uploaded byJoao Serra
- competency 001Uploaded byapi-341505564
- Stats GC Formula SheetUploaded bygeoz88
- Vygotsky.pptUploaded byVictor M. Morcillo
- Our ITIL Journey: Moving a Decentralized IT Organization to a Common Incident Management Process and Common ITSM Tool (241322223)Uploaded byEDUCAUSE
- Sandra Bai CVUploaded bySandra Bai
- Stat a IntroUploaded bydontcare16355
- 200102Sedimentation in Storage ReservoirsUploaded byjoaoaugusto79
- Challenges and SuccessesUploaded byShoutik Chakrabarti
- Canadian Medical Residency FM Guide for CarmsUploaded byJatinder Singh
- APA style guideUploaded byipjcta
- New Microsoft Office Word Document (4)Uploaded byPraveen Cp
- Big Five and Goal OrientationUploaded byMardye
- bed alarms and fall riskUploaded byapi-253792021
- presedex poster qayyumUploaded byapi-349637577
- SChapter_1Uploaded bydivvi
- weebly-digital unit plan - merging two culturesUploaded byapi-351765918
- HRM CH 10Uploaded byAbdur Rehman
- Preference for Group LeadershipUploaded byBilu