METODE PENELITIAN KUANTITATIF

VALIDITAS DAN RELIABILITAS PENGUKURAN

Bagian VI

Validitas dan Relialibilitas

METHODOLOGICAL GOODNESS / QUALITY CRITERIA Trustworthiness Constructiv ism Authenticity

Critical

• Historical situatedness • Wholeness

Classical

• Objectivity • External Validity • Internal validity

• Measurement validity and reliability • Design and analysis validity

Validitas dan Relialibilitas

Validitas dan Relialibilitas

MEASUREMENT RESULTS: variances/scores differences SOURCES OF VARIANCES/ SCORES DIFFERENCES
(Singleton, 1988; p.112; Selltiz, et.al., pp. 164-169)

TRUE DIFFERENCES
differences in the concept the measure intended to measure (Singleton, 1988); differences in the characteristics we are attempting to measure (Selltiz, 1976).

RANDOM ERRORS
measurement errors due to random or chance factors: (Singleton, 1988) or transient aspects of the person, of the situation of measurement, or of the measurement procedures that are likely to vary from one act of measurement to the next, even though the characteristics we are trying to measure has not changed. (Selltiz, 1976)

SYSTEMATIC ERRORS
biases inherent in the method or operational definition (Singleton,1988) ; is an error introduced into the measurement by some factor that systematically affects the characteristics being measured or the process of measurement (Selltiz, et el., 1976)

Validitas dan Relialibilitas

Measurement errors

MEASUREMENT ERRORS

RANDOM ERRORS

SYSTEMATIC ERRORS

• Kesalahan yang terjadi secara random/acak, akibat kondisi, proses, atau variasi prosedur pengukuran yang dilakukan.

Kesalahan yang terjadi secara sistematis, antara lain bersumber dari faktor-faktor yang inherent dalam alat ukur atau definisi operasional konsep yang diukur.

• Membias secara acak ke berbagai arah kemungkinan

• Membias ke satu arah kemungkinan tertentu

RELIABILITY
Validitas dan Relialibilitas

VALIDITY

Tingkatan sejauh mana pengukuran yang dilakukan memperoleh hasilyang konsisten (antar waktu, antar pengamat, antar indikator, dsb.)

Tingkatan sejauh mana pengukuran yang dilakukan benarbenar mengukur konsep yang semula akan diukur

The consistency of a measure (Bailey, 1987)

The degree to which a test measures what it purports to measure (Borg & Gall;
1971)

Validitas dan Relialibilitas

RELATIONSHIP BETWEEN RELIABILITY AND VALIDITY

• A MEASURE CAN BE RELIABLE BUT INVALID • RELIABILITY IS A NECESSARY CONDITION FOR VALIDITY

LOW RELIABILITY LOW VALIDITY • • • HIGH RELIABILITY • • LOW VALIDITY • •

• • •• •• • LOW RELIABILITY HIGH VALIDITY???

HIGH RELIABILITY HIGH VALIDITY
Validitas dan Relialibilitas

RELIABILITY IS A NECESSARY BUT NOT SUFFICIENT CONDITION FOR VALIDITY

high •• • • ••⊗ • • •••

RELIABILITY AND VALIDITY Reliability low • • • ⊗ • • • • • • • • • • • •

high

Validity • • •• • ••• •• • • ⊗

low
⊗ = actual parameter
Validitas dan Relialibilitas

• • • • • • • • • • • • • • • ⊗

= measure

VALIDITY AND RELIABILITY

FACE - in the judgment of
others

CONTENT
Pre-data Validity

SAMPLING - capture the
entire dimensions

CONCURRENT - preexisting
criterion

VALIDITY

CRITERION
Data-based -Validity

PREDICTIVE
criterion

- predicted

CONVERGENT - similar
construct MEASUREMENT QUALITY

CONSTRUCT
Data-based Validity

DIVERGENT
construct

- opposing

Validitas dan Relialibilitas

STABILITY RELIABILITY EQUIVALEN CE

OVER TIME OVER ALTERNATE / RATER internal consistency

HOMOGENE OVER INDICATORS ITY
Sources: Allen and Yen (1979); Seltiz et al.(1986); Sekaran (1992)

unidimensionality

Validitas dan Relialibilitas

VALIDITY ASSESSMENTS
CONTENT/ APPARENT VALIDITY FACE VALIDITY

. . . in the judgment of others -

Kesepakatan pakar tentang sejauh mana definisi operasional/ indikator yang dipergunakan suatu instrumen benar-benar mengukur konsep yang ingin diukur LOGICAL/SAMPLING VALIDITY . . . capture the entire dimensions Kesepakatan pakar tentang sejauh mana definisi operasional/ indikator-indikator suatu instrumen mewakili keseluruhan dimensi konsep yang diukur

CRITERIONRELATED VALIDITY Pragmatic validity

CONCURRENT VALIDITY

correlatred with a preexisting criterion - distinguish objects that differ in their present status - tingkat sejauh mana hasil
pengukuran berkorelasi dengan pengukuran konsep lain atau kondisi tertentu yang diasumsikan sebagai kriteria (e.g.: Kemampuan Index of Democracy menempatkan AS dan Korut dalam kategori berbeda) PREDICTIVE VALIDITY

correlated with future/predicted condition - distinguish objects that will differ in the future . Tingkat sejauh mana
hasil pengukuran suatu konsep mampu memprediksi keadaan di masa datang (Contoh: Test Potensi Akademik 2003 dan Index Prestasi Kumulatif ( 2005) CONSTRUC T VALIDITY Theoryrelated validity CONVERGENT VALIDITY

positively correlated with similar constructs
Tingkat sejauh mana hasil pengukuran suatu konsep ber-hubungan positif dengan hasil

Validitas dan Relialibilitas

pengukuran konsep lain yang secara teoretis harus sama (e.g.: Index Human Freedom Index dan Civil Liberties Index) DISCRIMINANT VALIDITY

uncorrelated or negatively correlated with opposing construct - Tingkat sejauh

mana hasil pengukuran suatu konsep berbeda (tidak berkorelasi / berkorelasi negatif) dengan pengukuran konsep lain yang secara teoretis harus berbeda (Contoh: Job Satisfaction Index dan Labor Turn-over Index)

Validitas dan Relialibilitas

RELIABILITY
The consistency of a measure (Bailey, 1987)

STABILITY

OVERTIME RELIABILITY:
Konsistensi hasil pengukuran dari waktu ke waktu

RELIABILITY RELIABILITY:

EQUIVALENCE

OVER-RATERS/ALTERNATE

Konsistensi hasil suatu pengukuran dng. pengukuran lain yang serupa, atau dng. pengukuran yang dilakukan pengamat lain (memakai instrumen yang sama) internal (summative rating

consistency scales)

HOMOGENEITY

OVER-INDICATORS:

Konsistensi hasil pengukuran antar indikator dalam suatu instrumen pengukuran

unidimensionality
Validitas dan Relialibilitas

(cumulative rating scales)

Validitas dan Relialibilitas

STABILITY ESTIMATES: OVERTIME RELIABILITY Pearson’s r Correlation Coefficient

SUBJECTS A 150.00 B 90.00 C 113.75 D 8.75 E 20.00 F 33.75 G 75.00
N=7 491.25

TEST-1 (X) 4.0 3.0 3.5 .5 1.0 1.5 2.5
∑X= 160

TEST-2 (Y) 3.75 3.00 3.25 1.75 2.00 2.25 3.00

X2 16.00 9.00 12.25 0.25 1.00 2.25 6.25

Y2

XY 14.06 9.00 10.56 3.06 4.00 5.06 9.00
XY=

∑Y= 19.00 ∑X2= 4700 ∑X2= 54.74

Pearson’s r Correlation Coefficient N∑ xy rxy = (∑x) (∑y) = 0.98 [ N∑x2 - (∑x)2 ] [ N∑y2 - (∑y)2 ]

Statistical significance

Validitas dan Relialibilitas

∝ = 0.05 df = n - 2 = 5 rc0.05;5 = 0.74 r xy = 0.98 > rc0.05;5 ( reject H 0)

critical value significant

HOMEGENEITY ESTIMATES: INTERNAL CONSISTENCY

(summative rating scales ) • Spearman-Brown coefficient Split-half reliability coefficients rX1X2 = 2 rY1Y2 _____________ 1 + rY1Y2

rX1X2 = reliabilitas pengukuran seluruhnya rY1Y2 = korelasi antara skor belahan 1 dan 2 Cronbach's α 2 [ s2 - ( s2Y1 + s2Y2 ) ] ______________________ s2Y1Y2

rX1X2 =

rX1X2 = reliabilitas pengukuran seluruhnya s2Y1 = varians skor belahan 1 s2Y2 = varians skor belahan 2
Validitas dan Relialibilitas

HOMEGENEITY ESTIMATES: INTERNAL CONSISTENCY
Pearson’s r Correlation Coefficient
N∑ x1x2 - (∑x1) (∑x2) rxy = [ N∑x12 - (∑x1)2 ] [ N∑ x22 - (∑x2)2 ]

Spearman-Brown coefficient 2 rx1,x2 rxy = 1 + rX1,X2

NEGARA A B C D E F G H I J K L

Skor HFI belahan 1 ( X1 ) 14 15 4 13 12 3 14 15 4 13 12 3

Skor HFI belahan 2 ( X2) 14 14 5 12 14 4 14 14 5 12 14 4

X12

X22

X1X2

∑X12

∑X22

∑X1X2

Validitas dan Relialibilitas

HOMEGENEITY ESTIMATES UNIDIMENSIONALITY

(cumulative rating scales )
COEFFICIENT OF REPRODUCIBILITY: . . . the percentage of original responses that could be reproduced by knowing the scale scores to summarize them (Babbie, 1992; p.186).
Contoh penghitungan coefficient of reproducibility skala Social Distance INDICATORS:

1.Bersedia menerima imigran Asia sebagai warganegara 2.Bersedia menerima imigran Asia sebagai rekan kerja 3.Bersedia menikah dengan imigran Asia Observatio n pattern Scale types 1 1 1 1 0 0 1 0 1 2 0 1 1 0 1 0 0 0 3 0 0 1 0 1 1 1 1 No. of Index Scale responde score scores nts s 10 7 2 8 1 4 5 7 44 number of 1 2 3 0 2 2 1 2 1 2 3 0 3 3 2 3

YE S 1 1 1

NO 0 0 0

Total scale errors 0 0 0 0 1 4 5 7 17

Mixed types

errors

CoR = item

1number of cases x number of

Validitas dan Relialibilitas

16 Sep 98 SPSS for MS WINDOWS Release 6.0

Page 6

RANKING INDONESIA : CIVIL LIBERTIES - POLITICAL RIGHTS - POLYARCHY HUMAN FREEDOM CNTRY Mexico SKorea Thailand India Singapore Egypt Philippine Malaysia Chile Bangladesh Saudi Indonesia Pakistan China Libya N = RCIVLIB8 43.0 33.5 33.5 33.5 52.0 43.0 33.5 52.0 43.0 52.0 69.0 52.0 33.5 62.0 62.0 71.0 RPOLRIG8 41.0 31.5 41.0 31.5 45.5 50.5 31.5 45.5 50.5 45.5 61.5 50.5 41.0 61.5 61.5 71.0 RPOLY85 18.0 18.0 8.5 8.5 18.0 18.0 18.0 18.0 25.0 25.0 43.5 25.0 28.5 43.5 43.5 45.0 RHDI87 29.0 24.0 38.0 61.0 25.5 55.0 45.0 34.0 22.0 67.0 46.0 52.0 62.0 44.0 43.0 71.0 RHFI85 36.0 38.5 38.5 38.5 45.0 45.0 49.0 na 53.5 58.0 60.0 62.5 62.5 66.5 68.0 71.0

Number of cases read:

71

Number of cases listed:

71

HUMAN FREEDOM INDEX
HFI disusun sebagai suatu Summative Rating Scale (Likert-type Scale) berdasarkan 40 indikator, a.l., kebebasan untuk menyatakan pendapat, melakukan oposisi, menentukan pasangan, kebebasan dari penganiayaan, dsb. oleh Charles Humana (1986). INDICATORS 4 Respect for rights or freedom 3 Some violations or infringement s 2 Substantial violation 1 Continuous or total denial

Rights to: 03. To assembly Freedom from: 07. Unlawful detention Freedom for: 19. Political
Validitas dan Relialibilitas

opposition Personal Rights 40. Homosexuality
HUMAN FREEDOM INDEX (HFI): Dalam laporan tahunan UNDP, Human Development Report 1991, mempergunakan data Human Rights Rating yang disusun oleh Charles Humana (World Human Rughts Guide 1986).

Validitas dan Relialibilitas

POLYARCHY INDEX
Indeks Polyarchy dikembangkan sebagai Cummulative Rating Scale (Guttman-type Scale). Mengukur pluralisme politik berdasarkan keberadaan perangkat tatanan kelembagaan yang memungkinkan dan menjamin adanya oposisi publik dan hak untuk berpatisipasi dalam proses-proses politik
. . . the set of institutional arrangements that permits public opposition and establishes the right to participate in politics (Coppedge and Reinicke, 1993; p.47).

CONCEPT

DIMENSI ONS
FREE & FAIR ELECTION

INDICATORS
1.No meaningful election are held 2.Marred by fraud and coercion 3.Meaningful fair election 1.All organizations are banned or controlled 2.Only nonpolitical organizations are allowed 3. Some independent political organizations are banned 4.Full freedom for political organization 1.All public dissent is suppressed 2.Some public dissent is suppressed 3.Full freedom of expression 1.No public alternative to

POLYARC HY

FREEDOM OF ORGANIZA TION

FREEDOM OF EXPRESSIO N AVAILABILI

TY OF ALTERNATI VE INFO SOURCES

official information 2.Alternative sources exist only for nonpolitical issues 3.There is preferential presentation of official views in the media 4.No preferential presentation of official views in the media

Disusun berdasarkan: Coppedge and Reinicke (1993) “Measuring Polyarchy”. In Inkeles, Inkeles, Alex (Ed.), On Measuring Democracy: Its Cosequences and Concomitants. New Brunswick, London: Transaction Publishers; pp. 47-68.

POLITICAL RIGHTS - CIVIL LIBERTIES Dua Indeks yang sebagai kombinasi dimaksudkan untuk menentukan tingkat kehidupan demokrasi. Suatu Summative Rating Scale (Likert-type Scale) yang dikembangkan oleh Raymond Gastil, berdasarkan 11 indikator Political Rights dan 14 indikator Civil Liberties
( lihat “The Comparative Survey of Freedom: Experiences and Suggestions”. In Inkeles, Inkeles, Alex (Ed.), On Measuring Democracy: Its Cosequences and Concomitants. New Brunswick, London: Transaction Publishers; pp. 21-46.).

Sample Checklist for Political Rights 1. Chief authority recently elected by a meaningful process 2. Legislature recently elected by a meaningful process Alternatives for 1 and 2: a. No choice and possibility of rejection b. No choice but some possibility of rejection c. Government or single-party selected candidates d. Choice possible only among governmentapproved candidates

e. Relatively open choices possible only in local elections f. Open choice possible within a restricted range g. Relatively open choices possible in all elections Sample Checklist for Civil Liberties 17. Free from unjustified political terror or imprisonment 18. Free trade unions, peasant organizations, or equivalent 19. Free businesses or cooperatives

- -

Correlation Coefficients

HFI85 HDI87

-

HFI85

HDI87

- -

CIVLIB88

POLRIG88

POLY85

GNP6588

.6130 ( 68) P= .000 .8142 ( 68) P= .000 .7612 ( 68) P= .000 .7102 ( 42) P= .000 .1118 ( 68) P= .364

-

CIVLIB88

.7174 ( 71) P= .000 .6706 ( 71) P= .000 .4652 ( 45) P= .001 .3144 ( 71) P= .008

-

POLRIG88

.9284 ( 71) P= .000 .8295 ( 45) P= .000 .1674 ( 71) P= .163

-

POLY85

.7830 ( 45) P= .000 .2306 ( 71) P= .053

-

GNP6588

R E L I A B I L I T Y N of Cases = 86.0 Guttman Split-half = 20 Items in part 1

A N A L Y S I S .9791

S C A L E (S P L I T) N of Items = 40 Unequal-length Spearman-Brown = 20 Items in part 2

.2514 ( 45) P= .096
.9820

-

Validitas dn Reliabilitas

26

FACTORIAL VALIDITY
CONFIRMATORY FACTOR ANALYSIS

LIBERALISM – CONSERVATISM A DUALISM CONCEPT
LIBERALISM CONSERVATISM DUALITY LIBERALISM CONSERVATISM DUALISM

CON1 CON2 CON3 LIB1 LIB2

FACTOR . . . a construct, a hypothetical entity, a latent variable that is assumed to underlie

LIB3

FACTOR I FACTOR II .03513 .9476 6 .09007 .9408 2 .05609 .8541 0 .8109 .50260 1 .9434 .05173 8 .9458 .10260 4

Validitas dn Reliabilitas

27

tests, scales, items, and indeed, measures of almost any kind
p. 569).

(Kerlinger, 1986;

PRINCIPLES TO INCREASE THE RELIABILITY OF MEASURES
(Neuman, 1997; pp. 147-148)

• • • •

Clearly conceptualize all constructs Increase the level of measurements Use multiple indicators of a variable Use pretest, pilot studies, and replication

Validitas dn Reliabilitas

28

Validitas dn Reliabilitas

29

ITEM-ANALYSIS UNTUK PENINGKATAN RELIABILITAS

JOB SATISFACTION SCALE
(Adopted from Schuessler’s Job Satisfaction Scale. In Miller, 1991; p.453)

Statements

SD A 5

DA 4

N 3

A 2

SA 1

1. There is too little variety in my job 2. I tend to get bored on the job 3. There must be better places to work 4. I would like more freedom on the job 5. I have to small a share in deciding matters that affect my work 6. My job means more to me than just money* 7. I am not satisfied with the the work I do 8. My job gives me a chance to do what I do best* 9. People feel like they belong where I work*
*Positive statements
12 Oct 98 SPSS for MS WINDOWS Release 6.0 Page 1

Statistics for SCALE
Validitas dn Reliabilitas

Mean 31.4733

Variance 4.2743

Std Dev 2.0674

N of Variables 9
30

N of Cases =

129.0

JOB SATISFACTION SCALE
R E L I A B I L I T Y Reliability Coefficients N of Cases = Alpha = .6198 129.0

A N A L Y S I S

-

S C A L E

(A L P H A)

N of Items =

9

Item-total Statistics

Scale Mean if Item Deleted Q1 Q2 Q3 Q4 Q5 Q6 Q7 28.0275 27.9570 27.9612 27.9756 27.9054 27.9399 28.1163

Scale Variance if Item Deleted 3.5993 3.3573 3.2213 2.4755 3.2929 3.2890 3.2524

Corrected ItemTotal Correlation .1503 .5014 .5466 .7012 .3719 .5468 .5300

Alpha if Item Deleted .6419 .5462 .5301 .4419 .5719 .5347 .5350
31

Validitas dn Reliabilitas

Q8 Q9

27.9008 28.0023

4.5462 4.8057

-.3015 -.4374
Page 5
Page 2

.6782 .7132

12 Oct 98 SPSS for MS WINDOWS Release 6.0
13 Oct 98 SPSS for MS WINDOWS Release 6.0

JOB SATISFACTION SCALE
R E L I A B I L I T Y A N A L Y S I S S C A L E (A L P H A)

Item-total Statistics Scale Mean if Item Deleted Q2 Q3 Q4 Q5 Q6 Q7 17.4627 17.4658 17.4873 17.4181 17.4485 17.6254 Scale Variance if Item Deleted 3.6783 3.5845 2.4132 2.9834 3.1811 3.2917 Corrected ItemTotal Correlation .3779 .3960 .8262 .6503 .7433 .6072 Alpha if Item Deleted .8343 .8326 .7372 .7829 .7700 .7938

Reliability Coefficients
Validitas dn Reliabilitas 32

N of Cases = Alpha = .8241

130.0

N of Items =

6

13 Oct 98 SPSS for MS WINDOWS Release 6.0

Page 3

WORDING EFFECTS IN REFERENDUM % Differences b/w cons and pros Common Market Should the United Kingdom come out of the Common Market?

+ 10.8 - 11.2

The Government recommends the acceptance of the renegotiated terms of British membership of the Common Market. Should the United Kingdom stay in the Common Market? Her Mayesty’s government believes that the nation’s best interests would be
Validitas dn Reliabilitas

- 16.2
33

served by accepting the favourably negotiated terms of our continued membership of the Common Market. Should the United Kingdom stay in the Common Market
Source: Butler and Kitzinger (1976; p.60)

Validitas dn Reliabilitas

34

BEBERAPA PRINSIP PENYUSUNAN INSTRUMEN PERTANYAAN
(Sumber: Neuman, 1997; pp.223-237)

• Avoid jargon, slang, and abbreviations
“Apakah suami anda memperlihatkan gejala Oedipus complex?”

• Avoid ambiguity, confusion, and vagueness
“Apakah anda berolahraga secara teratur?”

• Avoid emotional language and prestige bias
“Kalangan berpendidikan umumnya setuju agar film-film televisi bertema kekerasandan sex dilarang. Apakah anda setuju dengan pelarangan film-film televisi seperti itu?”

• Avoid double-barreled questions
“Apakah anda setuju dengan rencana untuk menyelenggarakan Sidang MPR bulan Oktober 1998 dan Pemilihan Umum bulan Mei 1999?”

• Avoid leading questions
“Haruskah pemerintah mengucurkan dana lebih banyak lagi untuk mengamankan jalanan ibukota yang semakin rawan kejahatan?”

• Avoid asking questions that are beyond respondents’ capabilities
“Berapa menitkah rata-rata anda menonton televisi selama tiga bulan terakhir ini?”

• Avoid false premises
“Pemda telah terlalu banyak memberi layanan kepada masyarakat. Apakah anda setuju layanan tersebut dikurangi demi menghemat biaya?”

Validitas dn Reliabilitas

35

• Avoid overlapping or unbalanced
“Bagaimanakah penilaian anda terhadap kinerja pimpinan anda? Luar biasa, istimewa, sangat bagus, bagus, atau memuaskan?

Validitas dn Reliabilitas

36

Sign up to vote on this title
UsefulNot useful