© C.

Thatchinamoorthy Phd Scholar 2016

 Meaning

 Definition

 Types

 Factors influencing reliability of test

 How to Improve reliability of test

© C.Thatchinamoorthy Phd Scholar 2016

 Reliability is one of theimportant
characteristics of any test.

 It refer to theprecision or accuracy of the

measurement of score.

 Reliability refers to the stability of a test measure or

protocol.

© Thatchinamoorthy, PhD scholar

 Rosenthal(1991): Reliability is a major concern when
a psychological test is used to measure some attribute
or behaviour.

 Anastasi (1968): Reliability refers to the consistency

of scores obtained by the same individuals when re-
examined with test on different occasions, or with
different sets of equivalent items, or under other
variable examining conditions.

© Thatchinamoorthy, PhD scholar

 RELIABILIT
Y
 Reliability is the extent to which an experiment,
test, or any measuring procedure shows the same
result on repeated trials. The researchers would
be unable to draw conclusions, formulate
theories, or make claims about the
generalizability of their research:
 Without the agreement of independent
observers able to replicate research
procedures, or
 the ability to use research tools and procedures
that produce consistent measurements,
 For researchers, four key types of reliability
are:
 Four important types

◦ Test –Retest Reliability

◦ Split – half or Internal Consistency Reliability

◦ Parallel forms reliability or Equivalent –

forms
◦ Inter-rater reliability

© Thatchinamoorthy, PhD scholar

 4. INTER-RATER RELIABILITY
• Inter-rater reliability is the extent to which two or
more individuals (coders or raters) agree on thetest
scores/interpretation.
• Inter-rater reliability assesses the consistency of how a
measuring system is implemented. For example, when two
or more teachers use a rating scale with which they are
rating the students’ oral responses in an interview (1
being most negative, 5 being most positive). If one
researcher gives a "1" to a student response, while
another researcher gives a "5," obviously the inter-rater
reliability would be inconsistent.
• Inter- rater reliability is dependent upon the ability of
two or more individuals to be consistent.
• Training, education and monitoring skills can enhance
inter-rater reliability.
 In test –retest reliability the single form of the test is
administered twice on the same sample with a
reasonable time gap.

 In this way two administration of the same form of

the two independent sets of scores.

 The two sets, when correlated, give the value of the

reliability coefficient.

© Thatchinamoorthy, PhD scholar

 Same Test
Same Sample
Different Times

© Thatchinamoorthy, PhD scholar

 Measure instrument at two times for multiple
persons.

 Compute correlation between the two measures.

 Assumesthere is no change in theunderlying

trait between time 1 and time 2.

© Thatchinamoorthy, PhD scholar

 Contributing factors:
◦ (1) Clear instructions for administrators,
research participants, and raters

◦ (2) Tasks/questions in participants’ first language or

target language at appropriate level of difficulty

◦ (3) Unambiguously phrased tasks/questions

 Other name Internal Consistency reliability

 It indicates the homogeneity of the test

 This method thetest is divided into two

equal or nearly halves

 Common way of this test is the odd-even method

© Thatchinamoorthy, PhD scholar

© Thatchinamoorthy, PhD scholar
Internal consistency is the extent to which all the items on a test or
procedures assess the same characteristic, skill or quality. It is a measure
of the precision between the various sections of the measuring
instruments used in a study. This type of reliability often helps
researchers interpret data and predict the value of scores and the limits
of the relationship among variables. For example, analyzing the internal
reliability of the items on a vocabulary quiz will reveal the extent to
which the quiz focuses on the examinee’s knowledge of words.
 Contributing
factors:
◦ (1) Careful item writing, guided by item specifications

◦ (2) Field test and item analysis

◦ (3) Construction of tests with reference to item performance

 This reliability various names such as,

◦ Alternative- forms reliability

◦ Equivalent –forms reliability

◦ Comparable – forms reliability

© Thatchinamoorthy, PhD scholar

 The alternative forms technique to estimate reliability
is similar to the test retest method, except that
different measures of a behaviour (rather than the
same measure) are collected at different times.

 If the correlation between the alternative forms is

low, it could indicate that considerable measurement
error is present, because two different scales were
used.

© Thatchinamoorthy, PhD scholar

© Thatchinamoorthy, PhD scholar
SOURCES OF ERROR

 Examinee (is a human being)

 Examiner (is a human being)

 Examination (is designed by and for

human beings)
 Group Variability

 Guessing by the examinees

 Environmental Conditions
 When the group of examinees being tested is
homogenous in ability, the reliability of the test scores
is likely to be lowered.

 But when the examinees vary widely in their range of

ability, that is, the group of examinees is a
heterogeneous one, the reliability of the test scores is
likely to be high.
 Guessing in a test is an important source of
unreliability.

 In two alternative response options there is a 50 %

chance of answering the items correctly on the basis of
the guess.
 Testing environment should be uniform

 Arrangement should be such that light, sound, and

other comforts are equal and uniform to all the
examinees.

 Otherwise it will tend to lower the reliability of the test

scores
 The group of examinees should be heterogeneous, that
is, the examinees should vary widely in their ability or
trait being measured.
 Items should be homogenous.
 Test should preferably be a longer one.
  C.Thatchinamoorhy (2015-16) Class Notes, AEX 812-
Advances in designs and techniques in Social Science
Research (2+1)

 Dr.T.Rathakrishnan (2009). Advances & Challenges in

Agricultural Extension & Rural Development.

 Rosenthal, R. and Rosnow, R. L. (1991). Essentials of

Behavioral Research: Methods and Data Analysis. Second
Edition. McGraw-Hill Publishing Company, pp. 46-65

 http://www.socialresearchmethods.net/kb/mtmmmat.htm

© Taylor & Francis 2014