Scale Development Process

Dr. Sandhya Sreekumar

1
 Scale Development
– Scale development:
• It is a process of representing the range of possible responses to a question
about a particular construct
• The goal of scale development is to precisely quantify what is to be
measured

– A scale is:
• The combined set of points that anchor the measurement tool

2
 Scale Development
• Item: An individual article or unit, especially one that is part of a list, collection,
or set.
• Variable: It is any characteristics, number, or quantity that can be measured or
counted. A variable may also be called a data item. Age, sex, business income.
• Construct: An idea or theory containing various conceptual elements,
typically one considered to be subjective and not based on empirical evidence.

3
Development of a Multi-item Scale
Develop Theory

Generate Initial Pool of Items: Theory, Secondary Data, and

Qualitative Research

Select a Reduced Set of Items Based on Qualitative Judgement

Collect Data from a Large Pretest Sample

Statistical Analysis

Develop Purified Scale

Collect More Data from a Different Sample

Evaluate Scale Reliability, Validity, and Generalizability

Final Scale

4
 Steps in Scale
Development
• Construct Definition
• Item Generation
• Content Validity & Pretesting
• Measurement Purification
• Verification of Dimensionality
• Nomo logical Validity
• Criterion related Validity
• Accounting for Known issues in measurement scales

5
 Construct Definition
– A construct is:
• A hypothetical variable comprised of a set of component responses or
behaviours that are thought to be related
– Construct development:
• The integrative process of identifying the subjective
properties of a
variable
• The goal of construct development is to precisely identify and define what
is to be measured

6
 Construct Development

The degree to which the construct consists of

Construct abstractness concrete versus subjective properties; the more
subjective, the more abstract the construct

Underlying a construct is the set of identifiable and

Construct dimensionality measurable components that constitute that
construct’s domain of observables

The process of establishing that the construct is

Construct validity valid, by testing for content, convergent,
discriminant and nomological validity

7
 Item Generation
Steps in Scale Development Number of Items

• Impossible to answer
• Thumb rules Devellis (2003) – 3 to 4 times large than final scale.
• Thumb rule for final scale for each dimension minimum 3 items.

8
 Item Generation
An Example SERVQUAL (Service quality)
• Initial Dimension – 10
• Initial Item Generation -97
• Final Dimensions – 5
• Number of items – 22
– Tangibles – 4
– Reliability -5
– Responsiveness -4
– Assurance – 4
– Empathy -5
– % Reduction in items = 77%
Parasuraman, A., Zeithaml, V. A., & Berry, L. L. (1988). Servqual. Journal of retailing, 64(1),
12- 40.

9
• Communication • Tangibles
• Credibility • Reliability
• Security • Responsiveness
• Competence • Assurance
• Courtesy • Empathy

10
 Item Generation
Characteristics of good or bad item
a) Exceptionally lengthy items
• It is fair to say that one of the things I seem to have a problem with much of the time is getting
my point across to other people.
• I often have difficulty making a point.
b) Reading difficulty
c) Redundancy –
• In my estimation pet lovers are kind people
• Vegetarians are kind in nature

11
 Item Generation
d) Multiple Negatives
• I am not in favor of corporations stopping funding for antinuclear groups
• I favor continued private support of groups advocating a nuclear ban
e) Double barreled
• How satisfied are you with your pay and job conditions?

12
 Validity
• The validity of a scale may be defined as the extent to which differences in observed scale
scores reflect true differences among objects on the characteristic being measured, rather than
systematic or random error. Perfect validity requires that there be no measurement error (XO
= XT, XR = 0, XS = 0).

• Content validity is a subjective but systematic evaluation of how well the content of a scale
represents the measurement task at hand.
• Criterion validity reflects whether a scale performs as expected in relation to other variables
selected (criterion variables) as meaningful criteria.

13
 Random and Systematic Error
Random Error
1) fluctuations in the person’s current mood.
2) misreading or misunderstanding the questions
3) measurement of the individuals on different days or in different
places.

These error may cancel out as you collect many samples

Systematic Error

Sources of error including the style of measurement, tendency

toward self-promotion, cooperative reporting, and other conceptual
variables are being measured.
 Content Validity
the degree to which an instrument has an appropriate sample of items for the
construct
being measured

1. Initial screening of items

2. Expert assessment of the applicability of the items to each dimension
3. Expert assessment of representativeness of the items to each dimension

15
 Pre-testing /Pilot Testing
• Pre-testing is the administration of the data collection instrument with a small set of
respondents from the population.
• The purpose of pre-testing is to identify problems with the data collection
instrument and find possible solutions.
• Recommended sample Size 10% of final sample size.

16
 Potential Sources of Error on
Measurement
1) Other relatively stable characteristics of the individual that influence the test score, such as intelligence,
social desirability, and education.
2) Short-term or transient personal factors, such as health, emotions, and fatigue.
3) Situational factors, such as the presence of other people, noise, and distractions.
4) Sampling of items included in the scale: addition, deletion, or changes in the scale items.
5) Lack of clarity of the scale, including the instructions or the items themselves.
6) Mechanical factors, such as poor printing, overcrowding items in the questionnaire, and poor design.
7) Administration of the scale, such as differences among interviewers.
8) Analysis factors, such as differences in scoring and statistical analysis.

17