Professional Documents
Culture Documents
Tejinder Sharma
Department of Commerce
Kurukshetra University,
Kurukshetra 136119
sharmatejinder@gmail.com
Measurement &
Questionnaire Design
• Agenda
– Measurement
– Scale development
– Questionnaire design
Defining Measurement
• Nominal
• Ordinal
• Interval
• Ratio
Nominal Scale
• One level measurement
• Numbers are used as labels to classify the products
• Used for demographic variables, types of products,
stores, etc.
• Eg. PIN numbers 136118 for Kurukshetra
• Analysis – counting, frequency, percentage, mode,
binomial test, chi square test, etc.
Ordinal Scale
• Provides information about ordered
relationship among the objects
• Contains all information of nominal scale and
measure of whether an object is more or less
characteristics of the other object
• For example – Ranking of preferences
• Statistical Analysis – Mathematical operations,
percentage, median, rank correlation, non-
parametric tests
Example of Ordinal Scale
Respondent Adidas Nike Reebok
A 2 1 3
B 1 2 3
C 2 3 1
D 1 3 2
E 1 2 3
F 1 3 2
G 2 3 1
H 2 1 3
I 1 2 3
J 2 1 3
Interval Scale
• Trilogy Approach
– Cognitive component (beliefs,
perceptions or knowledge)
– Affective component (emotions or
feelings)
– Conative component (intended or actual
behaviors)
Scale Development
• Process of developing reliable instrument,
which measures the desired variables
correctly and accurately
• Systematic and scientific process of
identifying the item variables, statements
and the scales for measuring them
• Ascertaining whether the instrument
correctly measures the desired variables
sufficiently and accurately
Classification of Scales
• Subject Orientation
– Designed to measure characteristics of the respondents
– Stimuli – response to develop categories (smokers/non-
smokers, etc)
• Response Form
– Categorical and comparative scales
• Degree of Subjectivity
– Personal preferences
• Scale properties
– Nominal, interval, ordinal, ratio
• Number of Dimensions
– Uni-dimensional, multi-dimensional
Scale Construction
Techniques
• Arbitrary Approach
– Scales developed on adhoc basis
– Developed on presumption
• Consensus Approach (Thurstone Differential Scale)
– Panel of judges evaluate the items chosen for inclusion in the
instrument
• Item Analysis Approach (Likert Scale)
– Individual items are tested by gropu of respondents
– Analysed on the basis of degree of discrimination
• Cumulative Scale (Guttman’s Scalogram)
– Conformity to some ranking of items in ascending or
descending order
• Factor Scales (Osgood’s Semantic Differential Scale)
– On the bass of intercorrelations to identify the commin factors
Important Scaling
Techniques
• Rating Scale
• Ranking Scale
Rating Scale
• Graphical Rating
– Points are put in a continuum
– Indicate rating by tick mark
Like Very Like Neutral Dislike Dislike Very
Much Somewhat Some What Much
Rating Scale Types
• Itemised Rating
– Presents a series of statements
– Respondent selects the test
• He is always involved in some friction with his fellow worker
• He is often at odds with one or more of his fellow workers
• He sometimes gets involved in friction
• He infrequently becomes involved in friction with others
• He almost ever gets involved in friction with his fellow workers
Ranking Scale
• Make comparative/relative
judgments
• Approaches
– Method of paired comparison
– Method of rank order
Method of Paired
Comparison
• Respondent expresses the attitude by
making choice between two objects
• Number of comparisons (N) to be made
depend upon number of objects (n)
N = n/2 (n-1) If n= 10, N = 45
• Reduce possible comparisons by sample
survey
• Paired Comparison can be converted to
interval data by the Thurstone’s Law of
Comparative Judgment & Guilford’s
composite standard method
Scale Construction
• Thurstone Scale
• Likert Scale
Differential Scale (Thurstone
Scale)
• Uses consensus approach
• Method used in measuring attitude
on single dimension
• Used to measure the issues like war,
religion, etc.
Differential Scale (Thurstone
Scale)
• Researcher gathers a large number of statements
to express a point of view
• Submitted to a panel of judges to arrange them in
11 groups ranging from one extreme to another
• Sorting by each judge yields composite position
of each item
• Items of disagreement are discarded
• Median position of items selected items is
decided
• Attitude comparison made on the basis of these
median scores
Likert Scale (Summated
Scale)
• Evaluates each item on its ability to
discriminate between those with high score and
those with low score
• Respondent indicates degree of agreement or
disagreement with the statements in the
instrument
• Each response is given a numerical score,
indicating favourableness or unfavourableness
and total score represents the attitude
Likert Scale (Summated Scale)
Procedure
• Collect large number of statements relevant
to the attitude
• Collect diverse statements which express
favourableness or unfavourableness
• Administer it to a group of respondents
• Do the coding 1 for lowest and 5/7 for the
highest
Likert Scale (Summated Scale)
Procedure
• Compute total score of each respondent
• Arrange the total scires to find out
discriminating power of each statement
• Identify top 25% and bottom 25%
statements, which express the attitudes
• Statements correlating with total score are
retained in the final instrument and rest
are discarded
Likert Scale (Summated
Scale)
• Advantages
– Easier than Thurstone Scale
– Without panel of judges
– More reliable as it considers each item statement
and respondent
• Limitations
– Just gives the difference in attitudes and does
not quantify the same
Criteria for good
measurement
• Reliability
• Validity
• Sensitivity
• Relevance
• Versatility
• Ease of response
Reliability
• Equivalence: Determined by
calculating the internal consistency
or homogeneity of the set of items
forming the scale. One way to
calculate equivalence reliability is to
use coefficient alpha.
Example
For each of the following items, circle the number that best represents
how you feel about making a sales call/presentation.
Definitely Definitely
do not Feel Feel
Jittery 1 2 3 4 5 6 7
Active 1 2 3 4 5 6 7
Intense 1 2 3 4 5 6 7
Energetic 1 2 3 4 5 6 7
Fearful 1 2 3 4 5 6 7
Vigorous 1 2 3 4 5 6 7
Lively 1 2 3 4 5 6 7
Tense 1 2 3 4 5 6 7
Steps for reliability in SPSS
• Analyse
– Scale
Reliability
– Alpha
– Statistics
Descriptives for Scale
Descriptives for Scale if item deleted
Inter-item Correlation
OK
Reliability Statistics
Cronbach Alpha Cronbach Alpha of N of Items
Standardised Items
0.706 0.708 6
• Pragmatic Validity
• How well the measure actually predicts
some other characteristic or behavior
• Predictive Validity: A measure is used
to predict something in the future
• Concurrent Validity: A measure is used
to predict something assessed at the
same point in time
Assessing Validity
• Content Validity: The adequacy with
which the domain of the characteristic is
captured by the measure. Also called face
validity.
• Construct Validity: Assessment of how
well the instrument captures the
construct, concept, or trait it is supposed
to be measuring
• Item to Total Correlation
Relationship between
Reliability and Validity
• Reliability is a necessary but not
sufficient condition for validity
• A measure may be reliable and not
valid
Sensitivity
• It is the ability of a measurement to
indicate changes or differences
• Eg : Three ad campaigns showed similar
sales. Possible reasons could be:
– The ads were similar
– The period of sales was brief and insensitive to
changes
– Sales might not be the right test to measure
the effectiveness of the ads
Relevance
. . . . a prepared set
of questions (or
measures) to which
respondents or
interviewers record
answers
Steps in Questionnaire
Design:
Step 1: Determine Specific Data to be sought
Step 2: Determine Interview Process
Step 3: Evaluate Question Content
Step 4: Determine Response Format
Step 5: Determine Wording
Step 6: Determine Questionnaire Structure
Step 7: Determine Physical Characteristics
Step 8: Pretest – Revise – Finalize Draft
Questionnaire Design –
Identify Information Needs
• Clarify the nature of the research problem and
objectives.
• Develop research questions to meet research
objectives.
• Identify Variables from Literature
• Develop Statements to measure each item
• Select the right scale
Questionnaire Design –
Clarification of Concepts:
• Opening Questions
• Research Topic Questions
• Classification Questions
Screening or Filter
Questions:
• . . . are used to ensure respondents included in
the study are those that meet the pre-
determined criteria of the target population.
Concept Identification.
• Conceptual definition – e.g., Service Quality.
As perceived by customers, it is the extent of
discrepancy between customers’ expectations
or desires and their perceptions.
Working Definition for Concept.
• Decompose definition into components.
• Search for items that are measurable.
Preparing and Presenting
Good Questions:
• Use simple words.
• Be brief.
• Avoid ambiguity.
• Avoid leading questions.
• Avoid double-barreled questions.
• Be careful about question order and
context effects.
• Check questionnaire layout.
• Prepare clear instructions.
Avoid Position Bias:
• Position Bias:
• “How important are flexible hours in evaluating job
• alternatives?”
• “What factors are important in evaluating job
• alternatives?”
• No Position Bias:
• “What factors are important in evaluating job
• alternatives?”
• “How important are flexible hours in evaluating job
• alternatives?”
Double-Barreled Questions: