References For Instrument Development

Questionnaire Design for Surveys and Laboratory Experiments:
Social And Cognitive Perspectives
Communication 239, Political Science 324R, Psychology 224
Jon A. Krosnick
Departments of Communication, Political Science, and Psychology
Stanford University
434 McClatchy Hall, 725-3031, Krosnick@stanford.edu
Spring, 2005
Building 50, Room 52H
Tuesdays, 3:15 – 6:05 pm
_______________________________________________________________________
This course will provide an introduction to general theories of the cognitive aspects of
question-answering in surveys. The focus will be on the cognitive processes in which
respondents engage when answering questions and on the social interactions among researchers,
interviewers, and respondents. The course will provide an overview of issues that one should
consider when designing a survey questionnaire or when interpreting the results of a survey.
A great deal of research has been conducted during the last 50 years exploring the effects
of different measurement strategies on the findings of survey research, and this work reveals a
great deal about the cognitive processes in which survey respondents engage during interviews.
We will review the major findings of this work and discuss their implications for understanding
cognitive aspects of question-answering. The primary goal of the discussions will be to identify
issues to be considered when writing survey questions to measure specified constructs. Taken
together, this literature makes many useful recommendations regarding how to avoid sources of
bias in measurement and how to maximize accuracy.
Most of our discussions will focus on the measurement of subjective psychological

phenomena, such as attitudes, beliefs, assessments of importance, and assessments of
satisfaction. There will be some secondary discussion of the measurement of objective
phenomena, primarily past behavior.
Course requirements are: (1) reading all required reading assignments prior to class
meetings when they will be discussed, (2) participating energetically and creatively in class
discussion, and (3) a final sit-down exam asking you to critique a questionnaire that I will give
you.
Topics to be Covered
1. General Introduction
2. Open versus Closed Questions
3. Closed Questions: Ratings versus Rankings
4. Closed Questions: Number of Scale Points/Magnitude Scaling
5. Closed Questions: Verbal vs. Numeric Scale Point Labels
6. Closed Questions: Response Order Effects
7. Attitudes and Non-attitudes
8. Acquiescence and Other Response Biases
9. Question Wording/Question Balance
10. Question Order
11. Attitude Recall Questions
12. Asking Why?

1
Readings to be Reviewed
Note: Readings marked with "*" are required. Readings marked with "+" are optional but
recommended. All other readings are fully optional.
General Readings on Questionnaire Design
Bradburn, N. M. (1983). Response effects. Pp. 289-328 in P. H. Rossi, J. D. Wright, &

A. B. Anderson (Eds.), Handbook of survey research. New York: Academic
Press.
Bradburn, N. M., & Sudman, S. Improving interview method and questionnaire design.
San Francisco: Jossey-Bass.
Cantril, H. (1944). Gauging public opinion. Princeton, N.J.: Princeton University Press.
+Converse, J. M., & Presser, S. (1986). Survey questions: Handcrafting the standardized
questionnaire. Beverly Hills, CA: Sage.
Kahn, R. L., & Cannell, C. F. (1957). The dynamics of interviewing (Chapters 5 and 6,
"The formulation of questions" and "The design of questionnaires"). New York:
Wiley.
Oppenheim, A. N. (1966). Questionnaire design and attitude measurement. New York:

Basic Books.
Payne, S. L. (1951). The art of asking questions. Princeton, N.J.: Princeton University
Press.
Sheatsley, P. B. (1983). Questionnaire construction and item writing. Pp. 195-230 in P.

H. Rossi, J. D. Wright, & A. B. Anderson (Eds.), Handbook of survey research.
New York: Academic Press.
Smith, T. W. (1987). The art of asking questions, 1936-1985. Public Opinion Quarterly,
51, S95-S108.
Sudman, S., & Bradburn, N. M. (1974). Response effects in surveys. Chicago: Aldine.
Sudman, S., & Bradburn, N. M. (1983). Asking questions. San Francisco: Jossey-Bass.
+Turner, C. F., & Martin, E. (1984). Surveying subjective phenomena (vol. 1 and 2).
New York: Russell Sage Foundation.
2
Warwick, D. P., & Lininger, C. A. (1975). The sample survey: Theory and practice.
(Chapter 6, "Questionnaire design."). New York: McGraw-Hill.
1. Introduction to Issues of Question Design
Andrews, F. M. (1984). Construct validity and error components of survey measures: A

structural modeling approach. Public Opinion Quarterly, 48, 409-442.
*Hippler, H., & Schwarz, N. (1987). Response effects in surveys. In H. Hippler, N.

Schwarz, & S. Sudman (Eds.), Social information processing and survey
methodology. New York: Springer-Verlag.
Kalton, G., & Schuman, H. (1982). The effect of the question on survey responses: A
review. The Journal of the Royal Statistical Society, 145, 42-73.
*Krosnick, J. A. (1991). Response strategies for coping with the cognitive demands of
attitude measures in surveys. Applied Cognitive Psychology, 5, 213-236.
Schuman, H. (1982). Artifacts are in the mind of the beholder. The American
Sociologist, 17, 21-28.
Schuman, H. (1986). Ordinary questions, survey questions, and policy questions. Public
Opinion Quarterly, 50, 432-442.
Schuman, H., & Kalton, G. (1985). Survey methods. In G. Lindzey and E. Aronson
(Eds.), Handbook of social psychology (vol. 1). New York: Random House.
Schuman, H., & Presser, S. (1977). Question wording as an independent variable in

survey analysis. Sociological Methods and Research, 6, 27-46.
*Tourangeau, R. (1987). Attitude measurement: A cognitive perspective. In H. Hippler,

N. Schwarz, & S. Sudman (Eds.), Social information processing and survey
methodology. New York: Springer-Verlag.
2. Open vs. Closed Questions
Belson, W. A., & Duncan, J. A. (1962). A comparison of the checklist and the open
response questioning systems. Applied Statistics, 11, 120-132.
*Converse, J. M. (1984). Strong arguments and weak evidence: The open/closed

questioning controversy of the 1940s. Public Opinion Quarterly, 48, 267-282.
Ehrlich, H. J., & Rinehart, J. W. (1965). A brief report on the methodology of stereotype
research. Social Forces, 43, 564-575.
3
Elig, T. W., & Frieze, I. H. (1979). Measuring causal attributions for success and
failure. Journal of Personality and Social Psychology, 37, 621-634.
Geer, J. G. (1988). What do open-ended questions measure? Public Opinion Quarterly,

52, 365-371.
Hurd, A. W. (1932). Comparisons of short answer and multiple choice tests covering
identical subject content. Journal of Educational Psychology, 26, 28-30.
Jenkins, J. G. (1935). Psychology in Business and Industry. New York: Wiley, p. 348-
351.
Lazarsfeld, P. E. (1944). The controversy over detailed interviews - an offer for

negotiation. Public Opinion Quarterly, 8, 38-60.
Lindzey, G. E., & Guest, L. (1951). To repeat - check lists can be dangerous. Public
Link, H. C. (1943). An experiment in depth interviewing. Public Opinion Quarterly, 7,

267-279.
Magill, W. H. (1934). The influence of the form of the item on the validity of
achievement tests. Journal of Educational Psychology, 25, 21-28.
Malmud, R. S. (1925). The controlled vs. the free completion. American Journal of
Psychology, 36, 401-411.
Mason, R., Boersma, L., & Faulkenberry, G. D. (1988). The use of open and closed
questions to identify holders of crystallized attitudes: The case of adoption of
erosion-control practices among farmers. Rural Sociology, 53, 96-109.
Metzner, H., & Mann, F. (1952). A limited comparison of two methods of data
collection: The fixed alternative questionnaire and the open-ended interview.
American Sociological Review, 17, 486-491.
Remmers, H. H., Marschat, L. E., Brown, A., & Chapman, I. (1923). An experimental
study of the relative difficulty of true-false, multiple-choice, and incomplete-
sentence types of examination questions. Journal of Educational Psychology, 14,
367-372.
Roslow, S., & Blankenship, A. B. (1939). Phrasing the question in consumer research:
Experimental studies on the form of question. Journal of Applied Psychology, 23,
612-622.
4
Roslow, S., Wulfeck, W. H., & Corby, P. G. (1940). Consumer and opinion research:
Experimental studies on the form of the question. Journal of Applied Psychology,
24, 334-346.
Rugg, D., & Cantril, H. (1944). The wording of questions. In H. Cantril (Ed.), Gauging
public opinion. Princeton: Princeton University Press, p. 37-38, 49.
Schuman, H., Ludwig, J., & Krosnick, J.A. (1986). The perceived threat of nuclear war,
salience, and open questions. Public Opinion Quarterly, 50, 519-536.
*Schuman, H., & Presser, S. (1981). Questions and answers in attitude surveys. New
York: Academic Press. Chapter 3, "Open versus closed questions."
Schuman, H., & Scott, J. (1987). Problems in the use of survey questions to measure
public opinion. Science, 236, 957-959.
Skott, H. E. (1943). Attitude research in the Department of Agriculture. Public Opinion

Quarterly, 7, 280-292.
3. Closed Questions: Ratings vs. Rankings
*Alwin, D. F., & Krosnick, J. A. (1985). The measurement of values in surveys: A

comparison of ratings and rankings. Public Opinion Quarterly, 49, 535-552.
Elig, T. W., & Frieze, I. H. (1979). Measuring causal attributions for success and
failure. Journal of Personality and Social Psychology, 37, 621-634.
Feather, N. T. (1973). The measurement of values: Effects of different assessment

procedures. Australian Journal of Psychology, 25, 221-231.
Guilford, J. P. (1954). Psychometric methods. New York: McGraw-Hill.
*Herzog, A. R., & Bachman, J. G. (1981). Effects of questionnaire length on response

quality. Public Opinion Quarterly, 45, 549-559.
*Krosnick, J. A., & Alwin, D. F. (1987). A test of the form-resistant correlation

hypothesis: Ratings, rankings, and the measurement of values. Public Opinion
Quarterly, 52, 526-538.
McIntyre, S. H., & Ryans, A. B. (1977). Time and accuracy measures for alternative
multidimensional scaling data collection methods: Some additional results.
Journal of Marketing Research, 14, 607-610.
Moore, M. (1975). Rating versus ranking in the Rokeach value survey: An Israeli
comparison. European Journal of Social Psychology, 5, 405-408.
5
Munson, J. M., & McIntyre, S. H. (1979). Developing practical procedures for the
measurement of personal values in cross-cultural marketing. Journal of
Marketing Research, 16, 48-52.
Rankin, W. L., & Grube, J. W. (1980). A comparison of ranking and rating procedures
for value system measurement. European Journal of Social Psychology, 10, 233-
246.
Reynolds, T. J., & Jolly, J. P. (1980). Measuring personal values: An evaluation of

alternative methods. Journal of Marketing Research, 17, 531-536.
Schulz, U., & May, T. (1989). The recording of social orientations with ranking and pair
comparison procedures. European Journal of Social Psychology, 19, 41-59.
4. Closed Questions: Number of Scale Points

Bass, B. M., Cascio, W. F., & O'Connor, E. J. (1974). Magnitude estimations of

expressions of frequency and amount. Journal of Applied Psychology, 59, 313-
320.
Bendig, A. W. (1953). The reliability of self-ratings as a function of amount of verbal

anchoring and of the number of categories on the scale. Journal of Applied
Bendig, A. W. (1954). Transmitted information and the length of rating scales. Journal
of Experimental Psychology, 47, 303-308.
Bendig, A. W., & Hughes, J. B. (1953). Effect of amount of verbal anchoring and
number of rating-scale categories upon transmitted information. Journal of
Experimental Psychology, 46, 87-90.
Birkett, N. J. (1986). Selecting the number of response categories for a Likert-type

scale. Proceedings of the American Statistical Association 1987 Annual
Meetings, Section on Survey Research Methods.
Bishop, G. F. (1987). Experiments with the middle response alternative in survey

questions. Public Opinion Quarterly, 51, 220-232.
Champney, H., & Marshall, H. (1939). Optimal refinement of the rating scale. Journal
of Applied Psychology, 23, 323-331.
6
Cox, E. P. (1980). The optimal number of response alternatives for a scale: A review.
Eagly, A. H., & Steffen, V. J. (1988). A note on assessing stereotypes. Personality and
Social Psychology Bulletin, 14, 676-680.
Finn, R. H. (1972). Effects of some variations in rating scale characteristics on the

means and reliabilities of ratings. Educational and Psychological Measurement,
32, 255-265.
Garner, W. R. (1960). Rating scales, discriminability, and information transmission.

Psychological Review, 67, 343-352.
Ghiselli, E. E. (1939). All or none versus graded response questionnaires. Journal of

Applied Psychology, 23, 405-413.
Green, P. E., & Rao, V. R. (1970). Rating scales and information recovery - how many
scales and response categories to use? Journal of Marketing, 34, 33-39.
Kalton, G., Roberts, J., & Holt, D. (1980). The effects of offering a middle response
option with opinion questions. Statistician, 29, 65-78.
Komorita, S. S., & Graham, W. K. (1965). Number of scale points and the reliability of
scales. Educational and Psychological Measurement, 25, 897-995.
Lehmann, D. R., & Hulbert, J. (1972). Are three-point scales always good enough?
Lipset, S. M., & Schneider, W. (1983). The confidence gap. New York: The Free Press,
p. 89-96.
Lissitz, R. W., & Green, S. B. (1975). Effect of the number of scale points on reliability:
A monte carlo approach. Journal of Applied Psychology, 60, 10-13
+Lodge, M. (1981). Magnitude scaling: Quantitative measurement of opinions. Beverly

Hills: Sage.
Lodge, M., & Tursky, B. (1979). Comparisons between category and magnitude scaling
of political opinion employing SRC/CPS items. American Political Science
Review, 73, 50-66.
Lodge, M., & Tursky, B. (1981) The social-psychophysical scaling of political opinion.
In B. Wegener (Ed.), Social attitudes and psychophysical measurement.
Hillsdale, N.J.: Erlbaum.
7
Lodge, M., Cross, D., Tursky, B., & Tannenhaus, J. (1975). The psychophysical scaling
and validation of a political support scale. American Journal of Political Science,
19, 611-649.
Lodge, M., Cross, D., Tursky, B., Foley, M. A., & Foley, H. (1976). The calibration and
cross-model validation of ratio scales of political opinion in survey research.
Social Science Research, 5, 325-347.
Lodge, M., Cross, D., Tursky, B., Tannenhaus, J., & Reeder, R. (1976). The
psychophysical scaling of political support in the 'real world'. Political
Methodology, 2, 159-182.
Matell, M. S., & Jacoby, J. (1971). Is there an optimal number of alternatives for Likert
scale items? Study I: Reliability and validity. Educational and Psychological
Measurement, 31, 657-674.
Matell, M. S., & Jacoby, J. (1972). Is there an optimal number of alternatives for Likert-
scale items? Effects of testing time and scale properties. Journal of Applied
McGuire, W. J. (1981). The probabilogical model of cognitive structure and attitude

change. In R. E. Petty, T. M. Ostrom, and T. C. Brock (Eds.), Cognitive
responses in persuasion. Hillsdale, NJ: Erlbaum.
Peterson, B. L. (1985). Confidence: Categories and confusion. GSS Technical Report

No. 50, National Opinion Research Center, University of Chicago.
Ramsay, J. O. (1973). The effect of number of categories in rating scales on precision of

estimation of scale values. Psychometrika, 38, 513-532.
Rosenstone, S. J., Hansen, J. M., & Kinder, D. R. (1986). Measuring change in personal
economic well-being. Public Opinion Quarterly, 50, 176-192.
York: Academic Press. Chapters 6, "Measuring a middle position."
Smith, T. W., & Peterson, B. L. (1985). The impact of number of response categories on
inter-item associations: Experimental and simulated results. Paper presented at
the American Sociological Association annual meetings, Washington, D.C.
Stember, H., & Hyman, H. (1949-1950). How interviewer effects operate through
question form. International Journal of Opinion and Attitude Research, 3, 493-
512.
Symonds, P. M. (1924). On the loss of reliability in rating due to coarseness of the scale.
Journal of Experimental Psychology, 456-461.
8
Warr, P., Barter, J., & Brownridge, G. (1983). On the independence of positive and
negative affect. Journal of Personality and Social Psychology, 44, 644-651.
Watson, D. (1988). The vicissitudes of mood measurement: Effects of varying

descriptors, time frames, and response formats on measures of positive and
*Wedell, D. H., & Parducci, A. (1988). The category effect in social judgment:
Experimental ratings of happiness. Journal of personality and Social Psychology,
55, 341-356.
Welch, R. E. (1972). The use of magnitude estimation in attitude scaling: Constructing a

measure of political dissatisfaction. Social Science Quarterly, 76-87.
5. Closed Questions: Verbal vs. Numerical Scale Point Labels
Albaum, G., & Murphy, B. D. (1988). Extreme response on a Likert scale.

Psychological Reports, 63, 501-502.

Bass, B. M., Cascio, W. F., & O'Connor, E. J. (1974). Magnitude estimations of

expressions of frequency and amount. Journal of Applied Psychology, 59, 313-
320.
Bendig, A. W. (1953). The reliability of self-ratings as a function of amount of verbal

anchoring and of the number of categories on the scale. Journal of Applied
Bendig, A. W., & Hughes, J. B. (1953). Effect of amount of verbal anchoring and
number of rating-scale categories upon transmitted information. Journal of
Experimental Psychology, 46, 87-90.
*Bradburn, N. M., & Miles, C. (1979). Vague qualifiers. Public Opinion Quarterly, 43,
92-101.
Bryant, G. D., & Norman, G. R. (19??). Expressions of probability: Words and

numbers. New England Journal of Medicine, 302, 411.
Budescu, D. V., & Wallsten, T. S. (1985). Consistency in interpretation of probabilistic

phrases. Organizational Behavior and Human Decision Making, 36, 391-405.
Chase, C. I. (1969). Often is where you find it. American Psychologist, 24, 1043.
9
Cliff, N. (1959). Adverbs as multipliers. Psychological Review, 66, 27-44.
Cohen, J. Dearnley, E. J., & Hansel, C. E. M. (1958). A quantitative study of meaning.

British Journal of Educational Psychology, 28, 141-148.
Eagly, A. H., & Steffen, V. J. (1988). A note on assessing stereotypes. Personality and
Finn, R. H. (1972). Effects of some variations in rating scale characteristics on the

means and reliabilities of ratings. Educational and Psychological Measurement,
32, 255-265.
Goocher, B. E. (1965). Effects of attitude and experience on the selection of frequency

adverbs. Journal of Verbal Learning and Verbal Behavior, 4, 193-195.
Hakel, M. D. (1968). How often is often? American Psychologist, 23, 533-534.
Koltko, M. E. (1989). How vagueness can ruin a survey: Comment on Pope,

Tabachnick, & Keith-Spiegel. American Psychologist, 44, 845-846.
Lichtenstein, S., & Newman, J. R. (1967). Empirical scaling of common verbal phrases
associated with numerical probabilities. Bulletin of the Psychonomic Society, 9,
563-564.
Madden, J. M., & Bourdon, R. D. (1964). Effects of variations in scale format on

judgment. Journal of Applied Psychology, 48, 147-151.
Miller, P. V. (1984). Alternative question forms for attitude scale questions in telephone
surveys. Public Opinion Quarterly, 48, 766-778.
Moore, P. G. (1977). The manager's struggles with uncertainty. Journal of the Royal
Statistical Association, 140, 129-165.
Murphy, K. R., & Constans, J. I. (1987). Behavioral anchors as a source of bias in

ratings. Journal of Applied Psychology, 72, 573-577.
Ostrom, T. M., & Upshaw, H. S. (1968). Psychological perspective and attitude change.
In A. G. Greenwald, T. C. Brock, and T. M. Ostrom (Eds.), Psychological
Foundations of Attitudes. New York: Academic Press.
Parducci, A. (1968). Often is often. American Psychologist, 23, 828.
Pepper, S., & Prytulak, L. S. (1974). Sometimes frequently means seldom: Context
effects in the interpretation of quantitative expressions. Journal of Research in
Personality, 8, 95-101.
10
Peters, D. L., & McCormick, E. J. (1966). Comparative reliability of numerically

anchored versus job-task anchored rating scales. Journal of Applied Psychology,
50, 92-96.
Pope, K. S., Tabachnick, B. G., & Keith-Spiegel, P. (1989). Reply to Koltko. American
Psychologist, 44, 846-847.
Reyna, V. F. (1981). The language of possibility and probability: Effects of negation on

meaning. Memory and Cognition, 9, 642-650..
24, 334-346.
Schwarz, N., & Scheuring, B. (1988). Judgments of relationship satisfaction: Inter- and
intraindividual comparisons as a function of questionnaire structure. European
Journal of Social Psychology, 18, 485-496.
Simpson, R. H. (1944). The specific meanings of certain terms indicating different

degrees of frequency. Quarterly Journal of Speech, 30, 328-330.
Stone, D. R., & Johnson, R. T. (1959). A study of words indicating frequency. Journal
of Educational Psychology, 50, 224-227.
Teigen, K. H. (1988). When are low-probability events judged to be 'probable?' Effects

of outcome-set characteristics on verbal probability estimates. Acta
Psychologica, 68, 157-174.
*Wallsten, T. S., Budescu, D. V., Rapoport, A., Zwick, R., & Forsyth, B. (1986).
Measuring the vague meanings of probability terms. Journal of Experimental
Psychology: General, 115, 348-365.
Warr, P., Barter, J., & Brownridge, G. (1983). On the independence of positive and
Watson, D. (1988). The vicissitudes of mood measurement: Effects of varying

descriptors, time frames, and response formats on measures of positive and
Wedell, D. H., & Parducci, A. (1988). The category effect in social judgment:
Experimental ratings of happiness. Journal of personality and Social Psychology,
55, 341-356.
Weisberg, H. F., & Miller, A. H. (1980). Evaluation of the feeling thermometers: A

report to the National Election Study Board based on data from the 1979 pilot
11
study. Unpublished manuscript, Center for Political Studies, Institute for Social
Research, University of Michigan.
Zaller, J. (1988). Vague questions get vague answers: An experimental attempt to

reduce response instability. Unpublished manuscript, University of California at
Los Angeles.
6. Closed Questions: Acquiescence and Other Response Biases
Bachman, J. G., & O'Malley, P. M. (1984). Black-white differences in self-esteem: Are

they affected by response styles? American Journal of Sociology, 90, 624-639.
Bachman, J. G., & O'Malley, P. M. (1984). Yea-saying, nay-saying, and going to

extremes: Black-white differences in response styles. Public Opinion Quarterly,
48, 491-509.
Bass, B. M. (1955). Authoritarianism or acquiescence? Journal of Abnormal and Social

Bentler, P. M., Jackson, D. N., & Messick, S. (1971). Identification of content and style:
A two-dimensional interpretation of acquiescence. Psychological Bulletin, 76,
186-204.
Bentler, P. M., Jackson, D. N., & Messick, S. (1972). A rose by any other name.
Psychological Bulletin, 77, 109-113.
Bishop, G. F., Oldendick, R. W., & Tuchfarber, A. J. (1982). Effects of presenting one
versus two sides of an issue in survey questions. Public Opinion Quarterly, 46,
69-85.
69-85.
Blankenship, A. B. (1940). The influence of the question form upon the response in a
public opinion poll. Psychological Record, 3, 345-422.
Block, J. (1965). The challenge of response sets. New York: Appleton-Century-Crofts.
Block, J. (1971). On further conjectures regarding acquiescence. Psychological

Bulletin, 76, 205-210.
Campbell, A., Converse, P. E., & Rogers, W. (1976). The quality of American life:
Perceptions, evaluations, and satisfaction. New York: Russell Sage Foundation.
Chapter 4.
12
Campbell, A., Converse, P. E., Miller, W. E., & Stokes, D. E. (1960). The American
Voter. New York: Wiley. Chapter 18.
Clancy, K. J., & Wachsler, R. A. (1971). Positional effects in shared-cost surveys.

Public Opinion Quarterly, 35, 258-265.
Couch, A., & Keniston, K. (1960). Yeasayers and naysayers: Agreeing response set as a
personality variable. Journal of Abnormal and Social Psychology, 60, 151-174.
Couch, A., & Keniston, K. (1961). Agreeing response set and social desirability.
Journal of Abnormal and Social Psychology, 62, 175-179.
Crandall, J. E. (1965). Some relationship among sex, anxiety, and conservatism of

judgment. Journal of Personality, 33, 99-107.
Cronbach, L. J. (1941). An experimental comparison of the multiple true-false and

multiple multiple-choice tests. Journal of Educational Psychology, 32, 533-543.
Cronbach, L. J. (1942). Studies of acquiescence as a factor in the true-false test. Journal

of Educational Psychology, 33, 401-415.
Cronbach, L. J. (1946). Response sets and test validity. Educational and Psychological
Measurement, 6, 475-494.
Cronbach, L. J. (1950). Further evidence on response sets and test design. Educational
and Psychological Measurement, 10, 3-31.
Foster, R. J. (1961). Acquiescent response set as a measure of acquiescence. Journal of

Abnormal and Social Psychology, 63, 155-160.
Gage, N. L., & Chatterjee, B. B. (1960). The psychological meaning of acquiescence

set: Further evidence. Journal of Abnormal and Social Psychology, 60, 280-283.
Goldsmith, R. E., Stith, M. T., & White, J. D. (1987). The Rokeach value survey and
social desirability. Journal of Social Psychology, 127, 553-554.
Goldsmith, R. E., White, J. D., & Stith, M. T. (1987). Yeasaying and the Rokeach value
survey: Interactions with age and race. Journal of Social Behavior and
Hamilton, D. L. (1968). Personality attributes associated with extreme response style.

Hedges, B. M. (1979). Question wording effects: Presenting one or both sides of the
case. Statistician, 28, 83-99.
13
Hitlin, R. (1976). On question wording and stability of response. Social Science

Research, 5, 39-41.
Hui, C. H., & Triandis, H. C. (1985). The instability of response sets. Public Opinion
Quarterly, 49, 253-260.
Jackman, M. R. (1973). Education and prejudice or education and response-set?

Jackson, D. N., & Messick, S. J. (1957). A note on ethnocentrism and acquiescence

response sets. Journal of Abnormal and Social Psychology, 54, 132-134.
Jackson, D. N., & Messick, S. J. (1958). Content and style in personality assessment.
Jackson, J. E. (1979). Bias in closed-ended issue questions. Political Methodology, 6,

393-424.
Lenski, G. E., & Leggett, J. C. (1960). Caste, class, and deference in the research
interview. American Journal of Sociology, 65, 463-467.
Lentz, T. F. (1938). Acquiescence as a factor in the measurement of personality.

Psychological Bulletin, 35, 659.
Lewis, N. A., & Taylor, J. A. (1955). Anxiety and extreme response preferences.
Educational and Psychological Measurement, 15, 111-116.
Martin, J. (1964). Acquiescence: Measurement and theory. British Journal of Social and
Clinical Psychology, 3, 216-225.
McClendon, M. J. (1989). Acquiescence: A test of the cognitive limitations and question

ambiguity hypotheses. Paper presented at the 1989 meetings of the American
Association for Public Opinion Research, St. Petersburg, Florida.
McClendon, M. J. (1991). Acquiescence and recency response-order effects in interview

surveys. Sociological Methods and Research.
Moum, T. (1988). Yea-saying and mood-of-the-day effects in self-reported quality of

life. Social Indicators Research, 20, 117-139.
O'Grady, K. E. (1988). The Marlowe-Crowne and Edwards social desirability scales: A

psychometric perspective. Multivariate Behavioral Research, 23, 87-101.
14
Peabody, D. (1961). Attitude content and agreement set in scales of authoritarianism,

dogmatism, anti-Semitism, and economic conservatism. Journal of Abnormal and
Social Psychology, 63, 1-11.
Peabody, D. (1964). Models for estimating content and set components in attitude and
personality scales. Educational and Psychological Measurement, 24, 255-269.
Peabody, D. (1966). Authoritarianism scales and response bias. Psychological Bulletin,

65, 11-23.
Phillips, D. L. (1973). Abandoning method. San Francisco: Jossey-Bass. (Chapters 2

and 3).
Raskin, E., & Cook, S. W. (1938). A further investigation of the measurement of an

attitude toward fascism. Journal of Social Psychology, 9, 201-206.
Ray, J. J. (1979). Is the acquiescent response style not so mythical after all? Some
results from a successful balanced F scale. Journal of Personality Assessment, 43,
638-643.
Ray, J. J. (1983). Reviving the problem of acquiescent response bias. Journal of Social
Ray, J. J., & Pratt, G. J. (1979). Is the influence of acquiescence on "Catchphrase" type
attitudes scale items not so mythical after all? Australian Journal of Psychology,
31, 73-78.
*Rorer, L. G. (1965). The great response-style myth. Psychological Bulletin, 63, 129-
156.
24, 334-346.
Schuman, H., & Duncan, O. D. (1974). Questions about attitude survey questions. In H.
Costner (Ed.), Sociological Methodology 1973-1974. San Francisco: Jossey-
Bass.
Schuman, H., & Presser, S. (1981). Questions and answers in attitude surveys. New
York: Academic Press. Chapter 7, "Balance and imbalance in questions."
York: Academic Press. Chapter 8, "The acquiescence quagmire."
Schuman, H., & Scott, J. (1989). Response effects over time: Two experiments.
Sociological Methods and Research, 17, 398-408.
15
Small, D. O., & Campbell, D. T. (1960). The effect of acquiescence response set upon
the relationship of the F-scale and conformity. Sociometry, 23, 69-71.
Stagner, R. (1941). A comparison of the Gallup and Fortune polls regarding American
intervention policy. Sociometry, 4, 239-258. (p. 247)
Toner, B. (1987). The impact of agreement bias on the ranking of questionnaire

response. Journal of Social Psychology, 127, 221-222.
Wright, J. D. (1975). Does acquiescence bias the 'index of political efficacy?' Public
7. Closed Questions: Response Order Effects
Becker, S. L. (1954). Why an order effect. Public Opinion Quarterly, 18, 271-278.
Belson, W. A. (1966). The effects of reversing the presentation order of verbal rating
scales. Journal of Advertising Research, 6, 30-37.
Bishop, G. F. (1987). Experiments with the middle response alternative in survey

questions. Public Opinion Quarterly, 51, 220-232.
Brook, D., & Upton, G. J. G. (1974). Biases in local government elections due to
position on the ballot paper. Applied Statistics, 23, 414-419.
Campbell, D. T., & Mohr, P. J. (1950). The effect of ordinal position upon responses to
items in a check list. Journal of Applied Psychology, 34, 62-67.
Carp, F. M. (1974). Position effects on interview responses. Journal of Gerontology, 29,

581-587.
Cochrane, R., & Rokeach, M. (1970). Rokeach's value survey: A methodological note.
Journal of Experimental Research in Personality, 4, 159-161.
Coney, K. A. (1977). Order-bias: The special case of letter preference. Public Opinion
Quarterly, 41, 385-388.
Coney, K. A. (1979). Reply. Public Opinion Quarterly, 43, 408-409.
Greenhalgh, C. (1979). On letter preference order bias. Public Opinion Quarterly, 43,
407-408.
Johnson, J. D. (1981). Effects of the order of presentation of evaluative dimensions for

bipolar scales in four societies. Journal of Social Psychology, 113, 21-27.
16
*Krosnick, J. A., & Alwin, D. F. (1987). An evaluation of a cognitive theory of

response order effects in survey measurement. Public Opinion Quarterly, 51,
201-219.
Mathews, C. O. (1929). The effect of the order of printed response words on an interest
questionnaire. Journal of Educational Psychology, 20, 128-134.
McClendon, M. J. (1986). Response-order effects for dichotomous questions. Social

Science Quarterly, 67, 205-211.
*McClendon, M. J. (1991). Acquiescence and recency response-order effects in

interview surveys. Sociological Methods and Research, 20, 60-103.
Mingay, D. J., & Greenwell, M. T. (1989) Memory bias and response-order effects.
Journal of Official Statistics, 5, 253-263.
Monsees, M. L., & Massey, J. T. (1979). Adapting procedures for collecting

demographic data in a personal interview to a television interview. Proceedings
of the American Statistical Association 1979 Annual Meetings.
Morgan, R. (1948). A note on question wording. Public Opinion Quarterly, 12, 38.
Mueller, J. E. (1970). Choosing among 133 candidates. Public Opinion Quarterly, 34,
395-402.
Payne, J. D. (1971). The effects of reversing the order of verbal rating scales in a postal
survey. Journal of the Marketing Research Society, 14, 30-44.
Payne, S. L. (1949). Case study in question complexity. Public Opinion Quarterly, 13,
653-658.
Payne, S. L. (1951). The art of asking questions. Princeton: Princeton University Press.
Perreault, W. D. (1975). Controlling order-effect bias. Public Opinion Quarterly, 39,

544-551.
Powers, E. A., Morrow, P., Goudy, W. J., & Keith, P. M. (1977). Serial order preference
in survey research. Public Opinion Quarterly, 41, 80-85.
Quinn, S. B., & Belson, W. A. (1969). The effects of reversing the order of presentation
of verbal rating scales in survey interviews. London: Survey Research Centre.
Rugg, D., & Cantril, H. (1944). The wording of questions. In H. Cantril (Ed.), Gauging
public opinion. Princeton: Princeton University Press.
17
York: Academic Press. Chapters 2, "Question order and response order," p. 56-74
only.
Upton, G. J. G., & Brook, D. (1974). The importance of positional voting bias in British
elections. Political Studies, 22, 178-190.
Upton, G. J. G., & Brook, D. (1975). The determination of the optimum position on a
ballot paper. Applied Statistics, 24, 279-287.
8. Attitudes and Non-Attitudes

Bishop, G. F., Oldendick, R. W., & Tuchfarber, A. J. (1980). Experiments in filtering

political opinions. Political Behavior, 2, 339-369.
69-85.
Bishop, G. F., Oldendick, R. W., & Tuchfarber, A. J. (1983). Effects of filter questions
in public opinion surveys. Public Opinion Quarterly, 47, 528-546.
Bishop, G. F., Oldendick, R. W., Tuchfarber, A. J., & Bennett, S. E. (1980). Pseudo-
Opinions on public affairs. Public Opinion Quarterly, 44, 198-209.
Bishop, G. F., Tuchfarber, A. J., & Oldendick, R. W. (1986). Opinions on fictitious

issues: The pressure to answer survey questions. Public Opinion Quarterly, 50,
240-250.
Bishop, G. F., Tuchfarber, A. J., & Oldendick, R. W., & Bennett, S. E. (1979). Effects
of opinion filtering and opinion floating: Evidence from a secondary analysis.
Political Methodology, 6, 293-309.
Budd, R. J. (1987). Response bias and the theory of reasoned action. Social Cognition,
5, 95-107.
*Converse, P. E. (1964). The nature of belief systems in mass publics. In D. E. Apter

(Ed.), Ideology and discontent. New York: Free Press.
18
Converse, P. E. (1970). Attitudes and non-attitudes: Continuation of a dialogue. In E.

R. Tufte (Ed.), The quantitative analysis of social problems. Reading, MA:
Addison-Wesley.
Edwards, J. D., & Ostrom, T. M. (1971). Cognitive structure of neutral attitudes.

Journal of Experimental Social Psychology, 7, 36-47.
Ehrlich, H. J. (1964). Instrument error and the study of prejudice. Social Forces, 43,
197-206.
Eysenck, H. J., & Crown, S. (1948). National stereotypes: An experimental and

methodological study. International Journal of Opinion and Attitude Research, 2,
26-39.
Gill, S. (1947). How do you stand on sin? Tide, 21, 72.
Kolson, K. L., & Green, J. J. (1970). Response set bias and political socialization
research. Social Science Quarterly, 51, 527-538.
Mason, R., Boersma, L., & Faulkenberry, G. D. (1988). The use of open and closed
questions to identify holders of crystallized attitudes: The case of adoption of
erosion-control practices among farmers. Rural Sociology, 53, 96-109.
York: Academic Press. Chapters 4 and 5, "The assessment of no opinion," and
"The fine line between attitudes and non-attitudes."
Schuman, H., & Scott, J. (1989). Response effects over time: Two experiments.
Sociological Methods and Research, 17, 398-408.
Smith, T. W. (1984). Nonattitudes: A review and evaluation. In C. F. Turner & E.

Martin (Eds.), Surveying subjective phenomena. New York: Russell Sage
Foundation.
9. Question Wording
Aldrich, J. H., Niemi, R. G., Rabinowitz, G., & Rohde, D. W. (1982). The measurement
of public opinion about public policy: A report on some new issue question
formats. American Journal of Political Science, 26, 391-414.
Blankenship, A. B. (1940). Does the question form influence public opinion poll
results? Journal of Applied Psychology, 24, 27-30.
19
Chaplin, W. F., & Buckner, K. E. (1988). Self-ratings of personality: A naturalistic

comparison of normative, ipsative, and idiothetic standards. Journal of
Converse, P. E., & Pierce, R. (1985). Measuring partisanship. Political Methodology,

11, 143-166.
Fee, J. F. (1981). Symbols in survey questions: Solving the problems of multiple word
meanings. Political Methodology, 7, 71-95.
*Hippler, H. J., & Schwarz, N. (1986). Not forbidding isn't allowing: The cognitive
basis of the forbid-allow asymmetry. Public Opinion Quarterly, 50, 87-96.

Research, 5, 39-41.
Houser, J. D. (1938). Measurement of the vital products of business. Journal of

Marketing, 2, 181-189.
Kenney, P. J., & Rice, T. W. (1988). The evaporating independents: Removing the
'independent' option from the NES party identification question. Public Opinion
Quarterly, 52, 231-239.
Kolson, K. L., & Green, J. J. (1970). Response set bias and political socialization
research. Social Science Quarterly, 51, 527-538.
+Krosnick, J. A. (1989). Question wording and reports of survey results: The case of
Louis Harris and Aetna Life and Casualty. Public Opinion Quarterly, 53, 107-
113.
Langer, G. (1989). Polling on prejudice: Questionable questions. Public Opinion,

May/June, 18-19, 57.
Link, H. C., & Freiberg, A. D. (1942). The problem of validity vs. reliability in public
opinion polls. Public Opinion Quarterly, 6, 87-98.
Lurie, W. A. (1938). The measurement of prestige and prestige-suggestibility. Journal

of Social Psychology, 9, 219-225.
Menefee, S. C. (1936). The effect of stereotyped words on political judgments.

Morgan, R. (1948). A note on question wording. Public Opinion Quarterly, 12, 38.
20
Mullen, B., Driskell, J. E., & Smith, C. (1989). Availability and social projection: The
effects of sequence of measurement and wording of question on estimates of
consensus. Personality and Social Psychology Bulletin, 15, 84-90.
Payne, S. L. (1951). The art of asking questions. Princeton, N.J.: Princeton University
Press.
*Petty, R. E., Rennier, G. A., & Cacioppo, J. T. (1987). Assertion versus interrogation
format in opinion surveys: Questions enhance thoughtful responding. Public
Quattrone, G. A., & Tversky, A. (1988). Contrasting rational and psychological analyses
of political choice. American Political Science Review, 82, 719-736.
24, 334-346.
Rossi, P. H., & Wright, J. D. (19??). Social science research and the politics of gun
control. In R. L. Shotland and M. M. Mark (Eds.), Social science and social
policy. Beverly Hills, CA: Sage.
Rugg, D. (1941). Experiments in wording questions: II. Public Opinion Quarterly, 5, 91-
92.
Schaffer, N. C. (1982). A General Social Survey experiment in generic words. Public

Schuman, H. (1988). Rejoinder. Public Opinion Quarterly, 52, 579-581.
Schuman, H., & Bobo, L. (1988). Survey-based experiments on white racial attitudes
toward residential integration. American Journal of Sociology, 94, 273-299.
Schuman, H., & Duncan, O. D. (1974). Questions about attitude survey questions. In H.
Costner (Ed.), Sociological Methodology 1973-1974. San Francisco: Jossey-
Bass.
York: Academic Press. Chapters 11, "Tone of wording."
Sigelman, L., & Presser, S. (1988). Measuring public support for the new Christian
right: The perils of point estimation. Public Opinion Quarterly, 52, 325-337.
Simpson, J. H. (1988). A reply to 'Measuring public support for the new Christian right:
The perils of point estimation." Public Opinion Quarterly, 52, 338-342.
21
Singer, E. (1988). To the editor of POQ. Public Opinion Quarterly, 52, 576-579.
Smith, T. W. (1987). That which we call welfare by any other name would smell
sweeter: An analysis of the impact of question wording on response patterns.
Sniderman, P. M., Piazza, T., & Kendrick, A. (1987). The dynamics of racial policy
attitudes. Paper presented at the annual meetings of the Midwest Political Science
Association, Chicago, Illinois.
Stagner, R. (1941). A comparison of the Gallup and Fortune polls regarding American
intervention policy. Sociometry, 4, 239-258. (p. 247)
Stagner, R. (1941). The prestige value of different types of leadership. Sociology and
Social Research, 404-413.
Studenski, P. (1939). How polls can mislead. Harpers, 180, 80-83.
Uhrbrock, R. S. (1950). Standardization of 724 rating scale statements. Personnel

Warr, P., Barter, J., & Brombridge, G. (1983). On the independence of positive and
Wembridge, E. R., & Means, E. R. (1918). Obscurities in voting upon measures due to
double-negative. Journal of Applied Psychology, 2, 156-163.
Winer, G. A., Hemphill, J., & Craig, R. K. (1988). The effect of misleading questions in
promoting nonconservation responses in children and adults. Developmental
10. Question Order
Abramson, P. R., Silver, B. D., & Anderson, B. A. (1987). The effects of question order
in attitude surveys: The case of the SRC/CPS citizen duty questions. American
Journal of Political Science, 31, 900-908.
Baron, R. S., & Roper, G. (1976). Reaffirmation of social comparison views of choice
shifts: Averaging and extremity effects in an autokenetic situation. Journal of
Personality and Social Psychology, 33, 521-530.
Bishop, G. F. (1987). Context effects on self-perceptions of interest in government and

public affairs. In H. Hippler, N. Schwarz, & S. Sudman (Eds.), Social
information processing and survey methodology. New York: Springer-Verlag.
22
Bishop, G. F., Oldendick, R. W., & Tuchfarber, A. J. (1980). Experiments in filtering

political opinions. Political Behavior, 2, 339-369.
*Bishop, G. F., Oldendick, R. W., & Tuchfarber, A. J. (1982). Political information

processing: Question order and context effects. Political Behavior, 4, 177-200.
Bishop, G. F., Oldendick, R. W., & Tuchfarber, A. J. (1984). What must my interest in
politics be if I just told you "I don't know"? Public Opinion Quarterly, 48, 510-
519.
Bishop, G. F., Oldendick, R. W., & Tuchfarber, A. J. (1985). The importance of

replicating a failure to replicate: Order effects on abortion items. Public Opinion
Quarterly, 49, 105-114.
Bradburn, N. M., & Mason, W. M. (1964). The effect of question order on responses.
*Budd, R. J. (1987). Response bias and the theory of reasoned action. Social Cognition,
5, 95-107.
Carp, F. M. (1974). Position effects on interview responses. Journal of Gerontology, 29,

581-587.
Carpenter, E. H., & Blackwood, L. G. (1979). The effect of question position on

responses to attitudinal questions. Rural Sociology, 44, 56-72.
Carper, J., & Doob, L. W. (1953-1954). Intervening responses between questions and
answers in attitude surveys. Public Opinion Quarterly, 17, 511-519.
Crespi, I., & Morris, D. (1984). Question order effect and the measurement of candidate
preference in the 1982 Connecticut elections. Public Opinion Quarterly, 48, 578-
591.
Feldman, J. M., & Lynch, J. G. (1988). Self-generated validity and other effects of
measurement on belief, attitude, intention, and behavior. Journal of Applied
Ferber, R. (1952). Order bias in a mail survey. Journal of Marketing, 17, 171-178.
Granberg, D. (1985). An anomaly in political perception. Public Opinion Quarterly, 49,

504-516.
23
Green, D. P., & Waxman, L. M. (1987). Direct threat and political tolerance: An
experimental analysis of the tolerance of blacks toward racists. Public Opinion
Quarterly, 51, 149-165.
Gross, E. J. (1964). The effect of question sequence on measures of buying interest.

Journal of Advertising Research, 4, 40-41.
Hayes, D. P. (1964). Item order and Guttman scales. American Journal of Sociology,
70, 51-58.
Higgins, E. T., & Lurie, L. (1983). Context, categorization, and memory: The "change-
of-standard" effect. Cognitive Psychology, 15, 525-547.
Higgins, E. T., & Stangor, C. (in press). A "change-of-standard" perspective on the

relations among context, judgment, and recall. Journal of Personality and Social
Psychology.

Research, 5, 39-41.
Hyman, H. H., & Sheatsley, P. B. (1950). The current status of American public
opinion. In J. C. Payne (Ed.), The teaching of contemporary affairs: Twenty-first
yearbook of the national council for the social studies. Washington, D. C.:
National Council for the Social Studies.
Johnson, W. R., Sieveking, N. A., & Clanton, E. S. (1974). Effects of alternative

positioning of open-ended questions in multiple-choice questionnaires. Journal of
Applied Psychology, 59, 776-778.
Kalton, G., Collins, M., & Brook, L. (1978). Experiments in wording opinion questions.
Applied Statistics, 27, 149-161.
Kirchner, W. K., & Uphoff, W. H. (1955). The effect of grouping scale items in union-
attitude measurement. Journal of Applied Psychology, 39, 182-183.
Knowles, E. S. (1988). Item context effects on personality scales: Measuring changes

the measure. Journal of Personality and Social Psychology, 55, 312-320.
Knowles, E. S., Coker, M. C., Cook, D. A., Diercks, S. R., Irwin, M. E., Lundeen, E. J.,
Neville, J. W., & Sibicky, M. E. (1991). Order effects within personality
measures. In N. Schwarz & S. Sudman (Eds.), Order effects in social and
psychological research. New York: Springer-Verlag.
Kraut, A. I., Wolfson, A. D., & Rothenberg, A. (1975). Some effects of position on
opinion survey items. Journal of Applied Psychology, 60, 774-776.
24
Mansbridge, J. J. (1986). Why we lost the ERA. Chicago: University of Chicago Press,
pp. 14-19.
McCauley, C., Kogan, N., & Teger, A. I. (1971). Order effects in answering risk
dilemmas for self and others. Journal of Personality and Social Psychology, 20,
423-424.
McClendon, M. J., & O'Brien, D. J. (1988). Question-order effects on the determinants

of subjective well-being. Public Opinion Quarterly, 52, 351-364.
McFarland, S. G. (1981). Effects of question order on survey responses. Public Opinion

Quarterly, 45, 208-215.
Metzner, H., & Mann, F. (1953). Effects of grouping related questions in questionnaires.
Moore, D. (1987). Question order effects and form resistant correlations: Some
discouraging findings. Paper presented at the annual meetings of the American
Association for Public Opinion Research, Hershey, PA.
Mullen, B., Driskell, J. E., & Smith, C. (1989). Availability and social projection: The
effects of sequence of measurement and wording of question on estimates of
consensus. Personality and Social Psychology Bulletin, 15, 84-90.
Nakamura, C. Y. (1959). Salience of norms and order of questionnaire items: Their

effect on responses to the items. Journal of Abnormal and Social Psychology, 59,
139-142.
Newman, S. E., & Benassi, V. A. (1989). Putting judgments of control into context:
Contrast effects. Journal of Personality and Social Psychology, 56, 876-889.
Pligt, J. V. D., Eiser, J. R., & Spears, R. (1987). Comparative judgments and
preferences: The influence of the number of response alternatives. British Journal
of Social Psychology, 26, 269-280.
Sayre, J. (1939). A comparison of three indices of attitude toward radio advertising.

Journal of Applied Psychology, 23, 23-33.
Schuman, H., Kalton, G., & Ludwig, J. (1981). Context and contiguity in survey
questionnaires. Public Opinion Quarterly, 47, 112-115.
*Schuman, H., & Ludwig, J. (1983). The norm of even-handedness in surveys as in life.
25
Schuman, H., & Presser, S. (1981). Questions and answers in attitude surveys. New
York: Academic Press. Chapter 2, "Question order and response order," p. 23-56
only.
Schuman, H., Presser, S., & Ludwig, J. (1981). Context effects on survey responses to
questions about abortion. Public Opinion Quarterly, 45, 216-223.
Schwarz, N., & Wyer, R. S. (1985). Effects of rank ordering stimuli on magnitude
ratings of these and other stimuli. Journal of Experimental Social Psychology, 21,
30-46.
Sigelman, L. (1981). Question-order effects on presidential popularity. Public Opinion

Quarterly, 45, 199-207.
Smith, R. H., Diener, E., & Wedell, D. H. (1989). Intrapersonal and social comparison
determinants of happiness: A range-frequency analysis. Journal of Personality
and Social Psychology, 56, 317-325.
Smith, T. W. (1982). Conditional order effects. Paper presented at the annual meeting
of the World Association for Public Opinion Research, Hunt Valley, Maryland,
May, 1982.
Strack, F., & Martin, L. L. (1987). Thinking, judging, and communicating: A process
account of context effects in attitude surveys. In H. Hippler, N. Schwarz, & S.
Sudman (Eds.), Social information processing and survey methodology. New
York: Springer-Verlag.
Swann, W. B., Pelham, B. W., & Chidester, T. R. (1988). Change through paradox:
Using self-verification to alter beliefs. Journal of Personality and Social
Tourangeau, R., & Rasinski, K. A. (1988). Cognitive processes underlying context

effects in attitude measurement. Psychological Bulletin, 103, 299-314.
*Tourangeau, R., Rasinski, K. A., & Bradburn, N. (1991). Measuring happiness in

surveys: A test of the subtraction hypothesis. Public Opinion Quarterly, 55, 255-
266.
Turner, C. F., & Krauss, E. (1978). Fallible indicators of the subjective state of the
nation. American Psychologist, 33, 456-470.
Weinstein, N. D. (1984). Why it won't happen to me: Perceptions of risk factors and
susceptibility. Health Psychology, 3, 431-457.
26
Willick, D. H., & Ashley, R. K. (1971). Survey question order and the political party
preferences of college students and their parents. Public Opinion Quarterly, 35,
189-199.
11. Attitude Recall Questions
Abramson, P. R., & Inglehart, R. (1992). Generational replacement and value change in
eight West European societies. British Journal of Political Science, 22, 183-228.
Aderman, D., & Brehm, S. S. (1976). On the recall of initial attitudes following
counterattitudinal advocacy: An experimental reexamination. Personality and
Allegeier, A. R., Byrne, D., Brooks, B., & Revnes, D. (1979). The waffle phenomenon:
Negative evaluations of those who shift attitudinally. Journal of Applied Social
Allsop, D., & Weisberg, H. F. (1988). Measuring change in party identification in an

election campaign. American Journal of Political Science, 32, 996-1017.
Andersen, K. (1979). Generation, partisan shift, and realignment: A glance back to the
new deal. In N. Nie, S. Verba, & J. Petrocik (Eds.). The changing American
voter. Cambridge, MA: Harvard University Press.
Anderson, R. C., & Pichert, J. W. (1978). Recall of previously unrecallable information

following a shift in perspective. Journal of Verbal Learning and Verbal Behavior,
17, 1-12.
Andrews, F. M., & Withey, S. B. (1976). Social indicators of well-being. New York:
Plenum Press.
Baddeley, A. D., Lewis, V., & Nimmo-Smith, I. (1978). When did you last ...? In M.
M. Gruneberg, P. E. Morris, & R. N. Sykes (Eds.), Practical aspects of memory.
London: Academic Press.
Bain, R. (1931). Stability in questionnaire response. American Journal of Sociology,

37, 445-453.
Baltes, P. (1968). Longitudinal and cross-sectional sequences in the study of age and
generation effects. Human Development, 11, 145-171.
Bartlett, F. C. (1932). Remembering: A study in experimental and social psychology.

Cambridge: Cambridge University Press.
Bem, D. J. (1965). An experimental analysis of self-persuasion. Journal of

Experimental Social Psychology, 1, 199-218.
27
Bem, D. J. (1967). Self-perception: An alternative interpretation of cognitive

dissonance phenomena. Psychological Review, 74, 183-200.
*Bem, D. J., & McConnell, H. K. (1970). Testing the self-perception explanation of

dissonance phenomena: On the salience of premanipulation attitudes. Journal of
Benewick, R. J., Birch, A. H., Blumler, J. G., & Ewbank, A. (1969). The floating voter
and the liberal view of representation. Political Studies, 17, 177-195.
Berent, M. K., & Krosnick, J. A. (1990). The accuracy of attitude recollections.

Unpublished data, The Ohio State University, Columbus, Ohio.
Blair, E., & Burton, S. (1987). Cognitive processes used by survey respondents to
answer behavioral frequency questions. Journal of Consumer Research, 14, 280-
288.
Blair, J., Menon, G., & Bickart, B. (1991). Measurement effects in self vs. proxy
responses to survey questions: An information-processing perspective. In P. P.
Biemer, R. M. Groves, L. E. Lyberg, N. A. Mathiowetz, & S. Sudman (Eds.),
Measurement errors in surveys. New York: Wiley.
Booth, A. (1971). The recall of news items. Public Opinion Quarterly, 34, 604-610.
Bower, G. H. (1981). Mood and memory. American Psychologist, 36, 129-148.
Bradburn, N. M., Rips, L. J., & Shevell, S. K. (1987). Answering autobiographical

questions: The impact of memory and inference on surveys. Science, 236, 157-
161.
Breksted, A. (1966). Factors influencing the reliability of anamnestic recall. Child

Development, 37, 601-612.
Bridge, R. G., Reeder, L. G., Kanouse, D., Kinder, D. R., Nagy, V. T., & Judd, C. M.
(1977). Interviewing changes attitudes - sometimes. Public Opinion Quarterly,
41, 56-64.
Burton, S., & Blair, E. (1991). Task conditions, response formulation processes, and
response accuracy for behavioral frequency questions in surveys. Public Opinion
Quarterly, 55, 50-79.
Butler, D., & Stokes, D. (1969). Political change in Britain. New York: St. Martin's
Press.
Cialdini, R. B., Braver, S. L., & Lewis, S. K. (1974). Attributional bias and the easily
persuaded other. Journal of Personality and Social Psychology, 30, 631-637.
28
Cialdini, R. B., Levy, A., Herman, C. P., & Evenbeck, S. (1973). Attitudinal politics:
The strategy of moderation. Journal of Personality and Social Psychology, 25,
100-108.
Cannell, C. F., & Fowler, F. J. (1963). A comparison of a self-enumerative procedure

and a personal interview: A validity study. Public Opinion Quarterly, 27, 250-
264.
Cannell, C. F., Oksenberg, L., & Converse, J. M. (1978). Experiments in interviewing

techniques: Field experiments in health reporting, 1971-1977. Washington, D.C.:
U.S. Department of Health, Education, and Welfare, DHEW Publication No.
(HRA) 78-3204.
Cialdini, R. B., Braver, S. L., & Lewis, S. K. (1974). Attributional bias and the easily
persuaded other. Journal of Personality and Social Psychology, 30, 631-637.
Cialdini, R. B., Levy, A., Herman, P., & Evenbeck, S. (1973). Attitudinal politics: The
strategy of moderation. Journal of Personality and Social Psychology, 25, 100-
108.
Conway, M., & Ross, M. (1984). Getting what you want by revising what you had.
Journal of Personality and Social Psychology, 47, 738-748.
Crowder, R. G. (1976). Principles of learning and memory. Hillsdale, NJ: Erlbaum.
Dakin, R. E., & Tennant, D. (1968). Consistency of response by event-recall intervals

and characteristics of respondents. Sociological Quarterly, 9, 73-84.
Duncan, O. D. (1975). Measuring social change via replication of surveys. In K. C.

Land & S. Spilerman (Eds.), Social indicator models. New York: Russell Sage.
Duncan, O. D., Schuman, H. & Duncan, B. (1973). Social change in a metropolitan

community. New York: Russell Sage Foundation.
Eich, E., Reeves, J. L., Jaeger, B., & Graff-Radford, S. B. (1985). Memory for pain:
Relation between past and present pain intensity. Pain, 23, 375-380.
Faunce, W. A. (1958). Automation and the automobile worker. Social Problems, 6, 68-
78.
Ferber, R. (1966). The reliability of consumer reports of financial assets and debts.
Urbana, IL: Bureau of Economic and Business Research, University of Illinois.
Ferber, R., Forsythe, J., Guthrie, H., & Maynes, E. S. (1969). Validation of a national
survey of consumer financial characteristics: savings accounts. Review of
Economics and Statistics, 51, 436-444.
Fink, R. (1960). The retrospective question. Public Opinion Quarterly, 24, 143-148.
29
Fischhoff, B. (1975). Hindsight =/ foresight: The effect of outcome knowledge on

judgment under uncertainty. Journal of Experimental Psychology: Human
Perception and Performance, 1, 288-299.
Fischhoff, B. (1977). Perceived informativeness of facts. Journal of Experimental

Psychology: Human Perception and Performance, 3, 349-358.
Goethals, G. R., & Reckman, R. F. (1973). The perception of consistency in attitudes.

Granberg, D., & Holmberg, S. (1991). Self-reported turnout and voter validation.
American Journal of Political Science, 35, 448-459.
Granberg, D. & Holmberg, S. (1992). The Hawthorne effect in election studies: The
impact of survey participation on voting. British Journal of Political Science, 22,
240-247.
Gutek, B. A. (1978). On the accuracy of retrospective attitudinal data. Public Opinion

Quarterly, 42, 390-401.
Hardin, E. (1960a). Computer automation, work environment, and employee

satisfaction: A case study. Industrial and Labor Relations Review, 13, 559-567.
Hardin, E. (1960b). The reactions of employees to office automation. Monthly Labor

Review, 83, 925-932.
Hardin, E. (1965). Perceived and actual change in job satisfaction. Journal of Applied
Hardin, E., & Hershey, G. L. (1960). Accuracy of employee reports on changes in pay.
Hastie, R., & Park, B. (1986). The relationship between memory and judgment depends
on whether the judgment task is online or memory-based. Psychological Review,
93, 258-268.
Hastorf, A., & Cantril, H. (1954). They saw a game. Journal of Abnormal and Social
Hawkins, S. A., & Hastie, R. (1990). Hindsight: Biased judgments of past events after
the outcomes are know. Psychological Bulletin, 107, 311-327.
Himmelweit, H. T., Biberian, M. J., & Stockdale, J. (1978). Memory for past vote:
Implications of a study of bias in recall. British Journal of Political Science, 8,
365-375.
30
Himmelweit, H. T., Humphreys, P., & Jaeger, M. (1985). How voters decide: A
longitudinal study of political attitudes and voting extended over fifteen years.
Philadelphia: Open University Press.
Huttenlocher, J., Hedges, L., Prohaska, V. (1988). Hierarchical organization in ordered

domains: Estimating the dates of events. Psychological Review, 95, 471-484.
Katz, R. S., Niemi, R. G., & Newman, D. (1980). Reconstructing past partisanship in
Britain. British Journal of Political Science, 10, 505-537.
Kelley, H. H. (1967). Attribution theory in social psychology. In D. Levine (Ed.),

Nebraska symposium on motivation (Vol. 15, pp. 192-240). Lincoln: University
of Nebraska Press.
Kenny, D. A., & Campbell, D. T. (1989). On the measurement of stability in over-time

data. Journal of Personality, 57, 445-481.
Ladd, E. C., & Hadley, C. D. (1975). Transformations of the American party system.
New York: Norton.
Lansing, J. B., Ginsburg, G., & Braaten, K. (1961). An investigation of response error.
Urbana, IL: Bureau of Economic and Business Research, University of Illinois.
Locander, W., Sudman, S., & Bradburn, N. (1976). An investigation of interview

method, threat, and response distortion. Journal of the American Statistical
Association, 71, 269-275.
Loftus, E. F. (1975). Leading questions and the eyewitness report. Cognitive

Loftus, E. F., & Zanni, G. (1975). Eyewitness testimony: The influence of the wording
of a question. Bulletin of the Psychonomic Society, 5, 86-88.
Lyons, T. F., & Dickinson, T. L. (1973). A comparison of perceived- and computed-

change measures over a three-year period. Journal of Applied Psychology, 58,
318-321.
Mann, F. C., & Hoffman, L. R. (1960). Automation and the worker. New York: Holt.
Markus, G. B. (1982). Political attitudes during an election year: A report of the 1980
NES panel study. American Political Science Review, 23, 338-359.
*Markus, G. B. (1986). Stability and change in political attitudes: Observed, recalled,

and "explained". Political Behavior, 8, 21-44.
McFarland, C., & Ross, M. (1987). The relation between current impressions and
memories of self and dating partners. Personality and Social Psychology Bulletin,
13, 228-238.
31
McFarland, C., Ross, M., & Conway, M. (1985). Self-persuasion and self-presentation
as mediators of anticipatory attitude change. Journal of Personality and Social
McFarland, C., Ross, M., & Giltrow, M. (1992). Biased recollections in older adults:
The role of implicit theories of aging. Journal of Personality and Social
Means, B., Nigam, A., Zarrow, M., Loftus, E. F., & Donaldson, M. S. (1989).
Autobiographical memory for health-related events. Washington, D.C.: U.S.
Department of Health and Human Services, DHHS Publication No. (OHS) 89-
1077.
Mednick, S. A., & Shaffer, J. B. P. (1963). Mothers' retrospective reports in child-

rearing research. American Journal of Orthopsychiatry, 33, 457-461.
Mischel, W., Ebbesen, E. B., & Zeiss, A. M. (1976). Determinants of selective memory
about the self. Journal of Consulting and Clinical Psychology, 44, 92-103.
Murphy, G., & Likert, R. (1967). Public opinion and the individual. New York: Russell
and Russell.
Natale, M., & Hantas, M. (1982). Effect of temporary mood states on selective memory
about the self. Journal of Personality and Social Psychology, 42, 927-934.
National Center for Health Statistics. (1965b). Health interview responses compared
with medical records. Vital and Health Statistics, PHS Publication No. 1000-
Series 2-No. 8. Washington, D.C.: U.S. Government Printing Office.
National Center for Health Statistics. (1965c). Comparison of hospitalization reporting

in three survey procedures. Vital and Health Statistics, PHS Publication No.
1000-Series 2-No. 8. Washington, D.C.: U.S. Government Printing Office.
National Center for Health Statistics. (1971). Effect of some experimental interviewing
techniques on reporting in the Health Interview Survey. Vital and Health
Statistics, PHS Publication No. 1000-Series 2-No. 41. Washington, D.C.: U.S.
Government Printing Office.
National Center for Health Statistics. (1972a). Effect of an extensive questionnaire and a
diary procedure. Vital and Health Statistics, Series 2-No. 49, DHEW Publication
No. (HSM) 72-1049 Health Services and Mental Health Administration.
Washington, D.C.: U.S. Government Printing Office.
National Center for Health Statistics. (1972b). Optimum recall period for reporting
persons injured in motor vehicle accidents. Vital and Health Statistics, Series 2-
No. 50, DHEW Publication No. (HSM) 72-1050. Washington, D.C.: U.S.
Government Printing Office.
32
National Center for Health Statistics. (1977). A summary of studies of interviewing

methodology. Vital and Health Statistics, Series 2-No. 69, DHEW Publication
No. (HRA) 77-1343. Washington, D.C.: U.S. Government Printing Office.
National Center for Health Statistics. (1965a). Reporting of hospitalization in the health
interview survey. Vital and Health Statistics, PHS Publication No. 1000-Series 2-
No. 6. Washington, D.C.: U.S. Government Printing Office.
Neter, J., & Waksberg, J. (1964). A study of response errors in expenditures data from
household interviews. Journal of the American Statistical Association, 59, 18-55.
Newcomb, T. M. (1943). Personality and social change. New York: Dryden.
Newcomb, T. M., Koenig, K. E., Flacks, R., & Warwick, D. P. (1967). Persistence and
change: Bennington College and its students after twenty-five years. New York:
Wiley.
Newman, W. R. (1976). Patterns of recall among television news viewers. Public

Niemi, R. G., Katz, R. S., & Newman, D. (1980). Reconstructing past partisanship: The
failure of the party identification recall questions. American Journal of Political
Science, 24, 633-651.
Olson, J. M., & Cal, A. V. (1984). Source credibility, attitudes, and the recall of past
behavior. European Journal of Social Psychology, 14, 203-210.
Parry, H. J., & Crossley, H. M. (1950). Validity of responses to survey questions.

Powers, E., & Bultena, G. (1972). Characteristics of decreased dropouts in longitudinal

research. Journal of Gerontology, 27, 530-535.
Powers, E. A., Goudy, W. J., & Keith, P. M. (1978). Congruence between panel and
recall data in longitudinal research. Public Opinion Quarterly, 42, 380-389.
Reiter, H. L. (1980). The perils of partisan recall. Public Opinion Quarterly, 44, 385-
388.
Robbins, L. C. (1963). The accuracy of parental recall of aspects of child development

and of child rearing practices. Journal of Abnormal and Social Psychology, 66,
261-270.
Roberts, J. V. (1985). The attitude-memory relationship after 40 years: A meta-analysis

of the literature. Basic and Applied Social Psychology, 6, 221-241.
Robinson, J. A. (1976). Sampling autobiographical memory. Cognitive Psychology, 8,

578-595.
33
Rogers, T. F. (1976). Interviews by telephone and in person: Quality of responses and

field performance. Public Opinion Quarterly, 40, 51-65.
Ross, M. (1989). Relation of implicit theories to the construction of personal histories.

Psychological Review, 96, 341-357.
Ross, M., & Shulman, R. F. (1973). Increasing the salience of initial attitudes:
Dissonance versus self-perception theory. Journal of Personality and Social
*Ross, M., & Conway, M. (1986). Remembering one's own past: The construction of
personal histories. In R. M. Sorrentino & E. T. Higgins (Eds.), Handbook of
motivation and cognition (pp. 122-144). New York: Guilford Press.
Ross, M., McFarland, C., & Fletcher, G. J. O. (1981). The effect of attitude on the recall
of personal histories. Journal of Personality and Social Psychology, 40, 627-634.
Ross, M., McFarland, C., Conway, M., & Zanna, M. P. (1983). Reciprocal relation
between attitudes and behavior recall: Committing people to newly formed
attitudes. Journal of Personality and Social Psychology, 45, 257-267.
Shaffer, D. R. (1975a). Another look at the phenomenological equivalence of pre- and

postmanipulation attitudes in the forced-compliance experiment. Personality and
Shaffer, D. R. (1975b). Some effects of consonant and dissonant attitudinal advocacy on

initial attitude salience and attitude change. Journal of Personality and Social
Skowronski, J. J., Betz, A. L., Thompson, C. P., & Shannon, L. (1991). Social memory
in everyday life: Recall of self-events and other-events. Journal of Personality
and Social Psychology, 60, 831-843.
Smith, T. W. (1984). Recalling attitudes: An analysis of retrospective questions on the

1982 GSS. Public Opinion Quarterly, 48, 639-649.
Snyder, M., & White, P. (1982). Moods and memories: Elation, depression, and the
remembering of the events of one's life. Journal of Personality, 50, 149-167.
Staw, B. M., & Ross, J. (1980). Commitment in an experimenting society: A study of

the attribution of leadership from administrative scenarios. Journal of Applied
Streib, G. (1966). Participants and drop-outs in a longitudinal study. Journal of

Gerontology, 21, 200-209.
Sudman, S., & Bradburn, N. (1973). Effects of time and memory factors on response in
surveys. Journal of the American Statistical Association, 68, 805-815.
34
Sudman, S., & Bradburn, N. M. (1974). Response effects in surveys: A review and
synthesis. Chicago: Aldine.
Teasdale, J. D., & Fogarty, S. J. (1979). Differential effects of induced mood on

retrieval of pleasant and unpleasant events from episodic memory. Journal of
Abnormal Psychology, 88, 248-257.
Teasdale, J. D., & Taylor, R. (1981). Induced mood and accessibility of memories: An
effect of mood state or of induction procedure? British Journal of Clinical
Tedeschi, J. T., Schlenker, B. R., & Bonoma, T. V. (1971). Cognitive dissonance:

Private retiocination or public spectacle. American Psychologist, 26, 685-695.
Trussell, R. E., & Elinson, J. (1959). Chronic illness in the United States, Vol. III.
Cambridge: Harvard University Press.
Wanous, J. P., & Reichers, A. E. (1992). Measuring "change": Remembered change vs.
longitudinal data. WPS 92-112, Working Paper Series, College of Business, Ohio
State University, Columbus, Ohio.
Weir, B. T. (1975). The distortion of voter recall. American Journal of Political

Science, 19, 53-62.
Weiss, C. (1968). Validity of welfare mothers' interview responses. Public Opinion

Quarterly, 32, 622-633.
+Wells, G. L., & Loftus, E. F. (1984). Eyewitness testimony: Psychological

perspectives. New York: Cambridge University Press.
Wenar, C., & Coulter, J. B. (1962). A reliability study of developmental histories. Child
Development, 33, 453-462.
White, L., & Edwards, J. N. (1990). Emptying the nest and parental well-being: An
analysis of national panel data. American Sociological Review, 55, 235-242.
Whyte, W. F. (1951). Pattern for industrial peace. New York: Harper.
Williams, M. D., & Hollan, J. D. (1981). The process of retrieval from very long-term
memory. Cognitive Science, 5, 87-119.
Withey, S. B. (1954). Reliability of recall of income. Public Opinion Quarterly, 18,

197-204.
Wixon, D. R., & Laird, J. D. (1976). Awareness and attitude change in the forced-
compliance paradigm: The importance of when. Journal of Personality and
35
Yarrow, M. R., Campbell, J. D., & Burton, R. V. (1970). Recollections of childhood: A

study of the retrospective method. Monographs of the Society for Research in
Child Development, 35, 1-83.
12. Asking Why
Bem, D. J. (1967). Self-perception: An alternative interpretation of cognitive dissonance

phenomena. Psychological Review, 74, 183-200.
Bem, D. J. & McConnell, H. K. (1970). Testing the self-perception explanation of

dissonance phenomena: On the salience of premanipulation attitudes. Journal of
Bishop, G. F. (1989). Think-aloud responses to survey questions: Illustrations of a new

qualitative technique. Presented at the annual conference of the American
Association for Public Opinion Research, St. Petersburg, Florida.
Blair, E. & Burton, S. (1987). Cognitive processes used by survey respondents to

answer behavioral frequency questions. Journal of Consumer Research, 14, 280-
288.
Boring, E. G. (1953). A history of introspection. Psychological Bulletin, 50, 169-189.
Bornstein, R. F., Leone, D. R., & Galley, D. J. (1987). The generalizability of

subliminal mere exposure effects: Influence of stimuli perceived without
awareness on social behavior. Journal of Personality and Social Psychology, 53,
1070-1079.
Brown, R. (1962). Models of attitude change. In R. Brown, E. Galanter, E. H. Hess, &

G. Mandler (Eds.), New directions in psychology (Vol. 1). New York: Holt,
Rinehart & Winston.
Burt, C. L. (1925). The young delinquent. London: University of Toronto Press.
Byrne, D. (1959). The effects of a subliminal food stimulus on verbal responses.

Chaiken, S. & Yates, S. (1985). Affective-cognitive consistency and thought-induced

attitude polarization. Journal of Personality and Social Psychology, 49, 1470-
1481.
Chapman, D W. (1932). Relative effects of determinate and indeterminate "Aufgaben."

American Journal of Psychology, 44, 163-174.
36
Comstock, C. (1921). On the relevancy of imagery to the processes of thought.

American Journal of Psychology, 32, 196-230.
Comte, A. (1896). The positive philosophy of Comte. London: Bell, (Originally

published, 1830-1842).
Davis, J. A. (1964). Great aspirations: The graduate school plans of America's college
students. Chicago: Aldine.
Dreben, E. K., Fiske, S. T., & Hastie, R. (1979). The independence of evaluative and
item information: Impression and recall order effects in behavior-based
impression formation. Journal of Personality and Social Psychology, 37, 1758-
1768.
Duval, S. & Wicklund's, R. A. (1972). A theory of objective self awareness. New York:
Academic.
Ericsson, K. A. & Simon, H. A. (1984). Protocol analysis: Verbal reports as data.

Cambridge, Mass: MIT Press.
Fazio, R. H., Chen, J. M., McDonel, E. C. & Sherman, S. J. (1982). Attitude

accessibility, attitude-behavior consistency, and the strength of the object-
evaluation association. Journal of Experimental Social Psychology, 18, 339-357.
Fishbein, M. & Ajzen, I. (1975). Belief, attitude, intention, and behavior: An

introduction to theory and research. Reading, MASS: Addison-Wesley.
Gaudet, H. (1955). A model for assessing changes in voting intention. In P. F.

Lazarsfeld & M. Rosenberg (Eds.), The language of social research. New York:
Free Press of Glencoe.
Gavanski, I. & Hoffman, C. (1986). Assessing influences on one's own judgments: Is

there greater accuracy for either subjectively important or objectively influential
variables? Social Psychology Quarterly, 49, 33-44.
Gavanski, I. & Hoffman, C. (1987). Awareness of influences on one's own judgments:

The roles of covariation detection and attention to the judgment process. Journal
of Personality and Social Psychology, 52, 453-463.
Genest, M. & Turk, D. C. (1981). Think-aloud approaches to cognitive assessment. In

T. Merluzzi, C. Glass, & M. Genest (Eds.), Cognitive assessment. N.Y.:
Guilford.
Goethals, G. R. & Reckman, R. F. (1973). The perception of consistency in attitudes.

Goode, W. J. (1956). After divorce. New York: Free Press of Glencoe.

37
Hastie, R. & Park, B. (1986). The relationship between memory and judgment depends
on whether the judgment task is memory-based or on-line. Psychological
Review, 93, 258-268.
Hawkins, D. (1970). The effects of subliminal stimulation on drive level and brand
preference. Journal of Marketing Research, 7, 322-326.
Hothersall, D. (1984). History of psychology. New York: Random House.
Hume, D. (1739). A treatise of human nature. London: Noon.
James, W. (1890). The principles of psychology. New York: Holt.
Jamieson, D. W., Lydon, J. E. & Zanna, M. P. (1987). Attitude and activity preference
similarity: Differential bases of interpersonal attraction for low and high self-
monitors. Journal of Personality and Social Psychology, 53, 1052-1060.
Kadushin, C. (1958). Individual decisions to undertake psychotherapy. Administrative

Science Quarterly, 3, 379-411.
Kelman, H. (1966). Deception in social research. Transaction, 3, 20-24.
Kimble, G. A. & Perlmuter, L. C. (1970). The problem of volition. Psychological

Review, 77, 361-384.
Kraut, R. E. & Lewis, S. H. (1981). Person perception and self-awareness: knowledge

of influences on one's own judgments. Journal of Personality and Social
Krosnick, J. A. (1991). Response strategies for coping with the cognitive demands of
attitude measures in surveys. Applied Cognitive Psychology, 5, 213-236.
Krosnick, J. A., Betz, A. L., Jussim, L. J., & Lynn, A. R. (1992). Subliminal
conditioning of attitudes. Personality and Social Psychology Bulletin, 18, 152-
162.
Kulpe, O. (1895). Outlines of psychology. Originally published, 1893. E. B. Titchener

(Trans.). New York: Macmillan.
Kunst-Wilson, W. R., & Zajonc, R. B. (1980). Affective discrimination of stimuli that

cannot be recognized. Science, 207, 557-558.
Langer, E. J. (1989). Mindfulness. Reading, Mass.: Addison-Wesley.
Latane, B. & Darley, J. (1970). The Unresponsive bystander: Why doesn't he help?
New York: Appleton-Century-Crofts.
Lazarsfeld, P. F. (1935). The art of asking why: Three principles underlying the
formulation of questionnaires. National Marketing Review, 1, 32-43.
38
Leahey, T. H. (1991). A history of modern psychology. New Jersey: Prentice Hall.
Lessler, J. T. (1987). Use of Laboratory Methods and Cognitive Science for the Design
and Testing of Questionnaires. Handbook of methods. Sweden.
Lessler, J., Tourangeau, R., & Salter, W. (1989). Questionnaire design in the cognitive
research laboratory. Vital and Health Statistics, Series 6, No. 1 (HHS Publication
No. PHS 89-1076). Washington, DC: U.S. Government Printing Office.
Lieberman, D. A. (1979). Behaviorism and the mind. A (limited) call for a return to
introspection. American Psychologist, 34, 319-333.
Locke, J. (1959). An essay concerning human understanding. New York: Dover,

(Originally published, 1690.)
Lodge, M., McGraw, K. M., & Stroh, P. (1989). An impression-driven model of

candidate evaluation. American Political Science Review, 83, 399-419.
Lusk, C. & Judd, C. M. (1988). Political expertise and the structural mediators of
candidate evaluations. Journal of Experimental Social Psychology, 24, 105-126.
Mackie, D. M. & Asuncion, A. G. (1990). On-line and memory-based modification of

attitudes: determinants of message recall-attitude change correspondence.
Markus, G. B. (1986). Stability and change in political attitudes: Observed, recalled,

and "Explained". Political Behavior, 8, 21-44.
Mayer, A., & Orth, J. (1901). Zur qualitativen Untersuchung der Association. Z.
Psychol. Physiol. Sinnesorg., 26, 1-13.
McDougall, W. (1922). Prolegomena to psychology. Psychological Review, 29, 1-43.
McGraw, K., Lodge, M. & Stroh, P. (1990). Processes of candidate evaluation: On-line
or memory based? A review and synthesis of four experiments. Unpublished
manuscript, Department of Political Science, State University of New York at
Stony Brook.
Millar, M. G. & Tesser, A. (1986). Effects of affective and cognitive focus on the
attitude-behavior relationship. Journal of Personality and Social Psychology, 51,
270-276.
Millar, M. G. & Tesser, A. (1989). The effects of affective-cognitive consistency and

thought on the attitude-behavior relation. Journal of Experimental Social
Newport, F. (1991). The Gallup poll monthly (no. 313). Princeton, NJ: The Gallup
Organization.
39
*Nisbett, R. E. & Wilson, T. D. (1977). Telling more than we can know: Verbal reports
on mental processes. Psychological Review, 84, 231-259.
Pryor, J. B., Gibbons, F. X., Wicklund, R. A. Fazio, & Hood, R. (1977). Self-focused
attention and self-report validity. Journal of Personality Psychology, 45, 513-527.
Quattrone, G. A. (1985). On the congruity between internal states and action.

Rahn, W. M. (1990). Perception and evaluation of political candidates: A social-

cognitive perspective. Unpublished Ph.D. dissertation, University of Minnesota.
Robles, R., Smith, R., Carver, C. S., & Wellens, A. R. (1987). Influence of subliminal
visual images on the experience of anxiety. Personality and Social Psychology
Bulletin, 13, 399-410.
Rossi, P. H. (1955). Why families move: A study in the social psychology of urban
residential mobility. New York: Free Press of Glencoe.
Rousseau, J., Johnson, E., & Stevens, D. (1984). The validity of verbal protocols.
Manuscript described in Zajonc, R. B. (1988). Prolegomena for the study of
access to mental events: Notes on Singer's chapter. In M. J. Horwitz (Ed.),
Psychodynamics and cognition. Chicago: University of Chicago Press.
Scheier, M. F., Buss, A. H., & Buss, D. M. (1978). Self-consciousness, self-report of

aggressiveness, and aggression. Journal of Research in Personality, 12, 133-140.
Schultz, D. (1981). A history of modern psychology. New York: Academic Press.
Schuman, H. (1966). The random probe: A technique for evaluating the validity of
closed questions. American Sociological Review, 31, 218-222.
Schuman, H. & Presser, S. (1981). Questions and answers in attitude surveys:

Experiments on question form, wording and context. New York: Academic
Press.
Sills, D. L. (1957). The volunteers: Means and ends in a national organization.

Glencoe, Ill.: Free Press.
Sills, D. L. (1961). On the art of asking "Why not?" Some problems and procedures in
studying acceptance of family planning. In All India Conference on Family
Planning (Fourth, 1961) Report of the Proceedings: 29th January-3rd February
1961, Hyderabad. Bombay: Family Planning Association of India.
Singer, J. L. (1988). Sampling ongoing consciousness and emotional experience:

Implications for health. In M. J. Horwitz (Ed.), Psychodynamics and cognition.
Chicago: University of Chicago Press.
40
Skinner, B. F. (1953). Science and human behavior. New York: Macmillan.
Skinner, B. F. (1957). Verbal behavior. New York: Appleton-Century-Crofts.
Smith, E. R. & Miller, F. D. (1978). Limits on perception of cognitive processes: A

reply to Nisbett and Wilson. Psychological Review, 85, 355-362.
Smith, E. R. & Miller, F. S. (1978). Limits on perception of cognitive processes: A

reply to Nisbett and Wilson. Psychological Review, 85, 355-362.
Snyder, M. (1982). When believing means doing: Creating links between attitudes and
behavior. In M. Zanna, E. Higgins, & C. Herman (Eds.), Consistency in social
behavior: The Ontario symposium. New Jersey: Lawrence Erlbaum Associates.
Snyder, M. & Ebbesen, E. B. (1972). Dissonance awareness: A test of dissonance theory

versus self-perception theory. Journal of Experimental Social Psychology, 8,
502-517.
Snyder, M. & Kendzierski, D. (1982). Acting on one's attitudes: Procedures for linking
attitude and behavior. Journal of Experimental Social Psychology, 18, 165-183.
Snyder, M. & Swann, W. B. (1976). When actions reflect attitudes: The politics of
impression management. Journal of Personality and Social Psychology, 34, 1034-
1042.
Stroh, P. K., Lodge, M. & Callaghan, K. (1990). Impression-driven and memory-based

explanations of political candidate evaluation. Presented at annual meeting of the
Midwest Political Science Association, Chicago, Illinois.
Tesser, A. (1978). Self generated attitude change. Advances in Experimental Social

Titchener, E. B. (1896). An outline of psychology. New York: Macmillan.
Titchener, E. B. (1912). The schema of introspection. American Journal of Psychology,

23, 485-508.
Turner, C. K. (1988). Don't blame memory for people's faulty reports on what
influences their judgments. Personality and Social Psychology Bulletin, 14, 622-
629.
Tyrer, P., Lewis, P., & Lee, I. (1978). Effects of subliminal and supraliminal stress on
symptoms of anxiety. Journal of Nervous and Mental Disease, 166, 88-95.
Watson, J. B. (1913). Psychology as the behaviorist views it. Psychological Review, 20,
158-177.
41
Weitz, B. & Wright, P. (1979). Retrospective self-insight on factors considered in

product evaluation. Journal of Consumer Research, 6, 280-294.
White, P. (1980). Limitations on verbal reports of internal events: A reputation of

Nisbett and Wilson and of Bem. Psychological Review, 87, 105-112.
Wicklund, R. A. (1982). Self-focused attention and the validity of self-reports. In M.

Zanna, E. T. Higgins, & C. Herman (Eds.), Consistency in social behavior: The
Ontario symposium. New Jersey: Lawrence Erlbaum Associates.
Willis, G. B., Royston, P. & Bercini, D. (1991). The use of verbal report methods in the
development and testing of survey questionnaires. Applied Cognitive
Wilson, T. D. (1985). Strangers to ourselves: The origins and accuracy of beliefs about
one's own mental states. In J. Harvey & G. Weary (Eds.), Attribution: Basic
issues and applications. New York: Academic Press.
Wilson, T. D. & Dunn, D. S. (1986a). Effects of introspection on attitude-behavior

consistency: Analyzing reasons versus focusing on feelings. Journal of
Experimental Social Psychology, 22, 249- 263.
Wilson, T. D. & Nisbett, R. E. (1978). The accuracy of verbal reports about the effects
of stimuli on evaluations and behavior. Social Psychology, 41, 118-131.
Wilson, T. D., Dunn, D. S., Kraft, D. & Lisle, D. J. (1989). Introspection, attitude
change, and attitude-behavior consistency: The disruptive effects of explaining
why we feel the way we do. Advances in Experimental Social Psychology, 22,
287-343.
Wilson, T. D., Hodges, S. D. & Pollack, S.E. (1992). Effects of explaining attitudes on
survey responses. Unpublished manuscript, University of Virginia.
Wilson, T. D., Kraft, D. & Dunn, D. S. (1989). The disruptive effects of explaining
attitudes: The moderating effect of knowledge about the attitude object. Journal
of Experimental Social Psychology, 25, 379-400.
Wilson, W. R. (1975). Unobtrusive induction of positive attitudes. Unpublished

doctoral dissertation, University of Michigan.
Wixon, D. R. & Laid, J. D. (1976). Awareness and attitude change in the forced-
compliance paradigm: The importance of when. Journal of Personality and
Wright, P. & Rip, P. (1980). Retrospective reports on consumer decision processes. In

J. Olson (Ed.), Advances in consumer research (Vol. 7). Urbana, IL: Association
for Consumer Research.
42
Wright, P. & Rip, P. D. (1981). Retrospective reports on the causes of decisions.

Zimbardo, P. G., Cohen, A. R., Weisenberg, M., Dworkin, L., & Firestone I. (1969).
The control of experimental pain. In P. G. Zimbardo, The cognitive control of
motivation. Glenview, Illinois: Scott, Foresman. Wilson, Hull, & Johnson, 1981.

References For Instrument Development

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

References For Instrument Development

Uploaded by

Copyright:

Available Formats

Questionnaire Design for Surveys and Laboratory Experiments:

Social And Cognitive Perspectives

Communication 239, Political Science 324R, Psychology 224

434 McClatchy Hall, 725-3031, Krosnick@stanford.edu

Most of our discussions will focus on the measurement of subjective psychological

2. Open versus Closed Questions

3. Closed Questions: Ratings versus Rankings

4. Closed Questions: Number of Scale Points/Magnitude Scaling

5. Closed Questions: Verbal vs. Numeric Scale Point Labels

6. Closed Questions: Response Order Effects

7. Attitudes and Non-attitudes

8. Acquiescence and Other Response Biases

9. Question Wording/Question Balance

10. Question Order

11. Attitude Recall Questions

12. Asking Why?

General Readings on Questionnaire Design

Bradburn, N. M. (1983). Response effects. Pp. 289-328 in P. H. Rossi, J. D. Wright, &

Oppenheim, A. N. (1966). Questionnaire design and attitude measurement. New York:

Sheatsley, P. B. (1983). Questionnaire construction and item writing. Pp. 195-230 in P.

1. Introduction to Issues of Question Design

Andrews, F. M. (1984). Construct validity and error components of survey measures: A

*Hippler, H., & Schwarz, N. (1987). Response effects in surveys. In H. Hippler, N.

Schuman, H., & Presser, S. (1977). Question wording as an independent variable in

*Tourangeau, R. (1987). Attitude measurement: A cognitive perspective. In H. Hippler,

2. Open vs. Closed Questions

*Converse, J. M. (1984). Strong arguments and weak evidence: The open/closed

Geer, J. G. (1988). What do open-ended questions measure? Public Opinion Quarterly,

Lazarsfeld, P. E. (1944). The controversy over detailed interviews - an offer for

Link, H. C. (1943). An experiment in depth interviewing. Public Opinion Quarterly, 7,

Skott, H. E. (1943). Attitude research in the Department of Agriculture. Public Opinion

3. Closed Questions: Ratings vs. Rankings

*Alwin, D. F., & Krosnick, J. A. (1985). The measurement of values in surveys: A

Feather, N. T. (1973). The measurement of values: Effects of different assessment

Guilford, J. P. (1954). Psychometric methods. New York: McGraw-Hill.

*Herzog, A. R., & Bachman, J. G. (1981). Effects of questionnaire length on response

*Krosnick, J. A., & Alwin, D. F. (1987). A test of the form-resistant correlation

Reynolds, T. J., & Jolly, J. P. (1980). Measuring personal values: An evaluation of

4. Closed Questions: Number of Scale Points

Andrews, F. M. (1984). Construct validity and error components of survey measures: A

Bass, B. M., Cascio, W. F., & O'Connor, E. J. (1974). Magnitude estimations of

Bendig, A. W. (1953). The reliability of self-ratings as a function of amount of verbal

Birkett, N. J. (1986). Selecting the number of response categories for a Likert-type

Bishop, G. F. (1987). Experiments with the middle response alternative in survey

Finn, R. H. (1972). Effects of some variations in rating scale characteristics on the

Garner, W. R. (1960). Rating scales, discriminability, and information transmission.

Ghiselli, E. E. (1939). All or none versus graded response questionnaires. Journal of

+Lodge, M. (1981). Magnitude scaling: Quantitative measurement of opinions. Beverly

McGuire, W. J. (1981). The probabilogical model of cognitive structure and attitude

Peterson, B. L. (1985). Confidence: Categories and confusion. GSS Technical Report

Ramsay, J. O. (1973). The effect of number of categories in rating scales on precision of

Watson, D. (1988). The vicissitudes of mood measurement: Effects of varying

Welch, R. E. (1972). The use of magnitude estimation in attitude scaling: Constructing a

5. Closed Questions: Verbal vs. Numerical Scale Point Labels

Albaum, G., & Murphy, B. D. (1988). Extreme response on a Likert scale.

Andrews, F. M. (1984). Construct validity and error components of survey measures: A

Bass, B. M., Cascio, W. F., & O'Connor, E. J. (1974). Magnitude estimations of

Bendig, A. W. (1953). The reliability of self-ratings as a function of amount of verbal

Bryant, G. D., & Norman, G. R. (19??). Expressions of probability: Words and

Budescu, D. V., & Wallsten, T. S. (1985). Consistency in interpretation of probabilistic

Cliff, N. (1959). Adverbs as multipliers. Psychological Review, 66, 27-44.

Cohen, J. Dearnley, E. J., & Hansel, C. E. M. (1958). A quantitative study of meaning.

Finn, R. H. (1972). Effects of some variations in rating scale characteristics on the