Quantitative Data Collection and Analysis:

Survey Research; Questionnaire Construction;

Types of surveys: Advantages and
Disadvantages
Survey Research
• Defined as "the collection of information from a sample of individuals
through their responses to questions"
• Allows for a variety of methods to recruit participants, collect data,
and utilize various methods of instrumentation.
• Can use quantitative research strategies (e.g., using questionnaires
with numerically rated items), qualitative research strategies (e.g.,
using open-ended questions), or both strategies (i.e., mixed
methods).
• As it is often used to describe and explore human behaviour, surveys
are therefore frequently used in social and psychological research
• Information has been obtained from individuals and groups through
the use of survey research for decades.
• It can range from asking a few targeted questions of individuals on a
street corner, to obtaining information related to behaviours and
preferences, to a more rigorous study using multiple valid and reliable
instruments.
• Common examples of less rigorous surveys include marketing or
political surveys of consumer patterns and public opinion polls.
• Has historically included large population-based data collection.
• The primary purpose of this type of survey research was to obtain
information describing characteristics of a large sample of individuals
of interest, relatively quickly.
• Large census surveys, obtaining information reflecting demographic
and personal characteristics and consumer feedback surveys are
prime examples.
• These surveys were often provided through the mail and were
intended to describe demographic characteristics of individuals or
obtain opinions on which to base programmes or products for a
population or group.
• More recently, survey research has developed into a rigorous
approach to research, with scientifically tested strategies detailing
who to include (representative sample), what and how to distribute
(survey method), and when to initiate the survey and follow up with
nonresponders (reducing nonresponse error), in order to ensure a
high-quality research process and outcome.
• Currently, the term "survey" can reflect a range of research aims,
sampling and recruitment strategies, data collection instruments, and
methods of survey administration.
• Given this range of options in the conduct of survey research, it is
imperative for the consumer/reader of survey research to understand
the potential for bias in survey research as well as the tested
techniques for reducing bias, in order to draw appropriate conclusions
about the information reported in this manner. 
Data Collection Methods
• Survey research may use a variety of data collection methods with the
most common being questionnaires and interviews.
• Questionnaires may be self-administered or administered by a
professional, may be administered individually or in a group, and
typically include a series of items reflecting the research aims.
• Questionnaires may include demographic questions in addition to
valid and reliable research instruments.
• It is helpful to the reader when authors describe the contents of the
survey questionnaire so that the reader can interpret and evaluate
the potential for errors of validity (e.g., items or instruments that do
not measure what they are intended to measure) and reliability (e.g.,
items or instruments that do not measure a construct consistently).
• Questionnaires may be in paper form and mailed to participants,
delivered in an electronic format via email or an Internet-based
program such as SurveyMonkey, or a combination of both, giving the
participant the option to choose which method is preferred.
• Using a combination of methods, survey administration can help to
ensure better sample coverage (i.e., all individuals in the population
having a chance of inclusion in the sample), therefore reducing
coverage error
• For example, if a researcher were to only use an Internet-delivered
questionnaire, individuals without access to a computer would be
excluded from participation.
• Self-administered mailed, group, or Internet-based questionnaires are
relatively low cost and practical for a large sample.
• Improving the visual appeal and graphics of surveys by using a font
size appropriate for the respondents, ordering items logically without
creating unintended response bias, and arranging items clearly on
each page can increase the response rate to electronic
questionnaires.
• Attending to these and other issues in electronic questionnaires can
help reduce measurement error (i.e., lack of validity or reliability) and
help ensure a better response rate.
• Conducting interviews is another approach to data collection used in
survey research.
Interviews
• Interviews may be conducted by phone, computer, or in person and
have the benefit of visually identifying the nonverbal response(s) of
the interviewee and subsequently being able to clarify the intended
question.
• An interviewer can use probing comments to obtain more information
about a question or topic and can request clarification of an unclear
response
• Interviews can be costly and time intensive, and therefore are
relatively impractical for large samples.
Mixed methods
• Used when no one method is adequate to address the planned
research aims, to reduce the potential for measurement and non-
response error, and to better tailor the study methods to the intended
sample
• For example, a mixed methods survey research approach may begin
with distributing a questionnaire and following up with telephone
interviews to clarify unclear survey responses
• Mixed methods might also be used when visual or auditory deficits
preclude an individual from completing a questionnaire or
participating in an interview.
Questionnaire construction
• A questionnaire that is poorly formatted can lead respondents to miss
questions, confuse respondents, or even cause them to throw the
questionnaire away.
• First, the questionnaire should be spread out and uncluttered.
• Researchers should not try to fit more than one question on a line
because that could cause the respondent to miss the second question
or get confused.
• Second, words should never be abbreviated in an attempt to save space or
make a questionnaire shorter. 
• Abbreviating words can be confusing to the respondent and not all
abbreviations will be interpreted correctly.
• This could cause the respondent to answer the question a different way or
skip it entirely.
• Lastly, ample space should be left between questions on each page.
Questions should not be too close together on the page or the respondent
might be confused as to when one question ends and another begins.
• Leaving a double space between each question is ideal.
Formatting individual questions
• In many questionnaires, respondents are expected to check one
response from a series of responses. There may be a square or circle
next to each response for the respondent to check or fill in, or the
respondent might be instructed to circle their response.
• Whatever method is used, instructions should be made clear and
displayed prominently next to the question. If a respondent indicates
their response in a way that is not intended, this could hold up data
entry or cause data to be miss-entered.
• Response choices also need to be equally spaced. For example, if
you’re response categories are "yes," "no," and "maybe," all three
words should be equally spaced from each other on the page.
• You do not want "yes" and "no" to be right next to each other while
“maybe” is three inches away.
• This could mislead respondents and cause them to choose a different
answer than intended. It could also be confusing to the respondent.
Question wording
• The wording of questions and response options in a questionnaire is very
important. Asking a question with the slightest difference in wording
could result in a different answer or could cause the respondent to
misinterpret the question.
• Oftentimes researchers make the mistake of making questions unclear
and ambiguous. Making each question clear and unambiguous seems like
an obvious guideline for constructing a questionnaire, however, it is
commonly overlooked.
• Often researchers are so deeply involved in the topic being studied and
have been studying it for so long that opinions and perspectives seem
clear to them when they might not be to an outsider.
• Conversely, it might be a new topic and one that the researcher has
only a superficial understanding of, so the question might not be
specific enough.
• Questionnaire items (both the question and the response categories)
should be so precise that the respondent knows exactly what the
researcher is asking.
Question wording
• Researchers should be cautious about asking respondents for a single
answer to a question that actually has multiple parts. This is called a
double-barreled question.
• As a general rule, whenever the word and appears in a question or
response category, the researcher is likely asking a double-barreled
question and measures should be taken to correct it and ask multiple
questions instead.
Ordering items in a questionnaire
• The order in which questions are asked can affect responses.
• First, the appearance of one question can affect the answers given to
later questions. For instance, if there are several questions at the
beginning of a survey that asks about the respondents’ views on
terrorism in the United States and then following those questions is
an open-ended question asking the respondent what they believe to
be dangers to the United States, terrorism is likely to be cited more
than it otherwise would be.
• It would be better to ask the open-ended question first before the
topic of terrorism is "put" into the respondents’ head.
• Efforts should be made to order the questions in the questionnaire so
they do not affect subsequent questions.
• This can be hard and nearly impossible to do with each question,
however, the researcher can try to estimate what the various effects
of different question orders would be and choose the ordering with
the smallest effect.
Questionnaire instructions
• Every questionnaire, no matter how it is administered, should contain
very clear instructions as well as introductory comments when
appropriate.
• Short instructions help the respondent make sense of the questionnaire
and make the questionnaire seem less chaotic. They also help put the
respondent in the proper frame of mind for answering the questions.
• At the very beginning of the survey, basic instructions for completing it
should be provided. The respondent should be told exactly what is
wanted: that they are to indicate their answers to each question by
placing a checkmark or X in the box beside the appropriate answer or by
writing their answer in the space provided when asked to do so.
• If there is one section on the questionnaire with closed-ended
questions and another section with open-ended questions, for
example, instructions should be included at the beginning of each
section.
• That is, leave instructions for the closed-ended questions just above
those questions and leave the instructions for the open-ended
questions just above those questions rather than writing them all at
the beginning of the questionnaire.
Advantages of Survey Research
• It is an inexpensive method of conducting research.
• Surveys are a practical solution for data gathering
• It is a fast way to get the results that you need.
• Surveys provide opportunities for scalability.
• It allows for data to come from multiple sources at once.
• Surveys give you the opportunity to compare results.
• It offers a straightforward analysis and visualization of the data.
• Survey respondents can stay anonymous with this research approach.
• Surveys can cover every component of any topic.
Disadvantages of survey research
• There is always a risk that people will provide dishonest answers.
• You might discover that some questions don’t get answers.
• There can be differences in how people understand the survey questions.
• Surveys struggle to convey emotions with the achievable results.
• Some answers can be challenging to classify.
• You must remove someone with a hidden agenda as soon as possible.
• Surveys don’t provide the same level of personalization.
• Some respondents will choose answers before reading the questions.
• Accessibility issues can impact some surveys.
• Survey fatigue can be a real issue that some respondents face.
Conclusion
• Survey research is a useful and legitimate approach to research that
has clear benefits in helping to describe and explore variables and
constructs of interest.
• Survey research, like all research, has the potential for a variety of
sources of error, but several strategies exist to reduce the potential
for error.