Subject: Advance Research Methods

Class: BS-Commerce 6th Semester

From Week 1 to 8: Advance Research Methods (Important Topics)

COURSE: WEEK 1&2

Q1. What is Survey Research, also explain the basic methods used for communication with
respondents.

Answer:

Survey Research

In research of human subjects, a survey is a list of questions aimed at extracting specific data from
a particular group of people. Surveys may be conducted by phone, mail, via the internet, and
sometimes face-to-face on busy street corners or in malls.

Communication Methods

1: Personal Interviews
In personal interviews, the interviewer and respondent meet face to face and discusses the
research questions through structured or semi-structured questions.

Types of Personal Interviews

1.1 Door-to-Door Interviews:

Personal interview conducted at respondent’s doorsteps in an effort to increase the participation

rate in the survey.

1.2 Mall intercept interview:

Personal interview conducted in a shopping mall.

1.3 Callback:

An attempt to recontact individuals selected for a sample who were not available initially

1.4 Telephone Interviews:

1
Personal interview conducted by telephone, the mainstay of commercial survey research.

Characteristics of telephonic interviews

• Speed
• Cost
• Absence of face-to-face contact
• Cooperation
• Limited duration
• Lack of visual medium

2: Computer-Assisted Telephone Interviewing

Actually Computer-Assisted Telephone Interview (CATI) Technology that allows answers to

telephone interviews to be entered directly into a computer for processing computerized Voice-
Activated Telephone Interview. Combining computerized telephone dialing and voice-activated
computer messages to allow researchers to conduct telephone interviews without human
interviewers.

3: Self-Administered Questionnaires

Survey in which the respondent takes the responsibility for reading and answering the questions.

4: Selecting the Appropriate Survey Research Design

To determine the appropriate technique, the researcher must ask several questions:

• Is the assistance of an interviewer necessary?

• Are respondents interested in the issues being investigated?
• Will cooperation be easily attained?
• How quickly is the information needed?
• Will the study require a long and complex questionnaire?
• How large is the budget?

5: Pre-Testing

Screening procedure that involves a trial run with a group of respondents to iron out fundamental
problems in the survey design.

Q2. Keeping in view the Survey Research? Also discuss the ethical issues in Survey Research.

Researcher Responsibilities to Participants

2
Researchers have some general obligations to the people who provide data in marketing research
studies. These include:

1. Participants should not be harmed.

2. Participants should not be deceived.
3. Participation should be willing and informed.
4. Data should be held in confidence.

Participants Should Not Be Harmed

The first obligation of researchers to participants is not to harm them in any way. Almost never
does physical harm become an issue in survey research, although in a few health surveys blood
samples may actually be taken. At an emotional level, the obligation not to harm participants
means that they shouldn’t be embarrassed, ridiculed, belittled, or generally subjected to mental
distress.
Taken to the extreme, the obligation not to hurt participants might seem to make research
impossible. The approach taken by most researchers who have pondered this issue is to apply two
standards. First, researchers should minimize the potential harm of a research project (and
maximize respondents’ convenience). This entails practices such as:
- Reminding research participants that they need not answer any question they prefer not to answer.
- Scheduling interviewing so as to minimize disruptions. Second, once the potential harm of a
research project has been minimized, compare its risks to risks that people face in everyday life. If
the risks posed by the research are no greater than those posed by everyday life, then the research
is not viewed as posing serious ethical issues.
Ethical issues do arise when a person who is asked to participate in research does not instantly
agree. Is it ethical to try to persuade someone to participate and, if so, how much effort is
permissible before it becomes harassment?
The approach that many researchers have adopted is to distinguish between hard and soft refusals.
A soft refusal is one that says "I’m too busy right now," or "This isn’t a good time." A hard refusal
is one where the person says "No, I’m not interested." Hard refusals are not followed up, but soft
refusals may be. Even for soft refusals, there will almost never be more than one follow-up effort.

3
For mail surveys, respondents who send back the questionnaire indicating that they don’t wish to
participate are not followed-up, but those who don’t mail it back receive subsequent reminder
mailings.

Participants Should Not Be Deceived

There are many ways in which researchers might deceive respondents. Is it ever ethically
permissible to deceive participants in research? Several situations can be distinguished.
Sales prospecting and fund raising under the guise of surveys: Because so many people are willing
to participate in research studies, some unscrupulous marketers and fund-raisers use phony
research, especially surveys, for purposes unrelated to obtaining information. This is a clear
violation of all ethical codes that deal with marketing research. If something is represented to
participants as research, then it should be legitimate research, and the results should not be used
for other purposes without the express consent of participants.
As an example of phony research, a major car company conducted a large telephone survey some
time ago, indicating to respondents that the purpose of the survey was to measure their attitudes
about cars and their buying intentions for market research purposes. This information was then
turned over to local dealers who made sales calls to likely buyers. Not only is this an unethical
deception of respondents, but such phony surveys ultimately lead to reduced cooperation on
legitimate surveys, killing the goose that lays the golden eggs.
One of the most common uses of phony research these days is to raise money. Fund raisers have
discovered that people are more likely to contribute money if they have a chance to express their
views on issues related to the fund raising. Thus, a brief survey, usually of four or five questions,
is included with a request for a contribution. The survey results are typically thrown away after
the checks are cashed and no attention is given to them. Fund raising under the guise of doing
research is deceptive and unethical.
Misidentification of the researcher or research sponsor: In my opinion, sponsor or researcher
misidentification is acceptable if it serves legitimate purposes of interviewer seurity or data
integrity and it does not encourage participation from people who otherwise would not participate.
Consider a survey about car attitudes done on behalf of the Ford Motor Company. If respondents
are told that Ford is the research sponsor, they may shade their answers to be more favorable
toward Ford cars. This gives a legitimate reason not to disclose the research sponsor. At the same
time, it seems unlikely that people who are willing to participate in a survey about automobiles
4
would suddenly withdraw that participation if they knew that Ford was the sponsor. Therefore, I
don’t see an ethical problem with failing to disclose the sponsor’s identity at the start of the survey.
If there is reason to believe that the sponsor’s identity will affect people’s willingness to participate
in the research, then the sponsor should be identified.
Deceiving participants about the length of the task: Imagine a researcher who wants to do
interviews that will take 90 minutes. The researcher doesn’t mention how long the interview will
last during the introduction because of a concern that many respondents will refuse to cooperate if
they know its length. In this case, this is deception by omission and is unethical.
In general, the length of an interview should be specified when seeking cooperation if it is possible
to do so. For interviews where the length varies depending on the answers, it often is possible to
specify a range.

Participation Should Be Willing and Informed

The whole ethical issue of how much information to give research participants so they can make
informed decisions on whether to participate is a difficult one. Respondents should be given
adequate information, but a researcher should not take 20 minutes to explain a ten-minute
questionnaire. Such detailed explanations increase the total time required from respondents with
no clear benefits.
The general principle that guides informed consent is that the amount of information should bear
some relationship to the risk involved in the study. Most survey research involves only small risks
for the participants, and only minimal information need be given at the beginning. For example, a
typical survey introduction would need only three kinds of information:
- Who is doing the study?
- What is the study about?
- How long will it take?
One might want to add :"Your participation in this study is, of course, completely voluntary. Feel
free to refuse to answer any question that you feel is too personal." Such a phrase makes
respondents’ rights more explicit, and has been found to reassure people and make them more
likely to participate.
It is unethical to falsify the name of the research firm or the client. If the research is done in-house
by a company, then that company should be willing to be identified. Some companies have

5
established in-house research operations and given them different names to conceal the parent
company’s identity, a borderline ethical behavior.

Data Should Be Held in Confidence

The final researcher obligation to participants is the obligation to hold data in confidence. Whether
or not a promise of confidentiality is made, research participants have a right to confidentiality
unless they specifically waive this right.
This means that data gathering organizations should routinely separate information that can
identify a specific respondent from the information the respondent gives. This usually cannot be
done at the time of a survey interview because it necessary to have identifying information for
purposes of verifying that the interview was conducted. It also may be necessary to call
respondents back if interviews are incomplete or crucial answers are unclear. However, once
verification and data editing are complete, identifying information should be separated from the
answers and never supplied to the client or anyone else. In fact, the identifying information should
be destroyed unless there are plans to conduct follow-up research in which the same respondents
are reinterviewed. In that situation, the identifying information should be retained in a file separate
from the data, and respondents should not be promised anonymity (though it is possible to promise
confidentiality).
Research participants may waive confidentiality if they wish. In some research projects,
participants are asked if they would be willing to be identified so that someone from the sponsoring
company can talk with them to follow up their comments. Since such follow-ups are done with the
permission of participants, there is no violation of ethical principles.
A rare but troublesome problem in research is to have data subpoenaed for court cases. When data
are subpoenaed, there are only limited legal safeguards for respondent confidentiality. If
identifying information has already been routinely removed, of course, then respondent
confidentiality can be maintained, but this information cannot be destroyed after being requested
by the courts

COURSE: WEEK 3&4

6
Q3. What is Observation in Research? Also discuss the basic key Dimensions of
Observational studies in Research.

Answer:

Observation in Research

Observation, as the name implies, is a way of collecting data through observing. Observation data
collection method is classified as a participatory study, because the researcher has to immerse
herself in the setting where her respondents are, while taking notes and/or recording.

Observation as a data collection method can be structured or unstructured. In structured or

systematic observation, data collection is conducted using specific variables and according to a
pre-defined schedule. Unstructured observation, on the other hand, is conducted in an open and
free manner in a sense that there would be no pre-determined variables or objectives.

Advantages of observation data collection method include direct access to research phenomena,
high levels of flexibility in terms of application and generating a permanent record of phenomena
to be referred to later. At the same time, observation method is disadvantaged with longer time
requirements, high levels of observer bias, and impact of observer on primary data, in a way that
presence of observer may influence the behavior of sample group elements.

It is important to note that observation data collection method may be associated with certain
ethical issues. Fully informed consent of research participant(s) is one of the basic ethical
considerations to be adhered to by researchers. At the same time, the behavior of sample group
members may change with negative implications on the level of research validity if they are
notified about the presence of the observer.

This delicate matter needs to be addressed by consulting with dissertation supervisor, and
commencing observation primary data collection process only after ethical aspects of the issue
have been approved by the supervisor.

7
 Basic key Dimensions of Observational studies

Controlled and Uncontrolled Studies

Some terminology:

In a controlled study, researchers are able to determine which of their subjects receive the
factor that is being tested for having a causal influence upon another factor. The alleged
causal factor is sometimes called the independent variable, while the alleged effect is called
the dependent variable.

Those given the alleged causal factor make up the experimental group, while those who don’t
receive such treatment belong in the control group. Ideally, both groups will be balanced with
respect to the subjects’ various other characteristics. Often this is achieved simply by
randomly assigning subjects to experimental or control groups.

In a blind study, the subjects do not know whether they are in the control group or not. In
a double blind study, the researchers gathering the data will not know which group a subject
belongs to. Blinding is meant to mitigate placebo effects – possible confounding of the study
by the subjects’ or researchers’ knowledge.

In uncontrolled or observational studies, researchers have no such control over whether their
subjects receive the treatment being investigated. Uncontrolled studies thus are subject to
worries about confounding variables.

An example:
Earlier this month, professor Yasmin Hurd of the Mount Sinai School of Medicine released
a study showing that rats exposed to the main ingredient in marijuana during their
adolescence showed a greater sensitivity to heroin as adults. The wire lit up with articles
announcing confirmation for the "gateway theory"—the claim that marijuana use leads to
harder drugs.
It's a theory that has long seemed to make intuitive sense, but remained unproven. The
federal government's last National Survey on Drug Use and Health, conducted in 2004,
counted about 97 million Americans who have tried marijuana, compared to 3 million who
have tried heroin (166,000 had used it in the previous month). That's not much of a rush
through the gateway. And a number of studies have demonstrated that your chances of
becoming an addict are higher if addiction runs in your family, or if heroin is readily
available in your community, or if you're a risk-taker. These factors can account for the total
number of heroin addicts, which could make the gateway theory superfluous. (Slate
Magazine, July 23, 2006)

This passage cites two studies, one is controlled, the other is not. Can you identify which?

It then warns us of potential confounding factors which might also explain the observed
correlation. What are these?

8
 PARTICIPANT VS NON-PARTICIPANT OBSERVATION
TYPE 1# PARTICIPANT OBSERVATION:
The participant observation means watching the events or situation or activities from inside by
taking part in the group to be observed. He freely interacts with the other group members,
participates in various activities of the group, acquires the way of life of the observed group or his
own, and studies their behaviour or other activities not as an outsider but by becoming a member
of that group.

Goode and Hatt define participant observation as “the procedure used when the investigator can
go disguise himself as to be accepted as a member of the group”. So, in this kind of observation
the observer has to stay as a member in the group he wants to study.

According to P.V. Young, “the participant observer using non-controlled observation, generally
lives or otherwise shares in the life of the group which he is studying”.

Some of the examples of studies using the method of participant observation are: W.F. White’s
study of Cornville social and Athletic Club and P.V. Young’s study of Molokan people. The
famous studies of Margaret Mead on primitive societies were also based on participant
observation.

For the success of participant observation, it is essential that the respondents being studied should
not have any doubt about the intention of the research worker. A fruitful result of participant
observation is very much dependent upon the resourcefulness, tactfulness, personality manners
and wit of the research worker.

Advantages of Participant Observation:

(a) Observation of natural behaviour:
The natural behaviour of the respondent can be studied by participant observation. When a group
knows that they are going to be observed by a stranger, they feel conscious, uncomfortable and
therefore neutrality in their behaviour and activity is lost. But in case of the participant observation,
the respondents do not know that they are being observed. So, their behaviour is not constrained
by the conscious feeling of being observed by a stranger.
9
(b) Closeness with the group:
In participant observation, the observer has a very good rapport with the respondents. He has a
very close primary relationship with the group members. Because of this he can participate in all
activities from a close angle and thus can better interpret the situation than a non-participant
observer.

(c) Studying the real character:

Often in order to study the actual behaviour, the group research requires close participation and
contact with the group members. Through participant observation the observer can make an
intensive and inclusive study of the group and can gain into the real character of such group.

(d) Better Understanding:

In participant observation the observer can better understood the feeling of the respondents than
an outsider. For example, a person who is actually living in a slum area can realize the feeling and
hardship of the slum dwellers in a better way than an outsider.

(e) Participation provides opportunity to learn more about an event:

The chief advantage of participant observation is that in it the observer gets an opportunity to
interact with the group regarding various activities of them. He can thus learn the significance of
these activities that are actually not open for observation. For example, if an observer participates
in a religious ceremony of a tribe, viz. “Chaitra Parba” of Gadaba tribe, he can not only observe
different aspects of the ceremony but also clear his doubt by asking various questions to the group
members or learn more about that ceremony by discussing with the group in this regard. It is
generally easier for the respondent to describe about the event on right occasion than before or
after it.

Disadvantages of Participant Observation:

In-spite of above advantages of participant observation it has also many disadvantages.

The following are the disadvantages of the participant observation:

(a) Lack of objectivity:

10
By becoming members of a group and participating very closely in it, the observer may lose his
objectivity. His emotional and sentimental association with the group kills his impartiality and
unbiased analysis. He may develop some soft corner for that group member and because of this;
he may often justify their evil activities as just activity.

(b) Often close association brings biased interpretation:

Because of his close association and emotional participation with the group members the
researcher creates a special position for himself in that group. He may be influenced or pleased by
this and begins to support them blindly. Due to this he observes the things from his own personal
point of view rather than scientific point of view.

(c) Misses important issues due to familiarity:

Due to much familiarity many crucial events appear to the participant observer as of little or no
significance. Therefore, he misses many of the important issues. But a stranger pays much attention
even to a small thing, as this appears new to him.

(d) Limited range of experience:

In participant observation the observer is confines himself to a particular group. So his experience
becomes very deep, but the range of his experience becomes very limited.

(e) Involvement in groupism:

The active participation and proximity of the observer with the group may involve him in quarrels
and group factionalism. He cannot avoid taking side of one faction. But if he does so, he loses his
status as an impartial observer whom everybody is ready to co-operate. So it destroys the very
purpose of the research and the researcher finds it very difficult to get proper information from the
group.

(f) Limits of participant observation:

There are certain situations in which the participant observation is not possible. For example, it is
not possible to observe criminals or prisoners.

11
TYPE 2# NON-PARTICIPANT OBSERVATION:
When the observer observes the group passively from a distance without participating in the group
activities, it is known as non-participant observation. Here he does not try to influence them or
take part in the group activities.

However, purely non-participant observation is extremely difficult. One cannot penetrate into the
heart of a matter without proper participation in it. One really cannot imagine a kind of relationship,
when the researcher is always present but never participates. This situation is hardly conducive for
both the observer and the group. A combination of both participant and non-participant method is
sometimes selected.

The observer actively participates in some of the ordinary activities and observes passively from
distance in others. Many sociologists therefore treat a non-participant observation in practice as
only a quasi-participant observation. It is easier for the observer to perform both the roles than to
disguise himself completely.

Advantages of Non-Participant Observation:

However, non-participant observation also has some advantages.

Following are the merits of participant observation:

(a) Objectivity and neutrality:
If an observer participates in the event actively and emotionally, he may try to justify the evil
things of the group as just things. In this frame of mind, he cannot analyze the phenomena with
neutrality. But in non-participant observation, the objectivity or neutrality can be maintained. The
observer in this type of observation gives a detached and unbiased view about the group.

(b) Command respect and co-operation:

In case of non-participant observation, the researcher plays an impartial role. Therefore, every
member of the group gives him a special status and co-operate with his study.

(c) More willingness of the respondent:

Often people do not feel shy to disclose their secrets, weaknesses or informal things to a stranger.
But they always become reluctant to disclose these things to a known person.

12
(d) Careful analysis:
In participant observation because of the much familiarity with the events, sometimes the observer
does not realize the significance of same events and neglects them. But in non- participant
observation the researcher does not even miss a minute thing. He carefully judges the merits and
demerits of each and every phenomenon under study.

(e) Freedom from groupism:

In non-participant observation the researcher always maintains his impartial status. His aloofness
from petty conflicts helps him to carry his research work more smoothly.

Disadvantages of Non-Participant Observation:

The following are the disadvantages of non-participant observation:
(a) Subjectivity:
In non-participant observation the observer does not have clarity about certain events on activities.
He cannot clear his doubts by asking various questions to the group members. Therefore, he has
to simply understand and interpret what he sees. This lack of understanding may make some of his
findings biased and colored by his personal prediction, belief and pre-conception.

(b) Inadequate observation:

The observer can observe only those events which take place in front of him. But that is not enough
and only a part of the phenomena as a vast range of information required for the research. He can
know many things about the group when he participates in the group and interacts with the group
members.

(c) Unnatural and formal information:

The members of a group become suspicious of a person who observes them objectively. In front
of an outsider or stranger they feel conscious and provide only some formal information’s in an
unnatural way. It creates bias and what the observer collects is not actual or normal thing but only
formal information’s.

(d) Inconvenience to the respondents:

The members of a particular group always feel uncomfortable when they know that their behaviour
is critically analyzed by an outsider. Therefore, in some cases the tribal do not allow an outsider

13
to watch their socio-cultural activities. It is always better for a researcher to become a member of
the group in order to learn much about it.

STRUCTURED VS UNSTRUCTURED OBSERVATIONAL STUDIES

As we have seen, observational studies can be of either the nonparticipant‐observer or the

participant‐observer type. Both of these, again, can be either structured or unstructured. Where the
observer has a predetermined set of categories of activities or phenomena planned to be studied, it
is a structured observational study. Formats for recording the observations can be specifically
designed and tailored to each study to suit the goal of that research. Structured observation is
generally quantitative in nature.

Usually, matters that pertain to the feature of interest, such as the duration and frequency of an
event (for instance, how long does it take to get a meal at a fast‐food restaurant?), as well as certain
activities that precede and follow it, are recorded. Environmental conditions (for instance, labor
conditions) and any changes in setting are also noted, if considered relevant. Task‐relevant
behaviors of the actors, their perceived emotions, verbal and nonverbal communication, and the
like, may also be recorded. Observations that are recorded in worksheets or field notes are then
systematically analyzed. At the beginning of a study, it is also possible that the observer has no
definite ideas of the particular aspects that need focus. Observing events as they take place may
also be a part of the plan as in many other forms of exploratory and qualitative research. In such
cases, the observer will record practically everything that is observed. Such a study will be an
unstructured observational study. Unstructured observational studies are claimed to be the
hallmark of qualitative research. Qualitative data analysis (Chapter 16) is used to analyze and
interpret what the researcher has seen.

Unstructured observation may eventually lead to a set of tentative hypotheses that are tested in
subsequent research that is deductive in nature. Hence, inductive discovery via observation can
pave the way for subsequent theory building and hypotheses testing.

14
 CONCEALED VS NON-CONCEALED OBSERVATIONAL STUDIES

Concealment of observation relates to whether the members of the social group under study are
told that they are being investigated. A primary advantage of concealed observation is that the
research subjects are not influenced by the awareness that they are being observed. Indeed,
reactivity or the extent to which the observer affects the situation under observation could be a
major threat to the validity of the results of observational studies. Unconcealed observation is
more obtrusive, perhaps upsetting the authenticity of the behavior under study.

What to observe?

A potential problem with observational studies is getting overwhelmed by massive amounts of

often disconnected data. For this reason, the researcher should try to keep a certain focus during
the various stages of the observation process. Generally speaking, the most important factor in
determining what to observe is the aim or purpose of the study. However, “[w]here to begin
looking depends on the research question, but where to focus or stop action cannot be determined
ahead of time” (Merriam, 1988, p. 97). Werner and Schoepfle (1987) discernthree consecutive
processes in observation that may provide an increasingly deep understanding of the setting
that is being studied: (1) descriptive observation, (2) focused observation, and (3) selective
observation.

In descriptive observation, the researcher is open to everything that is going on; data are collected
that describe the setting, the subjects, and the events that are taking place

What to observe in the descriptive observation stage?

Spradley (1980) distinguishes the following dimensions on which descriptive data might be
collected:
❖ Space layout of the physical setting such as the factory floor layout;
❖ Objects physical elements such as office equipment, machines, and power tools;
❖ Actors relevant details of the persons involved;
❖ Feelings, emotions, activities, actions, and goals of the actors;
❖ Events for instance meetings; and

15
 ❖ Time or the time sequence of events, feelings, actions, and the like.

What to observe in the focused and selective observation stages?

To help researchers decide on what to observe in the focused and selective observation stages,
DeWalt & DeWalt (2002) suggest that they:

❖ Observe events, actions, and behavior and look for a story line;
❖ Sort out the regular from the irregular activities;
❖ Look for variation in the storyline;
❖ Look for negative cases or exceptions; and,
❖ In case the observation is structured, develop a plan for systematic observation, including
an estimate of how many observations will be enough.

COURSE: WEEK 5&6

Q4. Define a Questionnaire? Also explain how to design a good questionnaire.

Answer:

Questionnaire

A questionnaire is a research instrument consisting of a series of questions for the purpose of

gathering information from respondents. The questionnaire was invented by the Statistical Society
of London in 1838.

How to design a good questionnaire

Questionnaires are a set of written questions designed to gather standardized information about the
opinions, preferences, experiences, intentions, and behavior of individuals, and can be devised for
the purposes of a scientific study. Traditionally, they have been said to contrast with surveys in the
sense that they do not collect mass data for further analysis, however the terms are largely used
interchangeably these days (and many research studies also use them together).

16
While questionnaires provide a comparatively cheap, prompt, and efficient means of obtaining
large amounts of information, questionnaire design is a multistage process that requires attention
to a number of aspects at the same time to gather the information you seek. Why exactly is that?

Depending on the kind of information you aim to acquire, questions need to be asked in varying
degrees of detail and in specific ways.

Given the same topic, it’s rather likely that different researchers will come up with different
questionnaires that vary widely in their choice of questions, a line of questioning, use of open-
ended questions, and length.

SIX STEPS TO GOOD QUESTIONNAIRE DESIGN

#1: Identify your research aims and the goal of your questionnaire

What kind of information do you want to gather with your questionnaire? What is your main
objective?

Ideally, there are already existing questionnaires with published results on the validity and
statistical evaluation of all tested questions that you can use (or borrow a couple of ideas from).

While this is quite common in academic research, it might be rather hard to reuse existing
questionnaires for commercial applications. In this case, you might have to scan papers and
internal reports for key metrics of interest and create a questionnaire that specifically addresses
these aspects.

#2: Define your target respondents

Clearly, you can’t test everyone – it’s rather plausible that there have to be certain restrictions with
respect to the target audience of your questionnaire. The selection of groups is a key factor for
maximizing the outcomes of your study.

To put it another way: You can run multiple questionnaire sessions over a longer period of time
with a single group (longitudinal design), or you can present your questionnaire once to two or
more groups (cross-sectional design).

17
While the former allows you to analyze how the questionnaire results of the group change over
time, the latter delivers insights into differences among groups.

#3: Develop questions

Smart questions are the cornerstone of every questionnaire. To make them work, they have to be
phrased in a way that prevents any misunderstandings or ambiguities. Quite frankly, it’s a lost
cause trying to analyze data from a questionnaire where people have mixed things up, selected
incorrect answers or haven’t been able to read or understand the questions at all.

It makes a significant difference whether you want to hand a questionnaire to children, adults, or
maybe even elderly respondents. It’s important to consider the cognitive, attentional, and sensory
competencies of your target group – handing out long questionnaires with a huge amount of
questions in small letter print and complicated phrasing might be information (and visual) overload
for any respondent group.

Additionally, remember to avoid jargon or technical language – the text needs to be fully
understood by anyone completing the questionnaire.

#4: Choose your question type

There’s a wide variety in how to phrase questions. In explorative questionnaires, you will find
mainly open questions, where respondents can fill in any answer (this makes sense whenever you
try to gain an understanding of the topics associated with your research question).

By contrast, quantitative questionnaires primarily include closed-questions, which have been

predefined by the researcher either in form of multiple choice answers or rating scales (such as the
Likert scale).

Here’s one example:

Open question:

“What did you like about the webinar?”

18
Closed question:

“The webinar was useful.”

[ ] Strongly agree

[ ] Agree

[ ] Cannot decide

[ ] Disagree

[ ] Strongly disagree

As is usually the case, both types of questions have benefits and drawbacks that are worth
considering in order to come up with a solid questionnaire design that does the trick for you.

Besides open and closed-format questions, there are several other types of questions that you can
use in your questionnaire.

#5: Design question sequence and overall layout

After optimizing each question separately it is time to improve the overall flow and layout of the
questionnaire.

Are there transitions from one question to the next? Are follow-up questions placed correctly? Are
skip-rules implemented (if needed) so that respondents can skip questions that do not apply to
them?

#6: Run a pilot

This stage is crucial for evaluation and optimization purposes. Any questionnaire should be handed
to a representative sample of your target audience before you push it to the masses.

During piloting, you can identify issues in readability and understanding, in phrasing and overall
arrangement. Pilot respondents should be monitored and interviewed closely.

19
You certainly want to avoid any inappropriate or problematic questions. Also, keep in mind to
evaluate your pilot data statistically to make sure that the analytic procedures of interest truly can
be applied to the data.

I hope this post helps you set out your questionnaire or survey design. If you’d like to learn more
about the fundamentals of experimental design, then download our free guide below.

COURSE: WEEK 7&8

Q5. What is the difference between Population and Sample size, and Sampling procedure?

Answer:

Population vs sample

A population is the entire group that you want to draw conclusions about.

A sample is the specific group that you will collect data from. The size of the sample is always
less than the total size of the population.

In research, a population doesn’t always refer to people. It can mean a group containing elements
of anything you want to study, such as objects, events, organizations, countries, species,
organisms, etc.
20
 Population vs sample

Population Sample

Advertisements for IT jobs in The top 50 search results for advertisements for IT jobs in the
the Netherlands Netherlands on May 1, 2020

Songs from the Eurovision Winning songs from the Eurovision Song Contest that were
Song Contest performed in English

Undergraduate students in the 300 undergraduate students from three Dutch universities who
Netherlands volunteer for your psychology research study

All countries of the world Countries with published data available on birth rates and GDP
since 2000

Collecting data from a population

Populations are used when your research question requires, or when you have access to, data from
every member of the population.

Usually, it is only straightforward to collect data from a whole population when it is small,
accessible and cooperative.

Example: Collecting data from a population. A high school administrator wants to analyze the
final exam scores of all graduating seniors to see if there is a trend. Since they are only interested
in applying their findings to the graduating seniors in this high school, they use the
whole population dataset.

For larger and more dispersed populations, it is often difficult or impossible to collect data from
every individual. For example, every 10 years, the federal US government aims to count every

21
person living in the country using the US Census. This data is used to distribute funding across the
nation.

However, historically, marginalized and low-income groups have been difficult to contact, locate
and encourage participation from. Because of non-responses, the population count is incomplete
and biased towards some groups, which results in disproportionate funding across the country.

In cases like this, sampling can be used to make more precise inferences about the population.

Collecting data from a sample

When your population is large in size, geographically dispersed, or difficult to contact, it’s
necessary to use a sample. You can use sample data to make estimates or test hypotheses about
population data.

Example: Collecting data from a sample You want to study political attitudes in young people.
Your population is the 300,000 undergraduate students in the Netherlands. Because it’s not
practical to collect data from all of them, you use a sample of 300 undergraduate volunteers from
three Dutch universities – this is the group who will complete your online survey.

Ideally, a sample should be randomly selected and representative of the population.

Using probability sampling methods (such as simple random sampling or stratified sampling)
reduces the risk of sampling bias and enhances both internal and external validity.

If your research is less concerned with generalizability, you can also use non-probability sampling
methods. Non-probability samples are chosen for specific criteria; they may be more convenient
or cheaper to access. Because of non-random selection methods, you can’t make valid statistical
inferences about the broader population.

Reasons for sampling

• Necessity: Sometimes it’s simply not possible to study the whole population due to its size
or inaccessibility.
• Practicality: It’s easier and more efficient to collect data from a sample.

22
 • Cost-effectiveness: There are fewer participant, laboratory, equipment, and researcher
costs involved.
• Manageability: Storing and running statistical analyses on smaller datasets is easier and
reliable.

Sampling procedure

Sampling is a process or technique of choosing a sub-group from a population to participate in the

study; it is the process of selecting a number of individuals for a study in such a way that the
individuals selected represent the large group from which they were selected (Ogula, 2005).

Q5. What is Sampling? Also discuss Probability and Non-Probability Sampling techniques
along with its main types.

Answer:

Sampling is a process or technique of choosing a sub-group from a population to participate in the

study; it is the process of selecting a number of individuals for a study in such a way that the
individuals selected represent the large group from which they were selected (Ogula, 2005).
Choosing part of a population to use to test hypotheses about the entire population. Used to choose the
number of participants, interviews, or work samples to use in the assessment process. First, we will
examine how sample is selected and the differences between a probability sample and a non-probability
sample.

Probability Sample Vs Non-Probability Sample

23
There are two main methods of sampling:

1. Probability sampling techniques and;

2. Non-probability sampling techniques

In probability sampling, respondents are randomly selected to take part in a survey or other mode
of research. For a sample to qualify as a probability sample, each person in a population must have
an equal chance of being selected for a study, and the researcher must know the probability that
an individual will be selected.

Probability sampling is the most common form of sampling for public opinion studies, election
polling, and other studies in which results will be applied to a wider population. This is the case
whether or not the wider population is very large, such as the population of an entire country, or
small, such as young females living in a specific town.

Non-probability sampling is when a sample is created through a non-random process. This could
include a researcher sending a survey link to their friends or stopping people on the street. This
type of sampling would also include any targeted research that intentionally samples from specific
lists such as aid beneficiaries, or participants in a specific training course. Non-probability
samples are often used during the exploratory stage of a research project, and in qualitative
research, which is more subjective than quantitative research, but are also used for research with
specific target populations in mind, such as farmers that grow maize.

24
Generally speaking, non-probability sampling can be a more cost-effective and faster approach
than probability sampling, but this depends on a number of variables including the target
population being studied. Certain types of non-probability sampling can also introduce bias into
the sample and results. For general population studies intended to represent the entire population
of a country or state, probability sampling is usually the preferred method.

Types of Probability Sampling

25
There are several sampling methods that fall under probability sampling. In each method, those
who are within the sample frame have some chance of being selected to participate in a study. Four
of the common types of probability sampling are:

Simple Random Sample: The most basic form of probability sampling, in a simple random
sample each member of a population is assigned an identifier such as a number, and those selected
to be within the sample are picked at random, often using an automated software program.

Stratified Random Sample: A stratified random sample is a step up from complexity from a
simple random sample. In this method, the population is divided into sub-groups, such as male and
female, and within those sub-groups a simple random sample is performed. This enables a random
sample that is representative of a larger population and its specific makeup, such as a country’s
population.
Cluster Sample: In cluster sampling, a population is divided into clusters which are unique, yet
represent a diverse group – for example, cities are often used as clusters. From the list of clusters,
a select number are randomly selected to take part in a study.

Systematic Sample: Using a systematic sample, participants are selected to be part of a sample
using a fixed interval. For example, if using an interval of 5, the sample may consist of the fifth,
10th, 15th, and 20th, and so forth person on a list.

Types of Non-Probability Sample

In non-probability sampling, those who participate in a research study are selected not by random,
but due to some factor that gives them the chance of participating in a study that others in the
population do not have. Types of non-probability sample include:

Convenience Sample: As its name implies, this method uses people who are convenient to access
to complete a study. This could include friends, people walking down a street, or those enrolled in
a university course. Convenience sampling is quick and easy, but will not yield results that can be
applied to a broader population.

26
Snowball Sample: A snowball sample works by recruiting some sample members who in turn
recruit people they know to join a sample. This method works well for reaching very specific
populations who are likely to know others who meet the selection criteria.

Quota Sample: In quota sampling, a population is divided into subgroups by characteristics such
as age or location and targets are set for the number of respondents needed from each subgroup.
The main difference between quota sampling and stratified random sampling is that a random
sampling technique is not used in quota sampling; For example, a researcher could conduct a
convenience sample with specific quotas to ensure an equal number of males and females are
included, but this technique would still not give every member of the population a chance of being
selected and thus would not be a probability sample.

Purposive or Judgmental Sample: Using a purposive or judgmental sampling technique, the

sample selection is left up to the researcher and their knowledge of who will fit the study criteria.
For example, a purposive sample may include only PhD candidates in a specific subject matter.
When studying specific characteristics this selection method may be used, however as the
researcher can influence those who are selected to take place in the study, bias may be introduced.

27