Can the Wording of a Question

Create Response Bias?

By: Kimberly Liang & Jessica Tam
Introduction:
Will high schoolers at Abraham Lincoln High School care about
ordering the “classic tapioca pearls” topping alongside a boba
drink, if they knew the tapioca contain high-risk cancer
chemicals?
Reason for choosing this topic:
-This was the first topic that
came into our mind (for both of
us) so we felt a connection/fate
to do this topic.

-Boba is a huge part of our lives,

especially towards the majority
of teenagers our age so we were
curious to see if there were any
downsides to consuming it.
SURVEY QUESTIONS:
Unbiased: Would you prefer a boba drink with or without the classic tapioca pearls?

☐ Boba with classic tapioca pearls

☐ Boba without classic tapioca pearls

Biased: Researchers from nutritional and medical studies have reportedly found
cancer-causing chemicals in the classical tapioca pearls that can be ordered as a
“topping” in a boba drink. Considering this information, would you prefer a boba drink
with or without the classic tapioca pearls?

☐ Boba with classic tapioca pearls

☐ Boba without classic tapioca pearls

HYPOTHESIS:
We predict that the biased question will
yield more “without” responses because
we believe that if students knew about the
risks of consuming tapioca pearls, there
will be significantly more students who
pick “without tapioca pearls” as there is
provided information that supports one
side of the question but not the other,
which we believe will result in response
bias.
Methodology
1. Randomly generate 2 advisories regardless of grade (simple random sampling) using randInt on
the calculator(Ms. Gerek will do this). Grade is not important in this survey because all high
schoolers belong to the same generation, so their answers to health questions will not vary
significantly enough to affect their answers, especially when the question poses only two possible
answers that are complete opposites.
2. In the first advisory chosen, the unbiased survey question will be given out to that class. The
second advisory chosen will be presented the biased survey question. This is decided ahead of
time.
3. Print out the 2 types of survey questions onto identical sheets of paper and cut them out into
identical sizes.
4. Go to the advisories ahead of time to schedule an appointment on Block Day, Wednesday, 11-1-
17.
5. Go to the advisories on the scheduled date in the same order as they were chosen.
6. Pass out the survey questions to each respective class.
7. Collect all the survey questions, including the non-responses.
Effectiveness:
- Used the same greeting to both classrooms in order to prevent
unintentional bias
- Advisories were chosen randomly, so every advisory had an equal chance
of being chosen
- Survey questions given to which advisories were decided ahead of time
- Survey questions were written with identical font, size, color, and
wording and were printed and cut into identical sizes of paper.
Results:
With Tapioca Without Tapioca Non-Response Total
Non- Pearls Pearls
Biased:
Number of Votes 11 4 5 20

Total 55% 20% 25% 100%

With Tapioca Without Tapioca Non-Response Total

Biased Pearls Pearls

Number of Votes 4 7 4 15

Total 26.67% 47.66% 26.67% 100%

Results:
Conclusion:
We can conclude that the wording of a question can result in response bias, due to our
data which shows a statistically significant difference between the responses to the
biased survey question and the unbiased survey question. Our biased question was
expected to yield more “without” responses, and our hypothesis was discovered to be
correct, as three more students chose “without” in the biased question than in the non-
biased question. Furthermore, our study shows that in a non-biased question, a large
majority of students do indeed prefer a boba drink “with” tapioca pearls, therefore we
can conclude the change in opinion must be the result of the wording of the question
that persuaded the students to choose otherwise. However, there were a fairly large
number of non-responses for the biased question that could have potentially changed
the overall results significantly. Thus, we plan for our future study to have a larger
sample size in order to yield more accurate results.