Peer Review 1

To: Media Service Director

From: Concerned Statistics Student
Date: 10/11/2018
Topic: Data Analysis of Website Navigation

This study is an experiment because instead of simply observing a group and monitoring what they do, we are
bringing in people and we are having them complete a task. We are involved. It is also a randomized block because
the subjects were divided into groups: faculty, staff, and student. Then the people from these groups were randomly
selected and placed in either instruction set 1 or instruction set 2. There were three types of variables in this study.
They are: Participant Type, Instruction Set, and Time. Participant type is where the subjects were divided into
faculty, staff, or student. This type of variable is categorical nominal and it is used to determine how the instructions
differ among different ages and demographics. Instruction Set is where those participants were divided into written
or illustrated instructions, labeled as Instruction set 1 and 2 and it is used in the study to determine if there is a faster
way to give instructions.. This variable is categorical nominal as well because there is no order to the instruction
sets. Time is how long it took the participants to complete the task given the instruction set. This variable is
quantitative because time can be averaged and it is used in the study to determine which instruction set is faster.
I would choose to use a bar graph because bar graphs are used to compare variables. I would make two
separate bar graphs. These two would differ by instruction set. The X axis would be the participant type and the Y
axis would be the time it took them to navigate the web page. I would do this so you can look at each bar graph and
have an easier time finding what the average time it took participants to navigate the page with instruction set 1 vs
how long it took them with instruction set 2, thus answering the question of the study: which instruction set is better.
One thing that might influence the study would be a response bias. Since participants were invited to
participate it is possible only people who believe they could properly navigate it would show up. Some confounding
variables of my study would be age or internet speed. Age could influence how fast someone may type or if they can
read what is on the screen. Internet speed could slow down response time because if one computer is operating
slower than another it will obviously change how long it takes someone to complete the task.

1)The setup is relatively clearly defined in your memo; however, it has a few mistakes that are more elaborated in
the second criterion response. I think the data analysis portion could be a little more clarified because the regular
person may not understand what a categorical nominal variable is.
2)You explicitly mention that it is an experiment; however, you provide an incomplete justification. It could be
improved by adding "via two different treatments to the very end of the first sentence.

3) I didn't take your approach but you are able to justify why a bar graph is a good way to look at the data. I was
initially confused about how the bar graph would be set up but you provide the necessary justification. However,
you don't explicitly incorporate how the two different treatments are graphed along with each of the blocks. (faculty,
student, and staff)

4)The confounding variable section is very well developed with your discussion of internet speed and age being
possible culprits. However, I am not completely sure if it is appropriate to characterize response bias as a
confounding variable. I wasn’t sure where to place the following comment but here it is: Your formatting seems a
little off, and I highly recommend that you check out
(https://owl.purdue.edu/owl/subject_specific_writing/professional_technical_writing/memos/sample_memo.html).
Peer Review #2
To: Media Services Director
From: Student Intern
Date: October 12, 2018
Subject: Proposed Analysis Plan Of Media Usability Study
The type of study performed was an experimental study and the design of the study was a randomized block design.
The blocks consisted of Student, Staff, and Faculty, while the groups within the blocks were participants given
instruction set 1 and participants given instruction set 2. There were 3 variables recorded, participant type,
instruction set provided, and time to complete. Participant type was a categorical nominal variable and served as the
major blocks of the study, as it was thought the 3 different participant types would respond differently to the type of
instructions received. Instruction set provided was a categorical ordinal variable and was randomly assigned within
each participant type to measure the time it took for members of each participant type to complete the task with the
instructions they were provided. Time to complete was a quantitative continuous variable that was the output based
on the participant type and instruction set provided to said participant type. With all of this in mind, I believe it
would be appropriate to summarize the data with 6 histograms. There would be a histogram for each participant
type, followed by the instructions et they were provided, i.e. students with instruction set 1, students with instruction
set 2, staff with instruction set 1, etc.. The x-axis for all of the histograms would be time to complete the task given
by the instruction set in minutes and the y-axis would be the number of participants. A five number summary could
then be taken of each of the observed groups to make quick comparisons between the groups. Doing this, we could
measure all the variables that were observed, however, there may be other factors that affected the results of the
survey. Participant type and instruction set provided serve additionally as explanatory variables to time to complete,
which is also the response variable, and there may be confounding variables affecting time to complete. Considering
the instructions are for how to navigate a website to reserve media rooms and equipment, I can imagine a possible
confounding variable being the technical abilities of the participants in the study. There may be participants of any
type that lack proficiency with computers, so they may take longer to complete their task, and the instruction set
those participants have to work with may potentially make that lack of proficiency an even greater factor in how
long it takes. While this may be a possibility, these participants would likely be outliers in the data, and these
outliers would be noticed in my proposed analysis plan.
1) The memo is very straightforward and clear, but it seems highly dense because of there being only one
paragraph. Separating into multiple paragraphs would help the reader (Media Services Director) better
understand the flow of your thoughts and pull out the important details. The confounding variable section is
very clear, justified.
2) You get off to a great start by mentioning that it is an experiment and you justify it later, by mentioning the
2 different sets of instructions. You very clearly mention what are the response variable and explanatory
variables. There appears to be enough detail so the media director will generally understand these details.
3) Something that could be improved is the lack of explicit mentions of how the data would be compared
using histograms. You don’t say why histograms would be useful in comparison. Additionally, I think it
would be easier for you to switch to boxplots because you place more importance on the five-number
summary than the histogram. They generally align well with the comparisons to be made and the primary
study question.
4) Your confounding variable of computer proficiency seems extremely plausible, and your explanation of it
makes it very easy for the reader to understand why that may influence the response variable.
 TO: Media Services Director
FROM: Student Intern
DATE: October 12, 2018
SUBJECT: Media Website Usability Study Data Analysis Plan
This memorandum proposes a data analysis plan to interpret the results from the Media Website
Usability Study to answer the primary study question of whether the type of instructions and type
of participants have an effect, on average, on the time to complete the task.
Study Type and Variables Used
The utility study used an experiment with a randomized block design to obtain the results. It was
an experiment as participants were manipulated in some form through being assigned a set of
instructions (written-only or illustrations-included). It was of a randomized block nature as
participants were first grouped based on the type of user they were (student, faculty or staff) and
then randomly assigned to one of the instruction types.
The time taken to perform the reservation task is the outcome of interest and therefore the
response variable. It is quantitative as it is numerical and arithmetic can be completed using it,
and it is continuous as there is an infinite amount of values it can be. The set of instructions and
the type of user are both explanatory variables as they both may affect the response variable.
They are also both categorical and nominal as they are not numerical and have no particular
ordering to them.
Method to Summarise Data
To optimally address the comparisons needed to answer the primary study question, side-by-side
box plots should be used as the graphical summary as they will display the quantitative variable
according to the categorical variables. These plots will visually show the five-number summaries
for the categorical variables and any existing outliers, allowing for easy comparisons between the
task completion times. The five-number summary, comprised of the minimum, first quartile,
median, third quartile, and maximum of the dataset, should be used as the numerical summary as
it eliminates outliers and provides effective information for the centre and spread of the outcomes
and the differences between them when sorted by types of instructions or participants.
One notable disadvantage of boxplots is that it does not show the shape of the distribution of the
data. If this is desired, histograms can be used in addition to the side-by-side box plots.
Confounding Variables
A potential confounding variable in this study is how comfortable the users are with using
technology. If users are more comfortable and experienced in making reservations online, this
could result in a shorter reservation task completion time relative to a user who is inexperienced
in using technology. This means that it is both an explanatory variable and affects the response
variable. This confounding variable should be measured if future usability studies are to be
conducted.
Statistical Tests
Confidence interval estimation and hypothesis testing could be conducted on the results to
determine accuracy of the results and the extent of the differences between the types of
instructions and participants on the task completion time.

1) I love how your memo is organized into sections. This makes it extremely easy for one to easily find the
information one might be looking for in your memo. One major fallacy of your memo is the technical level
you assume of the reader. I don’t think it would be appropriate to fail to explain confidence interval and
hypothesis testing when that is your proposed data analysis plan.
2) This portion of your memo was the strongest. Right from the get-go, you mention that the study conducted
is of experimental design (specifically randomized block), and kudos for providing the appropriate
justification. You also very explicitly mention what the variables are and of what types they are.
3) This section is also properly executed in your memo. However, the last section of your memo may be
overdoing it and confusing a reader with only basic statistical literacy. I love how you clearly define the
composition of each boxplot and how they can be used to make the appropriate comparisons.
4) You check all the boxes in this section as well. An individual’s comfort with technology could definitely
impact the response variable, and you show how. On an ending note, try to make sure that people who have
never taken any statistics classes can also understand your memo.