CS6750 Assignment M4

Anna He
annahe@gatech.edu

Abstract— This series of M assignments seeks to investigate and

redesign an aspect of Strava: an exercise tracking and social me -
dia app. In particular, the investigation will deep dive and dis-
cuss the interface while the user is engaged in a physical activity
i.e., running or cycling. The goal of this project is to use findings
from the investigation to improve the efficiency and ease of ac-
cessing Strava metrics during an activity.

1 QUALITATIVE EVALUATION

1.1 Type of Evaluation and Participant Recruitment

This qualitative evaluation will be conducted on a paper prototype using sur-

veys. Some participants will be classmates and peers in the CS6750 course.
These participants will be recruited via the discussion board or on PeerSurvey
directly. Other participants will consist of Strava users that I am acquainted and
friends with. These participants I will recruit by directly interacting with and
asking if they are willing to submit a survey to help my research. If they agree,
I will provide a link to the survey.

In either case, survey participants will receive a PeerSurvey link where the eval-
uation will take place. Survey results will be recorded through PeerSurvey and
later analysed. There will be no recordings of the participants completing the
survey.

1.2 Survey Questions

The participants will be shown an image of the current Strava interface and the
paper interface. The paper interface can be found in the Appendix. The follow-
ing questions will be asked:

 What age group are you in?

 How clearly does this design resemble a lock screen? I.e., can you tell
whether a smartphone is unlocked based on this screen?

1
  Answer options: Yes it is very clear; Yes it is somewhat clear, Neutral it
is neither clear nor unclear; No it is somewhat unclear; No it is very un-
clear
 What metrics can you identify based on this screen?
 Answer options: Mileage/Distance, Time Elapsed, Elevation, Pace, Heart
Rate, Power
 How clearly are you able to identify the second screen as a map?
 Answer options: Very clearly; Somewhat clearly, Neither clearly nor un-
clearly; Somewhat unclearly; Not clear at all
 How clearly are you able to identify what button(s) to press to stop the cur -
rent Strava activity?
 Answer options: Very clearly; Somewhat clearly, Neither clearly nor un-
clearly; Somewhat unclearly; Not clear at all
 Is it clear how to navigate between the first screen with metrics and the sec-
ond screen with the map?
 Answer options: Yes it is very clear; Yes it is somewhat clear, Neutral it
is neither clear nor unclear; No it is somewhat unclear; No it is very un-
clear
 How would you compare this lock screen experience to the existing Strava
experience that requires unlocking your smartphone?
 Answer options: Much improved, Somewhat improved, Neither better
nor worse, Somewhat worse, Much worse

1.3 Revisiting Data Inventory and Requirements

The main requirement was Usability, i.e., for the interface to be able to convey
information without on the lock screen without unlocking the phone. Specifi -
cally, the user should be able to see metrics including time elapsed, distance
travelled, and speed/pace. This evaluation gauges whether the design meets the
interface by directly asking participants whether they could see necessary data
while on the lock screen. The survey also asks to compare this the prototyped
experience with the existing one.

2 EMPIRICAL EVALUATION

The prototype to be developed empirically is the voice activated prototype

where users can speak a keyword aloud and the Strava app will respond with a

2
corresponding metric. For instance, if the user says “Time” the app may re-
spond with "3 minutes and 40 seconds.”

2.1 Experimental Conditions

There is only one experimental variable. The experimental variable is the

amount of time it takes for the user to receive some activity data using the ex -
isting interface and comparing it to the prototyped voice-activated interface.
The control conditions include:

 the environment that the participant is in during a Strava activity (e.g. the
same route, during the same day)
 the type of activity the participant partakes in (i.e., running or cycling)
 the metrics the participant will try to access during the experiment will stay
consistent between experiments.

2.2 Null and Alternative Hypothesis

The null hypothesis is that there is no relationship between the method of re -

ceiving metric data during an activity and the amount of time to receive that
data. It will be accepted that both designs are equal if insufficient data is col -
lected about the alternative hypothesis.

The alternative hypothesis is that a voice activated design to retrieve Strava

data will allow users to access their data more quickly and with less cognitive
burden. Users will not need to unlock their phone or take their eyes off the
road. However, they may need to wear earphones to hear the response clearly
which make it more difficult for the user to perceive their environment.

2.3 Experiment Method

This experiment will be conducted within-subjects. Participants will be split

into two groups: those that test the existing interface first and those that test
the prototyped interface first.

Each participant will complete a short Strava activity (e.g. running or biking for
5 minutes) during which they will try to access specific metrics (Distance, Time
Elapsed, Pace) using one interface. Participants will say aloud when they begin
accessing the metric and speak again once they have succeeded. Then, partici-

3
pants will complete the same Strava activity and try to access the same metrics
but using the alternate interface.

During both experiments, participants will be voice recorded. This will gener -
ate data on the length of time required to access data using either method. One
potential lurking variable may be some general fatigue accumulated while en-
gaged in physical activity. For instance, it is possible that the length of time to
retrieve a metric may simply increase with physical fatigue. For this reason,
some participants will experiment with the prototype first, while others will ex-
periment with the existing interface first.

This length of time will be analyzed for a statistically significant difference. The
mean and medians of the lengths of time will be calculated and compared. If a
statistically significant difference between the two trials exists, then we may
conclude that one design is better than the other.

3 PREDICTIVE EVALUATION

For the predictive evaluation, I will create two GOMS models to compare the
efficiency between them. There will be a model of my haptic tapping prototype
and a model of the existing interface. In my haptic prototype, the user taps their
phone externally twice to stop and start their activity. The phone will then vi-
brate twice if the activity is being paused, or three times if it is being resumed.
The goal is to better understand the operators, methods, and selection rules that
are required to complete the user task.

I will evaluate the user accomplishing the two tasks of pausing and resuming a
Strava activity via a haptic interface. The specific user’s goal is to pause and re -
sume a Strava activity when they are temporarily stopped, e.g. for a coffee
break or to use the restroom. The emphasis of the goal is efficiency, and it is
presumed that the users are experienced users of the interface. The only opera -
tors will be externally tapping the back of their smartphone device. The user
will not have access to the visual interface that currently exists. The evaluation
will be based on accomplishing the two tasks described above without evalua-
tion of a user’s navigation around the interface.

4
4 PREPARING TO EXECUTE 0.5

The two evaluations to be completed are the Qualitative and Predictive evalua -
tions. Because it is still early in the design cycle, the purpose of the evaluation is
formative. Both Qualitative and Predictive evaluations are appropriate since the
interface remains low fidelity.

Without a prototype ready for empirical evaluation, it will not be conducted in

the next assignment.

5
5 APPENDIX

Figure 1—Strava lock screen widget as it would

appear on iOS and Android Lock Screens.