EDPS 6440: Human-Computer Interaction

Dr. Kirsten Butcher

HCI Exercise #5: Results from Initial User Testing
Design Challenge
Currently, Salt Lake Community College (SLCC) has no job task training for new incoming
academic administrative assistants. When a new administrative assistant is hired, they are not
provided with any formal training of essential job functions. They often have to scour websites,
contact veteran personnel, as well as fragmented resources to find relevant information to
complete their job. This current ‘system’ is time consuming, ineffective, and often leads to
incorrect completion of tasks and frustration. Effective training on centralized processes (e.g.,
scheduling courses, budget management) is needed for new admins as well as to serve as a
reference for existing admins.
Design Goal(s)
The goal of this project is to design a centralized and effective training for new incoming
academic administrative assistants at Salt Lake Community College on essential job tasks in
order to be in alignment with the overall mission and goals of SLCC. This training will be
available to all admins asynchronously.
Testing Summary
There were a total of 9 users who tested our lo-fi paper prototype. They were given the
following scenario for instructions:

“You are an academic administrative assistant at Salt Lake Community College. Your
boss has asked for a Budget Availability Report but you can’t remember how to generate
the report. You are at the admin training website to refresh your memory.”

The tasks were presented on paper and changed by a team member based on the choices the
user made.

System Task(s)
The test user was given one system task to complete:
● Navigate within the training site to the “Budget Availability Training” module
Findings from User Testing
· Summary of Positive Findings
Test users were positive about the visual representation of the prototype and one user,
User A, closely matches the demographic for the stakeholder and found they had very little
issue navigating to the training modules. Others also felt that the content they were looking for
was clearly marked, making it relatively easy to find what they were looking for.

Those who chose to log in without hesitation were comfortable to navigate past the login to find
the info they were tasked with finding.

When the users navigated to the training materials, the majority were sure that they were in the
correct place to start/complete the task from the scenario.

“And now this is exactly what I was
hoping to find. A step-by-step
instructions [sic] that I can open
up on my computer in a separate
window while I do the actual thing
itself.”
Test user A looks at the training module page and
decides that she can keep it open as a reference as she
executes the task in the scenario.

Summary of Usability Issues
1. Almost half of the test users seemed to struggle the most with the initial navigation of the
site. They were unsure of whether they should login or click directly on the link that they
thought contained the information to help them complete their task.
“I don’t really want to login, so I
wonder if I can just get some stuff
on Budget Availability without
logging in because I can never
remember my logins, or I just find it
annoying to have to do it, so let’s
see if I can just grab that. So,
would it be resources or would it be
Training Modules. I usually avoid
training modules if possible because
they are long, so I am just going to
see what Resources has for me.”
Test User B is unsure of where to go but chooses the screen
that makes the most sense to her, which is Resources. Upon review, for the user who would not need actual training,
just access to the content, putting a static version containing screenshots would be best under the Resources page.

2. The users who did not log in first, but were forced
to log in despite selecting a menu item, were
confused about why the action of logging in had lead
them back to the homepage and did not direct them
to the page they selected initially.
“This looks like the same screen I was
at. Ahhh. Why do I have to read all of
this? I don’t even know if I’m in the
right place yet. I should be in the
training modules by now. I’m a little
confused.”
Test User B is redirected to the homepage after a forced login.
She is unclear about whether or not this is the same page, and is unclear about why the page she selected did not
automatically load after the login.

3. Once on the actual module page, users were not sure
they were looking at the training because they felt that
the page was not clearly marked.
“I’m reading my instructions but I’m
confused about what I’m supposed to do.
I’m confused because there are 2
screenshots with instructions that look
identical. There are no instructions for
what to hit so I guess I’ll hit the main
screenshot.” - Test User I
Test User I is not sure that the screen she is looking at is the actual
module she needs.

4. The majority of the users also felt that they
had to click too many times to get to the content
they needed. Others were confused about how
to get to the tutorial itself from the homepage.
“This is getting really, there’s a
lot of pages for me to go through to
get to one module...I feel like I’ve
gone through 4 or 5 pages before I
got to my actual training.”
Test User I felt that the 3 screens post-login were too many
to have to go through to reach the actual training. Moving
forward, it will be a simple correction to allow users to
access the module after 1 or 2 screens.

Data Table of Usability Issues

Number of Times Observed By Category

Description of Usability
Issue
Navigation

Content

Clicked on “Training
Module” to begin
searching
Unsure of whether or
not to login first

3

Confused about why
the site returned to the
homepage after they
logged in

7

User Assumptions

3

Minor

1

Minor
Moderate

Selected “Resources”
to find information

2

Confused if the training
module page was the
actual training or not

3

Felt there were too
many clicks to get
where they needed to
be

4

Total Number Issues

14

Critical

Minor
Critical

Moderate

3

6

Overall, navigation was the biggest problem. There are several navigation issues that we need
to address in order to make the site more user friendly. Several test users were confused about
why they were on certain pages or how to get to the page they wanted.

Appendix: Notes from Test Data

The notes are color coded to group similar usability or positive findings together for ease of
compilation.
Test 1
Training module 1st - clicks on training
Not sure what the username and password would be
Not sure if the page went back to the homepage
Confusion about whether to click begin or the training
Not clear if the next page is actually the training - need to mark that this is the training

3

Test 2
Unclear if needs to login to the site
Back to homepage seems confusing
Stated that the report training was obvious to find
Terms confusing but found the one she needed easily
Confused if the training was a form or the training
Test 3
Not sure if should login first
Name is clearly marked
Was clear about training
Test 4
Assumes to hit training modules first
Confused about going back to the homepage - expected to go straight to the modules after
signing in
Distracted by words
Complaining about too many steps - cumbersome
Clicks words instead of begin
Confused about what the tutorial is - even though it is numbered
Test 5
Clicked training first
Assumed info already in
Confused about returning to the homepage
Redundant to have to click the page twice (begin)
Too many clicks to get to the training

Test 6
Word portal is confusing
Need to login?
Resources
Confused where to go - just wants to get to the meat
Not sure of credentials
Still unclear of where to go
Acronyms are confusing
Seems time consuming - not clear what kind of training
Title not consistent
Too many clicks
Page size might be an issue in case of shrink when viewed on mobile devices

Test 7
Logged in first
Hit resources first
Hit Directory - assumed the directory was of the trainings
Hit Resources again - assumed the resources would contain the info (partially correct)
Confusion about begin on the training page (too many clicks?)
Test 8
Login first
Assumes standard login and password
return to homepage confusing

Test 9
Login first
return to homepage confusing