EME 6208 June 8, 2014

IMA 2 Usability Testing Report

handsonbanking.org



Group 3
Alex DeLange & Haley DeBaker





2
Introduction

Handsonbanking.org is an educational website, created as a free public service by
Wells Fargo. It is a program developed for learners of all ages, from kids to seniors, who
want to take charge of their own finances and to reach personal goals. Hands on
Banking offers all the basic tools, skills, and information about money and finances
needed for an individual to become more educated.

Each age group is set up differently, however the material being delivered is appropriate
for each age level. Within the Kid and Teen modules, there is an assessment at the
end for the students to complete while the Young Adult, Adult, and Senior modules
provide all the necessary information that is more pertinent for the age group.

As a group, we conducted a usability test on 4 separate students between the grade
levels of 7
th
grade to 12
th
grade. Each student was observed in a classroom setting.
Using a wide age range allowed us to get usable information from 2 different sections,
Kids and Teens.

Methodology
Sessions

The recruited students for this usability test were all observed in a classroom setting.
Before beginning the session, the students were advised to follow the directions
provided, try their best, and not to worry about making mistakes or being unable to
complete the assessment. Students were also asked to share their thoughts about the
website as they go through the process asked of them. Each individual session lasted
approximately 35 to 45 minutes.

Participants

Emily was a 7
th
grader at Pinellas Preparatory Academy; Nick was a 10
th
grader at Leto
High School; Lacy was an 11
th
grader from Leto High School; and Carla was an 11
th

grader from Leto High School. The tests were completed during the week of June 2,
2014. There were three (3) female participants and one (1) male participant.

Evaluation Tasks

There were five (5) tasks that we asked the students to complete or attempt to
complete:


3

1. Click on appropriate age level, kids or teen
2. Click on, then listen to and view the introduction module
3. Click on You and Your Money module and follow the prompts to
complete the activities
4. Click on the map icon to return to the main menu
5. Click on the Assessment module and follow the prompts to complete the
assessment. Disclaimer: Students should not expect to answer all
questions correctly. As they have not completed all of the modules. They
should just do their best and they should not feel bad if they get the
answers wrong.

Students were timed as they went through each of the five tasks. After all the tasks
were completed, each student answered an eight-question questionnaire on the ease of
use, navigation, and learnability.

Results
Task Completion Success Rate
All participants successfully completed Task 1 (select the appropriate age). All of the
four (100%) completed Task 2 (click on and listen to/watch the Introduction module). All
participants were able to complete Task 3 (click on and complete You and your money
module) and 100% were able to complete Task 4 (click on map to return to main menu).
None of the participants were able to complete Task 5 which required them to complete
an assessment for all modules. This was extremely difficult for all participants because
they had not completed all modules.
Participant Task 1 Task 2 Task 3 Task 4 Task 5
1 -
2 -
3 -
4 -
Success 4 4 4 4 0
Completion
Rates
100% 100% 100% 100% 0%


4

Learnability: Time taken to successfully complete task
Task 1 required participants to choose the appropriate age level on the homepage. The
mean time for this task was .57 minutes, with completion times ranging from 3 seconds
to one minute.
Task 2 required participants to click on and complete the Introduction module (mean
time = 3.5 minutes, with completion times ranging from two minutes to five minutes.
Task 3 required participants to click on and complete the You and your money
module. This module had an average completion time of 16.5 minutes, with times
ranging from 12 minutes to 20 minutes.
Task 4 required participants to click on the map button to return to the main page. This
task had a mean completion time of 4.5 seconds, with completion times ranging from 3
seconds to eight seconds.
Task 5 required participants to complete the assessment for all modules (mean time =
22.25 minutes). However, completion times ranged from 17 to 28 minutes.
Time on Task (in minutes)
P1 P2 P3 P4 Total Average
Task 1 0.03 1 0.5 0.75 2.28 0.57
Task 2 5 2 4 3 14 3.5
Task 3 12 18 16 20 66 16.5
Task 4 0.03 0.03 0.04 0.08 0.18 0.045
Task 5 17 18 28 26 89 22.25

Errors

The task in which participants made the most errors was Task One. It was difficult for
participants 1, 2, and 3 to determine which age range they should choose. However,
this was not a critical error, as it did not prevent them from successfully completing the
task.






5
P1 P2 P3 P4 Total
Task 1 X X X 3
Task 2 X 1
Task 3 X 1
Task 4 0
Task 5 X X X 3

Satisfaction

As indicated in the chart below, most questionnaire responses fell between Very Easy
or Easy. All 4 participants were satisfied with the website they were asked to test.
All four participants found completing the assessment to be Hard or Very Hard.
However, it is important to note that participants dissatisfaction with this portion of the
website was directly related to the fact that they had not completed all of the modules.
The tasks that participants found to be the easiest were finding the map to return to the
main menu and finding the assessment section; both of these tasks earned 3 Very
Easy rankings and one Easy ranking.
Very
Easy
Easy Not Easy
or Hard
Hard Very
Hard
Find appropriate age level 2 2
Find introduction section 2 2
Find You and Your Money
section
2 2
Complete You and Your
Money section
1 1 2
Navigating through You
and Your Money section
1 3
Find the map to return to
main menu
3 1
Find the Assessment
section
3 1
Complete Assessment
section
2 2


6

Recommendations
Task one asked the participant to select the appropriate age range- kid or teen.
Change Justification Severity
Make age ranges more
apparent for kids teens
and young adult sections.
Correct the grade range for
the teen section.
Several participants at the high school level were
not able to tell if they should choose the teen
section or the young adult section. The distinction
between the two was ambiguous. According to
the website, the teen section is in fact for 6
th

through 8
th
graders and the young adult section is
for high-schoolers. This is confusing because high
schoolers are teens and many middle schoolers
are not.
Medium
Task three asked the participant to click on and complete the You and your money
module.
Change Justification Severity
Highlight/underline new
vocabulary words used in the
module.
Link the highlighted
vocabulary words to the
glossary so that if the
participant hovers over them
a definition appears.
Include a description of the
tools available (including
glossary and calculator) in
the Introduction module.
Participants didnt understand much of the
higher-level vocabulary words used in the module
and struggled to use the glossary (or didnt know
that it existed.) Additionally, participants did not
know how to find the calculator and other tools
that would have made it easier to complete the
module.
Medium


Conclusion
Most of the participants found handsonbanking.org to be well-organized, informative,
and user-friendly. Handsonbanking.org is an excellent way for people of all age groups
to become financially literate. The website is especially useful for instructional use at
the secondary level. The modules are interactive and easy to navigate for students.
Additionally, instructional guides are available for teachers. Implementing the
recommendations and continuing to work with users (i.e., real lay persons) will ensure a
continued user-centered website.



7
Website Perceptions

Haley DeBaker

Learnability: In terms of learnability, handsonbanking.org is fairly easy for a novice
user to figure out. My test user, a 7
th
grader, was able to navigate the map and
modules without much instruction. She was also able to use the volume control to adjust
the volume as needed, again without instruction. The Introduction module was an
excellent opening activity to familiarize students with the program (however, even
without this module, I think students would easily be able to navigate the site). My test
subject is admittedly very bright, but I believe that students of varying ability levels will
have no problems navigating the site due to its interactive nature. The flow of the site
was very apparent- along the way, users are instructed where to click next. The only
complaint that I have in this category is that students are not instructed as to the
location or use of the tool menu or glossary. My test user Emily looked for a calculator
to use on a word problem and could not find it. My suggestion would be to make those
menus more apparent (perhaps move them to the top of the page) and mention them in
the introduction module. Because of this issue, I rate handsonbanking.org a 19/20 in the
learnability category.

Errors: The first error that was made was not the users fault, nor was it the websites
fault. The network was weak and the internet connection was lost. The user did not get
all of the questions correct during the assessment; much of this can be explained by the
fact that she did not complete all of the modules. None of these errors were catastrophic
or detrimental to the student successfully using the website. Emily had no problem
finding and clicking on the correct icons to continue through the modules. I would rate
the website 20/20 in the error category.

Satisfaction: Overall, the user was satisfied with the website. When I asked Emily if
there was anything she would change about the website to make it better, she couldnt
think of anything. Emily liked that the website was detailed. She also liked the map
page and thought it was easy to navigate. According the questionnaire that Emily filled
out right after our testing session, she found that it was very easy to find the
appropriate age level, find the You and your money section, find the map button, and
find the assessment section. Additionally, she found that it was easy to find the
introduction section, complete the You and your money section and navigate through
the You and your money section. The only item that Emily gave a low rating was
completing the assessment section. She rated this as difficult, but I believe this was
due to the fact that Emily had not completed all of the modules, rather than a deficiency
in the website. Overall, I would give the website an 18/20 in the satisfaction category.



8

Alex DeLange

Learnability: I felt that handsonbanking.org is an excellent way to teach learners how
to properly handle their money from a young age. The use of animation, text, and audio
is key to grabbing the attention to the new generation of learners. All my participants
were high school age and had no problem navigating through the site. The site
appeared to be very user friendly, with a few exceptions like the correct age range
between teen and young adult as well as the tool bar at the bottom, which consisted of
the glossary, tools, and contents. The students were asked to complete the assessment
at the end of the first module, but felt this was very difficult for them since they did not
go through each of the additional four modules. I would rate the website 20/20 on the
quality of the content and ease of navigation.

Errors: Each student navigated through the website seamlessly, but did come across a
few errors. Due to the fact that my participants were anywhere from 15-17 years of age,
they are considered teens, however were unsure if they should enter the teen or the
young adult module. They are at an age where they could be labeled as a young adult,
so I feel that the site could indicate the actual age ranges for each module. Another
small area of difficulty the students had was finding the glossary. There were many
financial words, such as discretionary or the rule of 72, that they did not know the
meaning to. This could easily be fixed if users could hover over certain words and the
definition would pop up instead of having to locate the glossary tool and open a new
window. I would rate the website 18/20 on errors.

Satisfaction: Overall, the students seemed pleased with the website and even learned
new information from the You and Your Money section. I feel that if they completed all
6 modules and took the assessment they would learn a lot and use what they learned in
their daily lives. Based on the questionnaires the students completed, they all stated
that navigating through the site and understanding the You and Your Money section
were easy to complete. I believe the use of all the audio, text, and animations were all
essential in captivating the audience in each respective age ranges. I would rate the
overall website 19/20 on satisfaction.


Observation Form

Participant Name: Emily A. (7
th
Grader)

Observer Name: Haley DeBaker


9

Observer Script:

Hi, Emily A. Thank you for taking the time to volunteer today to help me test out this
website called www.handsonbanking.org.

Were trying to see how the website works and if we can make it any easier or more fun
for students like you to use. Dont worry about making mistakes. Just have fun and try
your best to follow the directions given to you. Please share any thoughts as you are
visiting the site.


Tasks Start
Time
End
Time
Errors Comment
1. Click on
appropriate age
level, kids or teen.
9:45am 2
seconds
None She had no problems with
this.
2. Click on, then
listen to and view
the "introduction"
module
9:45am 9:50am Yes When the map page
opened, it was blank.
There were no icons.
Emily was confused and
tried to click on the map
icon. After sitting for a
minute or two, I stepped in
and determined that we
had lost our internet
connection (which
unfortunately happens
frequently at my school). I
disconnected and then
reconnected the internet.
After that, the page was up
and working with all of the
icons and Emily had no


10
problem locating the
Introduction module. The
error here was with our
internet connection. There
was not a problem with the
website or the user.
During the stage, Emily
located and used the
volume tool to adjust the
volume without prompting.
3. Click on, "You
and Your Money"
module and follow
the prompts to
complete the
activities.
9:50am 10:02am Yes Emily was a little confused
during the word problems.
She asked is there a
calculator? She then
pulled out her own
calculator. Emily didnt
seem to notice the tools
icon along the bottom of
the screen; if she would
have clicked on that, she
would have found a
calculator to use. Emily
was easily able to follow
along with the module by
clicking the next lesson
button and got all of the
word problems correct.
Emily also pulled out a
piece of scratch paper to
use for the word problems.
Something I noticed about
this module was that there
was no instruction on how
to complete the word


11
problems; students needed
to rely on their background
mathematics knowledge to
complete the problems.
4. Click on the
"map" button to
return to the main
menu.

10:02am 2 sec. None Emily had no problems
doing this.
5. Click on the
"Assessment"
module and follow
the prompts to
complete the
assessment.
Disclaimer:
(Students should not
expect to answer all
questions correctly,
as they have not
completed all of the
modules. They
should just do their
best and they
should not feel bad
if they get the
answers wrong.)
10:02am 10:19am None Even though a calculator
tool was up on the screen
for many of the problems,
Emily continued to use her
own calculator. Emily
really struggled on the
assessment and made the
comment this makes me
feel like a fail. I reassured
her again that it didnt
matter if she got the
questions wrong and she
should not expect to get
them all right since she
didnt complete all of the
modules. Emily was able
to successfully use the
credit calculator tool that
was provided for a few
questions. Emily got
several questions wrong,
and then read the hint(s)
and tried again.
Sometimes the hint(s)
helped and she got the


12
question correct the
second time; sometimes
she still got the question
incorrect the second time.
If she got the question
wrong the second time, the
quiz continued on to the
next question. Emily
audibly sighed during the
assessment; I could tell
that she was getting
frustrated. I feel that her
frustration was not with the
website, but rather with not
knowing how to answer the
questions. Emily is a very
good student and it really
bothered her to get
questions wrong. We
ended the test when we
got to the end of the
assessment; Emily did not
pass. She earned a 6/11
(53%). A 70% was needed
to pass.


Questionnaire from Emily A. (7
th
Grader)
Observer: Haley DeBaker


1. How difficult was it to find the appropriate age level?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard


13

2. How difficult was it to find the introduction section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

3. How difficult was it to find the You and Your Money section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

4. How difficult was it to complete the You and Your Money section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

5. How difficult was it to navigate through the You and Your Money section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

6. How difficult was it to find the map button to return to the main menu?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

7. How difficult was it to find the Assessment section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

8. How difficult was it to complete the Assessment section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

Observation

Participant Name: Nick

Observer Name: Alex DeLange

Observer Script:



14
Hi, Nick. Thank you for taking the time to volunteer today to help me test out this
website called www.handsonbanking.org.

Were trying to see how the website works and if we can make it any easier or more fun
for students like you to use. Dont worry about making mistakes. Just have fun and try
your best to follow the directions given to you. Please share any thoughts as you are
visiting the site.


Tasks Start
Time
End
Time
Errors Comment
1. Click on appropriate age
level, kids or teen.
8:17 8:18 Yes Not sure what level
to choose if students
are teen. Choose
teen or young adult?
2. Click on, then listen to
and view the "introduction"
module
8:18 8:20 0 Easy to find
3. Click on, "You and Your
Money" module and follow
the prompts to complete the
activities.
8:20 8:38 0 Was a little confused
whether to listen to
the audio or read the
text since she talked
fast
4. Click on the "map" button
to return to the main menu.

8:40 8:40 0 Easy to find
5. Click on the
"Assessment" module and
follow the prompts to
complete the assessment.
Disclaimer: (Students
should not expect to answer
all questions correctly, as
they have not completed all
8:40 9:02 Yes It was way too
difficult for him. He
needed to use a lot
of hints to complete
and didnt know what
some of the words
meant.


15
of the modules. They should
just do their best and they
should not feel bad if they
get the answers wrong.)

Questionnaire from Nick

1. How difficult was it to find the appropriate age level?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

2. How difficult was it to find the introduction section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

3. How difficult was it to find the You and Your Money section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

4. How difficult was it to complete the You and Your Money section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

5. How difficult was it to navigate through the You and Your Money section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

6. How difficult was it to find the map button to return to the main menu?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

7. How difficult was it to find the Assessment section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

8. How difficult was it to complete the Assessment section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard


16

Observation

Participant Name: Lacy

Observer Name: Alex DeLange

Observer Script:

Hi, Lacy. Thank you for taking the time to volunteer today to help me test out this
website called www.handsonbanking.org.

Were trying to see how the website works and if we can make it any easier or more fun
for students like you to use. Dont worry about making mistakes. Just have fun and try
your best to follow the directions given to you. Please share any thoughts as you are
visiting the site.


Tasks Start
Time
End
Time
Errors Comment
1. Click on appropriate age
level, kids or teen.
8:55 8:56 yes Not sure what level
to choose if students
are teen. Choose
teen or young adult?
2. Click on, then listen to
and view the "introduction"
module
8:56 9:00 0 Easy to do
3. Click on, "You and Your
Money" module and follow
the prompts to complete the
activities.
9:00 9:16 0 Helped student
understand how to
save money and
figure out how banks
work.
4. Click on the "map" button 9:16 9:17 0 Easy to find


17
to return to the main menu.

5. Click on the
"Assessment" module and
follow the prompts to
complete the assessment.
Disclaimer: (Students
should not expect to answer
all questions correctly, as
they have not completed all
of the modules. They should
just do their best and they
should not feel bad if they
get the answers wrong.)
9:17 9:45 yes Section was too
difficult for student,
especially since not
all modules were
completed. Felt as if
he needed more
hints in order to
succeed.

Questionnaire from Lacy

1. How difficult was it to find the appropriate age level?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

2. How difficult was it to find the introduction section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

3. How difficult was it to find the You and Your Money section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

4. How difficult was it to complete the You and Your Money section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

5. How difficult was it to navigate through the You and Your Money section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard


18

6. How difficult was it to find the map button to return to the main menu?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

7. How difficult was it to find the Assessment section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

8. How difficult was it to complete the Assessment section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

Observation

Participant Name: Carla

Observer Name: Alex DeLange

Observer Script:

Hi, Carla. Thank you for taking the time to volunteer today to help me test out this
website called www.handsonbanking.org.

Were trying to see how the website works and if we can make it any easier or more fun
for students like you to use. Dont worry about making mistakes. Just have fun and try
your best to follow the directions given to you. Please share any thoughts as you are
visiting the site.


Tasks Start
Time
End
Time
Errors Comment
1. Click on appropriate age
level, kids or teen.
8:20 8:21 Yes She liked the topic it
was about, but didnt
know which was the


19
appropriate age for
her.
2. Click on, then listen to
and view the "introduction"
module
8:21 8:24 0 She liked that it
explained what she
was about to learn.
3. Click on, "You and Your
Money" module and follow
the prompts to complete the
activities.
8:24 8:44 0 She thought it was a
little hard to complete
in some areas.
4. Click on the "map" button
to return to the main menu.

8:44 8:45 0 Easy to find
5. Click on the
"Assessment" module and
follow the prompts to
complete the assessment.
Disclaimer: (Students
should not expect to answer
all questions correctly, as
they have not completed all
of the modules. They should
just do their best and they
should not feel bad if they
get the answers wrong.)
8:45 9:11 Yes She felt it was way
too hard for her since
she didnt complete
the other modules.
She did however feel
that it helped her
learn more about
credit cards and
income.

Questionnaire from Carla

1. How difficult was it to find the appropriate age level?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

2. How difficult was it to find the introduction section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard



20
3. How difficult was it to find the You and Your Money section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

4. How difficult was it to complete the You and Your Money section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

5. How difficult was it to navigate through the You and Your Money section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

6. How difficult was it to find the map button to return to the main menu?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

7. How difficult was it to find the Assessment section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard

8. How difficult was it to complete the Assessment section?
1 Very easy 2 Easy 3 Not easy or hard 4 Hard 5 Very hard