Greater Second Baptist Church

Website Test Plan

Usability Testing & Design - Spring 2019

For Dr. Joe Williams

By:
Cameron Handie
Lisa Hubbard
Leighana Wood
Kristina Yarberry
Table of Contents
Executive Summary

Introduction

Methodology
Planning Phase
Heuristic Analysis Data
Design of Questionnaires
Usability Testing Participants
Length of Sessions
Scheduling
Tasks & Scenarios
Description of the Testing Scenarios
Scenario 1:
Scenario 2:
Scenario 3:
Scenario 4:
Testing Equipment
Conducting the Test
Usability Test Plan for GSB Website

Findings

Conclusions

Appendices

A.
Table 1: Nielsen’s Ten Heuristics

B.
User Research
Pre-test survey

C.
Personas & Empathy Maps

D.
Test Plan:
E.
Moderator Script
Introduction
Videotaping Permission
Introduction of the Evaluation Room
Introduction of the Team
Introduction to the Test

F.
Scenario 1:
Scenario 2:
Scenario 3:
Scenario 4:
G.
Pre-test Survey questions

H.
Post-Task Survey 1
Responses:
Post-Task Survey 2
Responses:
Post-Task Survey 3
Responses:
Post-Task Survey 4
Responses:
Post-Task Survey 5
Responses:
Post-Task Survey 6
Responses:
Post-Task Survey 7
Responses:

I.
Post-Test Survey

K.
Forms
Executive Summary
The purpose of this project is to test the usability and functionality of the Greater Second Baptist
Church (GSBC) of Little Rock’s website. We have a member of the church on our team, who
first led us to the website to show us its shortcomings. Each member of the group performed a
heuristic analysis from a different user “perspective,” testing the entire site and observing its
usability. We each created User Personas as well, to give us an idea of the types of users we
need to consider for the actual test of the website later on.

We compiled a survey early in the planning process to give us an idea of the general consensus
of opinion on the website’s functionality. We received several responses, with the most common
result being that we were not clear enough of the purpose of the questions. Nearly all users
assumed that we built the website and therefore wanted to be “kind”. We all heard from the
people we personally knew that they thought the website was terrible but thought we were being
graded on the website, not the testing of the website. So, they were nicer than they would have
been had they understood the process. As a result, we decided to be more clear when
performing the full usability test that the assignment is the testing of the site, not the site itself.

We have moved forward in the planning stage to create this test plan, laying the groundwork for
the project as a whole and for our expectations of it. As the testing continues, so will the
progress of the test plan; it will change as needed according to our findings.

The next step is to perform the actual usability test. Each group member has been assigned to
recruit at least three test participants, all of whom will be required to complete a Video
Permission Form prior to testing. As each participant uses the Morae Recorder software we
provide to test the GSBC website, their progress will be recorded, and we will observe via a
second computer and make a moderator available for assistance as needed.

Following the test, we will compile the results of all the usability tests and evaluate them. We will
be looking for what worked for our users and what didn’t. This will help us determine the general
idea of the website’s usability as it was experienced by our participants, and will thereby aid us
in developing suggestions for solutions for how to improve it in the future.
Introduction

The website being evaluated is GreaterSecond.org. This is a website that services a

Baptist church in the greater Little Rock area.

The main navigation consists of: Home, Our Leaders, Christian Leadership, Values and
Beliefs, About GSCC, Resources, and Gallery. Each main tab has two or more
subcategories. There is a second navigation panel at the bottom of nearly all the pages.
The secondary navigation consists of: Get Connected, Our Blog, Our Sermons, Events
Calendar, Photo Gallery, Online Donations, Have Questions, and Prayer Requests. The
secondary Navigation does not have any subsections.

Some initial observations are that there are a number of pages that display 404 Errors,
meaning that the link to the pages are broken and therefore cannot be displayed. There
are several images that do not fit within the allotted space and are not optimized for use
on the web. Having a current church member in the group, we are able to note that
there is quite a bit of information on the site that is outdated, including some staff
members, images that are several years old, and several events that occur regularly but
are not listed on the site.

Through heuristic analysis, we have identified the navigation nesting to be a main area
of concern and have chosen to focus many of our tasks based on the navigation and
ability for the user to locate different features of the site in order to complete specific
tasks.
Methodology
This section describes the methodology for conducting usability testing on the Greater Second
Baptist (GSB) website, including:

• Planning phase
• Participant information
• Equipment in the usability lab
• Task and scenario overview
• Test configuration

Planning Phase
Prior to conducting the usability test, the project team conducted a heuristic evaluation on the
GSB website. The heuristic evaluation is a test in which the usability team judges the usability of
an interface against a specific set of heuristics or principles. The heuristics and principles that
the usability team used were taken from Nielsen’s Ten Usability Heuristics. Table 1 describes in
detail Nielsen’s heuristics.

The purpose of the heuristic evaluation is to discover any potential usability weaknesses that
might intimidate the novice or beginner user from using the site. Acting as subject matter
experts, the four-member project team uncovered some areas of concern that the test
participants might encounter while interacting with the GSB interface.

The finding of the Heuristic Analysis are listed below:

Heuristic Analysis Data

1. Visibility of system status: Yes, but needs work.
a. About & Resources seems easy to find.
b. The text/ images are too large & not readable
c. Pictures are off centered
d. Everything shifts to the right of the screen
2. Match between system & real world: Yes, but needs work.
a. Rotator images drop down instead of scroll.. Seems odd.
b. Food pantry seems to be under the wrong tab.
3. User control and freedom: No,
a. if at the bottom of the webpage, you have to scroll to the top of the page to
navigate elsewhere.
b. Back buttons seem to be the only exit.
4. Consistency & standard: No,
a. Different fonts all over
b. Different icons for generic listings.
c. Calendar outdated.
d. The resources and about tabs seem to have confused info.
e. Photos in the gallery are 5 years old
f. Some of the leaders listed on this web site are no longer members of this
church
5. Error prevention: No, there is not any error prevention.
a. Many broken links.
b. Forms do have validation, however unsure who receives the form info.
6. Recognition over recall:Yes, but needs work.
a. Menu not as obvious
b. Not easy to navigate
7. Flexibility/efficiency of use: No
a. Online registration for some things, however unsure who receives the form
info.
b. Online contact forms, however unsure who receives the form info.
c. A Lot of broken links - 404 errors
8. Aesthetic/minimalist design: No, looks cluttered.
a. The text/ images are too large for allotted space & not readable.
b. Home page is super busy
9. Error help: Yes, but needs work.
a. With filling out forms with public/private information.
10. Help & documentation: Yes, but needs work.
a. Everything is labeled pretty well, but you can’t search for something
specific.
b. A lot of opportunities to give feedback
c. Not a lot of help info/ preventative.
Design of Questionnaires
The questionnaires were designed for the test participants to complete at different stages of the
usability test. The goals of these questionnaires are to garner some insight to the users
perceptions prior to, during, and immediately after the test. Comparing the participants’ answers
from the different stages of the test should reveal how the experience affected their overall
understanding and view of the tasks, as well as the interface as a whole.

The Pre-Test Questionnaire will be given prior to the usability test on the day of the test. The
purpose of the pre-test questionnaire is to obtain some insight of the test participants’
perceptions of the current interface and some of their concerns about the test. The
questionnaire is also used to gauge if there are any impairments that may skew the test results
or adaptations that need to be made for the user.

The Post-Task Questionnaires will be given immediately following each individual task. This
will be done so that the participant can express their impressions while the test experience is
still fresh in their memories. Participants will also be given the opportunity to give additional
comments after each of these questionnaires.

The Post-Test Questionnaire will be administered immediately after the completion of the full
usability test. The Post-Test Questionnaire mirrors the purpose of the Pre-Test Questionnaire,
with the goal of obtaining some insight into the test participants’ perceptions about the overall
use of the GSB interface, after interacting with it throughout the usability test.

All of the questionnaires and participant’s responses can be found in Appendices B, G, H, and I.

Usability Testing Participants

The GSB team will conduct eight sessions with volunteers who meet the criteria of our primary
user group (having recruited a total of twelve to allow for backup participants). We will select our
participants based on the user profile we created for this project (see Appendix C). The user
profile is defined as: “persons who would be searching for a new church, who have children or
understand the needs of children in a church setting, and/or who are concerned about the
beliefs and values of a church before attending.” Qualified participants will complete a pre-test
questionnaire verifying that they do meet the user profile we have designed.

Length of Sessions
Each session will be one hour, with the following segments:
• Welcome and pre-test: 5 minutes
• Task scenarios: 45 minutes
• Post-test questionnaire: 5 minutes
Scheduling
The usability tests will be administered in UALR Stabler Hall CRUX Lab on the 6th floor, and
UALR Student Union room 106C, beginning on March 26th and completed by April 15 th, 2019.
Tests will be completed as the users’ schedules allow.

Tasks & Scenarios

The tasks created for each scenario are based on a combination of our heuristic evaluation
findings and feedback from project sponsor, Dr. Joe Williams. Within each scenario, participants
will attempt tasks related to the areas of concern mainly focused on the content organization
and accessibility.

According to our original evaluation and heuristic analysis, primary areas of concern are the
site’s:
• Content
• Organization
• Community resources
• Sense of community
• Visual formatting/readability

The heuristic evaluation considered each of these concerns along with typical needs of a
church.

Description of the Testing Scenarios

Based on the results of the heuristic evaluation, the project team has generated four test
scenarios. We determined that these four detailed scenarios would be sufficient to observe any
issues that the test participants might encounter. The difficulty of the tasks will increase with the
advancement of the scenarios as the user becomes more comfortable, as well as follow the
common path our defined user might take to determine if they would want to attend the church
or not.

The goal of these scenarios is to allow the test participants to familiarize themselves with the
GSB website, perform tasks that a typical user would encounter, and identify areas of concern
that need to be addressed.

A summary of each of the scenarios is listed below. The Post-task Surveys can be found in
Appendix F.
Scenario 1: You are looking for a new church and have come across the
Greater Second Baptist Church website. You want to know if this church fits
with your beliefs.

1. Please navigate to the areas of the website you think would be relevant or helpful to
determine the beliefs of this church. When you are ready to move on to the next step
raise your hand

Scenario 2: You want to know if childcare options or programs are available

at GSB.

1. Where would you go to find children’s programs on the Greater Second Baptist Website?
Please do this now.

2. Where would you find if there is childcare offered by Greater Second Baptist? Please do
this now.

Scenario 3: Next you want to find relevant content about church

membership. Don’t forget to “Think Out Loud!”

1. Are you able to find out how to become a member of GSB?

2. Where would you go if you have questions about membership?

3. Are you able to send a message to someone on the church staff?

Scenario 4: You have decided that you want to try out GSB.

1. Please find the Sunday service times.

2. Are there services on any other days?

 3. Where can you find the Address for GSBC?

Testing Equipment
Participants will sit at a standard desktop computer workstation with internet access,
accompanied by the test moderator. The participant room contains the following computing and
monitoring equipment:
● Morae testing software
● Webcam
● Microphone
● User phone, for communication as needed

The test team will observe and record user sessions from the control room. Control room
equipment consists of:
● Morae observer software
● A standard Desktop computer with 2 large monitors

Conducting the Test

1. For each user, the moderator will begin with an overview of the testing environment and
what the user can expect during the test.
2. The user will then be asked to complete the Pre-Test Questionnaire.
3. Next, the moderator will introduce a test scenario. The moderator will sit with the
participants throughout the test to ensure user understanding and comfort level, and will
explain the next steps between scenarios. The moderator will observe quietly and let
participants complete each task independently without direction. The moderator will only
prompt or ask appropriate questions as needed. The user will be allowed all the time
needed to complete each task.
4. Following each task, the user will be asked to complete the Post-Task Questionnaire.
5. After all tasks have been completed, the user will be asked to complete the Post-Test
Questionnaire.

Usability Test Plan for GSB Website

All of the participants’ actions will be recorded through Morae logging software and video. The
participants will be encouraged to think aloud throughout the session. The moderator may take
notes of the comments made by the user during the testing. Each session will conclude with a
standardized 10-question survey known as the System Usability Scale (SUS), which is
conducted through the Morae software.
Following the user tests, each team member will analyze and evaluate the tests, looking for the
number of clicks per task, positive and negative nonverbal feedback, positive and negative
verbal feedback, completion of the tasks, failure to complete the tasks, and 404 Errors.
Summaries and findings will be compiled based on the observations of all team members.

A full list of the test plan details can be found in Appendix D.

Findings
It was determined that the website functioned in a way that most users would expect, however
the content and organization were lacking. For example, the main navigation was where users
expected to find it, however there was a secondary navigation located at the bottom of the
pages that housed content not found in the main navigation. Some users never found the
secondary navigation because of the location, as well as the fact that it had much different
styling than the main navigation.

Secondly, the items nested under the main navigation seemed to be confusing for several of the
users. For example, items listed under “About GSCC” seemed more fitting to go under
“Resources,” and vice-versa. Additionally, some of the main tabs are linked to a page and some
are not. Not only does clicking on the tabs that do not have anything linked not take you to a
new page, it also does not make the sub-menu items display. The nested items only display if
you hover over the tab without moving.

Thirdly, it is an industry standard that it should require no more than 3 clicks to get to a
destination on a web page. As you can see in the table below, 4 of the 9 tasks required
significantly more than 3 clicks to complete the task. The tasks themselves did not require
clicking, so the data shows solely the number of clicks it took to navigate to where the
information is housed.
Another significant finding is that the most important information is not prominently displayed.
Information such as service times, address, who to contact, and where children should go were
not listed in headers or sidebars, or anchored in places that could easily be seen. Users had to
dig through the website to find the information that they expected to easily find. When asked to
find the address in order to attend the church service, one exacerbated user stated, “After
searching all through this website for simple information, I don’t think I would even want to go to
this church!”

Throughout the testing process, the majority of user comments were negative, and the users
appeared physically frustrated. However, as noted in our Pre-Test Questionnaire, we found that
people tend to answer surveys more kindly or positively than their experience would lead you to
believe they ought to answer. This was depicted in the Post-Task surveys, where after huffing in
frustration, many users gave neutral responses.

Finally, one of the more significant responses we got was from an older user, who is quite
tech-savvy, and also an avid church-goer. This user stated, “The problem isn’t the functionality
of the website. The website performs the way one would expect. The navigation works, the
page scrolls the way I expect it to. The problem is the content just is not there. I can easily find
where answers should be. I can navigate to that area without problems, except when the page
link is broken and I get an error. But the real problem is that the content just isn’t on the site to
answer the basic questions a user would have. For example, look at the calendar. There is a
calendar, but there aren’t any events on it. I would hope a church has an event on Sundays, at
the very least, but nothing. Nothing is listed.”

This user diligently looked for answers. She investigated thoroughly, even asking to be let out of
the post-task surveys when she had a new idea about where she might be able to find an
answer after previously not being able to. Her comment that the content itself is lacking seemed
to summarize the frustrations of all the users we tested.
Conclusions
As a team, we have determined a number of changes that we believe would substantially
improve the functionality and usability of this website. Our suggestions are as follows:

1. Church service times, address, and phone number should be added to the header of
each page so that it can be easily found no matter what page of the site a user is on.
a. Currently, the service times are in a scrolling image that is displayed only on the
home page and are not even displayed in the first iteration of the scrolling
banner.
2. All services and activities should be added to the events calendar.
3. All images should be formatted to be responsive and have limits so that they may not
display larger than the allotted space.
4. Values and Beliefs should include stances on current issues.
a. Currently there are generic lists of beliefs and values, and even some scriptures
that indicate where they come from; however, there is no mention of where the
congregation or leadership stands on major social issues.
5. The secondary navigation should be moved to a more prominent location, such as a
sidebar, so that users know that the information is available.
6. The home page should be more concise, moving the sermon information to the pages
dedicated to that information and leaving the homepage for contact information and
important announcements.
7. All outdated information should be updated or removed, particularly in reference to staff
members who are no longer on staff.
8. Nested items on the main navigation should be restructured to display in a more
readable format and reorganized to display only active content.
9. The “Have Questions” content should be moved to the “About GSCC” landing page.
10. All main navigation elements should direct to an applicable landing page.
11. Commonly asked questions should be addressed with legitimate answers via significant
expansion of the content readily available on the website..
Appendices
A.
Table 1: Nielsen’s Ten Heuristics
Source: Carol Barnum, “Usability Testing and Research.” New York: Longman 2002. © 2002 Pearson Education, Inc.
Nielsen’s Ten Usability Heuristics
1. Visibility of System Status
2. Match between system and the
real world
3. User control and freedom
4. Consistency and standards
5. Error prevention
The system should always keep users informed
about what is going on, through appropriate
feedback within reasonable time.
The system should speak the users' language,
with words, phrases and concepts familiar to the
user, rather than system-oriented terms. Follow
real-world conventions, making information
appear in a natural and logical order.
Users often choose system functions by mistake
and will need a clearly marked "emergency exit"
to leave the unwanted state without having to go
through an extended dialogue. Support undo and
redo.
Users should not have to wonder whether
different words, situations, or actions mean the
same thing. Follow platform conventions
Even better than good error messages is a careful
design which prevents a problem from occurring
in the first place. Either eliminates error-prone
conditions or check for them and present users
with a confirmation option before they commit to
the action.
6. Recognition rather than recall
7. Flexibility and efficiency of use
8. Aesthetic and minimalist design
9. Help users recognize, diagnose,
and recover from errors
10. Help and documentation
Minimize the user's memory load by making
objects, actions, and options visible. The user
should not have to remember information from
one part of the dialogue to another. Instructions
for use of the system should be visible or easily
retrievable whenever appropriate.
Accelerators -- unseen by the novice user -- may
often speed up the interaction for the expert user
such that the system can cater to both
inexperienced and experienced users. Allow
users to tailor frequent actions.
Dialogues should not contain information, which is
irrelevant or rarely needed. Every extra unit of
information in a dialogue competes with the
relevant units of information and diminishes their
relative visibility.
Error messages should be expressed in plain
language (no codes), precisely indicate the
problem, and constructively suggest a solution.
Even though it is better if the system can be used
without documentation, it may be necessary to
provide help and documentation. Any such
information should be easy to search, focused on
the user's task, list concrete steps to be carried
out, and not be too large.
B.

User Research

Pre-test survey

Are you a current member of this

Timestamp Email Address church?
2/21/2019
20:59:20 jllove2525@gmail.com No
2/22/2019 2:15:36 mdmoody2@ualr.edu Yes
2/22/2019 2:28:38 kailyced12@gmail.com No
2/22/2019 camillehandie83@gmail.co
14:08:41 m Yes
2/22/2019
14:44:12 klleonard1@ualr.edu No
2/22/2019
16:58:42 1100reezo@gmail.com No
2/24/2019
11:50:12 jenn.sanders25@gmail.com No
2/25/2019 3:09:00 shaneikadperry@gmail.com Yes
2/25/2019
15:47:52 yxchang1@ualr.edu No
2/26/2019
21:50:15 shrronhandie@gmail.com Yes
2/28/2019 1:56:49 shantail.miller@gmail.com No

Are you seeking a

new church? What is your first impression of this site?
I thought it was a good site that offered lots of good information for new and current
No members
No welcoming
No It's wonderful!
 It looks nice and it brings back childhood memories of me and my family at this
No church
I see where they're going for and overall it's not the worst site I've seen, but it's just
No their optomization and organization are way out in left field.
No
No Wonderful!
Yes, No Outdated
No Bad quality, need resizing and structure
No unfinished
Yes Very one demensional

Are you able to find info on service

What is your main goal when coming to this site? times?
Just to check it out Yes
Get information on events happening st the church Yes
Church times and events Yes
Keep up with church events Yes
Find information about the church and it's events. Yes
N/A Yes
To find answers Yes
To see what's going on with the church. Yes
Resizing Yes
To find out service times, whats offered, etc No
Information about the different currently active ministries
available Yes, No

Are you able to find contact Please check all resources and services you were able to find useful
information? information for:
Yes
Food & clothing pantry, Care Center, After School Care, Prayer
Yes Requests
Food & clothing pantry, Care Center, After School Care, Prayer
Yes Requests
Yes Food & clothing pantry, Care Center, After School Care, Prayer
 Requests
Food & clothing pantry, Care Center, After School Care, Prayer
Yes Requests
Yes Prayer Requests
Yes Food & clothing pantry, Care Center, Prayer Requests
No Prayer Requests
Yes Food & clothing pantry, Care Center, After School Care
Care Center
Food & clothing pantry, Care Center, After School Care, Prayer
Yes Requests

If you are a current church member, how What is the end goal of the organizations, if you were to be
do you get involved in the organizations? accepted?

Just join in Make the church a better place

Not a current member
To be honest, i don’t . I don’t know
I would honestly bypass the site and just
talk to someone at the church.

“Pantry” in the previous question has a

typo This question feels weird.
I don't. N/A
Build a better site Fix up the site where its easy to use
The organizations goal would be to create a place of
fellowship and a place where like minded believers can
N/A congregate.

What would you suggest be changed about this site?

Maybe a place where members and visitors could watch some old videos of sermons.
Nothing the website is readable, easy to navigate and has character.
Bolder font
I’m no expert at websites, so i would only add more pictures of the church members and staff, and a little
more splash of color.
Clean up optimization and have better explained site portions and a cleaner navigation.
Nothing
See previous answer :)
Update ALL information&pictures
Resizing the images
Font choice, size of images as they did not load, etc.
Information should be kept up to date, contact links for the different ministry leaders, sermons/sermon
point should have its own tab, service times should be more readily visible- maybe even in Multiple
places
C.

Personas & Empathy Maps

Persona 1: Kelly Davis
31 years old
Single mother of two (2)
Full-time nurse
Technical Profile: Familiar with her smartphone and general computer
technology (as she was/is required to use it to get through nursing
school and to enter patient information at work).
Hobbies: Reading, cooking, spending time with family

Kelly is a busy working mom that is looking for somewhere safe to let her kids stay while she is
away at work. Her shifts as a nurse require long nights and weekends, and she doesn’t have
family members that live near enough to her to watch the kids for her while she works. She is in
search of services at a local church that can help her with this.

Empathy Map 1:

Think & Feel

● “I want my kids to be watched by someone trustworthy while I am working.”
● “I want my kids kept safe.”
● Stressed, concerned, worried, determined, pressed for time
Hear
● “A church is probably a good place to keep the kids busy.”
● “I’m sorry, I just don’t have the time to watch them for you tonight.”
● “Babysitter’s can’t be trusted.”
See
● Available services at local churches
● Lack of reliable family nearby that can help
● Need for quality care for her children
Say & Do
● “I’m going to find a place where I can rest easy knowing my kids are safe and cared for
while I’m at work.”
● Looking online for local churches/daycares that offer services on nights and weekends
● Contact churches or daycares that look promising
 Persona 2: Jordan Givens
Age:12
Income -0-
7th grade Student

I am an active member of GSBC and would like to use the

website to find out the following information

● What (if any) are the upcoming events

● Where do I listen to the latest sermon
● View photos from past events
● Find out what activities are available for the youth
● When and who do I call fro transportation for sunday service

Empathy Map 2:

Think & Feel:

● Are the sermons biblical?
● Will they really pray for me if I ask?
Hear:
● I enjoyed that sermon
See:
● Is this website up to date?
● What can I get involved in?
● This website is unorganized
Say & Do:
● This website is not easy to navigate
● I can request transportation
 Persona 3: Cambridge Hanaboro
Age: 23 years old
Single
Active church member
Seeking to get involved more

Empathy Map 3:
Thoughts/Feelings
● Eager
● Wants to get involved more.
● Can I help someone who needs it?
● What is my end goal with these events, if I am accepted?
● Anxious
Hear
● You’d make a great asset to their organization.
● Can you do this?
● Are you willing to help?
● What more could you do?
See
● Ways to help reduce hunger would be by VOLUNTEERING, DONATE AND BECOME A
SPONSOR.
● Volunteers are needed for...
Say/Do
● I can help.
● I will help.
● What are the programs that are offered?
 Persona 4: Community Member Seeking
Services offered by the church
Mid thirties
Single parent
Needs child care
Has limited resources
Interested in food pantry & clothing drive
Busy lifestyle
Uses web on phone mostly
Motivators:
Easy to find
Complete info
Makes life easier

Frustrations:
Out of date info
Questions not answered.

Empathy Map 4:
Think & Feel
● “I wish I could pay for my childcare online.”
● “I only have a few minutes to get this done...”
● Busy, tired
● Looking for a strong community
Hear
● “Have you checked out GSB?”
● “They have great kids programs”
See
● Many churches to choose from
● Churches getting more modern with web options
Say & Do
● “I want to find ways to give back.”
● Looking for a church with a lot of activities for kids
D.

Test Plan:
1. Where will your team gather general user research related to your testing project? (Hint:
Pew Internet Research and Nielsen are great places to start!)
a. What we can find online and what is on the website itself.
b. http://www.pewforum.org/datasets/
c. http://www.thearda.com/
d. https://blog.capterra.com/church-statistics-social-media/
e. https://churchmetrics.com/
f. http://hirr.hartsem.edu/research/fastfacts/fast_facts.html#WebsiteGrowth
2. What about specific research about your identified user group(s)? What additional
research can you do to understand them better? A survey? A focus group? Some
interviews? Card sorting?
a. We will come up with a google survey and send it out to a small group that
attends the church.
i. What would you go to the website to do/look for?
ii. Were you able to find what you were looking for?
iii. Could you find current information?
iv. What services does the church offer?
v. Were you able find info on these services?
vi. How easy was it for you to navigate this web site?
vii. Did you receive any error messages while using the web site?
3. What is the purpose of creating personas and empathy maps? How will they help your
testing project?
a. The purpose of creating personas and empathy maps are to place ourselves into
the user’s shoes so that we can have a better understanding of the user. Use the
website as a user with and/or without a bias.
1. Church members
a. People with kids
b. People without kids
2. Church seekers
3. Hearing impaired church goers
4. Community members seeking services offered by the church
4. What goals do your users bring to their use of the interface?
a. The goal for the user is to navigate through the website and find specific things.
They will need to pick around the website and see what they can find, what is
and is not possible, etc.
i. To find info
ii. To register for services
5. What expectations do they bring?
a. The expectations are if they are looking for a new church and want to join, if they
are wanting to find a church, event details, special requests, general information
of the church and its staff members, job opportunities, etc.
i. Current info
ii. Easy to navigate
6. What relevant experiences do they bring?
a. N/A
7. Think through the difference between tasks and goals, particularly as it applies to your
project. What common tasks can you identify for users, based on your user research?
a. Event dates/calendar
b. Services provided
i. Requirement to receive services
ii. Times and dates services are available
c. Beliefs and Theology of the church
d. Service times
e. Contact info
f. Leadership
g. Music/worship style
8. How does your heuristic analysis data add to your list of user tasks for the interface you
are testing?
a. N/A
9. Based on heuristic analysis, general user research, specific research on your user
group(s), and development of personas and empathy maps, what task-based scenarios
should tentatively be in your test plan? (These are the short scenarios found in the test
plan examples, not the longer ones found in the textbook.)
a. See #7
E.
Moderator Script

Introduction
Good Morning/Afternoon/Evening. How are you doing? I will be reading from a scripts for our
session today to ensure that all participants have the same information. Today we are testing
the Greater Second Baptist Church’s website. I would like to make it clear that we are testing
the website and not you. Please note that we did not build this site and we are not being graded
on the functionality of the site, but are learning to identify what does and does not work well in a
website. Thank you for your participation for today. With this participation, you will be giving my
team and I some very useful feedback for improving the website that you will be looking at in a
few minutes.

Videotaping Permission
As you know, we are going to record the session. The reason for recording the session is so
that the team members that are not with us today can review the session if we have any
questions while writing our report, and so that our sponsor who is unable to join us will have a
chance to review the session. Additionally, we could potentially use some video-clips in an
academic and professional presentations. At the conclusion of this class (May 2019) all videos
will be deleted.

Here is a permission form stating that you allow us to videotape the session, as well as record
your voice statements. Please take your time to review it. [Give the participant the permission
form to review.] Are you comfortable with signing this form? [If so, ask the participant to sign it. If
the participant has questions, respond to them.]

Introduction of the Evaluation Room

Before we start, I would like to give you a brief tour of the room. We will be recording the
session with the camera here on the computer as well as recording the screen. [Point to the
cameras.] They give the team a view of the screen as you see it as well as a view of your face.
Most of the time, the team will be looking at the screen you will be working on to see where you
are clicking. We have a small headshot of you in the corner of the screen to be sure we know
which test participant in performing the test.

Here is a microphone [point to the microphone] to pick up your comments as you complete the
tasks. The moderator will remain in the room to allow communication with the team during the
test, should you have any questions or concerns. Please note that we may not be able to
answer questions about the task, as they may skew the findings of our test, but you are
welcome to ask any questions you may have.
Introduction of the Team
Here to observe and document this testing session is my teammate, (insert individual team
member name.) [The team is made up of two caucasian females, one african american female,
and one african american male.] Our professor, Dr. Joe Williams, may also be here to help us
conduct the test.

Introduction to the Test

Today, we are going to be working with the Greater Second Baptist website. We’ll give you
some tasks to do on the site, then ask you some questions to get your feedback. We really
appreciate any feedback you are willing to give about your experience. In addition to being
video recorded, we would like you to “Think out loud” by telling us what you are doing and why
you are doing it.

During the session, I am going to sit here with you, and I may take a few notes for the final
report. Again, I would like to remind you that YOU cannot do anything “Wrong” in this session. If
you find that a task cannot be completed or that you are unable to find something within the site,
that is perfectly fine. The goal here is to find the shortcomings of the site itself. Before we start,
do you have any questions for me or the team?
F.
Scenario 1: You are looking for a new church and have come across the
Greater Second Baptist Church website. You want to know if this church fits
with your beliefs.

Please open the Google Chrome browser now and navigate to greatersecond.org. Please
remember to “Think Out Loud” while performing these tasks.

1. Please navigate to the areas of the website you think would be relevant or helpful
to determine the beliefs of this church. When you are ready to move on to the next
step raise your hand

Scenario 2: You want to know if childcare options or programs are available

at GSB.
2. Where would you go to find children’s programs on the Greater Second Baptist
Website? Please do this now.

3. Where would you find if there is childcare offered by Greater Second Baptist?
Please do this now.

Scenario 3: Next you want to find relevant content about church

membership. Don’t forget to “Think Out Loud!”
4. Are you able to find out how to become a member of GSB?

5. Where would you go if you have questions about membership?

6. Please send a message to someone on the church staff.

Scenario 4: You have decided that you want to try out GSB.
7. Please find the Sunday service times.

8. Are there services on any other days?

9. Where can you find the Address for GSBC?

G.
Pre-test Survey questions
Please go to https://greatersecond.org/ to answer the following questions:
Email address:

Are you a current member of this church?

Are you a seeking a new church?

What is your first impression of this site?

How does the overall appearance of the site display on your device?
a. Perfectly, no problems
b. Reasonably well, everything is readable
c. Poorly, some items are not readable
d. Terribly, most content does not display in a useable manner

What is your main goal when coming to this site?

Are you able to find info on service times?

Are you able to find contact information?

Please check all resources and services you were able to find useful information for:
a. Food & clothing pantry
b. Care Center
c. After School Care
d. Prayer Requests

If you are a current church member, how do you get involved in the organizations?

What do you think worked well about the site?

What would you suggest be changed about this site?

H.

Post-Task Survey 1
1. I think this information was easy to find: Strongly Disagree 1- 5 Strongly Agree
2. I think this information was in a logical location: Strongly Disagree 1- 5 Strongly Agree
3. I think I have a good understanding of what the beliefs are based on the information I
was able to find: Strongly Disagree 1- 5 Strongly Agree

Responses:
I think this information was easy to find.
Strongly Disagree 1 2 3 4 5 Strongly Agree
Cheryl
Dylan X
Jessica X
Judith X
Loren X
McKenzie
Tam X

I think this information was in a logical location.

Strongly Disagree 1 2 3 4 5 Strongly Agree

Cheryl
Dylan X
Jessica X
Judith
Loren X
McKenzie
Tam X

I think I have a good understanding of what this beliefs are based on the information I was able
to find.
Strongly Disagree 1 2 3 4 5 Strongly Agree
Cheryl
Dylan X Information is
available - but generic. I'm not sure how this church would compare to others in the Little Rock
area.
Jessica X The spiritual beliefs and basis of
those beliefs are clearly stated, however their application to the current and relevant issues and
the church's stand is absent
Judith X
Loren X
McKenzie
Tam X

Post-Task Survey 2
1. I think this information was easy to find: Strongly Disagree 1- 5 Strongly Agree
2. I think this information was in a logical location: Strongly Disagree 1- 5 Strongly Agree
3. I was satisfied with the information I was able to find on the site: Strongly Disagree 1- 5
Strongly Agree

Responses:
I think this information was easy to find.
Strongly Disagree 1 2 3 4 5 Strongly Agree
Cheryl
Dylan X children's
programming' was easy to find - not sure exactly what it is...I would assume that it is child care
during the worship service.
Jessica X I had to go to 'have questions' to determine if there
was a place for child care and it took several attempts at other logical links before I was able to
find it.
Judith X
Loren X
McKenzie
Tam X

I think this information was located in a logical location on the site.

Strongly Disagree 1 2 3 4 5 Strongly Agree

Cheryl
Dylan X
Jessica X
Judith X
Loren X
McKenzie
Tam X
I was satisfied with the information I was able to find on the site.

Strongly Disagree 1 2 3 4 5 Strongly Agree

Cheryl
Dylan X
Jessica X
Judith X There was no information. There was an error
message. I was redirected to the Home Page
Loren X
McKenzie
Tam X It was not easy to locate any information about childcare.
Actually, I did not locate any place on the site about where a child would be cared for, if I were to
attend a service or special program.

Post-Task Survey 3
1. I think this information was easy to find: Strongly Disagree 1- 5 Strongly Agree
2. I think this information was in a logical location: Strongly Disagree 1- 5 Strongly Agree
3. I believe I understand how to become a member if I were wanting to join: Strongly
Disagree 1- 5 Strongly Agree

Responses:
I think this information was easy to find.
Strongly Disagree 1 2 3 4 5 Strongly Agree
Cheryl
Dylan X
Jessica X
Judith X
Loren X
McKenzie
Tam X

I think this information was located in a logical location on the site.

Strongly Disagree 1 2 3 4 5 Strongly Agree

Cheryl
Dylan X
Jessica X
Judith X I would expect that information Under About GSCC
Loren X
McKenzie
Tam X

I believe I understand how to become a member if I were wanting to join.

Strongly Disagree 1 2 3 4 5 Strongly Agree

Cheryl
Dylan X
Jessica X
Judith X I believe the how is not the real question I would
have. I would be hesitant to complete the application, believing that was a commitment. I would
want the option to contact someone or have someone contact me.
Loren X
McKenzie
Tam X I only located a form to fill out with my personal
information. I did not see a person's name to contact for membership information.

Post-Task Survey 4
1. I was able to send a message: Strongly Disagree 1- 5 Strongly Agree
2. Sending a message was easy: Strongly Disagree 1- 5 Strongly Agree

Responses:

I was able to send a message.

Strongly Disagree 1 2 3 4 5 Strongly Agree
Cheryl
Dylan X
Jessica X yes i could send a message
if I knew who to send it to
Judith X
Loren X
McKenzie
Tam X

Sending a message was easy.

Strongly Disagree 1 2 3 4 5 Strongly Agree
Cheryl
Dylan X
Jessica X the information is unclear about the
roles of the individuals listed and how they may be able to answer or respond to my questions
Judith X
Loren X
McKenzie
Tam X

Post-Task Survey 5
1. I was able to find the Sunday Service Times: Strongly Disagree 1- 5 Strongly Agree
2. I think this information was easy to find: Strongly Disagree 1- 5 Strongly Agree
3. I think this information was in a logical location: Strongly Disagree 1- 5 Strongly Agree

Responses:
I was able find the Sunday service times.
Strongly Disagree 1 2 3 4 5 Strongly Agree
Cheryl
Dylan X
Jessica X I found times available for child care
and I assume that those are also service times but I am unsure of other Sunday Service options
Judith X
Loren X
McKenzie
Tam

I think this information was easy to find.

Strongly Disagree 1 2 3 4 5 Strongly Agree
Cheryl
Dylan X Could be missed
given the slide show set-up (if someone is quickly browsing) Maybe could also be placed in
another area to be more user friendly.
Jessica X
Judith X
Loren X
McKenzie
Tam

I think this information was located in a logical location on the site.

Strongly Disagree 1 2 3 4 5 Strongly Agree

Cheryl
Dylan X
Jessica X
Judith X I would have expected it as a Frequently
Asked Question. First logical place, 2nd, under Events Calendar, never assume people know
when your regular services are. They should be on your calendar. 3rd Get Connected, A person
browsing would believe that is an opportunity to join, the first meaningful connection.
Loren X
McKenzie
Tam

Post-Task Survey 6
1. I was able to find information on other service times: Strongly Disagree 1- 5 Strongly
Agree
2. I think this information was easy to find: Strongly Disagree 1- 5 Strongly Agree
3. I think this information was in a logical location: Strongly Disagree 1- 5 Strongly Agree

Responses:
I was able to find information on other service times.

Strongly Disagree 1 2 3 4 5 Strongly Agree

Cheryl
Dylan X
Jessica X
Judith X
Loren X
McKenzie
Tam X

I think this information was easy to find.

Strongly Disagree 1 2 3 4 5 Strongly Agree
Cheryl
Dylan X
Jessica X This website is extremely cumbersome and
not easy to navigate quickly.
Judith X
Loren X
McKenzie
Tam X I think the color used could be
more bold and to help draw attention to that area of the home screen. I think maybe centering
this information would be best.

I think this information was located in a logical location on the site.

 Strongly Disagree 1 2 3 4 5 Strongly Agree
Cheryl
Dylan X
Jessica X
Judith X Please see comments regarding Sunday Service
Times. There was one page that gave the 404 error Message, maybe that was the page
providing service times and days.
Loren X
McKenzie
Tam X

Post-Task Survey 7
1. I think this information was easy to find: Strongly Disagree 1- 5 Strongly Agree
2. I think this information was in a logical location: Strongly Disagree 1- 5 Strongly Agree
3. I was satisfied with the information I was able to find on the site: Strongly Disagree 1- 5
Strongly Agree
4. I believe I could go to the correct location based on the information I was able to find on
the site: Strongly Disagree 1- 5 Strongly Agree

Responses:
I think this information was easy to find.
Strongly Disagree 1 2 3 4 5 Strongly Agree
Cheryl
Dylan
Jessica X yes... one of the
easiest
Judith X Easy, because I am a web surfer, I always
expect information to be there, like the header/footer of stationery. I think it should be more
prominently displayed near the header of the landing page.
Loren X
McKenzie X Some of the information was
confusing. The wording, specifically in the navigation of the site, made it harder to complete
certain tasks.
Tam X

I think this information was located in a logical location on the site.

Strongly Disagree 1 2 3 4 5 Strongly Agree

Cheryl
Dylan
Jessica X it is most logical to
have it at the top, but many sites have the physical address at the bottom
Judith X see above. Logical for someone
familiar with web sites/design, not for a person just 'looking for a church to visit.'
Loren X
McKenzie X I think both sites were organized and
grouped like items together; however, I think the first site had too many 'resources' and the
second site had awkward names in the navigation.
Tam X

I was satisfied with the information I was able to find on the site.

Strongly Disagree 1 2 3 4 5 Strongly Agree

Cheryl
Dylan
Jessica X
Judith X
Loren X
McKenzie X I was happy to answer some
questions quickly and to find the address for both sites in logical places (like the footer and
contact us page).
Tam X

I believe I could go to the correct location based on the information I was able to find on the site.

Strongly Disagree 1 2 3 4 5 Strongly Agree

Cheryl
Dylan
Jessica X
Judith X I would have preferred a google link,
available when clicking the address, or a side pop up, providing area view or directions from
area landmarks.
Loren X
McKenzie X Some information was hard to find
because it was in places that I would not expect them to be in. Childcare was a hard one to find.
Tam
I.

Post-Test Survey
1. I think that I would like to use this website frequently: Strongly Disagree 1- 5 Strongly
Agree
2. I found this website unnecessarily complex: Strongly Disagree 1- 5 Strongly Agree
3. I thought that the website was easy to use: Strongly Disagree 1- 5 Strongly Agree
4. I think that I would need the support of a technical person to be able to use this website:
Strongly Disagree 1- 5 Strongly Agree
5. I found the various functions in this website were well integrated: Strongly Disagree 1- 5
Strongly Agree
6. I thought there was too much inconsistency in this website: Strongly Disagree 1- 5
Strongly Agree
7. I would imagine that most people would learn to use this website very quickly: Strongly
Disagree 1- 5 Strongly Agree
8. I found this website very cumbersome to use: Strongly Disagree 1- 5 Strongly Agree
9. I felt very confident using this website: Strongly Disagree 1- 5 Strongly Agree
10. I needed to learn a lot of things before I could get going with this website: Strongly
Disagree 1- 5 Strongly Agree

I think that I would like to use this system frequently

Strongly Disagree 1 2 3 4 5 Strongly Agree
Cheryl X
Dylan X
Jessica X
Judith X
Loren X
McKenzie
Tam X

I found the system unnecessarily complex

Strongly Disagree 1 2 3 4 5 Strongly Agree
Cheryl X
Dylan X
Jessica X
Judith X
Loren X
McKenzie
Tam X
I thought that the system was easy to use
Strongly Disagree 1 2 3 4 5 Strongly Agree
Cheryl X
Dylan X
Jessica X
Judith X
Loren X
McKenzie
Tam X

I think that I would need the support of a technical person to be able to use this system

Strongly Disagree 1 2 3 4 5 Strongly Agree

Cheryl X
Dylan X
Jessica X
Judith X
Loren X
McKenzie
Tam X

I found the various functions in this system were well integrated

Strongly Disagree 1 2 3 4 5 Strongly Agree

Cheryl X
Dylan X
Jessica X
Judith X
Loren X
McKenzie
Tam X

I thought there was too much inconsistency in this system

Strongly Disagree 1 2 3 4 5 Strongly Agree

Cheryl X
Dylan X
Jessica X
Judith X
Loren X
McKenzie
Tam X
I would imagine that most people would learn to use this system very quickly

Strongly Disagree 1 2 3 4 5 Strongly Agree

Cheryl X
Dylan X
Jessica X
Judith X
Loren X
McKenzie
Tam X

I found the system very cumbersome to use

Strongly Disagree 1 2 3 4 5 Strongly Agree
Cheryl X
Dylan X
Jessica X
Judith X
Loren X
McKenzie
Tam X

I felt very confident using the system

Strongly Disagree 1 2 3 4 5 Strongly Agree
Cheryl X
Dylan X
Jessica X
Judith X
Loren X
McKenzie
Tam X

I needed to learn a lot of things before I could get going with this system

Strongly Disagree 1 2 3 4 5 Strongly Agree

Cheryl X
Dylan X
Jessica X
Judith X
Loren X
McKenzie
Tam X
K.
Forms
Video release
_______________________________________________________________________
Photo/Video Release Form I, ______________________________________(please print),
grant permission to UALR and its agents and employees the irrevocable and unrestricted right
to reproduce the photographs and/or video images taken of me for the purpose of Usability
testing of the Greater Second Baptist Church’s website. I hereby release UALR and its legal
representatives for all claims and liability relating to said images or video. Furthermore, I grant
permission to use my statements that were given during session, with or without my name, for
the purpose of reporting without restriction. I waive my right to any compensation. I
acknowledge that I am
[ ] over the age of 18
[ ] the legal guardian of the following If legal guardian of model(s), please list name(s) here:

Name(s): ________________________________________________________________
Signature: _________________________________________Date: _________________
Address: ________________________________________________________________