Ace Hardware Test

Alex Parkman

2/10/2024

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 1
 Introduction
Acehardware.com serves as a place to view different things you can
buy from the store Ace Hardware. The website acts as a centralized
repository, offering easy accessibility to a variety of information and
resources within the Ace Hardware network.

A usability test is intended to determine the extent an interface facilitates a

user’s ability to complete routine tasks. Typically the test is conducted with a
group of potential users either in a usability lab, remotely (using e-meeting
software and telephone connection), or on-site with portable equipment.
Users are asked to complete a series of routine tasks. Sessions are recorded
and analyzed to identify potential areas for improvement to the web site.

The AceHardware.com usability engineers conducted an onsite

usability test using a live version of AceHardware.com located on the
test administrator’s laptop. The usability engineers captured the
participant’s face and comments using a phone camera recording the
reactions. The answers were logged in Microsoft Word.

Executive Summary
The AceHardware.com project team conducted an onsite usability test
at my house in Dahlonega on February 10th, 2024. The purpose of the test
was to assess the website's usability and ease of use.
One Attendee participated in test 1, the test lasted about 20 minutes.
He found the website to be very clear and straightforward and
completed the task with ease in about 2 minutes
The test determined a few issues with the website design, including:
• A clogged-up screen with lots of information was sometimes
overloading the participant’s emotions.
• Bad aesthetic color choices
• Confusing AI for older generations.
• Lack of reviews.

Methodology
Sessions
The test administrator contacted and recruited the participant from the
UNG intramural team list. The test administrator met up face to face with the
participant and asked him if he would be interested in participating in the
test, because of his background in tools.
The session lasted about 30 minutes; The testing session commenced
with the test administrator providing a comprehensive overview of the
process. Additionally, the participant completed a brief background
questionnaire to capture relevant demographic data. Subsequently, we
engaged with the provided task scenarios and actively navigated the Ace
Hardware website to locate the required information.

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 2
 After each task, the administrator asked the participant to rate the
interface on a 5-point Likert Scale with measures ranging from Strongly
Disagree to Agree Strongly.
• How easy was the website to use?
• How clear were the items to find?
• How easy was the text to read?

After this, the participant was asked to answer four more scaled
questions.
• Understandability
• Happiness
• Stress
• Satisfaction
Lastly, the administrator asked the participant three overall open-
ended questions.
• What did you find most helpful or user-friendly on the Ace
Hardware website?
• What was the most difficult or frustrating aspect of using the
website?
• If you could change one thing about the Ace Hardware website,
what would it be?

Participants
The participant is a student at UNG and a self-proclaimed
“Handyman.”
He was scheduled for a testing date on February 10th at 1 PM.
Role In Tools community:
Frequently shops at Ace Hardware, Home Depot, and Lowes. Does not do
handy work for occupation.

Evaluation Tasks/Scenarios
The participant attempted to complete the following tasks:
• Task 1: You need to find a hammer, but don’t need a specific brand
or model. You want something under $50 and durable.
• Task 2: You must order a gallon of blue interior paint in a specific
shade for a bedroom makeover.

Results
Task Completion Success Rate
The participant successfully completed both Tasks.

Task Completion Rates

Participant Task 1 Task 2
1 √ √
Completion
100% 100%
Rates

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 3
 Ease in Finding Information
The participant agreed it was easy to find a hammer that was under
50$ and was durable, he gave it a 5 out of 5 rating on ease. For the second
task, he had a few more issues, but still completed it with a 3/5 rating on
ease.

Keeping Track of Location in Site

The participant found it easy to keep track of his location in the site
when looking for the hammer (5) but when he was prompted to find the
paint, he struggled a little bit with choosing between the different types of
paint (3).

Predicting Information Section

The participant agreed that it was easy to predict where to find the
hammer (5) but the participant believed that it was relatively hard to find the
specific paint he needed (2).

[Display the results in a table (see example tabular display).]

Test 1 – Mean Task Ratings & Percent Agree
Ease – Location in Predict
Task Overall
Finding Info Site Section
1 – Find <$50 Hammer 5 5 5 5

2 – Find Blue Paint 3 3 2 2.66

*Percent Agree (%) = Agree & Strongly Agree Responses combined

Time on Task
The testing software recorded the time on task for each participant. Some
tasks were inherently more difficult to complete than others and is reflected
by the average time on task.

Task 1 took the participant 2 minutes to solve, and task 2 took six
minutes and thirty seconds to finish.

For example:
Time on Task
P1
Task 1 120

Task 2 395

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 4
 Overall Metrics
Overall Ratings
After task session completion, participants rated the site for eight overall
measures (See Attachment insert attachment letter here). These measures
include:
• Ease of use
• Frequency of use
• Difficulty of keeping track of where they were in the site
• How quickly most people would learn to use the site
• Getting information quickly
• Homepage’s content facilities exploration
• Relevancy of site content
• Site organization

Post-Task Overall Questionnaire

Strongly Strongly Mean

Disagree Neutral Agree
Disagree Agree Rating

Thought Website
1 4
was easy to use
Would use website
1 3
frequently
Found it difficult to
keep track of
1 4
where they were in
website
Thought most
people would learn
1 3
to use website
quickly
Can get
1 4
information quickly
Homepage’s
content makes me
1 2
want to explore
site
Site’s content
would keep me 1 2
coming back
Website is well
1 3
organized
*Percent Agree (%) = Agree & Strongly Agree Responses combined

4.6.2 Likes, Dislikes, Participant Recommendations

Upon completion of the tasks, participants provided feedback for what they
liked most and least about the website, and recommendations for improving
the website.

Liked Most
The following comments capture what the participants liked most:
• “When you’re looking for something broad, you will get a lot of
results that you will be interested in.

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 5
 Liked Least
The following comments capture what the participants liked the least:
• “I didn’t like the UI.”

Recommendations for Improvement

• “Fix the search bar and get rid of ads.”

Recommendations
The recommendations section provides recommended changes and
justifications driven by the participant success rate, behaviors, and
comments. Each recommendation includes a severity rating. The following
recommendations will improve the overall ease of use and address the areas
where participants experienced problems or found the interface/information
architecture unclear.
Use AI to make the search more accurate for specific items.

Change Justification Severity

• Use AI to make the search
more accurate for specific
items.
The Participant was not able to easily find the High
specific blue paint that he wanted, he was
instead bombarded with other types of paint,
such as paint markers and spray paint. This will
be a website killer for older and less tech-savvy
people.

Conclusion
The participant found AceHardware.com to be well set up and easy to use,
until he was forced to find a specific item. Then he was frustrated with the results
not being what he was looking for, which lowered his overall perception of the site
and the brand behind it. Using more specific keywords will ensure a better and more
successful website.

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 6