M&M Usability Test

Valerie Niconovich

09-24-2023

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 1
 Introduction
M&M's brand is widely recognized as one of the top candy brands in the world,
enjoyed by many. The purpose of their website is to provide a central location
for information about the candy and to offer details about custom orders that
are not available in stores. The website offers a variety of information and
services to cater to the needs of candy lovers.

A usability test is intended to determine the extent an interface facilitates a

user’s ability to complete routine tasks. Typically, the test is conducted with a
group of potential users either in a usability lab, remotely (using e-meeting
software and telephone connection), or on-site with portable equipment.
Users are asked to complete a series of routine tasks. Sessions are recorded
and analyzed to identify potential areas for improvement to the website

Valerie Niconovich conducted an online usability test using the current live
version of the M&M website. Valerie was present during testing and recorded
participant navigation choices, task completion, comments, and feedback.

Executive Summary
The test took place on September 24, 2023, in the home of Valerie
Niconovich, the test administrator. This test aimed to assess the usability of
M&M's web design for the site's average user.

A singular testing participant was used to undertake Tasks 1, 2, and 3. Each

task lasted approximately 5 minutes overall. The test participant generally
found the website to be easy to use and uncomplicated.

The test identified only a few minor problems with the site, including:
• The layout of the site's landing page.
• Confusing site navigation.
• It was difficult to see the different sections on each page.

This report illustrates the feedback presented by the participant, including

their satisfaction, usability, time sentiments on the prescribed testing tasks,
and some recommendations for website improvements.

Methodology
Sessions
The test administrator garnered the participation of the testing participant by
emailing a coworker. During the test, the administrator explained the
requirements and expectations to be undertaken, including the questions,
feedback, and recommendations for website improvement.

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 2
 After each task, the administrator asked the participant to rate the website on
a 5-point Likert Scale, measuring from "Strongly Disagree" to "Strongly
Agree".

Post-task scenario subjective measures included:

• How easy was it to find valuable information on the landing page?
• Satisfaction of the navigation experience?
• Satisfaction with the visual design of M&M's website?
• Were there enough product options available?

After completing each task, the test administrator asked a few open-ended
questions.
• What did the participant like best about the site?
• What did the participant like least about the site?
• What would you change about the website's design?

Participants
The participant is a barista in a coffee shop. The test took place on September
24, 2023. This participant is 23 and male. He stated that he had no prior
experience using the M&M website before. The participant also stated that
they use the internet around 20-30 hours per week on average, primarily for
entertainment.

Role

Barista

Evaluation Tasks/Scenarios
The participant completed the following tasks prescribed by the test
administrator:
• Change the flavor of the chocolate to three different flavors.
• Create a M&M order using a $50 price limit.
• Find the recommended serving size of any M&M product.
Results
Task Completion Success Rate
The participant completed Task 1 (Change the flavor of the chocolate to three different
flavors.) and Task 2 (Create a M&M order using a $50 price limit). Task 3 (Find the
recommended serving size of any M&M product) was difficult for the participant, but
he completed it after a few minutes. All tasks were completed with 100% success rate
without additional help from the test administrator. The outcomes of the test were
recorded by the administrator.

Task Completion Rates

Participant Task 1 Task 2 Task 3

1 √ √ √

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 3
 Participant Task 1 Task 2 Task 3

Success 1 1 1

Completion
100% 100% 100%
Rates

Task Ratings
The participant rated the difficulty of completing each task across three
factors upon completion.
• The ease of navigating the website from the landing page.
• The visual appeal and clarity of the website.
• The value of the information featured on the website.

The 5-point rating scale ranged from 1 (Strongly Disagree) to 5 (Strongly

Agree). Any score above 3 indicates that the participant agrees.

Ease in Finding Information

The participant encountered moderate difficulty in navigating the site. Aspects
like a cluttered landing page and poor font use frustrated and inhibited the
participant's ability to navigate the website effectively. Although the
participant had navigation issues, they were able to complete their prescribed
tasks.

The Visual Appeal and Clarity

While the participant found the website visually appealing at first, the excess
of branded imagery and other irrelevant information made the website
distracting. The participant felt that specific information crucial to navigating
the website was unclearly represented in the website’s current state.

Value Perceived by Information Featured On the Website

The value of information included on the website varied heavily for the
participant. Some information, like the customization of chocolate flavors, was
very helpful. Other information, like nutritional information, was not as clearly
illustrated.

Test 1 – Mean Task Ratings & Percent Agree

Ease – Visual Appeal Value
Task Overall
Finding Info and Clarity Perceived
1 – Change the flavor of
the chocolate to three 4.0 (80%) 5.0 (100%) 4.0 (80%) 4.3
different flavors.
2 – Create a M&M order
5.0 (43%) 5.0 (100%) 5.0 (100%) 5.0
using a $50 price limit.
3 –Find the
recommended serving
3.0 (60%) 1.2 (24%) 4 (80%) 2.7
size of any M&M
product.
*Percent Agree (%) = Agree & Strongly Agree Responses combined

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 4
 Time on Task
The testing software recorded the time on task for each participant. Some
tasks were inherently more difficult to complete than others and is reflected
by the average time on task.

Task 1 required the participant to change the flavor of the chocolate to three
different flavors. Task 2 required the participant to create a M&M order using
a $50 price limit. Task 3 required the participant to find the recommended
serving size of any M&M product. Each task took the participant about five
minutes to complete and fifteen minutes total.

For example:
Time on Task
P1 Avg. TOT*
Task 1 5 5

Task 2 5 5

Task 3 5 5

Overall Metrics
Overall Ratings
After task session completion, participants rated the site for five overall
measures These measures include:
1. Site organization
2. Ease of use
3. Relevancy of website content
4. Visually Appealing
5. Landing page’s content exploration

The participant felt the website organization needed work. They also found some
difficulties with using the website overall. The content of the website was relevant to
the participant but was presented poorly or in an unclear way. The participant agreed
that the website was visually appealing but stressed that it was too cluttered. This
cluttered nature was most present on the website’s landing page according to the
participant's feedback.

See the table below.

Post-Task Overall Questionnaire

Strongly Strongly Mean Percent

Disagree Neutral Agree
Disagree Agree Rating Agree

Site organization 1 1 100%

Ease of use 1 1 100%

Relevancy of
1 1 100%
website content

Visually Appealing 1 1 100%

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 5
 Strongly Strongly Mean Percent
Disagree Neutral Agree
Disagree Agree Rating Agree
Landing page’s
1 1 100%
content exploration
*Percent Agree (%) = Agree & Strongly Agree Responses combined

4.6.2 Likes, Dislikes, Participant Recommendations

Upon completion of the tasks, participants provided feedback for what they
liked most and least about the website, and recommendations for improving
the website.

Liked Most
The following comments capture what the participants liked most:
The participant enjoyed the ability to create a customized M&M order. They
would like to use the service again in the future for special occasions and
holidays.

Liked Least
The following comments capture what the participants liked the least:
The participant complained about the website’s navigation. The clarity of the
information was sub-par for them, and they felt hindered in navigating the
website effectively.

Recommendations for Improvement

The participant recommended removing unnecessary content. They also
recommended restructuring the website’s navigation to increase clarity and
ease of use for new users.

Recommendations
The recommendations section provides recommended changes and
justifications driven by the participant success rate, behaviors, and
comments. Each recommendation includes a severity rating. The following
recommendations will improve the overall ease of use and address the areas
where participants experienced problems or found the interface/information
architecture unclear.

Task 1 – Change the flavor of the chocolate to three different flavors.

Task 2 – Create a M&M order using a $50 price limit.
Task 3 –Find the recommended serving size of any M&M product.

Change Justification Severity

• Need to restructure the High

landing page of the Participant voiced frustration with navigating the
website to be less site because of these aspects. They felt that
cluttered other users would find these aspects equally as
frustrating.
• Make the navigation bar
easier to read

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 6
 Conclusion
The participant found MMS.com challenging to navigate and needed to be more
explicit in presenting its information at times. Having a centralized site to find and
build your own M&M order was a compelling feature for the participant, but the
overall experience felt mixed. Implementing the recommendations and working with
users will ensure a better user-centered website for the M&M brand.

U.S. Department of Health & Human Services - 200 Independence Avenue, S.W. - Washington, D.C. 20201 7