The

complete
guide
TO USER TESTING
WEBSITES, APPS,
AND PROTOTYPES
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS, AND PROTOTYPES
Understanding your customers is the key to any
successful business, to ensure that you provide
them with what they want and minimize what they
find frustrating or challenging.
Whether it’s a physical product or a digital
experience, including websites or apps, you’re more
likely to meet—and exceed—customer expectations
if you consider customer feedback throughout the
product or campaign ideation, development, and
optimization process. That way, in every step of the
process, you’re creating solutions and experiences
that match what your customers are seeking.
UserTesting is an on-demand human insights platform that allows you to get
qualitative data to understand intention and motivation driving customer
behaviors. Understanding why customers make the decisions that they do
helps you design and provide digital experiences that your customers will love.
With UserTesting, teams across your organization—from product to marketing
to UX research and design—are able to:
• Uncover customers’ needs and challenges as the starting point for
designing solutions and campaigns
• Make faster, more informed decisions based on known customer needs
• Take a customer-first approach in everything they work on
In this guide, we’ll cover how to plan, conduct, and analyze your customer
studies. Well-planned and well-executed studies will uncover actionable,
qualitative data that your teams can use to make faster, better decisions.
And this results in delightful experiences that keep customers satisfied and
coming back for more.
2
THE COMPLETE GUIDE TO UER TESTING WEBSITES, APPS, AND, PROTOTYPES

1. The first step toward gathering useful feedback is

Define your setting a clear and focused objective. If you don’t

know exactly what sort of information you’re looking
objective to obtain before you conduct your research, you risk
launching a study that fails to yield actionable insights.

3
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS AND PROTOTYPES

ASK YOURSELF: WHAT AM I TRYING TO LEARN?

Remember: you don’t need to uncover every issue or problem in a single, Example of a complex objective:

exhaustive study. While it’s tempting to try to cover as many challenges as

Can users easily find our products and make an informed
possible with your study, it’s much easier and more productive to run a series
purchase decision?
of smaller studies with one specific objective for each. You’re more likely to get
focused feedback that guides you towards a solution or supports a specific This objective contains three very different components
decision or change.
1. finding a product, 2. getting informed, and 3. making a purchase.

As you set your objective, think about the outcomes, results, or KPIs that have
Example of a better objective:
greatest impact on your business outcomes. This sets the focus on gathering
Can users find the information they need?
insights that align with strategies and activities that drive positive business
outcomes.

For example, if you notice that customers who visit a certain page of your
website don’t typically convert, you can run a customer study on that page
related to that KPI. Remember to ask open-ended questions to better
understand customers’ perceptions and how those opinions influence
their choices and behaviors. You could also then ask them to try to make a
purchase, to see if a usability issue impedes this activity. (Remember that if
you ask a study participant to complete a purchase on your site, you should
provide a promotional code.)

4
THE COMPLETE GUIDE TO UER TESTING WEBSITES, APPS, AND PROTOTYPES

When creating a new test using the UserTesting

platform, you can choose to create one from scratch,
or you can use a premade template or create one
based on the framework of a previous test.

2.
Create your test
NEW TEST FROM SCRATCH

Selecting the Test button will create a brand new test from scratch.

NEW TEST USING TEMPLATES

If you’re new to conducting customer studies and you’d like a bit more
guidance on structuring your study plan, try our pre-made templates. The
team at UserTesting created a series of templates based on popular and
frequently used objectives across product and campaign development cycles.

You can access them with just a single click from your UserTesting dashboard.

5
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS AND PROTOTYPES
You’ll see a list of template options. Scroll down and select the website or app
option that meets your needs, then review and customize before launching
your test.
NEW STUDY SIMILAR TO EXISTING TEST
You can also create a new test using the framework of an existing study.
First, locate the test you wish to replicate. Click on the “...” and then select
Create similar test.
You’ll then see the tasks and questions from the original test, which you can
edit until it’s finalized and and ready to share.
You can also add additional participants to a previously launched unmoderated
study by selecting Add more participants. You may choose this option if a
previously-run study didn’t yield conclusive findings and you want to get more
feedback before making decisions on next steps.
6
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS, AND PROTOTYPES

3.
Depending on your subscription, you may have the
Identify the best choice between conducting a recorded test or a live

way to get the interview. No hard and fast rules exist about which
option is the better choice for a given situation, but

answers you’re you should try to pair your expected outcome with the
type of study that will support your efforts.
seeking

7
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS, AND PROTOTYPES

RECORDED TEST

A self-guided test or study, also known as an unmoderated study, is an

unsupervised test completed by participants in their own time. You create
the test with a series of prompts, including tasks and questions, which the
participants follow as they complete the test. You’ll receive a recording of the
completed test, where you can see the participant’s screen and what each
participant is viewing as opinions and actions are narrated.

Recorded tests are great if you have a product, like a website or app, and
you want the participants to go about an activity or action as they would
in real life. For this reason, they tend to be about 10-20 minutes in length.
Because participants are completing this study on their own, and in their
native environment, it’s as close as you can get to being “a fly on the wall”—
observing how the consumer would naturally act and make decisions.

LIVE CONVERSATION
With UserTesting, you can choose between unmoderated studies or conducting live interviews
using Live Conversation
Live Conversation allows you to schedule and conduct real-time interviews.
By sharing your availability through the calendar, you can schedule live video
interviews as early as that same day. Simply log in during your appointment
time, and you can engage in dynamic conversation with the study participant,
either from the UserTesting panel or your unique audiences (using My Recruit).

Interviews are great if you’re looking to investigate a topic or have a free-form

conversation, such as a discovery call or early in the product development
cycle when you’re still learning about customer pain points and problems. For
this reason, Live Conversation studies tend to be 30-60 minutes in length.

8
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS, AND PROTOTYPES

Next, you need to share the product on which you’re

seeking feedback.

Product types include live properties, such as websites and apps,

or unreleased products, like prototypes, or websites and apps still in
development.

4.
Identify what
you’re studying
With UserTesting, you have the choice of testing a wide range of digital experiences

Conducting a test on live products entails providing a study participant with

a URL or the name of the app in a digital marketplace (such as the App Store
or Google Play). Similarly, many prototyping tools, such as InVision, or file
hosting solutions, such as Google Drive or Dropbox, allow you to download a
link for sharing the prototype. You can share the URL to get feedback on your
prototype (make sure to open sharing permissions to ensure that all study
participants are able to view your content).

The easiest way to test an app, released or unreleased, is to upload the .IPA or
.APK file during test setup. We’ll take care of all provisioning and certification,
and we’ll automatically upload the app to the participant’s mobile device at the
start of the test.

For more information testing mobile experiences, you can visit our Help
Center or refer to this mobile testing document. 9
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS, AND PROTOTYPES

In many cases, it’s helpful to get feedback from a

wide range of participants, spanning age, gender, and
income level. This helps you understand how a wider
audience perceives your products—whether products
are easy to use and relevant to many, or whether they
present challenges to some groups.

Or perhaps you have a very specific customer profile or persona in mind, and

5.
you want to make sure that you get feedback from people who meet a list of
criteria.

Identify the types

Depending on your subscription, you have the choice to share your study with
the UserTesting panel or with members of your network using My Recruit.

of people you want

to include in your
study

With UserTesting, you can get feedback from the proprietary, on-demand panel or your own
audiences using My Recruit

10
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS, AND PROTOTYPES

USERTESTING PANEL

The UserTesting panel is a proprietary, on-demand panel, with participants

located around the globe. People apply to be a part of the panel, and if they
pass our screening criteria, they become panel participants. They receive
notifications when a test matching their profile is available and can choose
to accept it. When it comes to unmoderated studies, most are completed
and returned by panel participants within 1–2 hours, because of the size and
responsiveness of our panel.

If you want to target a specific persona, you can adjust the demographics
and write screener questions when setting up each participant group within
your study. You can ask screener questions related to the participants’
psychographics, such as hobbies, attitudes, behaviors, professions and titles,
or anything that will help you locate precise participants.

You can select a single persona per study. Or, creating multiple participant You can create different groups within a single study, to get feedback on the same experience from
groups within the same study allows you to get feedback on the same multiple audiences

experience from different groups of people.

How to decide which audience(s) to study

MY RECRUIT
My Recruit allows you to share your study with unique audiences that might
be difficult to find in a panel, including people with specific demographics or
backgrounds. All you need is a way of connecting with the audience members,
like email addresses or a social media network, so you can share your study
with them directly using our auto-generated link.
As a general rule, you can launch your study to the UserTesting panel
to understand how audiences at large interact with and perceive your
products, especially for B2C products. It’s a great way to minimize the biases
that might occur from only soliciting feedback from your current customer
base or people who are geographically close to you. Using the panel is also
a great way to get responses very quickly— often in under 1 hour.
My Recruit is a good solution when you want to get feedback from very
specific people, including those who work in certain industries, hold specific
job titles, or have esoteric knowledge, backgrounds, or experiences. You’ll
need a means of contacting these individuals, such as email addresses.

11
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS, AND PROTOTYPES

It’s been demonstrated that five participants will

uncover 85 percent of the usability problems on
a website and that additional users will produce
diminishing returns. Resist the temptation to “boil the

6. ocean” by doubling or tripling the number of participants

in an attempt to uncover 100 percent of your usability

Determine problems. It’s easier and more efficient to run a test with
five participants, make changes, run another test with a
how many test different set of five users, and continue iterating until all
major challenges are resolved.
participants to
include If you’re looking for trends and insights beyond basic usability issues,
it is helpful to include a larger sample size. We recommend five to eight
participants per audience segment. For example, if you have three distinct
buyer personas, you’ll want to set up each persona within a test, for a total of
15–24 participants overall.

12
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS, AND PROTOTYPES

If you’re creating an unmoderated test, start assembling

your study by creating a test plan. Your test plan is the

7. list of instructions that your participants will follow, the

tasks they’ll complete, and the questions they’ll answer

Build your test plan during the study.

(for unmoderated Task:

An action or activity that you want a participant to complete.

studies only) Example of a task:

Go through the checkout process as far as you can without actually

making a purchase.

Question:

Something that elicits verbal feedback from the study participant.

Example of a question:

Was anything difficult or frustrating about this process?

13
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS AND PROTOTYPES

CREATE A SERIES OF TASKS

Tasks are the actions that you want the study participant to complete. They
complete one task before moving onto the next, so keep the ordering of tasks
PRO TIP:
in mind as it will guide the study participant from start to finish. As they work
through completing each task, they’ll be sharing their thoughts out loud, which When you’re not sure where to focus your test, try this tip. Run a very open-
will be captured on video and saved to your dashboard when the session is ended test using broad tasks like “Explore this website/app as you naturally
finished. would for 10 minutes, speaking your thoughts aloud.” You’re sure to uncover
new areas to study in a more targeted follow-up test.
When you’re creating tasks for your study, focus on the insights you want to
gather about the specific objective you determined. If you have a lot of areas
you want to test, we recommend breaking these up into different studies. In
most cases, it’s best to keep your study around 15 minutes in length, so keep
this in mind as you plan your tasks.

Consider using both broad and specific tasks in your study
Broad, open-ended tasks give your test participants minimal explanation
about how to perform the activity. They’re meant to help you learn how your
users think, and are useful when you’re seeking insight on branding, content,
layouts, or any “intangibles” of the user experience. Broad tasks are good
for observing natural user behavior—what people would do when given
the choice and the freedom—which can yield valuable insights on usability,
content, aesthetics, and users’ sentimental responses.
Example of a broad task:
Specific tasks are defined activities that participants must follow. They
provide the participant with clear guidance on what actions to take and what
features to speak about. Participants focus on a specific action, webpage, or
app, and speak precisely their perceptions of this experience.
One thing to keep in mind: specific tasks are most appropriate to situations in
which you have identified a problem area or a defined place where you desire
feedback.
Example of a specific task:
On this page, select an item that you would want to buy. Share how you
went about making this decision and selecting the item.

Find a hotel that you’d like to stay in for a vacation to Chicago next month.
Share your thoughts out loud as you go.

One thing to keep in mind:

Responses may vary significantly from one test participant to the next. Be
prepared to get diverse feedback when using broad tasks.

14
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS, AND PROTOTYPES

Plan out tasks using a logical flow WRITE CLEAR QUESTIONS

The structure of your study is important. We recommend starting with broad
tasks (such as, exploring the home page, then using search, then adding an
item to a basket) and moving in a logical flow toward specific tasks (such
as putting items in their online shopping cart in preparation for a purchase).
The more natural the flow, the more realistic the study will be, putting your
participant at ease and enabling better, more authentic feedback.
Once you’ve mapped out a sequence of tasks in your test plan, start writing
the questions. It’s critical to structure questions accurately so you get reliable
answers and gain the insights that will support your recommendations and
business decisions.

Example of poor logical flow:

Create an account > find an item > check out > search the site > evaluate
the global navigation options

Example of good logical flow:

Evaluate the global navigation options > search the site > find an item >
create an account > checkout

How you lay out your test plan should align with the objective originally set
for your study. If you’re interested in discovering the participants’ natural
journey or acquiring better understanding of their motives and rationale as
they navigate your products, then give them the freedom to use the product in
their own way. However, if you’re more focused on understanding participant
attitudes and behaviors in a well-defined context, then be specific in the tasks
you assign and the questions you ask.

Additionally, starting with broad and then moving to specific tasks is important
if you suspect that a specific task will require the participant to do something
complicated or something with a high risk of failure. Putting these types of
tasks near the end of the study prevents the test participant from getting
stuck or being thrown off track in the beginning, which could negatively
impact the remainder of your study.

15
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS AND PROTOTYPES
TIPS FOR GATHERING RESPONSES
Use clear, everyday language
You want to provide a seamless experience for participants completing your
study. Avoid industry jargon or internal phrases that participants might not
know. Terms like “sub-navigation” and “affordances” probably won’t resonate
with the average user, so don’t include them in your questions unless you’re
certain your actual target customers use those words.
Include the time frame
If you’re asking about some sort of frequency, such as how often a user visits
a particular site, make sure you define the timeline clearly at the start of the
question. This ensures consistency and accuracy in the responses that you
receive.
Example of time frame in question:
In the past six months, how often did you visit Amazon.com?
Frame questions for more standardized responses
Gathering opinion and preference data can be tricky. To ensure that you
collect actionable insights that support changes and decisions (instead of a
range of diverging opinions), it is critical that you standardize the experience
so that participants answer the same question and provide helpful feedback.
Ensure rating scales are balanced
Be fair, realistic, and consistent with the two ends of a rating spectrum. If
you’re asking participants to select the more accurate response from two
options, ensure that they’re weighing each end of the spectrum evenly.
Example of an unbalanced scale:
After going through the checkout process, to what extent
do you trust or distrust this company?
I distrust it slightly ←→ I trust it with my life
Example of a balanced scale:
After going through the checkout process, to what extent
do you trust or distrust this company?
I strongly distrust this company ←→ I strongly trust this company
16
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS AND PROTOTYPES

Separate questions to dig into vague or conceptual ideas Avoid leading questions

Some concepts are complex and can mean different things to different Sometimes, when asking questions, you can inadvertently influence the
people. For example, satisfaction is a complex concept, and people may participant’s response by including small hints in the phrasing of the questions
evaluate different things when concluding whether or not they’re satisfied with themselves. In doing so, you influence the outcome of your studies because
a product or experience. you get biased, inaccurate responses and results.

To ensure that your study yields actionable feedback, break up complex
concepts into separate questions. Then, consider all of the responses in
aggregate when analyzing the results. You can even create a composite
“satisfaction” rating based on the results from the smaller pieces.
Be sure that your questions are neutral, unbiased, and free of assumptions.
Example of a leading question:
“How much better is the new version than the original home page?”

Example of question on a complex concept: “Compare the new version of the home page to the original.
Which do you prefer?”
On a scale of 1 to 5, with 1 being “Very satisfied” and 5 being “Very
unsatisfied,” how would you rate your satisfaction with this children’s
online learning portal?

Example of breaking up questions on a complex concept:

On a scale of 1 to 5, with 1 being “Very easy to use” and 5 being “Very difficult
to use,” how would you rate the ease of use of this children’s online learning
portal?

On a scale of 1 to 5, with 1 being “Very appealing” and 5 being “Very

unappealing,” how would you rate the visual design of this children’s online
learning portal?

Remember: one of the great benefits of qualitative research is getting to the

“why” of human behavior. For every question, if you find yourself asking “why”
a person might respond a certain way, you have the opportunity to include
additional questions to better understand the source of an opinion.

17
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS, AND PROTOTYPES

Before you launch your test to all participants, we

recommend conducting a dry run (sometimes called
a pilot study) with just one or two participants. This will
determine whether there are any flaws or confusing
instructions within your original test plan and provide the
8. opportunity to make adjustments and improve the test

Launch a dry
plan before launching it fully.

run test Here’s a good structure to do this:

1. When creating a study, select one or two participants.

Then launch that test.

2. Review your videos and listen as they process each task and
question. Take note of any trouble they encounter while trying to
complete the tasks and questions or if any responses fail to provide
the type, level, or quality of insight you expected.

3. After reviewing your videos of those one or two sessions to

ensure that you are getting the types of feedback you are expecting,
you can select Add participants to include additional test
participants in the study.

18
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS, AND PROTOTYPES

You’ll receive email notifications when your unmoderated

studies have been completed (or when your Live
Conversation sessions have been scheduled). Once
9. you’ve received your results, you’ll have the opportunity
to analyze the responses and begin extracting insights
Analyze to support your recommendations and decisions. You’ll
know that you have sessions awaiting your review based
your results on the status under the test name.

On your dashboard, you’ll be able to see if you have sessions awaiting your review

19
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS AND PROTOTYPES

As you review your study results, you can take notes using the Notes & Clips
feature in the video player. The timestamp that’s captured alongside each
note will correspond with the related portion of the recording, making it easy
to jump back to that section of the recording later.

As you review answers to your questions, keep an eye out for similar
responses and themes as well as any major deviations. If a significant number
of your study participants provide similar feedback, this could signal an issue
that impacts your larger customer base and deserves some attention. Or, if
one or a small number of participants shares a unique piece of feedback, you
can hone in on this particular video to better understand why that participant
or participants had such a different experience.

Take note of user frustrations as well as things that users find particularly
helpful or exciting. By knowing what people love about your product
experience, you avoid the possibility of “fixing” something that’s not really
broken. Hearing about things that customers struggle with, as well as
enjoy, can support informed discussions on future product and experiential
improvements.

20
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS, AND PROTOTYPES

After you’ve uncovered your findings, you can share

them with your team and stakeholders to begin
discussing next steps.

Here are a few ways to share findings with your team:

10. • Create a highlight reel that aggregates critical video clips. This is
especially helpful when you have multiple participants with the same

Share opinion—the curation of this feedback can be compelling. You can even
create an Instant Highlight Reel from within a single task, to watch in rapid

your findings
success as each of your participants completes the task.

You have the option of creating a highlight reel based on a single task

21
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS, AND PROTOTYPES

• From your dashboard, you can choose to export your study into an Excel
file. Learn more here.
PRO TIP:
• Create charts to represent interesting data and findings from your
questions. This can be used to visually convey the volume of participants It’s important for everyone to remain objective and neutral—both when

who say similar things (such as a word cloud) or to display the difference
in opinions.
• Share participant quotations from the studies to back up your hypotheses
presenting and hearing feedback. Be careful not to place blame on
teammates. If you have a lot of negative findings, choose your words
carefully. “Users found this feature frustrating” is much easier to hear than
“This feature is terrible.”

or recommendations. Hearing straight from the Voice of the Customer is a

And while receiving feedback can be tough, it’s important to reiterate that
powerful step in aligning team members and other stakeholders.
the feedback presents an opportunity to improve products and experiences
for customers. Encourage team members to ask questions about the
• Share directly to your public Slack channels to speed insights to your
findings, and remind them not to make excuses.They’re there to learn about
team. Learn more here.
the customer experience, not to defend their design decisions.

• You can also share insights on Trello. With the UserTesting Power-Up, you
can view completed videos right on your Trello board. Learn more here.

22
THE COMPLETE GUIDE TO USER TESTING WEBSITES, APPS, AND PROTOTYPES

We encourage you to gather customer insights

throughout your product and campaign development
process and across multiple teams and departments.

• Early in your process, it will help you better understand customer pain

11. points and challenges, in order to more effectively brainstorm and

support product and solution ideation. You can also hone in on the right

Champion a
customers and audiences to assess market opportunities and analyze
product-market fit.

customer-focused
• You can understand usability challenges in your prototypes or early stage
products to course-correct before you’ve spent much time or resources

attitude throughout
on development.

• As you launch new digital experiences, you can monitor how customers

your organization
are reacting to and interacting with these new products. This yields ideas
on how to continue evolving your product.

Getting frequent feedback and insights from your customers is the best way
to keep your finger on the pulse of customer challenges and expectations.
This will ensure that you’re making the right decisions and taking the right
steps towards ensuring ongoing customer loyalty and satisfaction.

23
About UserTesting

UserTesting is an on-demand human insight platform that quickly gives companies a first-
person understanding of how their target audience behaves throughout any experience
and why.To learn more, visit www.usertesting.com.

HEADQUARTERS
690 5th Street
San Francisco, CA 94107
__

UserTesting.com
1-888-877-1882

© UserTesting, Inc. All rights reserved