user interviews company logo

The UX Research Field Guide

>

UX Research Methodologies

>

Qualitative vs. Quantitative Research

Qualitative vs. Quantitative Research

Knowing when to use which type of research is an essential instinct for UX researchers to hone. In the
last chapter, we gave an overview of the different types of user research methods. In this chapter, we’re
going to take a closer look at qualitative and quantitative research, in particular.

Why? Because combining qualitative and quantitative UX research methods is the most effective way to
create a comprehensive portrait of your customers’ wants and needs.

Properly acquired and analyzed, the high-volume of data in quantitative research helps you uncover
what is happening—trends, issues, and opportunities—and prove hypotheses.

Qualitative research, on the other hand, adds a layer of humanity to user data, delivering details that
add depth and a deeper understanding of not only what’s happening, but why it’s happening.

The trick to choosing the right user research method is knowing what kind of data you need to answer
your research question. In this chapter, we’ll go over the differences between qualitative and
quantitative UX research methods, the pros and cons of each one, and how they complement each
other in a mixed methods approach.

In this chapter:

The differences between qualitative and quantitative research

When to do qualitative vs. quantitative user research

Research methodologies for both qualitative and quantitative research

Recruiting participants for qualitative vs quantitative studies

Qualitative vs quantitative data analysis

The differences between qualitative and quantitative research

The primary difference between qualitative and quantitative UX research lies in the nature of the data—
which impacts how it is collected and then analyzed.
a simple user research decision tree for qualitative vs quantitative methods

A simple decision tree for quant v. qual

In qualitative research, researchers gain insight through direct observation of a small group of people,
often engaging in follow-up questions that dig deeper into the “why” behind certain attitudes and
behaviors. Qualitative research can also be adapted on the fly as researchers adjust study protocols
based on participant engagement and response.

Common qualitative research methods include interviews, focus groups, field studies, qualitative
usability tests, and co-design sessions.

Quantitative research, on the other hand, usually involves collecting large quantities of numerical data
from a much larger group of people, often indirectly through surveys or analytics tools. Insights are
derived using mathematical analysis of the data to quantify a problem by answering the questions,
“How much?” and “How many?” Because statistical significance is a key goal of qualitative research,
testing protocols are rigid and not subject to change over the course of a study.

Common quantitative research methods include user tests, surveys, click tests, card sorts, and A/B tests.

Let’s take a closer look

Qualitative vs quantitative UX research methods

a table comparing quantitative and qualitative research methods

Qualitative vs quantitative UX research methods

Whatever kind of product you’re working on, and wherever you are in the product development cycle,
it’s important to know which kind of UX research will get you the answers you need.

When to do qualitative user research

We love qualitative UX research at User Interviews. Qualitative methods can provide valuable insight
when you are trying to:

Come up with new ideas.

Gain deeper insight into customer needs.

Identify design problems.

Formulate hypotheses
The use cases for qualitative research stretch across the product development and design cycle—when
you’re brainstorming, in discovery, validating a concept, testing the usability and desirability of a
finished product, preparing a go-to-market strategy, iterating post launch, doing a redesign… every step
of the way.

Qualitative research brings the human element to bear on whatever question you’re exploring. It lets
you hear what people think in their own words. It helps you compile detailed information that is both
valuable in itself and also a great jumping off point for additional research directed at solving specific
challenges.

When to do quantitative user research

Stakeholders love hard data. Compiling data to drive a strategic research agenda with management is
far from the only reason to engage in quantitative UX research, but it is often the catalyst for major
research efforts.

Quantitative research can help you to:

Establish critical benchmarks.

Identify problem areas.

Begin to define specific problems.

Elevate one solution over another.

Prioritize projects.

It can also lay the foundation for (and help you make the case for) doing the qualitative research you
need to find solutions to problems.

Quantitative UX research is most commonly applied when you already have a working product and are
trying to evaluate its usability. It can also be very effective at uncovering the answers to broad, high-
level questions through statistical analysis to either validate or disprove your hypothesis.

Qualitative research methodologies

Qualitative research methods—which may be behavioral or attitudinal, moderated or unmoderated,

remote or in-person—can be further categorized into five different types.

Some kinds of qualitative research (like grounded theory) are infrequently utilized in a UX research
context. They’re more commonly used by researchers in the fields of sociology, anthropology, and other
social sciences from which they originated. But that’s not to say you couldn’t use grounded theory for
UX research, if a problem called for it.

Five types of qualitative research

1. Ethnographic research

In ethnographic research methods, researchers observe user behavior in participants’ natural

environment. This approach is a powerful way to understand context, observe real-world product use,
and gain deep cultural insights about participants’ day-to-day lives.

2. Narrative research

Humans are wired for story, which is why narrative research often packs an outsized punch. In this
approach, researchers run in-depth interviews with a very small number of participants in order to
create a cohesive narrative that reveals themes and patterns.

3. Phenomenology

Phenomenology uses a combination of interviews, observation, documentation, video recordings, etc. to

help researchers understand the what, how, and why behind a particular phenomenon or event. This
method helps describe and interpret lived experiences, which in turn helps uncover participants’
perceptions and motivations.

4. Grounded Theory

While phenomenology aims to describe an event, grounded theory aims to explain why an event
happens, by uncovering the social and psychological processes behind it. This is typically done through a
combination of user interviews with 20 to 60 participants and in-depth document research.

5. Case Study

The purpose of the case study is to relate, in detail, a real example of a specific type of experience. The
nature of a case study can be either explanatory or exploratory, but the aim in either case is to gain
access to deep understanding of how things happen in the real world.

Quantitative research methodologies

As with qualitative research, quantitative methods can be broken down into categories—which type of
quantitative approach is most appropriate depends on the nature of the issue you are researching, the
type of information you are after, and the study protocols involved.

The following decision tree illustrates the process:

example of a quantitative research decision tree

Source

Four types of quantitative research

1. Descriptive
Descriptive research uses a wide variety of methods to identify the characteristics and frequency of a
study topic while also looking at associated trends and categories. It relies on observation and
measurement to deliver insights into the what, where, when and how something happens.

2. Correlational

Correlational research looks at how two or more variables that are similar and interdependent relate to
each other. This type of research uses mathematical analysis to show how each variable affects the
other. Results are often presented using diagrams or statistics.

3. Quasi-experimental (aka causal-comparative)

This type of research looks at the cause-and-effect relationship between two variables—one dependent
and one independent—that are not related to each other.

4. Experimental

Used to verify an argument, experimental research takes a theoretical approach that focuses on a
theory and then helps researchers identify whether a given statement is right or wrong.

Recruiting participants for qualitative vs quantitative studies

Quality research—whether it’s qualitative or quantitative—requires quality participants. But the

challenges of recruiting are different depending on which research methods you’re using.

In quantitative research, you start with a large ‘population’—the entire group you want to study—and
then you create a ‘sample,’ which is a subset of the specific individuals who will participate in the study.
This process is often called ‘sampling.’ For this to work, you need two things: 1) a large sample size, and
2) a solid sampling design that guarantees a random sample, which is critical to ensure truly accurate
results.

In qualitative research (especially for B2B or other highly targeted studies) the key is finding the perfect
participants—people who fit your customer profile exactly in terms of not only demographics and
geographics, but also psychographics, behavior, and specific criteria relevant to the study parameters.
Recruiting participants for qualitative studies involves non-random sampling and appropriate screening
to deliver the best results.

User Interviews can help with both types of recruiting, but we’re especially great at helping researchers
fill qualitative studies that require vetted, good-fit participants who meet the specific criteria of your
study.

Find legit participants for your research study

Sign up for free

Qualitative vs quantitative data analysis

Once you’ve collected your data, what next?

Regardless of whether your data is qualitative or quantitative, it isn’t much use to anyone until it’s been
analyzed and synthesized. The methods you use to analyze your data will depend on the methods you
used to gather it—and unsurprisingly, there’s quite a lot of difference between quantitative and
qualitative research analysis.

We’ll be covering user research data analysis in more detail in a later chapter. But here’s what you need
to know in a nutshell.

Analyzing qualitative data

Qualitative data to yield a wealth of information, but not all of it is relevant to your research goals.
Qualitative analysis involves sifting through the raw data to find patterns, themes, and stories that tell
you something meaningful about the product, the user, or both.

Qualitative findings are:

Estimates, impressions, and interpretations

Informed by the knowledge and experience of the researcher

Unstructured or semi-structured in nature

They are used to determine:

General strengths and weaknesses of a design or process

Average scores across multiple participants

Themes and trends

Correlation or causation between two or more variables

They are analyzed using:

The application of metadata to help organize unstructured data

Qualitative content analysis, which tracks instances, positioning, and intended meaning of specific words
and phrases

Thematic analysis, which identifies themes and patterns

Discourse analysis, which looks at the nature of communication within a specific social context

Analyzing quantitative data

Quantitative analysis is comparatively (deceptively) straightforward. It involves a lot of number

crunching—but what you are doing at heart is trying to understand how people use a certain product,
what problems they may be experiencing, and where improvements could be made by looking for
patterns in the data.

Quantitative findings are:

Statistically meaningful

Replicable

Measurable data points expressed in numerical form

Structured data that is easy to organize and search using a relational database

They are used to determine:

How many

How much

How frequently

They are analyzed using:

Mathematical and statistical analysis

📖 Read more about UX Research Analysis and Synthesis

Comparing advantages and disadvantages

We know there’s a lot of information in this chapter, so before we wrap up with a brief discussion of
mixed methods, let's take a moment to summarize what we’ve learned about each type of research with
a list of pros and cons.

Advantages of qualitative research

Gives you first-hand insight into what people are really thinking and feeling

Reveals nuances and subtleties that quantitative data conceals

Uncovers the cause-and-effect connections that shape experience and drive behaviors

Works with a smaller sample size

Allows for a flexible and nuanced approach

Provides study participants with a more expansive way to express their feelings and share their
experience

Delivers emotionally driven, narrative-style evidence that can be very persuasive

Reveals why something is happening

Helps identify solutions to UX problems

Inspires future studies

Disadvantages of qualitative research

Incremental expense—can have a higher upfront cost than quantitative research since most teams
already run some sort of analytics by default

Time required—can take a long time to coordinate and run (though getting help with recruiting, one of
the most time-consuming parts of the process, can streamline things substantially)

Risk of researcher bias, which can influence conclusions consciously or subconsciously. Learn how to
mitigate your personal biases by practicing reflexivity as a qualitative researcher.

Non-traditional validation—the subjective, open-ended nature and small participant pools do not align
with conventional standards for reliability and validity such as statistically representative data

Non-replicable—neither the study nor the results can be replicated since you cannot control for
variables like the context, conditions, researcher knowledge, researcher approach, etc.

Narrow context—you cannot extrapolate results to make confident generalizations about a broader
audience.

Challenging analysis—accurate interpretation requires expert knowledge of both the subject matter and
qualitative research methodology.

Advantages of quantitative research

Provides some level of statistical significance

Reduces random noise

Allows for fast results

Lends itself to rapid analysis

Delivers comparatively more objective results*

Supports helpful data visualizations

Simplifies connecting the dots between UX improvements and team objectives

Enables test replication

*Caveat: The aim of quantitative research is to provide unbiased and objective results. But, despite best
intentions, there are always variables that can influence a study. Quantitative methods do offer a
greater opportunity to find statistical significance, but only when the data is collected, analyzed, and
presented correctly. For example, if a survey design is biased—even unintentionally—the data will also
be biased. Excellence in quantitative research requires a high level of expertise and knowledge about
each step in the process.

Disadvantages of quantitative research

Limited insight—quantitative research can reveal the what, but it can’t tell you the why. It can tell you
that there is an issue, but it can’t necessarily identify the specific problem, and it doesn’t provide a
solution.

Risk of confirmation bias—researchers may overlook certain generative insights due to an over focus on
evaluative testing.

Limited context—participants’ response options are narrowly defined. They do not have the opportunity
to explain their choices or ask for clarification about questions.

Reliance on researcher assumptions—the context-bound nature of responses means that researchers

need to make their own assumptions about how participants interpreted questions and why they chose
certain responses.

Large sample size requirement—statistical significance requires a large pool of participants.

High analytical skills required—an inadequate knowledge of how to apply statistical analysis can
influence data interpretation and negatively affect the accuracy of results.

For best results, mix methods

Oftentimes, the best way to answer your research question with both certainty and nuance is to take a
mixed methods approach.

Mixed methods user research is exactly what it sounds like—the practice of using both kinds of research
methods in a single study. Side note: If you’re blending multiple qualitative methods (and you often will)
or multiple quantitative methods—but not the two together—that’s referred to as hybrid research.
Similar, but different.

Many UX researchers opt for a mixed method approach. That’s because quantitative and qualitative
methods are actually highly complementary. Quantitative research helps you identify specific problems
by measuring the ‘what’ and the ‘how’, and provides hard data that can be quickly analyzed and
understood. Meanwhile, qualitative data helps you uncover the ‘why’ behind an issue or opportunity.
Each type of data helps fill in the gaps left by the other, giving you a holistic picture of exactly what’s
happening, why it’s happening, and how you should address it.
Mixing methods also helps you avoid making false assumptions that can cause you to travel down the
wrong path, wasting time and resources on developing a solution that users don’t really need.

For example, if quantitative research shows that a particular step in your conversion journey takes twice
as long as any other part of the process, you might assume that that step is simply more time-
consuming. However, if you then run a qualitative test to observe how people do that step and collect
direct feedback about the experience, you may learn that the instructions are confusing and users are
getting hung up on trying to interpret them.

The key to successful UX research is knowing which methods will give you the answers you need, and
how they can be combined with other methods to give you the most complete and accurate picture
possible.

Which is an excellent segway into the next chapter, How to Choose a User Research Method, which is all
about the frameworks user researchers use to decide on the specific methodologies their research
question requires.