Becoming A Spy - Covert Naturalistic Observation - UXmatters

6/21/2020 Becoming a Spy: Covert Naturalistic Observation :: UXmatters
Research
(/Topics/)
Becoming a Spy: Covert Naturalistic

Observation
Practical Usability
(https://www.uxmatters.com/columns/practical-
usability/)
Moving toward a more usable world
A column by Jim Ross
(https://www.uxmatters.com/authors/archives/2009/10/jim_ross.php)
November 1, 2013

1 Comment (https://www.uxmatters.com/mt/archives/2013/11/becoming-a-spy-
covert-naturalistic-observation.php#comments)
 1 Share
As I mentioned in a previous column, user research is unnatural

(https://www.uxmatters.com/mt/archives/2011/04/user-research-is-
unnatural-but-thats-okay-part-i.php). When we conduct a user TOP
experience study, we can’t help but introduce unnaturalness. Following
https://www.uxmatters.com/mt/archives/2013/11/becoming-a-spy-covert-naturalistic-observation.php#:~:text=Covert Naturalistic Observation,that they're being observed.&text=It is the opposite of,forms of overt natur… 1/16
“
ethical research
guidelines, we inform When we conduct a user
participants about the experience study, we can’t
study and get their
help but introduce
consent to participate, but
unnaturalness. … We inform
when people know we’re
participants about the study
observing them, their
and get their consent to
behavior changes. This is
participate, but when people
known as the Hawthorne
know we’re observing them,
Effect. [1]
”
their behavior changes.
Most researchers are
aware of these limitations
of research, but after a while, we tend to take them for granted and may
forget how strange it can feel for a participant to be in a usability lab.
We may try to overcome this artificiality by going into the field and
observing people in their natural environments, which certainly is more
natural than conducting research in a lab. But knowing that we’re
observing them can still be an uncomfortable experience for
participants, and we may forget that our presence affects their behavior.
Covert Naturalistic Observation

So, how can we observe natural behavior if our mere presence affects what
people do? Don’t tell them that you’re observing them. At this point, you may be
thinking: Wait a minute. Isn’t that unethical? I’ll get to that, so please read on. TOP
“
This type of study is known in
psychology, anthropology, and other Being covert means observing
social sciences as covert naturalistic behaviors in their natural
observation. It is the opposite of the contexts without any
techniques we typically use, which intervention or influence by
are forms of overt naturalistic the researcher and without
observation. Being covert means participants knowing that
”
observing behaviors in their natural
they’re being observed.
contexts without any intervention or
influence by the researcher and
without participants knowing that they’re being observed.
Covert naturalistic observation isn’t a completely new idea. In addition to its use
in the social sciences, it’s practiced by law enforcement agencies, spies, and
amateur people-watchers all over the world. But for some reason, its use in user
experience research is somewhat rare.
What Are Its Advantages?
“
By ensuring that participants are
unaware that we’re observing them, We can see what people really
we eliminate any effect that we have do rather than what they say
on participant behavior—the they do or what they show us
Hawthorne Effect. We can see what when they know we’re
”
people really do rather than what studying them.
they say they do or what they show
us when they know we’re studying
them. What better way to see natural behavior than to go out and observe what is
really happening outside a study?
What Are Its Disadvantages?

TOP
“
Being covert requires you to hide the
fact that you’re observing, which Being covert requires you to
makes it more difficult to observe and hide the fact that you’re
take notes. So it’s not very useful for observing, which makes it
studying interactions up close—such more difficult to observe and
as making detailed observations of a take notes. So it’s not very
person using a user interface. It’s useful for studying interactions
”
more suited for observing actions and
up close….
behaviors that you can see from a
distance.
Being covert eliminates your ability to ask questions or discuss what you’ve
observed with participants. So it can leave you with additional questions and
require you to make assumptions.
In the social sciences, some have criticized covert research as being ethically
unsound, deceptive, and an invasion of privacy. [2] Without the knowledge that
they are being observed, participants can’t provide their informed consent, and
they have no choice about whether to participate. However, despite these
objections, covert naturalistic observation has its advocates who argue that, in the
right situations, there are ways to do this research ethically. I’ll discuss those later
in this column.
When Should You Use Covert Naturalistic

Observation?
The best application of covert naturalistic observation is for studying the behavior
and interactions of groups of people in public places. So it’s ideally suited for
research projects for service design, workflow redesign, and process redesign.
Some examples of questions to which covert research could provide answers
include the following:
 How can we speed up the process and improve the passenger experience
TOP
of airport security lines?
“
 How can we make the emergency-
room waiting area a better The best application of covert
experience for patients and those naturalistic observation is for
who accompany them? studying the behavior and
 How can we improve customers’ interactions of groups of
shopping experience and increase people in public places. So it’s
sales in our clothing store? ideally suited for research
As I mentioned earlier, being projects for service design,
covert is difficult when you need to workflow redesign, and
”
see details close up—such as how process redesign.
people use a user interface. Overt
methods such as contextual inquiry
are better suited for those types of studies.
Tips on Covert Naturalistic Observation

Like most user research methods, covert naturalistic observation is more complex
than it may seem at first glance. Just as overt observation is more than just
watching people do things and asking questions about what they do, covert
observation is more than just people-watching. The following tips will help you to
get the most out of this method of research.
Gather Intelligence
Before you go out to observe, learn as much as you can about the domain that
you’re studying, the characteristics of user groups, and the behaviors and tasks
you’ll be observing. Review any previous research to which you have access. By
being informed, you’ll know more about what to look for, and you’ll be better able
to make sense of what you see.
Plan What You Want to Observe

TOP
“
If, for example, you wanted to study
what people do at the airport while Plan what you want to learn,
waiting for a flight, it would be a the questions you want to
mistake to think, I’ll just go to the answer, and how you’re going
airport and see what people do. to gather the information you
”
Without planning in advance what need.
your study will focus on, you would
quickly become overwhelmed by the
amount of information you’d take in. This makes it especially important for you to
plan what you want to learn, the questions you want to answer, and how you’re
going to gather the information you need. Knowing what to look for makes it easier
to focus your observation on what’s important and discard the rest.
Do Some Reconnaissance of the Location

As part of your planning, scout out the location ahead of time. Get answers to the
following questions to determine where and when to observe people and how to
appear natural and remain unnoticed.
 What’s the layout like?
 Where are the best places from which to observe?
 What’s the activity level like at certain times of the day?
 When is the best time to observe the behavior you want to see?
 What are people doing and what are they wearing?
If appropriate, notify the management or security of your research. For example,

if you’re observing people in an airport, the customers might not notice you, but
security might. It’s better to let them know what you’re doing in advance.
Observe as a Group
There are advantages to having two or more people observe as a group:
 Each person can observe behavior in a different location and focus on TOP
different people.
 Each person can focus on observing different behaviors or answering

different research questions.
 You’re less likely to miss something.
 You have someone to discuss your observations with.
If you work in a group, be sure to coordinate where each person will be, what
each person will do, and what each person should focus on.
Observe During Several Shorter Sessions

Instead of one long observation session, it’s usually better to break your research
into several shorter observation sessions. Doing shorter sessions has the following
advantages:
 You can see what happens over several sessions—perhaps at different
times of the day, observing different people, and observing different
situations.
 You can see whether there are repetitive behaviors, situations, and
patterns that recur over several sessions.
 It’s easier to remain mentally sharp, observant, and avoid fatigue.
 You’re less likely to be overwhelmed with too much information, so it’s
easier to remember what you observed.
 You’d appear more natural and would be less likely to get caught if you
didn’t hang around too long.
Blend In
To remain covert, you must blend in and can’t appear to be observing people.
That’s easier in some situations than others, but in most situations, the following
TOP
tips will help you to blend in:
“
 Dress the same way everyone else
does. Use a smartphone or tablet as
 Do what others are doing. cover—to appear like you’re
 Keep your level of eye contact doing something, not paying
”
normal. attention to your surroundings.
 Maintain a relaxed posture.
 Use a smartphone or tablet as
cover—to appear like you’re doing something, not paying attention to your
surroundings. You can pretend to talk on the phone or pretend to be
texting, while you’re actually observing and taking notes.
Observe Eﬀectively
Observation sounds deceptively simple. You just watch people, right? Actually, it
takes practice to observe effectively in an unstructured situation. Over time, you’ll
improve your observation skills and will be able to focus on what’s important.
Some things to look out for and note include the following:
 repeated behaviors and patterns
 unique behaviors
 body language
 the way people interact with other people, objects, and the environment
 the order of people’s actions
 similarities and differences between people
Don’t Be Obvious About Taking Notes

Remaining covert will prevent your taking notes as easily, obviously, or extensively
as you could in an overt observation such as a contextual inquiry. So don’t even try
to take detailed notes. Instead, rely more on your memory. Take very brief notes to
capture major themes or observations that will trigger your memory later on. When
the observation is over, you can type up more extensive notes from your memory, TOP
expanding on your brief notes.
“
As I mentioned earlier,
smartphones and tablets are great Remaining covert will prevent
cover for taking notes without your your taking notes as easily,
looking like you’re taking notes. To obviously, or extensively as
anyone who notices you, it looks like you could in an overt
you could be using an app, texting, or observation such as a
writing an email message. If you must
contextual inquiry. So don’t
take notes on paper, make it look like
even try to take detailed
you’re reading something and writing
notes. Instead, rely more on
”
notes about what you’re reading.
your memory.
Don’t Worry About
Recording
You’re primarily there to observe, not record the situation. So focus primarily on
observing, then if necessary, capture some photos or video at the end of the
session. The purpose of these photos or video is not to record the entire
observation. Their purpose is to capture important things, help you to remember
what you observed later on, and show those who didn’t accompany you on the
observations what you’ve observed.
To remain covert, you’ll need to take photos and record video very
surreptitiously. That’s another reason to wait until the end of your sessions to take
photos. If you get caught, you’ve already finished observing, so it doesn’t really
matter.
Smartphone cameras are ideal for this sort of research because they’re not as
obvious as traditional still or video cameras. If you turn off the flash and shutter
sound, people often won’t realize you’re taking pictures. Don’t record audio. It’s
not necessary, and recording audio without consent is illegal in most places.
Do a Debrief Afterward
TOP
“
Immediately after each observation
session, sit down and type up your Immediately after each
detailed notes, while the information observation session, sit down
is still fresh in your mind. First, capture and type up your detailed
whatever’s at the top of your mind— notes, while the information is
such as your overall observations, still fresh in your mind. First,
patterns, themes, conclusions, and capture whatever’s at the top
”
any further questions that you may
of your mind….
have. Then review the brief notes that
you took during the session to see
whether those spark any additional memories of things that you should note. You
may also want to draw a map or a layout of the physical space.
Once each individual observer types up his or her notes, get together with the
other observers to compare your notes and discuss what each of you observed.
Look for commonalities and differences. Have one person take notes on this
discussion. At the end of all of your observation sessions, combine the notes from
each observer, and do your analysis.
Do Covert Research Ethically

Covert research can be ethical if you’re careful not to violate any laws or ethical
boundaries. Conduct covert observation only in public places, where passersby
might reasonably observe the same situation. Consider whether there’s an
expectation of privacy in the situation or location. For example, it may be ethical to
observe customers shopping in a clothing store, but it’s certainly not okay to
observe them in a dressing room.
Focus on people and situations in general, not specific individuals. Unlike a spy
or police officers on a stakeout, you’re not targeting, following, or stalking a
specific person. You’re situated in a public place observing what people in general
do.
TOP
https://www.uxmatters.com/mt/archives/2013/11/becoming-a-spy-covert-naturalistic-observation.php#:~:text=Covert Naturalistic Observation,that they're being observed.&text=It is the opposite of,forms of overt nat… 10/16
Be careful to protect people’s privacy by not revealing their identities in photos

or videos or their names in reports. Block out their faces unless you’ve received
consent to show them.
Don’t deceive participants. You’ll try to blend in, so people won’t notice you, but
you’re not going under cover. If you get caught, you should just be honest about
what you’re doing. Many people won’t mind and might even find it interesting.
Combine Covert Research with Other Methods
“
Because covert naturalistic
observation doesn’t allow you to ask Covert naturalistic observation
questions or discuss what you … can bring up a lot of
observe with participants, it can bring questions and require you to
up a lot of questions and require you make assumptions. To answer
to make assumptions. To answer those questions and test your
those questions and test your assumptions, it’s useful to
assumptions, it’s useful to combine
combine covert research with
covert research with more traditional
more traditional overt
”
overt methods such as user
methods….
interviews or contextual inquiries. You
can also use traditional, overt
methods first, so you’ll have a better understanding of the subject matter when you
do your covert observation.
Comparing your covert and overt findings can be an interesting exercise. You
can determine whether there were any interesting differences between what
people did when they knew you were observing them versus when they didn’t
know you were observing them.
Should You Do Covert or Overt Research?
TOP
Covert or overt observation, which is better? It depends on the situation. Both

methods have advantages and disadvantages. Overt observation affects
participants’ behavior. Being covert eliminates this problem, but limits what you
can observe and prevents your asking questions. But as long as you’re aware of
these limitations and try to minimize them, you can choose overt and covert
methods as appropriate to your research and what would best enable you to
understand the participants’ experience.
References
[1] Shuttleworth, Martyn. “Hawthorne Effect (http://explorable.com/hawthorne-effect).”
(http://explorable.com/hawthorne-effect) Explorable.com, 2009. Retrieved October 10, 2013.
[2] Lugosi, Peter. “Between Overt and Covert Research: Concealment and Disclosure in an
Ethnographic Study of Commercial Hospitality
(https://www.academia.edu/319068/Between_Overt_and_Covert_Research_Concealment_and_Disclosure_In_An_Ethnographic_Study_o
(https://www.academia.edu/319068/Between_Overt_and_Covert_Research_Concealment_and_Disclosure_In_An_Ethnographic_Study_o
Qualitative Inquiry, 2006. Retrieved October 10, 2013.
In User Research (https://www.uxmatters.com/topics/research/user-research/)
Show Comments  Share…
Jim Ross
Senior UX Researcher at AnswerLab

Philadelphia, Pennsylvania, USA
TOP
(https://www.uxmatters.com/authors/archives/2009/10/jim_ross.php)Jim has spent

most of the 21st Century researching and designing intuitive and satisfying user
experiences. As a UX consultant, he has worked on Web sites, mobile apps,
intranets, Web applications, software, and business applications for financial,
pharmaceutical, medical, entertainment, retail, technology, and government clients.
He has a Masters of Science degree in Human-Computer Interaction from DePaul
University. Read More
Other Columns by Jim Ross
 Retaining Your Sanity as a User Researcher

(https://www.uxmatters.com/mt/archives/2020/03/retaining-
your-sanity-as-a-user-researcher.php)
 The Perils of Client Recruiting
(https://www.uxmatters.com/mt/archives/2019/12/the-perils-
of-client-recruiting.php)
 Fears About User Research
(https://www.uxmatters.com/mt/archives/2019/10/fears-about-
user-research.php)
 The Worst Ideas I’ve Heard for User Research
(https://www.uxmatters.com/mt/archives/2019/08/the-worst-
ideas-ive-heard-for-user-research.php)
TOP
Other Articles by Jim Ross
 Review of Information Architecture Evaluation Tools:

Chalkmark and Treejack
(https://www.uxmatters.com/mt/archives/2010/02/review-of-
information-architecture-evaluation-tools-chalkmark-and-
treejack.php)
 Eyetracking: Is It Worth It?
(https://www.uxmatters.com/mt/archives/2009/10/eyetracking-
is-it-worth-it.php)
Other Articles on User Research
 10 Best UXmatters Articles on Foundational User Research

(https://www.uxmatters.com/mt/archives/2020/06/10-best-
uxmatters-articles-on-foundational-user-research.php)
 Applying In-Depth Research and Analysis to Achieve Better
Outcomes
(https://www.uxmatters.com/mt/archives/2020/05/applying-
in-depth-research-and-analysis-to-achieve-better-
outcomes.php)
 Dealing with Clients Who Focus on Only One Aspect of a UX
Design
(https://www.uxmatters.com/mt/archives/2020/04/dealing-
with-clients-who-focus-on-only-one-aspect-of-a-ux-
design.php)
 Forgetting to Breathe: Mindful User Research
(https://www.uxmatters.com/mt/archives/2020/04/forgetting- TOP
to-breathe-mindful-user-research.php)
New on UXmatters
 Overcoming Designer’s Block

(https://www.uxmatters.com/mt/archives/2020/06/overcoming-
designers-block.php)
 Foundational Practices for Great Teamwork
(https://www.uxmatters.com/mt/archives/2020/06/foundational-
practices-for-great-teamwork.php)
 The Ooarding User Experience: A Comparative Study
(https://www.uxmatters.com/mt/archives/2020/06/the-
ooarding-user-experience-a-comparative-study.php)
 10 Best UXmatters Articles on Foundational User Research
(https://www.uxmatters.com/mt/archives/2020/06/10-best-
uxmatters-articles-on-foundational-user-research.php)
 Review: Figma Versus Adobe XD, Part 2
(https://www.uxmatters.com/mt/archives/2020/06/review-
ﬁgma-versus-adobe-xd-part-2.php)
Subscribe to Our Newsletter Followers

 86.4K (https://twitter.com/uxmatters)
About UXmatters (/about-
We’ll let you know when new articles appear on
us/)
Subscribing (/subscribing-
UXmatters. Subscribers
 32.4K (https://feedpress.me/UXmatters)
Your full name to-uxmatters.php)
Writing Articles (/about-
us/writing-for-
Sponsoring (/about-
Email address
uxmatters.php)
us/sponsoring-
TOP
Subscribe Volunteering (/about-

uxmatters.php)
us/volunteering-with-
Contact Us (/contact-us/)
uxmatters.php)
Founder, Publisher, & Editor in Chief: Pabini Gabriel-Petit (/authors/archives/2005/11/pabini_gabriel-petit.php) Site designed by
Copyright © 2005–2020 UXmatters & Our Authors. All rights reserved.
TOP

Becoming A Spy - Covert Naturalistic Observation - UXmatters

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Becoming A Spy - Covert Naturalistic Observation - UXmatters

Uploaded by

Copyright:

Available Formats

6/21/2020 Becoming a Spy: Covert Naturalistic Observation :: UXmatters

Becoming a Spy: Covert Naturalistic

As I mentioned in a previous column, user research is unnatural

Covert Naturalistic Observation

What Are Its Advantages?

What Are Its Disadvantages?

When Should You Use Covert Naturalistic

Tips on Covert Naturalistic Observation

Plan What You Want to Observe

Do Some Reconnaissance of the Location

If appropriate, notify the management or security of your research. For example,

 Each person can focus on observing different behaviors or answering

Observe During Several Shorter Sessions

Don’t Be Obvious About Taking Notes

Do Covert Research Ethically

Be careful to protect people’s privacy by not revealing their identities in photos

Combine Covert Research with Other Methods

Should You Do Covert or Overt Research?

Covert or overt observation, which is better? It depends on the situation. Both

In User Research (https://www.uxmatters.com/topics/research/user-research/)

Show Comments  Share…

Senior UX Researcher at AnswerLab

(https://www.uxmatters.com/authors/archives/2009/10/jim_ross.php)Jim has spent

Other Columns by Jim Ross

 Retaining Your Sanity as a User Researcher

Other Articles by Jim Ross

 Review of Information Architecture Evaluation Tools:

Other Articles on User Research

 10 Best UXmatters Articles on Foundational User Research

 Overcoming Designer’s Block

Subscribe to Our Newsletter Followers

Subscribe Volunteering (/about-

Copyright © 2005–2020 UXmatters & Our Authors. All rights reserved.

You might also like