Observational Research: Protocols and Logs 1

Observational Research: Protocols and Logs

Observation has led to some of the crucial discoveries in the natural, social, medical, and

applied sciences. Observational studies are also called ‘field research’ as they generally don’t

take place in a laboratory. An observation log is the instrument used to collect data in an

observational study.

Observation based research is generally inexpensive and can provide a more nuanced

look into phenomenon than some other kinds of data collection methods (such as surveys).

However, there is significant opportunity for bias to influence the collection of data in an

observational study as well as ethical concerns over privacy for human subjects.

Observation logs are versatile instruments used in many fields for many purposes. The

natural world, the human-made world, and humans themselves can be the subject of observation

in research. Observation logs can be used to gather qualitative (descriptive) or quantitative

(numerical) data. The types of information that the researcher is hoping to gather will determine

what the observation log will look like. This brief guide will focus on observationally based

research of human behavior.

Observational Research Involving Human Subjects

When it comes to observing human behavior, research can be divided into two very broad

categories – participant observation and non-participant observation. Participant observation is

often used in anthropology, ethnography, and linguistics; this method requires the researcher to

integrate with the population of people under study and can combine observations with

interviews and surveys. Non-participant observation is often used in sociology and psychology;
 Observational Research: Protocols and Logs 2

this method requires that the researcher not integrate with the population under study. Non-

participant observational studies often aim for “naturalistic observation” or observation that goes

unnoticed by the subjects under study.

Ethical Concerns:

Anonymity and confidentiality: When working with human subjects, researchers must do

all in their power to protect the privacy of their subjects. This is conceived of in two ways:

confidentiality and anonymity. When a subject’s identity is Note: Research institutions, like
universities, have Institutional Review
treated as confidential, the researcher has identifying information Boards (IRB’s) that review proposals
for research involving human and
about the research subject but does not share that information. animal subjects. These boards are
responsible for assisting researchers in
When a subject’s identity is anonymous, the researcher does not the engagement of ethical research;
however, IRB’s don’t replace the
have identifying information about the research subjects (which researcher’s careful consideration and
approach to ethical issue.
would, of course, prevent them from sharing it).

Vulnerable populations: Certain groups of people, such as children, refugees, and

prisoners, are considered vulnerable populations and research involving them is subject to

additional scrutiny. Special care must be taken in research involving vulnerable populations to

ensure their mental, emotional, and physical safety.

Bias: Bias is the manifestation of preference and prejudice; in observational research, this

can cause researchers to overlook, over-emphasize, ignore, or highlight phenomenon in a way

that is not congruent with reality. Bias is both an ethical concern as well as a concern of the

validity and reliability of research. To guard against bias – researchers typically use an

observation protocol or schedule (see below), oversight of their work by colleagues, and

randomization of samples (to the degree possible).

 Observational Research: Protocols and Logs 3

Before Observation

Before undertaking an observation study, the researcher must create a protocol

(sometimes called a schedule) to serve as a plan for the researcher’s behavior during observation.

This protocol must be set in place before the observation takes place to ensure continuity in the

actions of the researcher. The protocol should be appropriate to the research questions under

study. The following list outlines the major parameters that must be established by the researcher

before engaging in observational research:

1. Who will be observing and what level of participation will the observer engage(s) in?

o Do the research questions call for participant or non-participant observations?

2. Where the observations will take place?

o Often, the location of the observation will in large part determine the sample that

will be studied;

o Not only must the place of observation be fitting for the research, it must also be

legally, physically, and culturally accessible to the researcher.

o Safety concerns should be thought through when the location of research is being

decided.

3. What are the times, dates, and frequency for the observations?

o The time, date, and frequency of observation also contributes to determining the

sample for the research study;

4. What will be observed and recorded?

o In outlining what will be observed, what will not be recorded is implied; however,

researchers often explicitly articulate what will not be recorded during the study.
 Observational Research: Protocols and Logs 4

o When describing what will (and will not) be recorded by the researcher – the

greater the level of detail the better.

5. How will observations be recorded?

o Researchers have many options for recording their observations. The most

common ways are note-taking and video/audio recording; however, some fields

use specialized equipment for gathering kinds of observations they are interested

in.

o Different recording methodologies require different considerations; audio/video

recording requires a greater level of diligence on the part of the researcher to

maintain anonymity and confidentiality for research participants. However, note-

taking requires careful adherence to the protocol to ensure that data is not

misreported due to time constraints.

6. What equipment will be needed to complete the observations?

Making a plan for your results

After the observation schedule has been established, the researcher must record what

they intend to do with the observations that they will collect. This might be as simple as

comparing the amount of dog-walkers in the morning to the afternoon, or as complicated as

correlating a tiny phonemic sound with the gender and profession of millennials in Charlotte.

Many researchers already have an idea in their heads of how to treat their data when they begin

to design a research project. However, there are practical, conventional, and scientific reasons

why carefully writing out a plan for your results is of crucial importance:

1. The process of writing out a clear plan for your results demands that the process is
thought out carefully and will improve the rigor of your research;
 Observational Research: Protocols and Logs 5

2. Having a written plan for your results allows you to share that plan with others who can
provide feedback and guidance.

3. The scientific methods demands it. Without a plan for the treatment of your data, your
research plan sounds something like: “I’m gonna throw a net out there and if I see
something in the net that looks pretty good, I’ll keep it.” Instead, the scientific method
requires that we make a prediction about what we will catch (the results) and what we
think we can learn from those results (the treatment of the results). Even still, having a
plan for your results doesn’t require that you ignore additional points of interest that your
research may illuminate for you – as long as you also follow through on your plan.

4. Various “discourse communities” ask it of us. The scientific (academic/business etc.)

community has conventions or accepted practices that it expects of participants.

Discuss with your instructor and your peers on how best to go about working with the data your

research will collect. Keep in mind the guidance that the concept of interrater reliability has to

offer your research design (see box below for a brief introduction to interrater reliability).

Computer software has changed the speed, efficiency, and potentials of data analysis but can be

overwhelming to a new researcher. Make sure that you take advantages of the resources that your

college/university offers in support of your research. Professors and instructors can also help you

navigate software that you may be unfamiliar with.

 Observational Research: Protocols and Logs 6

What is Interrater reliability? Formally, the term interrater reliability refers to the
consistency with which different raters evaluate or score something. This term is used to
indicate the strength of the instruments.
This idea of measuring reliability by having a kind of oversight on the research through
multiple raters/researchers can be applied in many areas of the research process. In
observational research, this may mean having multiple people collecting observations. Often,
more than one person will compile results separately from each other and then the resulting
data will be compared. If there is a discrepancy, the observation logs are used to ensure an
accurate reporting of the results. On larger projects, a second person may check randomly
selected pieces of the results transcribed with the original observation logs.

THis idea

Design your observation log

Once you have established the kinds of information that you are gathering in your

observations and how you intend to treat the results, you should design an observation log. The

observation log assists researchers by prompting them on what to observe and report, organizing

their observations, and providing a space to record those observations. The observation log acts

as a guide to a researcher in the field, a map to help them find their way through a wealth of

sensory information. Observation logs (as well as interview questions/survey questions etc.) are

referred to as instruments.

Does the word instrument throw you off? Make you think of a saxophone?
One way to remember it, is to think about scientific instruments like surveys and
observation logs, more like a doctor’s instruments, like a scalpel and a stethoscope and less
like drum set or a clarinet.

Combatting bias in observationally based research should be approached in the

instrument design. Observation can also easily become conjecture, and for researchers new and
 Observational Research: Protocols and Logs 7

experienced alike, separating out the phenomenon being observed and the researcher’s reactions

to and interpretations of that phenomenon is crucial. One helpful strategy in achieving this is a

divided observation log. A divided observation log calls for designated areas to record

observations and separate areas for recording reactions/interpretations. Observation logs should

include ample areas for both kinds of recording to take place.

The observation log should include designated areas for the different pieces of information

established by your observation schedule. The list of what should be included in the protocol is

on pgs 3-4; specifically, points 1-4 should determine the main design elements of the observation

log:

1. Who will be observing and what level of participation will the observer(s) engage in?

2. Where the observations will take place?

3. What are the times, dates, and frequency for the observations?

4. What will be observed and recorded?

Observation logs are sometimes used within experimental research designs. For example, if a
researcher is testing a new drug on rodents, they will keep a detailed record of the rodents’
behavior that may be referred to as an observation log. This is typically referred to as analogue
observation as the observation is taking place in an artificial setting.

During Observation

After careful planning of the observational research, the researcher must then execute

their protocol and collect their data. Adhering to the observation protocol is incredibly important.

This ensures that data collection is a uniform process, regardless of which researcher is
 Observational Research: Protocols and Logs 8

collecting the data at any particular moment. Uniformity in that process will increase the

likelihood that the observational data will be robust.

In a laboratory setting, a much greater percentage of variables are able to be controlled

for than in field research. As such, a disciplined approach to observation and recording is of

crucial importance. Even a careful, well-thought through and detailed observation protocol won’t

cover every possible scenario that a researcher will find themselves in. Researchers may still

have to make choices about how they will handle an unforeseen event. The last line of defense in

maintaining a uniform data collection method is leveraging the guiding principles and aims

under which the observation protocol was built. For example, let’s take an observation session

that is intended to be non-participatory in order to observe behavior as naturally as possible. How

should the researcher react if a research subject notices the researcher and approaches them?

Under the guiding aim of observing natural behavior, the researcher must make a real-time

judgment of how to interact with that individual. They must also decide if a significant enough

breach of the observation protocol has occurred and whether or not the data from that session is

still usable.

The following example illustrates the application of these foundational concepts to a local problem.
Three students believe that one of the crosswalks on their college campus is dangerous and are
researching the safety of that crosswalk. After doing secondary research on pedestrian safety and
primary research on pedestrian behavior on campus generally, the students are ready to observe the
crosswalk in question. Below you will find the first draft of their observation protocol, notes
received from their peers and instructor, the final draft of their protocol, and their observation log.
 Observational Research: Protocols and Logs 9

Observation Protocol
Researchers:
Ayoti Korir, Kenwal Sidana, and Sarah Jackson.

Location:
The crosswalk in front of 280 Hicks Way, Amherst, MA 01002.

Timing:
The crosswalk will be observed in the afternoons during September
2017.

Participation:
The researchers will not be participating in the pedestrian
activity of crossing the crosswalk and the researchers will not
talk to any pedestrians.

Recording Method:
Observations will be recorded by hand and researchers will use a
handheld speedometer to record the speed of cars passing by.

Observations:
This study will record – how many people are using the crosswalk
and how fast cars are driven down this street
 Observational Research: Protocols and Logs 10

Observation Protocol
Researchers:
Ayoti Korir, Kenwal Sidana, and Sarah Jackson.

Are these the principle researchers or the people doing the observations?
You want to include both in the observation protocol

Location:
The crosswalk in front of 280 Hicks Way, Amherst, MA 01002.

This is pretty specific – that’s great! How will you guys know that you are
in the right position to make the best observations?

Timing:
The crosswalk will be observed in the afternoons during September
2017.
Which afternoons in September and at what time? How many
observation sessions are you aiming to complete. Also, why did you
choose afternoons? Would it perhaps be better to vary the times of day
that you will be observing?

Participation:
The researchers will not be participating in the pedestrian
activity of crossing the crosswalk and the researchers will not
talk to any pedestrians.

What is the plan if a pedestrian wants to talk to you? If your goal is

“unobtrusive observation” how will you go about ensuring that you are
unobtrusive?

Recording Method:
Observations will be recorded by hand and researchers will use a
handheld speedometer to record the speed of cars passing by.

Which cars? How will you decide which cars will have their speed
recorded? How will you record the speed of passing cars?

Observations:
This study will record – how many people are using the crosswalk
and how fast cars are driven down this street

In outlining what will be observed, what will not be recorded is implied; however, researchers
often explicitly articulate what will not be recorded during the study. When describing what will
(and will not) be recorded by the researcher – the greater the level of detail the better.
 Observational Research: Protocols and Logs 11

Observation Protocol
Researchers:
Ayoti Korir, Kenwal Sidana, and Sarah Jackson.
Observers:
Ayoti Korir and Sarah Jackson
Location:
The crosswalk in front of 280 Hicks Way, Amherst, MA 01002.
Observers will sit and record their observations from the bench
on the south side of the street. This should help the observer
remain inconspicuous and look natural. The bench on the north
side of the street doesn’t have a good enough view of the street
to be used for observations.
Timing:
The crosswalk will be observed for 3 hours every afternoon during
the first two weeks of September 2017. Observations will talk
place between 3-6pm. This time was chosen because it is the
busiest time of day with the most vehicles and pedestrians.
Participation:
The researchers will not be participating in the pedestrian
activity of crossing the crosswalk and the researchers will not
talk to any pedestrians. If a pedestrian approaches a researcher,
the researcher will politely inform the pedestrian that they are
“working” and disengage from the conversation.
Recording Method:
Pedestrian crossings will be recorded by hand and researchers
will use a handheld speedometer to observe the speed of vehicles
passing by. The speed of every 3rd vehicle that approaches the
bench will be recorded on the observation log.
Observations:
This study will record: how many people are using the crosswalk
during a particular time and how fast cars are driven down this
street. The number of people (adults and children) and dogs that
cross the crosswalk will be recorded. The weather conditions will
be also be recorded.
 Observational Research: Protocols and Logs 12

Observation Log #___

Name of observer:___________________________
Date of observation:____/____/_________
Location of observation: 280 Hicks Way, Amherst, MA 01002

Weather:

Car speeds(& num.)in mph: Observer reactions thoughts on

cars:

Number of Pedestrians (tally) Observer thoughts on

pedestrians

Total:
# of Dogs # of Children
(13 and under)
 Observational Research: Protocols and Logs 13

Observation Log #3 .

Name of observer: Kenwal Sidana

Date of observation:09/25/2017

Location of observation: 280 Hicks Way, Amherst, MA 01002

Weather: Sunny, 68◦F (google), very high winds – 28mph (google)
Car speeds(& num.) in mph: Observer reactions thoughts on
cars:
25 III 47 I
Cars go faster when there are 2—5
27 IIII II 23 III people around and slower when
24 IIII I 39 I no one is around!

34 III Some cars slow down for

empty crosswalk, others speed up!
32 I
Maybe because they want to like
33 III “beat the traffic (pedestrians).

29 IIII IIII IIII IIII

Number of Pedestrians (tally) Observer thoughts on

28 III pedestrians
IIII IIII IIII IIII IIII IIII IIII IIII
When people are already in crosswalk –
IIII IIII IIII IIII IIII IIII IIII IIII new people cross without looking

IIII IIII IIII IIII IIII IIII IIII IIII IIII IIII People with dogs look around for
cars earlier than people walking alone
IIII IIII IIII IIII IIII IIII IIII IIII IIII
IIII IIII IIII IIII IIII II People w/kids stop before crossing

Total: 202 Some people “look both ways” after they

already stopped into the crosswalk.
# of Dogs # of Children
(13 and under)
IIII IIII IIII total 14 III total 3