Lecture

Research Methods 09
Observational Research
(Exploratory studies)

Dr. Syed Muhammad Sajjad

Department of Cyber-Security and Data Science
Riphah Institute of Systems Engineering (RISE),
Riphah International University, Islamabad, Pakistan.
 Observational Research 
• Observational research, of which exploratory study 
is a subset, is useful 
• when you are trying to understand a real cyber 
system (techno‐social behavior) 
• This type of research is suited to answer open‐ended 
or comparatively broad research questions
• In general, observational research methods include 
sensing of real world environments and data mining 
for discovery of interesting artifacts.
 Observational Research 
• Sagarin et al. provide an insightful discussion on the limits 
and strengths of observational research in their book
1. Observation‐based studies find patterns, but patterns 
can’t be used to infer process
2. Observation‐based studies rely on the flawed approach 
of induction, rather than the more precise deductive 
approach, to reach conclusions
3. They are just a collection of un‐replicated anecdotes
4. They rely too much on correlations between variables
 Exploratory Study
• Exploratory studies are either datasets that are 
not in the immediate control of the investigator,
• Often the scope of the study and the dataset is 
much larger
• Descriptive Studies, are focused on preconceived 
behavior and a goal of describing the phenomena
• Descriptive studies typically have more interaction 
and control of the subject at hand
 Exploratory Study
• Exploratory studies consist of collecting, analyzing, 
and interpreting observations about known designs, 
systems, or models, or about abstract theories or 
subjects
• Studies are largely an inductive process to gain 
understanding
• Where the experimental process goes from a general 
theory to an understanding in specific, 
• Studies look at specific phenomena to search for 
patterns and arrive at a general theory of behavior 
 Exploratory Study
• The emphasis is on evaluation or analysis of data, 
not on creating new designs or models
• This is often called a qualitative study in the 
social and health sciences
• When you have no expectations or beliefs about 
how a system operates 
• But you want to study and understand it, 
exploratory study is a helpful method 
 Exploratory Study
• By observing the world around us we can start to 
discover patterns and sequences of events 
• That can lead to understanding the underlying 
principles and laws of behavior
• Exploratory studies enable researchers to start 
developing models of system operation
 Exploratory Study
• As some aspects of cyber security is practically 
uncontrollable, such as threats,
• When exploring topics and research questions 
around uncontrollable variables
• Exploratory studies are a useful approach to 
discover information about a system
 Types of Exploratory Study
1. Case Control Study
2. Ecological Study
3. Cross‐Sectional Study
4. Longitudinal/Cohort Study
 Types of Exploratory Study
1. Case Control Study
• Type of observational study where a set of
measurements can be divided into two groups
• The first group is called the case group exhibits the
signs or symptoms of the event or phenomena under
study
• The second group, the control group, does not
exhibit these systems
• For example, a researcher may want to look at the
rare cancer. The researcher would find a group of
individuals with cancer(the cases) and compare them
to a group of patients who are similar to the cases in
most ways but do not have cancer(controls)
 Types of Exploratory Study
2. Ecological Study
• Explore the entire population in either
geographic or temporal space (in a region or at a
particular time)
• The goal of an ecological study is to assess the
risk factors confronting a cyber population by
evaluating risk mitigations and the subsequent
adverse outcomes that affect the population
 Types of Exploratory Study
3. Cross‐sectional Study
• Sometimes called a census
• Cross‐sectional studies are related to, and sometimes
a branch of, ecological studies in that a time‐slice
sample is collected from a population
• A sample is collected from the entire population at a
specific period of time
• Cross‐sectional studies are often used to describe the
prevalence of certain
• Characteristics of the overall population or enable
deductive inferences to be made about the
information at hand
 Types of Exploratory Study
4. Longitudinal/Cohort Study
• Is a sequential observation of a cyber system or
cyber behavior over time
• The goal is to collect sufficient information over
time to ensure the entire lifecycle or complete
context of the system under evaluation is
considered
• The evolution of viruses over time
• A cohort study is a type of longitudinal study that
focuses on a specific group, over time, called a
cohort
 GATHERING DATA
• At the point of data collection, a researcher should
have a sufficient handle on the research questions
and approach they will be undertaking
• If you have insufficient clarity to start with, then you
run the risk of gathering data that does not end up
being relevant to your line of inquiry
• From questions about collection methodologies, to
formats and syntax, the task of collecting sufficient
data, or even knowing when to stop, can dwarf the
labor involved in the subsequent analysis.
 GATHERING DATA
To avoid this trap, as much as possible, a well‐crafted
research plan will alleviate several of these pitfalls, start
by exploring a few questions to help identify the scope,
objectives, and likely challenges in data gathering
• Have you reviewed your idea with your REC
(Research Ethics Committee) informally to seek
guidance or feedback?
• What types of questions are you asking (qualitative
vs quantitative, relationship, volume or routine
occurrences)? Clearly the questions must be
observable, but the dataset may not exist.
• How much data will you need? Do you know what
amount will be statistically significant for the
conclusions or answers you hope to address?
 GATHERING DATA
• Who owns the dataset? Privacy, access, control,
dissemination, and other issues will need to be
addressed beforehand
• Can you easily recollect or recreate the dataset? Or
are you stuck with the dataset from a single point in
time? Are you able to come back and revisit your
initial assumptions if they do not pan out?
• Are you sampling? Will you be able to collect the
entire population or dataset? This is very common in
network traffic. If you have to sample, what is the
method? What artifacts or challenges might it
introduce?
• Have you reviewed the full study plan with your REC
(Research Ethics Committee) ?
 GATHERING DATA
• Broadly there are two types of data: qualitative and
quantitative.
• Qualitative research includes collection and analysis of
descriptive data.
• Research involving humans often includes information
about their emotional state and social characteristics.
• It can be categorized and sometimes ordered but does not
provide the ability to mathematically quantify the data.
• Quantitative research involves the collection and analysis of
numerical data.
• It enables the quantification or statistical exploration and
explanation of data.
• Quantitative provides the most flexibility in analysis and
should be sought above qualitative when possible.