ROSELYN ESPINOSA

CHAPTER 3 – METHODOLOGY

To be a researcher we must think logically or systematically. Research activities must follow a certain order,
like doing inductive thinking that makes you ponder on specific ideas first, then move to more complex
concepts like conclusions or generalizations. Or, do the opposite of inductive thinking which is deductive
thinking that lets you start from forming generalizations to examining detail about the subject matter. Inductive
and Deductive thinking are not the only approaches that you can adhere to in planning your research work.
Depending on your topic and purpose, you free to choose from several approaches, methods, and types of
research.

Let’s defined what Methodology is

 According to Polit and Beck (2004) methodology refers to ways of obtaining, systematizing and analyzing
data.
 Creswell (2003) portrays methodology as a coherent group of methods that harmonize one another and that
have the capability to fit to deliver data and findings that will reflect the research question and suits the
researcher’s purpose.
 Bowling (2002) explains that methodology is the complete structure of the research study; the size and
sample methods, the practices and techniques utilized to collect data and the process to analyze data.

Qualitative Research
 Qualitative research refers to inductive, holistic, emic, subjective and process-oriented methods used to
comprehend, interpret and describe a phenomenon or setting. It is a systemic, subjective approach used to
describe meaning (Burns & Grove, 2003). Qualitative research is more associated with words, language
and experiences rather than measurements, statistics and numerical figures.

 Researcher’s using qualitative research take a person centred and holistic perspective to understand the
phenomenon, without focusing on specific concepts. The original context of the experience is unique, and
rich knowledge and insight can be generated in depth to present a lively picture of participants’ reality and
social context (Holloway, 2005).

 Regarding generation of knowledge, qualitative research is characterised as developmental and dynamic,

and does not use formal structured instruments. In turn it involves the systemic collection and analysis of
subjective narrative data in an organised and intuitive fashion to identify the characteristics and significance
of human experience (Holloway, 2005).

 Qualitative researchers are concerned with the emic perspective to explore the ideas and perceptions of the
participants. The researcher tries to examine the experience from the participant’s point of view in order to
interpret their words. The researcher therefore becomes involved and immersed in the phenomena to
become familiar with it. The immersion of the researcher helps to provide dense description from the
narrative data gathered from the participants, to interpret and portray their experiences, and to generate a
more comprehensive understanding. However, immersion cannot be obtained without a researcher-
participant trusting relationship. The relationship is built through basic interviewing and interpersonal
skills. In qualitative research, the researcher is required to be a good listener, non-judgemental, friendly,
honest and flexible. Brink and Wood (1998) point out that the qualitative data collection methods are
flexible and unstructured, capturing verbatim reports or observable characteristics and yield data that
usually do not take numerical forms.

 Quantitative Research focuses on numbers, statistics and relationship between variables. (Punch 2014;
Edmonds 2013; Lapan 2012)

1
Research Design
Burns and Grove (2002) define a research design as a blueprint for conducting a study with maximum control
over factors that may interfere with the validity of the findings. The overall purpose of this study is to contribute
further to the role-definition process.

General Structure and Writing Style

The function of a research design is to ensure that the evidence obtained enables you to effectively address the
research problem as unambiguously as possible. In social sciences research, obtaining evidence relevant to the
research problem generally entails specifying the type of evidence needed to test a theory, to evaluate a
program, or to accurately describe a phenomenon. However, researchers can often begin their investigations far
too early, before they have thought critically about what information is required to answer the study's research
questions. Without attending to these design issues beforehand, the conclusions drawn risk being weak and
unconvincing and, consequently, will fail to adequate address the overall research problem.

Given this, the length and complexity of research designs can vary considerably, but any sound design will do
the following things:

1. Identify the research problem clearly and justify its selection,

2. Review previously published literature associated with the problem area,
3. Clearly and explicitly specify hypotheses [i.e., research questions] central to the problem selected,
4. Effectively describe the data which will be necessary for an adequate test of the hypotheses and explain
how such data will be obtained, and
5. Describe the methods of analysis which will be applied to the data in determining whether or not the
hypotheses are true or false.

2
TYPES OF RESEARCH DESIGN

Action Research Design

Definition and Purpose
The essentials of action research design follow a characteristic cycle whereby initially an exploratory stance is
adopted, where an understanding of a problem is developed and plans are made for some form of
interventionary strategy. Then the intervention is carried out (the action in Action Research) during which time,
pertinent observations are collected in various forms. The new interventional strategies are carried out, and the
cyclic process repeats, continuing until a sufficient understanding of (or implement able solution for) the
problem is achieved. The protocol is iterative or cyclical in nature and is intended to foster deeper
understanding of a given situation, starting with conceptualizing and particularizing the problem and moving
through several interventions and evaluations.

What do these studies tell you?

1. A collaborative and adaptive research design that lends itself to use in work or community situations.
2. Design focuses on pragmatic and solution-driven research rather than testing theories.
3. When practitioners use action research it has the potential to increase the amount they learn consciously
from their experience. The action research cycle can also be regarded as a learning cycle.
4. Action search studies often have direct and obvious relevance to practice.
5. There are no hidden controls or preemption of direction by the researcher.

What these studies don't tell you?

1. It is harder to do than conducting conventional studies because the researcher takes on responsibilities
for encouraging change as well as for research.
2. Action research is much harder to write up because you probably can’t use a standard format to report
your findings effectively.
3. Personal over-involvement of the researcher may bias research results.
4. The cyclic nature of action research to achieve its twin outcomes of action (e.g. change) and research
(e.g. understanding) is time-consuming and complex to conduct.

Case Study Design

Definition and Purpose

A case study is an in-depth study of a particular research problem rather than a sweeping statistical survey. It is
often used to narrow down a very broad field of research into one or a few easily researchable examples. The
case study research design is also useful for testing whether a specific theory and model actually applies to
phenomena in the real world. It is a useful design when not much is known about a phenomenon.

What do these studies tell you?

1. Approach excels at bringing us to an understanding of a complex issue through detailed contextual

analysis of a limited number of events or conditions and their relationships.
2. A researcher using a case study design can apply a variety of methodologies and rely on a variety of
sources to investigate a research problem.
3. Design can extend experience or add strength to what is already known through previous research.
4. Social scientists, in particular, make wide use of this research design to examine contemporary real-life
situations and provide the basis for the application of concepts and theories and extension of methods.
5. The design can provide detailed descriptions of specific and rare cases.

What these studies don't tell you?

1. A single or small number of cases offers little basis for establishing reliability or to generalize the
findings to a wider population of people, places, or things.
2. The intense exposure to study of the case may bias a researcher's interpretation of the findings.
3. Design does not facilitate assessment of cause and effect relationships.
4. Vital information may be missing, making the case hard to interpret.
5. The case may not be representative or typical of the larger problem being investigated.

3
 6. If the criteria for selecting a case are because it represents a very unusual or unique phenomenon or
problem for study, then your interpretation of the findings can only apply to that particular case.

Causal Design
Definition and Purpose

Causality studies may be thought of as understanding a phenomenon in terms of conditional statements in the
form, “If X, and then Y.” This type of research is used to measure what impact a specific change will have on
existing norms and assumptions. Most social scientists seek causal explanations that reflect tests of hypotheses.
Causal effect (nomothetic perspective) occurs when variation in one phenomenon, an independent variable,
leads to or results, on average, in variation in another phenomenon, the dependent variable.

Conditions necessary for determining causality:

 Empirical association--a valid conclusion is based on finding an association between the independent
variable and the dependent variable.
 Appropriate time order--to conclude that causation was involved, one must see that cases were exposed
to variation in the independent variable before variation in the dependent variable.
 Nonspuriousness--a relationship between two variables that is not due to variation in a third variable.

What do these studies tell you?

1. Causality research designs helps researchers understand why the world works the way it does through
the process of proving a causal link between variables and eliminating other possibilities.
2. Replication is possible.
3. There is greater confidence the study has internal validity due to the systematic subject selection and
equity of groups being compared.

What these studies don't tell you?

1. Not all relationships are casual! The possibility always exists that, by sheer coincidence, two unrelated
events appear to be related
2. Conclusions about causal relationships are difficult to determine due to a variety of extraneous and
confounding variables that exist in a social environment. This means causality can only be inferred,
never proven.
3. If two variables are correlated, the cause must come before the effect. However, even though two
variables might be causally related, it can sometimes be difficult to determine which variable comes first
and therefore to establish which variable is the actual cause and which is the actual effect.

Cohort Design
Definition and Purpose

Often used in the medical sciences, but also found in the applied social sciences, a cohort study generally refers
to a study conducted over a period of time involving members of a population which the subject or
representative member comes from, and who are united by some commonality or similarity. Using a
quantitative framework, a cohort study makes note of statistical occurrence within a specialized subgroup,
united by same or similar characteristics that are relevant to the research problem being investigated, rather than
studying statistical occurrence within the general population. Using a qualitative framework, cohort studies
generally gather data using methods of observation. Cohorts can be either "open" or "closed."

 Open Cohort Studies [dynamic populations, such as the population of Los Angeles] involve a population
that is defined just by the state of being a part of the study in question (and being monitored for the
outcome). Date of entry and exit from the study is individually defined, therefore, the size of the study
population is not constant. In open cohort studies, researchers can only calculate rate based data, such as,
incidence rates and variants thereof.
 Closed Cohort Studies [static populations, such as patients entered into a clinical trial] involve
participants who enter into the study at one defining point in time and where it is presumed that no new
participants can enter the cohort. Given this, the number of study participants remains constant (or can
only decrease).

4
What do these studies tell you?

1. The use of cohorts is often mandatory because a randomized control study may be unethical. For
example, you cannot deliberately expose people to asbestos; you can only study its effects on those who
have already been exposed. Research that measures risk factors often relies on cohort designs.
2. Because cohort studies measure potential causes before the outcome has occurred, they can demonstrate
that these “causes” preceded the outcome, thereby avoiding the debate as to which is the cause and
which is the effect.
3. Cohort analysis is highly flexible and can provide insight into effects over time and related to a variety
of different types of changes [e.g., social, cultural, political, and economic, etc.].
4. Either original data or secondary data can be used in this design.

What these studies don't tell you?

1. In cases where a comparative analysis of two cohorts is made [e.g., studying the effects of one group
exposed to asbestos and one that has not], a researcher cannot control for all other factors that might
differ between the two groups. These factors are known as confounding variables.
2. Cohort studies can end up taking a long time to complete if the researcher must wait for the conditions
of interest to develop within the group. This also increases the chance that key variables change during
the course of the study, potentially impacting the validity of the findings.
3. Because of the lack of randomization in the cohort design, its external validity is lower than that of study
designs where the researcher randomly assigns participants.

Cross-Sectional Design
Definition and Purpose
Cross-sectional research designs have three distinctive features: no time dimension, a reliance on existing
differences rather than change following intervention; and, groups are selected based on existing differences
rather than random allocation. The cross-sectional design can only measure differences between or from among
a variety of people, subjects, or phenomena rather than change. As such, researchers using this design can only
employ a relative passive approach to making causal inferences based on findings.

What do these studies tell you?

1. Cross-sectional studies provide a 'snapshot' of the outcome and the characteristics associated with it, at a
specific point in time.
2. Unlike the experimental design where there is an active intervention by the researcher to produce and
measure change or to create differences, cross-sectional designs focus on studying and drawing
inferences from existing differences between people, subjects, or phenomena.
3. Entails collecting data at and concerning one point in time. While longitudinal studies involve taking
multiple measures over an extended period of time, cross-sectional research is focused on finding
relationships between variables at one moment in time.
4. Groups identified for study are purposely selected based upon existing differences in the sample rather
than seeking random sampling.
5. Cross-section studies are capable of using data from a large number of subjects and, unlike observational
studies, are not geographically bound.
6. Can estimate prevalence of an outcome of interest because the sample is usually taken from the whole
population.
7. Because cross-sectional designs generally use survey techniques to gather data, they are relatively
inexpensive and take up little time to conduct.

What these studies don't tell you?

1. Finding people, subjects, or phenomena to study that are very similar except in one specific variable can
be difficult.
2. Results are static and time bound and, therefore, gives no indication of a sequence of events or reveals
historical contexts.
3. Studies cannot be utilized to establish cause and affect relationships.
4. Provide only a snapshot of analysis so there is always the possibility that a study could have differing
results if another time-frame had been chosen.
5. There is no follow up to the findings.

5
Descriptive Design
Definition and Purpose
Descriptive research designs help provide answers to the questions of who, what, when, where, and how
associated with a particular research problem; a descriptive study cannot conclusively ascertain answers to why.
Descriptive research is used to obtain information concerning the current status of the phenomena and to
describe "what exists" with respect to variables or conditions in a situation.

What do these studies tell you?

1. The subject is being observed in a completely natural and unchanged natural environment. True
experiments, whilst giving analysable data, often adversely influence the normal behaviour of the
subject.
2. Descriptive research is often used as a pre-cursor to more quantitatively research designs, the general
overview giving some valuable pointers as to what variables are worth testing quantitatively.
3. If the limitations are understood, they can be a useful tool in developing a more focused study.
4. Descriptive studies can yield rich data that lead to important recommendations.
5. Approach collects a large amount of data for detailed analysis.

What these studies don't tell you?

1. The results from a descriptive research cannot be used to discover a definitive answer or to disprove a
hypothesis.
2. Because descriptive designs often utilize observational methods [as opposed to quantitative methods],
the results cannot be replicated.
3. The descriptive function of research is heavily dependent on instrumentation for measurement and
observation.

Experimental Design
Definition and Purpose
A blueprint of the procedure that enables the researcher to maintain control over all factors that may affect the
result of an experiment. In doing this, the researcher attempts to determine or predict what may occur.
Experimental Research is often used where there is time priority in a causal relationship (cause precedes effect),
there is consistency in a causal relationship (a cause will always lead to the same effect), and the magnitude of
the correlation is great. The classic experimental design specifies an experimental group and a control group.
The independent variable is administered to the experimental group and not to the control group, and both
groups are measured on the same dependent variable. Subsequent experimental designs have used more groups
and more measurements over longer periods. True experiments must have control, randomization, and
manipulation.

What do these studies tell you?

1. Experimental research allows the researcher to control the situation. In so doing, it allows researchers to
answer the question, “what causes something to occur?”
2. Permits the researcher to identify cause and affect relationships between variables and to distinguish
placebo effects from treatment effects.
3. Experimental research designs support the ability to limit alternative explanations and to infer direct
causal relationships in the study.
4. Approach provides the highest level of evidence for single studies.

What these studies don't tell you?

1. The design is artificial, and results may not generalize well to the real world.
2. The artificial settings of experiments may alter subject behaviours or responses.
3. Experimental designs can be costly if special equipment or facilities are needed.
4. Some research problems cannot be studied using an experiment because of ethical or technical reasons.
5. Difficult to apply ethnographic and other qualitative methods to experimental designed research studies.

Exploratory Design
Definition and Purpose
An exploratory design is conducted about a research problem when there are few or no earlier studies to refer
to. The focus is on gaining insights and familiarity for later investigation or undertaken when problems are in a
preliminary stage of investigation.
The goals of exploratory research are intended to produce the following possible insights:
 Familiarity with basic details, settings and concerns.
6
  Well-grounded picture of the situation being developed.
 Generation of new ideas and assumption, development of tentative theories or hypotheses.
 Determination about whether a study is feasible in the future.
 Issues get refined for more systematic investigation and formulation of new research questions.
 Direction for future research and techniques get developed.
What do these studies tell you?
1. Design is a useful approach for gaining background information on a particular topic.
2. Exploratory research is flexible and can address research questions of all types (what, why, how).
3. Provides an opportunity to define new terms and clarify existing concepts.
4. Exploratory research is often used to generate formal hypotheses and develop more precise research
problems.
5. Exploratory studies help establish research priorities.

What these studies don't tell you?

1. Exploratory research generally utilizes small sample sizes and, thus, findings are typically not
generalizable to the population at large.
2. The exploratory nature of the research inhibits an ability to make definitive conclusions about the
findings.
3. The research process underpinning exploratory studies is flexible but often unstructured, leading to only
tentative results that have limited value in decision-making.
4. Design lacks rigorous standards applied to methods of data gathering and analysis because one of the
areas for exploration could be to determine what method or methodologies could best fit the research
problem.

Historical Design
Definition and Purpose
The purpose of a historical research design is to collect, verify, and synthesize evidence from the past to
establish facts that defend or refute your hypothesis. It uses secondary sources and a variety of primary
documentary evidence, such as, logs, diaries, official records, reports, archives, and non-textual information
[maps, pictures, audio and visual recordings]. The limitation is that the sources must be both authentic and
valid.
What do these studies tell you?
1. The historical research design is unobtrusive; the act of research does not affect the results of the study.
2. The historical approach is well suited for trend analysis.
3. Historical records can add important contextual background required to more fully understand and
interpret a research problem.
4. There is no possibility of researcher-subject interaction that could affect the findings.
5. Historical sources can be used over and over to study different research problems or to replicate a
previous study.
What these studies don't tell you?
1. The ability to fulfil the aims of your research are directly related to the amount and quality of
documentation available to understand the research problem.
2. Since historical research relies on data from the past, there is no way to manipulate it to control for
contemporary contexts.
3. Interpreting historical sources can be very time consuming.
4. The sources of historical materials must be archived consistently to ensure access.
5. Original authors bring their own perspectives and biases to the interpretation of past events and these
biases are more difficult to ascertain in historical resources.
6. Due to the lack of control over external variables, historical research is very weak with regard to the
demands of internal validity.
7. It rare that the entirety of historical documentation needed to fully address a research problem is
available for interpretation; therefore, gaps need to be acknowledged.

7
Longitudinal Design
Definition and Purpose
A longitudinal study follows the same sample over time and makes repeated observations. With longitudinal
surveys, for example, the same group of people is interviewed at regular intervals, enabling researchers to track
changes over time and to relate them to variables that might explain why the changes occur. Longitudinal
research designs describe patterns of change and help establish the direction and magnitude of causal
relationships. Measurements are taken on each variable over two or more distinct time periods. This allows the
researcher to measure change in variables over time. It is a type of observational study and is sometimes
referred to as a panel study.

What do these studies tell you?

1. Longitudinal data allow the analysis of duration of a particular phenomenon.
2. Enables survey researchers to get close to the kinds of causal explanations usually attainable only with
experiments.
3. The design permits the measurement of differences or change in a variable from one period to another
[i.e., the description of patterns of change over time].
4. Longitudinal studies facilitate the prediction of future outcomes based upon earlier factors.

What these studies don't tell you?

1. The data collection method may change over time.
2. Maintaining the integrity of the original sample can be difficult over an extended period of time.
3. It can be difficult to show more than one variable at a time.
4. This design often needs qualitative research to explain fluctuations in the data.
5. A longitudinal research design assumes present trends will continue unchanged.
6. It can take a long period of time to gather results.
7. There is a need to have a large sample size and accurate sampling to reach representativeness.

Observational Design
Definition and Purpose
This type of research design draws a conclusion by comparing subjects against a control group, in cases where
the researcher has no control over the experiment. There are two general types of observational designs. In
direct observations, people know that you are watching them. Unobtrusive measures involve any method for
studying behaviour where individuals do not know they are being observed. An observational study allows a
useful insight into a phenomenon and avoids the ethical and practical difficulties of setting up a large and
cumbersome research project.

What do these studies tell you?

1. Observational studies are usually flexible and do not necessarily need to be structured around a
hypothesis about what you expect to observe (data is emergent rather than pre-existing).
2. The researcher is able to collect a depth of information about a particular behavior.
3. Can reveal interrelationships among multifaceted dimensions of group interactions.
4. You can generalize your results to real life situations.
5. Observational research is useful for discovering what variables may be important before applying other
methods like experiments.
6. Observation research designs account for the complexity of group behaviors.
7.
What these studies don't tell you?
1. Reliability of data is low because seeing behaviours occur over and over again may be a time consuming
task and difficult to replicate.
2. In observational research, findings may only reflect a unique sample population and, thus, cannot be
generalized to other groups.
3. There can be problems with bias as the researcher may only "see what they want to see."
4. There is no possibility to determine “cause and effect” relationships since nothing are manipulated.
5. Sources or subjects may not all be equally credible.
6. Any group that is studied is altered to some degree by the very presence of the researcher, therefore,
skewing to some degree any data collected (the Heisenburg Uncertainty Principle).

8
Philosophical Design
Definition and Purpose
Understood more as a broad approach to examining a research problem than a methodological design,
philosophical analysis and argumentation is intended to challenge deeply embedded, often intractable,
assumptions underpinning an area of study. This approach uses the tools of argumentation derived from
philosophical traditions, concepts, models, and theories to critically explore and challenge, for example, the
relevance of logic and evidence in academic debates, to analyse arguments about fundamental issues, or to
discuss the root of existing discourse about a research problem. These overarching tools of analysis can be
framed in three ways:
 Ontology -- the study that describes the nature of reality; for example, what is real and what is not, what
is fundamental and what is derivative?
 Epistemology -- the study that explores the nature of knowledge; for example, on what does knowledge
and understanding depend upon and how can we be certain of what we know?
 Axiology -- the study of values; for example, what values does an individual or group hold and why?
How are values related to interest, desire, will, experience, and means-to-end? And, what is the
difference between a matter of fact and a matter of value?
What do these studies tell you?
1. Can provide a basis for applying ethical decision-making to practice.
2. Functions as a means of gaining greater self-understanding and self-knowledge about the purposes of
research.
3. Brings clarity to general guiding practices and principles of an individual or group.
4. Philosophy informs methodology.
5. Refine concepts and theories that are invoked in relatively unreflective modes of thought and discourse.
6. Beyond methodology, philosophy also informs critical thinking about epistemology and the structure of
reality (metaphysics).
7. Offers clarity and definition to the practical and theoretical uses of terms, concepts, and ideas.
What these studies don't tell you?
1. Limited application to specific research problems [answering the "So What?" question in social science
research].
2. Analysis can be abstract, argumentative, and limited in its practical application to real-life issues.
3. While a philosophical analysis may render problematic that which was once simple or taken-for-granted,
the writing can be dense and subject to unnecessary jargon, overstatement, and/or excessive quotation
and documentation.
4. There are limitations in the use of metaphor as a vehicle of philosophical analysis.
5. There can be analytical difficulties in moving from philosophy to advocacy and between abstract
thought and application to the phenomenal world.

Sequential Design
Definition and Purpose
Sequential research is that which is carried out in a deliberate, staged approach [i.e. serially] where one stage
will be completed, followed by another, then another, and so on, with the aim that each stage will build upon the
previous one until enough data is gathered over an interval of time to test your hypothesis. The sample size is
not predetermined. After each sample is analysed, the researcher can accept the null hypothesis, accept the
alternative hypothesis, or select another pool of subjects and conduct the study once again. This means the
researcher can obtain a limitless number of subjects before finally making a decision whether to accept the null
or alternative hypothesis. Using a quantitative framework, a sequential study generally utilizes sampling
techniques to gather data and applying statistical methods to analyse the data. Using a qualitative framework,
sequential studies generally utilize samples of individuals or groups of individuals [cohorts] and use qualitative
methods, such as interviews or observations, to gather information from each sample.
 
What do these studies tell you?
1. The researcher has a limitless option when it comes to sample size and the sampling schedule.
2. Due to the repetitive nature of this research design, minor changes and adjustments can be done during
the initial parts of the study to correct and hone the research method. Useful design for exploratory
studies.
3. There is very little effort on the part of the researcher when performing this technique. It is generally not
expensive, time consuming, or workforce extensive.
4. Because the study is conducted serially, the results of one sample are known before the next sample is
taken and analyzed.

9
What these studies don't tell you?
1. The sampling method is not representative of the entire population. The only possibility of approaching
representativeness is when the researcher chooses to use a very large sample size significant enough to
represent a significant portion of the entire population. In this case, moving on to study a second or more
sample can be difficult.
2. Because the sampling technique is not randomized, the design cannot be used to create conclusions and
interpretations that pertain to an entire population. Generalizability from findings is limited.
3. Difficult to account for and interpret variation from one sample to another over time, particularly when
using qualitative methods of data collection.

10
https://www.scribbr.com/methodology/sampling-methods/
Published on September 19, 2019 by Shona McCombes. Revised on October 2, 2020.

When you conduct research about a group of people, it’s rarely possible to collect data from every person in that
group. Instead, you select a sample. The sample is the group of individuals who will actually participate in the
research.

To draw valid conclusions from your results, you have to carefully decide how you will select a sample that is
representative of the group as a whole. There are two types of sampling methods:

 Probability sampling involves random selection, allowing you to make statistical inferences about the
whole group.
 Non-probability sampling involves non-random selection based on convenience or other criteria,
allowing you to easily collect initial data.

Population vs sample
First, you need to understand the difference between a population and a sample, and identify the target
population of your research.
 The population is the entire group that you want to draw conclusions about.
 The sample is the specific group of individuals that you will collect data from.
The population can be defined in terms of geographical location, age, income, and many other characteristics.
It can be very broad or quite narrow: maybe you want to make
inferences about the whole adult population of your country;
maybe your research focuses on customers of a certain company,
patients with a specific health condition, or students in a single
school.
It is important to carefully define your target
population according to the purpose and practicalities of your
project.
If the population is very large, demographically mixed, and
geographically dispersed, it might be difficult to gain access to a
representative sample.

Sampling frame
The sampling frame is the actual list of individuals that the sample will be drawn from. Ideally, it should
include the entire target population (and nobody who is not part of that population).
Example
You are doing research on working conditions at Company X. Your population is all 1000 employees of the
company. Your sampling frame is the company’s HR database which lists the names and contact details of
every employee.
Sample size
The number of individuals in your sample depends on the size of the population, and on how precisely you want
the results to represent the population as a whole.
You can use a sample size calculator to determine how big your sample should be. In general, the larger the
sample size, the more accurately and confidently you can make inferences about the whole population.

11
Probability sampling methods
Probability sampling means that every member of the population has a chance of being selected. It is mainly
used in quantitative research. If you want to produce results that are representative of the whole population, you
need to use a probability sampling technique.
There are four main types of probability sample.

1. Simple random sampling

In a simple random sample, every member of the population has an equal chance of being selected. Your
sampling frame should include the whole population.
To conduct this type of sampling, you can use tools like random number generators or other techniques that are
based entirely on chance.
Example
You want to select a simple random sample of 100 employees of Company X. You assign a number to every
employee in the company database from 1 to 1000, and use a random number generator to select 100 numbers.

2. Systematic sampling
Systematic sampling is similar to simple random sampling, but it is usually slightly easier to conduct. Every
member of the population is listed with a number, but instead of randomly generating numbers, individuals are
chosen at regular intervals.
Example
All employees of the company are listed in alphabetical order. From the first 10 numbers, you randomly select a
starting point: number 6. From number 6 onwards, every 10th person on the list is selected (6, 16, 26, 36, and so
on), and you end up with a sample of 100 people.

If you use this technique, it is important to make sure that there is no hidden pattern in the list that might skew
the sample. For example, if the HR database groups employees by team, and team members are listed in order
of seniority, there is a risk that your interval might skip over people in junior roles, resulting in a sample that is
skewed towards senior employees.

3. Stratified sampling
Stratified sampling involves dividing the population into subpopulations that may differ in important ways. It
allows you draw more precise conclusions by ensuring that every subgroup is properly represented in the
sample.
To use this sampling method, you divide the population into subgroups (called strata) based on the relevant
characteristic (e.g. gender, age range, income bracket, job role).
Based on the overall proportions of the population, you calculate how many people should be sampled from
each subgroup. Then you use random or systematic sampling to select a sample from each subgroup.
Example
The company has 800 female employees and 200 male employees. You want to ensure that the sample reflects
the gender balance of the company, so you sort the population into two strata based on gender. Then you use
random sampling on each group, selecting 80 women and 20 men, which gives you a representative sample of
100 people.

4. Cluster sampling
Cluster sampling also involves dividing the population into subgroups, but each subgroup should have similar
characteristics to the whole sample. Instead of sampling individuals from each subgroup, you randomly select
entire subgroups.
If it is practically possible, you might include every individual from each sampled cluster. If the clusters
themselves are large, you can also sample individuals from within each cluster using one of the techniques
above.
This method is good for dealing with large and dispersed populations, but there is more risk of error in the
sample, as there could be substantial differences between clusters. It’s difficult to guarantee that the sampled
clusters are really representative of the whole population.
Example
The company has offices in 10 cities across the country (all with roughly the same number of employees in
similar roles). You don’t have the capacity to travel to every office to collect your data, so you use random
sampling to select 3 offices – these are your clusters.

12
Non-probability sampling methods
In a non-probability sample, individuals are selected based on non-random criteria, and not every individual has
a chance of being included.
This type of sample is easier and cheaper to access, but it has a higher risk of sampling bias, and you can’t use it
to make valid statistical inferences about the whole population.
Non-probability sampling techniques are often appropriate for exploratory and qualitative research. In these
types of research, the aim is not to test a hypothesis about a broad population, but to develop an initial
understanding of a small or under-researched population.

1. Convenience sampling
A convenience sample simply includes the individuals who happen to be most accessible to the researcher.
This is an easy and inexpensive way to gather initial data, but there is no way to tell if the sample is
representative of the population, so it can’t produce generalizable results.
Example
You are researching opinions about student support services in your university, so after each of your classes,
you ask your fellow students to complete a survey on the topic. This is a convenient way to gather data, but as
you only surveyed students taking the same classes as you at the same level, the sample is not representative of
all the students at your university.
2. Voluntary response sampling
Similar to a convenience sample, a voluntary response sample is mainly based on ease of access. Instead of the
researcher choosing participants and directly contacting them, people volunteer themselves (e.g. by responding
to a public online survey).

Voluntary response samples are always at least somewhat biased, as some people will inherently be more likely
to volunteer than others.

Example
You send out the survey to all students at your university and a lot of students decide to complete it. This can
certainly give you some insight into the topic, but the people who responded are more likely to be those who
have strong opinions about the student support services, so you can’t be sure that their opinions are
representative of all students.

3. Purposive sampling
This type of sampling involves the researcher using their judgement to select a sample that is most useful to the
purposes of the research.
It is often used in qualitative research, where the researcher wants to gain detailed knowledge about a specific
phenomenon rather than make statistical inferences. An effective purposive sample must have clear criteria and
rationale for inclusion.
Example
You want to know more about the opinions and experiences of disabled students at your university, so you
purposefully select a number of students with different support needs in order to gather a varied range of data
on their experiences with student services.

4. Snowball sampling
If the population is hard to access, snowball sampling can be used to recruit participants via other participants.
The number of people you have access to “snowballs” as you get in contact with more people.
Example
You are researching experiences of homelessness in your city. Since there is no list of all homeless people in the
city, probability sampling isn’t possible. You meet one person who agrees to participate in the research, and she
puts you in contact with other homeless people that she knows in the area.
Instruments https://www.editage.com/insights/what-is-a-research-instrument
A research instrument is a tool used to obtain, measure, and analyze data from subjects around the research
topic. You need to decide the instrument to use based on the type of study you are conducting: quantitative,
qualitative, or mixed-method.

For instance, for a quantitative study, you may decide to use a questionnaire, and for a qualitative study, you
may choose to use a scale. While it helps to use an established instrument, as its efficacy is already established,
you may if needed use a new instrument or even create your own instrument. You need to describe the
instrument(s) used in the Methods section of the paper.
13
Validation and Reliability Testing
Reliability and validity are concepts used to evaluate the quality of research. They indicate how well a method,
technique or test measures something. Reliability is about the consistency of a measure, and validity is about the
accuracy of a measure.

It’s important to consider reliability and validity when you are creating your research design, planning your
methods, and writing up your results, especially in quantitative research.
Reliability Validity

What does it tell The extent to which the results can be
you? reproduced when the research is
repeated under the same conditions.
How is it assessed? By checking the consistency of results
across time, across different observers,
and across parts of the test itself.
How do they relate? A reliable measurement is not always
valid: the results might be reproducible,
but they’re not necessarily correct.
The extent to which the results really
measure what they are supposed to measure.
By checking how well the results correspond
to established theories and other measures of
the same concept.
A valid measurement is generally reliable: if
a test produces accurate results, they should
be reproducible.

Understanding reliability vs validity

Reliability and validity are closely related, but they mean different things. A measurement can be reliable
without being valid. However, if a measurement is valid, it is usually also reliable.

What is reliability?
Reliability refers to how consistently a method measures something. If the same result can be consistently
achieved by using the same methods under the same circumstances, the measurement is considered reliable.
You measure the temperature of a liquid sample several times under identical conditions. The thermometer
displays the same temperature every time, so the results are reliable.

A doctor uses a symptom questionnaire to diagnose a patient with a long-term medical condition. Several
different doctors use the same questionnaire with the same patient but give different diagnoses. This indicates
that the questionnaire has low reliability as a measure of the condition.

What is validity?
Validity refers to how accurately a method measures what it is intended to measure. If research has high validity
that means it produces results that correspond to real properties, characteristics, and variations in the physical or
social world.
High reliability is one indicator that a measurement is valid. If a method is not reliable, it probably isn’t valid.
* If the thermometer shows different temperatures each time, even though you have carefully controlled
conditions to ensure the sample’s temperature stays the same, the thermometer is probably malfunctioning, and
therefore its measurements are not valid.
* If a symptom questionnaire results in a reliable diagnosis when answered at different times and with different
doctors, this indicates that it has high validity as a measurement of the medical condition.
- However, reliability on its own is not enough to ensure validity. Even if a test is reliable, it may not
accurately reflect the real situation.
* The thermometer that you used to test the sample gives reliable results. However, the thermometer has not
been calibrated properly, so the result is 2 degrees lower than the true value. Therefore, the measurement is not
valid.
* A group of participants take a test designed to measure working memory. The results are reliable, but
participants’ scores correlate strongly with their level of reading comprehension. This indicates that the method
might have low validity: the test may be measuring participants’ reading comprehension instead of their
working memory.

Validity is harder to assess than reliability, but it is even more important. To obtain useful results, the
methods you use to collect your data must be valid: the research must be measuring what it claims to
measure. This ensures that your discussion of the data and the conclusions you draw are also valid.

14
How are reliability and validity assessed?
Reliability can be estimated by comparing different versions of the same measurement. Validity is harder to
assess, but it can be estimated by comparing the results to other relevant data or theory. Methods of estimating
reliability and validity are usually split up into different types.
Types of reliability

Type of reliability What does it assess? Example

Test-retest The consistency of a measure across time: A group of participants complete a
do you get the same results when you repeat questionnaire designed to measure
the measurement? personality traits. If they repeat the
questionnaire days, weeks or months apart
and give the same answers, this indicates
high test-retest reliability.
Interrater The consistency of a measure across raters Based on an assessment criteria checklist,
or observers: do you get the same results five examiners submit substantially different
when different people conduct the same results for the same student project. This
measurement? indicates that the assessment checklist has
low inter-rater reliability (for example,
because the criteria are too subjective).
Internal consistency The consistency of the measurement itself: You design a questionnaire to measure self-
do you get the same results from different esteem. If you randomly split the results into
parts of a test that are designed to measure two halves, there should be a strong
the same thing? correlation between the two sets of results. If
the two results are very different, this
indicates low internal consistency.

Types of validity
The validity of a measurement can be estimated based on three main types of evidence. Each type can be
evaluated through expert judgement or statistical methods.
Types of validity
Type of validity What does it assess? Example
Construct The adherence of a measure to existing A self-esteem questionnaire could be
theory and knowledge of the concept being assessed by measuring other traits known or
measured. assumed to be related to the concept of self-
esteem (such as social skills and optimism).
Strong correlation between the scores for
self-esteem and associated traits would
indicate high construct validity.
Content The extent to which the measurement covers A test that aims to measure a class of
all aspects of the concept being measured. students’ level of Spanish contains reading,
writing and speaking components, but no
listening component.  Experts agree that
listening comprehension is an essential
aspect of language ability, so the test lacks
content validity for measuring the overall
level of ability in Spanish.
Criterion The extent to which the result of a measure A survey is conducted to measure the
corresponds to other valid measures of the political opinions of voters in a region. If the
same concept. results accurately predict the later outcome
of an election in that region, this indicates
that the survey has high criterion validity.

To assess the validity of a cause-and-effect relationship, you also need to consider internal validity (the design
of the experiment) and external validity (the generalizability of the results).

15
How to ensure validity and reliability in your research

The reliability and validity of your results depends on creating a strong research design, choosing appropriate
methods and samples, and conducting the research carefully and consistently.

Ensuring validity
If you use scores or ratings to measure variations in something (such as psychological traits, levels of ability or
physical properties), it’s important that your results reflect the real variations as accurately as possible. Validity
should be considered in the very earliest stages of your research, when you decide how you will collect your
data.
 Choose appropriate methods of measurement
Ensure that your method and measurement technique are high quality and targeted to measure exactly what you
want to know. They should be thoroughly researched and based on existing knowledge.
For example, to collect data on a personality trait, you could use a standardized questionnaire that is considered
reliable and valid. If you develop your own questionnaire, it should be based on established theory or findings
of previous studies, and the questions should be carefully and precisely worded.
 Use appropriate sampling methods to select your subjects
To produce valid generalizable results, clearly define the population you are researching (e.g. people from a
specific age range, geographical location, or profession). Ensure that you have enough participants and that they
are representative of the population.

Ensuring reliability
Reliability should be considered throughout the data collection process. When you use a tool or technique to
collect data, it’s important that the results are precise, stable and reproducible.
 Apply your methods consistently
Plan your method carefully to make sure you carry out the same steps in the same way for each measurement.
This is especially important if multiple researchers are involved.
For example, if you are conducting interviews or observations, clearly define how specific behaviours or
responses will be counted, and make sure questions are phrased the same way each time.
 Standardize the conditions of your research
When you collect your data, keep the circumstances as consistent as possible to reduce the influence of external
factors that might create variation in the results.
For example, in an experimental setup, make sure all participants are given the same information and tested
under the same conditions.

16
A step-by-step guide to data collection
Published on June 5, 2020 by Pritha Bhandari. Revised on July 3, 2020.

Data collection is a systematic process of gathering observations or measurements. Whether you are performing
research for business, governmental or academic purposes, data collection allows you to gain first-hand
knowledge and original insights into your research problem.
While methods and aims may differ between fields, the overall process of data collection remains largely the
same. Before you begin collecting data, you need to consider:
 The aim of the research
 The type of data that you will collect
 The methods and procedures you will use to collect, store, and process the data
To collect high-quality data that is relevant to your purposes, follow these four steps.

Step 1: Define the aim of your research

Before you start the process of data collection, you need to identify exactly what you want to achieve. You can
start by writing a problem statement: what is the practical or scientific issue that you want to address and why
does it matter?
Next, formulate one or more research questions that precisely define what you want to find out. Depending on
your research questions, you might need to collect quantitative or qualitative data:
 Quantitative data is expressed in numbers and graphs and is analyzed through statistical methods.
 Qualitative data is expressed in words and analyzed through interpretations and categorizations.
If your aim is to test a hypothesis, measure something precisely, or gain large-scale statistical insights, collect
quantitative data. If your aim is to explore ideas, understand experiences, or gain detailed insights into a specific
context, collect qualitative data. If you have several aims, you can use a mixed methods approach that collects
both types of data.

Examples of quantitative and qualitative research aims

You are researching employee perceptions of their direct managers in a large organization.
 Your first aim is to assess whether there are significant differences in perceptions of managers across
different departments and office locations.
 Your second aim is to gather meaningful feedback from employees to explore new ideas for how
managers can improve.
You decide to use a mixed-methods approach to collect both quantitative and qualitative data.

Step 2: Choose your data collection method

Based on the data you want to collect, decide which method is best suited for your research.
 Experimental research is primarily a quantitative method.
 Interviews/focus groups and ethnography are qualitative methods.
 Surveys, observations, archival research and secondary data collection can be quantitative or qualitative
methods.
Carefully consider what method you will use to gather data that helps you directly answer your research
questions.

17
Method When to use How to collect data
Experiment To test a causal relationship. Manipulate variables and measure their effects on
others.
Survey To understand the general Distribute a list of questions to a sample online, in
characteristics or opinions of a person or over-the-phone.
group of people.
Interview/focus To gain an in-depth understanding Verbally ask participants open-ended questions in
group of perceptions or opinions on a individual interviews or focus group discussions.
topic.
Observation To understand something in its Measure or survey a sample without trying to affect
natural setting. them.
Ethnography To study the culture of a community Join and participate in a community and record your
or organization first-hand. observations and reflections.
Archival research To understand current or historical Access manuscripts, documents or records from
events, conditions or practices. libraries, depositories or the internet.
Secondary data To analyze data from populations Find existing datasets that have already been collected,
collection that you can’t access first-hand. from sources such as government agencies or research
organizations.

Step 3: Plan your data collection procedures

When you know which method(s) you are using, you need to plan exactly how you will implement them. What
procedures will you follow to make accurate observations or measurements of the variables you are interested
in?
For instance, if you’re conducting surveys or interviews, decide what form the questions will take; if you’re
conducting an experiment, make decisions about your experimental design.
Operationalization
Sometimes your variables can be measured directly: for example, you can collect data on the average age of
employees simply by asking for dates of birth. However, often you’ll be interested in collecting data on more
abstract concepts or variables that can’t be directly observed.
Operationalization means turning abstract conceptual ideas into measurable observations. When planning how
you will collect data, you need to translate the conceptual definition of what you want to study into the
operational definition of what you will actually measure.

Example of operationalization

You have decided to use surveys to collect quantitative data. The concept you want to measure is the leadership
of managers. You operationalize this concept in two ways:
You ask managers to rate their own leadership skills on 5-point scales assessing the ability to delegate,
decisiveness and dependability.

You ask their direct employees to provide anonymous feedback on the managers regarding the same topics.
Using multiple ratings of a single concept can help you cross-check your data and assess the test validity of
your measures.

Sampling
You may need to develop a sampling plan to obtain data systematically. This involves defining a population, the
group you want to draw conclusions about, and a sample, the group you will actually collect data from.
Your sampling method will determine how you recruit participants or obtain measurements for your study. To
decide on a sampling method you will need to consider factors like the required sample size, accessibility of the
sample, and timeframe of the data collection.

18
Standardizing procedures
If multiple researchers are involved, write a detailed manual to standardize data collection procedures in your
study.
This means laying out specific step-by-step instructions so that everyone in your research team collects data in a
consistent way – for example, by conducting experiments under the same conditions and using objective criteria
to record and categorize observations.
This helps ensure the reliability of your data, and you can also use it to replicate the study in the future.

Creating a data management plan

Before beginning data collection, you should also decide how you will organize and store your data.
 If you are collecting data from people, you will likely need to anonymize and safeguard the data to
prevent leaks of sensitive information (e.g. names or identity numbers).
 If you are collecting data via interviews or pencil-and-paper formats, you will need to
perform transcriptions or data entry in systematic ways to minimize distortion.
 You can prevent loss of data by having an organization system that is routinely backed up.

Step 4: Collect the data

Finally, you can implement your chosen methods to measure or observe the variables you are interested in.

Examples of collecting qualitative and quantitative data

To collect data about perceptions of managers, you administer a survey with closed- and open-ended questions
to a sample of 300 company employees across different departments and locations.

The closed-ended questions ask participants to rate their manager’s leadership skills on scales from 1–5. The
data produced is numerical and can be statistically analyzed for averages and patterns.

The open-ended questions ask participants for examples of what the manager is doing well now and what they
can do better in the future. The data produced is qualitative and can be categorized through content analysis for
further insights.

To ensure that high quality data is recorded in a systematic way, here are some best practices:
 Record all relevant information as and when you obtain data. For example, note down whether or how
lab equipment is recalibrated during an experimental study.
 Double-check manual data entry for errors.
 If you collect quantitative data, you can assess the reliability and validity to get an indication of your
data quality.

Statistical Treatment of Data

Statistical treatment of data is when you apply some form of statistical method to a data set to transform it from
a group of meaningless numbers into meaningful output.

Statistical treatment of data involves the use of statistical methods such as:

 mean,
 mode,
 median,
 regression,
 conditional probability,
 sampling,
 standard deviation and
 distribution range.

19
These statistical methods allow us to investigate the statistical relationships between the data and identify
possible errors in the study.

In addition to being able to identify trends, statistical treatment also allows us to organize and process our data
in the first place. This is because when carrying out statistical analysis of our data, it is generally more useful to
draw several conclusions for each subgroup within our population than to draw a single, more general
conclusion for the whole population. However, to do this, we need to be able to classify the population into
different subgroups so that we can later break down our data in the same way before analyzing it.

Type of Errors https://www.discoverphds.com/blog/statistical-treatment-of-data

A fundamental part of statistical treatment is using statistical methods to identify possible outliers and errors.
No matter how careful we are, all experiments are subject to inaccuracies resulting from two types of errors:
systematic errors and random errors.

 Systematic errors are errors associated with either the equipment being used to collect the data or with
the method in which they are used.

 Random errors are errors that occur unknowingly or unpredictably in the experimental configuration,
such as internal deformations within specimens or small voltage fluctuations in measurement testing
instruments.

These experimental errors, in turn, can lead to two types of conclusion errors: type I errors and type II errors. A
type I error is a false positive which occurs when a researcher rejects a true null hypothesis. On the other hand,
a type II error is a false negative which occurs when a researcher fails to reject a false null hypothesis.

20