EMPIRICAL RESEARCH METHODS

Research Methods Class, March 8, 15 & 22, 2002

Daniel Moody, Visiting Associate Professor, NTNU & Senior Research
Fellow, Monash University (Melbourne, Australia)

What is Empirical Research?

Empirical research methods are a class of research methods in which empirical observations or
data are collected in order to answer particular research questions. While primarily used in
academic research, they can also be useful in answering practical questions.

Research Questions, Theoretical Models and Hypotheses

Empirical research normally starts with some a priori theory, which the researcher develops
to try to explain and/or predict what happens in the real world. The purpose of the research
is to test the theory and possibly refine it. In some cases, research is conducted to develop
theory (the grounded theory approach), but this is much harder (and more risky) to do.

Research Questions
Initially a research question is formulated. For example:
• Is programming language A more effective than programming language B?
• What are the critical success factors in implementing an ERP system?
Theoretical Models
To be empirically tested, the research question will need to be transformed into a theoretical
model, consisting of theoretical constructs (latent variables), causal relationships and meas-
ures (observed variables). The theoretical model is generally developed based on analysis of
the literature. The theoretical model forms the basis both for collecting and analysing data,
and may be modified as a result of the research.

Hypotheses
A hypothesis defines an expected relationship between variables (based on causal relation-
ships in the theoretical model), which can be empirically tested.

Qualitative vs Quantitative Methods

Empirical research methods can be divided into two categories:
• Quantitative research methods: such methods collect numerical data (data in the form
of numbers) and analyse it using statistical methods.
• Qualitative research methods: such methods collect qualitative data (data in the form
of text, images, sounds) drawn from observations, interviews and documentary
evidence, and analyse it using qualitative data analysis methods.

Page 1
Qualitative methods tend to be more appropriate in the early stages of research (exploratory
research) and for theory building. Quantitative methods tend to be more appropriate when
theory is well developed, and for purposes of theory testing and refinement. In practice, no
research method is entirely qualitative or quantitative (Yin, 1994). For example, a survey
may collect qualitative data using open ended questions as well as quantitative data using
closed questions; an experiment may include observations of participant behaviour as well
as measures of response time and accuracy; a case study may incorporate quantitative data
(e.g. system usage statistics) as well as qualitative data (e.g. interviews with users).

The selection of an appropriate research method is critical to the success of any research pro-
ject, and must be driven by the research question and the state of knowledge in the area be-
ing studied. In general, a combination of research methods may be most effective in achiev-
ing a particular research objective. For example, when a subject area is not well understood,
qualitative methods may be used to build theory and testable hypotheses. This theory may
then be tested using quantitative methods such as surveys and experiments. Mixing qualita-
tive and quantitative research methods is called triangulation of method. While most re-
searchers develop expertise in one style, the two types of methods have different, comple-
mentary strengths and when used together can lead to a more comprehensive understanding
of a phenomenon.

Quantitative Methods
The most common quantitative methods are:
• Experiment: apply a treatment, measure results (before and/or after): this is the
only method that can demonstrate causal relationships between variables. Experi-
mental research is associated with the traditional “scientific method” (the “Newto-
nian” model of science).
• Survey: ask questions (face to face interview, telephone, mail, internet)
• Historical data: look for patterns in historical data (e.g. IT investment patterns)
While quantitative methods tend to result in more convincing scientific evidence, they are
generally more difficult to apply in a real world context (field setting).

Qualitative Methods
The most common qualitative methods are:
• Case study: observations carried out in a real world setting (e.g. a software devel-
opment project, an operating theatre in a hospital). The objective is to immerse
yourself in the situation and gain a holistic understanding of the phenomena in its
natural setting.
• Action Research: apply a research idea in practice, evaluate results, modify idea
(cross between an experiment and case study)
Qualitative methods tend to be applied more easily in real world settings, but lack internal
validity (alternative explanations of results; lack of control) and external validity (usually
concerned with a single case: limits generalisability to other settings). Also, interpretation of
data is by nature much more subjective than quantitative methodsit is easy to read what
you want into the data.

Page 2
Data Analysis
Used to determine whether your theory is supported or not supported.
• Quantitative data analysis: use of statistical methods to identify patterns and rela-
tionships in the data.
• Qualitative data analysis: analysis is more subjective, and relies heavily on the re-
searcher’s knowledge and experience to identify patterns, extract themes and make
generalisations.

Topic Outline
Class 1 (March 8): Empirical Research Methods
Topics covered:
• Empirical research methods
• Quantitative vs qualitative methods
• Theory building
• Internal and external validity
Class 2 (March 15): Quantitative Research Methods
Topics covered:
• Experiments
• Surveys
• Historical analysis
Class 3 (March 22): Qualitative Research Methods
Topics covered:
• Case study
• Action research

Additional References
These represent “definitive” references on particular research topics. While not necessary in
this subject, they are essential reading if you intend to use any of these techniques in your
future research.

Theory Building
DUBIN, R. (1978): Theory Building (Revised Edition), The Free Press, New York.

Experimental Research
CAMPBELL, D.T. and STANLEY, J.C. (1966): Experimental and Quasi-Experimental Designs for
Research, Houghton Mifflin College.

Survey Research
DILLMAN, D.A. (2000): Mail and Internet Surveys: The Tailored Design Method (2nd ed.),
John Wiley & Sons, New York.

Page 3
Case Study Research
YIN, R. K. (1994) Case Study Research: Design Methods. Second Edition, Sage Publications,
San Francisco

Action Research
Action Research Resources: a web site with a wide collection of resources on action research
(including “How to Do an Action Research Thesis”)
http://www.scu.edu.au/schools/gcm/ar/arhome.html

Statistical Analysis
PEDHAZUR, E.J. and SCHMELKIN, L.P. (1991): Measurement, Design and Analysis: An Inte-
grated Approach, Lawrence Erlbaum Associates Inc, Hillsdale, NJ.

Qualitative Data Analysis

MILES, M.B. and A.M. HUBERMAN (1994): Qualitative data analysis: An expanded source
book, 2nd ed. Thousand Oaks: Sage Publications

Page 4