1.

Introduction: Some Principles

of Empirical Research
Bernd Hayo, Empirical Macroeconomics
Scientific principles of investigation

• The main difference between scientific and non-scientific

research is the application of specific methods for drawing
conclusions.
• The same methods can be applied by different individuals and,
therefore, allow potentially a replication of the results.
• Thus, scientific study entails developing specific research
methods.
• In economics, a difference is often made between theoretical and
empirical research.
• In empirical research, the attempt is made to draw conclusion
from observing events in the ‘real’ world.
– This, of course, assumes the possibility of external
observation in the first place. For a different perspective,
check, e.g., constructivism.
2 Hayo: Empirical Macroeconomics
 Drawing conclusions

• There are two ways of drawing conclusions:

– Deduction
• From general statement to specific statements.
– Induction
• From specific statements to general principles.
• When we work with data, we use induction.
– Many specific events share similar characteristics (e.g. every
time income goes up, consumption increases too).
– We use such findings to derive general statements.
– Note: Hume’s problem: we cannot easily justify inductive
conclusions.
3 Hayo: Empirical Macroeconomics
Validation versus rejection of claims
• From the philosophy of science (particularly Karl Popper) we
know that the conclusive power of rejection is much stronger
than that of validation.
• Example: Take the hypothesis that ‘all swans are white’.
– On the one hand, you can go around all parks with lakes or
rivers in Marburg and check whether there are white swans.
• However, in the end, you can only state that in Marburg,
within a given time period and using the available means
of observations, there were white swans but NOT that all
swans are white.
– On the other hand, the observation of even one black swan
can potentially destroy the hypothesis.

4 Hayo: Empirical Macroeconomics

Scientific world in uproar: black swan mystery solved!
• Finally, after many years of research, I found black swans in
New Zealand and, therefore, can reject the hypothesis that all
swans are white!

5 Hayo: Empirical Macroeconomics

The limits of the usefulness of rejection
• Remember that empirical analysis means observing the
environment we live in.
• Given the complexity of human behaviour, it is difficult to derive
general propositions.
• However, economics is based on general propositions and thus
extremely vulnerable to counterexamples.
• Thus, if we took rejection seriously, NO economic theory would
survive empirical testing.
• This is one reason why in practice economics is not based on
falsification.
• Instead, researchers tend be concerned with finding supporting
evidence for specific theories, i.e. they are more concerned with
verification.
6 Hayo: Empirical Macroeconomics
Potential flaws in theory and observation
• But there are also other reasons related to the potential
limitations of observations.
1. We may not be able to separate observation and
interpretation, i.e. neutral observation may be impossible.
2. Our means of observations could be flawed, e.g. a
microscope bending light in such a way to create a flawed
image of an object. In economics, most of our data are full
of errors and often just rough estimates, e.g. GDP.
3. When applying econometric methods, there are problems
related to model specification and statistical uncertainty.
• Thus, we face a dilemma when drawing conclusions: is the
theory flawed and should be rejected? Or are our methods of
observations so noisy that we cannot learn anything useful? It
could even be the case that both are flawed!
7 Hayo: Empirical Macroeconomics
Causality
• Quite often we are interested in finding out whether one
variable causes another variable

• Causal relationships can be

– singular, i.e. a specific event a causes another specific
event b
– general, i.e. an event of type A causes an event of type B

• In natural sciences, general causal relationships can be found,

in social sciences, this is rarely the case

8 Hayo: Empirical Macroeconomics

Causality in social sciences

• More useful in social sciences is a probabilistic causality:

– Prob(B|A) = r
– Translation: The probability that B occurs given A is equal to r

• This is close to the ‘disposition’ concept of causality:

– A has a force (disposition) to bring about B
– Problem: Often, disposition cannot be observed, which is a
problem for a ‘positivist‘ philosophy of science

9 Hayo: Empirical Macroeconomics

Other concepts of causality:

• Regularity concept of causality:

– A and B follow each other over time, i.e. if A, then B
– Problem: Regularities may occur without a causal relation

• Counterfactual concept of causality:

– Without A, B will not come about
– Problem: If A never occurs, we cannot say anything useful

10 Hayo: Empirical Macroeconomics

Testing for causality

• Concurrence method (John Duns Scotus)

– Under different conditions, which are all possible in
principle, B always occurs when A is present

Case Condition Consequence

1 ACD B
2 ACE B
3 AEFG B
4 ADEF B
…

11 Hayo: Empirical Macroeconomics

• Difference method (William of Ockham):
– B occurs if and only if A occurs, not matter what
combination of other factors are present

Case Condition Consequence

1 ACD B
2 ACE B
3 CDE Not B
4 ADEF B
…

12 Hayo: Empirical Macroeconomics

 Practical problems of investigating causality

• Control problem
– B is caused by K but A always comes about if K happens. If
we do not control for K, we will come to the conclusion that
B is caused by A, although K is the true cause

• Interference problem
– There is a factor S, which blocks the causal influence of A
on B. Thus, A causes B but only if S does not occur

• In practical empirical analyses, we have to face the problem

that we do not take into account K and S, as:
– we do not know about them
– we cannot measure them

13 Hayo: Empirical Macroeconomics

Case studies vs statistical analysis

• A phenomenon of interest is studied by looking at one or more

specific events or cases.
• One case
– Phenomenon occurred only once (e.g., the Great
Depression).
– But: the historical novelty of an event depends on its
interpretation (e.g. the recent financial crisis: is it unique or
just one of many financial crises we have observed over
time?).
– We may want to look at one case because it is of special
importance (e.g., the Great Depression) and thus deserves
a particularly deep analysis.

14 Hayo: Empirical Macroeconomics

 Advantages of studying one case
– The analyst has a profound knowledge about the
circumstances relating to this case.
– The analysis may generate hypotheses that may be of use
for future research.

Disadvantages of studying one case

– However, these hypotheses cannot be tested within the
framework of one case, as we should not use the same
dataset to develop and test hypotheses.
– A yardstick for comparing our case may be lacking, which
makes interpretation more difficult.
– Our case could be an outlier and of no further interest than
the event itself.
15 Hayo: Empirical Macroeconomics
• Several cases
– More than one relevant example exists.
• Relevance is again based on interpretation by the
analyst.
– The focus is on discovering similarities between the various
cases.
– The analyst can study some cases to generate hypotheses
and use the remaining ones for testing.

16 Hayo: Empirical Macroeconomics

 Many cases: Statistical analysis

– The more cases there are, the less likely it is that the
analyst can become familiar with each individual case.
– It could be the case, however, that studying single cases
will not yield insights about a common structure underlying
all cases.
– Thus, studying a large number of cases might generate
additional insights and prevent wrong inferences due to
induction based on single cases.
– Deriving conclusions from a sample of many similar cases
is the domain of statistical analysis.

17 Hayo: Empirical Macroeconomics

Comparative Analysis

• Comparing different situations is important for making educated

statements, not least those involving reference to data.
• Comparison is used in at least two contexts:
1. Comparison as yardstick
2. Comparison as a widely used method of investigation

18 Hayo: Empirical Macroeconomics

Comparison as yardstick
• Question: How do we know whether a measured value is large
or small?
• We don’t if we do not have a yardstick value!
• Example: In discussions, people often use numbers to support
their argument.
– ‘This new marketing policy is effective, as our sales
revenues increased by 5%’
– You hear such statements often but when taken on their
own, they do not contain much useful information!
– We need something like: Before implementing the new
marketing policy, our sales revenues grew by 3% or over the
same period, our competitors sales grew by 4%.

19 Hayo: Empirical Macroeconomics

So we need some sort of yardstick, based on:
• Theory
− We may be able to derive a useful yardstick based on
theoretical arguments.
− For instance, a price elasticity above one is called elastic, so
our estimated value of 0.18 indicates an inelastic response of
consumers to price increases.
• Observations
− Derive a useful yardstick based on empirical observations,
e.g., all products in this category have a price elasticity above
one, so our estimated value of 0.18 indicates a relatively
inelastic response of our consumers to price increases.
− Comparison can be made across units (over different
products) or across time (same product over different points
in time).
20 Hayo: Empirical Macroeconomics
Comparison as method

• Example: How can we explain differences in profits across

different firms?
• Solution method: We compare two (or more!) firms with different
profits and try to find out where the more successful one differs
from the less successful one.
• If the firms turn out to be equal except with regard to factor A,
we conclude that it is the influence of factor A that is responsible
for the difference in performance.
• This type of comparison is probably the most widely used
research method.

21 Hayo: Empirical Macroeconomics

• Some problems of the comparative approach are:
– Too many potentially relevant factors, not enough
empirically measurable variables.
• For instance, we cannot control for all the factors that are likely
going to be relevant for firms profits.
– Seemingly similar variables have different effects on
outcomes.
• For instance, unemployment rates can be defined in different
ways.
– Cultural context of concepts.
• For instance, seemingly similar concepts can be interpreted
quite differently in different cultures.
– Comparison must take place on same conceptual level.
• For instance, if we compare a market economy and a centrally-
planned economy, we can either look at the properties of these
system as ideal cases or at their practical implementation but
we should not mix these conceptual levels.
22 Hayo: Empirical Macroeconomics
Reliability vs validity

• One aspect of empirical research is to make sure that results

derived in one study can be replicated in another study.
• Very often, we have to work with ‘constructs’, i.e. concepts that
we believe to be important based on theory, but which we
cannot directly observe, e.g. utility.
• In principle, using different indicators to measure our constructs
or using the same indicators in different samples, we should be
able to get the same results.
• Internal reliability supports the internal consistency of our
indicators, i.e. whether they appear to measure the same
construct.
• External reliability supports the claim that our results are close
to reality.
23 Hayo: Empirical Macroeconomics
• In the case of two indicators, reliability can be measured by a
correlation. In the case of more indicators, a statistic like
Cronbach’s Alpha can be used, which is based on the average
correlation between indicators.
• Validity is a measure showing whether our indicators do what
they are supposed to do, namely being closely related to real-
world entities.
• Note: Validity requires reliability, but reliability does not imply
validity.
• Example: Assume we use GDP and GNI as indicators for a
country’s welfare. We will often find that the internal reliability of
these concepts is high (they are strongly correlated) and, after
many decades of refinement by statistical offices, external
reliability is likely high, too. However, it is possible that in spite
of high income, people may not feel ‘well’. Thus, although
reliable, our indicators would not be valid.
24 Hayo: Empirical Macroeconomics
Types of statistical analyses

• Descriptive statistics
– Aims at providing a summary of data, either using
appropriate numbers (statistics) or graphical means.
– These summaries can lead to the discovery of stylised facts
(what is a ‘stylised fact’?) about a phenomenon.
– It can already form the basis for an analysis, even though
we do not statistically test hypothesis.
– For instance, by graphing the development of our and our
competitors average firm revenues over time, we find that
ours are rising less steeply.

25 Hayo: Empirical Macroeconomics

• Statistical inference
– Allows testing of statistical hypotheses.
– By transforming economic hypotheses into statistical
hypotheses, we can test economic statements using
empirical data.
– Example: we assume that our customers are price sensitive.
• By estimating the price elasticity of consumption and
statistically testing its value against unity, we can draw
inferences about the underlying economic hypothesis.

26 Hayo: Empirical Macroeconomics

Types of data

• Data can contain different degrees of information, even though

they are coded as numbers.
– Nominal scale: Numbers only indicate differences in the
characteristics of the underlying observations, e.g., male
customers coded as 1, female customers coded as 2.
– Ordinal scale: observations can be ranked using the
numbers, e.g. marks given in a lecture course.
– Cardinal scale: Distances between numbers can be
interpreted, i.e., an observation coded as ‘4’ is two times as
large as one coded as ‘2’, e.g. income or height.

27 Hayo: Empirical Macroeconomics

Dimensions of data

• Data can have different dimensions

1. Cross-sectional data: Variation across units but not across
time, e.g. customer satisfaction survey undertaken in April
2011.
2. Time-series data: Variation across time but not across units,
e.g. a firm’s sales over the period August 2005 to April 2011.
3. Mix of cross-sectional and time series data.
– Repeated cross-sections
– Panel data

28 Hayo: Empirical Macroeconomics

References on scientific methodology in economics

• Blaug, M. (1992), The Methodology of Economics, Cambridge:

Cambridge University Press.

• Boumans, M. und J. Davis (2010), Economic Methodology:

Understanding Economics as a Science, Basingstoke: Palgrave
Macmillan.

• Hausman, D. (1992), The Inexact and Separate Science of

Economics, Cambridge: Cambridge University Press.

• Lavoie, D. (1990), Economics and Hermeneutics, ed., London:

Routledge.

29 Hayo: Empirical Macroeconomics