#1 What are the two principal ways to test theories?

Experimentation and Observation (case study analysis or large-n analysis).

#2 What are the four principal types of tests that emerge out of these two principal predictive

qualities? Also, how do each of these types of tests work in validating or invalidating a

theory/hypothesis?

Hoop tests - high certitude but no uniqueness; decisive negative test; failed test kills a theory but a

passed one gives it little support.

2. Smoking-gun tests - high uniqueness but no certitude; decisive positive test; passage strongly

confirms the theory, but failure infirms it very little.

3. Doubly-decisive tests - high uniqueness and high certitude; decisive both ways (passage strongly

confirms theory and failure kills it), convey the most information but are rare.

4. Straw-in-the-wind tests - low uniqueness and low certitude; indecisive both ways.

#3 What are the 10 "Helpful Hints for Testing Theories"?

1. Test as many theory’s hypotheses as possible.

2. Infer and test as many predictions of each hypotheses as possible.

3. Explain and defend the predictions you infer from your theory.

4. Select data that represent, as accurately as possible, the domain of the test

5. Consider and evaluate the possibility that the observed relationship between two variables is not

causal but rather results from the effect of the third variable.

6. When interpreting results, judge each theory on its own merits.

7. We can repair theories by replacing disconfirmed hypotheses with new explanatory hypotheses

proposing a different intervening process or by narrowing the scope of theory’s claims.

8. We can test theories against the null hypothesis or against each other.

9. One tests theory by asking if the empirical evidence confirms the theory’s predictions, not by

asking how many cases the theory can explain.

10. One does not test a theory by assessing the validity of its assumptions.

#4 What are the strengths and weaknesses of case studies?

- weaknesses
 - least opportunity for control for the effect of perturbing third variables (experiments,

then large-n analysis better on that front)

- case study results can’t be generalized to other cases

- strengths

- tests of predictions of within-case variance gain strong control from the uniform

character of the background conditions of the case

- we can control the effects of omitted variables by selecting for study cases with

extreme (high or low) values on the study variable

- testing predictions are unique, thus singular to their theory, providing decisive

evidence

- inferring and testing explanations that define how the independent causes the

dependent variable are often easier with case study than large-n methods

#5 What are the five main reasons/purposes case studies can offer?

- testing theories

- creating theories

- identifying antecedent conditions

- testing the importance of these antecedent conditions

- explaining cases of intrinsic importance

#6 What are the three general formats for testing that case studies offer?

- Controlled Comparison

- Congruence Procedures

- Process Tracing

#7 Regarding the specific testing format being used, what are the three steps that ought to be

followed?

- state the theory

- state the expectations about what we should observe in the case, if the theory is valid, and

what we should observe if it’s false

 - explore the case (or cases) looking for congruence or incongruity between expectation and

observation

#8 Elaborate on what are Controlled Comparisons?

Investigator explores paired observations in two or more cases asking if values on the pairs are

congruent or incongruent with the theory’s predictions. Theory passes if the DV are higher than in

case A than B. If they are much higher - theory has large importance. If a little higher - passes, but

small importance. Overall the weakest method.

#9 What are the two principal ways Controlled Comparisons are carried out?

John Stuart Mill’s „method of difference” or „method of agreement”

#10 Elaborate on what are Congruence Procedures.

The investigator explores the case looking for congruence or incongruence between values observed

on the dependent or independent variable and values predicted by the test hypothesis.

#11 What are the two principal ways Congruence Procedures are carried out?

Comparison to typical values or multiple within-case comparisons.

#12 What is Process Tracing and how does it work?

The investigator explores the chain of events or the decision-making process by which initial case

conditions are translated into outcomes. The cause-effect link is unwrapped and divided into smaller

steps, then the investigator looks for observable evidence on each step. Often unique.

#13 What are Delphi Methods?

Investigator mines the views of case participants or others who experienced the case for hypotheses,

using their memories and judgment.

#14 What are Delphi Methods for?

For using already established judgment that the investigator might not be able to state alone.

#15 Can Delphi Methods be used to test theories?

No

#16 How does one Create Theories via a case study?

Searching cases for associations between phenomena and for testimony by people who directly

experienced the case on their motives and beliefs about the case, which offer clues on cause and
effect. Once candidate causes and effects are framed in general terms the investigator has theories that

can be tested against other evidence and applied to other cases.

#17 How does one infer Antecedent Conditions via a case study?

Controlled comparison, congruence procedures, process tracing, delphi methods.

#18 How does one test Antecedent Conditions via a case study?

Controlled comparison, congruence procedures, process tracing.

#19 What are the four questions to assess the explanation for a specific case?

- Does the explanation exemplify a valid general theory? - it must be

- Is the covering law’s causal phenomenon present in the case? - it must be

- Are the covering law’s antecedent conditions met in the case? - they must be

- Are the covering law’s intervening phenomena observed in the case? - it should be

#20 What should an investigator do in testing cases?

Select cases that enable the most strong tests, explain and justify their predictions, and comment on

the strength of the test formed.

#21 What are the 5 procedures to interpret contradicting results?

- Infer and test additional predictions, with a special eye toward finding „hoop” and

„smoking-gun” tests.

- Double-check the accuracy of data used for past tests.

- Reconsider the predictions you inferred from the theory. Were they fair?

- Replicate your tests using new cases.

- Repair the theory in ways that enable it to pass flunked tests, by limiting the scope of its

claims or by removing flunked explanatory hypotheses.

#22 What are Van Evera's 11 cross-case criteria?

- data richness

- extreme values on the independent, dependent or condition variables

- large within-case variances in values on the independent, dependent or condition variables

- divergence of predictions made of case background conditions to the conditions of current

policy problems
 - the resemblance of case background conditions to the conditions of current policy problems

- prototypicality of case background conditions

- appropriateness for controlled comparison with other cases (mainly using Mill’s method of

difference)

- outlier character

- intrisinc importance

- appropriateness for replication of previous tests

- appropriateness for performing a previously omitted type of test

#23 What are the two general criteria that govern Van Evera's cross-case criteria?

- Investigators should select the cases, which best serve the purpose of their inquiry.

- Investigators should select the cases to maximize the strength and number of tests they let the

investigator perform.