How a research agenda can be organized, i.e what is the basis for a successful research project ?

A research agenda can be organized around four questions (we call these Frequently Asked Questions "FAQs", because they should be) : - the causal relationship of interest, - the experiment that could ideally be used to capture the causal effect of interest. - the identication strategy (Some research questions cannot be answered by any experiment - and the mode of inference.

What is a causal relationship useful for ?

A causal relationship is useful for making predictions about the consequences of changing circumstances or policies; it tells us what would happen in alternative (or counterfactual) worlds.

Could the experiment that would ideally be used to capture the causal effect of interest be hypothetical ?

Ideal experiments are most often hypothetical. Still, hypothetical experiments are worth contemplating because they help us pick fruitful research topics. The description of an ideal experiment also helps you formulate causal questions precisely.The mechanics of an ideal experiment highlight the forces you'd like to manipulate and the factors you'd like to hold constant.

What exactly does a Fundamentally Unidentified Question ( FUQ) look like ?

At first blush, questions about the causal effect of race or gender seems like good candidates because these things are hard to manipulate in isolation (imagine your chromosomes were switched at birth).

You created this PDF from an application that is not licensed to print to novaPDF printer (

data not generated by a randomized trial) to approximate a real experiment. we might randomly select some kids to start kindergarten at age 6. namely. And we usually have some such experiment in mind when we construct the theories.. a fact that raises achievement if school is worth anything. a pure maturation effect. Here is our econometrics forefather. Angrist and Krueger (1991) used the interaction between compulsory attendance laws in American schools and students-season of birth as a natural experiment to estimate the effects of finishing high school on wages (season of birth affects the degree to which high school students are constrained by laws allowing them to drop out on their birthdays).one of the 4 FAQ that are concerned with the nuts-and-bolts elements producing a specific econometrical study? The fourth research FAQ borrows language from Rubin (1991): what is your mode of statistical inference? The answer to this question describes the population to be that older kids tend to do better on tests. appealing for more explicit discussion of both kinds of experimental designs:"A design of experiments is an essential appendix to any quantitative . (1) experiments that we should like to make to see if certain real economic phenom ena. If they did. Although inference issues are rarely very exciting.The last chapter covers practical problems that arise once you've answered question number 4.Often inference is more complex. Describe ideal vs natural experiments .Now. There is no way Define the term identification strategy . and the assumptions made when constructing standard errors. as evidenced by their elementary school test scores. But the first group has spent more time in school. especially with data that are clustered or grouped.novapdf. We are interested in whether those held back learn more in school. The problem with this question .e. Trygve Haavelmo (1944).Sometimes inference is straightforward. Suppose we test those who started at age 6 in second grade and those who started at age 7 in first grade so everybody is tested at age 7.Give an example of a Fundamentally Unidentifieed Question (FUQ) ? To assess the effects of delayed school entry on learning. as when you use Census micro-data samples to study the American population. while others start at age 5.the effects of start age on first grade test scores . one of the 4 FAQ that are concerned with the nuts-and-bolts elements producing a specific econometrical study? Angrist and Krueger (1999) used the term identification strategy to describe the manner in which a researcher uses observational data (i. it m ight seem we can fix this by holding age constant instead of grade.would verify certain Discuss the mode of statistical inference. they would see that the experiments they have in mind may be grouped into two different classes. however. the ultimate success of even a well-conceived and conceptually exciting project turns on the details of You created this PDF from an application that is not licensed to print to novaPDF printer (http://www. The juxtaposition of ideal and natural experiments has a long history in econometrics.when artificially isolated from "other infuences". as is still typical. the sample to be used. although-unfortunately-most economists do not describe their design of experiments explicitly. and often quite technical.

The observed relation between class size and student achievement should not be taken at face value. the trainees earn less than plausible comparison groups Here too. What are the types of possible selection bias ? There are many types of possible selection bias.novapdf. that the students assigned to classes of di¤erent sizes are otherwise comparable.Sampling bias : is systematic error due to a non-random sample of a population. causing some members of the population to be less likely to be included than others.. resulting from the method of collecting samples. defined as a statistical sample of a population (or non-human factors) in which all participants are not equally balanced or objectively represented 2. A randomized trial overcomes this problem by ensuring that we are comparing apples to apples. So perhaps school systems can save money by hiring fewer teachers with no consequent reduction in achievement.e. since weaker students are often deliberately grouped into smaller classes.A trial may be terminated early at an extreme value (often for ethical reasons). simple comparisons of program participants with non-participants often show lower earnings for the participants. Results from the You created this PDF from an application that is not licensed to print to novaPDF printer (http://www. The term "selection bias" most often refers to the distortion of a statistical analysis. If the selection bias is not taken into account then certain conclusions drawn may be wrong.It is sometimes referred to as the selection effect.What is a "Selection bias " ? Selection bias is a statistical bias in which there is an error in choosing the individuals or groups to take part in a scientific study. selection bias is a natural concern since subsidized training programs are meant to serve men and women with low earnings potential. but the extreme value is likely to be reached by the variable with the largest variance.Time interval: Early termination of a trial at a time when its results support a desired conclusion. even if all variables Give an example explaining why random Assignment solves the selection problem ? Paradoxically. i. evidence from randomized evaluations of training programs generate mostly positive e¤ects Many studies of education production using non-experimental data suggest there is little or no link between class size and student learning. studies based on non-experimental comparisons of participants and non-participants often show that after training. In contrast. resulting in a biased sample. Not surprisingly. . therefore. including: 1.

com) . however. 4 In other cases. therefore. such trials are impractical. attempts to exploit cheaper and more readily available sources of variation. the resulting estimates have a causal interpretation. regression estim ates m ay or may not have a causal interpretation. E [y | x] is the conditional expectation function and the goal of econometrics is to estimate this function. Can we always nd a convincing natural experiment? Of course not.novapdf. regression is used with observational data. where E [e | x] = 0: E [e | x] = E [y − E [y | x] | x] = E [y | x] − E [y | x] = 0. to understand how y changes with x. In most cases. Much of the research we do. You created this PDF from an application that is not licensed to print to novaPDF printer (http://www. We hope to nd natural or quasi-experiments that mimic a randomized trial by changing the variable of interest while other factors are kept balanced. Wha ie a conditional expectation function ? Let y be an outcome and x be a vector of predictors. we would like an answer sooner rather than later.When the regressor of interest is randomly assigned. y = E [y | x] + e. What makes a regression causal ? Without the benet of random assignm ent. that is.Discuss the credibility and feasability of randomized trials ? In many cases. Then.