9 Methods and Procedures 143 Conditions Occur with Phenomena [-TeTe] ——b Positive Canon of Agreement Cc I D I E X ic) I F | G | es |X (conclude that C causes X) A [ B [ ‘|-_j¥% Negative Canon of Agreement e —| €L*[«¢] ——[*% (conclude that C causes X) Ni Ne Method of al B lé Te Difference A I B I c | ——— | x (conclude that C causes X) Figure 9.1. Representation of Mill's Methods of Agreement and Difference A brief aside on this matter of causation vs, relationship: Experimental meth- ods, whether those as classified by Mill or the statistical and econometric tech- niques that are used, do not have the capability to establish causation, Our tech- niques, be they observational or statistical, can establish only associations. They help us determine, often probabilistically, whether (and sometimes to. what degree) things are 1 ied to one another. Evidence of causation is derived by first144 Il The Research Project Design developing hypotheses of direction of causation from conceptual reasoning (the- ory), then examining for evidence of the expected relationship. If the empirical evidence exists to support the relationship, then it supports the hypothesis of cau- sation. The causative implications come from the conceptual reasoning (theory) rather than the empirical evidence. Research in all sciences is concerned with isolating and quantifying effects of individual conditions (variables). In economics, we try to understand and/or quantify the effects of the different variables on a particular economic phenome- non independent of the effects of the other relevant variables. It is important to understand the effect of the price of a given commodity on its consumption inde- pendent of (and in conjunction with) the effects of the prices of related commodi- ties, the effects of consumers’ incomes, and so forth. Understanding of these rela- tionships is important because we must understand the role of each to understand their collective influence. Thus, the problem of identifying and controlling for other variables that may affect the phenomenon being studied, and identifying interactive influences of those variables on the phenomenon, has demanded prominent attention in all the social and physical sciences. Control of related variables in the laboratory and field sciences has been attempted largely through variations of Mill's Method of Difference. In econom- ics, control of variables is more difficult because of the complexity of systems and phenomena studied and the infeasibility of controlled experiments in most cases. These factors have made necessary the adoption and adaptation of statisti- cal means of controlling for the effects of variables in economics, Economists were quick to adopt the methods of statistics in the early part of the twentieth cen- tury. As the multivariate statistical techniques, particularly regression analysi became feasible as empirical techniques, a means for statistical “control” of other forces was provided.' The field of econometrics grew from this emphasis, which has been fostered not only by the continued developments in mathematical statis- tics (statistical theory) but also by the technology of calculators and computers Advancements in computing technology alone have expanded empirical possibil- ities greatly; we routinely solve algorithms today that were impossible even ten years ago. MODELS IN ECONOMIC RESEARCH Economic models are abstractions from reality, developed in whole or part from theory, often expressed in mathematical format, and their purposes are to provide (1) explanations and predictions; (2) discovery; and (3) description and illustra- tion (Edwards, 1978; Ghebremedhin and Tweeten, 1994; Simkin, 1993, p. 67). They can be used with or without recourse to data (i.e., they may be empirical or solely theoretical). When models are constructed with the intent of estimating structure or parameters, the model constitutes a form of hypothesis.9 Methods and Procedures 145 The purpose of a model is to explain how a relationship or system works—to identify the factors or forces that are driving a phenomenon and explain with as much specificity as possible how those forces act and interact to cause the phe- nomenon. The adaptation of theory to a particular set of phenomena, and for a given purpose, forms a model. If the model explains how a set of phenomena works, it can be used to predict direction of change and identify how policy instruments may be used to affect that change (Edwards, 1978). Although the model structure for the study proposed in appendix A is not specified, when the study was conducted the empirical model drew heavily on the theoretical model of the conceptual framework, yet deviated in several important respects. Some of the deviations were based on data limitations and others on forces believed to have an impact on the policy structure that were not dealt with explicitly within the conceptual framework. ‘A model may be simple or complex. A simple model might contain a single relationship—for example, an estimated relationship between the price of apart- ment rents and the apartment vacancy rate. However, a single relationship does not, in itself, constitute a simple model; some single equation models may repre- sent very complex phenomena. Complex models often, however, contain multiple (often many) interrelated relationships or equations. Economic journals abound with examples of complex models. ‘When we merge data and theory in a model with a particular intent, we build an empirical model; this is the focus of econometricians and other model builders. Empirical models may be classified as econometric, optimization, or simulation Econometric models are stochastic (they estimate relationships and parameters with some probability of error) and positive (they estimate relationships and para- meters from data as they are produced by the actual working of the phenomena being studied—i.e., they use data produced by the phenomena studied). Opri- mization models (e.g., linear programming) are normative (they derive solutions according to a specified set of objectives—ic., the specified or “desired” objec- tive function). Optimization models may be nonstochastic or stochastic. When they are stochastic, the probability distributions are derived outside the model and imposed on the optimizing objective function. Simulation models are mathematical constructs that are positive in their intent (to simulate “reality”) and nonstochastic except in cases when simulations are con- ducted repeatedly with the driving conditions for successive simulations drawn from probability distributions of those conditions. Simulation models, while posi- tive in focus, are different from econometric models because they are inherently artificial constructions as opposed to derivations from observed phenomena. Opti- mization and simulation models typically involve no statistical hypothesis testing but are often the most effective in cases in which there is no way to observe the phe- nomena being studied directly. It also should be noted that the types of models are not mutually exclusive. Econometric models, for example, sometimes provide esti- mates of relationships that, in turn, are used for mathematical simulation,146 I The Research Project Design ‘We economists tend to be fascinated with and devoted to our models, occa- sionally with a certain amount of blind allegiance that can be detrimental? Models and modeling may embody the best that we, as a field of study, have to offer: and we may even be the best at the modeling activity among all fields of study. It represents a means for us to capture the inner workings of very com- plex phenomena and understand how and why things happen. It is a means for us to be scientific in the sense of testing theory and a means to provide (condi- tional) predictions about economic phenomena. But we also should recognize the limitations of models and bear in mind that the models are the means rather than the ends. Everything that is important in economics cannot be modeled: some may argue that the most important things cannot be modeled. Perhaps our most fundamental mistake is to fall prey to the temptation to pretend that some- thing does not exist if it cannot be quantified. Merely capturing thinking within models does not, in itself, legitimize the thinking. As Breimyer (1991. p. 252) states: Employment of mathematical methods does not substitute for rigorous theo- retical formulations, Rather, it itself requires more precise understanding of the constructs of which the discipline is constituted. A different but related point is addressed by McCloskey (1990a, pp. 1126-1127). Economists often mistake statistical significance for scientific sig- nificance. Regression analysis, while useful, is merely a detail of method. TYPES OF EMPIRICAL METHODS Economists use an array of empirical methods or techniques in conducting research and invest considerable attention and energy into choosing, testing, and evaluating these techniques for various applications and into developing or adapt- ing new techniques for applications. The classification of empirical methods pre- sented below is meant as a summary of the general types of techniques, not an exhaustive listing. The classification, while influenced by Williams (1984) and Johnson (1988), reflects the author's own thinking of how the different methods may be grouped. There are warranted differences of opinion on where some spe- cific tools should be classified in relation to others. Nevertheless, the grouping of empirical methods is into two main categories, statistical and econometric tools and operations research tools, plus a method that does not fit well in either cate- gory (the descriptive method). Some would argue that the descriptive method is not really an analytical empirical method because it is subjective (interpretive); it is included in the discussion here because it usually entails the application/use of data, albeit in an interpretative way.