108

CASE STUDY RESEARCH

. f ifyi d umentary evidence, including

2 Excellent suggestions regarding the ways 0 ven mg oc ff d by Barzun

the nontrivial problem of determining the actual author of a document, are 0 e~ r blem is and Graff (1986, pp. 109-133). An exemplary quantitative study of the authorship po

found in Mosteller and Wallac~ (19b84)k' (Yin 20(3) contains a complete multiple-case study

3. Chapter 9 of the companion 00 '.

that quantitatively analyzed a critical set of archlval records. .' b f" I d"

. h intervlewmg mem ers 0 a C ose

4. Such consistent responses are likely 10 occur 7 en m or the teachers in a closely knit

institution, such as the residents of ~ drug treatmehn probs:a . terviewed all are aware of the

t iracy anses because t ose emg m

school. The apparen consp be nrovi ding corroboratory evidence when in fact

"socially desirable" responses and appear to e provi

th I repeating their institution's mantra.

r:y are mere y , bo k (Yin 2003) for an example of a complete case

5 See Chapter 2 of the compamon 0 I, •

stud~ that is written in the form of narrative answers to the prolOCol questions.

5

Analyzing Case Study Evidence

Data analysis consists of examining, categorizing, tabulating, testing, or othelWise recombining both quantitative and qualitative evidence to address the initial propositions of a study. Analyzing case study evidence is especially difficult because the strategies and techniques have not been well defined. Familiarity with various tools and manipulative techniques is helpful, but every case study should nevertheless strive to have a general analytic strategy-defining priorities for what to analyze and why. Three strategies are relying on theoretical propositions, setting up a framework based on rival explanations, and developing case descriptions.

Any of these strategies can be used in practicing five specific techniques for analyzing case studies: pattern matching, explanation building, time-series analysis, logic models, and cross-case synthesis. The first four are applicable whether a study involves a single- or a multiple-case design, and every case study should consider these techniques. Regardless of the choice of strategies or techniques, a persistent challenge is to produce high-quality analyses, which require investigators to attend to all the evidence, display and present the evidence separate from any interpretation, and show adequate concern for exploring alternative interpretations.

AN ANALYTIC STRATEGY: MORE THAN FAMILIARITY WITH ANALYTIC TOOLS

Need for an Analytic Strategy

The analysis of case study evidence is one of the least developed and most difficult aspects of doing case studies. Too many times, investigators start case studies without having the foggiest notion about how the evidence is to be analyzed (despite Chapter 3's recommendation that the analytic approaches be developed as part of the case study protocol). Such investigations easily become stalled at the analytic stage; this author has known

109

110

CASE STUDY RESEARCH

i· '.

colleagues who have simply ignored their case study data for month after month, not knowing what to do with the evidence.

Because of the problem, the experienced case study investigator is likely to have great advantages over the novice at the analytic stage. Unlike statistical analysis, there are few fixed formulas or cookbook recipes to guide the novice (one of the few texts providing useful advice is Miles & Huberman, 1994). Instead, much depends on an investigator's own style of rigorous thinking, along with the sufficient presentation of evidence and careful consideration of alternative interpretations.

At the same time, investigators and especially novices continue to be oriented toward the search for formulas, recipes, or tools, hoping that familiarity with these devices will produce the needed analytic result. The tools are important and can be useful, but they are usually most helpful if you know what to look for (i.e., have an overall analytic strategy), which returns you back to your original problem.

For instance, computer-assisted routines with prepackaged software, such as nonnumerical unstructured data indexing, searching, and theorizing (NUD·IST) (e.g., Gahan & Hannibal, 1999) or Computer Assisted Qualitative Data Analysis Software (CAQDAS) (e.g., Fielding & Lee, 1998), have become increasingly used. The software helps you to code and categorize large amounts of narrative text, as might have been collected from open-ended interviews or from historic documents. Guidance on coding skills and techniques also has improved (e.g., Boyatzis, 1998; Strauss & Corbin, 1998).

The great benefit from such tools is when (a) the narrative texts represent

\ a verbatim record of an interviewee's remarks or the literal content of a file or historic document, and (b) the empirical study is trying to derive meaning and insight from the word usage and frequency pattern found in the texts. (Indeed, content analysis was a pre-computer-age technique . frequently used to analyze newspaper texts.) However, the verbatim or documentary records are likely to be only part of your total case study; alternatively, they may be the initial phase of a case study, aimed at

surfacing salient concepts or themes (e.g., Strauss & Corbin, 1998). In either situation, you still need an analytic strategy to address the larger or fuller case study. (The main exception would be when your inquiry focused entirely on learning lessons from the verbatim or documentary text-but you should be aware that such a study would be a study of verbal behavior and not necessarily of actual events.)

Another set of helpful analytic manipulations has been comprehensively described and summarized by Miles and Huberman (1994) and includes

ANALYZING CASE STUDY EVIDENCE

111

• Putting information into different arrays

• Making a matrix of categories and placing the evidence within such categories

• Creating data displays-flowcharts and other graphics-for examining the data

• Tabulating the frequency of different events

• ~xamining the complexity of such tabulations and their relationships by calculating second-order numbers such as means and variances

• Putting information in chronological order or using some other temporal scheme

T~ese ar: indeed us~fu.l and important manipulations and can put the ~vlde~ce 10 some preliminary order. Moreover, conducting such manipulanons IS one way of overcoming the stalling problem mentioned earlier. In this sense, manipulating the data at this early stage, or "playing with the data,". c~ be a fruitful activity. Without a broader strategy, however, you are still likely t~ encounter many false starts and potentially waste large chunks of your time. Furthermore, if after playing with the data a general strategy does not emerge (or if you are not facile in playing with the data to begin .wi.th), the entire case study analysis is likely to be in jeopardy.

In a SImilar mode, yet another investigator has espoused the coding of case study events into numerical form, to make the case study data conducive to statistical analysis (Pelz, 1981). Such "quantitative" case studies may be possible when you have an embedded unit of analysis within a case study, but the approach still fails to address the need for doing analysis at the level of the whole case, where there may be only a single or a few cases-far too few for quantification to be helpful.

. A ~igher priority than sheer familiarity with these tools and manipulations IS to have a general analytic strategy in the first place. Once you have a strategy, ~e tools may tum out to be extremely useful (or irrelevant). The strate~y will he~p you to treat the evidence fairly, produce compelling analytJ~ conclusions, and rule out alternative interpretations. The strategy also wIll. help you to use tools and make manipulations more effectively and efficiently, Three such strategies are described below after which five specific techniques for analyzing case study data are reviewed, A continued alert is to be aware of these choices before collecting your data, so that you can be sure your data will be analyzable.

Three General Strategies

i!

. Relying on theoretica~ propositions. The first and most preferred strategy i,1

IS to follow the theoretical propositions that led to your case study. The r

112

CASE STUDY RESEARCH

I] original objectives and design of the case study presumably were based on such propositions, which in tum reflected a set of research questions, reviews of the literature, and new hypotheses or propositions.

J The propositions would have shaped your data collection plan and there-

fore would have given priorities to the relevant analytic strategies. One example, from a study of intergovernmental relationships, followed the proposition that federal funds not only have redistributive dollar effects but also create new organizational changes at the local level (Yin, 1980). The basic proposition-the creation of a "counterpart" bureaucracy in the form of local planning organizations, citizen action groups, and other new offices within a local government itself, but all attuned to specific federal programs-was traced in case studies of several cities. For each city, the purpose of the case study was to show how the formation and modification in local organizations occurred after changes in related federal programs, as well as how these local organizations acted on behalf of the federal programs even though they might have been components of local government.

This proposition is an example of a theoretical orientation guiding the case study analysis. Clearly, the proposition helps to focus attention on certain data and to ignore other data. (A good test is to decide what data you might cite if you had only 5 minutes to defend a proposition in your case study.) The proposition also helps to organize the entire case study and to define alternative explanations to be examined. Theoretical propositions about causal relations-answers to "how" and "why" questions--can be extremely useful in guiding case study analysis in this manner.

Thinking about rival explanations. A second general analytic strategy. tries to define and test rival explanations. This strategy can be related to the first, in that the original theoretical propositions might have included rival hypotheses. However, the strategy is relevant even in the absence of such theoretical propositions and is especially useful in doing case study evaluations.

For instance, the typical hypothesis in an evaluation is that the observed outcomes were the result of an intervention supported by public or foundation funds. The simple or direct rival explanation would be that the observed outcomes were in fact the result of some other influence besides the intervention and that the investment of funds may not actually have been needed. Being aware (ahead of time) of this direct rival, your case study data collection should then have included attempts to collect evidence about the possible "other influences." Furthermore, you should have pursued your data collection about them vigorously-as if you were in fact trying to prove the salience of the other influences; then, if you had found

ANALYZING CASE STUDY EVIDENCE

113

Craft Rivals:

1. The Null Hypothesis

2. Threats to Validity

3. Investigator Bias

The observation is the result of chance circumstances only e.g., history. maturation, Instability, testing, instrumentation. regression, selection, experimental mortality, and selectlcomaturation interaction

e.g., "experimenter effect"; reactivity In field research

Real-Life Rivals:

4, Direct Rival

(Practice or Policy) 5. Commingled Rival (Practice or Policy)

6. Implementation Rival

7. Rival Theory

8. Super Rival

s. SOCietal Rival

An intervention ("suspect 2") other than the target lnterventlon ("suspect 1") accounts for the results ("the butler did 11")

other interventions and the target interventlon both contributed to the results ("it wasn't only me")

The Implementation process, not the substantive intervention. accounts for the results ("did we do it right?")

A theory different from the original theory explains the results better ("It's elementary. my dear Watson")

A force larger than but including the IntelVenllon accounts for the results ("it's bigger than both of us")

Social trends, not any particular force or intervention, account for the results ('he times they are a-chanqln")

Figure 5.1 Brief Descriptions of Different Kinds of Rival Explanations SOURCE: Yin (2000).

insufficient evidence, you would less likely be accused of "stacking the deck" in favor of the original hypothesis (Patton, 1990, p. 462).

The direct rival-that the original investment was not the reason for the observed outcomes-is but one of several types of rival explanations. Figure 5.1 classifies and itemizes many types of rivals (Yin, 2000), For each type, an informal and more understandable descriptor accompanies the formal social science categorization, hoping to make the gist of the rival clearer.

The list reminds us of three "craft" rivals that underlie all of our social

science research, and textbooks have given much attention to these craft rivals. However, the list also defines six "real-life" rivals that have received virtually no attention by other textbooks (nor, unfortunately, do most texts discuss the challenges and benefits of rival thinking or the use of rival ~xpl~natio~s). These real-life rivals are the ones that you should carefully identify pnor to your data collection (while not ignoring the craft rivals).

. Some real-life rivals also may not become apparent until you are in the midst of your data collection, and attending to them at that point is not only acceptable but also desirable. Overall, the more rivals that your analysis addresses and rejects, the more confidence you can place in your findings.

114

CASE STUDY RESEARCH

Rival explanations were a critical part of several of the case studies already contained in the BOXES cited earlier (e.g., refer to BOXES 1 and 11). The authors of these case studies used the rivals to drive their entire case study analysis. Additional examples of the use of rival explanations in case studies are presented in the companion to this book (Yin, 2003)_i

Developing a case description. A third general analytic strategy is to develop a descriptive framework for organizing the case study. This strategy is less preferable than the use of theoretical propositions or rival explanations but serves as an alternative when you are having difficulty making either of the other approaches work.

Sometimes, the original purpose of the case study may have been a descriptive one. This was the objective of the famous sociological study Middletown (Lynd & Lynd, 1929), which was a case study of a small midwestern city. What is interesting about Middletown, aside from its classic value as a rich and historic case, is its structure, reflected by its chapters:

• Chapter 1: Getting a Living

• Chapter II: Making a Home

• Chapter III: Training the Young

• Chapter IV: Using Leisure

• Chapter V: Engaging in Religious Practices

• Chapter VI: Engaging in Community Activities

These chapters cover a range of topics relevant to community life in the early 20th century, when Middletown was studied. The descriptive framework also organizes the case study analysis. (As an aside, a useful exercise is to note the structure of existing case studies---e.g., those cited in the BOXES throughout this book-and to examine their tables of contents, as an implicit clue to different analytic approaches.)

In other situations, the original objective of the case study may not have been a descriptive one, but a descriptive approach may help to identify the appropriate causal links to be analyzed---even quantitatively. BOX 24 gives an example of a case study that was concerned with the complexity of implementing a local public works program in Oakland, California. Such complexity, the investigators realized, could be described in terms of the. multiplicity of decisions that had to occur for implementation to succeed. This descriptive insight later led to the enumeration, tabulation, and hence quantification of the various decisions. In this sense, the descriptive approach was used to identify (a) an embedded unit of analysis (see

·1

I

ANALYZING CASE STUDY EVIDENCE

115

BOX 24

Quantifying the Descriptive Elements of a Case Study

Pressman and Wildavsky's (1973) book, Implementation: How Great Expectations in Washington Are Dashed in Oakland, is regarded as one of the foremost contributions to the study of implementation (Yin, 1982b). This is the process whereby some programmatic activity-an economic development project, a new curriculum in a school, or a crime prevention program, for example--is installed in a specific setting (e.g., organization or community). The process is complex and involves numerous individuals, organizational rules, social norms, and mixtures of good and bad intentions.

Can such a complex process also be the subject of quantitative inquiry and analysis? Pressman and Wildavsky offer one innovative solution. To the extent that successful implementation can be described as a sequence of decisions, an analyst can focus part of the case study on the number and types of such decisions or elements.

Thus, in their chapter titled "The Complexity of Joint Action," the authors analyze the difficulties in Oakland: To implement one public works program required a total of 70 sequential decisions-project approvals, negotiation of leases, letting of contracts, and so on. The analysis examined the level of agreement and the time needed to reach agreement at each of the 70 decision points. Given the normal diversity of opinion and slippage in time, the analysis illustrates-in a quantitative manner-the low probability of implementation success.

Chapter 2) and (b) an overall pattern of complexity that ultimately was used in a causal sense to "explain" why implementation had failed.

Summary. The best preparation for conducting case study analysis is to have a general analytic strategy. Three have been described, relying on theoretical propositions, rival explanations, and case descriptions. These three general strategies underlie the specific analytic techniques to be described below. Without such strategies (or alternatives to them), case studyanalysis will proceed with difficulty.

The remainder of this chapter covers specific analytic techniques to be used as part of and along with any of the general strategies. The techniques are especially intended to deal with the previously noted problems of developing internal validity and external validity in doing case studies (see Chapter 2).

116

CASE STUDY RESEARCH

SPECIFIC ANALYTIC TECHNIQUES

None of the analytic techniques should be considered easy to use, and all will need much practice to be used powerfully. Your objective should be to start modestly, work thoroughly and introspectively, and build. your own analytic repertoire over time. The reward will eventually e~erge in the f~nn of compelling case study analyses and, ultimately, compelling case studies.

Pattern Matching

For case study analysis, one of the most desirable techniques is using .a pattern-matching logic. Such a logic (Trochi~, 1989) compares .an empl~cally based pattern with a predicted one (or with several alternative predictions). Ifthe patterns coincide, the results can help a case study to strengthen its internal validity.

If the case study is an explanatory one, the patterns may be related to the dependent or the independent variables of study (or both). If the case study is a descriptive one, pattern matching is still relevant, as long a~ the predicted pattern of specific variables is defined prior to data collection,

Nonequivalent dependent variables as a pattern. The dependent-variables pattern may be derived from one of the more potent quas~-experim~n~~ research designs, labeled a "nonequivalent, dependent vanables de.s1gn (Cook & Campbell, 1979, p. 118). According to this design, an ex.penme~t or quasi-experiment may have multiple dependent variables-that 1S, a vanety of outcomes. If, for each outcome, the initially predicted ~alues have been found and at the same time alternative "patterns" of predicted values (including 'those deriving from methodological artifacts, or "threats" to validity) have not been found, strong causal inferences can b~ made.

For example, consider a single case in which you are studying the ~~ec!s of a newly decentralized office computer system. Your major .propoSltion 1S that-because each peripheral piece of equipment can work mdependent~y of any server-a certain pattern of organizational changes and stresses ~ln be produced. Among these changes and stresses, you s~ec~fy the following, based on propositions derived from previous decentralization theory:

! I il

II

;

• employees will create new applications for the office system, and these applications will be idiosyncratic to each employee;

• traditional supervisory links will be threatened, as manage.ment c.on~?1 over work tasks and the use of central sources of information will be diminished;

ANALYZTNG CASE STUDY EVIDENCE

117

• organizational conflicts will increase, due to the need to coordinate resources and services across the decentralized units; but, nevertheless,

• productivity will increase over the levels prior to the installation of the new system.

In this example, these four outcomes each represent different dependent variables, and you would assess each with different measures and instruments. To this extent, you have a study that has specified nonequivalent dependent variables. You also have predicted an overall pattern of outcomes covering each of these variables. If the results are as predicted, you can draw a solid conclusion about the effects of decentralization. However, if the results fail to show the entire pattern as predicted-that is, even if one variable does not behave as predicted-your initial proposition would have to be questioned.

This first case could then be augmented by a second one, in which another new office system had been installed, but of a centralized naturethat is, the equipment at all of the individual workstations had been networked. Now you would predict a different pattern of outcomes, using the same four dependent variables enumerated above. And now, if the results show that the decentralized system (Case 1) had actually produced the predicted pattern and that this first pattern was different from that predicted and produced by the centralized system (Case 2), you would be able to draw an even stronger conclusion about the effects of decentralization. In this situation, you have made a theoretical replication across cases. (In other situations, you might have sought a literal replication by identifying and studying two or more cases of decentralized systems.)

Finally, you might be aware of the existence of certain threats to the validity of this logic (see Cook & Campbell, 1979, for a full list of these threats). For example, a new corporate executive might have assumed office in Case 1, leaving room for a counterargument: that the apparent effects of decentralization were actually attributable to this executive's appointment and not to the newly installed office system. To deal with this threat, you would have to identify some subset of the initial dependent variables and show that the pattern would have been different (in Case 1) if the corporate executive had been the actual reason for the effects. If you only had a single-case study, this type of procedure would be essential; you would be using the same data to rule out arguments based on a potential threat to validity. Given the existence of a second case, as in our hypothetical example, you also could show that the argument about the corporate executive would not explain certain parts of the pattern found in Case 2 (in which the absence of the corporate executive should have been associated

118

CASE STUDY RESEARCH

BOX 25

Pattern Matching for Rival Explanations

A common policy problem is to understand the conditions under which new research findings can be made useful to society. Too often, people think that research serves only itself and does not meet practical needs.

This topic was the subject of several case studies in which a research project's results were known to have been used. The case studies investigated how and why this outcome had occurred, entertaining several rival explanations based on three prevailing models of research use; (a) a research, development, and diffusion model; (b) a problem-solving model; and (c) a social interaction model (COSMOS, 1984a). The events of each case were compared to those predicted by each model in a pattern-matching mode. For instance, the problem-solving model requires the prior existence of a problem, as a prelude to the initiation of a research project, but the condition is not recognized by the other two models. This is therefore an example of how different theoretical models can predict mutually exclusive events, facilitating effective comparisons.

For all of the cases that were studied (N = 9), the events turned out to match best a combination of the second and third models. The investigators had therefore used rival explanations to analyze the data within each case and a replication logic across cases.

with certain opposing outcomes). In essence, your goal is to identify all reasonable threats to validity and to conduct repeated comparisons, showing how such threats cannot account for the dual patterns in both of the hypothetical cases.

Rival explanations as patterns. The use of rival explanations, besides being a good general analytic strategy, also provides a good example of pattern matching for independent variables. In such a situation (for an example, see BOX 25), several cases may be known to have had a certain type of outcome, and your investigation has focused on how and why this outcome occurred in each case.

This analysis requires the development of rival theoretical propositions, articulated in operational terms. The desired characteristic of these rival explanations is that each involves a pattern of independent variables that is mutually exclusive: If one explanation is to be valid, the others cannot be. This means that the presence of certain independent variables (predicted by one explanation) precludes the presence of other independent variables

ANALYZrNG CASE STUDY EVIDENCE

119

(predicted by a rival explanation). The independent variables may involve several or many different types of characteristics or events, each assessed with different measures and instruments. The concern of the case study an~lysis, however, is with the overall pattern of results and the degree to WhICh the observed pattern matches the predicted one.

This type of pattern matching of independent variables also can be done either with a single case or with multiple cases. With a single case, the successful matching of the pattern to one of the rival explanations would be evidence for concluding that this explanation was the correct one (and that the other explanations were incorrect). Again, even with a single case, threats to validity-basically constituting another group of rival explanations-should be identified and ruled out. In addition, if this identical result were also obtained over multiple cases, literal replication of the single cases would have been accomplished, and the cross-case results might be stated even more assertively. Then, if this same result also failed to occur in a second group of cases, due to predictably different circumstances, theoretical replication would have been accomplished, and the initial result would stand yet more robustly.

Simpler patterns. This same logic can be applied to simpler patterns, having a minimal variety of either dependent or independent variables. In the simplest case, in which there may be only two different dependent (or independent) variables, pattern matching is possible as long as a different pattern has been stipulated for these two variables.

The fewer the variables, of course, the more dramatic the different patterns will have to be to allow any comparisons of their differences. Nevertheless, there are some situations in which the simpler patterns are both relevant and compelling. The role of the general analytic strategy would be to determine the best ways of contrasting any differences as sharply as possible and to develop theoretically significant explanations for the different outcomes.

Precision of pattern matching. At this point in the state of the art, the actual pattern-matching procedure involves no precise comparisons. Whether one is predicting a pattern of nonequivalent dependent variables, a pattern based on rival explanations, or a simple pattern, the fundamental comparison between the predicted and the actual pattern may involve no quantitative or statistical criteria. (Available statistical techniques are likely to be irrelevant because none of the variables in the pattern will have a "variance," each essentially representing a single data point.) The most precise quantitative result will likely occur if the study had set preestablished

120

CASE STUDY RESEARCH

benchmarks-for example, productivity will increase by IOo/o-and the value of the actual outcome was then compared to this benchmark.

Low levels of precision can allow for some interpretive discretion on the part of the investigator, who may be overly restrictive in claiming a pattern to have been violated or overly lenient in deciding that a pattern has been matched. You can make your case study stronger by developing more precise measures. In the absence of such precision, an important suggestion is to avoid postulating very subtle patterns, so that your pattern matching deals with gross matches or mismatches whose interpretation is less likely to be challenged.

Explanation BuDding

A second analytic technique is in fact a special type of pattern matching, but the procedure is more difficult and therefore deserves separate attention. Here, the goal is to analyze the case study data by building an explanation about the case.

As used in this chapter, the procedure is mainly relevant to explanatory case studies. A similar procedure, for exploratory case studies, has been commonly cited as part of a hypothesis-generating process (see Glaser & Strauss, 1967), but its goal is not to conclude a study but to develop ideas for further study.

Elements of explanations. To "explain" a phenomenon is to stipulate a presumed set of causal links about it. These causal links are similar to the independent variables in the previously described use of rival explanations. In most studies, the links may be complex and difficult to measure in any precise manner.

In most existing case studies, explanation building has occurred in narrative form. Because such narratives cannot be precise, the better case studies are the ones in which the explanations have reflected some theoretically significant propositions. For example, the causal links may reflect critical insights into public policy process or social science theory. The public policy propositions, if correct, can lead to recommendations for future policy actions (see BOX 26, part A, for an example); the social science propositions, if correct, can lead to major contributions to theory building (see BOX 26, part B, for an example).

Iterative nature of explanation building. The explanation-building process, for explanatory case studies, has not been well documented in

ANALYZING CASE STUDY EVIDENCE

BOX 26

A. Explanation Building in Multiple-Case Studies

In a mult~ple-~ase study, one goal is to build a general explanation that fits each o~the. tn~lvldual cases, even though the cases will vary in their details. The obJectlve IS analogous to multiple experiments.

~artha Derthick's (1972) ?'few Towns In-Town: Why a Federal Program Fall~~ IS a ~ook about a housing program under President Lyndon Johnson's ?-dnl1n~s~tlOn. The federal government was to give its surplus land-located In choice Inner-city areas-to local governments for housing developments But a~er 4 years, little progress had been made at the seven sites=San Antonio, Ne~ ~edford (M~ssachusetts), San Francisco, Washington, D.C., Atlanta, LOUISVIlle, and Chnton Township (Michigan}-and the program was considered a failure.

Derthick's account first analyzes the events at each of the seven sites.

Then, a genera~ explanation-that the projects failed to generate sufficient local support-e-rs found unsatisfactory because the condition was not domi~t at all of ~e sites. According to Derthick, local support did exist, but

federal officials ~ad nevertheless stated such ambitious objectives that som~ degree of failure was certain" (p. 91). As a result, Derthick builds a modified explanation and concludes that "the surplus lands program failed both because. the f~deral government had limited influence at the local level and because It set Impossibly high objectives" (p. 93).

B. Explanation Building in Multiple-Case Studies:

An Example From Another Field

An ~al~ic ?-pproa~h similar to Derthick's is used by Barrington Moore (1966) In hIS history titled ~ocial ~rigins of Dictatorship and Democracy. The boo~ serves as another illustration of explanation building in multiplecase studI,es, even though the cases are actually historical examples.

. M~ore. s book covers the transformation from agrarian to industrial societres In SIX diff~rent countries-England, France, the United States, China, Japan, and IndIa-and the general explanation of the role of the upper classes and th~ p~asantry is a basic theme that emerges. The explanation represented a significant contribution to the field of history.

operational te~s. However, the eventual explanation is likely to be a result of a senes of rterations:

• M~ing an i~itial theoretical statement or an initial proposition about policy or SOCIal behavior

121

122

CASE STUDY RESEARCH

• Comparing the findings of an initial case against such a statement or proposition

• Revising the statement or proposition

• Comparing other details of the case against the revision

• Comparing the revision to the facts of a second, third, or more cases

• Repeating this process as many times as is needed

In this sense, the final explanation may not have been fully stipulated at the beginning of a study and therefore differs from the pattern-matching approaches previously described. Rather, the case study evidence is examined, theoretical positions are revised, and the evidence is examined once again from a new perspective, in this iterative mode.

The gradual building of an explanation is similar to the process of refining a set of ideas, in which an important aspect is again to entertain other plausible or rival explanations. As before, the objective is to show how these explanations cannot be built, given the actual set of case study events. If this approach is applied to multiple-case studies (as in BOX 27), the result of the explanation-building process also may lead to starting a cross-case analysis, not simply an analysis of each individual case.

Potential problems in explanation building. You should be forewarned that this approach to case study analysis is fraught with dangers. Much analytic insight is demanded of the explanation builder. As the iterative process progresses, for instance, an investigator may slowly begin to drift away from the original topic of interest. Constant reference to the original purpose of the inquiry and the possible alternative explanations may help to reduce this potential problem. Other safeguards have already been covered by Chapters 3 and 4-that is, the use of a case study protocol (indicating what data were to be collected), the establishment of a case study database for each case (formally storing the entire array of data that were collected, available for inspection by a third party), and the following of a chain of evidence.

Time-Series Analysis

A third analytic technique is to conduct a time-series analysis, directly analogous to the time-series analysis conducted in experiments and quasi-experiments. Such analysis can follow many intricate patterns, which have been the subject of several major textbooks in experimental and clinical psychology with single cases (e.g., see Kratochwill, 1978); the interested reader is referred to such works for further detailed guidance. The more intricate and precise the pattern, the more that the time-series analysis also will lay a firm foundation for the conclusions of the case study.

ANALYZING CASE STUDY EVIDENCE

123

BOX 27

Empirical Cases With a Theoretical "Case"

A common way of using case studies is to present the empirical evidence from a set of cases to test a theoretical "case." For Kelling and Coles (1997), the theoretical case was that a successful police strategy-helping communities to claim control over public spaces through proactive, zero-tolerance prosecutions of disorderly and related conduct ("taking back the streets")emerged in the 1980s, resulting in substantial and subsequent reductions in more serious crimes. The authors argue their "case" by presenting evidence from specific case studies of New York City, Baltimore, San Francisco, and Seattle.

Especially relevant to case studies is an intriguing methodological analysis of qualitative research by Louise Kidder (1981), who demonstrated that certain types of participant-observer studies followed time-series designs, unbeknownst to the original investigators. For example, one study was concerned with the course of events that led to marijuana use, the hypothesis being that a sequence or "time series" of at least three conditions was necessary (Becker, 1963): initially smoking marijuana, later feeling its effects, and subsequently enjoying those effects. If a person experienced only one or two of these three steps but not all three, the hypothesis was that regular marijuana use would not follow. This type of postanalysis, on Kidder's part, needs to be repeated in the future to help reveal such implicit analytic techniques.

Simple time series. Compared to the more general pattern-matching analysis, a time-series design can be much simpler in one sense: In time series, there may only be a single dependent or independent variable. In these circumstances, when a large number of data points are relevant and available, statistical tests can even be used to analyze the data (see Kratochwill, 1978).

However, the pattern can be more complicated in another sense because the appropriate starting or ending points for this single variable may not be clear. Despite this problem, the ability to trace changes over time is a major strength of case studies-which are not limited to cross-sectional or static assessments of a particular situation. If the events over time have been traced in detail and with precision, some type of time-series analysis

124

CASE STUDY RESEARCH

always may be possible, even if the case study analysis involves some other techniques as well.

The essential logic underlying a time-series design is the match between a trend of data points compared to (a) a theoretically significant trend specified before the onset of the investigation versus (b) some rival trend, also specified earlier, versus (c) any other trend based on some artifact or threat to internal validity. Within the same single-case study, for instance, two different patterns of events may have been hypothesized over time. This is what Campbell (1969) did in his now-famous study of the Connecticut speed limit law (see also Chapter 2, Figure 2.1). One time-series pattern was based on the proposition that the new law (an "interruption" in the time series) had substantially reduced the number offatalities, whereas the other time-series pattern was based on the proposition that no such effect had occurred. The examination of the actual data points-that is, the annual number of fatalities over a period of years-was then made to determine which of the proposed time series best matched the empirical evidence. Such comparison of "interrupted time series" within the same case can be applied to many different settings.

In doing a multiple-case study, the same logic can be used, with different time-series patterns postulated for different cases. For instance, a case study about economic development in cities may have postulated the reasons that manufacturing-based cities would have more negative employment trends than those of service-based cities. The pertinent outcome data might have consisted of annual employment figures over a limited period of time, such as 10 years. In the manufacturing-based cities, the data might have been examined for a declining employment trend, whereas in the service-based cities, they might have been examined for a. rising employment trend. Similar analyses can be imagined with regard to the examination of crime trends over time within individual cities, changes in housing markets, trends in annual student achievement or test scores, and many other indicators.

Complex time series. The time-series designs can be more complex when the trends within a given case are postulated to be more complex. One can postulate, for instance, not merely rising or declining trends but some rise followed by some decline within the same case. This type of mixed pattern, across time, would be the beginning of a more complex time series. As always, the strength of the case study strategy would not merely be in' assessing this type of time series but also in having developed a rich ...•. explanation for the complex pattern of outcomes and in comparing the . explanation with the outcomes.

ANAL YZING CASE STUDY EVIDENCE

125

Even greater complexities arise in those instances in which a multiple set of variables-not just a single one-are relevant to a case study and in which each variable is predicted to have a different pattern over time. A s~dy of neighborhood change often assumes this characteristic. Typical neighborhood change theories, for instance, suggest that different time lags exist in the turnover rates among (a) the residential population, (b) commercial vendors and merchants, (c) local service institutions such as churches and public services, and (d) the housing stock (e.g., Yin, 1982a). When a neighborhood is undergoing racial change, upgrading, or other types of common transitions, all of these turnover rates may have to be studied over a 10- or 20-year period. The resulting curves, according to neighborhood change theories, will vary in predictable ways. For example, certain population changes (such as a subtle shift from small to larger families) are said to be followed first by certain changes in municipal services (such as school enrollment, as well as increases in the need for street services) but only later by turnover in commercial stores; furthermore, the types of churches may not change at all throughout this process.

Such a study frequently requires the collection of neighborhood indicators that in themselves are difficult to obtain, much less to analyze. However, if adequate time and effort have been set aside to conduct the necessary data collection and analysis, the result may be a compelling analysis-as in one study in which an "interrupted time-series design" was used to examine the long-term community effects of natural hazards. In this latter study, extensive data collection efforts were made in each of four communities, just to obtain the needed time-series data; the multiple-case results are described in BOX 28.

In general, although a more complex time-series creates greater problems for data collection, it also leads to a more elaborate trend (or set of trends) that can strengthen the analysis. Any match of a predicted with an actual time series, when both are complex, will produce strong evidence for an initial theoretical proposition.

Chronologies. The compiling of chronological events is a frequent techin case studies and may be considered a special form of time-series .•.. analysis. The chronological sequence again focuses directly on the major 'strength of case studies cited earlier-that case studies allow you to trace over time.

;, Yo~ ~hould ~ot think of the arraying of events into a chronology as a descriptive device only. The procedure can have an important analytic ;·ptllJX)se-to investigate presumed causal events-because the basic sequence of a cause and its effect cannot be temporally inverted. Moreover, the

126

CASE STUDY RESEARCH

BOX 28

Case Studies Using Complex Time-Series Analyses

A natural disaster-such as a hurricane, tornado, or flood---c~ be considered a major disruptive event for a community. Sales and business patterns, crimes, and other population trends might all be expected to change

as a result of such a disaster. .

Paul Friesema and his colleagues (1979) studied such changes m four communities that had suffered from major disasters: Yuba City, California, 1955; Galveston, Texas, 1961; Conway, Arkansas, 1965; and Topeka, Kansas 1966. In each of these case studies, the investigators collecte~ extensive time-series data for various economic and social indicators. Their analysis showed that the disastrous event, though having a s~ort-term eff~ct-that is, within a 12-month period-had few, ifany, long-t~rm eff~cts. Thls.analysis represents an excellent application of a complex time-series technique as the basis for a multiple-case study.

chronology is likely to cover many different types. of variable.s and not be limited to a single independent or dependent vanable. In this ~ense, ~e chronology can be richer and more insightful than general. time-series approaches. The analytic goal is to compare the chronology With .that pre- . dieted by some explanatory theory-in which the theory has specified one. :. or more of the following kinds of conditions:

• Some events must always occur before other events, with the reverse sequence : .

being impossible. ,:. :

• Some events must always be followed by other events, on a contingency basis.:'

• Some events can only follow other events after a prespecified interval of time .. : .

• Certain time periods in a case study may be marked by classes of events differ substantially from those of other time periods.

If the actual events of a case study, as carefully documented and mined by an investigator, have followed one predicte~ sequence of and not those of a compelling rival sequence, the smgle-case study again become the initial basis for causal inferences. Co~parison to cases, as well as the explicit consideration of threats to mternal will further strengthen this inference.

Summary conditions for time-series analysis. Wh~te~er ~he "'LlIf1w ......... , nature of the time series, the important case study objective IS to examme:

some relevant "how" and "why" questions about the relationship of events over time, not merely to observe the time trends alone. An interruption in a time series will be the occasion for postulating potential causal relationships; Similarly, a chronological sequence should contain causal postulates.

On those occasions when the use of time-series analysis is relevant to a case study, an essential feature is to identify the specific indicator(s) to be traced Over time, as well as the specific time intervals to be covered and the preswned temporal relationships among multiple events, stated by you prior to collecting the actual data. Only as a result of such prior specification are the relevant data likely to be collected in the first place, much less analyzed properly.

In contrast, if a study is limited to the analysis of time trends alone, as in a descriptive mode in which causal inferences are unimportant, a non--case study strategy is probably more relevant-for example, the economic analysis of consumer price trends over time.

Note, too, that without any hypotheses or causal propositions, chronologies become chronicles-valuable descriptive renditions of events but having no focus on causal inferences.

ANAL YZING CASE STUDY EVIDENCE

127

This fourth technique has become increasingly useful in recent years, especially in doing case study evaluations. The logic model deliberately stipulates a complex chain of events over time. The events are staged in repeated cause-effect-cause-effect patterns, whereby a dependent variable (event) at an earlier stage becomes the independent variable (causal event) for the next stage (peterson & Bickman, 1992; Rog & Huebner, 1992). The complexity arises from the fact that multiple stages may exist over an extended period of time.

• The use of logic models as an analytic technique consists of matching • empirically observed events to theoretically predicted events. Concep... tually, you therefore may consider the logic model technique to be another ••. of pattern matching. However, because of their sequential stages, logic models deserve to be distinguished as a separate analytic technique

. pattern matching .

-: . Joseph Wholey (1979) was at the forefront in developing logic models as

analytic technique. He first promoted the idea of a "program" logic tracing events when a public program intervention was intended to t;iDrOCllJCe a certain outcome or sequence of outcomes. The intervention could produce activities with their own immediate outcomes; these

::IWUII;:1UliiLt: outcomes could, in tum, produce some intermediate outcomes; in turn, the intennediate outcomes were supposed to produce final or -ummate outcomes.

128

CASE STUDY RESEARCH

To illustrate Wholey's (1979) framework with a hypothetical example, a school intervention could have been organized to improve students' performance in high-stakes accountability tests now prevalent in K-12 education. The hypothetical intervention might have led to a new set of classroom activities during an extra hour in the school day. These activities would have provided time for students to work with their parents on joint exercises (immediate outcome). The result of this immediate outcome would have been a report of increased understanding and satisfaction with the educational process on the part of students, parents, and teachers (intermediate outcome). Eventually, the exercises and the satisfaction would have led to increased learning of certain concepts by students (ultimate outcome).

In this example, the case study analysis would organize the empirical data to support (or to challenge) this logic model. Going beyond Wholey's (1979) approach and using the strategy of rival explanations espoused earlier in this book, the analysis also would entertain rival chains of events, as well as the potential importance of spurious external events. If the data were supportive of the initial chain, and no rivals could be substantiated, the analysis could claim a causal effect between the initial school intervention and the later increased learning. Alternatively, the conclusion might be reached that the specified series of events was illogical-for instance, that the school intervention had involved students at a different grade level than whose learning had been assessed.

The program logic model strategy can be used in a variety of circumstances, not just those in which a public policy intervention has occurred.' .. A key ingredient is the claimed existence of repeated cause-and-effect sequences of events, all linked together. The more complex the link, the more definitively the case study data can be analyzed to determine whether: a pattern match has been made with these events over time. Four types of logic models are discussed next. They mainly vary according to the unit of: analysis that might be relevant to your case study.

Individual-level logic model. The first type assumes that your case is about an individual person, with Figure 5.2 depicting the ... 0' .. """" .... course of events for a hypothetical youth. The events flow across a of boxes and arrows reading from left to right in the figure. It suggests

the youth may be at risk for becoming a member of a gang, may become involved in gang violence and drugs, and later may even be

of a gang-related criminal offense. Distinctive about this logic model the series of I 1 numbers associated with the various arrows in the Each of the 11 represents an opportunity, through some type of

IJI
o~ '0 ogt
1jca. t:1JI t:~
o ~ III C
lLZS! lLc! IIIIJI
I1.c
~ iC! ~i 111=
iiI o 0 ~!
j> glJl
0 fD~
Z
t @f
et 8
fiE
.... @ S-,S
C~ l
WC!/)
... -"-1-1 @
~@ --, ~ I
® t+
I"

IJI i
C @ ~!I
<'II ~~ .!'&il
<:) I"=--- E'gc E~c
~ J!l2 E> II
Ci (IIQ 0-8 g OI~
-, u~ UC
III
@t @+ G' C!

III
1I,",,c
E~~
J~>
ef
E
GI III ]a C Q) ~ 'iij .!!1
jotl.!:!c g$ 0·- <.>
[ij . e> ;e~ ~
.. 'iii ~ ,S! .~~cn~.!!l,j'l'C _
I~s~ 'I:: '0 ._ "C :5 e ~ c: ~
~ ::J a 0::' <11-.- 0::
'0'1:: ~ e>miij E[ij~ .g
~ ~8 CD 0 'O"SCl)'- >.~
> g CI. 0> ~ 'is. a. > .a _ .9 Z
8._ g.~'6 ~ 8.~a~&.1!!
. . . . . . . 129

130

CASE STUDY RESEARCH

intervention (e.g., community or public program), to prevent an individual youth from continuing on the flow. For instance, conununity development programs (number 1) might improve the social and economic conditions to reduce the youth's chances of becoming at risk in the first place. How a particular youth might have encountered and dealt with any or all of the 11 possible interventions might be the subject of a case study, with Figure 5.2 helping you to define the relevant data and their analysis.

Firm or organizational-level logic model. Logic models also can trace events taking place in an individual organization, such as a manufacturing firm. Figure 5.3 shows how changes in a firm (boxes 5 and 6) are claimed to lead to improved manufacturing (box 8) and eventually to improved business performance (boxes 10 and 11). The flow of boxes also reflects a hypothesis-that the initial changes were the result of external brokerage and technical assistance services (boxes 2 and 3). Given this hypothesis, the logic model therefore also contains rival or competing explanations (boxes 12 and 13). The data analysis for this case study would then consist of tracing the actual events over time, at a minimum giving close attention to their chronological sequence. The data collection also should have tried to identify ways in which the boxes were actually linked in real life, thereby corroborating the layout of the arrows connecting the boxes.

i ..

An alternative configuration for an organizational-level logic model.

Graphically, nearly all logic models construe a linear sequence (e.g., reading from left to right or from top to bottom). In real life, however, events can be . more dynamic, not necessarily progressing linearly. One such set of events might occur in relation to the "reforming" or "transformation" of an organization. For instance, business firms may undergo many significant operational changes, and the business's mission and culture (and even name) also may change. The significance of these changes warrants the notion that the entire business has been transformed (COSMOS, 2000).2 Similarly, schools. or school systems can sufficiently alter their way of doing business that . "systemic reform" is said to be occurring. In fact, major public initiatives deliberately aim at improving schools by encouraging the reform of entire school systems (i.e., school districts). However, neither the transformation nor reform processes are linear in at least two ways. First, changes may reverse course and not just progress in one direction. Second, the completed transformation or systemic reform is not necessarily an end point implied by the linear logic model (i.e., the final box in the model); continued transforming and reforming may be ongoing processes even over the long haul.

130

~ .----
~ lr~ il ~ i .. ~ H~ ~l
tJ!
)!j., CJ r!t ~ ~ ~ ~~Ij! ::~~ hi
8 -u PJ ~~ ..
~ ,,~ r to .. l:1
~ 0j~' ~~ .... ~ 0 ~ }£ ft ti f~i
!Il~ ~c G !Il ~ G !Ill'
....
~
~~_"" ~ ... ..
....- I
t-., E ~f i "iii •. ! ~~~
~ ~
a~ t-- j= .. h ME~J~!jlf ..
l':" J~l3~t ., ~ ~~r h
Il.,C () ~
t • .s
$
.., I
·s
'" H"i-~~~i ! ~ 'f-~IU~ .. 1
t§ ~~ if'l r: ~£~ ·~t ij
t ~ I!'
~ a.
() l3
{'E i ~'Jfi ii
II: i
.., .. j
810.. ~ d!
..... .-
.- I
"lUiun f~" j! j
-t ;5:81 :l!!l3 ..
I- .. 8:aE~ ~l~ ill
t_.
.. I
s ~ ~'iI t-' ~ B .. t~ ~
C\I ;.~,. . '~ ~ ~ .g'.~. i?~
E!Olfii .. 8~Jl :~~ ......
<0 .S ~
----' .9

131

132

CASE STUDY RESEARCH

Ba!I~line,

An Unaligned System

Illusttillive Movement toward Reforming

+

+

Mot< R~foF"'~

2 LCl'~ 1 R<fo __

Illustrative

liIiItKIJc#t.ool

Figure 5.4 Hypothetical States of an Education (K-12) Reforming System SOURCE: COSMOS Corporation (2001).

Figure 5.4 presents an alternatively configured logic ~ode1,. r~~ec~ing these conditions. This logic model (a) tracks all of the main activities m a school system (the initials are decoded in the key to the figure), (b) over four periods of time (each time interval might represent a 2- or 3-year

t

ANAL YZING CASE STUDY EVIDENCE

133

period of time). Systemic reform occurs when all of the activities are aligned and work together, and this occurs at 11 in Figure 5.4. At later stages, however, the reform may regress, represented by t4, and the logic model does not assume that the vacillations will even end at (.,. As a further feature of the logic model, the entire circle at each stage can be positioned higher or lower, representing the level of student performance-the hypothesis being that systemic reform will be associated with the highest performance. The pennants in the middle of the circle indicate the number of schools or classrooms implementing the desired reform practices, and this number also can vacillate. Finally, the logic model contains a "metric," whereby the positioning of the activities or the height of the circle can be defined as a result of analyzing actual data.

Program-level logic model. Returning to the more conventional linear model, Figure 5.5 contains a fourth and final example. Here, the model depicts the rationale underlying a major federal program, aimed at reducing the incidence of HIV/AIDS by supporting community planning and prevention initiatives. The program provides funds as well as technical assistance to 65 state and local health departments across the United States. The model was used to organize and analyze data from 14 case studies (COSMOS, 1999), including the collection of data on rival explanations, whose potential role also is shown in the model.

Summary. Using logic models represents a fourth technique for analyzing case study data. Four types of logic models, applying to different units of analysis and situations, have been presented. You should define your logic model prior to collecting data and then "test" the model by seeing how well the data support it.'

Cross-Case Synthesis

A fifth technique applies specifically to the analysis of multiple cases (the previous four techniques can be used with either single- or multiple-case studies). The technique is especially relevant if, as advised in Chapter 2, a case study consists of at least two cases. The analysis is likely to be easier and the findings likely to be more robust than having only a single case. BOX 29 presents an excellent example of the important research and research topics that can be addressed by having a "two-case" case study. Again, having more than two cases could strengthen the findings even further.

Cross-case syntheses can be performed whether the individual case studies have previously been conducted as independent research studies

134

CASE STUDY RESEARCH

Information about need. and results of TA

Community I'IInnl"ll i3roupo(cPGf

n Co-chalr.s

fI Members

Immediate Outcomes

Intermediate Outcomes

Unimata Outcomes

1A Mater1a1s and Workshops

Figure 5.5 Improving Community Planning for HrV/AIDS Prevention

SOURCE: COSMOS Corporation (1999).

(authored by different persons) or as a predesigned part of the same study. In either situation, the technique treats each individual case study as a separate study. In this way, the technique does not differ from other research syntheses-aggregating findings across a series of individual studies (see BOX 30). If there are large numbers of individual case studies available, the synthesis can incorporate quantitative techniques common to other research syntheses (e.g., Cooper & Hedges, 1994) or meta-analyses (e.g., Lipsey, 1992). However, if only a modest number of case studies is available, alternative tactics are needed.

One possibility is to create word tables that display the data from the individual cases according to some uniform framework. Figure 5.6 has an example of such a word table, capturing the findings from 14 case studies of organizational centers, with each center having an organizational partner (COSMOS, 1998). Of the 14 centers, 7 had received programmatic support and were considered intervention centers; the other 7 were selected as comparison centers. For both types of centers, data were collected about the center's ability to colocate (e.g., share facilities) with its partnering organization-this being only one of several outcomes of interest in the original study.

The overall pattern in the word table led to the conclusion that the intervention and comparison centers did not differ with regard to this particular

ANAL YZING CASE STUDY EVIDENCE

135

BOX 29

Using a "Two-Case" Case Study to Test a Policy-Oriented Theory

The international marketplace of the 1970s and 1980s was marked by Japan's prominence. Much of its strength was attributable to the role of centralized planning and support by a special governmental ministryconsidered by many to be an unfair competitive edge, compared to the policies in other countries. For instance, the United States was considered to have no counterpart support structures. Gregory Hooks's (1990) excellent case study points to a counterexample, frequently ignored by advocates: the role of the U.S. Department of Defense in implementing an industrial planning policy within defense-related industries.

Hooks provides quantitative data on two cases-the aeronautics industry and the microelectronics industry (the forerunner to the entire computer chip market and its technologies, such as the personal computer). One industry (aeronautics) has traditionally been known to be dependent on support from the federal government, but the other has not. In both cases, Hooks's evidence shows how the defense department supported the critical early development of these industries through financial support, the support of R&D, and the creation of an initial customer base for the industry's products. The existence of both cases, and not the aeronautics industry alone, makes the author's entire argument powerful and persuasive.

outcome. Additional word tables, reflecting other processes and outcomes of interest, were examined in the same way. The analysis of the entire collection of word tables enabled the study to draw cross-case conclusions about the intervention centers and their outcomes.

Complementary word tables can go beyond the single features of a case and array a whole set of features on a case-by-case basis. Now, the analysis can start to probe whether different groups of cases appear to share some similarity and deserve to be considered instances of the same "type" of general case. Such an observation can further lead to analyzing whether the arrayed case studies reflect subgroups or categories of general cases-raising the possibility of a typology of individual cases that can be highly insightful. This illustrative example shows how cross-case syntheses can become more complex and cover broader issues than simply analyzing single features.

An important caveat in conducting this kind of cross-case synthesis is that the examination of word tables for cross-case patterns will rely

136

CASE STUDY RESEARCH

BOX 30

Eleven Program Evaluations and a Cress-r'Case" Analysis

Dennis Rosenbaum (1986) collected II program evaluations as separate chapters in an edited book. The II evaluations had. been conducted by different investigators, had used a variety of methods, and were not case studies. Each evaluation was about a different community crime prevention intervention, and some presented ample quantitative evidence and employed statistical analyses. The evaluations were deliberately selected because nearly all had. shown positive results. A cross-vcase" analysis was conducted by the present author (Yin, 1986), treating each evaluation as ifit were a separate "case." The analysis dissected and arrayed the evidence from the II evaluations in the form of word tables. Generalizations about successful community crime prevention, independent of any specific intervention, were then derived by using a replication logic, given that all of the evaluations had shown positive results.

Centers

Characteristics of Co-Location

Intervention Centers;

1 Partnering staff are located in the same facility as Center 1 and follow Center

1 's policies that were in place prior to the partnership. Center 1 receives $25,000 annually from the partnership budget for software and peripherals, and communication and supplies.

2 As a business unit of Center 2, the partnerlng S1aff are housed within Center 2's

offices. Center 2's parent organization contributes $2,500 for space and $23,375 for Indirect expenses annually to the partnership budget.

3 Five partnership offices are co-located with Center 3's staff.

4 Center 4 and its pertner share office space.

S Center 5 staff and the partnering staff are located in the same building, but do

not share office space.

6 The two organizations are not co-located.

7 Partnering staff are located in Center 7's offices.

Comparison Centers.

8 Center 8 and its partner share office space in eight locations statewide.

9 Some sites are co-located.

10 Center 10 and its partner are not co-located.

11 The partnering and center staff share office space.

12 Center 12 and its partner's staff are located in the same building.

13 Center 13 and its partner's staff are located in the same office.

14 Center 14 shares office space With three regional partners.

Figure 5.6 Co-location oflnterorganizational Partners (14 Centers and Their Counterpart Organizations)

SOURCE: COSMOS Corporation (1998).

ANALYZING CASE STUDY EVIDENCE

137

strongly on argumentative interpretation, not numeric tallies. Chapter 2 has previously pointed out, however, that this method is directly analogous to cross-experiment interpretations, which also have no numeric properties when only a small number of experiments is available for analysis. A challenge you must be prepared to meet as a case study investigator is therefore to know how to develop strong, plausible, and fair arguments that are supported by the data.

PRESSING FOR A HIGH-QUALITY ANALYSIS

No matter what specific analytic strategy or techniques have been chosen, you must do everything to make sure that your analysis is of the highest quality. At least four principles underlie all good social science research (Yin, 1994a, 1997,1999) and require your attention.

First, your analysis should show that you attended to all the evidence.

Your analytic strategies, including the development of rival hypotheses, must be exhaustive. Your analysis should show how it sought as much relevant evidence as was available, and your interpretations should account for all of this evidence and leave no loose ends. Without achieving this standard, your analysis may be vulnerable to alternative interpretations based on the evidence that you had (inadvertently) ignored.

Second, your analysis should address, if possible, all major rival interpretations, If someone else has an alternative explanation for one or more of your findings, make this alternative into a rival. Is there evidence to address this rival? If so, what are the results? If not, should the rival be restated as a loose end to be investigated in future studies?

Third, your analysis should address the most significant aspect of your case study. Whether it is a single- or multiple-case study, you will have demonstrated your best analytic skills if the analysis focuses on the most important issue (preferably defined at the outset of the case study). In addition, by avoiding a detour to a lesser issue, your analysis will be less vulnerable to the possibility that the main issue was being avoided because of possibly negative findings.

Fourth, you should use your own prior, expert knowledge in your case study. The strong preference here is for you to demonstrate awareness of current thinking and discourse about the case study topic. If you know your subject matter as a result of your own previous investigations and publications, so much the better.

The case study in BOX 31 was done by a research team with academic credentials and strong practical experience. By taking several steps, the

138

CASE STUDY RESEARCH

BOX 31

Analytic Quality in a Multiple-Case Study of International Trade Competition

The quality of a case study analysis is not dependent solely on the techniques used, although they are important. Equally important is that the investigator demonstrate expertise in carrying out the analysis. This expertise was reflected in Magaziner and Patinkin's (1989) book, The Silent War:

Inside the Global Business Battles Shaping America s Future.

The authors organized their nine cases in excellent fashion. Across cases, major themes regarding America's competitive advantages (and disadvantages) were covered in a replication design. Within each case, the authors provided extensive interview and other documentation, showing the sources of their findings. (To keep the narrative reading smoothly, much of the data-in word tables, notes, and quantitative tabulations-were relegated to footnotes and appendices.) In addition, the authors showed that they had extensive personal exposure to the issues being studied as a result of nurnerous domestic and overseas visits.

Technically, a more explicit methodological section might have been helpful. However, the careful and detailed work, even in the absence of such a section, helps to illustrate what all investigators should strive to achieve.

authors demonstrate a care of empirical investigation whose spirit is worth considering in all case studies. The care is reflected in the presentation of the cases themselves, not by the existence of a stringent methodology section. If you can emulate these and other strategies in your analysis, your case study analysis also will be given appropriate respect and recognition.

SUMMARY

This chapter has presented several ways of analyzing case studies. First, the potential analytic difficulties can be reduced if you have a general strategy for analyzing the data-whether such a strategy is based on theoretical propositions, rival explanations, or descriptive frameworks. In the absence of such strategies, you may have to "play with the data" in a preliminary sense, as a prelude to developing a systematic sense of what is worth analyzing and how it should be analyzed.

ANALYZING CASE STUDY EVIDENCE

139

Second, given a general strategy, several specific analytic techniques are relevant. Of these, five (pattern matching, explanation building, time-series analysis, logic models, and cross-case syntheses) can be effective in laying the groundwork for high-quality case studies. For all five, a similar replication logic should be applied if a study involves multiple cases (thereby gaining external validity). Comparisons to rival propositions and threats to internal validity also should be made within each individual case.

None of these techniques is easy to use. None can be applied mechanically, following any simple cookbook procedure. Not surprisingly, case study analysis is the most difficult stage of doing case studies, and novice investigators are especially likely to have a troublesome experience. Again, one recommendation is to begin with a simple and straightforward case study (or, more preferably, a "two-case" design), even if the research questions are not as sophisticated or innovative as might be desired. Experience gained in completing such straightforward case studies will lead to the ability to tackle more difficult topics in subsequent case studies.

EXERCISES

1. Creating a general analytic strategy. Assume you have collected your case study data but still do not have an analytic strategy. Ask yourself how you might organize your case into separate chapters or sections. Use substantive titles and headings (e.g., instead of "Introduction," make the title say what the introduction is about, even if more than a few words are needed). Try different sequences of titles, noting how such differences might also alter your analytic strategy. Now choose one sequence and start sorting your data into the designated chapters or sections. You should be on your way to analyzing your case study.

2. Analyzing the analytic process. Select one of the case studies described in the BOXES of this book. Find one of the chapters (usually in the middle of the study) in which evidence is presented but conclusions also are being made. Describe how this linkage-from cited evidence to conclusions-e-occurs. Are data displayed in tables or other formats? Are comparisons being made?

3. Merging quantitative and qualitative data. Name some topic within a case study you might be conducting for which both qualitative and quantitative data might be relevant. Identify the two types of data, assume they have been collected successfully, and discuss the ways in which they would be combined or compared. What is the value of having each type of data?

4. Matching patterns. Name a case study that used a pattern-matching technique in its analysis. What peculiar advantages and disadvantages does it have to offer? How can the technique produce a compelling analysis even when applied to only a single case?

140

CASE STUDY RESEARCH

5. Constructing an explanation. Identify some observable changes that have been occurring in your neighborhood (or the neighborhood around your campus). Develop an explanation for these changes and indicate the critical set of evidence you would collect to support or challenge this explanation. If such evidence were available, would your explanation be complete? Compelling? Useful for investigating similar changes in another neighborhood?

6. Analyzing time-series trends. Identity a simple time series-for example, the number of students enrolled in your university for each of the past 20 years. How would you compare one period of time with another within the 20-year period? If the university admissions policies had changed during this time, how would you compare the effects of such policies? How might this analysis be considered part of a broader case study of your university?

NOTES

1. See Chapters 4, 5, 8, and 10. The first two of these chapters summarize the cases by deliberately focusing on the rival explanationts). The second two of these chapters contain complete case studies, showing how rival explanations were used as part of the case study analysis.

2. Chapter 10 of the companion book presents the full cross-case analysis of a study of transformed firms.

3. The companion book has many additional examples of the use of logic models in specific case studies. See Chapters 6, 8, and 10.

6

Reporting Case Studies

Reporting a case study means bringing its results and findings to closure. Regardless of the form of the report, similar steps underlie the case study composition: identifying the audience for the report, developing the compositional structure, and following certain procedures (such as having the report reviewed by informed persons who have been the subject of the case study), Once composed, the case study may be finlshed=or it may be joined with data collected through other methods as part of a broader, multimethod study. Such studies can be advantageous and represent a further challenge in doing case study research.

Whether serving as a finished case study or as part of a multimethod study, reporting case study results also is one of the most challenging aspects of doing case studies. The best general advice is to compose portions of the case stUdy early (e.g., the bibliography and t~~.!f1~!"~odOIQgy_$E;!(:Jlc;m), rather than waiting unUlthe·eri<fof the data analysis process. As for compositional structures, six alternatives are suggested: linear-analytic, comparative, chronological, theory-building, "suspense," and unsequenced structures. A final plea is to worry about producing high-quality and not just run-of-the-mill case studies.

As a general rule, the compositional phase puts the greatest demands on a case study investigator. The case study report does not follow any stereotypic form, such as a journal article in psychology. Moreover, the report need not be in written form only; you may have the opportunity to make an oral presentation about your case study. Because of this uncertain nature, researchers who do not like to compose may want to question their interest in doing case studies in the first place.

Of course, most investigators can eventually learn to compose easily and well, and inexperience in composing should not be a deterrent to doing case studies. However, much practice will be needed. Furthermore, to do good case studies, you should want to become good at composing-and not merely put up with it. One indicator of whether you are likely to succeed at this phase of the craft is whether term papers were easy or difficult to do in

141