THE EFFECT OF LABELS ON ANALYSIS

JEFFREY R. WELGAN

A Thesis Submitted to the Faculty of Mercyhurst College In Partial Fulfillment of the Requirements for The Degree of MASTER OF SCIENCE IN APPLIED INTELLIGENCE

DEPARTMENT OF INTELLIGENCE STUDIES MERCYHURST COLLEGE ERIE, PENNSLYVANIA JULY 2009

DEPARTMENT OF INTELLIGENCE STUDIES MERCYHURST COLLEGE ERIE, PENNSLYVANIA

THE EFFECT OF LABELS ON INTELLIGENCE ANALYSIS A Thesis Submitted to the Faculty of Mercyhurst College In Partial Fulfillment of the Requirements for The Degree of MASTER OF SCIENCE IN APPLIED INTELLIGENCE

Submitted By: JEFFREY R. WELGAN

Certificate of Approval:

___________________________________ Kristan J. Wheaton Associate Professor Department of Intelligence Studies

___________________________________ James G. Breckenridge Assistant Professor Department of Intelligence Studies

___________________________________ Phillip J. Belfiore Vice President Office of Academic Affairs July 2009

Copyright © 2009 by Jeffrey R. Welgan All rights reserved.

iii

DEDICATION This thesis is dedicated to the intelligence community, whose hard work often goes unnoticed.

iv

ACKNOWLEDGEMENTS I would first like to thank my wife, Chrissy, for the inspiration for this thesis. Without her constant support and encouragement, this thesis would not have been possible. I would also like to extend thanks to the professors of the Intelligence Studies program at Mercyhurst College for their guidance and support, and for pushing me to be a better analyst.

v

ABSTRACT OF THE THESIS

The Effect of Labels on Analysis By Jeffrey R. Welgan Master of Science in Applied Intelligence Mercyhurst College, 2009 Professor Kristan Wheaton, Chair

This thesis examines the notion of cognitive bias as it relates to intelligence analysis. Specifically, it reports the findings of an online experiment in which analysts in the field were given a mock intelligence report featuring one of five labels for hostile groups— either ‗group,‘ ‗insurgent,‘ ‗rebel,‘ ‗militia,‘ or ‗terrorist‘—and then the analysts were asked to complete a survey of questions regarding their impressions of the likelihood, severity, and sophistication of an attack if one were to occur. The overall findings resulted in no statistical significance, though statistical significance was discovered to be present for certain labels and certain questions. What this means for the intelligence community is that more research is needed before conclusive results can be determined. If cognitive bias plays a role in intelligence analysis, it could have

implications for the way in which intelligence reporting is done. Regardless, intelligence analysts are trained to be precise, accurate, and as unbiased as possible, and this is a goal of which analysts must continuously be conscious and for which analysts must constantly strive.

vi

TABLE OF CONTENTS COPYRIGHT PAGE DEDICATION ACKNOWLEDGEMENTS ABSTRACT TABLE OF CONTENTS LIST OF TABLES LIST OF FIGURES CHAPTER 1. INTRODUCTION 2. LITERATURE REVIEW 2a. Understanding The Cognitive Process Framing and Anchoring Heuristics and Cognitive Biases 2b. Understanding the Labels The Etymology of the Labels The ―Rebel‖ Label The ―Militia‖ Label The ―Insurgent‖ Label The ―Terrorist‖ Label Comparing the Labels 3. METHODOLOGY 3a. Research Design 3b. Sample, Population, and Participants 3c. Data Collection Instruments, Variables, & Materials 3d. Data Analysis Procedure 4. FINDINGS 4a. Questions 4&5: Overall Likelihood of an Attack 4b. Questions 6&7: Likelihood of a Successful Attack 4c. Questions 8&9: Likelihood of a Sophisticated Attack 4d. Questions 10&11: Severity of an Attack 4e. Question 12: Effect of an Overall Increase in Activity
vii

iii iv v vi vii ix x

1 3 3 4 9 13 13 17 20 23 27 32 36 36 37 40 43 46 47 49 51 53 55

4f. Question 13: Effect of Imagery Intelligence Indicating Increased Activity Near a Training Base or Command Post 4g. Question 14: Effect of Local Newspaper Reporting an Attack Will Not Occur 4h. Question 15: Effect of Communications Intelligence Reporting Increased Levels of Anti-Government Rhetoric 4i. Question 16: Effect of a Long History of Social Strife and Discontentment 4j. Question 17: Effect of the Region Being a Hot-Bed of Differing Political and Religious Ideologies, Widespread Poverty, and Poor Health Conditions 4k. Questions 18&19: Ranking of the Labels Based on Their Likelihood to Engage in Violence and Their Likelihood to Execute Severe Attacks 5. CONCLUSION 5a. Trends in Key Findings 5b. Research Recommendations 5c. Moving Forward REFERENCES APPENDICES Appendix A: Participation Consent Form Appendix B: Participation Debriefing Statement Appendix C: Institutional Review Board (IRB) Approval Appendix D: Scenario 1: Group Activity (Control Group) Appendix E: Scenario 2: Rebel Activity Appendix F: Scenario 3: Militia Activity Appendix G: Scenario 4: Insurgent Activity Appendix H: Scenario 5: Terrorist Activity Appendix I: The Effect of Labels on Analysis Survey STATISTICAL ANNEX

58 59

60 62

63

65 66 66 71 73 74 79 80 81 82 83 84 85 86 87 88 92

viii

LIST OF TABLES Table 3.1: Five-group posttest only experimental design Table 4.1: Scheffe test results for question #12 (overall increase in activity) Table 4.2: Tamhane test results for question #15 (effect of COMINT) Table 5.1: Positive, neutral, and negative responses to questions #4-17 36 56 61 68

ix

LIST OF FIGURES Figure 3.1: Number of participants by label received Figure 3.2: Number of participants by experience level Figure 3.3: Number of participants by intelligence field Figure 4.1: Significance values among questions Figure 4.2: Label comparison – overall likelihood of an attack Figure 4.3: Analytic confidence comparison of labels for overall likelihood of an attack Figure 4.4: Group label: likelihood of a successful attack Figure 4.5: Insurgent label: likelihood of a successful attack Figure 4.6: Terrorist label: likelihood of a successful attack Figure 4.7: Comparison of labels: likelihood of a sophisticated attack Figure 4.8: Comparison of labels: likelihood of a severe attack Figure 4.9: Analytic confidence comparison of labels for likelihood of a severe attack Figure 4.10: Responses to an overall increase in group activity Figure 4.11: Responses to IMINT indications of increased activity Figure 4.12: Responses to local newspaper reporting of non-violent intentions Figure 4.13: Outlying responses for question #14 (newspaper reporting of nonviolent intentions) Figure 4.14: Responses to COMINT indications of increased anti-government rhetoric Figure 4.15: Responses to long history of strife and discontentment Figure 4.16: Responses to region differing politically, ideologically, and economically Figure 4.17: Ranking of labels by violence & severity 39 39 40 46 48 49 49 50 51 52 53 54 55 58 59 60 62 63 64 65

x

INTRODUCTION
Intelligence analysts are constantly faced with labels throughout their daily work. In the national security realm, labels, such as terrorist, rebel, militia, and insurgent, are given to groups and are continually labels of interest to analysts and their decision makers. These particular labels have significant meaning, and many individuals have a preconceived idea, or cognitive biases, regarding the kinds of actions each of these particular groups conduct. The presence of cognitive biases when analyzing reports containing these labels can have a detrimental effect on analysis; as the effectiveness of intelligence depends directly upon the effectiveness of the intelligence analyst. If an analyst comes to his or her analysis using a flawed process, or inserts bias into the analytical process, the product of that analysis is virtually certain to contain more inconsistencies and therefore have increased inaccuracy. My thesis, The Effect of Labels on Intelligence Analysis, will explore terms used in the US Intelligence community and their affect on the analysis of a report. Specifically, I will examine the following labels given to certain groups: terrorist, rebel, militia, and insurgent. To determine whether or not intelligence analysts succumb to cognitive biases when faced with these particular labels, I will conduct an experiment using theoretical intelligence reports to demonstrate whether or not the use of particular labels change the analysis of a report, ceteris paribus. For example, how different is an analyst‘s analysis when the report contains a ―rebel‖ group rather than a ―terrorist‖ group? In addition to conducting an experiment that identifies whether analysts apply cognitive biases to these labels, the labels themselves must be examined.
1

Using

definitions from the Oxford English Dictionary (OED), governmental agencies and departments, as well as definitions from experts in the fields of the respective areas, I will clearly identify the similarities and differences between terrorist, rebel, militia, and insurgent groups. I will also attempt to explain how and why these labels are applied, as well as their underlying implications. The examination of the terms we use to label is significant, particularly in light of the US global war on terrorism and for national security interests. By using the same information and only switching which label (terrorist, rebel, militia, insurgent, or group) is used, my thesis will attempt to determine how much impact, if any, each label has on analysis. In addition, I will attempt to analyze the effect differences in experience levels and intelligence backgrounds (National Security, Military, Law Enforcement, and Competitive Intelligence) has on responses. This research is important because the use, or misuse, of particular labels can directly affect US government action or inaction.

2

LITERATURE REVIEW
In an effort to make this section more organized and easier to read, I have divided material into two main sections with subsequent subsections. The two main sections are: the cognitive process, and exploring the labels. It is important to understand the factors that influence our thinking before the labels in question are thoroughly examined; therefore, the first section covers the cognitive process and contains two subsections: framing and anchoring; and, heuristics and cognitive biases. The second section focuses on the meaning of the labels and is broken up into six subsections: the etymology of the labels; the rebel label; the militia label; the insurgent label; the terrorist label; and, comparing the labels. Understanding the Cognitive Processes People construct their own version of „reality‟ on the basis of information provided by the senses, but this sensory input is mediated by the complex mental processes that determine which information is attended to, how it is organized and the meaning attributed to it.1 - Richards Heuer

Many factors play into the cognitive process when labels are presented to analysts. All forms of communication have both a communicator and a receiver between which a message is conveyed. The difficulty comes when we try to understand the meaning of the message. Oftentimes messages can be successfully interpreted, meaning that the message has been received and understood in the manner it was intended. However, messages can also be misunderstood. The distortion of messages frequently is the result of individual perceptual screens. A perceptual screen, as defined by Debra
1

Heuer, Richards., Psychology of Intelligence Analysis, (Nova Science 1st ed, 2005), 17.

3

Nelson and James Quick, is ―a window through which we interact with people that influences the quality, accuracy, and clarity of the communication.‖2 Because our

individuality is molded by a multitude of factors such as religious beliefs, social upbringing, gender, age, values, culture, experience, needs, media, etc., messages can easily be distorted from one person to another.
Framing and Anchoring

We perceive and understand the world in terms of the way we structure and organize information cognitively. When we encounter new information, we make

decisions and assumptions about how that information should be categorized and organized in our minds. In the world of psychology and decision-making, this structuring of information is oftentimes referred to as framing. A frame can be defined as ―a structure for accounting for the data and guiding the search for more data.‖3 Frames are cognitively constructed to allow an individual to process information and events; they are based on three or four anchors, or pieces of information deemed relevant to the individual.4 These frames are not constructed

randomly, but rather, based on past experiences, both personal and cultural. Schoemaker & Russo concur with this notion and expand it further by stating, "The frames we use to view the world determine what we see, locking us into certain ideas and shutting out new possibilities."5
2

Nelson, Debra L,. & Campbell, James Quick., Organizational Behavior: Foundations, Realities, & Challenges, 5th ed. (Mason, OH.: Thomson South-Western. 2006). 250. 3 Klein, G. et al, ―A Data/Frame Theory of Sensemaking.‖ In Expertise out of Context; Proceedings of the Sixth International Conference on Naturalistic Decision Making, edited by R.R. Hoffman. (Mahwah, NJ: Erlbaum, 2004). 4 Ibid. 5 Schoemaker, P & Russo, J. ―Managing Frames to Make Better Decisions.‖ In Wharton on Making Decisions, edited by S. Hoch and H. Kunreuther. (New York: John Wiley & Sons, 2001). 131-55.

4

According to noted sociologist Stanley Goffman (1974), a frame is ―a culturally relative system of rules, principles, etc. that are used to organize society and guide individual behavior.‖6 In Barlett‘s (1932) words, ―a schema is a mental representation of the structure of event descriptions (e.g. stories), usually taking the form of regularlyoccurring and culture-specific sequences of dramatic sub-events.‖7 Schemas are readymade mental outlines into which individuals may enter new information in order to understand it in terms of their personal and cultural history. Richards Heuer describes everything we think we know about something in particular as a ―mind-set‖ or ―mental model.‖8 Heuer states: A mind-set is neither good nor bad. It is unavoidable. It is, in essence, a distillation of all that analysts think they know about a subject. It forms a lens through which they perceive the world, and once formed, it resists change.9 These mind-sets are pulled from our memory and aid in processing new information that is presented to us. Mind-sets, then, are synonymous with schema, which are stored in our long-term memory (LTM) as a method for the brain to retrieve information more efficiently each time we think about a particular subject. Like Russian nesting dolls, schemas are reoccurring frames, which are built upon certain anchors. When we examine schemas cross-culturally, they differ from group to group, which explains why certain labels evoke certain analyses. For example, we see this play out in society in terms of gender roles. This is particularly true for women, and most people will have a very similar preconceived notion of ―what that woman is like‖
6 7

Goffman, E., Frame Analysis: An Essay on the Organization of Experience, (NY: Harper, 1974). Bartlett, F.C., Remembering: A Study in Experimental and Social Psychology, (Cambridge, England: Cambridge UP, 1932). 8 Heuer, Richards., Psychology of Intelligence Analysis, (Nova Science 1st ed, 2005), 18. 9 Ibid., 77.

5

when they hear the phrase ―soccer mom,‖ or ―single mother,‖ or ―working woman.‖ When a person in our society hears each of these phrases, they will most likely have a similar picture of what each of these women look like, act like, even where these women might live. In fact, these labels have become so ingrained in our minds that they can be used to denote roles played in films, and everyone who watches a film will know who the ―soccer mom‖ was when they read the ending credits. We might expect a ―working woman‖ to be wearing a suit in New York City, but we would not picture a typical ―soccer mom‖ living in the same city. In much the same way that these labels for women operate to inform our judgments, decisions, and opinions, most people in our society will come to similar preconceived notions when they hear the labels ―terrorist,‖ or ―rebel group,‖ or ―insurgent,‖ or ―militia.‖ Frequently in the field of intelligence, information does not always make sense to those who analyze it. Data does not always fall into our mental schema, or if it does fit, our mind-sets will sometimes blur our perspective to the point where we steer ourselves into a rut. Heuer states: ―If information does not fit into what people know, or think they know, they have great difficulty processing it.‖10 Information processing is most difficult when the data obtained does not fit into the frame one has constructed. When this

occurs, the analyst must attempt to make sense of the data and the frame; this is known as ―sensemaking.‖11 Klein et al describe the sensemaking process as follows: Data are the interpreted signals of events; frames are the explanatory structures that account for the data. People react to data elements by trying to find or construct a story, script, a map, or some other type of structure to account for the data. At the same time, their repertoire of
10

Heuer, Richards., Psychology of Intelligence Analysis, (Nova Science 1st ed, 2005), 36. 11 Klein, G. et al, ―A Data/Frame Theory of Sensemaking.‖ In Expertise out of Context; Proceedings of the Sixth International Conference on Naturalistic Decision Making, edited by R.R. Hoffman. (Mahwah, NJ: Erlbaum, 2004), 12.

6

frames – explanatory structures – affects which data elements they consider and how they will interpret these data. We see sensemaking as the effort to balance these two entities – data and frames.12 Sensemaking, then, will be fulfilled when feedback reveals the congruence between the data and the frame. Jenny W. Rudolph explains that people make sense of situations from cognitively-framed internal images of external realities.13 Therefore, when new information is presented to analysts, the analyst tends to seek out the data that is, in one way or another, linked to an event or schema already stored in their memory. The analyst then uses this stored memory in conjunction with the newly presented data, creating an anchor for which to construct a new frame. Additional data that fits well into this newly constructed frame then gets added to solidify the frame and ultimately guide analysis. The analysis of the situation will make sense to the analyst if the data used works harmoniously with the frame they have constructed. However, the sensemaking process can leave an analyst with a false sense of security, as sensemaking does not necessarily mean that the analysis is accurate, due to the various different frames into which data can fit. In addition to the personal frames we construct, frames can be presented to us through context. If information is presented within a particular context, the person interpreting that data will oftentimes use the context that is presented to them rather than reinterpreting or reframing the data.14 This phenomenon is known as the ―concreteness

12

Klein, G. et al, ―A Data/Frame Theory of Sensemaking.‖ In Expertise out of Context; Proceedings of the Sixth International Conference on Naturalistic Decision Making, edited by R.R. Hoffman. (Mahwah, NJ: Erlbaum, 2004), 13 Rudolph, Jenny. W. Into the big muddy and out again. Unpublished Doctoral Thesis, (Boston College, Boston, MA., 2003). 14 Soman, Dilip., ―Framing, Loss Aversion, and Mental Accounting.‖ In The Blackwell Handbook of Judgment and Decision Making edited by Derek J. Koehler and Nigel Harvey, (Malden, Ma: Blackwell Publishing, 2004), 381.

7

principle.‖15 For example, an intelligence report stating that, ―Imagery intelligence has indicated an increase of human activity near what appears to be a training base or command post of the suspected insurgents. Imagery analysts report that the insurgents appear to have small arms and other weapons‖ might cause an analyst or decision maker to take that information and use it as truth; rather than examining the information, reframing it, and questioning what an ‗increase in human activity‘ means (i.e. prior number of insurgents compared to new number), what differences are apparent between a ‗training base‘ and ‗command post,‘ and exactly what type of weapons might this suspected insurgent group have. Daniel Kahneman, respected judgment & decisionmaking psychologist and Nobel laureate, states that "decision makers are generally quite passive and therefore inclined to accept any frame that they are presented with.‖16 Because decision makers rely so heavily on the information provided to them by analysts, analysts must be cautious in both using their own personal frames as well as in relying on context to provide a set frame for them. Because of information gaps and misleading anchors, there are strategies that analysts can use to generate and evaluate hypotheses. Hypothesizing is an attempt to fillin information gaps and to strengthen the frame using judgment – Minsky (1975) also referred to this process as ―slot-filling.‖17 Once hypotheses are formed, they can be disproven or reduced to a point where our certainty in them is so small that the hypothesis

15

Slovic, P., ―From Shakespeare to Simon: Speculations and Some Evidence About Man‘s Ability to Process Information.‖ Oregon Research Institute Monograph 12, no. 2 (1972). Thaler, R. & Johnson, E., ―Gambling With the House Money and Trying to Break Even: The Effects of Prior Outcomes on Risky Choice.‖ Management Science 36 (1990): 643-60. 16 Kahneman, D. & Tversky, A. (eds.), Choices, Values and Frames. (New York, NY: Russell Sage Foundation & Cambridge University Press, 2000), ix-xvii 17 Minsky, M., ―A Framework for Representing Knowledge.‖ In The Psychology of Computer Vision, edited by P. Winston (New York, NY: McGraw-Hill, 1975), 211-277.

8

is deemed not likely. Disproving hypotheses can also help in preventing satisficing. Satisficing is the process of ―choosing the first hypothesis that appears good enough rather than carefully identifying all possible hypotheses and determining which is most consistent with the evidence.”18 Therefore, the more hypotheses an analyst can reduce or eliminate, the more precise the analysis is likely to be.
Heuristics and Cognitive Biases

While the topics of cognitive bias and heuristics were not new to the field of cognitive psychology, Richards Heuer was the first to really apply their principles to the field of intelligence. Heuer defines cognitive bias as, ―a mental error that is consistent and predictable… [and the] awareness of the bias, by itself, does not produce a more accurate perception.‖19 The number of possible cognitive biases for a given situation is staggering, and different situations will likely elicit different biases – it is partly for this reason that cognitive biases are so difficult to overcome. However, Dr. Rob Johnston, in a study aimed to identify and explain the conditions and variables that negatively affect intelligence analysis, found that the most common cognitive bias among intelligence analysts was confirmation bias.20 Confirmation bias is the tendency to pay more attention to information that confirms one‘s viewpoints or beliefs rather than information that refutes it.21,22 Johnston‘s findings confirm a 1998 study of intelligence analysts by

18 19

Heuer, Richards., Psychology of Intelligence Analysis, (Nova Science 1st ed, 2005), 47. Ibid., 121-22. 20 Johnston, Rob., ―Analytic Culture in the US Intelligence Community: An Ethnographic Study.‖ (Washington D.C.: Center for the Study of Intelligence, Central Intelligence Agency, 2005). Chapter 2. <https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-andmonographs/analytic-culture-in-the-u-s-intelligence-community/chapter_2_research.htm> 21 Johnston, Rob., ―Analytic Culture in the US Intelligence Community: An Ethnographic Study.‖ (Washington D.C.: Center for the Study of Intelligence, Central Intelligence Agency, 2005). Chapter 2. <https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-andmonographs/analytic-culture-in-the-u-s-intelligence-community/chapter_2_research.htm> sa

9

Patterson, Woods, Sarter, and Watts-Perotti which found that when initial articles in open literature were misleading, the subsequent analyses would often succumb to distortion.23 Heuristics also contribute to our biases and decision-making. In its simplest definition, a heuristic can be described as a ‗rule of thumb,‘ or shortcut, that that aids us in making decisions. According to Gerd Gigerenzer, a heuristic must contain three criteria. First, heuristics must exploit evolved capacities – meaning that the heuristic is intrinsic to our nature and has evolved with us over time. This can be exemplified by the human ability to track an object in flight, also known as the gaze heuristic. Second, a heuristic must exploit structures of environment. This criterion implies that heuristics are not rational or irrational, good or bad, but rather relative to the environment in which they exist. Lastly, a heuristic must be distinct from ‗as-if‘ optimization models. Essentially, this criterion states that a heuristic cannot be ‗optimized‘ (or calculated) to explain away human behavior.24 Since we live in a world of bounded rationality, meaning that not all information is freely available to everyone, constraints are embedded into the analytical process. Constraints such as memory capacity, collection assets, time, cost, conflicting sources, etc. further complicate analysis, adding an ―optimization under constraints‖ factor to the

22

Mynatt, C.R., Doherty, M.E., & Tweeney, R. D. Confirmation Bias in a Simulated Research Environment. Quarterly Journal of Experimental Psychology, 29, 85-95 (1977). 23 Patterson, E.S., Woods, D.D., Sarter, N.B., & Watts-Perotti, J.C. Patterns in Cooperative Cognition. In COOP ‘98, Third International Conference on the Design of Cooperative Systems. 1998. 24 Gigerenzer, Gerd., ―Fast and Frugal Heuristics: The Tools of Bounded Rationality.‖ In The Blackwell Handbook of Judgment and Decision Making edited by Derek J. Koehler and Nigel Harvey, (Malden, Ma: Blackwell Publishing, 2004), 63-64.

10

already complex as-if optimization model.25 Unfortunately, these types of constraints are the norm for the intelligence analyst. In an effort to be more efficient, some heuristics are routine to the intelligence professional‘s analytical process. One such heuristic is the recognition heuristic. In essence, the recognition heuristic states that if two objects are presented, and only one of those objects are recognized, then the recognized object has a higher value in relation to the criterion presented.26 To take the recognition heuristic a step further, ecological rationality can influence choice. Ecological rationality is the level of correlation between an object and the criterion.27 Ecological rationality can be measured via recognition validity. Recognition validity is the proportion of times an object is recognized with a particular criterion, giving that object a higher criterion value than that of an object with a lower proportion to the same criterion.28 The recognition heuristic, then, can have a direct influence on the analysis of intelligence reports containing both labels and locations. If an analyst has a stronger recognition validity concerning a particular label (let‘s say ‗rebel‘ more-so than ‗insurgent‘) in a certain geographic location (‗Africa,‘ for example), the analyst would therefore be more likely to give the label with the greater recognition validity more weight when analyzing reports using that label and criterion. True heuristics have the qualities of being robust and fitting. Heuristics with robustness have an ability to be predictive about events in the future.
25

People use

Conlisk, John. ―Bounded Rationality and Market Fluctuations.‖ Journal of Economic Behavior & Organization 29. No. 2 (1996) 233-250. 26 Gigerenzer, Gerd., ―Fast and Frugal Heuristics: The Tools of Bounded Rationality.‖ In The Blackwell Handbook of Judgment and Decision Making edited by Derek J. Koehler and Nigel Harvey, (Malden, Ma: Blackwell Publishing, 2004), 68-69. 27 Ibid., 69. 28 Ibid.

11

heuristics because of their robustness; this is what makes them rules-of-thumb. They, more often than not, tend to ―work.‖ However, just because heuristics can be robust does not mean that we should rely on them, especially in a profession such as intelligence. Relying on rules of thumb, without using structured methods to aid the analytical process, can be dangerous and perhaps lead to intelligence failures. This is dangerous because rules of thumb are not equivalent to facts – they are not constants like the value of pi (3.14) or the Pythagorean Theorem (a2 + b2 = c2). In addition to robustness, Gigerenzer posits that heuristics must be able to fit past events or data that is already known. 29 The fit of a heuristic has a particular impact during the analysis of intelligence reports. Gigerenzer states that: In general, the predictive accuracy of a model increases with its fit, decreases with its number of adjustable parameters, and the difference between fit and predictive accuracy gets smaller with larger number of data points. The general lesson is that in judgments under uncertainty, one has to ignore information in order to make good predications. The art is to ignore the right kind.30 It is virtually certain that cognitive biases, heuristics, schemata, and frames are all inter-related and work simultaneous during the analytical process. These cognitive

processes are difficult to overcome and can impact the decisions we make as analysts.

29

Gigerenzer, Gerd., ―Fast and Frugal Heuristics: The Tools of Bounded Rationality.‖ In The Blackwell Handbook of Judgment and Decision Making edited by Derek J. Koehler and Nigel Harvey, (Malden, Ma: Blackwell Publishing, 2004), 78-79. 30 Ibid., 80.

12

Understanding the Labels The American Heritage Dictionary of the English Language describes a label as a term ―used to identify or classify a person or group of people or objects; an epithet.‖
31

In

essence, it is an intentional lingual shortcut with the purpose to communicate more easily. However, communicating more easily does not necessarily mean we are communicating more effectively. The use of labels inhibits the expression of true thoughts. For example, when I state that the Chechens attacked a small Russian convoy, what I truly mean to say is: a faction of extreme Chechens with deep-rooted hatred for Russia who are not supported by the government of Chechnya or comprised of peaceful Chechen citizens, attacked a convoy of Russian soldiers, some of which possibly being those who do not necessarily seek to harm or threaten Chechens, or some of which may be of similar ethnicity as the Chechens. It is evident that the use of a label, in this case Chechen, makes a statement a little easier to listen to, as our brain has less information to process. However, the label is used at the cost of accurately describing to whom or what I am truly referring. The Etymology of the Labels Before comparing the various labels‘ use today, it is important to examine their etymology. A word‘s etymology gives us a glimpse into how and why it has been used historically, which affects how and why it is used today. The term rebel, as we use it today to refers to ―one who resists, or rises in arms against, the established governing power; one who refuses or renounces allegiance or

31

The American Heritage Dictionary of the English Language. (New York: Houghton Mifflin Company, 2003).

13

obedience to his sovereign or the government of his country,‖ is first referenced in the Oxford English Dictionary (OED) circa 1400 as a reference to those whom Priam fought against in the Trojan War: ―He [Priam] was faryn to fight in a fer londe, To riche hym of Rebelles þatof þerewme held.‖32 It is traced through the centuries with notable references such as De Foe, in Crusoe (1719): ―I could… give Liberty, and take it away, and no Rebels among all my Subjects;‖ and Kipling, in Traffics & Discovery (1904): ―I shot my Bible full of bullets after Bloemfontein went… Take it and pray over it before we Federals help the British knock hell out of you rebels.‖33 The most recent entries in the OED for the term refer only to Muslims as rebels, and there are no recent references for rebels existing in the United States or any of its European allies. The OED defines militia, as I intend to discuss it, as ―a paramilitary force motivated by religious or political ideology, esp. one that engages in rebel or terrorist activities in opposition to a regular army.‖34 Originally, the term appeared as an allusion to the Spanish term milicia and the French milice, and its meaning has not changed dramatically from these Latin roots in 1937. Unlike rebel, the term militia has been recently used as a reference to United States citizens, as the OED cites the term as recent as the 1990s being applied to several right-wing groups who were opposed to gun control and mistrustful of the U.S. federal government: ―1995 Daily News (Halifax, Nova

Scotia)…The militia movement is already under scrutiny since the men arrested for the

32

"rebel, a. and n.1" The Oxford English Dictionary. 2nd ed. 1989. OED Online. Oxford University Press. 26 Aug. 2008 <http://dictionary.oed.com/cgi/entry/50198771>. 33 Ibid. 34 militia, n." The Oxford English Dictionary. Draft Revision 2008. OED Online. Oxford University Press. 26 Aug. 2008 <http://dictionary.oed.com/cgi/entry/00309562>.
14

Oklahoma City bombing were linked to a so-called citizens‘ militia—a pro-gun, antigovernment paramilitary group.‖35 Similarly to rebel, the OED defines an insurgent as ―one who rises in revolt against constituted authority; a rebel who is not recognized as a belligerent.‖36 References to insurgent are few in the OED, however, of note are Falconer‘s use of the word in Demagogue, 1765: ―His sanction will dismay, And bid th‘ insurgents tremble and obey;‖ and General Chalmers in 1812 in Great Britain, in which the term was used to refer to the original uprising of the American colonists in gaining independence: ―[Why] it was, that the vast strength of Britain did not beat down the colonial insurgents, not in one campaign, but in three.‖37 Ironically, the term terrorist in the OED is at once the most specific and the most vague, with two definitions that have political connotations. The first, specific definition reads: ―As a political term: Applied to the Jacobins and their agents and partisans in the French Revolution, esp. to those connected with the Revolutionary tribunals during the ‗Reign of Terror.‘‖38 The second, more vague definition describes a terrorist as ―anyone who attempts to further his views by a system of coercive intimidation.‖ 39 Despite its vague definition, the references for this term appear to be very specific. In its earliest uses, the term applied to specific members of an extreme revolutionary society in Russia, and then terrorist seems to disappear, according to the OED, until 1947 when it was used
35

militia, n." The Oxford English Dictionary. Draft Revision 2008. OED Online. Oxford University Press. 26 Aug. 2008 <http://dictionary.oed.com/cgi/entry/00309562>. 36 insurgent, a. and n." The Oxford English Dictionary. 2nd ed. 1989. OED Online. Oxford University Press. 26 Aug. 2008 <http://dictionary.oed.com/cgi/entry/50118488>. 37 Ibid. 38 terrorist." The Oxford English Dictionary. 2nd ed. 1989. OED Online. Oxford University Press. 26 Aug. 2008 <http://dictionary.oed.com/cgi/entry/50249599>. 39 Ibid.
15

to describe actions taken by Jews in the Middle East: ―The latest and worst of the outrages committed by the Jewish terrorists in Palestine—the blowing up of the King David Hotel in Jerusalem.‖40 In the most recent entries, we see the term used as a

reference to members of clandestine or expatriate organizations aiming to coerce established governments by acts of violence against them or their citizens. We see the term applied seemingly independent of nationality or religion, as the term has entries that use the term to reference Russians, Germans, Israelis, and Greeks. There are a number of common threads among all of these labels. All of the labels refer to groups of individuals who come together against a higher power or government. All use violence and/or coercion to further their cause, and all of them have a cause for which they have banned together. The differences come not so much in their denotative aspects as their connotative aspects. By strict definition alone, the labels would be difficult to distinguish from one another, but paired with their cultural contexts, which cumulatively create their connotative histories, we can make distinctions as to what kind of label will be applied to what kind of group. The ―Rebel‖ Label Academic literature defining the label ‗rebel‘ is lacking. The Merriam-Webster dictionary‘s adjective definition of rebel is, ―opposing or taking arms against a government or ruler, b: of or relating to rebels <the rebel camp>‖ and as a noun as, ―one who rebels or participates in a rebellion.‖41 The definition implies that a rebel, or rebel
40
41

Ibid.

rebel. (2009). In Merriam-Webster Online Dictionary. Retrieved April 15, 2009, <http://www.merriamwebster.com/dictionary/rebel>

16

group, is willing to use violent means to fight against a government or ruler. The primary motivation behind a group who rebels is therefore primarily political, and the violence is the method in which such a group achieves their goals – not through peaceful political methods of changing governments or the policies thereof. Michael Woldemarium, an Africanist Doctoral Fellow at the Woodrow Wilson International Center for Scholars, in discussing his paper Why Rebels Collide: Factionalism and Fragmentation in African Insurgencies used the definition given by the Peace Research Institute, Oslo (PRIO) Armed Conflict Dataset to describe rebel organizations.42 The PRIO Armed Conflict dataset describes a rebel organization as: a non-governmental group of people, formally organized, having announced a name for their group. It should also have used armed forces and violence to further the objectives of the group. The group's military activity must be part of a planned political campaign rather than spontaneous violence, thereby distinguishing it from certain guerilla movements. In the same way, the group must be involved in military activity that has resulted in at least twenty-five casualties.43 Woldemariam links the formation and fragmentation of African rebel groups to state failures in the late twentieth century, particularly the 1990‘s. These state failures are related to the inability of one or more rebel organizations or factions to gain control and consolidate state power. This continual cycle encourages the formation and

fragmentation of rebel groups, and it ultimately causes a period of endemic chaos and chronically failing governments. Woldermariam, again, reinforces the concept that a

42

Woldemariam, Michael. "Why Rebels Collide: Factionalism and Fragmentation in African Insurgencies." Africa: Conflict Transformation/Peacebuilding, Woodrow Wilson Center, Washington DC, August 19, 2008. 43 "Data on Armed Conflict - CSCW." PRIO - International Peace Research Institute, Oslo. <http://www.prio.no/CSCW/Datasets/Armed-Conflict/> & <http://www.prio.no/sptrans/1664678440/Codebook.pdf> (accessed April 15, 2009).

17

rebel group is one who opposes an established government using, or threat of using, violence. Nzamba Kitonga puts it this way: Whenever a ruling oligarchy, class or elite subjugates or dominates another and makes it impossible for that other through democratic legal and/or constitutional means to remove the oligarchy or to effect change within its superstructure, there results a dictatorship. And the only known means for the removal of a dictatorship is the use of armed insurrection. In a conflict of this nature, the subjugated class will call the ruling elite the oppressor, tyrant, and/or dictator. The ruling class will refer to the resisting group as rebel, subvert, treasoner, guerilla and in the extremist of cases, a terrorist.44 Ted Robert Gurr, in an extensive study that examined the cause for why people rebel, stated that the discontentment and frustration of a people toward a government lead to the potential for political violence.45 His concept of political violence best describes the acts of rebel groups. It states: …political violence refers to all collective attacks within a political community against the political regime, its actors – including competing political groups as well as incumbents – or its policies. The concept represents a set of events, a common property of which is the actual or threatened use of violence, but the explanation is not limited to that property. The concept subsumes revolution, ordinarily defined as fundamental sociopolitical change accomplished through violence. It also includes guerrilla wars, coup d‘état, rebellions, and riots.46 The frustration-anger-aggression theory underlies the causes for violent rebellions. This theory states that frustration, anger, and aggression have a direct correlation; and that the larger the population affected, the larger the scale of violence.47 Gurr states, ―Discontent leads men to political violence when their attitudes and beliefs focus it on political

44

Kitonga, Nzamba. "The Fight Against Terrorism in East Africa: Comments and Observations." East Africa Law Society Conference On ―Globalization And Terrorism: New Threats To Regional Integration‖, East Africa Law Society, Entebbe, Uganda, October 10, 2003. 45 Gurr, Ted R.. Why Men Rebel. Princeton: Princeton University Press, 1971. 46 Ibid., 3-4. 47 Ibid., Ch 2 & 3.

18

objects, and when institutional frameworks are weak enough, or opposition organizations strong enough, to give the discontented a sense of potency.‖48 Unfortunately, in an attempt for a rebel group to gain the desired attention from a government they oppose, rebels will often target the items of interest/value to that government. These items of interest and value, in the most severe cases of rebellion, are natural resources and civilians. Rebel groups that exemplify this point include the FDLR - Democratic Forces for the Liberation of Rwanda Hutu Rebels (Democratic Republic of Congo), Tutsi Rebels (Rwanda), Lord‘s Resistance Army (Uganda), and The Party of Islam (Somalia). The labeling for which groups are to be considered rebels, and which are to be considered terrorists, are highly subjective to who is doing the labeling. For example, Sheikh Hassan Dahir Aweys, a leader of the Eritrea-based Islamic Courts faction (one of the four rebel factions that comprise the The Party of Islam in Somalia), is named on the US list of terror suspects.49 It is possible that the overlap between labeling a group both as a rebel group and as a terrorist group may be intentional. Matthew O‘Rourke, in his article The Impact Of The "War On Terrorism" On Internal Conflicts, suggests that has been an intentional effort to reclassify rebel groups as terrorist organizations. O‘Rourke states: The campaign against terrorism has had political, as well as economic, repercussions. There has been a concerted effort to link conflicts to the terrorist attacks of September 11 and to reclassify opposition and rebel groups as "terrorists." Once rebel groups are classified as terrorists, governments feel less pressure to negotiate and become less willing to enter into a peace process. In many cases this disinclination towards negotiation leads a government to seek a military victory through the extermination of the rebel group. As well, links between rebel groups and international terrorist organizations, whether proven or not, are
48 49

Ibid., 155. U.S. Department of State. "E.O. 13224: Identified Terrorists and Groups." U.S. Department of State. <http://www.state.gov/s/ct/rls/fs/2001/6531.htm> (accessed April 16, 2009).

19

emphasized to isolate the groups and to justify a refusal to negotiate with them. Once a group is labeled "terrorist," its grievances, legitimate or not, are usually viewed as invalid, reducing international pressure on governments to work towards a negotiated settlement. And identifying an opposition group as "terrorists" helps a country obtain funding from the US as part of the war on terror. While this increase in funding may eventually lead to a military victory, it immediately intensifies ongoing conflicts.50 Whether the overlapping of labels is intentional or not, it is clear that an overlap does infact exist. What was of particular interest in O‘Rourke‘s statement was that he also accepts that some rebel groups do engage in terrorist activities, or are at least associated with known terrorist organizations. His statement evidences this: ―…links between rebel groups and international terrorist organizations, whether proven or not…‖ Whether or not what O‘Rourke posits is true, the mere fact that it may be possible is indicative of the similitude between rebel and terror groups. denotation as it is in connotation. The ―Militia‖ Label Historically, militias have formed because of a reaction to, or fear of, oppression (or the impression of oppression) of rights by a seemingly tyrannical government. 51 Militia activity has often been categorized as terrorist activity, especially when referring to groups who subscribe to extreme violence such as the Neo-Nazis, Ku Klux Klan (KKK), Second of June Movement, the Baader-Meinhof Gang, the Red Brigades, the The difference is not as obvious in

50

O'Rourke, Matthew. "The Impact Of The ‗War On Terrorism‘ On Internal Conflicts." The Ploughshares Monitor 26 (2005), <http://www.ploughshares.ca/libraries/monitor/monm05d.htm>. 51 Haider-Markel, Donald P., and Sean P. O'Brien. "Creating a "Well Regulated Militia": Policy Responses to Paramilitary Groups in the American States." Political Research Quarterly 50(3) (1997): 552. In JSTOR[database online]. Available from Mercyhurst College (accessed April 8, 2009).

20

Weathermen, and the Symbionese Liberation Army (SLA).52 Haider-Markel and O‘Brien posit that there is no agreed-upon definition of a militia, but that they believe a citizen militia is, ―any group of three or more persons organized for the stated purpose of defending their rights and property against a tyrannical government. Such a group must also engage in paramilitary training and/or maneuvers to prepare for ‗unlawful‘ behavior on the part of the government agents.‖53 The definition given by Haider-Markel and O‘Brien is significant as it essentially states that militias are organized and trained in a militaristic manner, implies that they are prepared to use violence as a means to achieve a goal, and asserts that their goal is to protect themselves (their rights and property) from a government that oppresses those rights. In a way, the definition makes it sound like the cause of a militia is justified, however, this is not always the case when we examine the vehemently violent groups mentioned earlier. Because the world is pluralistic, viewpoints are extremely subjective. While one person might define their ―rights‖ as one thing, another might disagree completely – leaving a situation in which one person‘s pursuit of liberty and protection of rights might actually infringe on another‘s liberty and rights. This subjective

interpretation of rights often causes extreme militias to engage in terroristic activities. Kurth Cronin identifies four types of terrorism that are categorized by motivation – these types are: left-wing terrorists, right-wing terrorists, ethnonationalist/separatist

52

Cronin, Kurth . "Behind the Curve: Globalization and International Terrorism." International Security, 27(3) (Winter 2002-2003): 39. In JSTOR[database online]. Available from Mercyhurst College (accessed April 9, 2009). 53 Haider-Markel, Donald P., and Sean P. O'Brien. "Creating a "Well Regulated Militia": Policy Responses to Paramilitary Groups in the American States." Political Research Quarterly 50(3) (1997): 552. In JSTOR[database online]. Available from Mercyhurst College (accessed April 8, 2009).

21

terrorists, and religious or "sacred" terrorists.54 Descriptions of ethnonationalist/separatist terrorists, and religious/sacred terrorists will be excluded from this section as they do not apply to militia groups/organizations. Left-wing terrorists, however, are described by Cronin as groups who align themselves with the communist movement and include groups such as the Second of June Movement, the Baader-Meinhof Gang, the Red Brigades, the Weathermen, and the SLA. Right-wing terrorists are described as groups who align themselves with fascism and include groups such as Neo-Nazis, Christian Patriots, and the KKK. The groups identified with both left- and right-wing terrorism have also been considered militia groups.55 Interestingly, the FBI has recognized some militia

organizations, and even ones that use ―militia‖ in their title, as domestic terrorist groups. These militias include The Freemen Organization, The Mountaineer Militia (TMM), The Washington State Militia, and The Phineas Priesthood.56 When referring to extremely violent militias and hate groups, the FBI often uses ―Militia‖ and ―domestic terrorist groups‖ interchangeably. In 1996, the FBI‘s Counterterrorism Threat Assessment and Warning Unit put out a report titled, ―Terrorism in the United States‖ which stated: Threats from domestic terrorism continue to build as militia extremists, particularly those operating in the western United States, gain new adherents, stockpile weapons, and prepare for armed conflict with the federal government. The potential for domestic right-wing terrorism remains a threat. Special interest groups also endure as a threat that could surface at any time.
54

Cronin, Kurth . "Behind the Curve: Globalization and International Terrorism." International Security, 27(3) (Winter 2002-2003): 39. In JSTOR[database online]. Available from Mercyhurst College (accessed April 9, 2009). 55 Anti Defamation League. "The Quiet Retooling of the Militia Movement: Recruitment." ADL: Fighting Anti-Semitism, Bigotry and Extremism. <http://www.adl.org/extremism/Militia/Recruitment.asp?m_flipmode=4> (accessed April 2, 2009). 56 FBI, Counterterrorism Threat Assessment and Warning Unit, National Security Division. ―Terrorism in the United States.‖ (1996): 7-8. www.fas.org/irp/threat/fbi_terror_96.pdf (accessed April 9, 2009).

22

Domestic terrorism involves groups or individuals who are based and operate entirely within the United States and its territories, and are directed at elements of the U.S. Government or population, without foreign influence. Domestic terrorist groups represent right-wing, left-wing, and special interest beliefs. Their causes spring from issues relating to American political and social concerns. 57

It is clear that the FBI recognizes that militias in the United States have used terrorist tactics as a means of achieving their goals. In addition, the causes of militia groups as listed by the FBI are reminiscent of the same causes listed by Gurr in Why Men Rebel.58 Although militias are most commonly associated with the western world, as evidenced by the OED, militias are not strictly a western phenomenon. There are a number of well-established militias in Africa and elsewhere in the world. One African militia group, the Janjaweed (in Sudan), has become increasingly popular in news reporting due to the crisis in Darfur. While the Janjaweed is overwhelmingly referred to as a ―militia‖ group by major news agencies such as CNN,59 BBC,60 and Reuters,61 their acts of violence on African ethnicities such as the Fur, Massaleet and Zagawa have been described as terroristic – citing rapes, child-killings, deprivation of water, and arson as common tactics of the Janjaweed. The ―Insurgent‖ Label Insurgency, as defined by Bard E. O‘Neill, is ―a struggle between a nonruling group and the ruling authorities in which the nonruling group consciously uses political
57 58

Ibid. Gurr, Ted R.. Why Men Rebel. Princeton: Princeton University Press, 1971. Ch 2 & 3. 59 CNN. "Janjaweed Militia: News & Videos about Janjaweed Militia - CNN.com." CNN.com - Breaking News, U.S., World, Weather, Entertainment & Video News. <http://topics.cnn.com/topics/janjaweed_militia> (accessed April 07, 2009). 60 BBC. "Sudan's Shadowy Arab Militia." BBC NEWS | News Front Page. <http://news.bbc.co.uk/1/hi/world/africa/3613953.stm> (accessed April 06, 2009). 61 Reuters. "Sudan Man Accused Of Aiding Darfur war Crime Court." Reuters.com - World News, Financial News, Breaking US & International News. <http://www.reuters.com/article/homepageCrisis/idUSLM432733._CH_.2400> (accessed April 07, 2009).

23

resources (e.g., organizational expertise, propaganda, and demonstrations) and violence to destroy, reformulate, or sustain the basis of legitimacy of one or more aspects of politics.‖62 The use of violence to achieve these goals set insurgents apart from

sociopolitical protests and movements. While there are many types of insurgents – such as, anarchist, egalitarian, traditionalist, apocalyptic-utopian, pluralist, secessionist, reformist, preservationist, and commercialist – they all employ the use of violence to alter a current political system. Interestingly, O‘Neill, in his explanation of the traditionalist insurgent, identifies groups/organizations as ―reactionary-traditionalist‖ insurgents that are more commonly referred to as terrorist groups. O‘Neill states: Within the category of traditionalist insurgents, one also finds more zealous groups seeking to reestablish an ancient political system that they idealize as a golden age. We refer to this subtype as reactionarytraditionalist. It includes a plethora of present day Islamic militants, or jihadis (―holy warriors‖), such as the Shia-based Party of God (Hezbollah) in Lebanon and the Islamic Call (Al-Dawa al-Islamiyya) in Iraq, as well as Sunni-based organizations like the Armed Islamic Group in Algeria, Hamas (―zeal,‖ the acronym for Harakat al-Muqawama al-Islamiyya, or Islamic Resistance Movement) in Palestine, the Islamic Group (Jama‘a alIslamiyya) in Egypt and, of course, Al Qaida (―the Base‖).63 The reasoning behind classifying these jihadi groups as insurgents rather than strictly as terrorists is due to their ultimate goal of overthrowing, an existing political system through a mechanism of violence – whereas a terrorist organization may only want to change policy and not necessarily overthrow it (although some terrorist organizations might also want to overthrow a government). Terrorism is just one tactic utilized by insurgents. Terrorism, as O‘Neill uses it, is a method of carrying out violence,

62

O'Neill, Bard E.. Insurgency and Terrorism: From Revolution to Apocalypse; 2nd Ed., Revised. Washington, D.C.: Potomac Books, 2005. 15. 63 Ibid., 21-22.

24

specifically ―the threat or use of physical coercion, primarily against noncombatants, especially civilians, to create fear in order to achieve various political objectives.‖64 The other two forms of warfare at the insurgents‘ disposal are guerilla and conventional warfare. O‘Neill‘s usage of terrorism and terrorist is similar to that of Paul Pillar in that it is a method or a specific act of violence.65 In fact, O‘Neill often uses the label terrorist synonymously with insurgents, using the label terrorist when referring to insurgents who engage in terrorism. He uses the term ―transnational terrorism‖ to refer to non-state terrorist organizations, such as Al Qaida, that view the world as a battlefield. Transnational terrorists still have the same ultimate goal in mind – to change a policy, however, the war on the policies they try to transform are played out on a much larger scale. Transnational terrorism is differentiated from international terrorism, which is used to describe similar terrorist acts by ―individuals or groups controlled by sovereign states‖
66

Insurgent terrorism is not as sporadic or mindless as it may oftentimes seem.

Insurgents who use terrorism have specific long, medium, and short term goals: The long-term goal is, of course, to change the political community, political system, authorities, or policies. The intermediate goal of terrorism is not so much the desire to deplete the government‘s physical resources as it is to erode its psychological support by instilling fear into officials and their domestic and international supporters. In the short term, terrorists often pursue one or more objectives, such as extracting particular concessions (eg., payment of ransom or the release of prisoners), gaining publicity, undermining or seeking to join a negotiating process, demoralizing the population through the creation of widespread disorder, provoking repression by the government, enforcing obedience and
64

O'Neill, Bard E.. Insurgency and Terrorism: From Revolution to Apocalypse; 2nd Ed., Revised. Washington, D.C.: Potomac Books, 2005. 33. 65 Pillar, Paul R.. Terrorism and U.S. Foreign Policy. Washington, D.C.: Brookings Institution Press, 2001. 18. 66 O'Neill, Bard E.. Insurgency and Terrorism: From Revolution to Apocalypse; 2nd Ed., Revised. Washington, D.C.: Potomac Books, 2005. 33.

25

cooperation from those inside and outside the movement, enhancing the political stature of specific factions within an insurgent movement, and fulfilling the need to avenge losses inflicted upon the movement.67 The CIA‘s understanding of insurgency and insurgents as outlined their Guide to the Analysis of Insurgency, is strikingly similar to O‘Neill‘s.68 Although the CIA identifies fewer types of insurgencies, placing them within four broad categories (Politically Organized, Militarily Organized, Traditionally Organized, and Urban); terrorism tactics, guerrilla warfare and conventional warfare are also identified as the forms of warfare conducted by insurgent groups.69 The goals of insurgents, as identified by the CIA, are also very similar to those identified by O‘Neill. However, unlike

O‘Neill, the CIA blatantly states that one of the objectives of insurgent groups is to control a particular area. ―This objective differentiates insurgent groups from purely terrorist organizations, whose objectives do not include the creation of an alternative government capable of controlling a given area or country.‖70 While insurgent and terrorist organizations have the same goals of changing an existing political order, the difference lies in actual physical occupation of a territory. The CIA‘s differentiation contradicts O‘Neill‘s claim that Al Qaida is in fact an insurgent organization – as Al Qaida does not intend to occupy and control territories such as the US, but rather it intends to change specific western policies and influences. Jim

67

O'Neill, Bard E.. Insurgency and Terrorism: From Revolution to Apocalypse; 2nd Ed., Revised. Washington, D.C.: Potomac Books, 2005. 34-35. 68 Central Intelligence Agency, Guide to the Analysis of Insurgency, C05332177. Washington DC,; Central Intelligence Agency, 2. Released January 05, 2009. <http://www.fas.org/irp/cia/product/insurgency.pdf> 69 Ibid., 5. 70 Ibid., 2.

26

Ruvalcaba describes terrorism in this sense as ―an auxiliary tactic that insurgents use as part of a broader strategy rather than an exclusive one.‖71 The ―Terrorist‖ Label “…as the meaning and usage of the word have changed over time to accommodate the political vernacular and discourse of each successive era, terrorism has proved increasingly elusive in the face of attempts to construct one consistent definition.” Bruce Hoffman

In his book, Inside Terrorism, Bruce Hoffman attempts to settle on a definition of terrorism that avoids the reckless labeling of terrorism under a wide range of acts. Hoffman believes that the history of the word reveals a usage that describes a ―reflection of the current political and social tenor of the times.‖72 He defines terrorism in a way that attempts to distinguish itself from guerillas, ordinary criminals, and assassins. Hoffman states that, ―Virtually any especially abhorrent act of violence that is perceived as directed against society –whether it involves the activities of anti-government dissidents or governments themselves, organized crime syndicates or common criminals, rioting mobs or persons engaged in militant protest, individual psychotics or lone extortionists – is often labeled ‗terrorism.‖73 Hoffman finds three main problems the terrorist label has encountered. The first problem is with the OED‘s definition of terrorist as being ―too literal and too historical‖

71 72

Ruvalcaba, Jim. "Understanding Iraq's Insurgency." Al Nakhlah Article 7 (Spring 2004): 1. Hoffman, Bruce. Inside Terrorism. New York: Cambridge University Press, 1998. 13. 73 Ibid.

27

to be applied to the present times.74 The second contention how subjective the decision is to use the label terrorist. Hoffman asserts that the decision depends ―largely on whether one sympathizes with or opposes the person/group/cause concerned.‖75 This point counters the opinion that terrorism can be defined in the same manner in which US Supreme Court Justice Potter Stewart defined pornography – that we may not be able to define it precisely, but we know it when we see it.76 The final contention deals with media‘s failure to be consistent with the usage of the terrorist label and its differentiation with other labels such as ‗guerilla,‘ ‗fundamentalist,‘ and ‗commando.‘77 It is for these three main reasons that the definitions for, and the usage of, the terrorist label is so disputed. Through his research, Hoffman boils terrorism down to:78 Ineluctably political in aims and motives Violent - or equally important, threatens violence; Designed to have far-reaching psychological repercussions beyond the immediate victim or target; Conducted by an organization with an identifiable chain of command or conspiratorial cell structure (whose members wear no uniform or identifying insignia); and Perpetrated by a subnational group or non-state entity We may therefore now attempt to define terrorism as the deliberate creation and exploitation of fear through violence or the threat of violence in the pursuit of political change. All terrorist acts involve violence or the threat of violence. Terrorism is specifically designed to have far-reaching psychological effects beyond the immediate victim(s) or object of the terrorist attack. It is meant to instill feat within, and thereby
74 75

Hoffman, Bruce. Inside Terrorism. New York: Cambridge University Press, 1998. 14. Ibid., 31. 76 Jacobellis v. Ohio, 378 U. S. 184 (1964) 77 Hoffman, Bruce. Inside Terrorism. New York: Cambridge University Press, 1998. 36-37. 78 Ibid., 43.

28

intimidate, a wider ‗target audience‘ that might include a rival ethnic or religious group, an entire country, a national government or political party, or public opinion in general. Terrorism is designed to create power where there is none or to consolidate power where there is very little. Through the publicity generated by their violence, terrorists seek to obtain the leverage, influence and power they otherwise lack to effect political change on either a local or an international scale.79 Many similarities to Hoffman‘s definition of terrorism also emerge in Michael Kronenwetter‘s, Terrorism: A Guide to Events and Documents, particularly in the historical examination and criticisms of government agency definitions. Kronenwetter also posits that terrorism has also existed in the United States for a long time, but that it has been labeled as other things (resembling Hoffman‘s argument that the OED reflects a usage consistent with the current political culture of the time). He illustrates this point with examples of Post-Civil War acts by the Ku Klux Klan being called ‗resistance to reconstruction;‘ the 1886 Chicago Haymarket Square bomb being described as ‗labor troubles;‘ ‗vigilantism‘ when African Americans were being lynched in the first half of the twentieth century, and racism when whites bombed black churches in the second half; and the Unabomber bombings described as ‗madness.‘80 Kronenwetter states that terrorists have three main elements: 1) they are malicious actions or threats; 2) they are ―directed against people who are regarded as innocent, or who are protected by the laws and conventions of modern warfare;‖ and 3) ―designed (at

79 80

Hoffman, Bruce. Inside Terrorism. New York: Cambridge University Press, 1998. 43-44. Kronenwetter, Michael. Terrorism: A Guide to Events and Documents. New York: Greenwood Press, 2004. vii.

29

least partly) to frighten, intimidate, or otherwise influence populations or governments.‖ 81 Kronenwetter‘s argument differs from Hoffman in the sense that Hoffman‘s definition implies a strong political motivation behind terrorist acts, whereas Kronenwetter leaves that aspect open to interpretation. Another difference is that Kronenwetter draws-out an element of innocence among the victims of terrorism; Hoffman does not. What the two definitions do have in common, however, is the element of violence and the psychological impact left with the victims of terrorism. Like Hoffman and Kronenwetter, Paul Pillar asserts that the terrorist label has been used too loosely – even to the point to flippantly describe someone else‘s political policies. Pillar believes that the US Department of State‘s statutory definition for

keeping statistics on international terrorism is as good a definition as any other. The statute defines terrorism as, ―premeditated, politically motivated violence perpetrated against noncombatant targets by subnational groups or clandestine agents, usually intended to influence an audience.‖82 Five key elements are listed in the State

Department‘s definition: premeditation, violence, political motivation, noncombatant targets, and perpetrators that are comprised of either subnational groups or clandestine agents. This definition is an ideal blend of Hoffman‘s and Kronenwetter‘s definitions. The only element left out of this definition, but included in the other two, was the psychological aspect; however, Pillar picks up on this point by stating: There is one other respect in which terrorism must be conceived somewhat more broadly than the statutory definition above. Terrorism as an issue is not just a collection of incidents that have already occurred; it is at least as much a matter of what might occur in the future. The threat of a terrorist
81

Kronenwetter, Michael. Terrorism: A Guide to Events and Documents. New York: Greenwood Press, 2004. 8. 82 22 U.S.C. 2656f (d). "Chapter 7 -- Legislative Requirements and Key Terms." U.S. Department of State. http://www.state.gov/s/ct/rls/crt/2007/103715.htm (accessed March 30, 2009).

30

attack is itself terrorism…Terrorism in general, even when conducted with conventional means, tends to have greater psychological impact relative to the physical harm it causes than do other lethal activities, including warfare.83 Pillar believes that when writing policy on terrorism, we shouldn‘t hyper-focus on trying to define what terrorism is or is not. Instead, Pillar suggests taking a commonsensible approach to what terrorism is by paralleling the understanding of the label/act with Justice Stewart‘s understanding of pornography.84 Lastly, Pillar states that most-ofall, ―terrorism is a method – a particularly heinous and damaging one – rather than a set of adversaries or the causes they pursue. Terrorism is a problem of what people (or groups, or states) do, rather than who they are or what they are trying to achieve.‖85 While there are similarities between Hoffman‘s, Kronenwetter‘s, and Pillar‘s definition of terrorist and terrorism (as well as many others), the definitional differences and lack of consensus is evident. The definitional quagmire is likely to continue on for decades, especially as nations attempt to agree on a broad enough definition to include a wide-variety of actors engaging in asymmetrical warfare, but yet a narrow enough one so that nations are not self-included in those definitional parameters. That said, there are three high-frequency components that run through definitions of terrorism: 1) the use of violence, 2) political objectives, and 3) the intent to place fear into victims.86

83

Pillar, Paul R.. Terrorism and U.S. Foreign Policy. Washington, D.C.: Brookings Institution Press, 2001. 14, 24. 84 Ibid., 16-17. 85 Ibid., 18. 86 Jongman, A.J., and Alex P. Schmid. Political Terrorism. New Brunswick: Transaction Publishers, 1988. Jongman and Schmid analyzed 109 different definitions of 'terrorism' and identified frequency trends among components used to describe the term.

31

Comparing the Labels The definitions of the rebel, militia, insurgent, and terrorist labels are all quite ambiguous and yet all are particularly very similar. They differ not so much in

denotation as they do in connotation. All four of the labels share the same core concepts and have common threads that tie them all together; however, when we hear each of them individually each label evokes a specific emotion and mental image. Our cognitive biases push beyond the denotations of the labels and create the connotations for which think about them on a daily basis. All the labels share the threads of being political in nature, use violence as a method to achieve their goals, and can even participate in terrorist tactics (blatantly targeting innocents and using fear as a weapon). All the groups are formed in opposition to a current establishment, particularly political, but also sometimes ideological. The groups form as a reaction to discontent with the establishment – following the frustrationanger-aggression theory that was outlined by Ted Gurr. The more widespread the

frustration and discontent, the more the group becomes angry. The scope broadens with an increasingly discontented population, and in turn, the intensity potential also increases. As the intensity potential increases, the potential for violence also increases. The biggest difference between the labels it this: While all groups (rebel, militia, insurgent, and terrorist) can use terrorism as a tactic to achieve their goals, not all groups have to engage in terrorism. Terrorism is a common tool in the tactical toolbox for many of the extremely violent groups that are categorized as one of these labels; however, rebel groups, militias, and insurgents can also use solely conventional and guerilla tactics to achieve their goals. This fact, however, does not explain why rebel groups, militias, and

32

insurgents who decide to use terrorist tactics are still categorized as such groups – rather than categorized as terrorist groups/organizations. In addition, the groups we label as terrorists do not strictly use terrorist tactics – they also engage in other forms of warfare such as guerrilla warfare and cyber warfare. Whether the reasoning is embedded into

our socio-linguistic nature, or whether it is an intentional process decided by governments as O‘Rourke suggests, the fact still remains that we do consider some groups as rebels, militias, and insurgents despite any terrorist tactics they may employ. Aside from the assertion that rebel groups, militias, and insurgents do not have to engage in terrorism (although the labeling of groups becomes increasingly more difficult with groups that actively engage in multiple forms of violence – including terrorism) little distinguishes one label from another. The literature suggests that the difference between a terrorist group and an insurgent group is that insurgent groups want to physically control an area after they overthrow an established government. However, while the direct goal of a terrorist group may not necessarily include overthrowing a government, but rather changing the policies of that government, most terrorist organizations would not be dissatisfied with the complete takeover of a government they oppose. In addition, the insurgent‘s aspect of ―controlling‖ an area is ambiguous and left open to interpretation – begging the question: Do you have to physically commit yourself to a particular area in order to control that area? The denotation differences between rebel groups and militias are less obvious. Connotatively, the difference suggests that rebel groups are less organized than militia groups, giving the group a more rag-tag feel to it. However, this is evidently not the case

33

when the violence spurred by rebel groups is closely examined. Denotatively, the biggest difference between rebel groups and militias is that militia groups do not have to be actively engaged in violence. Militia groups must only have the potential to engage in violence against an established government. Rebel groups, on the other hand, are

considered as such because they are actively engage in violent opposition to an established government. Both groups can utilize multiple forms of violence, including terrorism, as a tactic. While there are some notable differences between the labels, the rebel label and the insurgent label are seemingly alike. Both groups oppose an established government and desire to take it over, even if violence is necessary. Both groups are actively engaged in violence, not just holding the potential to be violent. And lastly, both groups use, or can use, a variety of tactics to achieve their goals. Connotatively, we think that insurgent groups are more likely than rebel groups to use terrorist tactics, however this is most likely due to our cognitive biases and the popularity of insurgent reporting in media networks (attributed mostly to the war in Iraq). Defining these labels is an extremely difficult task and it is not my aim to do so in this thesis. My intention is to illustrate the marked similarities of these labels and to point-out that the human cognitive process likely has something to do with how we view and analyze these groups. While the use of labels can be quite helpful in processing information, labels with similar denotations and divergent connotations can be troublesome to the analyst. When definitions begin to overlap and blur together without any methods for clarifying the intent of one label from another, the quality of analysis is

34

at risk. My thesis will examine to what extent the quality of analysis is at risk, if it is indeed at risk as the differing connotations of these labels would suggest.

35

METHODOLOGY

Research Design In order to test whether or not cognitive biases affect the way an intelligence professional analyzes reports using the labels in question (rebel, militia, insurgent, and terrorist), I conducted an experiment that combined two research designs: a cross-

sectional survey and a five-group experimental design. A cross-sectional survey is a survey design that gathers information on a population at s single point in time.87 This design allows for respondent data to be cross-analyzed against other respondents. Crosssectional surveys are one of the easiest survey experiments to conduct, as they do not have to track participant responses over a period of time. Due to time constraints, the cross-sectional survey was most appropriate for this study. Since multiple labels were being tested against each other, a five-group experimental design was used. The fivegroup experimental design is a modified two-group posttestonly experimental design that uses a total of five groups (rather than two) in order to determine whether the labels affect analysis. True experimental designs, like the five-group design used in this study, use randomly assigned (R) participants. In this type of design, one group is assigned a program (indicated with an ‗X‘), which in the case of this study is the use of neutrally emotive label – ―group.‖ A pretest is not necessary with this type of experiment as the random assignment of participants is assumed to be probabilistically Table 3.1 R group R rebel R militia R insurgent R terrorist X O O O O O

87

"Survey Methods." School of Information - University of Texas. http://www.ischool.utexas.edu/~palmquis/courses/survey.html (accessed April 20, 2009).

36

equivalent.88 After the program is implemented, all groups are measured (indicated with the ‗O‘) and compared against one another to measure the effect of the program (see Table 1). In the case of this experiment, the group with the program acted as the control group and also served as the comparison group against the other groups. The posttestonly randomized experiments hold strong against single-group and multiple-group threats to internal validity.89 In total, five mock intelligence reports (See appendices E-I) with accompanying surveys (See appendix J) were created – each report and survey used one of the labels in question (rebel, militia, insurgent, and terrorist) and one report and survey acting as the control group using the label ―group.‖ All five reports and surveys were identical in wording and context with the exception of which label was used. Sample, Population, and Participants Participation in the experiment was limited to populations associated with intelligence analysis. These populations included analysts from the national security sector, military intelligence, law enforcement intelligence, competitive intelligence, and students of intelligence studies. In order to target populations that were related to

intelligence analysis, I recruited students of the Mercyhurst Intelligence program (both graduate and undergraduate students), as well as tapped into the Mercyhurst College Intelligence Studies Program‘s faculty contact network. The Mercyhurst contact network included intelligence-related organizations such as the International Association of Law Enforcement Intelligence Analysts (IALEIA), the Society of Competitive Intelligence

88

Research Methods Knowledge Base. "Two-Group Experimental Designs." Social Research Methods. http://www.socialresearchmethods.net/kb/expsimp.php (accessed April 20, 2009). 89 Ibid.

37

Professionals (SCIP), and the International Association for Intelligence Educators (IAFIE). In addition, a link to the experiment was posted on Professor Kristan

Wheaton‘s Sources and Methods blog, which has a strong following of intelligence professionals.90 The link to the survey was also sent out to other intelligence

professionals with which the Mercyhurst faculty and I have personal contact. Participation in the experiment was on a voluntary basis and participants did not receive any compensation for their efforts. The participants in my study did not come from a population requiring any special protection (i.e. children, elderly, criminals, etc.) and none of the participants were purposefully deceived. There were no physical or psychological risks associated with the survey. Participants were presented with an informed consent form (See Appendix B) that was posted on the hosting site; consent to participate was indicated by clicking the hyperlink that directed participants to the actual mock intelligence report and survey.91 After the survey was completed, participants were redirected to a debriefing statement that was also posted on the host site (See Appendix C).92 The procedure and experiment was approved by Mercyhurst College‘s Institutional Review Board (See Appendix D) on November 12, 2008. The link to the experiment was originally hosted through Mercyhurst College‘s Institute for Intelligence Studies (MCIIS) website. The original host site yielded 222 participants. Unfortunately, the Intelligence Program switched servers part way through

90

Wheaton, Kristan. "Surveys, Surveys, Surveys! (It's Research Season...)." Sources And Methods. http://sourcesandmethods.blogspot.com/2009/01/surveys-surveys-surveys-its-research.html (accessed April 21, 2009). 91 Pollard, Kristine. "Jeffrey Welgan Thesis." Institute for Intelligence Studies at Mercyhurst College. http://74.125.95.132/search?q=cache:acF9XiY2Ly0J:www.cirat.com/welganthesis.php+http://www.cirat.co m/welganthesis.php&cd=1&hl=en&ct=clnk&gl=us&client=firefox-a (accessed April 21, 2009). 92 Pollard, Kristine. "Jeffrey Welgan Thesis - Thank You." Institute for Intelligence Studies at Mercyhurst College. http://www.cirat.com/welganthanks.php (accessed April 21, 2009).

38

the winter term causing a broken link to the experiment.

This was particularly

unfortunate as the survey encouraged participants to pass the link along to other intelligence professionals who might be interested in participating in the survey. The broken link halted participation momentum. The link was later reposted on the

Mercyhurst College‘s Center for Intelligence Research Analysis and Training (CIRAT) website; however, I was able to only gain eleven more intelligence professionals to participate before the conclusion of the survey.93 In intelligence participated experiment. illustrates the total, 233 Figure 3.1

professionals in Figure number my 3.1 of

participants by label received in the experiment. In order to randomly intelligence assign the Figure 3.2

report/survey

among the participants, I used an automatic hyperlink randomizer that sent

93

Pollard, Kristine. "Jeffrey Welgan Thesis." Institute for Intelligence Studies at Mercyhurst College. http://74.125.95.132/search?q=cache:acF9XiY2Ly0J:www.cirat.com/welganthesis.php+http://www.cirat.co m/welganthesis.php&cd=1&hl=en&ct=clnk&gl=us&client=firefox-a (accessed April 21, 2009).

39

participants to one of the five surveys.94 Although participants varied by experience level, the majority of participants (65%) had less than three years of intelligence experience (see Figure 3.2), This large proportion is attributed to the number of student participants I was able to recruit through the Intelligence Studies program at Mercyhurst College as well as the number of alumni the college is still in contact with. Figure 3.3 illustrates the number of participants by the field of intelligence with which they most associate themselves. Again, the participants who have identified themselves as students of Figure 3.3 intelligence represent the largest portion (28%) of the responses collected in the experiment. Military intelligence analysts and national security-related intelligence analysts represent the second and third largest populations with 24% and 22% respectively.

Data Collection Instruments, Variables, and Materials In order to collect respondent data, I used a paid subscription survey service through freeonlinesurveys.com. This survey service was used for its relatively

inexpensive monthly fee and for its ease of use. Freeonlinesurveys.com allows the user
94

"CGI Scripts: Random Link." CGI Spy. www.cgispy.com/scripts/rlink.html (accessed April 21, 2009).

40

to create multiple surveys with unique uniform resource locators (urls), allowing them to be used in the hyperlink randomizer on the hosting site. The survey service also allowed the survey results to be downloaded into a Microsoft Excel Spreadsheet. The mock

intelligence reports (using one of the five labels) were presented to the participants at the commencement of their survey. The mock intelligence reports provided the invariable data for which the participants had to respond and analyze. The reports focused around seven primary data anchors: A setting of the Northeast and Horn of Africa Regions An overall increase of activity for that particular label Imagery intelligence indicating an increase in human activity near an apparent training base or command post Local newspaper report that the label ―does not plan to incite, or participate in, any violence in the future despite that group‗s discontentment with the current establishment‖ Recent COMINT indicating an increase in anti-government rhetoric from the label based upon social injustice and oppression A long history of social strife and discontentment in the region The region being a hot-bed of differing political and religious ideologies, as well as a region of poverty and poor health conditions The surveys also consisted of three questions relating to the background of the participants and sixteen questions relating to the participants‘ analysis of the mock intelligence report. Unfortunately, the first question, asking whether or not the participant has ever been an intelligence analyst, had to be discarded from the results. Feedback from some of the participants, mostly students of intelligence, indicated that there was some confusion with this question, causing discrepancies with responses. The exclusion of the data from this question is unlikely to have significantly changed the overall results of the experiment, as populations related to intelligence analysis were specifically targeted. The other two background questions asked what area of intelligence the
41

participants were most associated with, as well as how many years of experience the participants had. The remainder of the questions focused around the participants‘ analysis of the mock intelligence report. sections: 1. A section asking about the participants‘ assessment of the situation. The participants had to answer questions regarding the overall likelihood of an attack, the likelihood that an attack would be successful, the likelihood that an attack would be sophisticated, and a question inquiring about the likelihood that an attack would be severe. The participants had to answer these questions on a 1-10 scale – with ―1‖ indicating that it was virtually certain not to be, and ―10‖ indicating that it was virtually certain. Each question in this section was paired with a question asking for the participant‘s analytical confidence – this was also based on a 1-10 scale, with ―1‖ indicating an extremely low level of analytical confidence, and a ―10‖ indicating an extremely high level of analytical confidence. 2. A section that indicated the value placed on some of the data anchors provided within the mock intelligence report. These questions were posed as open-ended statements that the participants had to complete using a Likert Scale of 10 possibilities; one the low end the participants could respond with, ―highly decreases the likelihood of an attack,‖ and on the high end with ―highly increases the likelihood of an attack.‖ These openended statements focused around invariable data within the report, such The questions could be categorized into three different

42

as: the overall increase in group activity; Imagery indicating increases in human activity near a training base or command post; the local newspaper report that the label ―does not plan to incite, or participate in, any violence in the future despite that group‗s discontentment with the current establishment; COMINT indicating an increase in anti-government rhetoric; the long history of social strife and discontentment in the region; and the region being a hot-bed of differing political and religious ideologies, as well as a region of poverty and poor health conditions. 3. Lastly, there was a two-question section asking the participants to rank the labels based on the which labels were most likely to engage in violence, and another question asking the participants to rank the labels according to which were most likely to execute severe attacks.

Data Analysis Procedures In order to analyze the data from the surveys effectively, the data had to first be consolidated into one large master database. Since the data from freeonlinesurveys.com was downloaded into separate Excel spreadsheets, the data was easily consolidated by cutting and pasting the information from each separate survey into one single spreadsheet. An extra column was then added to the spreadsheet that identified the label given to each participant. To test for statistical significance of responses among the different labels, the data was analyzed using a SPSS program. A statistics professor, Dr. Hemagini Deshmukh, in Mercyhurst College‘s math department conducted the SPSS analysis, ensuring accuracy of the results. Statistical significance testing was run for questions four through
43

seventeen. Responses were tested among the labels for each question independently. Additionally, differences among labels for all questions together were tested. Questions four through eleven asked participants to assess the: overall likelihood of an attack, likelihood of a successful attack, likelihood of a sophisticated attack, and the likely severity of an attack, as well as the analytical confidence levels for each of those questions. Questions twelve through seventeen asked participants to assess the effects of the various data anchors on the likelihood of an attack occurring. In order to test for normality among the labels, the Kolmogorov-Smirnov and Shapiro-Wilk tests were used in conjunction with a Lilliefors Significance Correction.95 If these two tests resulted in a P-value of less than =0.05 , then an additional Normal Quantile Plot was necessary to meet the normality assumption. To test the homogeneity of variance, the Levene test was used. Since more than two groups were being compared, it was necessary to conduct a 1-way ANOVA (Analysis of Variance) test. This test assured that equal variance assumptions were satisfied and whether or not there was statistical significance between the labels. Statistical significance is present when

ANOVA significance levels are less than =0.05 between the groups. In the instance where statistical significance was observed in the ANOVA testing, additional testing was necessary to identify what labels were statistically significant. The Scheffe test was used to discover which labels were statistically significant. This test was used due to its ability to account for a differing number of participants for each label tested. Lastly, to test differences among the labels for all the questions combined the

95

The Lilliefors Significance Correction adjusts for the estimation of population parameters such as, mean and variance (or standard deviation) rather than having known values. The Lilliefors correction generally provides a better test than an uncorrected approach.

44

Kruskal Wallis test was used. Statistical significance is achieved when the Chi-Square test P-value is less than =0.05. In instances where all assumptions for the 1-way ANOVA test were not satisfied, the Welch/Brown-Forsythe test had to be used to ensure robustness for the equality of the means were satisfied. In addition to statistical significance testing, the Pivot Table feature in Microsoft excel was used to analyze the results of the surveys. This feature consolidated the data in a way that allowed me to quickly compare the participants‘ responses across the different labels. The data was then used to create graphs that visually represented the results. While the graphs did not illustrate whether or not responses were statistically significant, they did illustrate areas where response levels differed enough to suggest that there may be a possibility for difference. These possible differences could be caused for a variety of reasons but represent areas where further research may be needed.

45

FINDINGS
No statistical significance was found between any of the labels with the exception of two instances. Statistical significance was found between the ‗group‘ and ‗rebel‘ labels for the likelihood of an attack for question #12 (the effect of overall increases in activity), and statistical significance was found between the ‗group‘ and ‗militia‘ labels for question #15 (the effect of COMINT reporting increased levels of anti-government rhetoric).

Figure 4.1

Statistical significance between the ‗group‘ and ‗rebel‘ label regarding the overall increase in activity had a 1-way ANOVA P-value of 0.017 and a Scheffe P-value of 0.036, which are both significantly lower than the required =0.05 values. The ANOVA test found that there was a difference between the labels, while the Sheffe test identified which labels were significantly different. The difference between the labels indicates that
46

more weight is given to this piece of evidence (data anchor) when assessing the likelihood of an attack for rebel groups compared to a neutrally emotive label such as ‗group.‘ Regarding statistical significance between the ‗group‘ and ‗militia‘ labels regarding how COMINT indications affect the likelihood of an attack, the Tamhane test had to be used in place of the 1-way ANOVA test because variance assumptions were not satisfied using the Levene‘s test. The Tamhane test resulted in a P-value of 0.042 between the ‗group‘ and ‗militia‘ labels, indicating that analysts place more weigh on the COMINT data anchor compared to a neutrally emotive label. To better organize the findings of the questions, I have organized the findings into subsections according to the variable questions asked in the survey. Each subsection discusses the statistical significance levels of the responses as well as any other relevant findings observed. The subsection discussing the results of questions 18 and 19 did not get analyzed for statistical significance, as they required the analysts to rank the violence and severity levels of the labels.

Questions 4&5: Overall likelihood of an attack Statistical analysis of participants‘ assessment regarding the overall likelihood of an attack across all the labels used in this experiment (Question #4) resulted in no statistical significance. All labels met the tests for normality and equal variance

assumptions. In addition, none of the labels contained any outliers in the participants‘ responses.

47

Although none of the responses were statistically significant from each other, it is important to note some of the percentage differences among the likelihood levels. Figure 4.2

Participants who received the ‗group‘ label gave the largest volume of ―unlikely‖

responses, equaling 13% of the responses and surpassed the 5-7% range of the other labels (approximately twice as
*Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’ ‘M,’ ‘I,’ and ‘G’ to indicate the terrorist, rebel, militia, insurgent, and group labels respectively

much). In addition, the ‗rebel‘ label received the highest volume of ―highly likely‖ responses, comprising of 7% of the respondents‘ assessments – this response far outweighed the ―highly likely‖ responses from the participants who received the other labels (7% versus the 2-3% range). The percent difference of the ‗rebel‘ label indicates that this particular label possibility evokes some type of cognitive bias, or at least a particular emotive response, when included in the African context. The ‗group‘

percentage of ―unlikely‖ responses are possibly indicative of lesser amounts of cognitive bias, or are at least less emotive responses, when compared to the other labels. Statistical analysis of participants‘ analytic confidence levels for the overall likelihood of an attack (Question #5) also resulted in no statistical significance. All labels meet the tests for normality and equal variance assumptions even though several outlier responses were noted for all labels except ‗insurgent‘ and ‗rebel.‘

48

Participants who received the ‗terrorist‘ and ‗rebel‘ labels indicate the possibility of being more confident in their responses when compared to the other labels. This possibility is Figure 4.3

evidenced by both labels having a 7% analytic confidence level of ―9‖ on a 10-point scale; an answer of ―10‖ represents an extremely high confidence level.
*Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’ ‘M,’ ‘I,’ and ‘G’ to indicate the terrorist, rebel, militia, insurgent, and group labels respectively

Questions 6&7: Likelihood of a Successful Attack Statistical participants‘ analysis assessment of Figure 4.4

regarding the likelihood of a successful attack across all the labels in (Question no #6)

resulted

statistical

significance among any of the labels in question. All labels meet the tests for normality

49

and equal variance assumptions.

In addition, no outliers were present among the

responses, satisfying another requirement for normality. The Shapiro-Wilk test for normality did indicate that some slight variance among the labels was possible, particularly between the ‗group,‘ ‗insurgent,‘ and ‗terrorist‘ labels. These differences do indicate that this area might need further testing. The trend line of the bar graphs of these labels also evidenced this variance possibility. The trend line of the participants‘ responses for the ‗group‘ label illustrate a downward slope – representing the possibility that this label is least likely to execute successful attacks (see Figure 4.4). The trend line for the ‗insurgent‘ responses illustrate a slope of near zero (see Figure 4.5), indicating that the Figure 4.5

participants who received this label had difficulty placing a value on the success level of insurgent attacks. The levelness of the trend line may be caused in part because of the varying success levels of insurgent attacks against US and coalition forces in Iraq.

50

Figure 4.6 Lastly, the responses to the ‗terrorist‘ label indicated an

increasing trend line regarding the success of an attack (see Figure 4.6). The increasing trend line a possibility among

indicates

respondents that terrorists, as a whole, are more likely than the other labels to execute successful attacks. This trend could be attributed to the amount of attention received by the media when terrorist attacks are successful compared to the amount of media attention received when terrorist attacks are thwarted. Statistical analysis of participants‘ analytic confidence levels for the likelihood of a successful attack (Question #7) resulted in no statistical significance among the labels. All labels met the tests for normality and equal variance assumptions even though several outlier responses were noted for all labels except ‗group,‘ ‗insurgent,‘ and ‗rebel.‘

Questions 8&9: Likelihood of a Sophisticated Attack No statistical significance was identified between responses concerning the sophistication levels of an attack (Question #8) among the different labels. Normality and equal variance assumptions were met for all the labels. Several outliers were present among the responses for the ‗group‘ and ‗insurgent‘ labels, however normality was still satisfied despite the outliers.
51

When visually representing the respondents‘ answers by percentages in a doughnut graph, responses to the ‗insurgent‘ label differ greatly from the other labels. The doughnut graph illustrates a much larger number of overall responses that indicate that the sophistication of an attack by an insurgent group is unlikely. This is evident by the total of responses of ‗somewhat less than even,‘ ‗somewhat unlikely,‘ Figure 4.7

‗unlikely,‘ and ‗highly unlikely‘ responses – comprising of 87% of the participants‘ responses of a negative sophistication level (see Figure 4.7).
*Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’ ‘M,’ ‘I,’ and ‘G’ to indicate the terrorist, rebel, militia, insurgent, and group labels respectively

When comparing the negative sophistication responses against the other labels‘ negative sophistication responses, the ‗insurgent‘ label received a far more negative sophistication response (terrorist=72%, rebel=69%, militia=68%, and group=75%). Again, this trend may be possibly contributed to the sporadic insurgent attacks against US and coalition forces in Iraq. Statistical analysis of participants‘ analytic confidence levels for the likelihood of a successful attack (Question #9) resulted in no statistical significance among the labels. All labels met the tests for normality and equal variance assumptions – one single outlier was noted for the ‗group‘ label, but single outliers are acceptable for normality assumptions.

52

Questions 10&11: Severity of an Attack Statistical analysis of participants‘ assessment regarding the likelihood of an attack being severe (Question #10) resulted in no statistical significance. All labels meet the tests for normality and equal variance assumptions. In addition, only one single outlier was present with the ‗group‘ label, which also meets Figure 4.8

normality assumptions. The doughnut graph

shows some interesting results however (see Figure 4.8). The ‗terrorist‘ label received the largest volume of ―not severe‖ answers participants (36%) assessing among the
*Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’ ‘M,’ ‘I,’ and ‘G’ to indicate the terrorist, rebel, militia, insurgent, and group labels respectively

potential severity level if an attack were to occur – this assessment is only 2% greater than the ‗group‘ label. In contrast, the ‗militia‘ label received the highest volume of ―very severe‖ (3%) and ―severe‖ (24%) answers, indicating that this label dishes out the most potential to conduct severe attacks (76% of respondents assessed this label of conducting attacks that were ―somewhat severe‖ or greater). The ‗rebel‘ label was assessed as the second most-severe label with 73% of respondents giving the label an assessment of at least ―somewhat severe.‖

53

Statistical analysis of participants‘ analytic confidence for severity levels of an attack (Question #11) resulted in no statistical significance among the labels. The

‗group‘ label and the ‗terrorist‘ label met normality assumptions via the KolmogorovSmirnov and Shapiro-Wilk tests. All other labels met the tests for normality and equal variance assumptions via Quantile plots. Although the analytic confidence levels of the participants among the labels did not achieve statistical significance, analysis of the responses represented by percentage level alludes to a possibility that analysts are more confident when assessing the terrorist and rebel groups, at least in the context of Africa. The doughnut graph illustrating the difference in confidence levels across the labels. The ‗terrorist‘ label received the highest levels of confidence among all of the other labels. It was the only label that received an analytic confidence level of ―10‖ (representing an extremely high confidence level), of which 5% of respondents answered. 7% of the participants with this label had a

confidence level of ―9,‖ and 24% had a confidence level of ―8.‖ Therefore, more than one-third of the participants who received this label had a confidence level greater than ―8.‖ Participants who had the Figure 4.9

‗rebel‘ label also had reasonably high levels of analytic confidence. 10% of these participants had confidence

levels of ―9,‖ while an additional

*Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’ ‘M,’ ‘I,’ and ‘G’ to indicate the terrorist, rebel, militia, insurgent, and group labels respectively

54

19% had a confidence level of ―8.‖ The higher levels of analytical confidence are possible indicators that these labels cause a stronger sense of certainty among analysts when assessing the severity of a possible attack.

Question 12: Effect of an Overall Increase in Activity Statistical analysis of participants‘ assessment of how indications of an overall increase in activity affect the likelihood of an attack (Question #12) resulted in statistical significance. The Scheffe test revealed the cause of the statistical significance to be between the ‗group‘ label (control group) and the ‗rebel‘ label. All labels meet the tests for normality and equal In Figure 4.10

variance

assumptions.

addition, while outliers were present for the ‗insurgent,‘ ‗militia,‘ and ‗rebel‘ labels, normality was still maintained. The doughnut graph (Figure 4.10) comparing the participants‘ responses regarding the effect of increased activity on
*Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’ ‘M,’ ‘I,’ and ‘G’ to indicate the terrorist, rebel, militia, insurgent, and group labels respectively

the likelihood of an attack reaffirms the statistical significance. The graph indicates the presence of ―highly increases the likelihood of an attack‖ assessments (at 2% - however, this may be caused in part by an outlier) for the ‗rebel‘ label, as well as a large volume

55

(19%) of ―Does not affect the likelihood of an attack‖ assessments from participants receiving the ‗group‘ label. In addition to the statistical difference between the ‗group‘ and ‗rebel‘ labels, the doughnut graph also illustrates a fairly large difference with the militia label as well. Three percent of the participants also gave the ‗militia‘ label assessments that the overall increase in activity ―highly increases the likelihood of an attack.‖ Furthermore, the statistical significance level between the ‗group‘ label and the ‗militia‘ label was scored as 0.233 – which was the closer than the ‗insurgent‘ and ‗terrorist‘ labels (see Table 4.1).

Table 4.1

56

While this is by no means statistically significant, it does illustrate a potential area that could use further research since no other labels received this strong of an assessment. The reasoning behind the responses might be due, at least in part, because of the amount of rebel and militia violence in Africa. Increased activity may possibly be a clear

indication that these two particular labels (rebel and possibly militia) are more likely to attack, especially in light of the rebel and militia clashes in places such as Darfur. The difference between the ‗group‘ label compared to the ‗rebel‘ and ‗militia‘ labels reveal the possibility that the ‗group‘ label is less emotive, or has less cognitive biases associated with it, than the other two labels.

57

Question 13: Effect of Imagery Intelligence Indicating Increased Activity Near a Training Base or Command Post Statistical analysis of participants‘ assessment regarding the effect of imagery intelligence (IMINT) indicating increased activity near a training base/command post on the likelihood of an attack (Question #13) resulted in no statistical significance. All labels did satisfy normality and variance testing. Interestingly, the doughnut graph (Figure 4.11) illustrates that this IMINT reporting put less weight on the likelihood that an insurgent group will attack compared to the other labels. This assertion is evidenced by 25% of the participants who received the ‗insurgent‘ label assessing that IMINT ―Does not affect the likelihood of an attack‖ – leaving a gap of 10% between the next closest label, Figure 4.11

‗group.‘ This possibility is most likely attributed to the media reporting of

insurgent attacks in Iraq against US and coalition forces, which possibly
*Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’ ‘M,’ ‘I,’ and ‘G’ to indicate the terrorist, rebel, militia, insurgent, and group labels respectively

leaves the analyst with some sense of vividness bias

toward the seemingly sporadic and fast-past attacks of insurgent groups. If analysts are left with the impression that insurgent groups use a fast-paced and sporadic modus

58

operendi (MO), then IMINT would not likely be an accurate indicator that an attack is likely (unless, of course, the IMINT was streaming in real time to the analyst).

Question 14: Effect of Local Newspaper Reporting an Attack Will Not Occur Statistical analysis of participants‘ assessment regarding how local newspaper reporting that an attack will not occur affects the likelihood of an attack (Question #14) resulted in no statistical significance. Although there were several outliers for all the labels in question, the labels did satisfy all tests for normality and equal variance assumptions. The outliers are more evident when examining the graph comparing the Figure 4.12

respondents‘ Fifty percent

assessments. of the

participants across all the labels in questions indicated that local newspaper reporting ―does not affect the likelihood of
*Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’ ‘M,’ ‘I,’ and ‘G’ to indicate the terrorist, rebel, militia, insurgent, and group labels respectively

an attack‖ (represented in purple in Figure 4.12). In particular, participants who received the ‗terrorist‘ label varied widely in their assessments of the how the newspaper reporting affects the likelihood of an attack. The participants indicated significant responses on both ends of the likelihood continuum, with 10% assessing that the reporting ―decreases the likelihood of an attack,‖ and 7%
59

assessing that the reporting, ―significantly increases the likelihood of an attack.‖ Outlying responses were also significant for the ‗rebel‘ and ‗militia‘ labels (See Figure 4.13).

Figure 4.13

Question 15: Effect of Communications Intelligence Reporting Increased Levels of Anti-Government Rhetoric Statistical analysis of participants‘ assessment regarding the effect of communications intelligence (COMINT) indicating increased levels of anti-government rhetoric on the likelihood of an attack (Question #13) resulted in statistical significance. All labels did satisfy normality testing using the Quantile plot. The Welch test found statistical significance between the different labels. Because variance levels were not equal, the Tamhane Hoc test was needed in order to find what labels were significantly different from each other. This test found a statistically significant difference between the ‗group‘ label and the ‗militia‘ label (See Table 4.2 on page 61).

60

Table 4.2

The difference between the ‗group‘ label and the ‗militia‘ label likely stems from the overwhelming response (88%) that COMINT will, in various levels of likelihood, will be an indicator that an attack is more likely – only 68% of the ‗group‘ label participants rated it the same way. In addition to the difference between the ‗group‘ and ‗militia‘ labels, the ‗insurgent‘ label was close to statistical significance from the ‗militia‘ label, indicating that there might be a possibility that these two labels differ regarding how COMINT is assessed and associated with them. Interestingly, assessments of the

61

‗insurgent‘ label were more similar to the ‗group‘ label than any of the other labels (75% responded that COMINT will, in various levels of likelihood, be an indicator of an attack). Fifty percent of those who received the ‗insurgent‘ label stated that COMINT only ―somewhat increases the likelihood of an attack.‖ Also of interest is Figure 4.14 the fact that none of the participants stated that COMINT will, in any varying level, decrease the likelihood of an

attack.

This is not so and of
*Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’ ‘M,’ ‘I,’ and ‘G’ to indicate the terrorist, rebel, militia, insurgent, and group labels respectively

interesting in

itself, but it is the only piece of evidence (or

data anchor) that evoked either a neutral or positive indication assessment – all other data anchors had both positive and negative likelihood responses.

Question 16: Effect of a Long History of Social Strife and Discontentment Statistical analysis of participants‘ assessment regarding the effect of a long history of social strife and discontentment on the likelihood of an attack (Question #16) resulted in no statistical significance. All labels met the tests for normality and equal variance assumptions. In addition, only a few outliers were present with the ‗insurgent‘ and ‗militia‘ labels, which also met normality assumptions.

62

Although no statistical significance was noted among the labels, analysis of the doughnut graph indicates a fairly large response (24%) of ―does not affect the likelihood of an attack‖ assessments by the participants of the ‗terrorist‘ label. This response percentage is larger compared to the other labels tested, as 16-17% of their participants assessed those labels in the same manner. The ‗rebel‘ and ‗militia‘ Figure 4.15 labels both received a relatively high volume of responses (7%) indicating that the history of the region ―highly increases the In

likelihood of an attack.‖

addition, 83% of participants for the both the ‗rebel‘ and ‗militia‘
*Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’ ‘M,’ ‘I,’ and ‘G’ to indicate the terrorist, rebel, militia, insurgent, and group labels respectively

labels indicated that the history at least ―somewhat increases the

likelihood of an attack.‖ The large volumes for these two labels could possibly be associated with the amount of violence in Africa attributed to rebel and militia groups; however further testing would be necessary to confirm this.

Question 17: Effect of the Region Being a Hot-Bed of Differing Political and Religious Ideologies, Widespread poverty, and Poor Health Conditions Statistical analysis of participants‘ assessment regarding how different political and ideological views, as well as widespread poverty and poor health conditions, affects the likelihood of an attack (Question #17) resulted in no statistical significance. All
63

labels meet the tests for normality and equal variance assumptions. In addition, only one outlier was present for the ‗insurgent‘ and ‗group‘ labels – meeting normality assumptions. Analysis of the doughnut graph indicates that this question may benefit from further testing as the ‗group‘ label responses appear to quite different responses compared to the labels being tested. For example, 17% of the ‗group‘ label participants assessed that this data anchor ―does not affect the likelihood of an attack.‖ This response volume is larger than that of the other labels (9-12% response volume). In addition, the ‗rebel‘ and ‗militia‘ labels had higher volumes of ―highly increases the likelihood of an attack‖ assessments (10% and 7% respectively) compared to the other labels (see Figure 4.16). These responses are Figure 4.16

indicative that the ‗group‘ label is more neutrally emotive than the other labels. addition, the ‗rebel‘ In and

‗militia‘ labels draw out more cognitive biases, or are at least more emotive, than the other labels tested. While these
*Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’ ‘M,’ ‘I,’ and ‘G’ to indicate the terrorist, rebel, militia, insurgent, and group labels respectively

results are not statistically significant, they do correspond with the statistically significant, and other findings, in this experiment – that the ‗rebel‘ and ‗militia‘ labels are more likely to engage in violent

64

activities in the context of Africa and other data anchors included in the mock intelligence report. Questions 18&19: Ranking of the Labels Based on Their Likelihood to Engage in Violence and Their Likelihood to Execute Severe Attacks

To rank the labels by their levels of violence and severity (Questions 18 and 19 respectively), the scoring

Figure 4.17

averages (ranked from 1 to 4) for the labels were taken and For

compared to each other.

levels of violence, the labels (in order from most to least violent) were assessed as: Insurgent, Rebel, Terrorist, and Militia. The severity assessments were nearly the inverse of the violence ranking, being ranked (most to least) as: Terrorist, Militia, Rebel, and Insurgent.

65

CONCLUSION Trends in Key Findings My hypothesis that these particular labels have significant meaning, and many individuals have a preconceived idea, or cognitive biases, regarding the kinds of actions each of these particular groups conduct must be rejected at this time due to an overall lack in statistical significance across the labels. There are two main possibilities that might explain why the hypothesis was disproved. The first possibility is that, in general, cognitive biases do not apply actually to these labels. This is the most desired possibility because, if this were true, intelligence analysts do not allow labels to convolute the importance of evidence, or in other words, they avoid the label as an anchoring bias. The other possibility is that having the setting of the intelligence report focus around Africa, because of the presence of all the labels and the amount of violence they engage in, allowed for distinct similarities among the assessments for the labels in question. If this were true, it is possible that the participants had similar assessments about this specific scenario; but if given a different context, analysts may have different responses. I posit that the latter is more likely to be true. However, further in-depth research will be necessary to fully conclude that labels do or do not have an effect on intelligence analysis. Although the hypothesis must be rejected, some important conclusions can be drawn out from the findings. First, the statistically significant results must be discussed. Statistical testing revealed that the ‗rebel‘ label differed from the control group (labeled as ‗group‘) when participants assessed the effect of an overall increase in activity on the likelihood of an attack. The difference is indicative that the ‗rebel‘ label is perceived as a
66

greater threat than the other labels when reporting indicates an overall increase in activity. The fact that the other labels were not statistically significant is likely attributed to the cognitive biases toward rebel group activity in Africa. Cognitive biases and heuristics such as vividness bias and the availability heuristic might play into the perceived actions of rebel groups in Africa. In addition, illusory correlation biases (the assumption that there is a relationship between a particular action and an effect) might also be skewing the results. The same biases and heuristics could also be affecting the responses of how COMINT affects the likelihood of an attack for militia groups. Statistical testing also found significance between the ‗group‘ label and the ‗militia‘ label for this data anchor. This difference could possibly be attributed to the assumption that all or most African militia groups use technology that could be intercepted by communication collection methods. If analysts presume that militia groups use more sophisticated communications equipment, this might explain why the ‗militia‘ label was statistically significant from the ‗group‘ label. For both questions that had statistical significance, further research will be necessary to accurately answer why they were significantly different. Analysis of the responses across all questions displayed possible trends for some of the labels. In general, the ‗group‘ label had a tendency to evoke assessments

suggesting that this label is less likely than the other labels to engage in acts of violence. Specifically, compared to the other labels, there was a trend for the ‗group‘ label to be assessed on lower levels for the overall likelihood of an attack and for the success of an attack. In addition, data anchors such as the increased activity, the COMINT, and the ideology/social issues tended to have less of an impact on the likelihood of an attack for

67

the ‗group‘ label. This trend indicates that the ‗group‘ label is likely to have fewer biases associated with it, or at a minimum, is less emotive than the other labels tested. In contrast to the ‗group‘ label, the ‗rebel‘ label and the ‗militia‘ label showed trends of being assessed as the most likely labels to engage of acts of violence. The ‗rebel‘ label showed greater likelihood in areas where participants assessed the overall likelihood of an attack (as well more confidence in this assessment) and the severity of an attack (and its associated analytic confidence). In addition, this label was assessed to be more likely to attack with data anchors such as increased activity, history, and ideology/socioeconomics. The ‗militia‘ label showed greater likelihood in areas where participants assessed the severity of an attack. In addition, this label was assessed to be more likely to attack with data anchors such as increased activity, COMINT, history, and ideology/socioeconomics. Table 5.1 illustrates questions in which the labels either had a positive response (red, and ranked with ―-1‖), neutral response (yellow, and ranked with ―0‖), or negative (green, and ranked with ―1‖) response to the assessment of the situation or the effect of the data anchor. The positive responses indicate an increased likelihood, while the negative responses indicate decreased likelihood.
Table 5.1

Question Group Insurgent Militia Rebel Terrorist P-Value

4 -1 0 0 1 0 0.71

5 0 0 0 1 1 0.514

6 -1 0 0 0 1 0.602

7 0 0 0 0 0 0.485

8 0 -1 0 0 0 0.259

9 0 0 0 0 0 0.713

10 0 0 1 1 -1 0.162

11 0 0 0 1 1 0.444

12 -1 -1 1 1 0 0.017

13 0 -1 0 0 0 0.18

14 0 0 0 0 0 0.886

15 -1 -1 1 0 0 0.011

16 0 0 1 1 -1 0.646

17 -1 0 1 1 0 0.192

68

Supporting the possibility that participants are susceptible to cognitive biases, despite the overall lack of statistical significance, is the vastly different response for the ‗terrorist‘ label in question 10 and question 19. Both questions pertained to the severity of an attack. Question 10 asked participants, basing their assessment on the information provided within the mock intelligence report, to assess how severe an attack would be if it were to occur. Question 19 asked the participants to rank the ‗rebel,‘ ‗militia,‘

‗insurgent,‘ and ‗terrorist‘ labels according to their likely severity levels – since participants were exposed to only one of these labels in the report, their assessments for question 19 would not have stemmed from information contained within the mock intelligence report. Analysis of responses for question 10 across all labels in question indicates that the participants assessed the severity of the labels (from most severe to least severe) to be: militia, rebel, insurgent, and terrorist. This result was found through the sum of

―somewhat severe,‖ ―severe,‖ and ―very severe‖ answers and then comparing the volume of those responses across each of the label being tested. When participants were asked to rank the labels by their level of severity (and out of the context of the data anchors that were provided in the mock intelligence report), the ‗terrorist‘ label was ranked as being the most severe out of all the labels. The other labels followed the same order (militia, rebel, & insurgent) as in question 10, however, the ‗terrorist‘ label jumped from being the least severe in question 10 to the most severe in question 19. The difference in assessing the severity of the ‗terrorist‘ label is a possible

69

indication that the context of the mock intelligence report drew out cognitive biases in the participants‘ analyses. Despite the possible cognitive biases pertaining to the ‗terrorist‘ label, we can conclude the ranking of severity for the other labels is likely to be accurate (since the order matched in both questions 10 and 19). It is likely that analysts believe that attacks by militia groups are generally more severe than attacks by rebel groups, and that rebel attacks are generally more severe than attacks by insurgents. Further testing of the ‗terrorist‘ label will need to be conducted to find out where exactly this label fits on the severity continuum. Another trend discovered in the key findings is among the relationship between the ‗insurgent‘ label and information derived from more sophisticated collection techniques such as IMINT and COMINT. For both questions asking participants to assess the effect of IMINT and COMINT on the likelihood of an attack, participants with the ‗insurgent‘ label indicated that these data anchors have less of an impact for this label when compared to the other labels. This trend may possibly be attributed to a belief that insurgents are less likely than the other labels to use communication devices that can be intercepted, or at a minimum, the communications intelligence reports have less weight for insurgent groups because of their MO. The same may be true for IMINT, this type of intelligence may also be a less reliable source for indicating likely insurgent attacks, at least when compared to the other labels. The variance in participants‘ assessments of how local news reporting contradicting other intelligence affects the likelihood of an attack was also worthy to note. Due to the amount of outlying responses to this question, as well as the variance in

70

assessments across all the labels, is indicative that there is a significant level of uncertainty with regards to how credibility of foreign, local news. This variance could possibly be attributed to the lack of identifying the source of the local news, which would help to explain the variance. Without sourcing the news agency, it appears that some analysts trusted the reporting, while others likely found it to be a deception tactic. Only two of the questions resulted in statistical significance, however, a few of the other questions had lower P-values that were approaching statistical significance. Questions asking about the sophistication of an attack (question #8, P-value of 0.259), the severity of an attack (question #10, P-value of 0.162), the effect of IMINT (question #13, P-value of 0.180), and the effect of differing ideologies and socioeconomic turmoil (question #17, P-value of 0.192) possibly show indications that these data anchors may have more relevance on the analysis of these labels. Research Recommendations Since the findings of my experiment aren‘t entirely conclusive, the continuation of this research is important. There are many avenues that can be taken to continue the research on the effect of labels on analysis, and all possible avenues should be researched, compiled, and compared before any solid conclusions about labeling effects are made. If further research is conducted using my experiment model, I would suggest repeating the same experiment, but changing out the location of scenario. It would be interesting to compare the results of my participants, who had to assess these labels in the context of Africa, against multiple other locations. My initial thought is that if the results of the African setting were compared against a setting such as Iraq, Afghanistan, or
71

Columbia, we would likely see varied responses across the labels. However, due to the size of each experiment (as it compares five different labels), this type of experiment might be difficult to execute and may take the longest amount of time to complete. Another option, which might prove to be quicker and reduce the biases associated with the setting of a scenario, would be to conduct the same experiment but change out the African setting for an imaginary country. This experiment could be quickly

conducted using the same data anchors and questions that were used in my experiment. The best option, in my post-experiment opinion, would be to use a Bayesian statistical model to assess the weight given to a label when analyzing intelligence. Again, all five labels would have to be tested separately and no specific setting should be included. Rather than having participants read a mock intelligence report and then answer questions, individual pieces of evidence (data anchors) would be presented to the participant one at a time. The participants would then have to assess the likelihood of an attack (on a 1-10 scale) given that particular piece of evidence (or given the next additional piece of evidence). If this methodology were used, two main things could be analyzed. First, the researcher would be able to accurately compare the weight given to the label for each piece of evidence presented. Second, after computing the responses for all pieces of evidence given (using Bayesian statistics), an overall assessment, in numerical form, for each label would be evident. The assessments for each label could then be compared against each other using the statistical methods used in this experiment such as the 1-way ANOVA test. One final option for further research would be to use a KISS (Keep It Simple Stupid) approach. This could be conducting with questions asking the participants to

72

rank the labels according to various characteristics (i.e. severity, success, sophistication, etc.). Although this method may be simple in form, it is not as robust as other methods such as Bayes. As a way to provide more pointed information about the groups being tested, as well as possibly simplifying the research, future studies should consider looking at only one group of intelligence professionals at a time. For example, consider conducting the same research that was used in this study, but rather than trying to incorporate multiple intelligence professions in one study, use only intelligence analysts in the field of national security, or law enforcement, or military, or students, etc. If research was conducted in this manner, there is great likelihood that the researcher will be able to effectively examine the variances among participants‘ assessments by experience level. Research delving into the analytical differences among analysts and categorized by experience level, would provide invaluable insight into the intelligence profession.

Moving Forward Cognitive biases and heuristics will continue to be an obstacle for the intelligence analyst to overcome. Since biases and heuristics can have a negative effect on the way we think about the information presented to us, it is in the best interest of the analyst to ensure that he or she has done everything possible to reduce bias and increase analytic accuracy. Richards Heuer has suggested methods such as Analysis of Competing

Hypotheses as a means to assist the intelligence analyst in the battle to overcome bias.96 Methods such as ACH, as well as other sound and structured methodologies, prove to be the best approach to reduce bias, increase transparency, and improve analysis.
96

Heuer, Richards., Psychology of Intelligence Analysis, (Nova Science 1st ed, 2005), Ch 8.

73

If intelligence analysts make a habit out of using structured methodologies to evaluate the information presented to them, it will force them to recognize alternate possibilities that are often hidden by bias. Once all options are out-on-the-table, the methodologies will aid the analyst in making the best possible assessment. Lastly, it is important for analysts to be as specific as possible when faced with a term that labels. Whenever possible, rather than using a term such as ‗terrorist,‘ or ‗rebel,‘ analysts should try to use the recognized name of the group, such as Al Qaida or the Justice and Equality Movement (JEM). In addition, when faced with larger reports containing multiple pieces of evidence, analysts should attempt to break the report up into its parts, separating each piece of evidence from the whole. Each piece of evidence should then be analyzed individually, without the context of the label. After this has been done, all pieces of evidence can then be compiled and analyzed as a whole. When analysts take away labels (or replace them with specific names) and focus more on the evidence by analyzing them individually, the better the chances will be for reducing cognitive bias, avoiding heuristics, and improving intelligence – and this should always be a primary goal for the analyst.

REFERENCES

22 U.S.C. 2656f (d). "Chapter 7 -- Legislative Requirements and Key Terms." U.S.Department of State. http://www.state.gov/s/ct/rls/crt/2007/103715.htm (accessed March 30, 2009). Anti Defamation League. "The Quiet Retooling of the Militia Movement: Recruitment." ADL: Fighting Anti-Semitism, Bigotry and Extremism. http://www.adl.org/extremism/Militia/Recruitment.asp?m_flipmode=4 (accessed April 2, 2009).

74

Bartlett, F.C., Remembering: A Study in Experimental and Social Psychology, (Cambridge, England: Cambridge UP, 1932). BBC. "Sudan's Shadowy Arab Militia." BBC NEWS | News Front Page. http://news.bbc.co.uk/1/hi/world/africa/3613953.stm (accessed April 06, 2009). Central Intelligence Agency, Guide to the Analysis of Insurgency, C05332177. Washington DC,; Central Intelligence Agency, 2. Released January 05, 2009. http://www.fas.org/irp/cia/product/insurgency.pdf "CGI Scripts: Random Link." CGI Spy. www.cgispy.com/scripts/rlink.html (accessed April 21, 2009). CNN. "Janjaweed Militia: News & Videos about Janjaweed Militia - CNN.com." CNN.com - Breaking News, U.S., World, Weather, Entertainment & Video News. http://topics.cnn.com/topics/janjaweed_militia (accessed April 07, 2009). Conlisk, John. ―Bounded Rationality and Market Fluctuations.‖ Journal of Economic Behavior & Organization 29. No. 2 (1996) 233-250. Cronin, Kurth . "Behind the Curve: Globalization and International Terrorism." International Security, 27(3) (Winter 2002-2003). "Data on Armed Conflict - CSCW." PRIO - International Peace Research Institute, Oslo. http://www.prio.no/CSCW/Datasets/Armed-Conflict/ & http://www.prio.no/sptrans/1664678440/Codebook.pdf (accessed April 15, 2009). FBI, Counterterrorism Threat Assessment and Warning Unit, National Security Division. ―Terrorism in the United States.‖ (1996): 7-8. www.fas.org/irp/threat/fbi_terror_96.pdf (accessed April 9, 2009). Gigerenzer, Gerd., ―Fast and Frugal Heuristics: The Tools of Bounded Rationality.‖ In The Blackwell Handbook of Judgment and Decision Making edited by Derek J. Koehler and Nigel Harvey, (Malden, Ma: Blackwell Publishing, 2004). Goffman, E., Frame Analysis: An Essay on the Organization of Experience, (NY: Harper, 1974). Gurr, Ted R.. Why Men Rebel. Princeton: Princeton University Press, 1971. Haider-Markel, Donald P., and Sean P. O'Brien. "Creating a "Well Regulated Militia": Policy Responses to Paramilitary Groups in the American States." Political Research Quarterly 50(3) (1997). Heuer, Richards., Psychology of Intelligence Analysis, (Nova Science 1st ed, 2005).
75

Hoffman, Bruce. Inside Terrorism. New York: Cambridge University Press, 1998. ―insurgent, a. and n." The Oxford English Dictionary. 2nd ed. 1989. OED Online. Oxford University Press. 26 Aug. 2008 http://dictionary.oed.com/cgi/entry/50118488. Jacobellis v. Ohio, 378 U. S. 184 (1964) Johnston, Rob., ―Analytic Culture in the US Intelligence Community: An Ethnographic Study.‖ (Washington D.C.: Center for the Study of Intelligence, Central Intelligence Agency, 2005). Chapter 2. https://www.cia.gov/library/center-for-thestudy-of-intelligence/csi-publications/books-and-monographs/analytic-culture-inthe-u-s-intelligence-community/chapter_2_research.htm Jongman, A.J., and Alex P. Schmid. Political Terrorism: A new Guide to Actors, Authors, Concepts, Data Bases, Theories, and Literature. New Brunswick: Transaction Publishers, 1988. Kahneman, D. & Tversky, A. (eds.), Choices, Values and Frames. (New York, NY: Russell Sage Foundation & Cambridge University Press, 2000), ix-xvii Kitonga, Nzamba. "The Fight Against Terrorism in East Africa: Comments and Observations." East Africa Law Society Conference On ―Globalization And Terrorism: New Threats To Regional Integration‖, East Africa Law Society, Entebbe, Uganda, October 10, 2003. Klein, G. et al, ―A Data/Frame Theory of Sensemaking.‖ In Expertise out of Context; Proceedings of the Sixth International Conference on Naturalistic Decision Making, edited by R.R. Hoffman. (Mahwah, NJ: Erlbaum, 2004). Kronenwetter, Michael. Terrorism: A Guide to Events and Documents. New York: Greenwood Press, 2004. vii. ―militia, n." The Oxford English Dictionary. Draft Revision 2008. OED Online. Oxford University Press. 26 Aug. 2008 http://dictionary.oed.com/cgi/entry/00309562. Minsky, M., ―A Framework for Representing Knowledge.‖ In The Psychology of Computer Vision, edited by P. Winston (New York, NY: McGraw-Hill, 1975). Mynatt, C.R., Doherty, M.E., & Tweeney, R. D. Confirmation Bias in a Simulated Research Environment. Quarterly Journal of Experimental Psychology, 29, 85-95 (1977). Nelson, Debra L,. & Campbell, James Quick., Organizational Behavior: Foundations, Realities, & Challenges, 5th ed. (Mason, OH.: Thomson South-Western. 2006).

76

O'Neill, Bard E.. Insurgency and Terrorism: From Revolution to Apocalypse; 2nd Ed., Revised. Washington, D.C.: Potomac Books, 2005. O'Rourke, Matthew. "The Impact Of The ‗War On Terrorism‘ On Internal Conflicts." The Ploughshares Monitor 26 (2005), http://www.ploughshares.ca/libraries/monitor/monm05d.htm. Patterson, E.S., Woods, D.D., Sarter, N.B., & Watts-Perotti, J.C. Patterns in Cooperative Cognition. In COOP ‘98, Third International Conference on the Design of Cooperative Systems. 1998. Pillar, Paul R.. Terrorism and U.S. Foreign Policy. Washington, D.C.: Brookings Institution Press, 2001. Pollard, Kristine. "Jeffrey Welgan Thesis." Institute for Intelligence Studies at Mercyhurst College. http://74.125.95.132/search?q=cache:acF9XiY2Ly0J:www.cirat.com/welganthesi s.php+http://www.cirat.com/welganthesis.php&cd=1&hl=en&ct=clnk&gl=us&cli ent=firefox-a (accessed April 21, 2009). Pollard, Kristine. "Jeffrey Welgan Thesis - Thank You." Institute for Intelligence Studies at Mercyhurst College. http://www.cirat.com/welganthanks.php (accessed April 21, 2009). ―rebel.‖ (2009). In Merriam-Webster Online Dictionary. Retrieved April 15, 2009, http://www.merriam-webster.com/dictionary/rebel "rebel, a. and n.1" The Oxford English Dictionary. 2nd ed. 1989. OED Online. Oxford University Press. 26 Aug. 2008 http://dictionary.oed.com/cgi/entry/50198771. Research Methods Knowledge Base. "Two-Group Experimental Designs." Social Research Methods. http://www.socialresearchmethods.net/kb/expsimp.php (accessed April 20, 2009).

Reuters. "Sudan Man Accused Of Aiding Darfur war Crime Court." Reuters.com - World News, Financial News, Breaking US & International News. http://www.reuters.com/article/homepageCrisis/idUSLM432733._CH_.2400 (accessed April 07, 2009). Rudolph, Jenny. W. Into the big muddy and out again. Unpublished Doctoral Thesis, (Boston College, Boston, MA., 2003). Ruvalcaba, Jim. "Understanding Iraq's Insurgency." Al Nakhlah Article 7 (Spring 2004).

77

Schoemaker, P & Russo, J. ―Managing Frames to Make Better Decisions.‖ In Wharton on Making Decisions, edited by S. Hoch and H. Kunreuther. (New York: John Wiley & Sons, 2001). Slovic, P., ―From Shakespeare to Simon: Speculations and Some Evidence About Man‘s Ability to Process Information.‖ Oregon Research Institute Monograph 12, no. 2 (1972). Soman, Dilip., ―Framing, Loss Aversion, and Mental Accounting.‖ In The Blackwell Handbook of Judgment and Decision Making edited by Derek J. Koehler and Nigel Harvey, (Malden, Ma: Blackwell Publishing, 2004). "Survey Methods." School of Information - University of Texas. http://www.ischool.utexas.edu/~palmquis/courses/survey.html (accessed April 20, 2009). ―terrorist." The Oxford English Dictionary. 2nd ed. 1989. OED Online. Oxford University Press. 26 Aug. 2008 http://dictionary.oed.com/cgi/entry/50249599. Thaler, R. & Johnson, E., ―Gambling With the House Money and Trying to Break Even: The Effects of Prior Outcomes on Risky Choice.‖ Management Science 36 (1990): 643-60. U.S. Department of State. "E.O. 13224: Identified Terrorists and Groups." U.S. Department of State. http://www.state.gov/s/ct/rls/fs/2001/6531.htm (accessed April 16, 2009). Wheaton, Kristan. "Surveys, Surveys, Surveys! (It's Research Season...)." Sources And Methods. http://sourcesandmethods.blogspot.com/2009/01/surveys-surveyssurveys-its-research.html (accessed April 21, 2009). Woldemariam, Michael. "Why Rebels Collide: Factionalism and Fragmentation in African Insurgencies." Africa: Conflict Transformation/Peacebuilding, Woodrow Wilson Center, Washington DC, August 19, 2008.

78

Appendices

79

Appendix A Participation Consent Form
JEFFREY WELGAN THESIS Participation Consent Form The purpose of this research is to test whether or not any cognitive biases exist among intelligence professionals and students when they are given a common intelligence report. Your participation involves a short instruction section online, followed by a short mock-intelligence report. Following the intelligence report will be an online survey comprised of 19 questions. This process should take no longer than 20 minutes. Your identity will remain completely anonymous to the researcher and throughout the entire thesis process. There are no foreseeable risks or discomforts associated with your participation in this study. Participation is voluntary and you have the right to opt out of the study at any time for any reason without penalty. By submitting your answers to the survey, you hereby acknowledge that your involvement in this research is voluntary and agree that the use of your data can be used for the purpose of this research. In addition, by clicking on the link below, you are confirming that you are at least 18 years of age or have parental consent to participate in this survey. If you choose to participate, you will be directed to freeonlinesurveys.com. freeonlinesurveys.com does not collect any personal data, or IP addresses from its participants. Click here to participate in the survey. (This link will take you to an intelligence report and survey- PLEASE NOTE: if the survey does not appear when you click on the link above - please press F5 until it does.) If you have any further questions about this research feel free to contact me at jeff.welgan@gmail.com Research at Mercyhurst College which involves human participants is overseen by the Institutional Review Board. Questions or problems regarding your rights as a participant should be addressed to Tim Harvey; Institutional Review Board Chair; Mercyhurst College; 501 East 38th Street; Erie, Pennsylvania 16546-0001; Telephone (814) 824-3372. tharvey@mercyhurst.edu

80

Appendix B Participation Debriefing Statement JEFFREY WELGAN THESIS - THANK YOU
Thank you for participating in this research process. I appreciate your contribution and willingness to support the student research process. It is of utmost importance that you do not discuss the content of your scenario, or your answers to the survey, with any possible future participants. Discussion with any prospective participants may manipulate or alter their analysis. If you know of any other analysts that would be interested or willing to participate in this research, please send them to: http://www.cirat.com/welganthesis.php The purpose of this study was to determine whether or not cognitive biases exist among intelligence professionals and students. Particular focus is being given on the labels that are commonly used in many of today's intelligence reports – these labels are terrorist, rebel, militia, insurgent, and group. Five intelligence reports are being distributed as a part of this research, all identical to one another with the exception of which label is being used. Your answers to the questionnaire will be compiled with other participants' answers that have received the same intelligence report. The different intelligence reports will then be compared against other participants' answers to see whether or not analysis changes solely based on which labels were used. It is of utmost importance that you do not discuss the content of your scenario, or your answers to the survey, with any possible future participants. Discussion with any prospective participants may manipulate or alter their analysis. This research is important because the use, or misuse, of particular labels can directly affect US government action or inaction. Because Intelligence assists in the decisions of policy makers, it is important to be precise and cautious in our use of such labeling terms, and this is the purpose and significance of my research into this topic. Thank you once again for your participation. If you have any further question about my research you can contact me at jeff.welgan@gmail.com.

81

Appendix C Institutional Review Board (IRB) Approval

Insert Approval Page here

82

Appendix D Scenario 1: Group Activity (Control Group)

**THIS IS A MOCK SCENARIO. THE INFORMATION INCLUDED IS FABRICATED AND NOT BASED ON ANY FACTUAL EVENTS**

SUBJECT: GROUP ACTIVITY 1. Summary. Northeast Africa and the Horn of Africa (HOA) regions have multiple groups. Groups in the region have varying levels of violence and success. Some groups remain benign with fluctuations in activity, while other groups consistently maintain high levels of violence. Over the past month, multiple sources have indicated an increase in activity for an unidentified group. The American Embassy in Djibouti has been unable to assess the likelihood of an attack on Djibouti interests in the region (due to the wide variety of groups) and is reaching out to the intelligence community for outside perspective and analysis. 2. Over the past month there has been an overall increase in group activity. Imagery intelligence has indicated an increase of human activity near what appears to be a training base or command post of the suspected group. Imagery analysts report that the group appears to have small arms and other weapons however, due to poor satellite coverage and cloud cover, analytic confidence in the imagery has decreased. Several local newspapers have indicated that the group ‖does not plan to incite, or participate in, any violence in the future despite discontentment with the current establishment.‖ Newspapers have added that the group was considering a peaceful protest, however no specific dates have been set. 3. Recent COMINT has indicated an increase in anti-government rhetoric from the group based upon the alleged social injustice and oppression. The Northeast and HOA regions have a long history of social strife and discontentment. These social tensions stem from the region being a hot-bed of differing political and religious ideologies, as well as widespread poverty and poor health conditions. 4. Comment. In order to assist us in analyzing the likelihood of an attack on Djibouti interests, and the level of sophistication if an attack were to occur, please fill out the attached survey. Thank you for your help and support.

END REPORT

83

Appendix E Scenario 2: Rebel Activity
**THIS IS A MOCK SCENARIO. THE INFORMATION INCLUDED IS FABRICATED AND NOT BASED ON ANY FACTUAL EVENTS**

SUBJECT: REBEL ACTIVITY 1. Summary. Northeast Africa and the Horn of Africa (HOA) regions have multiple rebel groups. Rebels in the region have varying levels of violence and success. Some rebel groups remain benign with fluctuations in activity, while other rebel groups consistently maintain high levels of violence. Over the past month, multiple sources have indicated an increase in activity for an unidentified rebel group. The American Embassy in Djibouti has been unable to assess the likelihood of an attack on Djibouti interests in the region (due to the wide variety of rebel groups) and is reaching out to the intelligence community for outside perspective and analysis. 2. Over the past month there has been an overall increase in rebel activity. Imagery intelligence has indicated an increase of human activity near what appears to be a training base or command post of the suspected rebels. Imagery analysts report that the rebels appear to have small arms and other weapons however, due to poor satellite coverage and cloud cover, analytic confidence in the imagery has decreased. Several local newspapers have indicated that the rebels ―do not plan to incite, or participate in, any violence in the future despite discontentment with the current establishment.‖ Newspapers have added that the rebels were considering a peaceful protest, however no specific dates have been set. 3. Recent COMINT has indicated an increase in anti-government rhetoric from the rebels based upon the alleged social injustice and oppression. The Northeast and HOA regions have a long history of social strife and discontentment. These social tensions stem from the region being a hot-bed of differing political and religious ideologies, as well as widespread poverty and poor health conditions. 4. Comment. In order to assist us in analyzing the likelihood of an attack on Djibouti interests, and the level of sophistication if an attack were to occur, please fill out the attached survey. Thank you for your help and support.

END REPORT

84

Appendix F Scenario 3: Militia Activity
**THIS IS A MOCK SCENARIO. THE INFORMATION INCLUDED IS FABRICATED AND NOT BASED ON ANY FACTUAL EVENTS**

SUBJECT: MILITIA ACTIVITY 1. Summary. Northeast Africa and the Horn of Africa (HOA) regions have multiple militias. Militias in the region have varying levels of violence and success. Some militias remain benign with fluctuations in activity, while other militias consistently maintain high levels of violence. Over the past month, multiple sources have indicated an increase in activity for an unidentified militia. The American Embassy in Djibouti has been unable to assess the likelihood of an attack on Djibouti interests in the region (due to the wide variety of militias) and is reaching out to the intelligence community for outside perspective and analysis. 2. Over the past month there has been an overall increase in militia activity. Imagery intelligence has indicated an increase of human activity near what appears to be a training base or command post of the suspected militia. Imagery analysts report that the militia appears to have small arms and other weapons however, due to poor satellite coverage and cloud cover, analytic confidence in the imagery has decreased. Several local newspapers have indicated that the militia ‖does not plan to incite, or participate in, any violence in the future despite discontentment with the current establishment.‖ Newspapers have added that the militia was considering a peaceful protest, however no specific dates have been set. 3. Recent COMINT has indicated an increase in anti-government rhetoric from the militia based upon the alleged social injustice and oppression. The Northeast and HOA regions have a long history of social strife and discontentment. These social tensions stem from the region being a hot-bed of differing political and religious ideologies, as well as widespread poverty and poor health conditions. 4. Comment. In order to assist us in analyzing the likelihood of an attack on Djibouti interests, and the level of sophistication if an attack were to occur, please fill out the attached survey. Thank you for your help and support.

END REPORT

85

Appendix G Scenario 4: Insurgent Activity
**THIS IS A MOCK SCENARIO. THE INFORMATION INCLUDED IS FABRICATED AND NOT BASED ON ANY FACTUAL EVENTS**

SUBJECT: INSURGENT ACTIVITY 1. Summary. Northeast Africa and the Horn of Africa (HOA) regions have multiple insurgents. Insurgents in the region have varying levels of violence and success. Some insurgents remain benign with fluctuations in activity, while other insurgents consistently maintain high levels of violence. Over the past month, multiple sources have indicated an increase in activity for an unidentified insurgent group. The American Embassy in Djibouti has been unable to assess the likelihood of an attack on Djibouti interests in the region (due to the wide variety of insurgent groups) and is reaching out to the intelligence community for outside perspective and analysis. 2. Over the past month there has been an overall increase in insurgent activity. Imagery intelligence has indicated an increase of human activity near what appears to be a training base or command post of the suspected insurgents. Imagery analysts report that the insurgents appear to have small arms and other weapons however, due to poor satellite coverage and cloud cover, analytic confidence in the imagery has decreased. Several local newspapers have indicated that the insurgents ―do not plan to incite, or participate in, any violence in the future despite discontentment with the current establishment.‖ Newspapers have added that the insurgents were considering a peaceful protest, however no specific dates have been set. 3. Recent COMINT has indicated an increase in anti-government rhetoric from the insurgents based upon the alleged social injustice and oppression. The Northeast and HOA regions have a long history of social strife and discontentment. These social tensions stem from the region being a hot-bed of differing political and religious ideologies, as well as widespread poverty and poor health conditions. 4. Comment. In order to assist us in analyzing the likelihood of an attack on Djibouti interests, and the level of sophistication if an attack were to occur, please fill out the attached survey. Thank you for your help and support.

END REPORT

86

Appendix H Scenario 5: Terrorist Activity
**THIS IS A MOCK SCENARIO. THE INFORMATION INCLUDED IS FABRICATED AND NOT BASED ON ANY FACTUAL EVENTS**

SUBJECT: TERRORIST ACTIVITY

1. Summary. Northeast Africa and the Horn of Africa (HOA) regions have multiple terrorist groups. Terrorists in the region have varying levels of violence and success. Some terrorists groups remain benign with fluctuations in activity, while other terrorist groups consistently maintain high levels of violence. Over the past month, multiple sources have indicated an increase in activity for an unidentified terrorist group. The American Embassy in Djibouti has been unable to assess the likelihood of an attack on Djibouti interests in the region (due to the wide variety of terrorist groups) and is reaching out to the intelligence community for outside perspective and analysis. 2. Over the past month there has been an overall increase in terrorist activity. Imagery intelligence has indicated an increase of human activity near what appears to be a training base or command post of the suspected terrorists. Imagery analysts report that the terrorists appear to have small arms and other weapons however, due to poor satellite coverage and cloud cover, analytic confidence in the imagery has decreased. Several local newspapers have indicated that the terrorists ―do not plan to incite, or participate in, any violence in the future despite discontentment with the current establishment.‖ Newspapers have added that the terrorists were considering a peaceful protest, however no specific dates have been set. 3. Recent COMINT has indicated an increase in anti-government rhetoric from the terrorists based upon the alleged social injustice and oppression. The Northeast and HOA regions have a long history of social strife and discontentment. These social tensions stem from the region being a hot-bed of differing political and religious ideologies, as well as widespread poverty and poor health conditions. 4. Comment. In order to assist us in analyzing the likelihood of an attack on Djibouti interests, and the level of sophistication if an attack were to occur, please fill out the attached survey. Thank you for your help and support.

END REPORT

87

Appendix I The Effect of Labels on Analysis Survey
Welgan Thesis: [Label was inserted here] Activity

Thank you for taking time to participate in my survey. Your participation is important to my thesis and to the Intelligence Community.

[Intelligence Report was inserted here]

Basing your responses from the intelligence report you were given, please respond to the evidence by stating the likelihood of an attack. Thank you once again, Jeff Welgan

1) Are you now, or have you ever been, an intelligence analyst? Students without intelligence experience (i.e. a minimum of internship experience) should answer this question "NO." a. b. Yes No

2) What area of Intelligence are you most associated with? a. b. c. d. e. f. National Security Military Law Enforcement Competitive Intelligence Student of Intelligence Other

3) How many years of intelligence-related experience do you have? a. b. c. d. e. f. g. h. i. 0 0-1 1-3 3-5 5-10 10-15 15-20 20-25 greater than 25

4) Overall, based on the information you have available to you, what is the likelihood of an attack on Djibouti interests? a. b. c. d. e. Virtually certain an attack will occur Highly likely Likely Somewhat likely Somewhat greater than even

88

f. g. h. i. j.

Somewhat less than even Somewhat unlikely Unlikely Highly unlikely Virtually certain an attack will not occur

5) Given the information you have received, what is your approximate level of analytic confidence for your estimate in the above question? a. b. c. d. e. f. g. h. i. j. 10 - Extremely High 9 8 7 6 5 4 3 2 1 - Extremely Low

6) What is the likelihood that an attack would be successful? a. b. c. d. e. f. g. h. i. j. Virtually certain an attack will be successful Highly likely Likely Somewhat likely Somewhat greater than even Somewhat less than even Somewhat unlikely Unlikely Highly unlikely Virtually certain an attack will not succeed

7) What is your analytic confidence for the above question? a. b. c. d. e. f. g. h. i. j. 10 - Extremely High 9 8 7 6 5 4 3 2 1 - Extremely Low

8) What is the likelihood than an attack would be sophisticated? a. b. c. d. e. f. g. h. i. j. Virtually certain an attack will be sophisticated Highly likely Likely Somewhat likely Somewhat greater than even Somewhat less than even Somewhat unlikely Unlikely Highly unlikely Virtually certain an attack will not be sophisticated

9) What is your analytic confidence for the above question? a. 10 - Extremely High

89

b. c. d. e. f. g. h. i. j.

9 8 7 6 5 4 3 2 1 - Extremely Low

10) If an attack were to occur, how severe do you think it would be? a. b. c. d. Very severe Severe Somewhat severe Not severe

11) What is your analytic confidence for the above question? a. b. c. d. e. f. g. h. i. j. 10 - Extremely High 9 8 7 6 5 4 3 2 1 - Extremely Low

12) An overall increase in group activity... a. b. c. d. e. f. g. h. i. Highly increases the likelihood of an attack Significantly increases the likelihood of an attack Increases the likelihood of an attack Somewhat increases the likelihood of an attack Does not affect the likelihood of an attack Somewhat decreases the likelihood of an attack Decreases the likelihood of an attack Significantly decreases the likelihood of an attack Highly decreases the likelihood of an attack

13) Imagery indicating an increase in human activity near the training base or command post of the group.... a. b. c. d. e. f. g. h. i. Highly increases the likelihood of an attack Significantly increases the likelihood of an attack Increases the likelihood of an attack Somewhat increases the likelihood of an attack Does not affect the likelihood of an attack Somewhat decreases the likelihood of an attack Decreases the likelihood of an attack Significantly decreases the likelihood of an attack Highly decreases the likelihood of an attack

14) The local newspapers' indication that the group "does not plan to incite, or participate in, any violence in the future despite discontentment with the current establishment... a. b. c. d. e. Highly increases the likelihood of an attack Significantly increases the likelihood of an attack Increases the likelihood of an attack Somewhat increases the likelihood of an attack Does not affect the likelihood of an attack

90

f. g. h. i.

Somewhat decreases the likelihood of an attack Decreases the likelihood of an attack Significantly decreases the likelihood of an attack Highly decreases the likelihood of an attack

15) Recent COMINT indicating an increase in anti-government rhetoric from the group based upon social injustice and oppression... a. b. c. d. e. f. g. h. i. Highly increases the likelihood of an attack Significantly increases the likelihood of an attack Increases the likelihood of an attack Somewhat increases the likelihood of an attack Does not affect the likelihood of an attack Somewhat decreases the likelihood of an attack Decreases the likelihood of an attack Significantly decreases the likelihood of an attack Highly decreases the likelihood of an attack

16) The long history of social strife and discontentment in Northeast Africa and the HOA region... a. b. c. d. e. f. g. h. i. Highly increases the likelihood of an attack Significantly increases the likelihood of an attack Increases the likelihood of an attack Somewhat increases the likelihood of an attack Does not affect the likelihood of an attack Somewhat decreases the likelihood of an attack Decreases the likelihood of an attack Significantly decreases the likelihood of an attack Highly decreases the likelihood of an attack

17) This region being a hot-bed of differing political and religious ideologies, as well a region of widespread poverty and poor health conditions... a. b. c. d. e. f. g. h. i. Highly increases the likelihood of an attack Significantly increases the likelihood of an attack Increases the likelihood of an attack Somewhat increases the likelihood of an attack Does not affect the likelihood of an attack Somewhat decreases the likelihood of an attack Decreases the likelihood of an attack Significantly decreases the likelihood of an attack Highly decreases the likelihood of an attack

18) Rank the following groups by their likelihood to engage in violence- 1 being most likely, 4 being least likely. a. b. c. d. Militia Terrorist Rebel Insurgent

19) Rank the following by their likelihood of executing severe attacks- 1 being most severe, 4 being least severe. a. b. c. d. Militia Terrorist Rebel Insurgent

91

Statistical Annex

Is there a difference in responses for Q. 4 among different groups? Null: There is no difference in responses for Q. 4 among different groups. Alternative: There is a significant difference in responses for Q. 4 among different groups. There are more than 2 groups to compare so need to use 1-way ANOVA provided assumptions are satisfied. Samples are independent as groups are different from each other and were given separate treatment.

From above table, both Kolmogorov-Smirnov and Shapiro-Wilk show that P-value (given by Sig. column) for each group is less than 0.05. Thus normality assumption is not satisfied and need to look at Normal quantile plots to check normality. Note normality is robust!

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Control.

92

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Insurgent.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Militia.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Rebel.

93

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Terrorist.

Box plot shows no outliers for each group. Thus another requirement of normality is satisfied.

Above table gives Levene‘s test P-value = 0.441 (given by column Sig.). It is larger than α = 0.05 thus equal variance assumption is satisfied. All the assumptions for 1-way ANOVA satisfied.

94

From table ANOVA, F-test value = 0.535 and F-test P-value = 0.710. (F-test P-value = 0.710) > (α = 0.05), thus fail to reject the null hypothesis. Conclusion: There is no significant difference in responses for Q. 4 among different groups at 5% level. Is there a difference in responses for Q. 5 among different groups? Null: There is no difference in responses for Q. 5 among different groups. Alternative: There is a significant difference in responses for Q. 5 among different groups. There are more than 2 groups to compare so need to use 1-way ANOVA provided assumptions are satisfied. Samples are independent as groups are different from each other and were given separate treatment.

95

From above table, both Kolmogorov-Smirnov and Shapiro-Wilk show that P-value (given by Sig. column) for each group is less than 0.05. Thus normality assumption is not satisfied and need to look at Normal quantile plots to check normality. Note normality is robust!

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Control.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Insurgent.

96

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Militia.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Rebel.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Terrorist.

97

Box plot shows several outliers for all the groups except group Insurgent and Rebel. The decision is to keep the outliers as normality is satisfied even with the outliers.

Above table gives Levene‘s test P-value = 0.948 (given by column Sig.). It is larger than α = 0.05 thus equal variance assumption is satisfied. All the assumptions for 1-way ANOVA satisfied.

98

From table ANOVA, F-test value = 0.82 and F-test P-value = 0.514. (F-test P-value = 0.514) > (α = 0.05), thus fail to reject the null hypothesis. Conclusion: There is no significant difference in responses for Q. 5 among different groups at 5% level. Is there a difference in responses for Q. 6 among different groups? Null: There is no difference in responses for Q. 6 among different groups. Alternative: There is a significant difference in responses for Q. 6 among different groups. There are more than 2 groups to compare so need to use 1-way ANOVA provided assumptions are satisfied. Samples are independent as groups are different from each other and were given separate treatment.

From above table, both Kolmogorov-Smirnov and Shapiro-Wilk show that P-value (given by Sig. column) for each group is less than 0.05. Thus normality assumption is not satisfied and need to look at Normal quantile plots to check normality. Note normality is robust!

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Control.

99

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Insurgent.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Militia.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Rebel.

100

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Terrorist.

Box plot shows no outliers for each group. Thus another requirement of normality is satisfied.

Above table gives Levene‘s test P-value = 0.602 (given by column Sig.). It is larger than α = 0.05 thus equal variance assumption is satisfied. All the assumptions for 1-way ANOVA satisfied.

101

From table ANOVA, F-test value = 0.437 and F-test P-value = 0.782. (F-test P-value = 0.782) > (α = 0.05), thus fail to reject the null hypothesis. Conclusion: There is no significant difference in responses for Q. 6 among different groups at 5% level. Is there a difference in responses for Q. 7 among different groups? Null: There is no difference in responses for Q. 7 among different groups. Alternative: There is a significant difference in responses for Q. 7 among different groups. There are more than 2 groups to compare so need to use 1-way ANOVA provided assumptions are satisfied. Samples are independent as groups are different from each other and were given separate treatment.

102

From above table, both Kolmogorov-Smirnov and Shapiro-Wilk show that P-value (given by Sig. column) for each group is less than 0.05. Thus normality assumption is not satisfied and need to look at Normal quantile plots to check normality. Note normality is robust!

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Control.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Insurgent.

103

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Militia.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Rebel.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Terrorist.

104

Box plot shows several outliers for all the groups except group Control, Insurgent and Rebel. The decision is to keep the outliers as normality is satisfied even with the outliers.

Above table gives Levene‘s test P-value = 0.553 (given by column Sig.). It is larger than α = 0.05 thus equal variance assumption is satisfied. All the assumptions for 1-way ANOVA satisfied.

105

From table ANOVA, F-test value = 0.866 and F-test P-value = 0.485. (F-test P-value = 0.485) > (α = 0.05), thus fail to reject the null hypothesis. Conclusion: There is no significant difference in responses for Q. 7 among different groups at 5% level. Is there a difference in responses for Q. 8 among different groups? Null: There is no difference in responses for Q. 8 among different groups. Alternative: There is a significant difference in responses for Q. 8 among different groups. There are more than 2 groups to compare so need to use 1-way ANOVA provided assumptions are satisfied. Samples are independent as groups are different from each other and were given separate treatment.

From above table, both Kolmogorov-Smirnov and Shapiro-Wilk show that P-value (given by Sig. column) for each group is less than 0.05. Thus normality assumption is not satisfied and need to look at Normal quantile plots to check normality. Note normality is robust!

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Control.

106

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Insurgent.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Militia.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Rebel.

107

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Terrorist.

Box plot shows several outliers for all the groups except group Control, Insurgent and Rebel. The decision is to keep the outliers as normality is satisfied even with the outliers.

Above table gives Levene‘s test P-value = 0.123 (given by column Sig.). It is larger than α = 0.05 thus equal variance assumption is satisfied. All the assumptions for 1-way ANOVA satisfied.

108

From table ANOVA, F-test value = 1.331 and F-test P-value = 0.259. (F-test P-value = 0.259) > (α = 0.05), thus fail to reject the null hypothesis. Conclusion: There is no significant difference in responses for Q. 8 among different groups at 5% level. Is there a difference in responses for Q. 9 among different groups? Null: There is no difference in responses for Q. 9 among different groups. Alternative: There is a significant difference in responses for Q. 9 among different groups. There are more than 2 groups to compare so need to use 1-way ANOVA provided assumptions are satisfied. Samples are independent as groups are different from each other and were given separate treatment.

109

From above table, both Kolmogorov-Smirnov and Shapiro-Wilk show that P-value (given by Sig. column) for each group is less than 0.05 except Terrorist group. Thus normality assumption is not satisfied and need to look at Normal quantile plots to check normality for all the groups except Terrorist group. Note normality is robust!

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Control.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Insurgent.

110

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Militia.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Rebel.

Box plot shows several outliers for all the groups except group Control and Rebel. The decision is to keep the outliers as normality is satisfied even with the outliers.

111

Above table gives Levene‘s test P-value = 0.518 (given by column Sig.). It is larger than α = 0.05 thus equal variance assumption is satisfied. All the assumptions for 1-way ANOVA satisfied.

From table ANOVA, F-test value = 0.531 and F-test P-value = 0.713. (F-test P-value = 0.713) > (α = 0.05), thus fail to reject the null hypothesis. Conclusion: There is no significant difference in responses for Q. 9 among different groups at 5% level. Is there a difference in responses for Q. 10 among different groups? Null: There is no difference in responses for Q. 10 among different groups. Alternative: There is a significant difference in responses for Q. 10 among different groups. There are more than 2 groups to compare so need to use 1-way ANOVA provided assumptions are satisfied. Samples are independent as groups are different from each other and were given separate treatment.

112

From above table, both Kolmogorov-Smirnov and Shapiro-Wilk show that P-value (given by Sig. column) for each group is less than 0.05. Thus normality assumption is not satisfied and need to look at Normal quantile plots to check normality. Note normality is robust!

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Control.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Insurgent.

113

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Militia.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Rebel.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Terrorist.

114

Box plot shows an outlier for the group Control. Presence of single outlier is acceptable for normality assumption.

Above table gives Levene‘s test P-value = 0.314 (given by column Sig.). It is larger than α = 0.05 thus equal variance assumption is satisfied. All the assumptions for 1-way ANOVA satisfied.

115

From table ANOVA, F-test value = 1.651 and F-test P-value = 0.162. (F-test P-value = 0.162) > (α = 0.05), thus fail to reject the null hypothesis. Conclusion: There is no significant difference in responses for Q. 10 among different groups at 5% level. Is there a difference in responses for Q. 11 among different groups? Null: There is no difference in responses for Q. 11 among different groups. Alternative: There is a significant difference in responses for Q. 11 among different groups. There are more than 2 groups to compare so need to use 1-way ANOVA provided assumptions are satisfied. Samples are independent as groups are different from each other and were given separate treatment.

From above table, Kolmogorov-Smirnov show that P-value (given by Sig. column) for each group is less than 0.05 except for group Control and Shapiro-Wilk show that P-value (given by Sig. column) for each group is less than 0.05 except for group Terrorist. Thus normality assumption is not satisfied for all the groups except group Control and Terrorist and need to look at Normal quantile plots to check normality. Note normality is robust!

116

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Insurgent.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Militia.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Rebel.

117

Box plot shows two outliers for the group Militia. The decision is to keep the outlier as normality is satisfied even with the outlier.

Above table gives Levene‘s test P-value = 0.737 (given by column Sig.). It is larger than α = 0.05 thus equal variance assumption is satisfied. All the assumptions for 1-way ANOVA satisfied.

118

From table ANOVA, F-test value = 0.936 and F-test P-value = 0.444. (F-test P-value = 0.444) > (α = 0.05), thus fail to reject the null hypothesis. Conclusion: There is no significant difference in responses for Q. 11 among different groups at 5% level. Is there a difference in responses for Q. 12 among different groups? Null: There is no difference in responses for Q. 12 among different groups. Alternative: There is a significant difference in responses for Q. 12 among different groups. There are more than 2 groups to compare so need to use 1-way ANOVA provided assumptions are satisfied. Samples are independent as groups are different from each other and were given separate treatment.

From above table, both Kolmogorov-Smirnov and Shapiro-Wilk show that P-value (given by Sig. column) for each group is less than 0.05. Thus normality assumption is not satisfied and need to look at Normal quantile plots to check normality. Note normality is robust!

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Control.

119

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Insurgent.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Militia.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Rebel.

120

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Terrorist.

Box plot shows an outlier for the group Insurgent, Militia and Rebel. The decision is to keep the outlier as normality is satisfied even with the outlier.

Above table gives Levene‘s test P-value = 0.190 (given by column Sig.). It is larger than α = 0.05 thus equal variance assumption is satisfied. All the assumptions for 1-way ANOVA satisfied.

121

From table ANOVA, F-test value = 3.072 and F-test P-value = 0.017. (F-test P-value = 0.017) < (α = 0.05), thus reject the null hypothesis. Conclusion: There is a significant difference in responses for Q. 12 among different groups at 5% level. Now we need to see where the difference lies. As the each group sample sizes are not same, need to use Scheffe Post Hoc test.

122

From above table, responses for Control and Rebel groups are significantly different from each other for Q. 12 at 5% level (SPSS flags it with an asterisk). Also for this group comparison (Scheffe P-value = 0.036) < (α = 0.05), this means difference is significant. Is there a difference in responses for Q. 13 among different groups? Null: There is no difference in responses for Q. 13 among different groups. Alternative: There is a significant difference in responses for Q. 13 among different groups. There are more than 2 groups to compare so need to use 1-way ANOVA provided assumptions are satisfied. Samples are independent as groups are different from each other and were given separate treatment. 123

From above table, both Kolmogorov-Smirnov and Shapiro-Wilk show that P-value (given by Sig. column) for each group is less than 0.05. Thus normality assumption is not satisfied and need to look at Normal quantile plots to check normality. Note normality is robust!

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Control.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Insurgent.

124

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Militia.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Rebel.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Terrorist.

125

Box plot shows two outliers for the group Rebel. The decision is to keep the outliers as normality is satisfied even with the outliers.

Above table gives Levene‘s test P-value = 0.023 (given by column Sig.). It is less than α = 0.05 thus equal variance assumption is not satisfied. All the assumptions for 1-way ANOVA are not satisfied. We need to use Welch test instead of 1way ANOVA.

126

From the above table Welch-test value = 1.598 and Welch-test P-value = 0.180. (Welch-test P-value = 0.180) > (α = 0.05), thus fail reject the null hypothesis. Conclusion: There is no significant difference in responses for Q. 13 among different groups at 5% level. Is there a difference in responses for Q. 14 among different groups? Null: There is no difference in responses for Q. 14 among different groups. Alternative: There is a significant difference in responses for Q. 14 among different groups. There are more than 2 groups to compare so need to use 1-way ANOVA provided assumptions are satisfied. Samples are independent as groups are different from each other and were given separate treatment.

From above table, both Kolmogorov-Smirnov and Shapiro-Wilk show that P-value (given by Sig. column) for each group is less than 0.05. Thus normality assumption is not satisfied and need to look at Normal quantile plots to check normality. Note normality is robust!

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Control.

127

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Insurgent.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Militia.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Rebel.

128

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Terrorist.

Box plot shows several outliers for all the groups. The decision is to keep the outliers as normality is satisfied even with the outliers.

Above table gives Levene‘s test P-value = 0.565 (given by column Sig.). It is larger than α = 0.05 thus equal variance assumption is satisfied. All the assumptions for 1-way ANOVA satisfied.

129

From table ANOVA, F-test value = 0.288 and F-test P-value = 0.886. (F-test P-value = 0.886) > (α = 0.05), thus fail reject the null hypothesis. Conclusion: There is no significant difference in responses for Q. 14 among different groups at 5% level. Is there a difference in responses for Q. 15 among different groups? Null: There is no difference in responses for Q. 15 among different groups. Alternative: There is a significant difference in responses for Q. 15 among different groups. There are more than 2 groups to compare so need to use 1-way ANOVA provided assumptions are satisfied. Samples are independent as groups are different from each other and were given separate treatment.

130

From above table, both Kolmogorov-Smirnov and Shapiro-Wilk show that P-value (given by Sig. column) for each group is less than 0.05. Thus normality assumption is not satisfied and need to look at Normal quantile plots to check normality. Note normality is robust!

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Control.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Insurgent.

131

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Militia.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Rebel.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Terrorist.

132

Box plot shows one or two outliers for all the groups except control. The decision is to keep the outliers as normality is satisfied even with the outliers.

Above table gives Levene‘s test P-value = 0.013 (given by column Sig.). It is less than α = 0.05 thus equal variance assumption is not satisfied. All the assumptions for 1-way ANOVA are not satisfied. We need to use Welch test instead of 1way ANOVA.

133

From the above table Welch-test value = 3.412 and Welch-test P-value = 0.011. (Welch-test P-value = 0.011) < (α = 0.05), thus reject the null hypothesis. Conclusion: There is a significant difference in responses for Q. 15 among different groups at 5% level. Now we need to see where the difference lies. Need to use Tamhane Post Hoc test as variances are not equal.

From above table, Control and Militia groups are significantly different from each other for Q. 15 at 5% level (SPSS flags it with an asterisk). Also for this group comparison (Tamhane Pvalue = 0.042) < (α = 0.05), this means difference is significant. Is there a difference in responses for Q. 16 among different groups? Null: There is no difference in responses for Q. 16 among different groups. Alternative: There is a significant difference in responses for Q. 16 among different groups.

134

There are more than 2 groups to compare so need to use 1-way ANOVA provided assumptions are satisfied. Samples are independent as groups are different from each other and were given separate treatment.

From above table, both Kolmogorov-Smirnov and Shapiro-Wilk show that P-value (given by Sig. column) for each group is less than 0.05. Thus normality assumption is not satisfied and need to look at Normal quantile plots to check normality. Note normality is robust!

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Control.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Insurgent.

135

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Militia.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Rebel.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Terrorist.

136

Box plot shows few outliers for the groups, Insurgent and Militia. The decision is to keep the outliers as normality is satisfied even with the outliers.

Above table gives Levene‘s test P-value = 0.421 (given by column Sig.). It is larger than α = 0.05 thus equal variance assumption is satisfied. All the assumptions for 1-way ANOVA satisfied.

137

From table ANOVA, F-test value = 0.624 and F-test P-value = 0.646. (F-test P-value = 0.646) > (α = 0.05), thus fail reject the null hypothesis. Conclusion: There is no significant difference in responses for Q. 16 among different groups at 5% level. Is there a difference in responses for Q. 17 among different groups? Null: There is no difference in responses for Q. 17 among different groups. Alternative: There is a significant difference in responses for Q. 17 among different groups. There are more than 2 groups to compare so need to use 1-way ANOVA provided assumptions are satisfied. Samples are independent as groups are different from each other and were given separate treatment.

From above table, both Kolmogorov-Smirnov and Shapiro-Wilk show that P-value (given by Sig. column) for each group is less than 0.05. Thus normality assumption is not satisfied and need to look at Normal quantile plots to check normality. Note normality is robust!

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Control.

138

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Insurgent.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Militia.

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Rebel.

139

Plot shows that most of the points are close to the diagonal line, there is no significant curvature and no concentration of points at any particular location. Thus normality is satisfied for group Terrorist.

Box plot shows one outlier for the groups, Insurgent and Control. This is acceptable norm with the normality assumption.

Above table gives Levene‘s test P-value = 0.416 (given by column Sig.). It is larger than α = 0.05 thus equal variance assumption is satisfied. All the assumptions for 1-way ANOVA satisfied.

140

From table ANOVA, F-test value = 1.537 and F-test P-value = 0.192. (F-test P-value = 0.192) > (α = 0.05), thus fail reject the null hypothesis. Conclusion: There is no significant difference in responses for Q. 17 among different groups at 5% level.

141

Is there a difference in responses for all the questions together among different groups? Null: There is no difference in responses for all the questions together among different groups. Alternative: There is a significant difference in responses for all the questions together among different groups. There are more than 2 groups to compare so need to use 1-way ANOVA provided assumptions are satisfied. Samples are independent as groups are different from each other and were given separate treatment.

From above table, both Kolmogorov-Smirnov and Shapiro-Wilk show that P-value (given by Sig. column) for each group is less than 0.05. Thus normality assumption is not satisfied and need to look at Normal quantile plots to check normality. Note normality is robust!

Plot shows that points are not in a linear format and close to the diagonal line; there is significant curvature and break in the points. Thus normality is not satisfied for group Control.

142

Plot shows that points are not in a linear format and close to the diagonal line; there is significant curvature and break in the points. Thus normality is not satisfied for group Insurgent.

Plot shows that points are not in a linear format and close to the diagonal line; there is some curvature and break in the points. Thus normality is not satisfied for group Militia.

Plot shows that points are not in a linear format and close to the diagonal line; there is some curvature and break in the points. Thus normality is not satisfied for group Rebel.

143

Plot shows that points are not in a linear format and close to the diagonal line; there is some curvature and break in the points. Thus normality is not satisfied for group Terrorist.

Box plot shows no outliers for all the groups.

For each group, normality is not satisfied. Thus we cannot use 1-way ANOVA. We need to use nonparametric (for non-normal data) method, Kruskal-Wallis test.

144

From table Test Statistics, Chi-Square test value = 1.364 and Chi-Square - test P-value = 0.850. (Chi-Square - test P-value = 0.850) > (α = 0.05), thus fail reject the null hypothesis. Conclusion: There is no significant difference in responses for all the questions together among different groups at 5% level.

145

Sign up to vote on this title
UsefulNot useful