The Effect of Labels On Analysis

THE EFFECT OF LABELS ON ANALYSIS
JEFFREY R. WELGAN
A Thesis
Submitted to the Faculty of Mercyhurst College
In Partial Fulfillment of the Requirements for
The Degree of
MASTER OF SCIENCE
IN
APPLIED INTELLIGENCE
DEPARTMENT OF INTELLIGENCE STUDIES

MERCYHURST COLLEGE
ERIE, PENNSLYVANIA
JULY 2009
DEPARTMENT OF INTELLIGENCE STUDIES
MERCYHURST COLLEGE
ERIE, PENNSLYVANIA
THE EFFECT OF LABELS ON INTELLIGENCE ANALYSIS
A Thesis
Submitted to the Faculty of Mercyhurst College
In Partial Fulfillment of the Requirements for
The Degree of
MASTER OF SCIENCE
IN
APPLIED INTELLIGENCE
Submitted By:
JEFFREY R. WELGAN
Certificate of Approval:
___________________________________
Kristan J. Wheaton
Associate Professor
Department of Intelligence Studies
___________________________________
James G. Breckenridge
Assistant Professor
Department of Intelligence Studies
___________________________________
Phillip J. Belfiore
Vice President
Office of Academic Affairs
July 2009
Copyright © 2009 by Jeffrey R. Welgan
All rights reserved.
iii
DEDICATION
This thesis is dedicated to the intelligence community,
whose hard work often goes unnoticed.
iv
ACKNOWLEDGEMENTS
I would first like to thank my wife, Chrissy, for the inspiration for this thesis. Without
her constant support and encouragement, this thesis would not have been possible.
I would also like to extend thanks to the professors of the Intelligence Studies program at
Mercyhurst College for their guidance and support, and for pushing me to be a better
analyst.
v
ABSTRACT OF THE THESIS
The Effect of Labels on Analysis
By
Jeffrey R. Welgan
Master of Science in Applied Intelligence
Mercyhurst College, 2009
Professor Kristan Wheaton, Chair
This thesis examines the notion of cognitive bias as it relates to intelligence
analysis. Specifically, it reports the findings of an online experiment in which analysts in
the field were given a mock intelligence report featuring one of five labels for hostile
groups— either ‗group,‘ ‗insurgent,‘ ‗rebel,‘ ‗militia,‘ or ‗terrorist‘—and then the
analysts were asked to complete a survey of questions regarding their impressions of the
likelihood, severity, and sophistication of an attack if one were to occur. The overall
findings resulted in no statistical significance, though statistical significance was
discovered to be present for certain labels and certain questions. What this means for the
intelligence community is that more research is needed before conclusive results can be
determined. If cognitive bias plays a role in intelligence analysis, it could have
implications for the way in which intelligence reporting is done. Regardless, intelligence
analysts are trained to be precise, accurate, and as unbiased as possible, and this is a goal
of which analysts must continuously be conscious and for which analysts must constantly
strive.
vi
TABLE OF CONTENTS
COPYRIGHT PAGE iii
DEDICATION iv
ACKNOWLEDGEMENTS v
ABSTRACT vi
TABLE OF CONTENTS vii
LIST OF TABLES ix
LIST OF FIGURES x
CHAPTER
1. INTRODUCTION 1
2. LITERATURE REVIEW 3
2a. Understanding The Cognitive Process 3
Framing and Anchoring 4
Heuristics and Cognitive Biases 9
2b. Understanding the Labels 13
The Etymology of the Labels 13
The ―Rebel‖ Label 17
The ―Militia‖ Label 20
The ―Insurgent‖ Label 23
The ―Terrorist‖ Label 27
Comparing the Labels 32
3. METHODOLOGY 36
3a. Research Design 36
3b. Sample, Population, and Participants 37
3c. Data Collection Instruments, Variables, & Materials 40
3d. Data Analysis Procedure 43
4. FINDINGS 46
4a. Questions 4&5: Overall Likelihood of an Attack 47
4b. Questions 6&7: Likelihood of a Successful Attack 49
4c. Questions 8&9: Likelihood of a Sophisticated Attack 51
4d. Questions 10&11: Severity of an Attack 53
4e. Question 12: Effect of an Overall Increase in
Activity 55
vii
4f. Question 13: Effect of Imagery Intelligence
Indicating Increased Activity Near a Training Base
or Command Post 58
4g. Question 14: Effect of Local Newspaper Reporting
an Attack Will Not Occur 59
4h. Question 15: Effect of Communications Intelligence
Reporting Increased Levels of Anti-Government
Rhetoric 60
4i. Question 16: Effect of a Long History of Social
Strife and Discontentment 62
4j. Question 17: Effect of the Region Being a Hot-Bed
of Differing Political and Religious Ideologies,
Widespread Poverty, and Poor Health Conditions 63
4k. Questions 18&19: Ranking of the Labels Based on
Their Likelihood to Engage in Violence and Their
Likelihood to Execute Severe Attacks 65
5. CONCLUSION 66
5a. Trends in Key Findings 66
5b. Research Recommendations 71
5c. Moving Forward 73
REFERENCES 74
APPENDICES 79
Appendix A: Participation Consent Form 80
Appendix B: Participation Debriefing Statement 81
Appendix C: Institutional Review Board (IRB) Approval 82
Appendix D: Scenario 1: Group Activity (Control Group) 83
Appendix E: Scenario 2: Rebel Activity 84
Appendix F: Scenario 3: Militia Activity 85
Appendix G: Scenario 4: Insurgent Activity 86
Appendix H: Scenario 5: Terrorist Activity 87
Appendix I: The Effect of Labels on Analysis Survey 88
STATISTICAL ANNEX 92
viii
LIST OF TABLES
Table 3.1: Five-group posttest only experimental design 36

Table 4.1: Scheffe test results for question #12 (overall increase in activity) 56
Table 4.2: Tamhane test results for question #15 (effect of COMINT) 61
Table 5.1: Positive, neutral, and negative responses to questions #4-17 68
ix
LIST OF FIGURES
Figure 3.1: Number of participants by label received 39

Figure 3.2: Number of participants by experience level 39
Figure 3.3: Number of participants by intelligence field 40
Figure 4.1: Significance values among questions 46
Figure 4.2: Label comparison – overall likelihood of an attack 48
Figure 4.3: Analytic confidence comparison of labels for overall likelihood of an
attack 49
Figure 4.4: Group label: likelihood of a successful attack 49
Figure 4.5: Insurgent label: likelihood of a successful attack 50
Figure 4.6: Terrorist label: likelihood of a successful attack 51
Figure 4.7: Comparison of labels: likelihood of a sophisticated attack 52
Figure 4.8: Comparison of labels: likelihood of a severe attack 53
Figure 4.9: Analytic confidence comparison of labels for likelihood of a severe
attack 54
Figure 4.10: Responses to an overall increase in group activity 55
Figure 4.11: Responses to IMINT indications of increased activity 58
Figure 4.12: Responses to local newspaper reporting of non-violent intentions 59
Figure 4.13: Outlying responses for question #14 (newspaper reporting of non-
violent intentions) 60
Figure 4.14: Responses to COMINT indications of increased anti-government
rhetoric 62
Figure 4.15: Responses to long history of strife and discontentment 63
Figure 4.16: Responses to region differing politically, ideologically, and
economically 64
Figure 4.17: Ranking of labels by violence & severity 65
x
INTRODUCTION
Intelligence analysts are constantly faced with labels throughout their daily work.
In the national security realm, labels, such as terrorist, rebel, militia, and insurgent, are
given to groups and are continually labels of interest to analysts and their decision
makers. These particular labels have significant meaning, and many individuals have a
preconceived idea, or cognitive biases, regarding the kinds of actions each of these
particular groups conduct. The presence of cognitive biases when analyzing reports
containing these labels can have a detrimental effect on analysis; as the effectiveness of
intelligence depends directly upon the effectiveness of the intelligence analyst. If an
analyst comes to his or her analysis using a flawed process, or inserts bias into the
analytical process, the product of that analysis is virtually certain to contain more
inconsistencies and therefore have increased inaccuracy.
My thesis, The Effect of Labels on Intelligence Analysis, will explore terms used
in the US Intelligence community and their affect on the analysis of a report.
Specifically, I will examine the following labels given to certain groups: terrorist, rebel,
militia, and insurgent. To determine whether or not intelligence analysts succumb to
cognitive biases when faced with these particular labels, I will conduct an experiment
using theoretical intelligence reports to demonstrate whether or not the use of particular
labels change the analysis of a report, ceteris paribus. For example, how different is an
analyst‘s analysis when the report contains a ―rebel‖ group rather than a ―terrorist‖
group?
In addition to conducting an experiment that identifies whether analysts apply
cognitive biases to these labels, the labels themselves must be examined. Using
1
definitions from the Oxford English Dictionary (OED), governmental agencies and
departments, as well as definitions from experts in the fields of the respective areas, I will
clearly identify the similarities and differences between terrorist, rebel, militia, and
insurgent groups. I will also attempt to explain how and why these labels are applied, as
well as their underlying implications.
The examination of the terms we use to label is significant, particularly in light of
the US global war on terrorism and for national security interests. By using the same
information and only switching which label (terrorist, rebel, militia, insurgent, or group)
is used, my thesis will attempt to determine how much impact, if any, each label has on
analysis. In addition, I will attempt to analyze the effect differences in experience levels
and intelligence backgrounds (National Security, Military, Law Enforcement, and
Competitive Intelligence) has on responses. This research is important because the use, or
misuse, of particular labels can directly affect US government action or inaction.
2
LITERATURE REVIEW
In an effort to make this section more organized and easier to read, I have divided
material into two main sections with subsequent subsections. The two main sections are:
the cognitive process, and exploring the labels. It is important to understand the factors
that influence our thinking before the labels in question are thoroughly examined;
therefore, the first section covers the cognitive process and contains two subsections:
framing and anchoring; and, heuristics and cognitive biases. The second section focuses
on the meaning of the labels and is broken up into six subsections: the etymology of the
labels; the rebel label; the militia label; the insurgent label; the terrorist label; and,
comparing the labels.
Understanding the Cognitive Processes
People construct their own version of „reality‟ on the basis of information

provided by the senses, but this sensory input is mediated by the complex
mental processes that determine which information is attended to, how it
is organized and the meaning attributed to it.1
- Richards Heuer
Many factors play into the cognitive process when labels are presented to
analysts. All forms of communication have both a communicator and a receiver between
which a message is conveyed. The difficulty comes when we try to understand the
meaning of the message. Oftentimes messages can be successfully interpreted, meaning
that the message has been received and understood in the manner it was intended.
However, messages can also be misunderstood. The distortion of messages frequently is
the result of individual perceptual screens. A perceptual screen, as defined by Debra
1
Heuer, Richards., Psychology of Intelligence Analysis, (Nova Science 1st ed, 2005), 17.
3
Nelson and James Quick, is ―a window through which we interact with people that
influences the quality, accuracy, and clarity of the communication.‖2 Because our
individuality is molded by a multitude of factors such as religious beliefs, social
upbringing, gender, age, values, culture, experience, needs, media, etc., messages can
easily be distorted from one person to another.
Framing and Anchoring
We perceive and understand the world in terms of the way we structure and
organize information cognitively. When we encounter new information, we make
decisions and assumptions about how that information should be categorized and
organized in our minds. In the world of psychology and decision-making, this structuring
of information is oftentimes referred to as framing.
A frame can be defined as ―a structure for accounting for the data and guiding the
search for more data.‖3 Frames are cognitively constructed to allow an individual to
process information and events; they are based on three or four anchors, or pieces of
information deemed relevant to the individual.4 These frames are not constructed
randomly, but rather, based on past experiences, both personal and cultural. Schoemaker
& Russo concur with this notion and expand it further by stating, "The frames we use to
view the world determine what we see, locking us into certain ideas and shutting out new
possibilities."5
2
Nelson, Debra L,. & Campbell, James Quick., Organizational Behavior: Foundations, Realities, &
Challenges, 5th ed. (Mason, OH.: Thomson South-Western. 2006). 250.
3
Klein, G. et al, ―A Data/Frame Theory of Sensemaking.‖ In Expertise out of Context; Proceedings of the
Sixth International Conference on Naturalistic Decision Making, edited by R.R. Hoffman. (Mahwah, NJ:
Erlbaum, 2004).
4
Ibid.
5
Schoemaker, P & Russo, J. ―Managing Frames to Make Better Decisions.‖ In Wharton on Making
Decisions, edited by S. Hoch and H. Kunreuther. (New York: John Wiley & Sons, 2001). 131-55.
4
According to noted sociologist Stanley Goffman (1974), a frame is ―a culturally
relative system of rules, principles, etc. that are used to organize society and guide
individual behavior.‖6 In Barlett‘s (1932) words, ―a schema is a mental representation of
the structure of event descriptions (e.g. stories), usually taking the form of regularly-
occurring and culture-specific sequences of dramatic sub-events.‖7 Schemas are ready-
made mental outlines into which individuals may enter new information in order to
understand it in terms of their personal and cultural history.
Richards Heuer describes everything we think we know about something in
particular as a ―mind-set‖ or ―mental model.‖8 Heuer states:
A mind-set is neither good nor bad. It is unavoidable. It is, in essence, a

distillation of all that analysts think they know about a subject. It forms a
lens through which they perceive the world, and once formed, it resists
change.9
These mind-sets are pulled from our memory and aid in processing new information that
is presented to us. Mind-sets, then, are synonymous with schema, which are stored in our
long-term memory (LTM) as a method for the brain to retrieve information more
efficiently each time we think about a particular subject.
Like Russian nesting dolls, schemas are reoccurring frames, which are built upon
certain anchors. When we examine schemas cross-culturally, they differ from group to
group, which explains why certain labels evoke certain analyses. For example, we see
this play out in society in terms of gender roles. This is particularly true for women, and
most people will have a very similar preconceived notion of ―what that woman is like‖
6
Goffman, E., Frame Analysis: An Essay on the Organization of Experience, (NY: Harper, 1974).
7
Bartlett, F.C., Remembering: A Study in Experimental and Social Psychology, (Cambridge, England:
Cambridge UP, 1932).
8
9
Ibid., 77.
5
when they hear the phrase ―soccer mom,‖ or ―single mother,‖ or ―working woman.‖
When a person in our society hears each of these phrases, they will most likely have a
similar picture of what each of these women look like, act like, even where these women
might live. In fact, these labels have become so ingrained in our minds that they can be
used to denote roles played in films, and everyone who watches a film will know who the
―soccer mom‖ was when they read the ending credits. We might expect a ―working
woman‖ to be wearing a suit in New York City, but we would not picture a typical
―soccer mom‖ living in the same city. In much the same way that these labels for women
operate to inform our judgments, decisions, and opinions, most people in our society will
come to similar preconceived notions when they hear the labels ―terrorist,‖ or ―rebel
group,‖ or ―insurgent,‖ or ―militia.‖
Frequently in the field of intelligence, information does not always make sense to
those who analyze it. Data does not always fall into our mental schema, or if it does fit,
our mind-sets will sometimes blur our perspective to the point where we steer ourselves
into a rut. Heuer states: ―If information does not fit into what people know, or think they
know, they have great difficulty processing it.‖10 Information processing is most difficult
when the data obtained does not fit into the frame one has constructed. When this
occurs, the analyst must attempt to make sense of the data and the frame; this is known as
―sensemaking.‖11 Klein et al describe the sensemaking process as follows:
Data are the interpreted signals of events; frames are the explanatory
structures that account for the data. People react to data elements by
trying to find or construct a story, script, a map, or some other type of
structure to account for the data. At the same time, their repertoire of
10
11
Erlbaum, 2004), 12.
6
frames – explanatory structures – affects which data elements they
consider and how they will interpret these data. We see sensemaking as
the effort to balance these two entities – data and frames.12
Sensemaking, then, will be fulfilled when feedback reveals the congruence between the
data and the frame. Jenny W. Rudolph explains that people make sense of situations
from cognitively-framed internal images of external realities.13 Therefore, when new
information is presented to analysts, the analyst tends to seek out the data that is, in one
way or another, linked to an event or schema already stored in their memory. The analyst
then uses this stored memory in conjunction with the newly presented data, creating an
anchor for which to construct a new frame. Additional data that fits well into this newly
constructed frame then gets added to solidify the frame and ultimately guide analysis.
The analysis of the situation will make sense to the analyst if the data used works
harmoniously with the frame they have constructed. However, the sensemaking process
can leave an analyst with a false sense of security, as sensemaking does not necessarily
mean that the analysis is accurate, due to the various different frames into which data can
fit.
In addition to the personal frames we construct, frames can be presented to us
through context. If information is presented within a particular context, the person
interpreting that data will oftentimes use the context that is presented to them rather than
reinterpreting or reframing the data.14 This phenomenon is known as the ―concreteness
12
Erlbaum, 2004),
13
Rudolph, Jenny. W. Into the big muddy and out again. Unpublished Doctoral Thesis, (Boston College,
Boston, MA., 2003).
14
Soman, Dilip., ―Framing, Loss Aversion, and Mental Accounting.‖ In The Blackwell Handbook of
Judgment and Decision Making edited by Derek J. Koehler and Nigel Harvey, (Malden, Ma: Blackwell
Publishing, 2004), 381.
7
principle.‖15 For example, an intelligence report stating that, ―Imagery intelligence has
indicated an increase of human activity near what appears to be a training base or
command post of the suspected insurgents. Imagery analysts report that the insurgents
appear to have small arms and other weapons‖ might cause an analyst or decision maker
to take that information and use it as truth; rather than examining the information,
reframing it, and questioning what an ‗increase in human activity‘ means (i.e. prior
number of insurgents compared to new number), what differences are apparent between a
‗training base‘ and ‗command post,‘ and exactly what type of weapons might this
suspected insurgent group have. Daniel Kahneman, respected judgment & decision-
making psychologist and Nobel laureate, states that "decision makers are generally quite
passive and therefore inclined to accept any frame that they are presented with.‖16
Because decision makers rely so heavily on the information provided to them by analysts,
analysts must be cautious in both using their own personal frames as well as in relying on
context to provide a set frame for them.
Because of information gaps and misleading anchors, there are strategies that
analysts can use to generate and evaluate hypotheses. Hypothesizing is an attempt to fill-
in information gaps and to strengthen the frame using judgment – Minsky (1975) also
referred to this process as ―slot-filling.‖17 Once hypotheses are formed, they can be
disproven or reduced to a point where our certainty in them is so small that the hypothesis
15
Slovic, P., ―From Shakespeare to Simon: Speculations and Some Evidence About Man‘s Ability to
Process Information.‖ Oregon Research Institute Monograph 12, no. 2 (1972).
Thaler, R. & Johnson, E., ―Gambling With the House Money and Trying to Break Even: The Effects of
Prior Outcomes on Risky Choice.‖ Management Science 36 (1990): 643-60.
16
Kahneman, D. & Tversky, A. (eds.), Choices, Values and Frames. (New York, NY: Russell Sage
Foundation & Cambridge University Press, 2000), ix-xvii
17
Minsky, M., ―A Framework for Representing Knowledge.‖ In The Psychology of Computer Vision,
edited by P. Winston (New York, NY: McGraw-Hill, 1975), 211-277.
8
is deemed not likely. Disproving hypotheses can also help in preventing satisficing.
Satisficing is the process of ―choosing the first hypothesis that appears good enough
rather than carefully identifying all possible hypotheses and determining which is most
consistent with the evidence.”18 Therefore, the more hypotheses an analyst can reduce or
eliminate, the more precise the analysis is likely to be.
Heuristics and Cognitive Biases
While the topics of cognitive bias and heuristics were not new to the field of
cognitive psychology, Richards Heuer was the first to really apply their principles to the
field of intelligence. Heuer defines cognitive bias as, ―a mental error that is consistent
and predictable… [and the] awareness of the bias, by itself, does not produce a more
accurate perception.‖19 The number of possible cognitive biases for a given situation is
staggering, and different situations will likely elicit different biases – it is partly for this
reason that cognitive biases are so difficult to overcome. However, Dr. Rob Johnston, in
a study aimed to identify and explain the conditions and variables that negatively affect
intelligence analysis, found that the most common cognitive bias among intelligence
analysts was confirmation bias.20 Confirmation bias is the tendency to pay more attention
to information that confirms one‘s viewpoints or beliefs rather than information that
refutes it.21,22 Johnston‘s findings confirm a 1998 study of intelligence analysts by
18
19
Ibid., 121-22.
20
Johnston, Rob., ―Analytic Culture in the US Intelligence Community: An Ethnographic Study.‖
(Washington D.C.: Center for the Study of Intelligence, Central Intelligence Agency, 2005). Chapter 2.
<https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-
monographs/analytic-culture-in-the-u-s-intelligence-community/chapter_2_research.htm>
21
Johnston, Rob., ―Analytic Culture in the US Intelligence Community: An Ethnographic Study.‖
(Washington D.C.: Center for the Study of Intelligence, Central Intelligence Agency, 2005). Chapter 2.
<https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-
monographs/analytic-culture-in-the-u-s-intelligence-community/chapter_2_research.htm> sa
9
Patterson, Woods, Sarter, and Watts-Perotti which found that when initial articles in open
literature were misleading, the subsequent analyses would often succumb to distortion.23
Heuristics also contribute to our biases and decision-making. In its simplest
definition, a heuristic can be described as a ‗rule of thumb,‘ or shortcut, that that aids us
in making decisions. According to Gerd Gigerenzer, a heuristic must contain three
criteria. First, heuristics must exploit evolved capacities – meaning that the heuristic is
intrinsic to our nature and has evolved with us over time. This can be exemplified by the
human ability to track an object in flight, also known as the gaze heuristic. Second, a
heuristic must exploit structures of environment. This criterion implies that heuristics are
not rational or irrational, good or bad, but rather relative to the environment in which they
exist. Lastly, a heuristic must be distinct from ‗as-if‘ optimization models. Essentially,
this criterion states that a heuristic cannot be ‗optimized‘ (or calculated) to explain away
human behavior.24
Since we live in a world of bounded rationality, meaning that not all information
is freely available to everyone, constraints are embedded into the analytical process.
Constraints such as memory capacity, collection assets, time, cost, conflicting sources,
etc. further complicate analysis, adding an ―optimization under constraints‖ factor to the
22
Mynatt, C.R., Doherty, M.E., & Tweeney, R. D. Confirmation Bias in a Simulated Research
Environment. Quarterly Journal of Experimental Psychology, 29, 85-95 (1977).
23
Patterson, E.S., Woods, D.D., Sarter, N.B., & Watts-Perotti, J.C. Patterns in Cooperative Cognition. In
COOP ‘98, Third International Conference on the Design of Cooperative Systems. 1998.
24
Gigerenzer, Gerd., ―Fast and Frugal Heuristics: The Tools of Bounded Rationality.‖ In The Blackwell
Handbook of Judgment and Decision Making edited by Derek J. Koehler and Nigel Harvey, (Malden, Ma:
Blackwell Publishing, 2004), 63-64.
10
already complex as-if optimization model.25 Unfortunately, these types of constraints are
the norm for the intelligence analyst.
In an effort to be more efficient, some heuristics are routine to the intelligence
professional‘s analytical process. One such heuristic is the recognition heuristic. In
essence, the recognition heuristic states that if two objects are presented, and only one of
those objects are recognized, then the recognized object has a higher value in relation to
the criterion presented.26 To take the recognition heuristic a step further, ecological
rationality can influence choice. Ecological rationality is the level of correlation between
an object and the criterion.27 Ecological rationality can be measured via recognition
validity. Recognition validity is the proportion of times an object is recognized with a
particular criterion, giving that object a higher criterion value than that of an object with a
lower proportion to the same criterion.28
The recognition heuristic, then, can have a direct influence on the analysis of
intelligence reports containing both labels and locations. If an analyst has a stronger
recognition validity concerning a particular label (let‘s say ‗rebel‘ more-so than
‗insurgent‘) in a certain geographic location (‗Africa,‘ for example), the analyst would
therefore be more likely to give the label with the greater recognition validity more
weight when analyzing reports using that label and criterion.
True heuristics have the qualities of being robust and fitting. Heuristics with
robustness have an ability to be predictive about events in the future. People use
25
Conlisk, John. ―Bounded Rationality and Market Fluctuations.‖ Journal of Economic Behavior &
Organization 29. No. 2 (1996) 233-250.
26
27
Ibid., 69.
28
Ibid.
11
heuristics because of their robustness; this is what makes them rules-of-thumb. They,
more often than not, tend to ―work.‖ However, just because heuristics can be robust does
not mean that we should rely on them, especially in a profession such as intelligence.
Relying on rules of thumb, without using structured methods to aid the analytical process,
can be dangerous and perhaps lead to intelligence failures. This is dangerous because
rules of thumb are not equivalent to facts – they are not constants like the value of pi
(3.14) or the Pythagorean Theorem (a2 + b2 = c2). In addition to robustness, Gigerenzer
posits that heuristics must be able to fit past events or data that is already known. 29 The
fit of a heuristic has a particular impact during the analysis of intelligence reports.
Gigerenzer states that:
In general, the predictive accuracy of a model increases with its fit,

decreases with its number of adjustable parameters, and the difference
between fit and predictive accuracy gets smaller with larger number of
data points. The general lesson is that in judgments under uncertainty, one
has to ignore information in order to make good predications. The art is to
ignore the right kind.30
It is virtually certain that cognitive biases, heuristics, schemata, and frames are all
inter-related and work simultaneous during the analytical process. These cognitive
processes are difficult to overcome and can impact the decisions we make as analysts.
29
30
Ibid., 80.
12
Understanding the Labels
The American Heritage Dictionary of the English Language describes a label as a
term ―used to identify or classify a person or group of people or objects; an epithet.‖ 31

In
essence, it is an intentional lingual shortcut with the purpose to communicate more easily.
However, communicating more easily does not necessarily mean we are communicating
more effectively. The use of labels inhibits the expression of true thoughts. For example,
when I state that the Chechens attacked a small Russian convoy, what I truly mean to say
is: a faction of extreme Chechens with deep-rooted hatred for Russia who are not
supported by the government of Chechnya or comprised of peaceful Chechen citizens,
attacked a convoy of Russian soldiers, some of which possibly being those who do not
necessarily seek to harm or threaten Chechens, or some of which may be of similar
ethnicity as the Chechens. It is evident that the use of a label, in this case Chechen,
makes a statement a little easier to listen to, as our brain has less information to process.
However, the label is used at the cost of accurately describing to whom or what I am truly
referring.
The Etymology of the Labels
Before comparing the various labels‘ use today, it is important to examine their
etymology. A word‘s etymology gives us a glimpse into how and why it has been used
historically, which affects how and why it is used today.
The term rebel, as we use it today to refers to ―one who resists, or rises in arms
against, the established governing power; one who refuses or renounces allegiance or
31
The American Heritage Dictionary of the English Language. (New York: Houghton Mifflin Company,
2003).
13
obedience to his sovereign or the government of his country,‖ is first referenced in the
Oxford English Dictionary (OED) circa 1400 as a reference to those whom Priam fought
against in the Trojan War: ―He [Priam] was faryn to fight in a fer londe, To riche hym of
Rebelles þatof þerewme held.‖32 It is traced through the centuries with notable references
such as De Foe, in Crusoe (1719): ―I could… give Liberty, and take it away, and no
Rebels among all my Subjects;‖ and Kipling, in Traffics & Discovery (1904): ―I shot my
Bible full of bullets after Bloemfontein went… Take it and pray over it before we
Federals help the British knock hell out of you rebels.‖33 The most recent entries in the
OED for the term refer only to Muslims as rebels, and there are no recent references for
rebels existing in the United States or any of its European allies.
The OED defines militia, as I intend to discuss it, as ―a paramilitary force
motivated by religious or political ideology, esp. one that engages in rebel or terrorist
activities in opposition to a regular army.‖34 Originally, the term appeared as an allusion
to the Spanish term milicia and the French milice, and its meaning has not changed
dramatically from these Latin roots in 1937. Unlike rebel, the term militia has been
recently used as a reference to United States citizens, as the OED cites the term as recent
as the 1990s being applied to several right-wing groups who were opposed to gun control
and mistrustful of the U.S. federal government: ―1995 Daily News (Halifax, Nova
Scotia)…The militia movement is already under scrutiny since the men arrested for the
32
"rebel, a. and n.1" The Oxford English Dictionary. 2nd ed. 1989. OED Online. Oxford
University Press. 26 Aug. 2008 <http://dictionary.oed.com/cgi/entry/50198771>.
33
Ibid.
34
militia, n." The Oxford English Dictionary. Draft Revision 2008. OED Online. Oxford
14
Oklahoma City bombing were linked to a so-called citizens‘ militia—a pro-gun, anti-
government paramilitary group.‖35
Similarly to rebel, the OED defines an insurgent as ―one who rises in revolt
against constituted authority; a rebel who is not recognized as a belligerent.‖36
References to insurgent are few in the OED, however, of note are Falconer‘s use of the
word in Demagogue, 1765: ―His sanction will dismay, And bid th‘ insurgents tremble
and obey;‖ and General Chalmers in 1812 in Great Britain, in which the term was used to
refer to the original uprising of the American colonists in gaining independence: ―[Why]
it was, that the vast strength of Britain did not beat down the colonial insurgents, not in
one campaign, but in three.‖37
Ironically, the term terrorist in the OED is at once the most specific and the most
vague, with two definitions that have political connotations. The first, specific definition
reads: ―As a political term: Applied to the Jacobins and their agents and partisans in the
French Revolution, esp. to those connected with the Revolutionary tribunals during the
‗Reign of Terror.‘‖38 The second, more vague definition describes a terrorist as ―anyone
who attempts to further his views by a system of coercive intimidation.‖ 39 Despite its
vague definition, the references for this term appear to be very specific. In its earliest
uses, the term applied to specific members of an extreme revolutionary society in Russia,
and then terrorist seems to disappear, according to the OED, until 1947 when it was used
35
militia, n." The Oxford English Dictionary. Draft Revision 2008. OED Online. Oxford
36
insurgent, a. and n." The Oxford English Dictionary. 2nd ed. 1989. OED Online.
Oxford University Press. 26 Aug. 2008 <http://dictionary.oed.com/cgi/entry/50118488>.
37
Ibid.
38
terrorist." The Oxford English Dictionary. 2nd ed. 1989. OED Online. Oxford
39
Ibid.
15
to describe actions taken by Jews in the Middle East: ―The latest and worst of the
outrages committed by the Jewish terrorists in Palestine—the blowing up of the King
David Hotel in Jerusalem.‖40 In the most recent entries, we see the term used as a
reference to members of clandestine or expatriate organizations aiming to coerce
established governments by acts of violence against them or their citizens. We see the
term applied seemingly independent of nationality or religion, as the term has entries that
use the term to reference Russians, Germans, Israelis, and Greeks.
There are a number of common threads among all of these labels. All of the
labels refer to groups of individuals who come together against a higher power or
government. All use violence and/or coercion to further their cause, and all of them have
a cause for which they have banned together. The differences come not so much in their
denotative aspects as their connotative aspects. By strict definition alone, the labels
would be difficult to distinguish from one another, but paired with their cultural contexts,
which cumulatively create their connotative histories, we can make distinctions as to
what kind of label will be applied to what kind of group.
The ―Rebel‖ Label
Academic literature defining the label ‗rebel‘ is lacking. The Merriam-Webster
dictionary‘s adjective definition of rebel is, ―opposing or taking arms against a
government or ruler, b: of or relating to rebels <the rebel camp>‖ and as a noun as, ―one
who rebels or participates in a rebellion.‖41 The definition implies that a rebel, or rebel
40
Ibid.
41
rebel. (2009). In Merriam-Webster Online Dictionary. Retrieved April 15, 2009, <http://www.merriam-
webster.com/dictionary/rebel>
16
group, is willing to use violent means to fight against a government or ruler. The primary
motivation behind a group who rebels is therefore primarily political, and the violence is
the method in which such a group achieves their goals – not through peaceful political
methods of changing governments or the policies thereof.
Michael Woldemarium, an Africanist Doctoral Fellow at the Woodrow Wilson
International Center for Scholars, in discussing his paper Why Rebels Collide:
Factionalism and Fragmentation in African Insurgencies used the definition given by the
Peace Research Institute, Oslo (PRIO) Armed Conflict Dataset to describe rebel
organizations.42 The PRIO Armed Conflict dataset describes a rebel organization as:
a non-governmental group of people, formally organized, having

announced a name for their group. It should also have used armed forces
and violence to further the objectives of the group. The group's military
activity must be part of a planned political campaign rather than
spontaneous violence, thereby distinguishing it from certain guerilla
movements. In the same way, the group must be involved in military
activity that has resulted in at least twenty-five casualties.43
Woldemariam links the formation and fragmentation of African rebel groups to state
failures in the late twentieth century, particularly the 1990‘s. These state failures are
related to the inability of one or more rebel organizations or factions to gain control and
consolidate state power. This continual cycle encourages the formation and
fragmentation of rebel groups, and it ultimately causes a period of endemic chaos and
chronically failing governments. Woldermariam, again, reinforces the concept that a
42
Woldemariam, Michael. "Why Rebels Collide: Factionalism and Fragmentation in African
Insurgencies." Africa: Conflict Transformation/Peacebuilding, Woodrow Wilson Center, Washington DC,
August 19, 2008.
43
"Data on Armed Conflict - CSCW." PRIO - International Peace Research Institute, Oslo.
<http://www.prio.no/CSCW/Datasets/Armed-Conflict/> &
<http://www.prio.no/sptrans/1664678440/Codebook.pdf> (accessed April 15, 2009).
17
rebel group is one who opposes an established government using, or threat of using,
violence. Nzamba Kitonga puts it this way:
Whenever a ruling oligarchy, class or elite subjugates or dominates

another and makes it impossible for that other through democratic legal
and/or constitutional means to remove the oligarchy or to effect change
within its superstructure, there results a dictatorship. And the only known
means for the removal of a dictatorship is the use of armed insurrection. In
a conflict of this nature, the subjugated class will call the ruling elite the
oppressor, tyrant, and/or dictator. The ruling class will refer to the
resisting group as rebel, subvert, treasoner, guerilla and in the extremist of
cases, a terrorist.44
Ted Robert Gurr, in an extensive study that examined the cause for why people
rebel, stated that the discontentment and frustration of a people toward a government lead
to the potential for political violence.45 His concept of political violence best describes
the acts of rebel groups. It states:
…political violence refers to all collective attacks within a political

community against the political regime, its actors – including competing
political groups as well as incumbents – or its policies. The concept
represents a set of events, a common property of which is the actual or
threatened use of violence, but the explanation is not limited to that
property. The concept subsumes revolution, ordinarily defined as
fundamental sociopolitical change accomplished through violence. It also
includes guerrilla wars, coup d‘état, rebellions, and riots.46
The frustration-anger-aggression theory underlies the causes for violent rebellions. This
theory states that frustration, anger, and aggression have a direct correlation; and that the
larger the population affected, the larger the scale of violence.47 Gurr states, ―Discontent
leads men to political violence when their attitudes and beliefs focus it on political
44
Kitonga, Nzamba. "The Fight Against Terrorism in East Africa: Comments and Observations." East
Africa Law Society Conference On ―Globalization And Terrorism: New Threats To Regional Integration‖,
East Africa Law Society, Entebbe, Uganda, October 10, 2003.
45
Gurr, Ted R.. Why Men Rebel. Princeton: Princeton University Press, 1971.
46
Ibid., 3-4.
47
Ibid., Ch 2 & 3.
18
objects, and when institutional frameworks are weak enough, or opposition organizations
strong enough, to give the discontented a sense of potency.‖48
Unfortunately, in an attempt for a rebel group to gain the desired attention from a
government they oppose, rebels will often target the items of interest/value to that
government. These items of interest and value, in the most severe cases of rebellion, are
natural resources and civilians. Rebel groups that exemplify this point include the FDLR
- Democratic Forces for the Liberation of Rwanda Hutu Rebels (Democratic Republic of
Congo), Tutsi Rebels (Rwanda), Lord‘s Resistance Army (Uganda), and The Party of
Islam (Somalia). The labeling for which groups are to be considered rebels, and which
are to be considered terrorists, are highly subjective to who is doing the labeling. For
example, Sheikh Hassan Dahir Aweys, a leader of the Eritrea-based Islamic Courts
faction (one of the four rebel factions that comprise the The Party of Islam in Somalia), is
named on the US list of terror suspects.49
It is possible that the overlap between labeling a group both as a rebel group and
as a terrorist group may be intentional. Matthew O‘Rourke, in his article The Impact Of
The "War On Terrorism" On Internal Conflicts, suggests that has been an intentional
effort to reclassify rebel groups as terrorist organizations. O‘Rourke states:
The campaign against terrorism has had political, as well as economic,

repercussions. There has been a concerted effort to link conflicts to the
terrorist attacks of September 11 and to reclassify opposition and rebel
groups as "terrorists." Once rebel groups are classified as terrorists,
governments feel less pressure to negotiate and become less willing to
enter into a peace process. In many cases this disinclination towards
negotiation leads a government to seek a military victory through the
extermination of the rebel group. As well, links between rebel groups and
international terrorist organizations, whether proven or not, are
48
Ibid., 155.
49
U.S. Department of State. "E.O. 13224: Identified Terrorists and Groups." U.S. Department of State.
<http://www.state.gov/s/ct/rls/fs/2001/6531.htm> (accessed April 16, 2009).
19
emphasized to isolate the groups and to justify a refusal to negotiate with
them. Once a group is labeled "terrorist," its grievances, legitimate or not,
are usually viewed as invalid, reducing international pressure on
governments to work towards a negotiated settlement. And identifying an
opposition group as "terrorists" helps a country obtain funding from the
US as part of the war on terror. While this increase in funding may
eventually lead to a military victory, it immediately intensifies ongoing
conflicts.50
Whether the overlapping of labels is intentional or not, it is clear that an overlap does in-
fact exist. What was of particular interest in O‘Rourke‘s statement was that he also
accepts that some rebel groups do engage in terrorist activities, or are at least associated
with known terrorist organizations. His statement evidences this: ―…links between rebel
groups and international terrorist organizations, whether proven or not…‖ Whether or
not what O‘Rourke posits is true, the mere fact that it may be possible is indicative of the
similitude between rebel and terror groups. The difference is not as obvious in
denotation as it is in connotation.
The ―Militia‖ Label
Historically, militias have formed because of a reaction to, or fear of, oppression
(or the impression of oppression) of rights by a seemingly tyrannical government. 51
Militia activity has often been categorized as terrorist activity, especially when referring
to groups who subscribe to extreme violence such as the Neo-Nazis, Ku Klux Klan
(KKK), Second of June Movement, the Baader-Meinhof Gang, the Red Brigades, the
50
O'Rourke, Matthew. "The Impact Of The ‗War On Terrorism‘ On Internal Conflicts." The Ploughshares
Monitor 26 (2005), <http://www.ploughshares.ca/libraries/monitor/monm05d.htm>.
51
Haider-Markel, Donald P., and Sean P. O'Brien. "Creating a "Well Regulated Militia": Policy Responses
to Paramilitary Groups in the American States." Political Research Quarterly 50(3) (1997): 552. In
JSTOR[database online]. Available from Mercyhurst College (accessed April 8, 2009).
20
Weathermen, and the Symbionese Liberation Army (SLA).52 Haider-Markel and O‘Brien
posit that there is no agreed-upon definition of a militia, but that they believe a citizen
militia is, ―any group of three or more persons organized for the stated purpose of
defending their rights and property against a tyrannical government. Such a group must
also engage in paramilitary training and/or maneuvers to prepare for ‗unlawful‘ behavior
on the part of the government agents.‖53
The definition given by Haider-Markel and O‘Brien is significant as it essentially
states that militias are organized and trained in a militaristic manner, implies that they are
prepared to use violence as a means to achieve a goal, and asserts that their goal is to
protect themselves (their rights and property) from a government that oppresses those
rights. In a way, the definition makes it sound like the cause of a militia is justified,
however, this is not always the case when we examine the vehemently violent groups
mentioned earlier. Because the world is pluralistic, viewpoints are extremely subjective.
While one person might define their ―rights‖ as one thing, another might disagree
completely – leaving a situation in which one person‘s pursuit of liberty and protection of
rights might actually infringe on another‘s liberty and rights. This subjective
interpretation of rights often causes extreme militias to engage in terroristic activities.
Kurth Cronin identifies four types of terrorism that are categorized by motivation
– these types are: left-wing terrorists, right-wing terrorists, ethnonationalist/separatist
52
Cronin, Kurth . "Behind the Curve: Globalization and International Terrorism." International Security,
27(3) (Winter 2002-2003): 39. In JSTOR[database online]. Available from Mercyhurst College (accessed
April 9, 2009).
53
Haider-Markel, Donald P., and Sean P. O'Brien. "Creating a "Well Regulated Militia": Policy Responses
to Paramilitary Groups in the American States." Political Research Quarterly 50(3) (1997): 552. In
JSTOR[database online]. Available from Mercyhurst College (accessed April 8, 2009).
21
terrorists, and religious or "sacred" terrorists.54 Descriptions of ethnonationalist/separatist
terrorists, and religious/sacred terrorists will be excluded from this section as they do not
apply to militia groups/organizations. Left-wing terrorists, however, are described by
Cronin as groups who align themselves with the communist movement and include
groups such as the Second of June Movement, the Baader-Meinhof Gang, the Red
Brigades, the Weathermen, and the SLA. Right-wing terrorists are described as groups
who align themselves with fascism and include groups such as Neo-Nazis, Christian
Patriots, and the KKK.
The groups identified with both left- and right-wing terrorism have also been
considered militia groups.55 Interestingly, the FBI has recognized some militia
organizations, and even ones that use ―militia‖ in their title, as domestic terrorist groups.
These militias include The Freemen Organization, The Mountaineer Militia (TMM), The
Washington State Militia, and The Phineas Priesthood.56 When referring to extremely
violent militias and hate groups, the FBI often uses ―Militia‖ and ―domestic terrorist
groups‖ interchangeably. In 1996, the FBI‘s Counterterrorism Threat Assessment and
Warning Unit put out a report titled, ―Terrorism in the United States‖ which stated:
Threats from domestic terrorism continue to build as militia extremists,

particularly those operating in the western United States, gain new
adherents, stockpile weapons, and prepare for armed conflict with the
federal government. The potential for domestic right-wing terrorism
remains a threat. Special interest groups also endure as a threat that could
surface at any time.
54
Cronin, Kurth . "Behind the Curve: Globalization and International Terrorism." International Security,
27(3) (Winter 2002-2003): 39. In JSTOR[database online]. Available from Mercyhurst College (accessed
April 9, 2009).
55
Anti Defamation League. "The Quiet Retooling of the Militia Movement: Recruitment." ADL: Fighting
Anti-Semitism, Bigotry and Extremism.
<http://www.adl.org/extremism/Militia/Recruitment.asp?m_flipmode=4> (accessed April 2, 2009).
56
FBI, Counterterrorism Threat Assessment and Warning Unit, National Security Division. ―Terrorism in
the United States.‖ (1996): 7-8. www.fas.org/irp/threat/fbi_terror_96.pdf (accessed April 9, 2009).
22
Domestic terrorism involves groups or individuals who are based and operate
entirely within the United States and its territories, and are directed at
elements of the U.S. Government or population, without foreign influence.
Domestic terrorist groups represent right-wing, left-wing, and special interest
beliefs. Their causes spring from issues relating to American political and
social concerns. 57
It is clear that the FBI recognizes that militias in the United States have used terrorist
tactics as a means of achieving their goals. In addition, the causes of militia groups as
listed by the FBI are reminiscent of the same causes listed by Gurr in Why Men Rebel.58
Although militias are most commonly associated with the western world, as
evidenced by the OED, militias are not strictly a western phenomenon. There are a
number of well-established militias in Africa and elsewhere in the world. One African
militia group, the Janjaweed (in Sudan), has become increasingly popular in news
reporting due to the crisis in Darfur. While the Janjaweed is overwhelmingly referred to
as a ―militia‖ group by major news agencies such as CNN,59 BBC,60 and Reuters,61 their
acts of violence on African ethnicities such as the Fur, Massaleet and Zagawa have been
described as terroristic – citing rapes, child-killings, deprivation of water, and arson as
common tactics of the Janjaweed.
The ―Insurgent‖ Label
Insurgency, as defined by Bard E. O‘Neill, is ―a struggle between a nonruling
group and the ruling authorities in which the nonruling group consciously uses political
57
Ibid.
58
Gurr, Ted R.. Why Men Rebel. Princeton: Princeton University Press, 1971. Ch 2 & 3.
59
CNN. "Janjaweed Militia: News & Videos about Janjaweed Militia - CNN.com." CNN.com - Breaking
News, U.S., World, Weather, Entertainment & Video News.
<http://topics.cnn.com/topics/janjaweed_militia> (accessed April 07, 2009).
60
BBC. "Sudan's Shadowy Arab Militia." BBC NEWS | News Front Page.
<http://news.bbc.co.uk/1/hi/world/africa/3613953.stm> (accessed April 06, 2009).
61
Reuters. "Sudan Man Accused Of Aiding Darfur war Crime Court." Reuters.com - World News,
Financial News, Breaking US & International News.
<http://www.reuters.com/article/homepageCrisis/idUSLM432733._CH_.2400> (accessed April 07, 2009).
23
resources (e.g., organizational expertise, propaganda, and demonstrations) and violence
to destroy, reformulate, or sustain the basis of legitimacy of one or more aspects of
politics.‖62 The use of violence to achieve these goals set insurgents apart from
sociopolitical protests and movements. While there are many types of insurgents – such
as, anarchist, egalitarian, traditionalist, apocalyptic-utopian, pluralist, secessionist,
reformist, preservationist, and commercialist – they all employ the use of violence to alter
a current political system.
Interestingly, O‘Neill, in his explanation of the traditionalist insurgent, identifies
groups/organizations as ―reactionary-traditionalist‖ insurgents that are more commonly
referred to as terrorist groups. O‘Neill states:
Within the category of traditionalist insurgents, one also finds more

zealous groups seeking to reestablish an ancient political system that they
idealize as a golden age. We refer to this subtype as reactionary-
traditionalist. It includes a plethora of present day Islamic militants, or
jihadis (―holy warriors‖), such as the Shia-based Party of God (Hezbollah)
in Lebanon and the Islamic Call (Al-Dawa al-Islamiyya) in Iraq, as well as
Sunni-based organizations like the Armed Islamic Group in Algeria,
Hamas (―zeal,‖ the acronym for Harakat al-Muqawama al-Islamiyya, or
Islamic Resistance Movement) in Palestine, the Islamic Group (Jama‘a al-
Islamiyya) in Egypt and, of course, Al Qaida (―the Base‖).63
The reasoning behind classifying these jihadi groups as insurgents rather than strictly as
terrorists is due to their ultimate goal of overthrowing, an existing political system
through a mechanism of violence – whereas a terrorist organization may only want to
change policy and not necessarily overthrow it (although some terrorist organizations
might also want to overthrow a government). Terrorism is just one tactic utilized by
insurgents. Terrorism, as O‘Neill uses it, is a method of carrying out violence,
62
O'Neill, Bard E.. Insurgency and Terrorism: From Revolution to Apocalypse; 2nd Ed., Revised.
Washington, D.C.: Potomac Books, 2005. 15.
63
Ibid., 21-22.
24
specifically ―the threat or use of physical coercion, primarily against noncombatants,
especially civilians, to create fear in order to achieve various political objectives.‖64 The
other two forms of warfare at the insurgents‘ disposal are guerilla and conventional
warfare.
O‘Neill‘s usage of terrorism and terrorist is similar to that of Paul Pillar in that it
is a method or a specific act of violence.65 In fact, O‘Neill often uses the label terrorist
synonymously with insurgents, using the label terrorist when referring to insurgents who
engage in terrorism. He uses the term ―transnational terrorism‖ to refer to non-state
terrorist organizations, such as Al Qaida, that view the world as a battlefield.
Transnational terrorists still have the same ultimate goal in mind – to change a policy,
however, the war on the policies they try to transform are played out on a much larger
scale. Transnational terrorism is differentiated from international terrorism, which is
used to describe similar terrorist acts by ―individuals or groups controlled by sovereign

66
states‖ Insurgent terrorism is not as sporadic or mindless as it may oftentimes seem.
Insurgents who use terrorism have specific long, medium, and short term goals:
The long-term goal is, of course, to change the political community,

political system, authorities, or policies. The intermediate goal of
terrorism is not so much the desire to deplete the government‘s physical
resources as it is to erode its psychological support by instilling fear into
officials and their domestic and international supporters. In the short term,
terrorists often pursue one or more objectives, such as extracting particular
concessions (eg., payment of ransom or the release of prisoners), gaining
publicity, undermining or seeking to join a negotiating process,
demoralizing the population through the creation of widespread disorder,
provoking repression by the government, enforcing obedience and
64
65
Pillar, Paul R.. Terrorism and U.S. Foreign Policy. Washington, D.C.: Brookings Institution Press, 2001.
18.
66
25
cooperation from those inside and outside the movement, enhancing the
political stature of specific factions within an insurgent movement, and
fulfilling the need to avenge losses inflicted upon the movement.67
The CIA‘s understanding of insurgency and insurgents as outlined their Guide to
the Analysis of Insurgency, is strikingly similar to O‘Neill‘s.68 Although the CIA
identifies fewer types of insurgencies, placing them within four broad categories
(Politically Organized, Militarily Organized, Traditionally Organized, and Urban);
terrorism tactics, guerrilla warfare and conventional warfare are also identified as the
forms of warfare conducted by insurgent groups.69 The goals of insurgents, as identified
by the CIA, are also very similar to those identified by O‘Neill. However, unlike
O‘Neill, the CIA blatantly states that one of the objectives of insurgent groups is to
control a particular area. ―This objective differentiates insurgent groups from purely
terrorist organizations, whose objectives do not include the creation of an alternative
government capable of controlling a given area or country.‖70
While insurgent and terrorist organizations have the same goals of changing an
existing political order, the difference lies in actual physical occupation of a territory.
The CIA‘s differentiation contradicts O‘Neill‘s claim that Al Qaida is in fact an insurgent
organization – as Al Qaida does not intend to occupy and control territories such as the
US, but rather it intends to change specific western policies and influences. Jim
67
Washington, D.C.: Potomac Books, 2005. 34-35.
68
Central Intelligence Agency, Guide to the Analysis of Insurgency, C05332177. Washington DC,;
Central Intelligence Agency, 2.
Released January 05, 2009. <http://www.fas.org/irp/cia/product/insurgency.pdf>
69
Ibid., 5.
70
Ibid., 2.
26
Ruvalcaba describes terrorism in this sense as ―an auxiliary tactic that insurgents use as
part of a broader strategy rather than an exclusive one.‖71
The ―Terrorist‖ Label
“…as the meaning and usage of the word have changed over time to accommodate the
political vernacular and discourse of each successive era, terrorism has proved
increasingly elusive in the face of attempts to construct one consistent definition.”
- Bruce Hoffman
In his book, Inside Terrorism, Bruce Hoffman attempts to settle on a definition of
terrorism that avoids the reckless labeling of terrorism under a wide range of acts.
Hoffman believes that the history of the word reveals a usage that describes a ―reflection
of the current political and social tenor of the times.‖72 He defines terrorism in a way that
attempts to distinguish itself from guerillas, ordinary criminals, and assassins. Hoffman
states that, ―Virtually any especially abhorrent act of violence that is perceived as
directed against society –whether it involves the activities of anti-government dissidents
or governments themselves, organized crime syndicates or common criminals, rioting
mobs or persons engaged in militant protest, individual psychotics or lone extortionists –
is often labeled ‗terrorism.‖73
Hoffman finds three main problems the terrorist label has encountered. The first
problem is with the OED‘s definition of terrorist as being ―too literal and too historical‖
71
Ruvalcaba, Jim. "Understanding Iraq's Insurgency." Al Nakhlah Article 7 (Spring 2004): 1.
72
Hoffman, Bruce. Inside Terrorism. New York: Cambridge University Press, 1998. 13.
73
Ibid.
27
to be applied to the present times.74 The second contention how subjective the decision is
to use the label terrorist. Hoffman asserts that the decision depends ―largely on whether
one sympathizes with or opposes the person/group/cause concerned.‖75 This point
counters the opinion that terrorism can be defined in the same manner in which US
Supreme Court Justice Potter Stewart defined pornography – that we may not be able to
define it precisely, but we know it when we see it.76 The final contention deals with
media‘s failure to be consistent with the usage of the terrorist label and its differentiation
with other labels such as ‗guerilla,‘ ‗fundamentalist,‘ and ‗commando.‘77 It is for these
three main reasons that the definitions for, and the usage of, the terrorist label is so
disputed.
Through his research, Hoffman boils terrorism down to:78
Ineluctably political in aims and motives

Violent - or equally important, threatens violence;
Designed to have far-reaching psychological repercussions beyond
the immediate victim or target;
Conducted by an organization with an identifiable chain of
command or conspiratorial cell structure (whose members wear no
uniform or identifying insignia); and
Perpetrated by a subnational group or non-state entity
We may therefore now attempt to define terrorism as the deliberate creation and
exploitation of fear through violence or the threat of violence in the pursuit of political
change. All terrorist acts involve violence or the threat of violence. Terrorism is
specifically designed to have far-reaching psychological effects beyond the immediate
victim(s) or object of the terrorist attack. It is meant to instill feat within, and thereby
74
Hoffman, Bruce. Inside Terrorism. New York: Cambridge University Press, 1998. 14.
75
Ibid., 31.
76
Jacobellis v. Ohio, 378 U. S. 184 (1964)
77
Hoffman, Bruce. Inside Terrorism. New York: Cambridge University Press, 1998. 36-37.
78
Ibid., 43.
28
intimidate, a wider ‗target audience‘ that might include a rival ethnic or religious group,
an entire country, a national government or political party, or public opinion in general.
Terrorism is designed to create power where there is none or to consolidate power where
there is very little. Through the publicity generated by their violence, terrorists seek to
obtain the leverage, influence and power they otherwise lack to effect political change on
either a local or an international scale.79
Many similarities to Hoffman‘s definition of terrorism also emerge in Michael
Kronenwetter‘s, Terrorism: A Guide to Events and Documents, particularly in the
historical examination and criticisms of government agency definitions. Kronenwetter
also posits that terrorism has also existed in the United States for a long time, but that it
has been labeled as other things (resembling Hoffman‘s argument that the OED reflects a
usage consistent with the current political culture of the time). He illustrates this point
with examples of Post-Civil War acts by the Ku Klux Klan being called ‗resistance to
reconstruction;‘ the 1886 Chicago Haymarket Square bomb being described as ‗labor
troubles;‘ ‗vigilantism‘ when African Americans were being lynched in the first half of
the twentieth century, and racism when whites bombed black churches in the second half;
and the Unabomber bombings described as ‗madness.‘80
Kronenwetter states that terrorists have three main elements: 1) they are malicious
actions or threats; 2) they are ―directed against people who are regarded as innocent, or
who are protected by the laws and conventions of modern warfare;‖ and 3) ―designed (at
79
Hoffman, Bruce. Inside Terrorism. New York: Cambridge University Press, 1998. 43-44.
80
Kronenwetter, Michael. Terrorism: A Guide to Events and Documents. New York: Greenwood Press,
2004. vii.
29
least partly) to frighten, intimidate, or otherwise influence populations or governments.‖ 81
Kronenwetter‘s argument differs from Hoffman in the sense that Hoffman‘s definition
implies a strong political motivation behind terrorist acts, whereas Kronenwetter leaves
that aspect open to interpretation. Another difference is that Kronenwetter draws-out an
element of innocence among the victims of terrorism; Hoffman does not. What the two
definitions do have in common, however, is the element of violence and the
psychological impact left with the victims of terrorism.
Like Hoffman and Kronenwetter, Paul Pillar asserts that the terrorist label has been
used too loosely – even to the point to flippantly describe someone else‘s political
policies. Pillar believes that the US Department of State‘s statutory definition for
keeping statistics on international terrorism is as good a definition as any other. The
statute defines terrorism as, ―premeditated, politically motivated violence perpetrated
against noncombatant targets by subnational groups or clandestine agents, usually
intended to influence an audience.‖82 Five key elements are listed in the State
Department‘s definition: premeditation, violence, political motivation, noncombatant
targets, and perpetrators that are comprised of either subnational groups or clandestine
agents. This definition is an ideal blend of Hoffman‘s and Kronenwetter‘s definitions.
The only element left out of this definition, but included in the other two, was the
psychological aspect; however, Pillar picks up on this point by stating:
There is one other respect in which terrorism must be conceived somewhat

more broadly than the statutory definition above. Terrorism as an issue is
not just a collection of incidents that have already occurred; it is at least as
much a matter of what might occur in the future. The threat of a terrorist
81
Kronenwetter, Michael. Terrorism: A Guide to Events and Documents. New York: Greenwood Press,
2004. 8.
82
22 U.S.C. 2656f (d). "Chapter 7 -- Legislative Requirements and Key Terms." U.S. Department of State.
http://www.state.gov/s/ct/rls/crt/2007/103715.htm (accessed March 30, 2009).
30
attack is itself terrorism…Terrorism in general, even when conducted with
conventional means, tends to have greater psychological impact relative to
the physical harm it causes than do other lethal activities, including
warfare.83
Pillar believes that when writing policy on terrorism, we shouldn‘t hyper-focus on
trying to define what terrorism is or is not. Instead, Pillar suggests taking a common-
sensible approach to what terrorism is by paralleling the understanding of the label/act
with Justice Stewart‘s understanding of pornography.84 Lastly, Pillar states that most-of-
all, ―terrorism is a method – a particularly heinous and damaging one – rather than a set
of adversaries or the causes they pursue. Terrorism is a problem of what people (or
groups, or states) do, rather than who they are or what they are trying to achieve.‖85
While there are similarities between Hoffman‘s, Kronenwetter‘s, and Pillar‘s
definition of terrorist and terrorism (as well as many others), the definitional differences
and lack of consensus is evident. The definitional quagmire is likely to continue on for
decades, especially as nations attempt to agree on a broad enough definition to include a
wide-variety of actors engaging in asymmetrical warfare, but yet a narrow enough one so
that nations are not self-included in those definitional parameters. That said, there are
three high-frequency components that run through definitions of terrorism: 1) the use of
violence, 2) political objectives, and 3) the intent to place fear into victims.86
83
Pillar, Paul R.. Terrorism and U.S. Foreign Policy. Washington, D.C.: Brookings Institution Press, 2001.
14, 24.
84
Ibid., 16-17.
85
Ibid., 18.
86
Jongman, A.J., and Alex P. Schmid. Political Terrorism. New Brunswick: Transaction Publishers, 1988.
Jongman and Schmid analyzed 109 different definitions of 'terrorism' and identified frequency trends
among components used to describe the term.
31
Comparing the Labels
The definitions of the rebel, militia, insurgent, and terrorist labels are all quite
ambiguous and yet all are particularly very similar. They differ not so much in
denotation as they do in connotation. All four of the labels share the same core concepts
and have common threads that tie them all together; however, when we hear each of them
individually each label evokes a specific emotion and mental image. Our cognitive
biases push beyond the denotations of the labels and create the connotations for which
think about them on a daily basis.
All the labels share the threads of being political in nature, use violence as a
method to achieve their goals, and can even participate in terrorist tactics (blatantly
targeting innocents and using fear as a weapon). All the groups are formed in opposition
to a current establishment, particularly political, but also sometimes ideological. The
groups form as a reaction to discontent with the establishment – following the frustration-
anger-aggression theory that was outlined by Ted Gurr. The more widespread the
frustration and discontent, the more the group becomes angry. The scope broadens with
an increasingly discontented population, and in turn, the intensity potential also increases.
As the intensity potential increases, the potential for violence also increases.
The biggest difference between the labels it this: While all groups (rebel, militia,
insurgent, and terrorist) can use terrorism as a tactic to achieve their goals, not all groups
have to engage in terrorism. Terrorism is a common tool in the tactical toolbox for many
of the extremely violent groups that are categorized as one of these labels; however, rebel
groups, militias, and insurgents can also use solely conventional and guerilla tactics to
achieve their goals. This fact, however, does not explain why rebel groups, militias, and
32
insurgents who decide to use terrorist tactics are still categorized as such groups – rather
than categorized as terrorist groups/organizations. In addition, the groups we label as
terrorists do not strictly use terrorist tactics – they also engage in other forms of warfare
such as guerrilla warfare and cyber warfare. Whether the reasoning is embedded into
our socio-linguistic nature, or whether it is an intentional process decided by
governments as O‘Rourke suggests, the fact still remains that we do consider some
groups as rebels, militias, and insurgents despite any terrorist tactics they may employ.
Aside from the assertion that rebel groups, militias, and insurgents do not have to
engage in terrorism (although the labeling of groups becomes increasingly more difficult
with groups that actively engage in multiple forms of violence – including terrorism)
little distinguishes one label from another.
The literature suggests that the difference between a terrorist group and an
insurgent group is that insurgent groups want to physically control an area after they
overthrow an established government. However, while the direct goal of a terrorist group
may not necessarily include overthrowing a government, but rather changing the policies
of that government, most terrorist organizations would not be dissatisfied with the
complete takeover of a government they oppose. In addition, the insurgent‘s aspect of
―controlling‖ an area is ambiguous and left open to interpretation – begging the question:
Do you have to physically commit yourself to a particular area in order to control that
area?
The denotation differences between rebel groups and militias are less obvious.
Connotatively, the difference suggests that rebel groups are less organized than militia
groups, giving the group a more rag-tag feel to it. However, this is evidently not the case
33
when the violence spurred by rebel groups is closely examined. Denotatively, the biggest
difference between rebel groups and militias is that militia groups do not have to be
actively engaged in violence. Militia groups must only have the potential to engage in
violence against an established government. Rebel groups, on the other hand, are
considered as such because they are actively engage in violent opposition to an
established government. Both groups can utilize multiple forms of violence, including
terrorism, as a tactic.
While there are some notable differences between the labels, the rebel label and
the insurgent label are seemingly alike. Both groups oppose an established government
and desire to take it over, even if violence is necessary. Both groups are actively engaged
in violence, not just holding the potential to be violent. And lastly, both groups use, or
can use, a variety of tactics to achieve their goals. Connotatively, we think that insurgent
groups are more likely than rebel groups to use terrorist tactics, however this is most
likely due to our cognitive biases and the popularity of insurgent reporting in media
networks (attributed mostly to the war in Iraq).
Defining these labels is an extremely difficult task and it is not my aim to do so in
this thesis. My intention is to illustrate the marked similarities of these labels and to
point-out that the human cognitive process likely has something to do with how we view
and analyze these groups. While the use of labels can be quite helpful in processing
information, labels with similar denotations and divergent connotations can be
troublesome to the analyst. When definitions begin to overlap and blur together without
any methods for clarifying the intent of one label from another, the quality of analysis is
34
at risk. My thesis will examine to what extent the quality of analysis is at risk, if it is
indeed at risk as the differing connotations of these labels would suggest.
35
METHODOLOGY
Research Design
In order to test whether or not cognitive biases affect the way an intelligence
professional analyzes reports using the labels in question (rebel, militia, insurgent, and
terrorist), I conducted an experiment that combined two research designs: a cross-
sectional survey and a five-group experimental design. A cross-sectional survey is a
survey design that gathers information on a population at s single point in time.87 This
design allows for respondent data to be cross-analyzed against other respondents. Cross-
sectional surveys are one of the easiest survey experiments to conduct, as they do not
have to track participant responses over a period of time. Due to time constraints, the
cross-sectional survey was most appropriate for this study. Table 3.1
Since multiple labels were being tested against each
R group X O
other, a five-group experimental design was used. The five-
R rebel O
group experimental design is a modified two-group posttest-
R militia O
only experimental design that uses a total of five groups
R insurgent O
(rather than two) in order to determine whether the labels
R terrorist O
affect analysis. True experimental designs, like the five-group
design used in this study, use randomly assigned (R) participants. In this type of design,
one group is assigned a program (indicated with an ‗X‘), which in the case of this study is
the use of neutrally emotive label – ―group.‖ A pretest is not necessary with this type of
experiment as the random assignment of participants is assumed to be probabilistically
87
"Survey Methods." School of Information - University of Texas.
http://www.ischool.utexas.edu/~palmquis/courses/survey.html (accessed April 20, 2009).
36
equivalent.88 After the program is implemented, all groups are measured (indicated with
the ‗O‘) and compared against one another to measure the effect of the program (see
Table 1). In the case of this experiment, the group with the program acted as the control
group and also served as the comparison group against the other groups. The posttest-
only randomized experiments hold strong against single-group and multiple-group threats
to internal validity.89
In total, five mock intelligence reports (See appendices E-I) with accompanying
surveys (See appendix J) were created – each report and survey used one of the labels in
question (rebel, militia, insurgent, and terrorist) and one report and survey acting as the
control group using the label ―group.‖ All five reports and surveys were identical in
wording and context with the exception of which label was used.
Sample, Population, and Participants
Participation in the experiment was limited to populations associated with
intelligence analysis. These populations included analysts from the national security
sector, military intelligence, law enforcement intelligence, competitive intelligence, and
students of intelligence studies. In order to target populations that were related to
intelligence analysis, I recruited students of the Mercyhurst Intelligence program (both
graduate and undergraduate students), as well as tapped into the Mercyhurst College
Intelligence Studies Program‘s faculty contact network. The Mercyhurst contact network
included intelligence-related organizations such as the International Association of Law
Enforcement Intelligence Analysts (IALEIA), the Society of Competitive Intelligence
88
Research Methods Knowledge Base. "Two-Group Experimental Designs." Social Research Methods.
http://www.socialresearchmethods.net/kb/expsimp.php (accessed April 20, 2009).
89
Ibid.
37
Professionals (SCIP), and the International Association for Intelligence Educators
(IAFIE). In addition, a link to the experiment was posted on Professor Kristan
Wheaton‘s Sources and Methods blog, which has a strong following of intelligence
professionals.90 The link to the survey was also sent out to other intelligence
professionals with which the Mercyhurst faculty and I have personal contact.
Participation in the experiment was on a voluntary basis and participants did not
receive any compensation for their efforts. The participants in my study did not come
from a population requiring any special protection (i.e. children, elderly, criminals, etc.)
and none of the participants were purposefully deceived. There were no physical or
psychological risks associated with the survey. Participants were presented with an
informed consent form (See Appendix B) that was posted on the hosting site; consent to
participate was indicated by clicking the hyperlink that directed participants to the actual
mock intelligence report and survey.91 After the survey was completed, participants were
redirected to a debriefing statement that was also posted on the host site (See Appendix
C).92 The procedure and experiment was approved by Mercyhurst College‘s Institutional
Review Board (See Appendix D) on November 12, 2008.
The link to the experiment was originally hosted through Mercyhurst College‘s
Institute for Intelligence Studies (MCIIS) website. The original host site yielded 222
participants. Unfortunately, the Intelligence Program switched servers part way through
90
Wheaton, Kristan. "Surveys, Surveys, Surveys! (It's Research Season...)." Sources And Methods.
http://sourcesandmethods.blogspot.com/2009/01/surveys-surveys-surveys-its-research.html (accessed April
21, 2009).
91
Pollard, Kristine. "Jeffrey Welgan Thesis." Institute for Intelligence Studies at Mercyhurst College.
http://74.125.95.132/search?q=cache:acF9XiY2Ly0J:www.cirat.com/welganthesis.php+http://www.cirat.co
m/welganthesis.php&cd=1&hl=en&ct=clnk&gl=us&client=firefox-a (accessed April 21, 2009).
92
Pollard, Kristine. "Jeffrey Welgan Thesis - Thank You." Institute for Intelligence Studies at Mercyhurst
College. http://www.cirat.com/welganthanks.php (accessed April 21, 2009).
38
the winter term causing a broken link to the experiment. This was particularly
unfortunate as the survey encouraged participants to pass the link along to other
intelligence professionals who might be interested in participating in the survey. The
broken link halted participation momentum. The link was later reposted on the
Mercyhurst College‘s Center for Intelligence Research Analysis and Training (CIRAT)
website; however, I was able to only gain eleven more intelligence professionals to
participate before the conclusion of the survey.93
In total, 233 Figure 3.1
intelligence professionals
participated in my
experiment. Figure 3.1
illustrates the number of
participants by label received
in the experiment. In order to
randomly assign the Figure 3.2
intelligence report/survey
among the participants, I
used an automatic hyperlink
randomizer that sent
93
Pollard, Kristine. "Jeffrey Welgan Thesis." Institute for Intelligence Studies at Mercyhurst College.
http://74.125.95.132/search?q=cache:acF9XiY2Ly0J:www.cirat.com/welganthesis.php+http://www.cirat.co
m/welganthesis.php&cd=1&hl=en&ct=clnk&gl=us&client=firefox-a (accessed April 21, 2009).
39
participants to one of the five surveys.94
Although participants varied by experience level, the majority of participants
(65%) had less than three years of intelligence experience (see Figure 3.2), This large
proportion is attributed to the number of student participants I was able to recruit through
the Intelligence Studies program at Mercyhurst College as well as the number of alumni
the college is still in contact with.
Figure 3.3 illustrates the number of participants by the field of intelligence with
which they most associate themselves. Again, the participants who have identified
themselves as students of
Figure 3.3
intelligence represent the
largest portion (28%) of the
responses collected in the
experiment. Military
intelligence analysts and
national security-related
intelligence analysts represent the second and third largest populations with 24% and
22% respectively.
Data Collection Instruments, Variables, and Materials
In order to collect respondent data, I used a paid subscription survey service
through freeonlinesurveys.com. This survey service was used for its relatively
inexpensive monthly fee and for its ease of use. Freeonlinesurveys.com allows the user
94
"CGI Scripts: Random Link." CGI Spy. www.cgispy.com/scripts/rlink.html (accessed April 21, 2009).
40
to create multiple surveys with unique uniform resource locators (urls), allowing them to
be used in the hyperlink randomizer on the hosting site. The survey service also allowed
the survey results to be downloaded into a Microsoft Excel Spreadsheet. The mock
intelligence reports (using one of the five labels) were presented to the participants at the
commencement of their survey.
The mock intelligence reports provided the invariable data for which the
participants had to respond and analyze. The reports focused around seven primary data
anchors:
A setting of the Northeast and Horn of Africa Regions

An overall increase of activity for that particular label
Imagery intelligence indicating an increase in human activity near an
apparent training base or command post
Local newspaper report that the label ―does not plan to incite, or
participate in, any violence in the future despite that group‗s
discontentment with the current establishment‖
Recent COMINT indicating an increase in anti-government rhetoric from
the label based upon social injustice and oppression
A long history of social strife and discontentment in the region
The region being a hot-bed of differing political and religious ideologies,
as well as a region of poverty and poor health conditions
The surveys also consisted of three questions relating to the background of the
participants and sixteen questions relating to the participants‘ analysis of the mock
intelligence report. Unfortunately, the first question, asking whether or not the participant
has ever been an intelligence analyst, had to be discarded from the results. Feedback
from some of the participants, mostly students of intelligence, indicated that there was
some confusion with this question, causing discrepancies with responses. The exclusion
of the data from this question is unlikely to have significantly changed the overall results
of the experiment, as populations related to intelligence analysis were specifically
targeted. The other two background questions asked what area of intelligence the
41
participants were most associated with, as well as how many years of experience the
participants had.
The remainder of the questions focused around the participants‘ analysis of the
mock intelligence report. The questions could be categorized into three different
sections:
1. A section asking about the participants‘ assessment of the situation. The
participants had to answer questions regarding the overall likelihood of an
attack, the likelihood that an attack would be successful, the likelihood
that an attack would be sophisticated, and a question inquiring about the
likelihood that an attack would be severe. The participants had to answer
these questions on a 1-10 scale – with ―1‖ indicating that it was virtually
certain not to be, and ―10‖ indicating that it was virtually certain. Each
question in this section was paired with a question asking for the
participant‘s analytical confidence – this was also based on a 1-10 scale,
with ―1‖ indicating an extremely low level of analytical confidence, and a
―10‖ indicating an extremely high level of analytical confidence.
2. A section that indicated the value placed on some of the data anchors
provided within the mock intelligence report. These questions were posed
as open-ended statements that the participants had to complete using a
Likert Scale of 10 possibilities; one the low end the participants could
respond with, ―highly decreases the likelihood of an attack,‖ and on the
high end with ―highly increases the likelihood of an attack.‖ These open-
ended statements focused around invariable data within the report, such
42
as: the overall increase in group activity; Imagery indicating increases in
human activity near a training base or command post; the local newspaper
report that the label ―does not plan to incite, or participate in, any violence
in the future despite that group‗s discontentment with the current
establishment; COMINT indicating an increase in anti-government
rhetoric; the long history of social strife and discontentment in the region;
and the region being a hot-bed of differing political and religious
ideologies, as well as a region of poverty and poor health conditions.
3. Lastly, there was a two-question section asking the participants to rank the
labels based on the which labels were most likely to engage in violence,
and another question asking the participants to rank the labels according to
which were most likely to execute severe attacks.
Data Analysis Procedures
In order to analyze the data from the surveys effectively, the data had to first be
consolidated into one large master database. Since the data from freeonlinesurveys.com
was downloaded into separate Excel spreadsheets, the data was easily consolidated by
cutting and pasting the information from each separate survey into one single
spreadsheet. An extra column was then added to the spreadsheet that identified the label
given to each participant.
To test for statistical significance of responses among the different labels, the data
was analyzed using a SPSS program. A statistics professor, Dr. Hemagini Deshmukh, in
Mercyhurst College‘s math department conducted the SPSS analysis, ensuring accuracy
of the results. Statistical significance testing was run for questions four through
43
seventeen. Responses were tested among the labels for each question independently.
Additionally, differences among labels for all questions together were tested. Questions
four through eleven asked participants to assess the: overall likelihood of an attack,
likelihood of a successful attack, likelihood of a sophisticated attack, and the likely
severity of an attack, as well as the analytical confidence levels for each of those
questions. Questions twelve through seventeen asked participants to assess the effects of
the various data anchors on the likelihood of an attack occurring.
In order to test for normality among the labels, the Kolmogorov-Smirnov and
Shapiro-Wilk tests were used in conjunction with a Lilliefors Significance Correction.95
If these two tests resulted in a P-value of less than =0.05 , then an additional Normal
Quantile Plot was necessary to meet the normality assumption. To test the homogeneity
of variance, the Levene test was used. Since more than two groups were being compared,
it was necessary to conduct a 1-way ANOVA (Analysis of Variance) test. This test
assured that equal variance assumptions were satisfied and whether or not there was
statistical significance between the labels. Statistical significance is present when
ANOVA significance levels are less than =0.05 between the groups.
In the instance where statistical significance was observed in the ANOVA testing,
additional testing was necessary to identify what labels were statistically significant. The
Scheffe test was used to discover which labels were statistically significant. This test was
used due to its ability to account for a differing number of participants for each label
tested. Lastly, to test differences among the labels for all the questions combined the
95
The Lilliefors Significance Correction adjusts for the estimation of population parameters such as, mean
and variance (or standard deviation) rather than having known values. The Lilliefors correction generally
provides a better test than an uncorrected approach.
44
Kruskal Wallis test was used. Statistical significance is achieved when the Chi-Square
test P-value is less than =0.05.
In instances where all assumptions for the 1-way ANOVA test were not satisfied,
the Welch/Brown-Forsythe test had to be used to ensure robustness for the equality of the
means were satisfied.
In addition to statistical significance testing, the Pivot Table feature in Microsoft
excel was used to analyze the results of the surveys. This feature consolidated the data in
a way that allowed me to quickly compare the participants‘ responses across the different
labels. The data was then used to create graphs that visually represented the results.
While the graphs did not illustrate whether or not responses were statistically significant,
they did illustrate areas where response levels differed enough to suggest that there may
be a possibility for difference. These possible differences could be caused for a variety of
reasons but represent areas where further research may be needed.
45
FINDINGS
No statistical significance was found between any of the labels with the exception
of two instances. Statistical significance was found between the ‗group‘ and ‗rebel‘
labels for the likelihood of an attack for question #12 (the effect of overall increases in
activity), and statistical significance was found between the ‗group‘ and ‗militia‘ labels
for question #15 (the effect of COMINT reporting increased levels of anti-government
rhetoric).
Figure 4.1
Statistical significance between the ‗group‘ and ‗rebel‘ label regarding the overall
increase in activity had a 1-way ANOVA P-value of 0.017 and a Scheffe P-value of
0.036, which are both significantly lower than the required =0.05 values. The ANOVA
test found that there was a difference between the labels, while the Sheffe test identified
which labels were significantly different. The difference between the labels indicates that
46
more weight is given to this piece of evidence (data anchor) when assessing the
likelihood of an attack for rebel groups compared to a neutrally emotive label such as
‗group.‘
Regarding statistical significance between the ‗group‘ and ‗militia‘ labels
regarding how COMINT indications affect the likelihood of an attack, the Tamhane test
had to be used in place of the 1-way ANOVA test because variance assumptions were not
satisfied using the Levene‘s test. The Tamhane test resulted in a P-value of 0.042
between the ‗group‘ and ‗militia‘ labels, indicating that analysts place more weigh on the
COMINT data anchor compared to a neutrally emotive label.
To better organize the findings of the questions, I have organized the findings into
subsections according to the variable questions asked in the survey. Each subsection
discusses the statistical significance levels of the responses as well as any other relevant
findings observed. The subsection discussing the results of questions 18 and 19 did not
get analyzed for statistical significance, as they required the analysts to rank the violence
and severity levels of the labels.
Questions 4&5:
Overall likelihood of an attack
Statistical analysis of participants‘ assessment regarding the overall likelihood of
an attack across all the labels used in this experiment (Question #4) resulted in no
statistical significance. All labels met the tests for normality and equal variance
assumptions. In addition, none of the labels contained any outliers in the participants‘
responses.
47
Although none of the responses were statistically significant from each other, it is
important to note some of the

Figure 4.2
percentage differences among
the likelihood levels.
Participants who received the
‗group‘ label gave the largest
volume of ―unlikely‖
responses, equaling 13% of
the responses and surpassed
the 5-7% range of the other *Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’
‘M,’ ‘I,’ and ‘G’ to indicate the terrorist, rebel, militia, insurgent,
labels (approximately twice as and group labels respectively
much). In addition, the ‗rebel‘ label received the highest volume of ―highly likely‖
responses, comprising of 7% of the respondents‘ assessments – this response far
outweighed the ―highly likely‖ responses from the participants who received the other
labels (7% versus the 2-3% range). The percent difference of the ‗rebel‘ label indicates
that this particular label possibility evokes some type of cognitive bias, or at least a
particular emotive response, when included in the African context. The ‗group‘
percentage of ―unlikely‖ responses are possibly indicative of lesser amounts of cognitive
bias, or are at least less emotive responses, when compared to the other labels.
Statistical analysis of participants‘ analytic confidence levels for the overall
likelihood of an attack (Question #5) also resulted in no statistical significance. All
labels meet the tests for normality and equal variance assumptions even though several
outlier responses were noted for all labels except ‗insurgent‘ and ‗rebel.‘
48
Participants who received
Figure 4.3
the ‗terrorist‘ and ‗rebel‘ labels
indicate the possibility of being
more confident in their responses
when compared to the other
labels. This possibility is
evidenced by both labels having a
7% analytic confidence level of
―9‖ on a 10-point scale; an answer

*Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’
of ―10‖ represents an extremely ‘M,’ ‘I,’ and ‘G’ to indicate the terrorist, rebel, militia, insurgent,
and group labels respectively
high confidence level.
Questions 6&7:
Likelihood of a Successful Attack
Statistical analysis of
Figure 4.4
participants‘ assessment
regarding the likelihood of a
successful attack across all
the labels (Question #6)
resulted in no statistical
significance among any of the
labels in question. All labels
meet the tests for normality
49
and equal variance assumptions. In addition, no outliers were present among the
responses, satisfying another requirement for normality.
The Shapiro-Wilk test for normality did indicate that some slight variance among
the labels was possible, particularly between the ‗group,‘ ‗insurgent,‘ and ‗terrorist‘
labels. These differences do indicate that this area might need further testing. The trend
line of the bar graphs of these labels also evidenced this variance possibility. The trend
line of the participants‘ responses for the ‗group‘ label illustrate a downward slope –
representing the possibility that this label is least likely to execute successful attacks (see
Figure 4.4).
The trend line for the

Figure 4.5
‗insurgent‘ responses illustrate a
slope of near zero (see Figure
4.5), indicating that the
participants who received this
label had difficulty placing a
value on the success level of
insurgent attacks. The levelness
of the trend line may be caused in part because of the varying success levels of insurgent
attacks against US and coalition forces in Iraq.
50
Figure 4.6
Lastly, the responses to the
‗terrorist‘ label indicated an
increasing trend line regarding the
success of an attack (see Figure
4.6). The increasing trend line
indicates a possibility among
respondents that terrorists, as a
whole, are more likely than the other labels to execute successful attacks. This trend
could be attributed to the amount of attention received by the media when terrorist attacks
are successful compared to the amount of media attention received when terrorist attacks
are thwarted.
Statistical analysis of participants‘ analytic confidence levels for the likelihood of
a successful attack (Question #7) resulted in no statistical significance among the labels.
All labels met the tests for normality and equal variance assumptions even though several
outlier responses were noted for all labels except ‗group,‘ ‗insurgent,‘ and ‗rebel.‘
Questions 8&9:
Likelihood of a Sophisticated Attack
No statistical significance was identified between responses concerning the
sophistication levels of an attack (Question #8) among the different labels. Normality
and equal variance assumptions were met for all the labels. Several outliers were present
among the responses for the ‗group‘ and ‗insurgent‘ labels, however normality was still
satisfied despite the outliers.
51
When visually representing the respondents‘ answers by percentages in a
doughnut graph, responses to the ‗insurgent‘ label differ greatly from the other labels.
The doughnut graph illustrates a much larger number of overall responses that indicate
that the sophistication of an attack

Figure 4.7
by an insurgent group is unlikely.
This is evident by the total of
responses of ‗somewhat less than
even,‘ ‗somewhat unlikely,‘
‗unlikely,‘ and ‗highly unlikely‘
responses – comprising of 87% of
the participants‘ responses of a
negative sophistication level (see *Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’
Figure 4.7). and group labels respectively
When comparing the negative sophistication responses against the other labels‘
negative sophistication responses, the ‗insurgent‘ label received a far more negative
sophistication response (terrorist=72%, rebel=69%, militia=68%, and group=75%).
Again, this trend may be possibly contributed to the sporadic insurgent attacks against US
and coalition forces in Iraq.
Statistical analysis of participants‘ analytic confidence levels for the likelihood of
a successful attack (Question #9) resulted in no statistical significance among the labels.
All labels met the tests for normality and equal variance assumptions – one single outlier
was noted for the ‗group‘ label, but single outliers are acceptable for normality
assumptions.
52
Questions 10&11:
Severity of an Attack
Statistical analysis of participants‘ assessment regarding the likelihood of an
attack being severe (Question #10) resulted in no statistical significance. All labels meet
the tests for normality and equal variance assumptions. In addition, only one single
outlier was present with the ‗group‘
label, which also meets Figure 4.8
normality assumptions.
The doughnut graph
shows some interesting results
however (see Figure 4.8). The
‗terrorist‘ label received the
largest volume of ―not severe‖
answers (36%) among *Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’
participants assessing the and group labels respectively
potential severity level if an attack were to occur – this assessment is only 2% greater
than the ‗group‘ label. In contrast, the ‗militia‘ label received the highest volume of
―very severe‖ (3%) and ―severe‖ (24%) answers, indicating that this label dishes out the
most potential to conduct severe attacks (76% of respondents assessed this label of
conducting attacks that were ―somewhat severe‖ or greater). The ‗rebel‘ label was
assessed as the second most-severe label with 73% of respondents giving the label an
assessment of at least ―somewhat severe.‖
53
Statistical analysis of participants‘ analytic confidence for severity levels of an
attack (Question #11) resulted in no statistical significance among the labels. The
‗group‘ label and the ‗terrorist‘ label met normality assumptions via the Kolmogorov-
Smirnov and Shapiro-Wilk tests. All other labels met the tests for normality and equal
variance assumptions via Quantile plots.
Although the analytic confidence levels of the participants among the labels did
not achieve statistical significance, analysis of the responses represented by percentage
level alludes to a possibility that analysts are more confident when assessing the terrorist
and rebel groups, at least in the context of Africa. The doughnut graph illustrating the
difference in confidence levels across the labels. The ‗terrorist‘ label received the highest
levels of confidence among all of the other labels. It was the only label that received an
analytic confidence level of ―10‖ (representing an extremely high confidence level), of
which 5% of respondents answered. 7% of the participants with this label had a
confidence level of ―9,‖ and 24% had a confidence level of ―8.‖ Therefore, more than
one-third of the participants who

Figure 4.9
received this label had a confidence
level greater than ―8.‖
Participants who had the
‗rebel‘ label also had reasonably high
levels of analytic confidence. 10% of
these participants had confidence
levels of ―9,‖ while an additional *Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’
and group labels respectively
54
19% had a confidence level of ―8.‖ The higher levels of analytical confidence are
possible indicators that these labels cause a stronger sense of certainty among analysts
when assessing the severity of a possible attack.
Question 12:
Effect of an Overall Increase in Activity
Statistical analysis of participants‘ assessment of how indications of an overall
increase in activity affect the likelihood of an attack (Question #12) resulted in statistical
significance. The Scheffe test revealed the cause of the statistical significance to be
between the ‗group‘ label (control group) and the ‗rebel‘ label. All labels meet the tests
for normality and equal

Figure 4.10
variance assumptions. In
addition, while outliers were
present for the ‗insurgent,‘
‗militia,‘ and ‗rebel‘ labels,
normality was still maintained.
The doughnut graph
(Figure 4.10) comparing the
participants‘ responses regarding *Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’
the effect of increased activity on and group labels respectively
the likelihood of an attack reaffirms the statistical significance. The graph indicates the
presence of ―highly increases the likelihood of an attack‖ assessments (at 2% - however,
this may be caused in part by an outlier) for the ‗rebel‘ label, as well as a large volume
55
(19%) of ―Does not affect the likelihood of an attack‖ assessments from participants
receiving the ‗group‘ label.
In addition to the statistical difference between the ‗group‘ and ‗rebel‘ labels, the
doughnut graph also illustrates a fairly large difference with the militia label as well.
Three percent of the participants also gave the ‗militia‘ label assessments that the overall
increase in activity ―highly increases the likelihood of an attack.‖ Furthermore, the
statistical significance level between the ‗group‘ label and the ‗militia‘ label was scored
as 0.233 – which was the closer than the ‗insurgent‘ and ‗terrorist‘ labels (see Table 4.1).
Table 4.1
56
While this is by no means statistically significant, it does illustrate a potential area
that could use further research since no other labels received this strong of an assessment.
The reasoning behind the responses might be due, at least in part, because of the amount
of rebel and militia violence in Africa. Increased activity may possibly be a clear
indication that these two particular labels (rebel and possibly militia) are more likely to
attack, especially in light of the rebel and militia clashes in places such as Darfur. The
difference between the ‗group‘ label compared to the ‗rebel‘ and ‗militia‘ labels reveal
the possibility that the ‗group‘ label is less emotive, or has less cognitive biases
associated with it, than the other two labels.
57
Question 13:
Effect of Imagery Intelligence Indicating Increased
Activity Near a Training Base or Command Post
Statistical analysis of participants‘ assessment regarding the effect of imagery
intelligence (IMINT) indicating increased activity near a training base/command post on
the likelihood of an attack (Question #13) resulted in no statistical significance. All
labels did satisfy normality and variance testing.
Interestingly, the doughnut graph (Figure 4.11) illustrates that this IMINT
reporting put less weight on the likelihood that an insurgent group will attack compared
to the other labels. This assertion is evidenced by 25% of the participants who received
the ‗insurgent‘ label assessing that IMINT ―Does not affect the likelihood of an attack‖ –
leaving a gap of 10% between

Figure 4.11
the next closest label,
‗group.‘ This possibility is
most likely attributed to the
media reporting of
insurgent attacks in Iraq
against US and coalition
forces, which possibly
leaves the analyst with *Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’ ‘M,’ ‘I,’
and ‘G’ to indicate the terrorist, rebel, militia, insurgent, and group labels
some sense of vividness bias respectively
toward the seemingly sporadic and fast-past attacks of insurgent groups. If analysts are
left with the impression that insurgent groups use a fast-paced and sporadic modus
58
operendi (MO), then IMINT would not likely be an accurate indicator that an attack is
likely (unless, of course, the IMINT was streaming in real time to the analyst).
Question 14:
Effect of Local Newspaper Reporting an Attack Will Not Occur
Statistical analysis of participants‘ assessment regarding how local newspaper
reporting that an attack will not occur affects the likelihood of an attack (Question #14)
resulted in no statistical significance. Although there were several outliers for all the
labels in question, the labels did satisfy all tests for normality and equal variance
assumptions.
The outliers are more Figure 4.12
evident when examining the
graph comparing the
respondents‘ assessments.
Fifty percent of the
participants across all the
labels in questions indicated
that local newspaper reporting *Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’
―does not affect the likelihood of and group labels respectively
an attack‖ (represented in purple in Figure 4.12).
In particular, participants who received the ‗terrorist‘ label varied widely in their
assessments of the how the newspaper reporting affects the likelihood of an attack. The
participants indicated significant responses on both ends of the likelihood continuum,
with 10% assessing that the reporting ―decreases the likelihood of an attack,‖ and 7%
59
assessing that the reporting, ―significantly Figure 4.13
increases the likelihood of an attack.‖ Outlying
responses were also significant for the ‗rebel‘
and ‗militia‘ labels (See Figure 4.13).
Question 15:
Effect of Communications Intelligence Reporting
Increased Levels of Anti-Government Rhetoric
Statistical analysis of participants‘ assessment regarding the effect of
communications intelligence (COMINT) indicating increased levels of anti-government
rhetoric on the likelihood of an attack (Question #13) resulted in statistical significance.
All labels did satisfy normality testing using the Quantile plot. The Welch test found
statistical significance between the different labels. Because variance levels were not
equal, the Tamhane Hoc test was needed in order to find what labels were significantly
different from each other. This test found a statistically significant difference between
the ‗group‘ label and the ‗militia‘ label (See Table 4.2 on page 61).
60
Table 4.2
The difference between the ‗group‘ label and the ‗militia‘ label likely stems from
the overwhelming response (88%) that COMINT will, in various levels of likelihood, will
be an indicator that an attack is more likely – only 68% of the ‗group‘ label participants
rated it the same way. In addition to the difference between the ‗group‘ and ‗militia‘
labels, the ‗insurgent‘ label was close to statistical significance from the ‗militia‘ label,
indicating that there might be a possibility that these two labels differ regarding how
COMINT is assessed and associated with them. Interestingly, assessments of the
61
‗insurgent‘ label were more similar to the ‗group‘ label than any of the other labels (75%
responded that COMINT will, in various levels of likelihood, be an indicator of an
attack). Fifty percent of those who received the ‗insurgent‘ label stated that COMINT
only ―somewhat increases the likelihood of an attack.‖
Also of interest is
Figure 4.14
the fact that none of the
participants stated that
COMINT will, in any
varying level, decrease
the likelihood of an
attack. This is not so
interesting in and of
itself, but it is the only *Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’ ‘M,’ ‘I,’
piece of evidence (or respectively
data anchor) that evoked either a neutral or positive indication assessment – all other data
anchors had both positive and negative likelihood responses.
Question 16:
Effect of a Long History of Social Strife and Discontentment
Statistical analysis of participants‘ assessment regarding the effect of a long
history of social strife and discontentment on the likelihood of an attack (Question #16)
resulted in no statistical significance. All labels met the tests for normality and equal
variance assumptions. In addition, only a few outliers were present with the ‗insurgent‘
and ‗militia‘ labels, which also met normality assumptions.
62
Although no statistical significance was noted among the labels, analysis of the
doughnut graph indicates a fairly large response (24%) of ―does not affect the likelihood
of an attack‖ assessments by the participants of the ‗terrorist‘ label. This response
percentage is larger compared to the other labels tested, as 16-17% of their participants
assessed those labels in the same manner.
The ‗rebel‘ and ‗militia‘

Figure 4.15
labels both received a relatively
high volume of responses (7%)
indicating that the history of the
region ―highly increases the
likelihood of an attack.‖ In
addition, 83% of participants for
the both the ‗rebel‘ and ‗militia‘

*Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’ ‘M,’
‘I,’ and ‘G’ to indicate the terrorist, rebel, militia, insurgent, and labels indicated that the history at
group labels respectively
least ―somewhat increases the
likelihood of an attack.‖ The large volumes for these two labels could possibly be
associated with the amount of violence in Africa attributed to rebel and militia groups;
however further testing would be necessary to confirm this.
Question 17:
Effect of the Region Being a Hot-Bed of Differing Political and Religious Ideologies,
Widespread poverty, and Poor Health Conditions
Statistical analysis of participants‘ assessment regarding how different political
and ideological views, as well as widespread poverty and poor health conditions, affects
the likelihood of an attack (Question #17) resulted in no statistical significance. All
63
labels meet the tests for normality and equal variance assumptions. In addition, only one
outlier was present for the ‗insurgent‘ and ‗group‘ labels – meeting normality
assumptions.
Analysis of the doughnut graph indicates that this question may benefit from
further testing as the ‗group‘ label responses appear to quite different responses compared
to the labels being tested. For example, 17% of the ‗group‘ label participants assessed
that this data anchor ―does not affect the likelihood of an attack.‖ This response volume
is larger than that of the other labels (9-12% response volume). In addition, the ‗rebel‘
and ‗militia‘ labels had higher volumes of ―highly increases the likelihood of an attack‖
assessments (10% and 7% respectively) compared to the other labels (see Figure 4.16).
These responses are Figure 4.16

indicative that the ‗group‘
label is more neutrally emotive
than the other labels. In
addition, the ‗rebel‘ and
‗militia‘ labels draw out more
cognitive biases, or are at least
more emotive, than the other

*Note: The Rings (from outer to inner) are labeled with a ‘T,’ ‘R,’ ‘M,’ ‘I,’
labels tested. While these
respectively
results are not statistically
significant, they do correspond with the statistically significant, and other findings, in this
experiment – that the ‗rebel‘ and ‗militia‘ labels are more likely to engage in violent
64
activities in the context of Africa and other data anchors included in the mock
intelligence report.
Questions 18&19:
Ranking of the Labels Based on Their Likelihood to Engage in Violence
and Their Likelihood to Execute Severe Attacks
To rank the labels by Figure 4.17
their levels of violence and
severity (Questions 18 and 19
respectively), the scoring
averages (ranked from 1 to 4) for
the labels were taken and
compared to each other. For
levels of violence, the labels (in
order from most to least violent) were assessed as: Insurgent, Rebel, Terrorist, and
Militia. The severity assessments were nearly the inverse of the violence ranking, being
ranked (most to least) as: Terrorist, Militia, Rebel, and Insurgent.
65
CONCLUSION
Trends in Key Findings
My hypothesis that these particular labels have significant meaning, and many
individuals have a preconceived idea, or cognitive biases, regarding the kinds of actions
each of these particular groups conduct must be rejected at this time due to an overall
lack in statistical significance across the labels. There are two main possibilities that
might explain why the hypothesis was disproved. The first possibility is that, in general,
cognitive biases do not apply actually to these labels. This is the most desired possibility
because, if this were true, intelligence analysts do not allow labels to convolute the
importance of evidence, or in other words, they avoid the label as an anchoring bias. The
other possibility is that having the setting of the intelligence report focus around Africa,
because of the presence of all the labels and the amount of violence they engage in,
allowed for distinct similarities among the assessments for the labels in question. If this
were true, it is possible that the participants had similar assessments about this specific
scenario; but if given a different context, analysts may have different responses. I posit
that the latter is more likely to be true. However, further in-depth research will be
necessary to fully conclude that labels do or do not have an effect on intelligence
analysis.
Although the hypothesis must be rejected, some important conclusions can be
drawn out from the findings. First, the statistically significant results must be discussed.
Statistical testing revealed that the ‗rebel‘ label differed from the control group (labeled
as ‗group‘) when participants assessed the effect of an overall increase in activity on the
likelihood of an attack. The difference is indicative that the ‗rebel‘ label is perceived as a
66
greater threat than the other labels when reporting indicates an overall increase in
activity. The fact that the other labels were not statistically significant is likely attributed
to the cognitive biases toward rebel group activity in Africa. Cognitive biases and
heuristics such as vividness bias and the availability heuristic might play into the
perceived actions of rebel groups in Africa. In addition, illusory correlation biases (the
assumption that there is a relationship between a particular action and an effect) might
also be skewing the results.
The same biases and heuristics could also be affecting the responses of how
COMINT affects the likelihood of an attack for militia groups. Statistical testing also
found significance between the ‗group‘ label and the ‗militia‘ label for this data anchor.
This difference could possibly be attributed to the assumption that all or most African
militia groups use technology that could be intercepted by communication collection
methods. If analysts presume that militia groups use more sophisticated communications
equipment, this might explain why the ‗militia‘ label was statistically significant from the
‗group‘ label. For both questions that had statistical significance, further research will be
necessary to accurately answer why they were significantly different.
Analysis of the responses across all questions displayed possible trends for some
of the labels. In general, the ‗group‘ label had a tendency to evoke assessments
suggesting that this label is less likely than the other labels to engage in acts of violence.
Specifically, compared to the other labels, there was a trend for the ‗group‘ label to be
assessed on lower levels for the overall likelihood of an attack and for the success of an
attack. In addition, data anchors such as the increased activity, the COMINT, and the
ideology/social issues tended to have less of an impact on the likelihood of an attack for
67
the ‗group‘ label. This trend indicates that the ‗group‘ label is likely to have fewer biases
associated with it, or at a minimum, is less emotive than the other labels tested.
In contrast to the ‗group‘ label, the ‗rebel‘ label and the ‗militia‘ label showed
trends of being assessed as the most likely labels to engage of acts of violence. The
‗rebel‘ label showed greater likelihood in areas where participants assessed the overall
likelihood of an attack (as well more confidence in this assessment) and the severity of an
attack (and its associated analytic confidence). In addition, this label was assessed to be
more likely to attack with data anchors such as increased activity, history, and
ideology/socioeconomics. The ‗militia‘ label showed greater likelihood in areas where
participants assessed the severity of an attack. In addition, this label was assessed to be
more likely to attack with data anchors such as increased activity, COMINT, history, and
ideology/socioeconomics.
Table 5.1 illustrates questions in which the labels either had a positive response
(red, and ranked with ―-1‖), neutral response (yellow, and ranked with ―0‖), or negative
(green, and ranked with ―1‖) response to the assessment of the situation or the effect of
the data anchor. The positive responses indicate an increased likelihood, while the
negative responses indicate decreased likelihood.
Table 5.1
Question 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Group -1 0 -1 0 0 0 0 0 -1 0 0 -1 0 -1
Insurgent 0 0 0 0 -1 0 0 0 -1 -1 0 -1 0 0
Militia 0 0 0 0 0 0 1 0 1 0 0 1 1 1
Rebel 1 1 0 0 0 0 1 1 1 0 0 0 1 1
Terrorist 0 1 1 0 0 0 -1 1 0 0 0 0 -1 0
P-Value 0.71 0.514 0.602 0.485 0.259 0.713 0.162 0.444 0.017 0.18 0.886 0.011 0.646 0.192
68
Supporting the possibility that participants are susceptible to cognitive biases,
despite the overall lack of statistical significance, is the vastly different response for the
‗terrorist‘ label in question 10 and question 19. Both questions pertained to the severity
of an attack. Question 10 asked participants, basing their assessment on the information
provided within the mock intelligence report, to assess how severe an attack would be if
it were to occur. Question 19 asked the participants to rank the ‗rebel,‘ ‗militia,‘
‗insurgent,‘ and ‗terrorist‘ labels according to their likely severity levels – since
participants were exposed to only one of these labels in the report, their assessments for
question 19 would not have stemmed from information contained within the mock
Analysis of responses for question 10 across all labels in question indicates that
the participants assessed the severity of the labels (from most severe to least severe) to
be: militia, rebel, insurgent, and terrorist. This result was found through the sum of
―somewhat severe,‖ ―severe,‖ and ―very severe‖ answers and then comparing the volume
of those responses across each of the label being tested.
When participants were asked to rank the labels by their level of severity (and out
of the context of the data anchors that were provided in the mock intelligence report), the
‗terrorist‘ label was ranked as being the most severe out of all the labels. The other labels
followed the same order (militia, rebel, & insurgent) as in question 10, however, the
‗terrorist‘ label jumped from being the least severe in question 10 to the most severe in
question 19. The difference in assessing the severity of the ‗terrorist‘ label is a possible
69
indication that the context of the mock intelligence report drew out cognitive biases in the
participants‘ analyses.
Despite the possible cognitive biases pertaining to the ‗terrorist‘ label, we can
conclude the ranking of severity for the other labels is likely to be accurate (since the
order matched in both questions 10 and 19). It is likely that analysts believe that attacks
by militia groups are generally more severe than attacks by rebel groups, and that rebel
attacks are generally more severe than attacks by insurgents. Further testing of the
‗terrorist‘ label will need to be conducted to find out where exactly this label fits on the
severity continuum.
Another trend discovered in the key findings is among the relationship between
the ‗insurgent‘ label and information derived from more sophisticated collection
techniques such as IMINT and COMINT. For both questions asking participants to
assess the effect of IMINT and COMINT on the likelihood of an attack, participants with
the ‗insurgent‘ label indicated that these data anchors have less of an impact for this label
when compared to the other labels. This trend may possibly be attributed to a belief that
insurgents are less likely than the other labels to use communication devices that can be
intercepted, or at a minimum, the communications intelligence reports have less weight
for insurgent groups because of their MO. The same may be true for IMINT, this type of
intelligence may also be a less reliable source for indicating likely insurgent attacks, at
least when compared to the other labels.
The variance in participants‘ assessments of how local news reporting
contradicting other intelligence affects the likelihood of an attack was also worthy to
note. Due to the amount of outlying responses to this question, as well as the variance in
70
assessments across all the labels, is indicative that there is a significant level of
uncertainty with regards to how credibility of foreign, local news. This variance could
possibly be attributed to the lack of identifying the source of the local news, which would
help to explain the variance. Without sourcing the news agency, it appears that some
analysts trusted the reporting, while others likely found it to be a deception tactic.
Only two of the questions resulted in statistical significance, however, a few of
the other questions had lower P-values that were approaching statistical significance.
Questions asking about the sophistication of an attack (question #8, P-value of 0.259), the
severity of an attack (question #10, P-value of 0.162), the effect of IMINT (question #13,
P-value of 0.180), and the effect of differing ideologies and socioeconomic turmoil
(question #17, P-value of 0.192) possibly show indications that these data anchors may
have more relevance on the analysis of these labels.
Research Recommendations
Since the findings of my experiment aren‘t entirely conclusive, the continuation
of this research is important. There are many avenues that can be taken to continue the
research on the effect of labels on analysis, and all possible avenues should be
researched, compiled, and compared before any solid conclusions about labeling effects
are made.
If further research is conducted using my experiment model, I would suggest
repeating the same experiment, but changing out the location of scenario. It would be
interesting to compare the results of my participants, who had to assess these labels in the
context of Africa, against multiple other locations. My initial thought is that if the results
of the African setting were compared against a setting such as Iraq, Afghanistan, or
71
Columbia, we would likely see varied responses across the labels. However, due to the
size of each experiment (as it compares five different labels), this type of experiment
might be difficult to execute and may take the longest amount of time to complete.
Another option, which might prove to be quicker and reduce the biases associated
with the setting of a scenario, would be to conduct the same experiment but change out
the African setting for an imaginary country. This experiment could be quickly
conducted using the same data anchors and questions that were used in my experiment.
The best option, in my post-experiment opinion, would be to use a Bayesian
statistical model to assess the weight given to a label when analyzing intelligence. Again,
all five labels would have to be tested separately and no specific setting should be
included. Rather than having participants read a mock intelligence report and then
answer questions, individual pieces of evidence (data anchors) would be presented to the
participant one at a time. The participants would then have to assess the likelihood of an
attack (on a 1-10 scale) given that particular piece of evidence (or given the next
additional piece of evidence). If this methodology were used, two main things could be
analyzed. First, the researcher would be able to accurately compare the weight given to
the label for each piece of evidence presented. Second, after computing the responses for
all pieces of evidence given (using Bayesian statistics), an overall assessment, in
numerical form, for each label would be evident. The assessments for each label could
then be compared against each other using the statistical methods used in this experiment
such as the 1-way ANOVA test.
One final option for further research would be to use a KISS (Keep It Simple
Stupid) approach. This could be conducting with questions asking the participants to
72
rank the labels according to various characteristics (i.e. severity, success, sophistication,
etc.). Although this method may be simple in form, it is not as robust as other methods
such as Bayes.
As a way to provide more pointed information about the groups being tested, as
well as possibly simplifying the research, future studies should consider looking at only
one group of intelligence professionals at a time. For example, consider conducting the
same research that was used in this study, but rather than trying to incorporate multiple
intelligence professions in one study, use only intelligence analysts in the field of national
security, or law enforcement, or military, or students, etc. If research was conducted in
this manner, there is great likelihood that the researcher will be able to effectively
examine the variances among participants‘ assessments by experience level. Research
delving into the analytical differences among analysts and categorized by experience
level, would provide invaluable insight into the intelligence profession.
Moving Forward
Cognitive biases and heuristics will continue to be an obstacle for the intelligence
analyst to overcome. Since biases and heuristics can have a negative effect on the way
we think about the information presented to us, it is in the best interest of the analyst to
ensure that he or she has done everything possible to reduce bias and increase analytic
accuracy. Richards Heuer has suggested methods such as Analysis of Competing
Hypotheses as a means to assist the intelligence analyst in the battle to overcome bias.96
Methods such as ACH, as well as other sound and structured methodologies, prove to be
the best approach to reduce bias, increase transparency, and improve analysis.
96
Heuer, Richards., Psychology of Intelligence Analysis, (Nova Science 1st ed, 2005), Ch 8.
73
If intelligence analysts make a habit out of using structured methodologies to
evaluate the information presented to them, it will force them to recognize alternate
possibilities that are often hidden by bias. Once all options are out-on-the-table, the
methodologies will aid the analyst in making the best possible assessment.
Lastly, it is important for analysts to be as specific as possible when faced with a
term that labels. Whenever possible, rather than using a term such as ‗terrorist,‘ or
‗rebel,‘ analysts should try to use the recognized name of the group, such as Al Qaida or
the Justice and Equality Movement (JEM). In addition, when faced with larger reports
containing multiple pieces of evidence, analysts should attempt to break the report up into
its parts, separating each piece of evidence from the whole. Each piece of evidence
should then be analyzed individually, without the context of the label. After this has been
done, all pieces of evidence can then be compiled and analyzed as a whole. When
analysts take away labels (or replace them with specific names) and focus more on the
evidence by analyzing them individually, the better the chances will be for reducing
cognitive bias, avoiding heuristics, and improving intelligence – and this should always
be a primary goal for the analyst.
REFERENCES
22 U.S.C. 2656f (d). "Chapter 7 -- Legislative Requirements and Key Terms."

U.S.Department of State. http://www.state.gov/s/ct/rls/crt/2007/103715.htm
(accessed March 30, 2009).
Anti Defamation League. "The Quiet Retooling of the Militia Movement: Recruitment."
ADL: Fighting Anti-Semitism, Bigotry and Extremism.
http://www.adl.org/extremism/Militia/Recruitment.asp?m_flipmode=4 (accessed
April 2, 2009).
74
Bartlett, F.C., Remembering: A Study in Experimental and Social Psychology,
(Cambridge, England: Cambridge UP, 1932).
BBC. "Sudan's Shadowy Arab Militia." BBC NEWS | News Front Page.
http://news.bbc.co.uk/1/hi/world/africa/3613953.stm (accessed April 06, 2009).
Central Intelligence Agency, Guide to the Analysis of Insurgency, C05332177.

Washington DC,; Central Intelligence Agency, 2. Released January 05, 2009.
http://www.fas.org/irp/cia/product/insurgency.pdf
"CGI Scripts: Random Link." CGI Spy. www.cgispy.com/scripts/rlink.html (accessed

April 21, 2009).
CNN. "Janjaweed Militia: News & Videos about Janjaweed Militia - CNN.com."
CNN.com - Breaking News, U.S., World, Weather, Entertainment & Video News.
http://topics.cnn.com/topics/janjaweed_militia (accessed April 07, 2009).
Conlisk, John. ―Bounded Rationality and Market Fluctuations.‖ Journal of Economic

Behavior & Organization 29. No. 2 (1996) 233-250.
Cronin, Kurth . "Behind the Curve: Globalization and International Terrorism."

International Security, 27(3) (Winter 2002-2003).
"Data on Armed Conflict - CSCW." PRIO - International Peace Research Institute, Oslo.
http://www.prio.no/CSCW/Datasets/Armed-Conflict/ &
http://www.prio.no/sptrans/1664678440/Codebook.pdf (accessed April 15,
2009).
FBI, Counterterrorism Threat Assessment and Warning Unit, National Security Division.
―Terrorism in the United States.‖ (1996): 7-8.
www.fas.org/irp/threat/fbi_terror_96.pdf (accessed April 9, 2009).
Gigerenzer, Gerd., ―Fast and Frugal Heuristics: The Tools of Bounded Rationality.‖ In
The Blackwell Handbook of Judgment and Decision Making edited by Derek J.
Koehler and Nigel Harvey, (Malden, Ma: Blackwell Publishing, 2004).
Goffman, E., Frame Analysis: An Essay on the Organization of Experience, (NY: Harper,
1974).
Gurr, Ted R.. Why Men Rebel. Princeton: Princeton University Press, 1971.
Haider-Markel, Donald P., and Sean P. O'Brien. "Creating a "Well Regulated Militia":
Policy Responses to Paramilitary Groups in the American States." Political
Research Quarterly 50(3) (1997).
Heuer, Richards., Psychology of Intelligence Analysis, (Nova Science 1st ed, 2005).
75
Hoffman, Bruce. Inside Terrorism. New York: Cambridge University Press, 1998.
―insurgent, a. and n." The Oxford English Dictionary. 2nd ed. 1989. OED Online. Oxford
University Press. 26 Aug. 2008 http://dictionary.oed.com/cgi/entry/50118488.
Jacobellis v. Ohio, 378 U. S. 184 (1964)
Johnston, Rob., ―Analytic Culture in the US Intelligence Community: An Ethnographic

Study.‖ (Washington D.C.: Center for the Study of Intelligence, Central
Intelligence Agency, 2005). Chapter 2. https://www.cia.gov/library/center-for-the-
study-of-intelligence/csi-publications/books-and-monographs/analytic-culture-in-
the-u-s-intelligence-community/chapter_2_research.htm
Jongman, A.J., and Alex P. Schmid. Political Terrorism: A new Guide to Actors,
Authors, Concepts, Data Bases, Theories, and Literature. New Brunswick:
Transaction Publishers, 1988.
Kahneman, D. & Tversky, A. (eds.), Choices, Values and Frames. (New York, NY:
Russell Sage Foundation & Cambridge University Press, 2000), ix-xvii
Kitonga, Nzamba. "The Fight Against Terrorism in East Africa: Comments and
Observations." East Africa Law Society Conference On ―Globalization And
Terrorism: New Threats To Regional Integration‖, East Africa Law Society,
Entebbe, Uganda, October 10, 2003.
Klein, G. et al, ―A Data/Frame Theory of Sensemaking.‖ In Expertise out of Context;

Proceedings of the Sixth International Conference on Naturalistic Decision
Making, edited by R.R. Hoffman. (Mahwah, NJ: Erlbaum, 2004).
Kronenwetter, Michael. Terrorism: A Guide to Events and Documents. New York:

Greenwood Press, 2004. vii.
―militia, n." The Oxford English Dictionary. Draft Revision 2008. OED Online. Oxford
Minsky, M., ―A Framework for Representing Knowledge.‖ In The Psychology of

Computer Vision, edited by P. Winston (New York, NY: McGraw-Hill, 1975).
Mynatt, C.R., Doherty, M.E., & Tweeney, R. D. Confirmation Bias in a Simulated

Research Environment. Quarterly Journal of Experimental Psychology, 29, 85-95
(1977).
Nelson, Debra L,. & Campbell, James Quick., Organizational Behavior: Foundations,
Realities, & Challenges, 5th ed. (Mason, OH.: Thomson South-Western. 2006).
76
O'Neill, Bard E.. Insurgency and Terrorism: From Revolution to Apocalypse; 2nd Ed.,
Revised. Washington, D.C.: Potomac Books, 2005.
O'Rourke, Matthew. "The Impact Of The ‗War On Terrorism‘ On Internal Conflicts."

The Ploughshares Monitor 26 (2005),
http://www.ploughshares.ca/libraries/monitor/monm05d.htm.
Patterson, E.S., Woods, D.D., Sarter, N.B., & Watts-Perotti, J.C. Patterns in Cooperative
Cognition. In COOP ‘98, Third International Conference on the Design of
Cooperative Systems. 1998.
Pillar, Paul R.. Terrorism and U.S. Foreign Policy. Washington, D.C.: Brookings
Institution Press, 2001.
Pollard, Kristine. "Jeffrey Welgan Thesis." Institute for Intelligence Studies at

Mercyhurst College.
http://74.125.95.132/search?q=cache:acF9XiY2Ly0J:www.cirat.com/welganthesi
s.php+http://www.cirat.com/welganthesis.php&cd=1&hl=en&ct=clnk&gl=us&cli
ent=firefox-a (accessed April 21, 2009).
Pollard, Kristine. "Jeffrey Welgan Thesis - Thank You." Institute for Intelligence Studies
at Mercyhurst College. http://www.cirat.com/welganthanks.php (accessed April
21, 2009).
―rebel.‖ (2009). In Merriam-Webster Online Dictionary. Retrieved April 15, 2009,

http://www.merriam-webster.com/dictionary/rebel
"rebel, a. and n.1" The Oxford English Dictionary. 2nd ed. 1989. OED Online. Oxford
Research Methods Knowledge Base. "Two-Group Experimental Designs." Social

Research Methods. http://www.socialresearchmethods.net/kb/expsimp.php
(accessed April 20, 2009).
Reuters. "Sudan Man Accused Of Aiding Darfur war Crime Court." Reuters.com - World
News, Financial News, Breaking US & International News.
http://www.reuters.com/article/homepageCrisis/idUSLM432733._CH_.2400
(accessed April 07, 2009).
Rudolph, Jenny. W. Into the big muddy and out again. Unpublished Doctoral Thesis,
(Boston College, Boston, MA., 2003).
Ruvalcaba, Jim. "Understanding Iraq's Insurgency." Al Nakhlah Article 7 (Spring 2004).
77
Schoemaker, P & Russo, J. ―Managing Frames to Make Better Decisions.‖ In Wharton
on Making Decisions, edited by S. Hoch and H. Kunreuther. (New York: John
Wiley & Sons, 2001).
Slovic, P., ―From Shakespeare to Simon: Speculations and Some Evidence About Man‘s
Ability to Process Information.‖ Oregon Research Institute Monograph 12, no. 2
(1972).
Soman, Dilip., ―Framing, Loss Aversion, and Mental Accounting.‖ In The Blackwell
Handbook of Judgment and Decision Making edited by Derek J. Koehler and
Nigel Harvey, (Malden, Ma: Blackwell Publishing, 2004).
"Survey Methods." School of Information - University of Texas.

http://www.ischool.utexas.edu/~palmquis/courses/survey.html (accessed April 20,
2009).
―terrorist." The Oxford English Dictionary. 2nd ed. 1989. OED Online. Oxford
Thaler, R. & Johnson, E., ―Gambling With the House Money and Trying to Break Even:
The Effects of Prior Outcomes on Risky Choice.‖ Management Science 36
(1990): 643-60.
U.S. Department of State. "E.O. 13224: Identified Terrorists and Groups." U.S.
Department of State. http://www.state.gov/s/ct/rls/fs/2001/6531.htm (accessed
April 16, 2009).
Wheaton, Kristan. "Surveys, Surveys, Surveys! (It's Research Season...)." Sources And
Methods. http://sourcesandmethods.blogspot.com/2009/01/surveys-surveys-
surveys-its-research.html (accessed April 21, 2009).
Woldemariam, Michael. "Why Rebels Collide: Factionalism and Fragmentation in

African Insurgencies." Africa: Conflict Transformation/Peacebuilding, Woodrow
Wilson Center, Washington DC, August 19, 2008.
78
Appendices
79
Appendix A
Participation Consent Form
JEFFREY WELGAN THESIS

Participation Consent Form
The purpose of this research is to test whether or not any cognitive biases exist
among intelligence professionals and students when they are given a common
Your participation involves a short instruction section online, followed by a short

mock-intelligence report. Following the intelligence report will be an online survey
comprised of 19 questions. This process should take no longer than 20 minutes.
Your identity will remain completely anonymous to the researcher and throughout
the entire thesis process.
There are no foreseeable risks or discomforts associated with your participation in

this study. Participation is voluntary and you have the right to opt out of the study
at any time for any reason without penalty.
By submitting your answers to the survey, you hereby acknowledge that your
involvement in this research is voluntary and agree that the use of your data can be
used for the purpose of this research. In addition, by clicking on the link below, you
are confirming that you are at least 18 years of age or have parental consent to
participate in this survey. If you choose to participate, you will be directed to
freeonlinesurveys.com. freeonlinesurveys.com does not collect any personal data, or
IP addresses from its participants.
Click here to participate in the survey.
(This link will take you to an intelligence report and survey- PLEASE NOTE: if the
survey does not appear when you click on the link above - please press F5 until it
does.)
If you have any further questions about this research feel free to contact me at
jeff.welgan@gmail.com
Research at Mercyhurst College which involves human participants is overseen by

the Institutional Review Board. Questions or problems regarding your rights as a
participant should be addressed to Tim Harvey; Institutional Review Board Chair;
Mercyhurst College; 501 East 38th Street; Erie, Pennsylvania 16546-0001;
Telephone (814) 824-3372. tharvey@mercyhurst.edu
80
Appendix B
Participation Debriefing Statement
JEFFREY WELGAN THESIS - THANK YOU
Thank you for participating in this research process. I appreciate your contribution
and willingness to support the student research process.
It is of utmost importance that you do not discuss the content of your scenario, or
your answers to the survey, with any possible future participants. Discussion with
any prospective participants may manipulate or alter their analysis.
If you know of any other analysts that would be interested or willing to participate
in this research, please send them to:
http://www.cirat.com/welganthesis.php
The purpose of this study was to determine whether or not cognitive biases exist
among intelligence professionals and students. Particular focus is being given on the
labels that are commonly used in many of today's intelligence reports – these labels
are terrorist, rebel, militia, insurgent, and group. Five intelligence reports are being
distributed as a part of this research, all identical to one another with the exception
of which label is being used.
Your answers to the questionnaire will be compiled with other participants' answers
that have received the same intelligence report. The different intelligence reports
will then be compared against other participants' answers to see whether or not
analysis changes solely based on which labels were used.
It is of utmost importance that you do not discuss the content of your scenario, or
your answers to the survey, with any possible future participants. Discussion with
any prospective participants may manipulate or alter their analysis.
This research is important because the use, or misuse, of particular labels can
directly affect US government action or inaction. Because Intelligence assists in the
decisions of policy makers, it is important to be precise and cautious in our use of
such labeling terms, and this is the purpose and significance of my research into this
topic.
Thank you once again for your participation.
If you have any further question about my research you can contact me at
jeff.welgan@gmail.com.
81
Appendix C
Institutional Review Board (IRB) Approval
Insert Approval Page here
82
Appendix D
Scenario 1: Group Activity (Control Group)
**THIS IS A MOCK SCENARIO. THE INFORMATION INCLUDED IS FABRICATED

AND NOT BASED ON ANY FACTUAL EVENTS**
SUBJECT: GROUP ACTIVITY
1. Summary. Northeast Africa and the Horn of Africa (HOA) regions have multiple groups.
Groups in the region have varying levels of violence and success. Some groups remain benign
with fluctuations in activity, while other groups consistently maintain high levels of violence.
Over the past month, multiple sources have indicated an increase in activity for an unidentified
group. The American Embassy in Djibouti has been unable to assess the likelihood of an attack
on Djibouti interests in the region (due to the wide variety of groups) and is reaching out to the
intelligence community for outside perspective and analysis.
2. Over the past month there has been an overall increase in group activity. Imagery intelligence
has indicated an increase of human activity near what appears to be a training base or command
post of the suspected group. Imagery analysts report that the group appears to have small arms
and other weapons however, due to poor satellite coverage and cloud cover, analytic confidence
in the imagery has decreased. Several local newspapers have indicated that the group ‖does not
plan to incite, or participate in, any violence in the future despite discontentment with the current
establishment.‖ Newspapers have added that the group was considering a peaceful protest,
however no specific dates have been set.
3. Recent COMINT has indicated an increase in anti-government rhetoric from the group based
upon the alleged social injustice and oppression. The Northeast and HOA regions have a long
history of social strife and discontentment. These social tensions stem from the region being a
hot-bed of differing political and religious ideologies, as well as widespread poverty and poor
health conditions.
4. Comment. In order to assist us in analyzing the likelihood of an attack on Djibouti interests,

and the level of sophistication if an attack were to occur, please fill out the attached survey.
Thank you for your help and support.
END REPORT
83
Appendix E
Scenario 2: Rebel Activity

SUBJECT: REBEL ACTIVITY
1. Summary. Northeast Africa and the Horn of Africa (HOA) regions have multiple rebel
groups. Rebels in the region have varying levels of violence and success. Some rebel groups
remain benign with fluctuations in activity, while other rebel groups consistently maintain high
levels of violence. Over the past month, multiple sources have indicated an increase in activity
for an unidentified rebel group. The American Embassy in Djibouti has been unable to assess the
likelihood of an attack on Djibouti interests in the region (due to the wide variety of rebel groups)
and is reaching out to the intelligence community for outside perspective and analysis.
2. Over the past month there has been an overall increase in rebel activity. Imagery intelligence
post of the suspected rebels. Imagery analysts report that the rebels appear to have small arms
in the imagery has decreased. Several local newspapers have indicated that the rebels ―do not
establishment.‖ Newspapers have added that the rebels were considering a peaceful protest,
3. Recent COMINT has indicated an increase in anti-government rhetoric from the rebels based
health conditions.

END REPORT
84
Appendix F
Scenario 3: Militia Activity

SUBJECT: MILITIA ACTIVITY
1. Summary. Northeast Africa and the Horn of Africa (HOA) regions have multiple militias.
Militias in the region have varying levels of violence and success. Some militias remain benign
with fluctuations in activity, while other militias consistently maintain high levels of violence.
Over the past month, multiple sources have indicated an increase in activity for an unidentified
militia. The American Embassy in Djibouti has been unable to assess the likelihood of an attack
on Djibouti interests in the region (due to the wide variety of militias) and is reaching out to the
intelligence community for outside perspective and analysis.
2. Over the past month there has been an overall increase in militia activity. Imagery intelligence
post of the suspected militia. Imagery analysts report that the militia appears to have small arms
in the imagery has decreased. Several local newspapers have indicated that the militia ‖does not
establishment.‖ Newspapers have added that the militia was considering a peaceful protest,
3. Recent COMINT has indicated an increase in anti-government rhetoric from the militia based
health conditions.

END REPORT
85
Appendix G
Scenario 4: Insurgent Activity

SUBJECT: INSURGENT ACTIVITY
1. Summary. Northeast Africa and the Horn of Africa (HOA) regions have multiple insurgents.
Insurgents in the region have varying levels of violence and success. Some insurgents remain
benign with fluctuations in activity, while other insurgents consistently maintain high levels of
violence. Over the past month, multiple sources have indicated an increase in activity for an
unidentified insurgent group. The American Embassy in Djibouti has been unable to assess the
likelihood of an attack on Djibouti interests in the region (due to the wide variety of insurgent
groups) and is reaching out to the intelligence community for outside perspective and analysis.
2. Over the past month there has been an overall increase in insurgent activity. Imagery
intelligence has indicated an increase of human activity near what appears to be a training base or
command post of the suspected insurgents. Imagery analysts report that the insurgents appear to
have small arms and other weapons however, due to poor satellite coverage and cloud cover,
analytic confidence in the imagery has decreased. Several local newspapers have indicated that
the insurgents ―do not plan to incite, or participate in, any violence in the future despite
discontentment with the current establishment.‖ Newspapers have added that the insurgents were
considering a peaceful protest, however no specific dates have been set.
3. Recent COMINT has indicated an increase in anti-government rhetoric from the insurgents
based upon the alleged social injustice and oppression. The Northeast and HOA regions have a
long history of social strife and discontentment. These social tensions stem from the region being
a hot-bed of differing political and religious ideologies, as well as widespread poverty and poor
health conditions.

END REPORT
86
Appendix H
Scenario 5: Terrorist Activity

SUBJECT: TERRORIST ACTIVITY
1. Summary. Northeast Africa and the Horn of Africa (HOA) regions have multiple terrorist
groups. Terrorists in the region have varying levels of violence and success. Some terrorists
groups remain benign with fluctuations in activity, while other terrorist groups consistently
maintain high levels of violence. Over the past month, multiple sources have indicated an
increase in activity for an unidentified terrorist group. The American Embassy in Djibouti has
been unable to assess the likelihood of an attack on Djibouti interests in the region (due to the
wide variety of terrorist groups) and is reaching out to the intelligence community for outside
perspective and analysis.
2. Over the past month there has been an overall increase in terrorist activity. Imagery
intelligence has indicated an increase of human activity near what appears to be a training base or
command post of the suspected terrorists. Imagery analysts report that the terrorists appear to
have small arms and other weapons however, due to poor satellite coverage and cloud cover,
analytic confidence in the imagery has decreased. Several local newspapers have indicated that
the terrorists ―do not plan to incite, or participate in, any violence in the future despite
discontentment with the current establishment.‖ Newspapers have added that the terrorists were
considering a peaceful protest, however no specific dates have been set.
3. Recent COMINT has indicated an increase in anti-government rhetoric from the terrorists
based upon the alleged social injustice and oppression. The Northeast and HOA regions have a
long history of social strife and discontentment. These social tensions stem from the region being
a hot-bed of differing political and religious ideologies, as well as widespread poverty and poor
health conditions.

END REPORT
87
Appendix I
The Effect of Labels on Analysis Survey
Welgan Thesis: [Label was inserted here] Activity
Thank you for taking time to participate in my survey. Your participation is important to my thesis and to the Intelligence
Community.
[Intelligence Report was inserted here]
Basing your responses from the intelligence report you were given, please respond to the evidence by stating the likelihood of
an attack.
Thank you once again,
Jeff Welgan
1) Are you now, or have you ever been, an intelligence analyst? Students without intelligence experience (i.e. a minimum of
internship experience) should answer this question "NO."
a. Yes
b. No
2) What area of Intelligence are you most associated with?
a. National Security
b. Military
c. Law Enforcement
d. Competitive Intelligence
e. Student of Intelligence
f. Other
3) How many years of intelligence-related experience do you have?
a. 0
b. 0-1
c. 1-3
d. 3-5
e. 5-10
f. 10-15
g. 15-20
h. 20-25
i. greater than 25
4) Overall, based on the information you have available to you, what is the likelihood of an attack on Djibouti interests?
a. Virtually certain an attack will occur

b. Highly likely
c. Likely
d. Somewhat likely
e. Somewhat greater than even
88
f. Somewhat less than even
g. Somewhat unlikely
h. Unlikely
i. Highly unlikely
j. Virtually certain an attack will not occur
5) Given the information you have received, what is your approximate level of analytic confidence for your estimate in the
above question?
a. 10 - Extremely High
b. 9
c. 8
d. 7
e. 6
f. 5
g. 4
h. 3
i. 2
j. 1 - Extremely Low
6) What is the likelihood that an attack would be successful?
a. Virtually certain an attack will be successful

b. Highly likely
c. Likely
d. Somewhat likely
h. Unlikely
i. Highly unlikely
j. Virtually certain an attack will not succeed
7) What is your analytic confidence for the above question?
b. 9
c. 8
d. 7
e. 6
f. 5
g. 4
h. 3
i. 2
8) What is the likelihood than an attack would be sophisticated?
a. Virtually certain an attack will be sophisticated

b. Highly likely
c. Likely
d. Somewhat likely
h. Unlikely
i. Highly unlikely
j. Virtually certain an attack will not be sophisticated
89
b. 9
c. 8
d. 7
e. 6
f. 5
g. 4
h. 3
i. 2
10) If an attack were to occur, how severe do you think it would be?
a. Very severe
b. Severe
c. Somewhat severe
d. Not severe
b. 9
c. 8
d. 7
e. 6
f. 5
g. 4
h. 3
i. 2
12) An overall increase in group activity...
a. Highly increases the likelihood of an attack

b. Significantly increases the likelihood of an attack
c. Increases the likelihood of an attack
d. Somewhat increases the likelihood of an attack
e. Does not affect the likelihood of an attack
f. Somewhat decreases the likelihood of an attack
g. Decreases the likelihood of an attack
h. Significantly decreases the likelihood of an attack
i. Highly decreases the likelihood of an attack
13) Imagery indicating an increase in human activity near the training base or command post of the group....

14) The local newspapers' indication that the group "does not plan to incite, or participate in, any violence in the future
despite discontentment with the current establishment...

90
15) Recent COMINT indicating an increase in anti-government rhetoric from the group based upon social injustice and
oppression...

16) The long history of social strife and discontentment in Northeast Africa and the HOA region...

17) This region being a hot-bed of differing political and religious ideologies, as well a region of widespread poverty and poor
health conditions...

18) Rank the following groups by their likelihood to engage in violence- 1 being most likely, 4 being least likely.
a. Militia
b. Terrorist
c. Rebel
d. Insurgent
19) Rank the following by their likelihood of executing severe attacks- 1 being most severe, 4 being least severe.
a. Militia
b. Terrorist
c. Rebel
d. Insurgent
91
Statistical Annex
Is there a difference in responses for Q. 4 among different groups?
Null: There is no difference in responses for Q. 4 among different groups.
Alternative: There is a significant difference in responses for Q. 4 among different groups.
There are more than 2 groups to compare so need to use 1-way ANOVA provided assumptions
are satisfied.
Samples are independent as groups are different from each other and were given separate
treatment.
From above table, both Kolmogorov-Smirnov and Shapiro-Wilk show that P-value (given by Sig.
column) for each group is less than 0.05. Thus normality assumption is not satisfied and need to
look at Normal quantile plots to check normality. Note normality is robust!
Plot shows that most of the points are close

to the diagonal line, there is no significant
curvature and no concentration of points at
any particular location. Thus normality is
satisfied for group Control.
92
satisfied for group Insurgent.

satisfied for group Militia.

satisfied for group Rebel.
93
satisfied for group Terrorist.
Box plot shows no outliers for each group.

Thus another requirement of normality is
satisfied.
Above table gives Levene‘s test P-value = 0.441 (given by column Sig.). It is larger than α =
0.05 thus equal variance assumption is satisfied.
All the assumptions for 1-way ANOVA satisfied.
94
From table ANOVA, F-test value = 0.535 and F-test P-value = 0.710.
(F-test P-value = 0.710) > (α = 0.05), thus fail to reject the null hypothesis.
Conclusion: There is no significant difference in responses for Q. 4 among different groups

at 5% level.
are satisfied.
treatment.
95


96


97
Box plot shows several outliers for all the
groups except group Insurgent and Rebel.
The decision is to keep the outliers as
normality is satisfied even with the outliers.
98

at 5% level.
are satisfied.
treatment.

99


100
Box plot shows no outliers for each group.

Thus another requirement of normality is
satisfied.
101

at 5% level.
are satisfied.
treatment.
102


103


104
groups except group Control, Insurgent and
Rebel. The decision is to keep the outliers
as normality is satisfied even with the
outliers.
105

at 5% level.
are satisfied.
treatment.

106


107

groups except group Control, Insurgent and
outliers.
108

at 5% level.
are satisfied.
treatment.
109
column) for each group is less than 0.05 except Terrorist group. Thus normality assumption is
not satisfied and need to look at Normal quantile plots to check normality for all the groups
except Terrorist group. Note normality is robust!


110


groups except group Control and Rebel. The
decision is to keep the outliers as normality
is satisfied even with the outliers.
111

at 5% level.
are satisfied.
treatment.
112


113


114
Box plot shows an outlier for the group
Control. Presence of single outlier is
acceptable for normality assumption.
115

at 5% level.
are satisfied.
treatment.
From above table, Kolmogorov-Smirnov show that P-value (given by Sig. column) for each
group is less than 0.05 except for group Control and Shapiro-Wilk show that P-value (given by
Sig. column) for each group is less than 0.05 except for group Terrorist. Thus normality
assumption is not satisfied for all the groups except group Control and Terrorist and need to look
at Normal quantile plots to check normality. Note normality is robust!
116


117
Box plot shows two outliers for the group
Militia. The decision is to keep the outlier as
normality is satisfied even with the outlier.
118

at 5% level.
are satisfied.
treatment.

119


120
Box plot shows an outlier for the group

Insurgent, Militia and Rebel. The decision is
to keep the outlier as normality is satisfied
even with the outlier.
121
(F-test P-value = 0.017) < (α = 0.05), thus reject the null hypothesis.
Conclusion: There is a significant difference in responses for Q. 12 among different groups

at 5% level.
Now we need to see where the difference lies. As the each group sample sizes are not same, need
to use Scheffe Post Hoc test.
122
From above table, responses for Control and Rebel groups are significantly different from
each other for Q. 12 at 5% level (SPSS flags it with an asterisk). Also for this group
comparison (Scheffe P-value = 0.036) < (α = 0.05), this means difference is significant.
are satisfied.
treatment.
123


124


125
Box plot shows two outliers for the group
outliers.
Above table gives Levene‘s test P-value = 0.023 (given by column Sig.). It is less than α = 0.05
thus equal variance assumption is not satisfied.
All the assumptions for 1-way ANOVA are not satisfied. We need to use Welch test instead of 1-
way ANOVA.
126
From the above table Welch-test value = 1.598 and Welch-test P-value = 0.180.
(Welch-test P-value = 0.180) > (α = 0.05), thus fail reject the null hypothesis.

at 5% level.
are satisfied.
treatment.

127


128

groups. The decision is to keep the outliers
outliers.
129
(F-test P-value = 0.886) > (α = 0.05), thus fail reject the null hypothesis.

at 5% level.
are satisfied.
treatment.
130


131


132
Box plot shows one or two outliers for all
the groups except control. The decision is to
keep the outliers as normality is satisfied
even with the outliers.
Above table gives Levene‘s test P-value = 0.013 (given by column Sig.). It is less than α = 0.05
thus equal variance assumption is not satisfied.
All the assumptions for 1-way ANOVA are not satisfied. We need to use Welch test instead of 1-
way ANOVA.
133
From the above table Welch-test value = 3.412 and Welch-test P-value = 0.011.
(Welch-test P-value = 0.011) < (α = 0.05), thus reject the null hypothesis.
Conclusion: There is a significant difference in responses for Q. 15 among different groups

at 5% level.
Now we need to see where the difference lies. Need to use Tamhane Post Hoc test as variances
are not equal.
From above table, Control and Militia groups are significantly different from each other for
Q. 15 at 5% level (SPSS flags it with an asterisk). Also for this group comparison (Tamhane P-
value = 0.042) < (α = 0.05), this means difference is significant.
134
are satisfied.
treatment.


135


136
Box plot shows few outliers for the groups,
Insurgent and Militia. The decision is to
keep the outliers as normality is satisfied
even with the outliers.
137

at 5% level.
are satisfied.
treatment.

138


139
Box plot shows one outlier for the groups,

Insurgent and Control. This is acceptable
norm with the normality assumption.
140

at 5% level.
141
Is there a difference in responses for all the questions together among different groups?
Null: There is no difference in responses for all the questions together among different groups.
Alternative: There is a significant difference in responses for all the questions together among
different groups.
are satisfied.
treatment.
Plot shows that points are not in a linear

format and close to the diagonal line; there
is significant curvature and break in the
points. Thus normality is not satisfied for
group Control.
142
is significant curvature and break in the
points. Thus normality is not satisfied for
group Insurgent.

is some curvature and break in the points.
Thus normality is not satisfied for group
Militia.

Rebel.
143
Terrorist.
Box plot shows no outliers for all the groups.
For each group, normality is not satisfied. Thus we cannot use 1-way ANOVA. We need to use
nonparametric (for non-normal data) method, Kruskal-Wallis test.
144
From table Test Statistics, Chi-Square test value = 1.364 and Chi-Square - test P-value = 0.850.
(Chi-Square - test P-value = 0.850) > (α = 0.05), thus fail reject the null hypothesis.
Conclusion: There is no significant difference in responses for all the questions together
among different groups at 5% level.
145

The Effect of Labels On Analysis

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

The Effect of Labels On Analysis

Uploaded by

Copyright:

Available Formats

THE EFFECT OF LABELS ON ANALYSIS

Submitted to the Faculty of Mercyhurst College

In Partial Fulfillment of the Requirements for

DEPARTMENT OF INTELLIGENCE STUDIES

THE EFFECT OF LABELS ON INTELLIGENCE ANALYSIS

This thesis is dedicated to the intelligence community,

whose hard work often goes unnoticed.

The Effect of Labels on Analysis

Master of Science in Applied Intelligence

Mercyhurst College, 2009

Professor Kristan Wheaton, Chair

This thesis examines the notion of cognitive bias as it relates to intelligence

analysis. Specifically, it reports the findings of an online experiment in which analysts in

groups— either ‗group,‘ ‗insurgent,‘ ‗rebel,‘ ‗militia,‘ or ‗terrorist‘—and then the

findings resulted in no statistical significance, though statistical significance was

determined. If cognitive bias plays a role in intelligence analysis, it could have

COPYRIGHT PAGE iii

TABLE OF CONTENTS vii

Table 3.1: Five-group posttest only experimental design 36

Figure 3.1: Number of participants by label received 39

intelligence depends directly upon the effectiveness of the intelligence analyst. If an

inconsistencies and therefore have increased inaccuracy.

in the US Intelligence community and their affect on the analysis of a report.

militia, and insurgent. To determine whether or not intelligence analysts succumb to

In addition to conducting an experiment that identifies whether analysts apply

well as their underlying implications.

The examination of the terms we use to label is significant, particularly in light of

and intelligence backgrounds (National Security, Military, Law Enforcement, and

misuse, of particular labels can directly affect US government action or inaction.

comparing the labels.

Understanding the Cognitive Processes

People construct their own version of „reality‟ on the basis of information

meaning of the message. Oftentimes messages can be successfully interpreted, meaning

However, messages can also be misunderstood. The distortion of messages frequently is

the result of individual perceptual screens. A perceptual screen, as defined by Debra

individuality is molded by a multitude of factors such as religious beliefs, social

easily be distorted from one person to another.

Framing and Anchoring

organize information cognitively. When we encounter new information, we make

of information is oftentimes referred to as framing.

individual behavior.‖6 In Barlett‘s (1932) words, ―a schema is a mental representation of

occurring and culture-specific sequences of dramatic sub-events.‖7 Schemas are ready-

understand it in terms of their personal and cultural history.

Richards Heuer describes everything we think we know about something in

particular as a ―mind-set‖ or ―mental model.‖8 Heuer states:

A mind-set is neither good nor bad. It is unavoidable. It is, in essence, a

efficiently each time we think about a particular subject.

group,‖ or ―insurgent,‖ or ―militia.‖

―sensemaking.‖11 Klein et al describe the sensemaking process as follows:

from cognitively-framed internal images of external realities.13 Therefore, when new

In addition to the personal frames we construct, frames can be presented to us

through context. If information is presented within a particular context, the person

reinterpreting or reframing the data.14 This phenomenon is known as the ―concreteness

indicated an increase of human activity near what appears to be a training base or

context to provide a set frame for them.

eliminate, the more precise the analysis is likely to be.

Heuristics and Cognitive Biases

refutes it.21,22 Johnston‘s findings confirm a 1998 study of intelligence analysts by

Heuristics also contribute to our biases and decision-making. In its simplest

in making decisions. According to Gerd Gigerenzer, a heuristic must contain three

the norm for the intelligence analyst.

In an effort to be more efficient, some heuristics are routine to the intelligence

professional‘s analytical process. One such heuristic is the recognition heuristic. In