Teaching Strategic Intelligence

Through Games

Kristan J. Wheaton
Associate Professor
Department Of Intelligence Studies
Mercyhurst College
501 E. 38th St
Erie, PA 16546
814 824 2023
kwheaton@mercyhurst.edu
 Teaching Strategic Intelligence
Through Games

Strategic intelligence is considered by intelligence professionals to be the highest form of

the analytic art.   There is a tremendous demand for this type of intelligence product and a
lack of trained professionals capable of producing it.  Developing effective teaching
methods for this challenging subject, therefore, is an area of ongoing concern for the
business, law enforcement and national security intelligence communities.

Previous research suggests that a game-based approach to teaching can be successful but
no report so far has examined game-based learning in intelligence analysis.  I
hypothesized that a game-based approach to teaching strategic intelligence analysis
would increase learning and improve performance while also increasing student
satisfaction with the course.

This paper reports the initial results and lessons learned from teaching three full courses
(2 undergraduate and one graduate) in strategic intelligence using games as a teaching
tool.  The paper will begin by examining the unique challenges in teaching strategy,
strategic decisionmaking and the types of intelligence that supports those efforts. This
will be followed by a short discussion concerning games-based learning generally before
examining in detail the specific approaches used in these three courses.

This paper will also examine both the learning outcomes and student satisfaction with the
courses.  Finally, this paper will discuss appropriate course modifications for
undergraduate and graduate students when teaching advanced subjects with games-based
on the evidence from this study.

What is a strategy and what are strategic decisions?

"As war is a game through its objective nature, so also is it

through its subjective. -- Carl von Clausewitz, On War,
Chapter 1.1

1
A note on the references in this paper: Wherever possible, I have hyperlinked directly to the sources of
my information. My intent is to make it easy for readers to rapidly access and evaluate the sources I have
used in the preparation of this paper. Where an adequate online source is not available, I have used
standard footnotes.

KJW--Teaching Strategic Page 2

While there are many definitions of strategy and strategic decisions, for the purposes of
this paper, a strategy is an idea or set of ideas about how to accomplish a goal and
strategic decisions are ones which typically put at risk a substantial portion of an entity's
disposable resources.

Defining strategy broadly is important. Far too often, strategy is also associated with
terms such as "long-term" or "large" and strategic thinking is something accomplished
only at corporate headquarters or by generals and kings.

Defining strategic decisions in the context of the resources risked by the entity (person or
organization) making the decision puts the role of strategy into perspective. Under this
definition, it is possible for the exact same decision to be strategic in one case and tactical
(or even trivial) in another context. For example, imagine an individual who owns a
successful dry cleaning store. Deciding to open up another branch of the store in a
different part of town is clearly a strategic decision for this owner. This owner will likely
spend many of his disposable resources (time, money, personnel) getting the new branch
set up and operating efficiently.

The same decision, to open another branch in the same town by the owner of a chain of
7000 dry cleaning stores across the US does not have the same strategic quality as in the
first case. In fact, such a decision, in such a large, national organization, might not even
be made at the owner’s level. It is entirely possible that such a decision would be pushed
down to regional or even sub-regional levels.

More importantly, defining strategy in terms of the resources at risk broadens the scope
of what arguably constitutes strategic intelligence as well.   Under this definition, strategy
is not confined to large, powerful organizations.  Small businesses, police units and even
students can have strategies and, in turn, require strategic intelligence to support their
decision-making processes.

What is intelligence and what is the role of intelligence in the formulation of

strategy?

"Well is the Game called great! I was four days a scullion at

Quetta, waiting on the wife of the man whose book I stole. And
that was part of the Great Game! From the South—God knows
how far—came up the Mahratta, playing the Great Game in
fear of his life. Now I shall go far and far into the North
playing the Great Game. Truly, it runs like a shuttle throughout
all Hind. -- Rudyard Kipling, Kim, Chapter 12

For many people, “intelligence” is an even more misunderstood word than “strategy”.
Conjuring up images of James Bond or, at least George Smiley, intelligence, for many, is
exclusively about secrets and spying. This has been patently untrue for some time,
however. As early as 1949, Sherman Kent, the father of intelligence analysis at the
Central Intelligence Agency, claimed that as much as 80% of information needed in

KJW--Teaching Strategic Page 3

intelligence work came from open, non-secret, sources.2 In a world that moves 21
exabytes of information via the internet each month, the role of both secrets and spying in
intelligence, while still important, is clearly further diminished.

In recent years, in fact, intelligence has moved from a narrow government function to a
broad reaching discipline. Intelligence-led policing is a highly regarded public safety
strategy while competitive intelligence has, for many years, been a driver for some
successful businesses. Likewise, commercial intelligence agencies, such as IJet and
Stratfor, provide intelligence analysis services to private clients. Even non-governmental
organizations, such as the Southern Poverty Law Center, have established intelligence
organizations to cover issues of interest to them. In addition, intelligence studies
programs, like the one at Mercyhurst have sprung up all over the US and abroad. The
International Association For Intelligence Education now boasts some 20 colleges and
universities among it members.

There are several common themes running through these activities that help define
intelligence3: First, intelligence is about those organizations and activities that are
outside your control but are relevant to your entity's success or failure. In short,
intelligence is externally focused. Ever since Moses sent scouts "through the Negev and
on into the hill country" of Canaan to see "what the land is like and whether the people
who live there are strong or weak, few or many” intelligence has been about "the other
guy"; the enemy, the criminal or the competitor.

Second, intelligence uses information from all sources and, more than that, most of this
information is unstructured. Very few disciplines truly deal with information from all
sources. Intelligence analysts, however routinely have to integrate economic, political,
military, cultural and other types of data into a forecast for decisionmakers within an
organization. In addition, much of the information used by intelligence analysts is
incomplete or unverified and may even be the result of a deliberate attempt to deceive.
This messy, dirty, unstructured data requires a unique set of analytic methodologies.
Traditional statistical methods often don't work, for example, with the kinds of anecdotal
data normal in intelligence work.

Third, intelligence is designed to reduce the level of uncertainty for a decisionmaker. It

is part of the decisionmaking process. If intelligence is about the question, "What is
going on out there and how is it likely to change?" then the other half of the
decisionmaking question (the so-called operational half) is "What can we/should we/will
we do about it?"

Finally, intelligence is a process. It is something that happens, that is both iterative and
reflexive; not something that just is. While emphasis is typically placed on the output of
this process, the intelligence product, the quality and utility of this product (as with any
2
Kent, Sherman. Strategic Intelligence for American World Policy. Princeton, NJ: Princeton University
Press, 1949.
3
For a more exhaustive discussion of these themes and how they lead to this definition of intelligence, see
Wheaton and Beerbower, Towards A New Definition Of Intelligence, Stanford Law and Policy Review,
Vol. 17, Issue 2 (2006), p. 319-330.

KJW--Teaching Strategic Page 4

product) is a direct function of the process used to create it. More importantly, since
intelligence is a process, it can be improved upon through careful and intelligent change,
improving, in turn, the quality and utility of the final intelligence product.

Intelligence products come in many flavors but one of the most useful distinctions is
between descriptive and estimative products. Descriptive products outline the relevant
facts, figure, personalities and issues surrounding a topic of interest. Estimative products,
on the other hand, attempt to forecast what will likely happen because of the intersection
of those facts, figures, personalities, etc. Of the two, decisionmakers typically value
quality estimative intelligence over even the very best descriptive products. In a world
dominated by a 24/7 news cycle and supported by the vast resources of the internet,
providing mere facts and figures is rarely enough to justify the expense of a dedicated
intelligence unit.

In short, strategic intelligence should be the foundation for all strategic planning.
Without some sense of how the external world will likely change to support or hinder
attempts to achieve a person’s or organization’s strategic goals, any allocation of
resources in support of those goals will likely be sub-optimal.

Why games?

"...Life is a kind of Chess, in which we have often points to

gain, and competitors or adversaries to contend with, and in
which there is a vast variety of good and ill events... By
playing at Chess then, we may learn: 1st, Foresight... 2nd,
Circumspection (and) 3rd, Caution..." -- Benjamin Franklin,
The Morals Of Chess

Mercyhurst College has the oldest and largest full time, residential Intelligence Studies
program in the world. With 350 student on campus in Erie, PA and 10 full time faculty
representing all three major sub-disciplines of intelligence (business, law enforcement
and national security), Mercyhurst produces qualified entry-level intelligence analysts
with both undergraduate and graduate degrees.

Critical to the success of this program (now in its 18th year) is its focus on application.
Like an engineering or architecture (or, for that matter, games design) program, the goal
of the curriculum is to produce graduates who understand both theory and practice and
are ready, at an entry-level, to apply this knowledge to real world problems.

Strategic Intelligence, as taught at Mercyhurst, is the capstone course of this program at

both the graduate and undergraduate levels.4 The primary purpose of the course is to
4
At the undergraduate level, the course is referred to simply as Strategic Intelligence and at the graduate
level the course is called Managing Strategic Intelligence Analysis.  Both classes are project based but the
topics chosen for the graduate class are conceptually more difficult.  In addition, there is more emphasis in
the graduate class on managing small groups and on other managerial level tasks such as budgeting and
personnel selection.  Given that graduate students come to our Masters in Applied Intelligence program
from all disciplines, many of the core concepts are the same in both classes with the primary difference

KJW--Teaching Strategic Page 5

integrate and apply the knowledge and skills gained in earlier courses while adding the
specific additional knowledge and skills necessary to prepare a complete strategic
intelligence product -- to up their game, as it were, as analysts.

The centerpiece of this course is a strategic intelligence project for a real-world

decisionmaker. Previous clients have included a number of US national security agencies
and organizations such as the Defense Intelligence Agency, the National Security
Agency, the National Intelligence Council; law enforcement organizations such as the
Federal Bureau of Investigation and the Drug Enforcement Agency; Fortune 500
businesses such as Target Corporation and local firms, such as Dispatch Printing; and
international organizations including the European Parliament, the Iraqi government,
Dutch counter-terror organizations and a Spanish bank.

The intelligence requirements (a fancy name for the questions) posed by these
organizations are as diverse as the organizations themselves. A sampling of the kinds of
questions typically asked of the student-analysts in the class includes:

- What are the most important and most likely impacts on, and threats
to, US national interests (including but not limited to political,
military, economic and social interests) resulting from infectious and
chronic human disease originating outside the US over the next 10-15
years?

- What are the likely current best practices or combination of best

practices utilized by suburban public high schools with respect to
curriculum, buildings and green initiatives?

- What are the likely causes for objection and consent to the ratification
of the Fissile Material Cutoff Treaty (FMCT) by "states of interest"
(i.e. states of concern and de facto states) over the next 5 years?

- What are the current and future direct threats to (a Fortune 500
company’s) existing global supply chain over the next 12-24 months?

- What is the current severity and effectiveness of the insurgency in the

North and South Caucasus regions (in regards to the quantitative and
geographic growth and spread of violence) and how is it likely to
change between now and the 2008 Russian Presidential election?

- Who are the likely innovative users and what are the likely innovative
uses of wiki technology over the next two years?

Students rarely have any previously acquired expertise in these subjects at the beginning
of the course. Instead, they have acquired during their time at Mercyhurst a set of skills
and techniques that allow them to rapidly orient themselves within the domain of the

arising in the expectations regarding performance.

KJW--Teaching Strategic Page 6

question and to begin to generate meaningful analytic estimates in response to these
questions fairly quickly. This is a good thing as Mercyhurst operates on a 10-week term
system and the finished intelligence product is due before the end of the term.5

Students on a team working on a strategic project are expected to efficiently organize

themselves to accomplish all of the analytic and administrative tasks associated with their
project. The professor in this class provides mentorship, methodological guidance,
moral support as well as tangible resources to help the team complete the project. The
professor does not, however, provide “answers”. In short, while the professor may help
the students draw the map, the students pick their own path. The successful completion
of a large-scale project such as this is designed to give these students confidence in their
skills and abilities as they approach graduation – a second major purpose of the class.

In addition to the project, which takes up approximately 80% of the total time of the
course (including time spent both in and out of class), the remaining coursework focuses
on three overlapping themes:

- Strategic theory
- The current practice of strategic intelligence
- Review of previously learned concepts that are particularly appropriate in a
strategic environment.

For the first six years of the course, I taught this remaining 20% using a standard mix of
lecture, discussion and classroom exercise.  Despite various tweaks, it became obvious to
me that the overwhelming emphasis of the class on the current project – an emphasis that
I both encouraged and approved of – severely weakened the impact of the remaining 20%
of the course.  Unfortunately, it was in this last 20% where the course materials designed
to accomplish the third major goal of the course -- preparing these students for the kinds
of strategic intelligence challenges they are likely to face throughout their careers, as well
as information important to the success of the previous two goals -- largely lay.

Inspired by the speakers at the Game Education Summit at Carnegie Mellon University in
June, 2009 (and, in particular, by Prof. Ian Schrieber’s lecture on Innovative Teaching
Through Game Design), I decided to integrate games into the syllabus such that they
were the fundamental pedagogical approach for this remaining 20%.

I knew from my own experience that games could be an effective way to teach strategy
and strategic intelligence. I realized that much of my own understanding of these
concepts had originated with a variety of wargames and other type games I had either
played or designed over the years.6 Likewise, these games encouraged me to delve
5
For an example of such a product, see www.nie.wikispaces.com . This product responded to the first of
the six questions listed above and was requested by the National Intelligence Council. You can see their
review of the product at http://www.dni.gov/nic/research_globaldisease.html
6
I began playing wargames when I was 12, wrote occasionally for the Dragon Magazine and designed
simple computer games and simulations while in college and unsuccessfully ran a small games design
company in the 1980’s. I developed an expert system designed to simulate the activities of an attorney in
giving advice regarding landlord-tenant disputes while in law school. I also designed several simulations

KJW--Teaching Strategic Page 7

deeper into the literature regarding strategy and strategic intelligence. It was precisely
this type of virtuous circle that I hoped to set up in my own class.

I also knew that there is an increasing body of literature about the effectiveness of games-
based learning strategies in the classroom. Studies have been conducted, the results
published and briefed and respected individuals outside the gaming industry have
endorsed games-based learning. Games-based learning forms a critical component to the
US Department of Education’s national education technology plan. Thus, despite
detractors, the hypothesis that a games-based approach to strategic intelligence would be
effective does not seem entirely out of line.

Most comments regarding the efficacy of games-based learning initiatives center on the
fact that they are fun in one of the 14 different ways that researchers define that term.
The fun translates to increased attention to the subject and increased attention, in turn,
facilitates learning.

My goals were more ambitious. I wanted to try to address all three of the major
objectives of this course. First, I wanted them to improve as analysts, specifically by
improving their ability to see patterns and connections buried deeply in unstructured data
sets that were confusing, incomplete, of unknown reliability and possibly deceptive.
These conditions are far more common in intelligence analysis than not. I wanted my
students to expand their ways of thinking about intelligence problems -- to develop a
flexibility of mind that would help them no matter what kind of problem the world threw
at them. I wanted them to become better at discovering solutions on their own rather than
merely getting good at recognizing the “right” solution when it was handed to them. I
expected to see this most clearly, albeit qualitatively, in classroom discussions and, in the
one class where implemented, the students' weekly writing assignments.

Second, I also wanted the students to gain confidence in their skills as analysts. I
believed that a learning environment dominated by games would foster a more creative,
exploratory atmosphere and that this would translate into better final products.

Third, I wanted the students to remember not just the experience but also some more
general lessons learned that would apply the next time they encountered a strategic
intelligence project. This would be the most difficult goal to measure but I believed that I
would be able to get at least anecdotal feedback from former students several months
after the classes had ended.

Finally, I wanted to measure student satisfaction with the course. For this, I would use
the results from the Student Instructional Report-II (SIR-II). There is a good bit of
discussion about the value and accuracy of the SIR and other student evaluations of
teaching effectiveness among college and university faculty.7 It is particularly difficult, I
while in the army (including the first dedicated tactical air defense simulation) and developed a simulation
to understand transitions from authoritarian rule as part of my masters work in Russian and East European
Area Studies while at Florida State University.
7
Conversations with several psychologists who study games and game-based learning during the 2010
Game Education Summit (where this paper was presented) highlighted another important issue in using the

KJW--Teaching Strategic Page 8

think, to use these scores in a project-based course, where the survey is administered in
week eight but the major learning event of the course takes place in week ten, the last
week, when the students present the results of their analysis to their decisionmaker (It is
sort of like asking a high school biology student what they think of biology before they
get to dissect the frog). That said, I had SIRs data from most of the previous classes and,
whatever effect the timing of the SIR had on each class's results, the results could still be
compared effectively from class to class. My expectation was that this unique, non-
traditional approach, if integrated correctly with the core material of the course coupled
with the fun of being able to play games would increase student satisfaction with the
course.

How, specifically, were games used in class?

“Indeed, experiments have shown that the more mental work readers have
to do to infer a cause from a set of facts, the more memorable the causal
inference will be.” – The Trouble With Intuition, Daniel J. Simons and
Christopher F. Chabris

While I wanted games to be a central tool within the context of the class, I established
some fairly rigorous pre-conditions for myself before deciding which games and how
exactly they would be used in class. First, I did not want the games to detract from the
project. The experience gained from working on a real world project for a real world
decisionmaker trumped, in my mind, any value games might bring to learning the
essential lessons of strategic intelligence. In addition, these projects have significant
tangible value to the students beyond the knowledge they gain from them. In many cases
over the years, these projects have led to jobs (or at least job offers) either directly or
indirectly.

I was worried that the games might detract in two ways. First, the games were to add
value, if at all, to only that 20% of the class that was indirectly involved with the project.
I did not want to create an environment where a student felt they had to choose between
playing a game and getting work done on the project. Second, I did not want the course
to become about the game or games. I know from experience that games can be
genuinely compelling. There are many good strategic simulations (Diplomacy, The Total
War series) around which a strategic course could easily be built. In these cases, I saw
the games not just competing for time with the project but actually overwhelming the
project completely.

Beyond the potential for detraction from the project, the games I did select needed to
resonate in a meaningful way with the core concepts of the course. I either needed a
single game through which I could articulate all of the core concepts of the course more
or less in the order they needed to be presented (unlikely) or find a number of different
games which could help me accomplish the same goals. Of course, if I were to use a

SIRs:  These tests were not designed to evaluate alternative forms of pedagogy.  In fact, the only options on
the SIR are "Lecture", "Discussion" or "Combination".  It may well be that this tool is wholly inappropriate
for evaluating game-based courses.

KJW--Teaching Strategic Page 9

number of games, the time to learn the new rules of each game would become a factor as
well.

If I were to use a number of games, then I also believed I needed to include a variety of
game types and genres. I know from experience that not all game types appeal to all
game players. In fact, I believe that one of the major hurdles to overcome with regard to
game-based learning will be the re-packaging of core concepts in any given subject into a
variety of game genres such that at least one approach will work for every student.

Finally, I was very conscious of the cost. Textbooks are expensive and I believe that, in
many cases, they are unreasonably so. I could not see adding to that burden.

In the end, I settled on using casual online or downloadable games or games, such as
World Of Warcraft, that came with free trials (for a complete list of the games and the
core concepts, see the course syllabus attached at Annex A). The exception to this
general rule was the addition of one "old-school", paper-pencil wargame, Defiant Russia.

Students were required to play the game before each class.  In addition, students were
required to come to some defensible conclusion about how the game related to the topic
of that particular class and to be prepared to discuss it when they came to class. I
indicated to the students that the relationships between the games and the topics were
rarely obvious and that in some cases there were many possible defensible conclusions. 
In many cases, I informed them, there might appear, on the surface, to be no real
connection between the game and the topic.  I wanted them to have to think hard about
the possible connections, to evaluate them and to come to a conclusion that they were
prepared to defend.

Classroom time was devoted in part to examining what the students saw compared with
what I saw as the essential connections.  A careful matching of games and topics yielded
a fairly high overlap between what the students saw and what I hoped they would see.  In
addition, I had the genuine pleasure of having students come up with unique and deep
interpretations far beyond my expectations (I will discuss this in greater detail in the
subsequent posts).

For example, in the class where we discussed strategic intelligence requirements (i.e. the
questions that intelligence professionals are asked at the strategic level), students had to
play World of Warcraft (WOW) or some other quest-based game. While there are many
defensible answers to the question regarding the connection between WOW and
intelligence requirements, I was able to leverage the student's experience with a well-
formed quest (typical of the high-end MMORPGs) and contrast it with the consequences
of poorly formed intelligence requirements. This, in turn, gave the students another way
of looking at their upcoming meeting with their decisionmaker where they would be
receiving the intelligence requirement relevant to their particular project.

As a result of my experience with the first two classes, I required the third class in which
I used this approach to write down their conclusions and post them to Mercyhurst’s

KJW--Teaching Strategic Page 10

Blackboard course management software prior to class. This writing assignment was
modest (100-150 words) but it allowed me to better prepare for the class itself.

While students were required to play the game and to come to a conclusion, I also made a
wide variety of supplemental readings available.  These provided "hints' to the
connections between the game and the topic that I saw.  I believed that, in some cases
(particularly where the material was more in the way of a review), students would be able
to come to reasonable conclusions without any additional reading.  I also believed that
students that were not overly familiar with the topic under consideration would be more
engaged in the reading if they were working to answer a question, even one as difficult as
the one I posed.

So, how did it all work out?

While mostly anecdotal, the available evidence suggests that students significantly
increased their ability to see patterns and connections buried deeply in unstructured data
sets, my first goal. This was particularly obvious in the graduate class where I required
students to jot down their conclusions prior to class.

An example of the growth I witnessed from week one to week ten would likely be helpful
at this point. The same student wrote both examples below and I consider this example
to be representative of the whole:

Week 2 response – “This game (The Space Game: Missions) was

predominantly about budgets and space allocation….  Strategy and
forethought goes into where you place lasers, missile launchers and solar
stations, so that you don’t run out of minerals to power those machines
and so repair stations are available for those that are rundown.  It’s clear
that resource and space allocation are key elements for a player to win this
game, just as it is for the Intelligence Community and analysts to win
a war.”

Week 8 response: “I think if Chess dropped acid it’d become the

Thinking Game. When the computer player was contemplating its next
move colorful lines and curves covered the board… To me, Chess was
always a one-on-one game; a conflict, if you will, between black versus
white… Samuel Huntington states up front that he believes that conflict
will no longer be about Princes and Emperors expanding empires or
influencing ideologies, but rather about conflicts among different cultures:
“The great divisions among humankind and the dominating source of
conflict will be cultural.” Civilizations and cultures are not black and
white, however; they’re not defined by nation-state borders. There are
colors and nuances in culture requiring a change in mindset and in strategy
to approach these new problems.”

KJW--Teaching Strategic Page 11

While difficult to assess quantitatively, literature from the critical thinking community
helps assess the degree of change here.8 In particular, the Guide to Rating Critical
Thinking developed by Washington State University identifies seven broad categories for
assessing if and to what degree critical thinking is taking place:

- Identification of the question or problem,

- willingness to articulate a personal position or argument
- willingness to consider other positions or perspectives
- identification and assessment of key assumptions
- identification and assessment of supporting data
- considers the influence of context on the problem
- identifies and assesses conclusions, implications and consequences

While such a list may not be perfect, there is certainly nothing on it that is inconsistent
with good intelligence practice. Likewise, when reading the representative example
above with these criteria in mind, the increase in nuance, the willingness to challenge an
acknowledged authority, the nimble leaps from one concept to another all become even
more obvious. The majority of the students in the class showed this kind of growth over
the course of the term both in the quality of the classroom discussions and in their written
reports.

In addition to seeing an improvement in students’ ability to detect deep patterns in

complex and disparate data sets, I also wanted that increased ability to translate into
better quality intelligence products for the decisionmakers who were sponsoring projects
in the class. Here the task was somewhat easier. I have solicited and received good
feedback from each of the decisionmakers involved in the 78 strategic intelligence
projects my students have worked on over the last 7 years. This feedback, leavened with
a sense of the cognitive complexity of the requirement, yields a rough but useful
assessment of how “good” each final project turned out to be.

Mapping this overall assessment onto a 5 point scale (where a 3 indicates average “A”
work9, a 2 and 1 indicates below and well below A work respectively, a 4 indicates A+ or
young professional work and a 5 indicates professional quality work), permits a

8
There is a widespread belief among intelligence professionals that teaching critical thinking methods will
improve intelligence analysis (See David Moore’s comprehensive examination of this thesis in his book
Critical Thinking And Intelligence Analysis). A minority of authors are less willing to jump on this
particular bandwagon (See Behavioral and Psychosocial Considerations in Intelligence Analysis: A
Preliminary Review of Literature on Critical Thinking Skills by the Air Force Research Laboratory, Human
Effectiveness Division) citing a lack of empirical evidence pointing to a correlation between critical
thinking skills and improved analysis.
9
“A” is average for the Mercyhurst Intelligence Studies seniors and second year graduate students
permitted to take Strategic Intelligence. In order to be employable in this highly competitive field, the
program requires students to maintain a cumulative 3.0 GPA simply to stay in the program. In addition, the
major is widely considered to be “challenging” and those who do not see themselves in the career of
intelligence analysis, upon reflection, often change majors. As a result, GPAs of the seniors and second
year graduate students are often quite high. The graduating class of 2010, for example, averaged a 3.66
GPA.

KJW--Teaching Strategic Page 12

comparison of the average quality of the work across various years.
Average Quality Score For Student Generated Strategic Intelligence
Products

5
4.5 4.57
4 4.09
3.8 3.92
3.67 3.64 3.73
3.5
3
Score

2.5
2
1.5
1
0.5
0
2002 2003 2004 2005 2006 2007 2008 2009 2010
Year

The chart above summarizes the results for each year. While the subjectivity inherent in
some of the evaluations possibly influenced some of the individual scores, the size of the
data pool suggests that some of these variations will be eliminated or at least smoothed
out through averaging.

There are, to be sure, a number of possible reasons to explain the surge in quality
evidenced by the most recent year group. The students could be naturally better analysts,
the quality of instruction leading up to the strategic course could have dramatically
improved, the projects could have been simpler or the results could be a statistical
artifact.

None of these reasons, in my mind, however, hold true. While additional statistical
analysis has yet to be completed, the hypothesis that games-based learning improves the
quality of an intelligence product appears to have some validity and is, at least, worthy of
further exploration.

My third goal for a games-based approach was to better lock in those ideas that would
likely be relevant to future strategic intelligence projects attempted by the students, most
likely after graduation. To get some sense if the games-based approach was successful in
this regard, I sent each of the students in the three classes a letter requesting their general
input regarding the class along with any suggestions for change or improvement. I sent
these letters approximately five months after the undergraduate classes had finished and
approximately 2.5 months after the end of the graduate class.

KJW--Teaching Strategic Page 13

Seventeen of the 75 students (23%) who took one of the three courses responded to the
email and a number of students stopped by to speak to me in person. In the end, over
40% of those who took the class responded to my request for feedback in one way or
another. This evidence, while still anecdotal, was consistent – games helped the students
remember the concepts better.

Comments such as, “Looking back, I can remember a lot of the concepts simply because
the games remind me of them” or “I am of the opinion that the only reason that the
[lessons] stood out was because they were different from any other class most students
have taken” were often mixed in with suggestions on how to improve the course. The
verbal feedback was even more encouraging, with reports of discussions and even
arguments centered on the games and their “meaning” weeks and months after the course
was completed.

The evidentiary record, in summary, is clearly incomplete but encouraging. Games–

based learning appears to have increased intelligence students’ capacity for sensemaking,
to have improved the results of their intelligence analysis and to allow the lessons learned
to persist and even encourage new exploration of strategic topics months after the course
has ended.

\What did the students think about it?

The SIRs actually measure a number of variables and identifying those that might be
most closely associated with the underlying pedagogy of a course are difficult to identify.
Instead, I chose to look at just one of the SIR-generated ratings, the Overall Evaluation of
the course. I believe that this is the single best indicator of effectiveness. A large change
here (in either the positive or the negative direction) would be a clear indication of
success or failure.

Furthermore, my assumption at the beginning of the course was that there would be a
large change in one direction or the other. I assumed that students would either love this
approach or hate it and that this would be reflected in the SIR results. The chart below,
which contains the weighted average of the Overall Evaluation score (1-5 with 5 being
best) for all classes taught in a particular year, indicates that I was wrong:

KJW--Teaching Strategic Page 14

 SIR Results for Strategic Intelligence 2003-2009

5
4.71
4.4 4.4 4.48
4.15 4.04
4

0
2002 2003 2004 2005 2006 2007 2008 2009 2010

Clearly, while students did not love it, they did not hate it either. The drop in score from
recent years could simply be attributed to the fact that the course changed from a fairly
well-oiled series of lectures and exercises to something that had the inevitable squeaks
and bumps of a new approach. Feedback from the student surveys given after the course
was over, while extremely helpful in providing suggestions for improving the class, gave
no real insight into the causes of this modest but obvious drop in student satisfaction.

Comparing this chart with the previous one concerning the quality of the final product
yields an even more interesting picture:

KJW--Teaching Strategic Page 15

 SIRs vs. Results

5
4.5
4
3.5
Score (1-5)

3
Results
2.5
SIR
2
1.5
1
0.5
0
2002 2003 2004 2005 2006 2007 2008 2009 2010
Year Group

This chart seems to be saying that the more a student thinks they are getting out of class
(as represented in their Overall Evaluation of the course) the better their final strategic
intelligence project is likely to be. This holds true, it seems, as long as strategic
intelligence is taught through more or less traditional methods of lecture, discussion and
classroom exercises. Once the underlying structure of the course is centered on games,
however, the students are less satisfied but actually perform better where it matters most
– on real-live projects for real-world decisionmakers.

Taken at face value (and ignoring, for the moment the possibility that this is all a
statistical anomaly), a possible explanation is that the students don’t realize what they are
getting “for free” from the games-based approach. Other researchers have noted that
information that had to be actively taught, assessed, re-taught and re-assessed in other
models is passively (and painlessly) acquired in a games-based environment.

I noted this effect myself in my thesis research into modeling and simulating transitions
from authoritarian rule. My goal, in that study, was to develop a predictive model; not to
teach students about the target country. One of my ancillary results, however, was that
students routinely claimed that they learned more about the target country in three hours
of playing the game than in a semester’s worth of study.

This “knowledge for free” aspect of the games-based model was nowhere more obvious
than in the fairly detailed understanding of the geography of western Russia acquired by
the students in all three classes while playing the boardgame, Defiant Russia. While this
information was available in the form of the game map, learning the geography was not
explicitly part of the instructions. Students rapidly understood, however, that they had to
understand the terrain in order to maximize their results within the game. Furthermore,

KJW--Teaching Strategic Page 16

an understanding of the geography of western Russia was critical to the formulation of
strategic options.

This raises a broader question regarding games based learning:  If students don't know
they are learning, how can they evaluate the learning process?  While I have not had time
to dig deeply into the literature regarding implicit learning, I intend to.  Giving students a
tangible sense of what they are learning in a game based environment may be one of the
biggest challenges to overcome with the approach, at least in higher education.

What else did you learn?

Many students have provided excellent feedback for improving the course. The single
most requested ‘tweak’ is, surprisingly, to include more games like Defiant Russia. The
old-school boardgame with its dice, hex maps and counters seemed to encourage a
thoughtful, collaborative (at least within the two teams playing) learning experience. In
addition, the idea of replaying history was clearly appealing to many of the students.
Only one of the students had played anything similar prior to this class and it was unclear
if any would voluntarily play something like Defiant Russia again but the
overwhelmingly positive response to the game in the feedback suggests that there is still a
place for these types of games in educational environments.

The main problem with a game like Defiant Russia and using it in an educational setting
is the amount of time it takes to play. For two experienced players, the game can move
very quickly. However, when playing it as I did, with two teams of inexperienced
players, the first turn can last the better part of an hour. The popularity of this experience
demands, however, that I take an additional look at how I might be able to carve out time
for another game like it.

Several other comments surfaced routinely. First, there was a fairly common request to
cut back on the number of games or to cut back on the games as the end of the course
approached. This request seemed to be driven by two separate reasons. The first was
that the lessons learned lost some of their potency, as students had to rapidly drop one
game only to pick up and analyze another. The second was that, for people who did not
routinely play games, learning the rules to new games – even casual games -- every
couple of days was difficult.

On the one hand, “more time on fewer subjects” is classic pedagogical advice; on the
other, “practice makes perfect” is also sound. One of my goals was to encourage the
students to not only be better but also quicker thinkers; to identify the patterns in
complex, confusing issues rapidly and flexibly. The incessant drumbeat of games over
the course of the term seemed to accomplish this. Another goal, however, was to lock in
knowledge important to the practice of strategic intelligence. This kind of learning
requires reflection and reflection takes time. Clearly, the right answer lies in properly
balancing these competing goals. How to do that in the context of a specific syllabus is
the real question and one that I will spend the next several months pondering.

KJW--Teaching Strategic Page 17

Another suggestion that seemed to make sense was to do a better job of explaining how
games-based learning worked. I provided students with some explanation and resources
early on in the course but decided not to spend much time discussing this unique
pedagogical approach. Given the feedback and the results of this study, it probably
makes some sense to discuss this approach more fully with the students. In fact, it is my
intent to give them a copy of this paper when classes begin in the fall.

Finally, there is one recommendation that I am considering with some hesitation: Make
the connections between the games and the topics covered in the course “more clear”.
My instincts say that this would be a mistake; that the purpose of the course is to
challenge students deeply, to make them travel unlit paths in darkened forests, to attempt
to climb insurmountable mountains. I would rather have them try and fail for, the way I
have constructed the course, there is no penalty in failing, only in not trying.

Clearly, here, too, the question is one of balance. At some point, the connection between
the game and the topic can be so obtuse as to be impossible to find except through dumb
luck. Likewise, simple connections do little to foster the sense of exploration and
discovery I think it critical to this approach.

Beyond these more or less common themes, I have received a wide variety of other
suggestions (including some game recommendations) that I intend to examine in detail
before the next time I teach the class. Regardless of what changes, additions or deletions
I make, the conclusion seems inescapable: Games-based learning, while not a perfect
pedagogical approach, has merit worth exploring when teaching strategic intelligence.

KJW--Teaching Strategic Page 18

Annex 1: Syllabus

STRATEGIC INTELLIGENCE
COURSE SYLLABUS

(3 Credits) – Winter 09
Mercyhurst College
Erie, Pa

INSTRUCTOR: Kristan J. Wheaton

E-mail: kwheaton@mercyhurst.edu
Office Phone: 814 824 2023
Office Hours: TBD

TEXTS:

Clavell, James (ed.), Sun Tzu’s The Art of War (Delacorte Press, New York, 1983)
Machiavelli, Nikolo, The Prince (Bantam Classics, New York, 1984)
Howard, Michael, Clausewitz (Oxford University Press, Oxford, 1983)
Huntington, Samuel, The Clash of Civilizations
Tetlock, Phillip, Expert Political Judgment
Kidder, Rushworth; How Good People Make Tough Choices : Resolving the
Dilemmas of Ethical Living (Fireside, 1996)

Additional readings are also available on Blackboard

Also available online

Required

COURSE DESCRIPTION:

Strategic estimative intelligence is perhaps the single most difficult skill an analyst must
master. Done correctly, it helps decisionmakers allocate resources today in such a way
that it positively impacts results for years to come. Done incorrectly, decisionmakers can
squander time, money and even lives before a mistake is caught and corrected. This
course is designed to teach the analyst strategic theory and the practice of strategic
intelligence. In addition, the analyst will apply these principles to “real-life” questions
posed by senior decisionmakers involved in national government, law enforcement,
business and academia.

COURSE OBJECTIVES:

The Analyst will:

1. Learn and apply the fundamentals of the strategic estimative process.

KJW--Teaching Strategic Page 19

2. Learn the fundamentals of strategy.
3. Actively participate as the member of a group in a large-scale estimative project.
4. Prepare and brief an intelligence estimate on a current issue to a senior decisionmaker
involved with that issue
5. Gain a significant level of understanding of small group dynamics and the
fundamentals of small group leadership

COURSE REQUIREMENTS:

The course contains a number of interlocked themes including, but not limited to,
strategic theory, the practice of strategic intelligence and the application of those
principles to a “real-life” problem. Analysts will participate throughout the course as the
member of a group in a large-scale estimative project.

Students will be expected to meet with the decisionmakers who have defined the real-life
problem at the time and convenience of those decisionmakers. Students will attend all
face-to-face or videoconference meetings in appropriate attire.

Much of the work of the course will be done via the web or through the Blackboard
system. Students are expected to check Blackboard and their Mercyhurst e-mail service
regularly.

Analysts will play or review all assigned games prior to the class for which they are
assigned. Students should come to a conclusion about how the game they are playing
relates to the topic of the class before entering class. Note:
- The relationship is rarely obvious, though some may see it immediately.
- The additional readings provided each week contain hints pointing to the
connection between the game and the topic. These readings are optional.

GRADING SYSTEM:

Attendance/Class Participation = 20% A= 93-100%

Terms of Reference = 10% B+=90-92% B=83-89%
Final Product = 35% C+=80-82% C=70-79%
Final Brief = 35% D=60-69% F=<59%

ADMINISTRATIVE ISSUES:

Class attendance is required. Every student is expected to attend every class. Students
are responsible for all readings in textbooks and any handouts, including those readings
that may be placed on reserve in the library. Students are expected to have read the
materials assigned prior to class. Neither cheating nor plagiarism will be tolerated.

KJW--Teaching Strategic Page 20

OUTLINE:

Week 1: Class 1 – Course Outline, Intro to the MIE, Elements of Strategy, the Role
of Intelligence
 Teams Assigned
 Game: Desktop Tower Defense
Class 2 – Developing Strategic Background Information, Assessing
Strategic Requirements
 Game: World Of Warcraft (or other quest-based game)
Class 3 – Requirements Meeting with DM NLT 2359 11 DEC 2009

Week 2: Class 4 –Theory Review: The Intelligence Process; Terms of Reference,

Budgeting (TOR)
 Game: The Space Game: Missions
 Budget Exercise Assigned

Class 5 – Group Conceptual Modeling, Managing Analytic Teams

 Game: Budget Hero
Class 6 – Group Meetings with Instructor (Actual times TBD)
 Draft TOR due 24 hours before Group Meeting
 Budget Exercise due NLT 14 DEC 2359

Week 3: Class 7 – Method Review: Case Studies

 Game: Train (Not playable) – WWI Medic ?
Class 8 – Strategy Overview: The Art of War
 Game: Go
Class 9 – Group Meetings with Instructor
 Group Conceptual Model due

Week 4: Class 10 – Method Review: ACH And Source Reliability

 Game: This Is The Only Level
 Get TOR approved before 2359 21 DEC 2009

Week 5: Class 11 – Strategy Overview: The Prince

 Game: Samarost
Class 12 – Method Review: Multi-criteria Intelligence Matrices (MCIM)
 Game: Clue
Class 13 – Group Meetings with Instructor

Week 6: Class 14 – Theory Review: Communicating With Decisionmakers

 Game: Auditorium
Class 15 – Strategy Overview: Clausewitz
 Game: The Game Of Life
Class 16 -- Group Meetings with Instructor

KJW--Teaching Strategic Page 21

Week 7: Class 17 – Strategy Overview: Clash of Civilizations
 Game: Dice Wars, Thinking Machine 4
Class 18 – Crisis Intelligence: A Strategic Perspective
No Group Meetings this week due to MLK Birthday

Week 8: No Class – Battle Week!

Week 9: Class 19 – Method Review: Decision Trees And Bayes

 Game: Bow Street Runners
Class 20 – Theory Review: Evaluating Intelligence
 Game: Global Conflicts – Child Soldier
Class 21 -- Group Meetings with Instructor
Draft Key Findings Due 24 hours before Group Meeting

Week 10: Class 22 -- Expert Political Judgment

Class 23 – How Good People Make Tough Choices
Class 24 -- Group Meetings with Instructor
Draft Final MIE due 24 hours before Group Meeting
Draft Final Briefings due at Group Meeting

Week 11/12: Final MIE due on date of presentation

Presentation and Critique of MIE NLT 19 FEB 1700

KJW--Teaching Strategic Page 22