Instructors Manual On Strategic Thinking

Critical Thinking for Strategic Intelligence
Instructors Materials
Chapter 1: Who Are the Customers?

Review case study, Yemen: the Next Afghanistan?
Does the situation continue to pose a serious threat to US interests?

Yes, AQAP is highly likely to remain a serious threat despite the departure of President
Saleh. The death of al-Awlaki is a major milestone, but other leaders almost certainly will
emerge to further his work. Moreover, a disintegrating economic and political situation in
Yemen could create conditions highly favorable to AQAP to both recruit and train future
terrorists. The Saleh regime supported US counterterrorism objectives for the most part and
such support could become more questionable in the future.
Who would you consider the key customer in preparing this report?
Most counterterrorism experts believe that Yemen could easily and quickly emerge as one of
the most dangerous locations for hatching and launching terrorist acts against US interests
and the West more broadly. For government analysts, the US President (or in other countries
the Prime Minister or Chief of State) should be viewed as the principal customer for an
analysis such as this.
o What would be the most important issues for the White House and the Presidents key
counterterrorism advisors?
Key issues for the President and the White House would be: How does the United
States assess the current threat from Yemen? How might the threat evolve over the
next few years (in both positive and negative ways)? Who and what dynamics could
increase or diminish the potential threat?
o How would you define the key issue for the Secretary of State?
Some key issues for the Secretary of State would be: How can the United States best
support domestic and foreign pressure for reform and the move to more democratic
institutions in Yemen? Who are the key partners? Who can exert the most effective
influence in this process? How can the United States and the international community
best mitigate potential problems created by a Yemen that becomes less stable in the
future?
o How would you define the key issue for the Secretary of Defense?
For the Secretary of Defense, some key issues would be: How do we sustain an
effective counterinsurgency campaign in Yemen given all the economic and political
uncertainties? Do alternative basing options need to be considered for US forces and
force projection capabilities to effectively counter any threat posed by forces resident
in Yemen? How would increased political instability in Yemen or the establishment of
a new form of government affect the military balance of power in the region? What
1
could the US military do to influence these dynamics in ways that best support US
interests?
Should the issue be posed as a threat, an opportunity, a key decision point, or some
combination of these?
At this point in the story, the two issues most deserving of attention are whether Yemen poses
a major threat and whether it offers the United Statesand the West more broadlya major
opportunity for shaping a more positive future. Government analysts traditionally tend to
focus much more heavily on threats, but policymakers always appreciate learning at the
same time any insights analysts might offer regarding potential opportunities to shape the
issue in ways that further national interests.
Are there critical unknowns, uncertainties, or contradictory information that should be

highlighted?
Several critical unknowns that require additional research or collection tasking include
learning more about popular support for or grievances against the government, the potential
role of tribal elites and Sunni Islamist extremists, the willingness of foreign powers to
interveneand the willingness of humanitarian organizations to provide reliefif conditions
deteriorate, and basic economic trends.
Chapter 2. What Are the Key Questions?

Review case study, Countering the Iranian Nuclear Threat: Stuxnet and Its Broader
Implications.
What are the key questions a senior policymaker is likely to ask about the Stuxnet attack?
Ensure that the responses satisfy the five characteristics of a good question.
The Stuxnet attack almost certainly would prompt a number of questions ranging from the
tactical to the strategic for any policymaker, including:
o Who is responsible for the attack?
o How much damage was done?
o Can the damage be repaired and how long will it take?
o How far did it set back the Iranian nuclear program?
o How are the Iranians likely to respond?
o What implications might the attack have on the broader international legal system?
o Were new precedents established regarding the future of cyber warfare?
What are the key questions those responsible for the security of industrial security systems
are likely to ask about the Stuxnet attack? Ensure that the responses satisfy the five
characteristics of a good question.
Officials responsible for industrial security most likely would ask the same set of tactical
questions, but a much different set of operational questions, including:
o Were any of our systems damaged by the attack? If so, what is the extent of the
damage and what has been done to correct any damage?
o Do the attackers pose any future threat to our systems?
o Can the code be reconfigured to threaten any other systems?
o Could we detect such an attack and what are we doing to prevent it from happening?
How would you use the Five Ws and H to identify the key question for a senior policymaker?
First, the analyst should generate the best answer to each of the Five Ws and H questions:
o Who? The United States and Israel are cited as the most likely perpetrators of the
Stuxnet attack, but the possibility of another nation with sophisticated cyber
capabilities or even a highly skilled hacker cannot be ruled out until more is known
about the attack.
o What? Research undertaken in the last two years has removed most of the initial
uncertainty. We now can say with a high level of confidence that the two digital
bombs were released in a direct attack on Iranian nuclear facilities, and we know
most of the defining elements of the attack.
o When? The attack commenced in July 2009 and continued to do damage through
mid-2010.
o Why? The motivation for the attack almost certainly was to disrupt the Iranian
nuclear program, focusing in particular on the processes for enriching uranium
sufficiently to use it in nuclear weapons.
o Where? The two most obvious points of attack were the Bashehr nuclear reactor and
the Natanz uranium enrichment facility. Other sites may also have been targeted, but
we have no reporting of problems arising at other facilities.
o How? Subsequent research by computer experts has revealed that a virus was used to
change a small piece of code in a component produced by the German company
Siemens that the Iranians use to manage the operation of turbines at the Bushehr
reactor and nuclear centrifuges at the Natanz facility.
For the senior policymaker, the answers to What? When? Where? and How? are fairly wellestablished, leaving Who? and Why? as the two questions most likely to demand their
attention. For those responsible for industrial security, far more attention should be focused
on the What? and the How?
How would you organize a paper on the implications of the Stuxnet attack using the Question
Method?
The first step would be to establish who is the key customer for the paper. Then a small but diverse
group of analysts would be assembled to brainstorm a series of key questions the customer would be
most likely to ask, probably drawn from the sets of questions provided above. The task of the group
then would be to reorganize this list of questions from the most pressing to the least pressing from the
perspective of the key customer. The paper would be organized around the final set of questions,
addressing the most important question first and the question deemed to be least pressing last.
Chapter 3: What Is the Broader Context for the Analysis?

Review case study, Financial Crises in the United States: Chronic or Avoidable? Assume this
case study was written in 2011 for generalists seeking to learn more about the US financial crisis
that began in late 2007.
Describe the broader political and economic context when this article was written. How does
this historical and theoretical background apply to the financial crisis of 2008?
When this article was written in 2011, the United States was only starting to show the signs
of recovering from a deep recession that had lasted for more than three years. US
presidential elections would be held within the year. A major topic of conversation was what
had caused the recession and what would be the best strategy for overcoming the suffering
incurred since 2007. A major focus of the presidential campaign was how to create jobs
while simultaneously taking definitive action to reduce a burgeoning federal deficit. Although
only sporadic mention was made by the candidates of the various economic schools of
thought, the positions espoused by the various schools were influential in framing the debate.
In the political discussions and debate surrounding the 2008 financial crisis, what was the
context, and how was this portrayed?
Much of the discussion in 2011 was focused on how to restart the economy or more
specifically: how to create more jobs, how to reduce the federal deficit, whether this might
include raising some taxes, and whether to consider major reductions in public entitlements.
Some argued that the deregulation from the 1980s and 1990s was detrimental to the
economyor at least inappropriately handledand called for increased regulation. Others
countered that drastic action was needed to reduce the size of government and to eliminate
many or most of the regulations facing the business community to stimulate more business
activity and create jobs. They contended that care had to be taken not to strangle a faintly
recovering economy. In fact, they believed that even more stimulus might be needed,
justifying a short-term increase in federal spending. Such increased spending, in their view,
could be offset by raising taxes on the most wealthy.
What are the key characteristics and strategies employed by the various economic schools of
thought? Construct a matrix showing their most common criticisms of government, their
preferred role for government, and the primary sources and proponents of their arguments.
See Figure IV.1 Economic Schools of Thought: Key Characteristics and Strategies.
Figure IM.1 Economic Schools of Thought: Key Characteristics and Strategies

Economic
School of
Thought
Most Common
Criticism of
Government
Preferred Role of
Government
Leading Proponents
and Sources
Keynesian
Economics
Too Little
Regulation
Supports proactive
government responses and
reliance on fiscal stimulus
policies during recessions
John Maynard Keynes, The

General Theory of Employment,
Interest and Money (1936).
Market
Failure
Theory
Inability to
Overcome Market
Inefficiencies
To intervene when free

markets fail (such as with
externalities) although this
too can lead to government
failure
Concept introduced by Victorian

philosopher Henry Sidgwick and
later developed by Herbert A.
Simon, Hugh Gravelle, and Ray
Rees.
Welfare
Economics
Failure to
Intervene to
Provide Services
Assumes a benevolent and

competent government that
should be generous in
providing social services and
opportunities for its citizens
The early Neoclassical approach

was developed by Edgeworth,
Sidgwick, Marshall, and Pigou and
the New Welfare Economics
approach is based on the work of
Pareto, Hicks, and Kaldor.
Chicago
School/
Monetarism
Too Much
Regulation
A laissez-faire policy and

reliance on monetary policy
to spur long-run economic
growth
Milton Friedman, A Monetary

History of the United States
(1963).
Distorting Prices
A laissez-faire approach
driven by the purposeful
actions of individuals and
free markets
Writings of Carl Menger, Eugen

von Bhm-Bawerk, and Friedrich
von Wieser in late-19th and early20th century Vienna.
Austrian
Theory of
Market
Process
List the key issues your customer (a well-informed, general reader) would like you to address
in this article.
Some of the key issues the general reader would like to see addressed are:
o Which positions of the various schools of economic thought best explain what caused
the financial crises of the past few decades?
o Which positions of the various schools of economic thought best explain the ultimate
policy outcomes of the financial crises of the 1980s? Who tells the best story?
o Which positions appear most relevant to address how to proceed in dealing with
todays financial and economic challenges?
o To what extent was current policy fashioned by the philosophies of the various
schools and which individuals were the most involved in propagating those views?
o To what extent are current policies a function of where policymakers come down with
regard to the various schools of thought?
o To what extent do voter ideologies, positions, and interests drive policy?
o Are the exogenous political variables cited in the case study ultimately more
influential than the philosophies expounded by the various schools?
5
How well did the author answer the so what question in the article?
The author provided substantial amounts of useful information relating to the so what
question, but did not organize this paper around this question, opting to lay out the various
arguments and let most of the facts speak for themselves.
Chapter 4: How Should I Conceptualize My Product?

Assume that you are a US State Department intelligence analyst who has been asked to prepare
an intelligence assessment focusing on the topic Yemen: the Next Afghanistan. Draw from the
case study to perform the following tasks:
Write a sentence or two describing each of the AIMS of the paper.

Audience. The primary audience for this assessment would be the originator of US foreign
policy in the State Department dealing with Yemen, such as the Assistant Secretary of State
for the Middle East. In some cases, a more senior official might be deemed the primary
recipient, such as the Undersecretary for Political Affairs, the Director of Policy &
Planning, or the Secretary of State.
Intelligence Question. The key intelligence question is whether the evolving dynamic in
Yemen will force the United Statesand particularly the State Departmentto devote
substantial and increasing resources and policy attention to the country in the coming year
or two.
Message. The message is that this could certainly prove to be the case if the succession
process is not smooth and a vacuum of leadership begins to emerge, allowing terrorist
elements to gain a stronger foothold in the country and threaten US interests.
Story. The story is intended to take readers through a short history of political, economic,
and cultural dynamics in the country to create a framework they can use for evaluating how
events are most likely to unfold in the coming year or so.
Create a sample TOR.

A sample TOR could include the following:
o The period of the estimate is two years.
o Additional research will be needed on the behavior of key tribal leaders and other
officials likely to wield the most power after Presidents Salehs expected departure
in 2012. Research will also have to be conducted to generate defensible economic
projections for 2012 and 2013, particularly in the oil sector.
o Key information gaps include the likely growing role of tribal elites, popular support
for political leaders, and the capability of social media to mobilize and articulate
grievances.
o Strategies for filling these gaps include the submission of well-tailored intelligence
collection tasking as well as debriefings of former State Department officials who
worked closely with Yemenis in the past.
o A colleague who is an economist will be tasked with conducting the economic trends
analysis.
o A professor who is knowledgeable on the topic and has delivered useful papers in the
past will be asked to write a ten-page paper discussing likely directions in tribal
politics and who is likely to emerge as a key power broker.
o The analysis would benefit from three techniques: a Key Assumptions Check, a
Multiple Scenarios Generation exercise that focuses heavily on identifying the most
critical key drivers, and the development of a set of validated and diagnostic
indicators for tracking future developments and best anticipating how Yemen will
actually evolve.
o A timeline will be established with the following target dates: Key Assumptions Check
in 1 week; Research and interview completed in 3 weeks; economic contribution
in 3.5 weeks; Multiple Scenarios Generation and Indicators workshop in 4 weeks;
first draft in 6 weeks; complete coordination in 7 weeks; final draft to customer
in 2 months.
Review the Getting Started Checklist (Figure 4.2) and identify five things you need to focus
on before beginning to draft the assessment.
1. (Question 2). What is the key concern regarding Afghanistan that we need to address in
the Yemeni paper? Key concerns include the role of tribal politics, the impact of the drug
culture, the implications of weak central government, and/or the difficulty of extracting a
super power from the situation after it becomes heavily engaged.
2. (Question 5) Do I need to establish exactly for whom I am writing this paper? If for the
Assistant Secretary, then I will not need to provide a lot of background information; if for
the Secretary, then a lot more attention will have to be paid to tribal and cultural issues
and past political maneuverings.
3. (Questions 7, 8 & 11). This topic will not likely have a single answer. Consequently, we
should give serious consideration to generating several possible scenarios and focusing
the paper on key drivers of future events. A Key Assumptions Check would also be very
useful as we kick off the project and help us challenge whether there are true parallels
between Yemen and Afghanistan.
4. (Question 9). A lot of the critical background information relates to internal tribal
politics. We usually do not get good reporting on this topic. This suggests it would be
prudent to identify some professors with richer knowledge of this topic who can
contribute to the product.
5. (Question 10). We should identify and reach out immediately to some Yemeni as well as
Afghan experts, including former political and consular officers and perhaps former
ambassadors who could provide valuable insights and perspective on this topic. Their
knowledge and insights will help focus research and surface new issues and key drivers
we have yet to consider.
Chapter 5. What Is My Analytic Approach?

Review case study, The End of the Era of Aircraft Carriers. Briefly answer the following
questions:
How would you define the issue for a global strategist, naval commander, weapons designer,
or a senior policy official?
Key customers are likely to view the issue from different perspectives, reflecting what is most
important in their particular job. For example:
o A global strategist would focus on what impact the demise of the aircraft carrier as a
platform to project national power will have on interstate relations and interstate
conflict. What new instruments such as cyber might fill this vacuum?
o A naval commander would want to test the assertions that a carrier strike group is
increasingly vulnerable and explore whether new technologies could reverse this
problem. He or she might ask what will be the priority missions of the carrier battle
group of the future.
o A weapons designer would be most interested in learning how the demand for
offensive and defensive weapons technologies will evolve over the next ten years.
How will the nature of conflict change and what types of new weapons need to be
developed?
o A senior policy official would have a shorter time horizon, focusing on how quickly
this shift is likely to occur, what implications it has for foreign relations with key
global partners and adversaries, and how a vision of a more cooperative maritime
future could best be achieved.
What information is each customer likely to request?

Key customers will have different requirements, reflecting what is needed to do their jobs.
For example:
o A global strategist would want more information on the economies of nations with
aircraft carriers and expert projections on how soon they may be forced to
decommission the ships. He or she might also want to know whether military
strategists in the countries mentioned in the case study are already adjusting their
plans and procurements in response to these drivers.
o A naval commander would want more information on anti-ship missile, torpedo, and
drone capabilities; more data on the prospects for developing better defensive
capabilities; and estimates of how many countries are likely to possess the weapons
needed to deter or defeat a carrier battle group.
o A weapons designer would be most interested in more specifics on current and
projected capabilities of both the offensive and defensive weapons systems discussed
in this case study. Another issue would be whether sufficient funding will be available
to support the development of new weapons systems.
o A senior policy official would want to know how national leaders in other countries
are dealing with this phenomenon and what expectations are being generated about
the future role of the United States in projecting a global naval presence.
8
Where would you look for information on the new technologies and evolving naval
strategies?
A major source of such information would be popular journals and other periodicals in the
areas of military strategy and doctrine, weapons development and procurement, emerging
technologies, and economic projections on a global and national scale. Other sources
include Internet sites that specialize in naval capabilities and weapons, testimony before the
US Congress, and foreign government publications on naval strategy.
Is your analytic argument descriptive, explanatory, evaluative, or estimative? Does sufficient

data exist to address this issue?
Although this case study falls into the category of estimative analysis, elements of the three
other types of argument are also reflected in the study. For example, the data provided in the
chart on global naval resources is descriptive and the discussion of why the United States is
currently unchallenged as a naval super power is explanatory. The sections on the shrinking
global naval presence and the three key drivers contain both evaluative and estimative
analysis.
How could a conference of experts help to answer the key questions?

Much of the analysis in this case study is speculative. A conference of experts would provide
an efficient mechanism for validating some of the trends presented. Assembling a group of
experts from a variety of disciplines, including naval strategy, weapons developments,
technology, economics, and foreign relations, could stimulate a rich discussion of the key
drivers and likely outcomes by forcing most participants to think outside their traditional
box. Such a group of experts would help bound the issue in terms of the seriousness of
current vulnerabilities and how quickly new and more threatening weapons technologies are
likely to emerge. Ideally, the experts would pool their expertise to construct several credible
scenarios of how this issue will evolve, providing policymakers with a firmer baseline for
future planning.
Chapter 6. Can Collaboration Contribute to a Better Answer?

Review case study, The End of the Era of Aircraft Carriers.
The article involves analysis of military, political, economic, and technological topics. The
author was an expert in only one of these areas. What would be a good strategy to get the
authors peers in the other disciplines to collaborate in producing the article?
The most important strategies focus on the beginning stages of drafting. A critical first step
would be for the author to alert his or her colleagues who work on different aspects of the
topic of the authors intent to write a paper. Colleagues might contribute information or even
papers they wrote that touch on key aspects, saving the author time in drafting. They might
also point the author to key sources of information or other papers that relate to the topic,
once again saving research time. More important, the initial contacts would reveal if
colleagues have already formed strong views on the topic that could differ from those of the
author. The question then becomes what information and insight helped to form these
opinions and how credible is this contrary evidence.
Another way to spur greater collaboration would be to invite colleagues to a concept paper
session or a Terms of Reference meeting. The author would benefit from any good ideas that
emerge from the discussions and might even persuade colleagues to contribute sections to
the paper, thus capturing their expertise and saving drafting time. Colleagues should be
given advance notice when a draft will be distributed for coordination to ensure they have
sufficient time to ask probing questions and provide useful commentary. A good critical
thinker will plan to have sufficient time to process the comments of his or her colleagues and
recraft the paper accordingly. Attribution and credit for any contributions should be
provided in accordance with office practice.
When considering the six imperatives, which would you expect to offer the greatest
opportunities and which the greatest challenges?
The article makes a bold statement that will almost certainly capture the attention of senior
officials. The drafting team would almost certainly share a sense of mission criticality and
would mutually benefit from working together in representing different analytic disciplines
and possibly different organizational cultures. If the author adopts a collaborative approach
to drafting the paper as outlined in the previous question, this should assist in developing
mutual trust.
Access and agility could pose a challenge depending on where the collaborating team
members are located and whether the existing IT systems support collaborative endeavors.
Unfortunately, collaboration usually works well when teammates work in the same office or
building, but can prove challenging if separated geographically, especially if over several
time zones.
Taking time at the start of a project to establish a common understanding could prove
particularly importantand challengingin drafting this paper given the non-traditional
approach. In the best of circumstances, the authors superiors would assume that the need to
offer incentives falls mostly on their shoulders. The managers of the author and the
contributors could consult at the beginning of the project to determine how best to ensure
that all team members get credit for contributing to the final product. The author might also
offer incentives in the form of agreeing to provide input to papers contributors plan to write
or to volunteer to provide peer review for their future papers.
If the analyst had access to a collaboration cell, how might the cell have assisted him or her?
A collaboration cell could assist the author by creating a collaborative network for the
drafting team or establishing a wiki or blog to facilitate collaboration. Cell members might
also assist the author and the team in applying the What If? Analysis methodology or other
structured techniques like Structured Brainstorming or a Key Assumptions Check, if
appropriate. If major gaps exist in the information or expertise, collaboration cell members
might be able to locate the needed expertise elsewhere within or outside the organization and
facilitate contact.
10
If conflict arises in coordinating this article within the authors organization, would you
expect the arguments to center on different facts, different interpretations, or different goals
and objectives?
o For this mostly speculative topic, differences of fact probably would be resolved
without much conflict by reviewing the credibility of sources and the presence of
other facts that provide independent confirmation. Analysts often work from different
data sets, and they need to share their entire data base with their peers. The
challenge is to identify early on when and where analysts are working from different
data bases or their facts are in conflict.
o Most of the conflict is likely to revolve around differences of interpretation. Analysts
from different functional disciplines process data in different ways; this could impede
dialogue.
o Serious differences could emerge, however, over goals and objectives because the
paper posits change that would be a major departure from several decades of
military practice. Naval or military analysts could be deeply engrained in mindsets
that support the value of surface naval dominance and might resist the argument that
both technology and economics will force a major reassessment of the costs and
benefits of aircraft carrier battle groups.
Chapter 7: How Do Models Help My Analysis?

in 2008.
What models did the author discuss in the paper? How did this affect the politics and
politicization of the problem?
The case study provided an overview of the prominent economic schools of thought including
the Keynesians, Chicago School/Monetarists, Austrians, Market Failure Theory, Welfare
Economics, and the Public Choice/Political Economy. Presenting a broad spectrum of
economic and political thought helped establish an objective baseline for efforts later in the
paper to evaluate the impact of each school and which were the most influential.
Figure IV.1 The Financial Crisis: Economic Schools of Thought represents another example
of how the influence of economic schools of thought on decision makers can be modeled in a
visual format. In this instance, the author sketched a new model of how economic schools of
thought influence the economic decision making process and frame interpretations of history.
The model suggests these dynamics could also be self-reinforcing.
Would you say these models were drawn from Traditional Analysis, Structured Analysis,
Empirical Analysis, or Quasi-Quantitative Analysis?
These models were drawn from a combination of traditional analysis, empirical analysis, and
quasi-quantitative analysis. The schools of thought were developed before the emergence of
structured analytic techniques as a form of analysis.
11
o Traditional/Structured Analysis: There is a definite qualitative analysis component to

all schools of thought. In fact, the Austrian tradition claims to be an extension and
elaboration of Adam Smiths works. While Smith is commonly known as the first
economist, he was actually a political philosophereconomics only became a
separate field much later. Smith (and later Hayek, Mises, other Austrians, Keynes,
and Chicago School economists) have derived much of their philosophies from
qualitatively analyzing history and drawing conclusions and policy
recommendations.
o Empirical Analysis: Chicago School economists/Monetarists fall more into the realm
of quantitative analysis. For example, they focus on questions such as how much to
contract or expand the money supply, and which method is most appropriateboth
largely empirical questions. Keynesians can be quantitative in their estimates of
stimulus package requirements. Both schools use empirical data to analyze and
justify their policy decisions or, if their policies were not implemented fully, argue
why the policies failed.
o Quasi-Quantitative Analysis: This form of analysis is mostly associated with the
Austrian school. Generally Austrian economists are not very quantitative but will
systemically analyze patterns to assess tendency and direction of an economy, based
on various indicators. They may perform some structured analysis, but they are not
explicit about using structured analytic techniques.
How did the use of the models facilitate your understanding of the issue?
The review of the various schools of thought and the key tenets adhered to by each school
helped the reader understand both the complexity of the issue and the underpinnings of the
political debate within the general public about which policies are likely to prove most
effective in improving economic performance in the United States and the world. Many
readers probably recognized the teachings of the Keynesian and Chicago School theorists
but were less familiar with the positions of the Austrians and other schools. Given the
growing influence of the Austrian and Public Choice schools, the case study helped put their
views in perspective. The use of a summary graphic to represent key streams of thought and
their impact on economic decision making also helped to clarify some issues and provide a
context or framework for discussing and debating the impact of the various schools.
Did the use of models suggest any key information gaps?

The economic frameworks focused on key economic decision making processes but gave
short shrift to politics and made no mention of social and demographic trends. The author
did not explore the parallel economic and financial crisis in Europe and did not touch on
how other international players (China, Brazil, India, the Middle East oil producers) could
impact the system. Another key information gap is the lack of a concluding discussion of
what current data says about whose interpretation of history is correct.
12
Chapter 8: What Types of Information Are Available?

Review case study, Puzzling Food Poisonings in Germany.
Where should we look to find authoritative information about the food poisoning outbreak
reported in Germany?
In cases such as this, a plethora of information can usually be found on the Internet.
Unfortunately, much of that information is speculation or the repackaging of information
published by other sources, mostly newspapers, wire services, and 24-hour television news
channels. Separating the wheat from the chaff can prove a major challenge. The key is to
trace all press reporting back to an original source and then evaluate the credibility of that
source before entering it into your database or using it in your paper. A more efficient way to
collect critical data is to identify government press spokespersons or official government web
sites that are responsible for informing the public about the outbreak. They are often more
comprehensive in their reporting and are less likely to continue to repeat information that
has already been released.
As you start your research, what is known and what additional information is needed?
A careful review of the available information at the start of this case study reveals that little
solid information is available except that growing numbers of people are becoming ill from
an E. coli bacteria. Most of the reporting focuses on individuals in northern Germany, but
the absence of reports from other geographic locations does not necessarily mean that the
outbreak is isolated to one area. In fact, in this case study similar outbreaks were reported in
North America and France, and the disease could have easily spread to many other locations
given the high volumes of cross-border traffic. Key unknowns include the food source that is
responsible for the poisoning, the origin of that food source, the type of strain, how easily
one could become infected, and how lethal the bacteria were. While most of the press
reporting was focused on what food was responsible for making people sick, over the long
run answers to the other questions were far more important to track down.
Another caution is that, even assuming honesty on the part of the information sources, some
had good reason to withhold information. German officials, in fact, were criticized for
speaking too soon. Experts initially had thought the source of the outbreak was Spanish
cucumbers, but had not verified this before publicly announcing it. The announcement
unnecessarily caused millions of Euros in damage to Spanish farmers. As a result,
governments and other sources of expertise are likely to be even more reluctant to provide
information in the future.
What are examples of primary and secondary sources used in this case study?
Primary sources used in the study include Professor Weidmann, who was interviewed by
Time, an official WHO report, German Health Minister Bahr quoted by BBC, WHO official
Flemming Scheutz quoted by Nature, and the statement by the European Food Safety
Authority reported by the Guardian. All the other sources qualify as secondary sources in
that they summarize what other people have said or reported. All of this information was
derived from reputable news sources, but much was still conflicting, confusing, and in the
case of the Spanish cucumbers inaccurate.
13
What tangible evidence is used in this case study? Is the case based mostly on tangible or
testimonial evidence? Which is the more trustworthy?
Examples of tangible evidence include official WHO and other European documents as well
as the forensic evidence reported by German scientific organizations and the Chinese
institute that identified the genome. The preponderance of the reporting, however, was
testimonial evidence in the form of press interviews of government officials, knowledgeable
academics, and ordinary citizens. Caution should be taken in using much of the press
reporting, the bulk of which is second and third-hand.
Chapter 9: Can I Trust the Sources?

For this chapter two case studies have been used: Puzzling Food Poisonings in Germany for
assessing the quality of sources and Countering the Iranian Nuclear Threat: Stuxnet and its
Broader Implications to explore the potential for deception.
Review the case study Puzzling Food Poisonings in Germany to examine the quality of the
sourcing.
What demonstrable tangible evidence is presented in this case? How would you rate its
credibility?
Examples of demonstrable tangible evidence include the WHO Global Alert and Response
Report posted on the WHO website and the Reports of the Robert Koch Institute, Germanys
national disease control agency. Both sources rate High in all three categories: authenticity,
the reliability of the sensing mechanism, and the accuracy of the representation. Key findings
contained sufficient detail, and the sources have well-established reputations for accurate
reporting. Nevertheless, caution should be observed in situations of this kind because
government officials are under extreme pressure to tell the public something even if it has not
been completely verified. German officials fell into this trap when they first reported that
Spanish cucumbers might be the origin of the infections.
Several individuals and institutions are quoted in this case study. How would you rate their
competence and credibility?
All the sources quoted in this case study received relatively high ratings (see Figure IM.2).
Many other sources could have been included in the paper that would have received much
lower ratings, particularly those that involved individual or press speculation about the
origin of the disease.
Are there any examples where the credibility of the collector could be confused with the
credibility of the source?
Two examples where the credibility of the collector could be confused with the credibility of
the source include early reporting from CNN that the cause of the outbreak was cucumbers
from Spain and press reporting from Hamburg that Spanish produce was to blame.
14
Figure IM.2 Credibility of Human Source Reporting on German Food Poisoning Event
Competence
Professor Weidmann,
Cornell University
WHO Global Alert &
Response Report
German Health
Minister Bahr
WHO official
Flemming Scheutz
European Food
Safety Authority
Report
Credibility
Access
Expertise
Objectivity
Veracity
Observational
Sensitivity
Cultural
Perspective
Medium
High
High
High
Medium
Medium
High
High
High
High
High
High
High
High
Medium
High
Medium
High
High
High
High
High
High
High
High
High
High
High
High
High
Source: Randolph Pherson, Pherson Associates, LLC
Were any sub-sources used whose credibility was not established?

Some of the press reporting involved sub-sources whose identity was not established,
preventing readers from evaluating the credibility of the sub-sources. The best example of
this was the article from the Sky Valley Chronicle in Washington State that provided no
information on any original sources used by the reporter.
Implications, to explore the issue of deception.
Which of the six reasons for being concerned about deception might be relevant to the story
of the Iranian Stuxnet attack?
In the early stages of the attack, the primary source of information was the Iranian
government, which would be highly motivated to deceive and well practiced in conducting
deception operations. The Iranians also would stand to lose a lot if they were truthful
because any admission of damage would offer the attackers valuable feedback on the success
of the operation and insight on how best to stage additional attacks. From the perspective of
Iranian decision makers, the ideal strategy would be to convince the perpetrators that the
attacks did little damage and the costs of mounting such a sophisticated attack greatly
outweighed the benefits.
Would the Iranians have reason to deceive other countries and the scientific community
about the extent of damage done by the Stuxnet virus on their systems and programs? Why?
How would this be reflected in the reporting?
The Iranian government would have every reason to deceive the world about the impact of
the Stuxnet virus. It would not want adversaries or global public opinion to believe its
nuclear systems were vulnerable to such attacks; it also would not want likely perpetrators to
15
learn how effective the attack actually was. For these reasons, official Iranian reporting
should be expected to deny or downplay the impact of the attacks.
Would Israel and the United States have reason to deceive the world about whether either
country was involved in propagating the Stuxnet virus? Why? How would this be reflected in
the reporting?
Israel and the United States would be highly motivated to deny any involvement in the
Stuxnet attack. One of the biggest challenges in the cyber realm is to identify the actual
source of malicious code. This makes plausible denial an easy standard to attain. Moreover,
admitting responsibility for the attack could inflame Iranian public opinion, provide the
Iranian government with justification to retaliate, and potentially violate international law.
What questions should analysts have asked when first receiving reports that a computer virus
had infected the cockpits of US Predator and Reaper drones? In what way could one argue
that such reports constitute deception?
Several questions are directly relevant and raise the possibilityin the early stagesthat
either a deception operation was being conducted or analysts were engaging in selfdeception. The reports linking the virus attack on the drones were speculation based on blog
reporting on the Internet that spread rapidly across the cyber community. This would give a
potential deceiver plausible deniability largely because the actual initial source of the report
would be difficult to determine. What made the speculation compelling is the Iranians would
have motive to retaliate, giving the initial reports a germ of credibility. Moreover, the
Iranians have a history of being deceptive and much to gain or lose if the drone program is
cancelled or suspended because it is deemed too vulnerable to cyber attack. Lastly, the
Iranians could easily track how the US military was reacting to the reports given the amount
of press speculation spawned by the initial reports.
Which of the seven rules of the road would be most useful in assessing whether the drone
virus reports might be inaccurate or deceptive?
Key rules to consider would include:
o Rule Two: Seek the opinion of those closest to the attack. In fact, once military
authorities conducted their own investigation of the unit and its computers, the
reported facts of the case changed dramatically and speculation of an Iranian source
proved unfounded.
o Rule Three is highly applicable because no tangible proof exists. The reporting either
was acknowledged speculation or emanated from Internet sources that could not be
verified.
o Rule Four: Similarly, the Internet usually qualifies as verbal intelligence in that
the veracity of what is stated in a message can be extremely difficult or impossible to
validate.
o Rule Seven correctly prompts the analyst to avoid falling into the trap of satisficing.
In this case, the Iranians would have motive and might have capability; lacking a
serious examination of alternative explanations, the analyst should avoid jumping to
the conclusion that the Iranians were responsible.
16
Chapter 10. How Should I Evaluate Sources on the Internet?

Implications.
Which Internet sources used in this case study should be investigated more closely?
Several reports quote journalists from well-respected publications or television networks
such as The New York Times, CNN, and ABCand can be assumed to be reliable.
Haaretz.com, Wired, and IDB News/PCWorld Business Center are less well known;
prudence suggests probing the track record of the named reporters as a double check.
Additional Internet research reveals that all three sources appear reputable.
o Yossi Melman is a senior commentator for the Israeli daily Haaretz. He specializes in
intelligence, security, terrorism, and strategic issues and has written seven books on
these topics, including The Nuclear Sphinx of Tehran: Mahmoud Ahmadinejad and
the State of Iran.
o Noah Shachtman is a contributing editor at Wired magazine and editor of its national
security blog, "Danger Room." He has reported from Afghanistan, Israel, Iraq,
Qatar, Kuwait, the Pentagon, Los Alamos, and from military bases around the United
States. He has written about technology and national security for The New York
Times Magazine, Slate, Salon, Esquire, Popular Science, The New York Post, Foreign
Policy, Popular Mechanics, The American Prospect Online, The Forward, and The
Bulletin of the Atomic Scientists, among others. 1
o Robert McMillan is a senior writer for IDG News Service. IDG stands for the
International Data Group, which claims it is the worlds leading technology media,
events, and research company with media brands including Computerworld,
Macworld, Network World, PCWorld, and TechWorld that reach an audience
of more than 280 million technology buyers in 97 countries. 2
The remaining source is the blog by Tyler Durden on Zero Hedge. An investigation into this
source raises some serious questions. Zero Hedge is a financial blog founded in January
2009 that claims 333,000 visitors a month, mostly from Wall Street.3 It was founded by
Daniel Ivandjiiski, who claims he writes for the blog as part of a group of writers that uses
the pseudonym Tyler Durden, a character in the movie Fight Club. Ivandjiiski is a 30year-old Bulgarian immigrant banned from working in the brokerage business for insider
trading. A former hedge-fund analyst, he is also a zealous believer in a sweeping conspiracy
theories. 4
Zero Hedge is best known for alleging that Goldman Sachs was using sophisticated, highspeed computers to siphon hundreds of millions of dollars in illegitimate trading profits from
the New York Stock Exchange, which subsequently led the US Security and Exchange
Commission to consider banning that form of computerized trading. 5 Given the anonymity of
Accessed on December 11, 2011 at http://www.noahshachtman.com/about.html

Accessed on December 11, 2011 at http://www.networkworld.com/Home/rmcmillan.html
Damien Hoffman, "First Amendment Award for Outstanding Journalism: Best Blog Zero Hedge" Wall St. Cheat
Sheet (August 4, 2009). Includes interview with co-founder "Tyler Durden." Accessed on December 11, 2011
4
Joe Hagan, "The Dow Zero Insurgency" New York magazine (September 27, 2009). Retrieved December 11, 2011
5 Joe Hagan, "The Dow Zero Insurgency" New York magazine (September 27, 2009). Retrieved December 11, 2011
2
3
17
the source and the maverick reputation of the blog, additional research would be warranted
to confirm the allegations made in the reporting.
Complete the checklist shown in Figure 10.2 for the most suspect Internet source.
See Figure IM.3.
Figure IM.3 Four Step Web Page Evaluation Checklist: zerohedge.com

Name of Website:
URL / Address:
Zero Hedge
http:// www.zerohedge.com
1. Type
Advocacy x Business Info/Ref x News

Personal Entertainment
Meta-tags Who are they

trying to attract to their
website (view / source)?
Keywords?
Older Versions How did
the site evolve? (check
archive.org)
2. Content
Accuracy
- Errors of fact or logic
- Misspellings, poor grammar
- Incorrect dates
Authority
- Author unqualified, uncited
- Poor reputation
- Sources undocumented
Objectivity
- Any blatant bias (terms, etc)?
- Persuasive aim?
- Single or multiple Point of
View?
- Any sponsors or advertising?
Currency
- Out-of date references
- When was it last updated?
- Any dead links? (also check
www.brokenlinkcheck.com)
Founded in January 2009 by Daniel Ivandjiiski, a former hedge-fund

analyst banned from working in the brokerage business for insider
trading. According to archive.org, it has been crawled 260 times
going back to February 13, 2009.
How reliable is the information?

Reliable Cant say x Unreliable
Several articles from reputable journals state that the blog is known
for promoting conspiracy theories. At times, however, its muckraking
can reveals true problems or vulnerabilities in the financial system
such as its revelations about inappropriate flash trading at Goldman
Sachs.
Author is anonymous. A group of unknown individuals write the blog
under the pseudonym of Tyler Durden, a character in the movie Fight
Club. The blog claims anonymity is required to protect contributors
from retaliation from the targets of their investigative reports.
The founder of the website was banned for insider trading and
obviously has a grudge against Wall Street. He advocates a total
crash of the stock market that would lead to the elimination of the big
banks altogether and the reinstatement of genuine free-market
capitalism.
www.brokenlinkcheck.com was used to check 1660 web pages
revealing only 20 broken or dead links, suggesting the site is active
and well maintained.
18
Coverage
- Any significant omissions?
Reporting is highly biases and lacking of presentations of balancing

perspectives.
3. Owner / Author:
Full legal company name:
-Check links to copyright
and privacy statements.
Who registered the
domain?
Who incorporated the
company? Officers /
directors?
ABC Media, Ltd
The domain is registered to ABC Media, Ltd. in Sophia, Bulgaria. No

suspicious information was noted.
The administrative and technical contact is listed as:
Hristozov, Georgi abcmedialtd@hotmail.com
P.O. Box 814, Sofia, Sofia 1000, BG
Hristozov is described by LinkedIn as a software developer at the
Bulgarian Academy of Sciences
4. Affiliations and Associations?

Who do they link to?
Shared premises? Google
Phone numbers and
addresses.
Who links to them?
- Nature of association
- Effect on credibility
What do others say about
them? Google names. Check
for urban legend, hoax, or
fraud.
Is it consistent with similar
sites?
Zero Hedge website has extensive links and postings to other blogs,
websites and articles covering global financial issues.
Zero Hedge website has extensive links and postings from other blogs,
websites and articles covering global financial issues.
Snopes.com reports no hits for zerohedge.com.
The fraud.com and webglimpse.net search sites revealed no hits.
Yes
Do any of the exceptions to traditional sourcing rules apply to the sourcing used in this case
study?
No.
How likely is it that the answer to the following question can be found on the Internet:
Did Iran conduct a virus attack on US drones in retaliation for Stuxnet?
The answer to that question cannot be found on the Internet. At best, one might be able to
find speculation that Iran was responsible for the attack, but the chances of someone
providing actual evidence that Iran was responsible is minuscule. The only authoritative
source would be a statement by an Iranian official that Iran conducted the operation
accompanied by concrete evidence proving that it launched the attacka standard that is
almost certainly never likely to be met given Iranian concerns that the United States could
use such a statement to justify retaliation. The best answer to that question lies in
constructing an argument laying out the pros and cons and systematically evaluating the
evidence and the logic that would support such an assertion. In addition, the argument
should demonstrate that any alternative explanations are less likely to be true.
19
Chapter 11. Are My Key Assumptions Well-Founded?

At the outset of this case, what key assumptions were reporters (and analysts) making about
the outbreak of E. coli in Germany? How many were supported, in need of caveats, and
unsupported?
A summary of the key assumptions is provided in Figure IM.4. In this exercise, four of the
assumptions are supported, two are caveated, and four are unsupported. The caveats are that
1) the bacteria is contagious only if direct physical contact is made, and 2) initial research
suggests that the strain is uncommon, but additional research is required to determine the
exact strain and its origin.
Figure IM.4 German Food Poisoning Key Assumptions Check

Number
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Key Assumption
The strain of bacteria was more toxic than most.
The food source of infection was cucumbers.
The food source of the infection was raw vegetables.
The geographic source of the infection was Spain.
The geographic source of the infection was northern Germany
Official reports citing the likely source of contamination would
cause serious economic damage to farmers.
The strain has never before been seen before in humans.
The strain was highly contagious.
Victims could be treated with antibiotics.
Washing ones hands and cooking all vegetables would reduce
the chances of infection.
Rating
S
U
U
U
U
S
C
C
S
S
Legend:
S = Supported
C = With Caveats
U = Unsupported
Were one in four key assumptions unsupported as is usually the case?

In this example, the ratio of Unsupported to Caveated and Supported assumptions is higher
than the average of one in four. In this case, four out of ten assumptions are Unsupported.
This is not surprising given the lack of solid information at the onset of the crisis and public
pressure to provide useful data people can use to avoid becoming infected.
How would a Key Assumptions Check exercise helped German officials working this case?
If analysts supporting German officialsand reporters for that matterhad conducted a Key
Assumptions Check as the breakout was in its early stages, the chances of suggesting
incorrect sources of infection probably would have been reduced.
20
A reasonable assumption initially made by the experts was that the strain of bacteria
originated in animals. This assumption was reasonable because most E. coli strains that lead
to outbreaks originate from animals, which is transmitted to humans through food and water.
Officials were quick to test this assumption in the lab, however, and it was fairly quickly
found to be untrue. As a result, the scientists concluded that humans were the most likely
source of the outbreak, even if food was the way in which it was transmitted.
If a Key Assumptions Check were conducted, what key uncertainties would have been identified
suggesting priority topics for investigation?
A Key Assumptions Check would have identified several obvious areas for investigation,
including what food source was causing the problem and where the source originated. More
importantly, a Key Assumptions Check might have prompted officialsand reportersto
delve more deeply into what strain of virus was causing the outbreak, how contagious and
lethal it was, what would be the most effective treatment, and whether more care should be
taken before announcing possible sources of the bacteria.
Chapter 12. Can I Make My Case?

Implications.
What is the primary assertion made by the analyst in this case study? What key evidence and
logic does the analyst provide to support this claim?
The primary assertion made in the article is that a computer worm, dubbed the Stuxnet virus,
was inserted into Iranian computers to sabotage Irans nuclear program. The motive for
such an attack is well established in the first page of the article that discusses international
concern that Iran will soon acquire the capability to make a nuclear weapon. The process by
which the worm was inserted into Iranian computer systems is well documented by several
sources as is the impact on the nuclear program.
What are two key secondary assertions and the key evidence and logic used to support the
claims?
Key secondary assertions include:
o The Stuxnet virus did substantial damage to the Bushehr nuclear reactor and the
Natanz uranium enrichment facility.
o The United States and/or Israel are responsible for launching the Stuxnet virus.
o Iran retaliated by infecting the computer controls of US Predator and Reaper drones.
Substantial evidence is provided to support the first claim, weaker evidence to support the
second, and no evidence to support the last. The case study uses several independent sources,
including reports from the Iranian government to document both directly and indirectly the
amount of damage done by the worm to both the Bushehr nuclear reactor and the Natanz
uranium enrichment facility. The assertion that the United States and/or Israel were
responsible for the attack is supported by logic (public statements made by leaders of both
countries that Iran should not become a nuclear state) and unconfirmed press reporting from
21
unnamed sources that Israel had built a test facility and Siemens had participated in a
program in Idaho to identify vulnerabilities in computer controllers identical to those used
by the Iranians. No specific evidence is provided, however, that the US or Israeli
governments were involved in implementing the actual attack. The third claim that Iran
retaliated by infecting the control systems of US drones is pure speculation, later disputed by
US Department of Defense press statements.
Are any contrary views or evidence presented that would challenge the key assertion or the
secondary assertions that you identified?
The case study presents the views of Iranian officials that challenge the contention that Iran
is intent on developing a nuclear weapons capability as well as early reports that a virus had
seriously damaged some of their nuclear programs. The study also presents an alternative
hypothesis that much of the damage could have resulted from the purchase of flawed
equipment provided by Western intelligence agencies through fake companies. In the case of
the virus in the Predator control systems, a credible alternative explanation is provided by a
US Air Force spokesperson.
Can you find examples of each type of analysisdescriptive, explanatory, evaluative, and
estimativeas it appears in the case study?
The technical discussion of how the virus most likely was inserted into the Iranian computers
and the way it operated is a good example of Descriptive Analysis. The exploration of
whether the United States and/or Israel were responsible for the attack is a good example of
Explanatory Analysis as it tries to link several distinct events to historical trends as well as
US and Israeli motives for launching such an attack. An example of Evaluative Analysis
would be the discussion of factors or criteria relating to the broader concern that someone
could customize the open source worm to launch potentially more potent attacks in the
future. The last two paragraphs of the case study pose several challenging questions that
would require a response using Estimative Analysis: How will cyber attacks be distinguished
from other forms of conventional attack in the future? Will new international norms be
established and how would they be enforced?
Can you find any examples of logical fallacies in the case study?
The assertion by the Zero Hedge blogger that Iran may have been infected the Predator
control systems in retaliation for the Stuxnet attack is a good example of several common
fallacies, including Hasty Generalization and Post Hoc, Ergo Propter Hoc. The Stuxnet
attack had attracted a lot of attention in the cyber world. When another report surfaced of a
possible cyber intrusion of US systems, a Hasty Generalization emerged suggesting that it
was a retaliatory attack. The fact that the Predator virus occurred after the Stuxnet attack is
not sufficient to argue that the first event caused the subsequent event to occur.
22
Chapter 13. Did I Consider Alternative Hypotheses?

Were sufficient alternative hypotheses concerning the origin of the infection considered?
No. Attention focused immediately on Spanish cucumbers. A good critical thinker would
generate a list of possible alternative causes of the outbreak, noting in which countries the
bacteria infection had been found, what food products had been digested before people
became ill, and where suspected food products may have been grown or processed. Such a
list could be lengthy, suggesting that it would be inadvisable to focus attention on one or two
lead hypotheses too early in the investigation.
One approach for generating alternative hypotheses is to employ the journalists list of Who?
What? Why? When? Where? and How? to stimulate ideas.
o Who: An assumption was made that the who was a farmer or group of farmers and
that the outbreak was unintentional. In fact, none of the press articles challenge this
assumption. However, unconsidered alternatives include terrorism or even industrial
sabotage. A potential null hypothesis would be that someone caused this outbreak
to occur versus it occurring naturally. When one considers the many millions of
dollars in damage this outbreak caused to Spanish and other European farmers,
industrial sabotage especially would have been a reasonable hypothesis to explore.
o What: At first, cucumbers were thought to have been the source. Subsequently, the
list expanded to all salad vegetables and finally to bean sprouts. While alternative
hypotheses were considered over the course of the outbreak, they were not considered
simultaneously and tested in this way. Even the null hypothesis of not Spanish
cucumbers could have been useful. Had these multiple hypothesessuch as all
salad vegetablesbeen tested at the same time, attention might not have been as
focused on Spanish cucumbers, and the actual source may have been found more
quickly.
o When: Not applicable.
o Where: As with the what question, the source of the outbreak was assumed to be
from Spain, then when that was not verified, northern Germany. A null hypothesis
would have been not Spain. This could have stimulated investigators to consider
other possible locations earlier. Had multiple locations been considered
simultaneously, this could have also saved some time, along with the overall costs to
European farmers.
o Why: Similar to the who question, motive was not considered. At the time, the
outbreak could have been a deliberate attack by terrorists or an act of industrial
sabotage. When it is an accident or a naturally occurring outbreak, the why
question is less relevant, but it could be helpful to explore reasons for improper food
processing.
o How: Consideration of the how raises several questions, such as whether people
became ill because the food was infected, packaging materials were infected, or the
bacteria were outside on the skin and the food was not washed.
23
Was sufficient attention paid to whether it was a version of a known strain of E. coli or an
unknown strain?
Yes. Scientists worked diligently to identify the particular strain of virus causing people to
become ill. No premature announcements were made by German officials until the scientific
work was completed.
Which techniquesituational logic, identifying historical analogies, or applying models or

theorieswould be most appropriate in generating alternative hypotheses in this situation?
The lead approach in this situation would be to use historical analogies. Outbreaks like this
have occurred many times in the past, and scientists have developed solid protocols for
investigating the cause of the illness in the most efficient manner possible. German health
officials almost certainly consulted with WHO and centers for disease control in other
countries that had dealt with similar situations. Strong pressure from the public and
politicians, however, can make it difficult to follow the protocols as assiduously as desired.
This can lead to premature announcements that cause unnecessary harm both economically
and politically.
How would an ACH exercise have helped analysts work this problem?
At the outset, conducting an ACH exercise could prove problematic because the number of
alternative hypotheses could be in the hundreds. One solution for this problem is to conduct
an initial ACH exercise using broader categories such as 1) the strain of bacteria is human
or animal based or 2) the source of contamination is an indigenous food product, an
imported food product, a food processing plant, or contaminated fields where the food was
grown, and 3) the point of outbreak is northern Germany, elsewhere in Germany, elsewhere
in Europe, or outside of Europe. As several of these options are eliminated, the use of the
ACH methodology becomes more manageable.
Chapter 14: How Do I Deal with Politicization?

that began in 2008.
Did the author present a balanced picture of the positions advocated by the various schools of
economic thought or did the author reflect a personal political agenda?
The author provided a balanced overview of the schools attempting to compare and contrast
the key precepts espoused by each school. In the concluding pages of the case study, the
author narrowed the discussion to four schools, but that is understandable as they appear to
exert the most influence in the system. The author also provided a judgment as to which
schools exerted the most influence in the crisis of the 1980s and the later mortgage crisis but
supported these views with appropriate but limited evidence and argumentation. In a more
exhaustive study, the author would be expected to provide more statistical data and other
evidence to buttress key assertions.
24
Did the organization of the article support the thesis that the case study was a rigorous
analysis and not political advocacy?
The author attempted to lay out a detailed chronology of events to establish a baseline for
evaluating her thesis as presented later in the paper. The author also provided an objective
overview of all the various schools of thought before analyzing which were the most
influential or correct in the authors opinion.
What structured analytic techniques could have the author employed to better deflect charges
of politicization?
o A detailed chronology or timeline would have helped the reader capture key events
and the gaps in time between them.
o A chart outlining the key assumptions attributed to each school of thought would
provide some useful and possibly unanticipated insights.
o A What If? Analysis that posited, for example, that the Austrians were most correct
and discussed what would happen if their views became prominent over the next ten
years would help the reader understand better the most likely strategic implications
of adopting this school of thought. Similarly, What If? papers could be written for the
Keynesian School or the Chicago School and the comparisons across all three would
be illuminating.
o Lastly, the author could have employed a Premortem Analysis and a Structured SelfCritique to ensure that there were no unfounded assumptions, logical fallacies, or
major gaps in the information and the line of analysis.
If you were to include a section on opportunities analysis in the paper, what themes would
you highlight?
The primary theme of an opportunities analysis probably would be that the future cannot be
predicted and that several future scenarios should be constructed to assess which schools of
thought offer the best insights for decisionmakers. This would require developing a robust set
of key drivers, a story for each scenario, and a list of indicators to track which scenarios are
most likely emerging. The identification of key drivers would be particularly beneficial to
policymakers signaling where it would be most effective to target new policies.
Chapter 15: How Might I Be Spectacularly Wrong?

Review case study, Countering the Iranian Nuclear Threat: Stuxnet and its Broader
Implications.
First conduct a Premortem Assessment on the case study.

o Consider each of the following questions as it relates to the case study and write
down your response to each in a sentence or two: Who? What? When? Where? Why?
and How?
o Conduct a brainstorming session asking how each of the answers you generated for
each question might be spectacularly wrong.
25
Who? The United States and Israel are cited as the most likely perpetrators of the
Stuxnet attack, but the possibility of another nation with a sophisticated cyber
capabilities or even a highly skilled hacker cannot be ruled out until more is known about
the attack.
Russia, China, and other Western powers such as the UK, France, and Germany
probably have the cyber capability to launch the attack. Russia and China would have
fewer motives, but the European countries would be at least as motivated as the United
States given their greater geographic proximity.
What? Research undertaken in the last two years has removed most of the initial
uncertainty; we now can say with a high level of confidence that the two digital bombs
were released in a direct attack on Iranian nuclear facilities and we know most of the
defining elements of the attack.
Could there be other code that has not been examined carefully that could indicate
additional attacks were attempted?
When? The attack commenced in July 2009 and continued to do damage through mid2010.
Are we certain that the Stuxnet worm is not still causing damage at locations we have not
yet identified? The absence of evidence is not necessarily evidence of absence.
Why? The motivation for the attack almost certainly was to disrupt the Iranian nuclear
program, focusing in particular on the processes for enriching uranium sufficiently to use
in nuclear weapons.
Was the motivation to disrupt operations or to disable totally both facilities?
If it was the latter, then the operation may be viewed by the perpetrator as relatively
unsuccessful. Attention should focus on whether a more powerful attack is being planned
that could have broader repercussions both within Iran and in other countries.
Where? The two most obvious points of attack were the Bashehr nuclear reactor and the
Natanz uranium enrichment facility. Other sites may also have been targeted, but no
reports of problems arising at other facilities are noted.
Our assumption that the attack was unleashed against only two targets may deserve more
scrutiny. We should check our assessment of the code language to determine if other
facilities in Iran and possibly in other countries might also have been targeted.
How? Subsequent research by computer experts has revealed that a virus was used to
change a small piece of code in a component produced by the German company Siemens
that the Iranians use to manage the operation of turbines at the Bushehr reactor and
nuclear centrifuges at the Natanz facility.
Are we certain all the damage was done by the Stuxnet code? Could many or most of the
centrifuges have been destroyed by other factors including inferior imported parts and
direct sabotage? Could the Stuxnet attack been intended to mask the activities of agent
saboteurs operating at both facilities?
26
Decide what additional work needs to be done to buttress the analysis in the case study and
whether any key judgments need to be adjusted.
The Premortem Assessment could spur additional research or collection aimed at answering
the following questions:
o What nations or groups would have sufficient motive and capability to launch the
Stuxnet attack?
o Can the code be examined to determine if there were any other intended targets of the
attack?
o Can we confirm that Stuxnet would have caused all the damage known to have
occurred?
o Is there any reporting that problems are continuing to occur at either facility or at
any other facility in Iran?
Conduct a Structured Self-Critique of the analysis in the case study focusing in particular on
the following questions:
o Have alternative explanations or hypotheses been considered?
Have we overlooked the possibility that other factors (inferior imported equipment,
sabotage) might have caused some or all of the reported damage? What if the Stuxnet
attack occurred simultaneously with other attacks?
o Was a broad range of diverse opinions solicited?
Most of the attention has focused on examining computer code and the vulnerabilities
of controllers for industrial machinery. It might be useful to enlist some Red Team
players who specialize in finding systems vulnerabilities by physically penetrating
such facilities. Including regional political specialists might also provide more
insight into who conducted the attack and what their ultimate objectives were.
o Were key assumptions identified and critically examined? Were unsupported
assumptions identified? Did the group consider how this could change the analysis?
Are we correct in assuming that only two sites were targeted by the Stuxnet worm?
That the Stuxnet attack could only be launched by a country with a highly
sophisticated offensive cyber capability? That the attack will significantly retard
nuclear programs in Iran? That the bug at Cheech Air Force Base is more a nuisance
than a threat? That the attack will not have unintended and undesirable
consequences? That Iran cannot or will not retaliate in kind or some other way?
o Are there significant information gaps? Have we considered how the lack of this data
could have influenced the key analytic judgments?
A key information gap is the identity of the perpetrator. If the person or country that
designed the worm was fairly unsophisticated, then the risk of other systems
becoming infected outside of Iran could be much greater, substantially increasing the
potential damage that has, or could soon be done, by the attack.
Another unknown is the extent to which, if at all, the United States and Israel might
have collaborated on developing the attack. If evidence later emerged regarding
close and continuing collaboration, what would that imply about future actions each
27
state might take against Iran? And what would it imply if no such collaboration
existed or had since ceased?
o Does the government, company, or competitor have a motive, opportunity, or means
to engage in deception to influence policy or change our behavior?
Many of the key players in this case study have strong motives to deceive. The state or
group that launched the attack has every reason to deceive the world that it was the
perpetrator of the attack. It does not want to be held responsible for the damage
done, it does not want to give the Iranian government a justification to retaliate, and
it could be accused by the international community of engaging in an unjustified act
of war. If neither of the prime suspects were the perpetrator, then the actual sponsor
has even more reason to be deceptive to avoid international opprobrium and
potential retaliation. The Iranians, for their part, have reason to obscure and
minimize the true impact of damage done to dissuade future attacks.
On the other hand, the computer experts quoted in the case study have no motives for
not telling the truth. In fact, the more accurate their analysis, the more their
reputations will be enhanced. Siemens also is highly motivated to ensure that as much
is known about the attacks in order to reduce the vulnerability of its future systems to
such attacks.
Chapter 16. Is My Argument Persuasive?

that began in 2008.
What was the primary thesis or argument of the paper?

The primary thesis of this paper is that political schools of thought are important, but not the
only factors that influence economic policy decision making in the wake of a financial crisis.
A secondary thesis is that the current political and economic decision making process in the
United States is not capable of initiating systematic structural reforms. Another secondary
thesis is that exogenous political variables have undercut good economic decision making
and are likely to continue to do so in the future.
How does the author establish credibility in making this argument?

The author establishes credibility by first providing a detailed chronology of past financial
crises that sets a framework for analyzing how the crisis was handled. The author also
establishes credibility by discussing all the key schools of economic thought before focusing
on those deemed most influential.
28
Which schools does the author suggest provide the most useful insights in explaining what
caused the financial crisis of 2008? Which of the five techniques does the author rely on the
most to make her or his argument?
The narrative and the graphics argue that the Keynesians and the Monetarists were the most
influential schools in the 1980s crisis, and the Austrians and Public Choice schools gained
significant influence in the more recent financial crisis. The author relies mostly on contrast
to structure the paper, showing how each school of thought differs from the other and how
each school of thought would recommend a substantially different policy approach. Knowing
that this subject is not an easy one for most general readers to grasp, the author also relies
heavily on simplicity to present the basics of each school of thought without bogging down
the reader in too much economic theory or legislative detail.
To which of the seven challenges discussed in the chapter is the author most susceptible?
o The largest challenge in drafting the paper was to avoid the trap of
oversimplification. Each school of thought has written multiple theses on the issues
covered in this case study. Care has to be taken to generalize without sacrificing the
sophistication of the baseline arguments. Given the scope and page limitations of the
case study, incorporating all major aspects of each school of thought, each factor,
and each influential person or group, would have been a difficult task. In addition,
much of the case study focuses on the financial crisis of the 1980s to establish a
baseline for analysis. This left little room to provide an in-depth discussion of the
more recent dynamics beginning in 2007.
o A parallel concern is whether the broad scope of analysis made the author subject to
the trap of vagueness. Perhaps it would have been useful to ask an economist wellschooled in this topic to edit the paper sharpening the presentation of facts as well as
lines of analysis. Another useful exercise would be to narrow the scope of the analysis
to assess if further detail makes a meaningful contribution.
o Considering the timeliness of the case when it was written in 2011, fast moving events
in the political world are likely to make the paper out-of-date as the debate
progresses, more information becomes available, and trends become clearer. The
possibility of this becoming a problem is a question that can only be answered by
someone now reading this case study.
o In addition, primarily due to time constraints, the challenge of familiarity was
something that could have biased the authors analysis. The analyses and schools of
thought were those with which the author was already familiar. While the author is
generally well versed in the spectrum of schools of thought, a more systematic
literature review would have helped to reduce this bias.
o A similar challenge the author faced was satisficing. This was also due to time
constraints and is likely to be an ever-present problem in both analysis and original
research. As with the challenge of familiarity, a more systematic literature review
would have helped mitigate this potential problem.
29
o A good technique for reducing the potential for mismatched data or improper
interpretations is to seek peer review from an outside economist (or economists).
Seek out experts with differing opinions.
Chapter 17: How Should I Portray Probability and Levels of Confidence?

Review case study, Yemen: the Next Afghanistan? Draft a final paragraph or two presenting
an analysis that answers the question: Will Yemen become another Afghanistan in the next five
years?
How would you assess the chances that Yemen will become a less reliable partner in the
counterterrorism struggle over the next five years? What words, percentages, or bettors odds
would you assign to this judgment?
Based on the evidence presented in the case study, a strong case can be made that the
situation will get worse and not better. This prognosis could change, however, based on
developments on the ground after the case study was published in 2012. That is why we
recommend that the students draft a concluding paragraph or two that captures the most
current developments in Yemen. Assuming that the fundamentals do not change dramatically,
most students will draft a sentence stating that the chances of Yemen becoming a less reliable
partner are likely, probable, between 50 and 80 percent, or two chances out of three.
What is the rationale behind your choice of a word, percentage, or bettors odd in the answer
you provided above? In other words, if you added the word because to the end of the
sentence above, what reasons would you give to complete the sentence?
Factors that could be cited to support the assessment that Yemen will become a less reliable
partner are the political pressures within the political elite, the political and social fallout
resulting from the loss of oil revenue, the relative weakness of civil society, the power of the
tribal elites, the lack of internal security, and the uncertainty of foreign assistance.
What level of confidence do you have in the sources used to support the analysis on whether
Yemen is likely to become another Afghanistan?
Most analysts would describe their level of confidence in the sources as between medium and
high. The analysis in the article is well-sourced and the author provides multiple
knowledgeable sources to buttress the analytic judgments. The inclusion of reliable classified
sources with good access could easily raise the level of confidence to high. If the situation
deteriorates significantly and reporters and country experts are denied access to the country
or to key actors or must increasingly depend on new and untested sources, then the level of
confidence could drop to low.
What level of confidence do you have in your overall assessment regarding Yemens future
reliability as a partner in the counterterrorism effort?
Depending on how developments unfold, the level of confidence in the analytic judgment
should be medium and possibly low if current conditions deteriorate sharply.
30
Chapter 18. How Can Graphics Support My Analysis?

Did the analyst incorporate an appropriate number of graphics in the article?

Yes, the graphics served three functions: presenting a large amount of data in the most
efficient way possible, demonstrating visually the threat posed by anti-ship missiles, and
summarizing the paper in a single graphic. The case study could be criticized for not
including a map showing country locations, the Strait of Malacca, and the Gulf of Aden. A
longer study could have included additional graphics summarizing anti-ship missile
capabilities and providing national budget projections for all nations that now have aircraft
carriers.
Did any graphic summarize a large amount of information in a single chart?

Figure III.1 attempts to summarize three sets of data in one overarching graphic: total
military expenditures for the states with the largest naval forces and aircraft carriers,
military spending as a percent of Gross Domestic Product (GDP), and countries with
aircraft carriers. Both vertical and horizontal bars as well as icons of aircraft carriers were
used to simplify the data presentation. Since the United States is the dominant naval power,
all the statistics relating to the United States are arrayed in a single column to the left to
show that the United States has a larger budget and more carriers than the other twelve
countries combined.
Did any graphic add meaning or provide unique insights?

All three graphics met this test to some degree.
o The chart Largest Military Budgets and Active Aircraft Carriers graphically
illustrates the dominance of the United States by showing it has a larger budget and
more carriers than the other twelve countries combined.
o The chart of anti-ship missile ranges shows how vulnerable aircraft carrier battle
groups are to new anti-ship weapons technology. It also underscores the point that
anti-ship missiles can be launched from both land and sea.
o The chart Key Drivers Reducing the Size of the Global Aircraft Carrier Fleet
captures in one graphic the primary thesis of the paperthree key drivers are likely
to force a major reduction in the global aircraft carrier fleetand uses icons to show
how the number of aircraft carriers may be reduced by as much as half in 2022.
How well did the author summarize the key message of the case study in a single graphic?
The test of a good summary graphic is whether the paper can be briefed effectively using that
one graphic alone. This graphic is fairly effective in that it:
o Shows the current number of aircraft carriers in the outer ring and the projection for
2022 in the inner ring.
o Identifies which countries have aircraft carriers.
o Illustrates the three key drivers and provides explanatory text for the role each driver
is likely to play in reducing the global carrier fleet.
31
o Reflects the dominant position of the United States, which occupies the entire top half
of the circle.
The graphic, however, does not capture the relative sizes of military expenditures for each
country nor does it provide any geographic identifier information.
Chapter 19. How Do I Present My Message in the Most Compelling Way?

Does the title of the article effectively capture both the what and the so what?
The article does a good job of capturing the what, but leaves the so what up to the
readers imagination. It points to the end of an era, but avoids using titles such as Aircraft
Carriers: No Longer Dominant, The Diminishing Value of Aircraft Carriers, or Future
without a Global Naval Presence that would overstate the main thesis of the paper. The
article was written in case study format; as a result, the so what did not appear until later
in the paper.
Is the article organized according to the inverted pyramid model, which organizes the
analysis from the most important concept, thought, or idea to the least critical information?
This paper follows the format of a case study, not that of an intelligence analysis. Most case
studies present their information chronologically and then pose questions at the end for
further discussion. This case study follows that pattern with an introduction, discussion of
current capabilities, identification of three drivers that are likely to force change, and a
conclusion. If this paper were written as an intelligence analysis, a new title would have been
chosen such as Aircraft Carrier Fleets: Growing Vulnerabilities and Shifting Priorities.
The paper would be reordered with an initial paragraph summarizing the thesis, a
presentation of three key drivers, and a separate and expanded section laying out the
implications of these trends. Much of the historical data would be reduced to graphics or
appendices.
The case study, however, adheres to the rule to present the most important information first,
and the least critical information last when presenting information in each section. For
example, the bullets describing the naval capabilities of Russia, the UK, France, and China
are ordered from largest to smallest navy. Similarly, the three technologies that threaten
aircraft carriersanti-ship missiles, torpedoes, and dronesare ordered in terms of the
seriousness of the threat posed.
Do all the paragraphs begin with good topic sentences?

Almost all the paragraphs have a solid initial summary topic sentence. One exception to this
rule might be the paragraph in italics that introduces the What If? Analysis methodology. In
the section, Succumbing to Growing Budgetary Pressures, a case can be made for crafting
stronger topic sentences, but care should be taken not to impede the flow of the argument.
32
Is the key message clearly stated?

The primary thesis is that three drivers could well force a major reduction in aircraft carrier
fleets by 2022. This theme is clearly argued given the case study format. After providing
useful background information, the theme is introduced as a recent trend in the section The
Shrinking Global Naval Presence. The following section begins with a paragraph
highlighted in italics that states the theme explicitly. And the next three sections play out the
theme in detail. In addition, the second graphic makes the same argument visually, using
bold arrows to show how each key driver is helping to shrink the global carrier fleet to
almost half its current size.
How well does the formata What If? Analysiswork for communicating the basic
message?
By presenting the theses in the context of a What If? Analysis, the authors have sought to
avoid an emotional debate over the fate of the US aircraft carrier fleet. As noted in the
article, the size of a nations naval fleet is often cited as a measure of that countrys power
and influence. Some argue that any reduction in the size of the US Navy would signal a
diminished influence and prestige. Large parts of the US defense industry are also highly
vested in the continuing construction and maintenance of naval vessels. The intent of a What
If? paper is to set these arguments aside for the moment and focus attention more
productively on the key drivers and implications by making the fact of a diminished naval
presence a hypothetical possibility.
What percentage of the sentences are in active voice? Should more sentences be in active
voice? Which ones?
This case study provides a good example of the value of using active versus passive voice.
Over 90 percent of the sentences are in active voice which brings a crisp flow to the writing.
The few exceptions are mostly topic sentences used to introduce a new subject or summarize
the current state of affairs. Once the What If? hypothesis is introduced in the italicized
paragraphImagine it is 2022 the tense shifts to past tense, but almost all of the
sentences are still in active voice..
Chapter 20. How Do I Know When I Am Finished?

What potential vulnerabilities would a Structured Self-Critique of the case study reveal?
Applying the Structured Self-Critique technique to the case study could prompt the following
questions:
o Have all the key drivers been identified?
o Are any of the key assumptions unsupported or needing caveats regarding the
strength of the tribal elites, the likelihood that oil and other natural resources will
soon run out, and popular dissatisfaction with the regime?
33
o Do we lack sufficient information on internal elite politics, the strength of the youth
movement, economic trends, and the willingness of key elites to broker a smooth
succession process?
o How reliable is the information provided by the substantive experts? Could they be
victims of the paradox of expertise defined as the tendency to make all
developments fit into an established analytic framework when fundamental shifts have
occurred that require new frameworks for understanding the underlying dynamics?
o What impact could changes in the broader environment, including the future strength
of the Taliban in Afghanistan, the withdrawal of US forces from Iraq, and
presidential elections in the United States have on the analysis?
How well does the article adhere to Figure 20.5, the Nine Principles of Effective Writing?
The case study rates poorly on the first principleto put the conclusions up front. The style
of writing is more academic than governmental, and the reader has difficulty extracting the
key judgments from the paper. It scores highly on several other principlesit follows the
case study format, is organized logically, uses precise language, and presents the case study
in a relatively distinct manner. More use of the active voice would be helpful, and a good
self-edit would have helped to sharpen the focus of the article.
Would Figure 20.4, the Four Golden Ws, or Figure 20.5, the Evaluating Major Assessments
Checklist, provide a more appropriate tool for self-editing this product? What flaws or
weaknesses in the case study would use of the proper checklist reveal?
This is a longer assessment and the appropriate checklist is Evaluating Major Assessments
Checklist. Key weaknesses that use of the Evaluating Major Assessments Checklist would
reveal include the need to:
o Address the so what question explicitly.
o State the key judgment at the outset of the paper and organize the draft in such a way
that supports the key judgment throughout.
o State levels of confidence more clearly and explicitly.
o Assess what is likely to happen next. In this instance, a good strategy would be to
generate several alternative scenarios and provide lists of indicators to allow the
customer to better anticipate which scenario is most likely to emerge.
o Explicitly identify key assumptions in a text box.
o Indicate opportunities for key customers to exploit.
o Use graphics more effectively to help carry the story.
What courses of action might use of Figure 20.7, the Critical Thinkers Checklist, suggest?
Key weaknesses that use of the Critical Thinkers Checklist would reveal include the need to:
o Articulate a clear line of analysis, putting the key judgment or bottom line up front.
o Present the reasons for any uncertainty and describe levels of confidence in the
sources and the key judgments.
o Incorporate graphics effectively to advance the storyline.
34

Instructors Manual On Strategic Thinking

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Instructors Manual On Strategic Thinking

Uploaded by

Copyright:

Available Formats

Critical Thinking for Strategic Intelligence

Chapter 1: Who Are the Customers?

Does the situation continue to pose a serious threat to US interests?

Are there critical unknowns, uncertainties, or contradictory information that should be

Chapter 2. What Are the Key Questions?

Chapter 3: What Is the Broader Context for the Analysis?

Figure IM.1 Economic Schools of Thought: Key Characteristics and Strategies

John Maynard Keynes, The

To intervene when free

Concept introduced by Victorian

Assumes a benevolent and

The early Neoclassical approach

A laissez-faire policy and

Milton Friedman, A Monetary

Writings of Carl Menger, Eugen

Chapter 4: How Should I Conceptualize My Product?

Write a sentence or two describing each of the AIMS of the paper.

Create a sample TOR.

Chapter 5. What Is My Analytic Approach?

What information is each customer likely to request?

Is your analytic argument descriptive, explanatory, evaluative, or estimative? Does sufficient

How could a conference of experts help to answer the key questions?

Chapter 6. Can Collaboration Contribute to a Better Answer?

Chapter 7: How Do Models Help My Analysis?

o Traditional/Structured Analysis: There is a definite qualitative analysis component to

Did the use of models suggest any key information gaps?

Chapter 8: What Types of Information Are Available?

Chapter 9: Can I Trust the Sources?

Source: Randolph Pherson, Pherson Associates, LLC

Were any sub-sources used whose credibility was not established?

Chapter 10. How Should I Evaluate Sources on the Internet?

Accessed on December 11, 2011 at http://www.noahshachtman.com/about.html

Figure IM.3 Four Step Web Page Evaluation Checklist: zerohedge.com

Advocacy x Business Info/Ref x News

Meta-tags Who are they

Founded in January 2009 by Daniel Ivandjiiski, a former hedge-fund

How reliable is the information?

Reporting is highly biases and lacking of presentations of balancing

ABC Media, Ltd

The domain is registered to ABC Media, Ltd. in Sophia, Bulgaria. No

4. Affiliations and Associations?

Chapter 11. Are My Key Assumptions Well-Founded?

Figure IM.4 German Food Poisoning Key Assumptions Check

Were one in four key assumptions unsupported as is usually the case?

Chapter 12. Can I Make My Case?

Chapter 13. Did I Consider Alternative Hypotheses?

Which techniquesituational logic, identifying historical analogies, or applying models or

Chapter 14: How Do I Deal with Politicization?

Chapter 15: How Might I Be Spectacularly Wrong?

First conduct a Premortem Assessment on the case study.

Chapter 16. Is My Argument Persuasive?

What was the primary thesis or argument of the paper?

How does the author establish credibility in making this argument?

Chapter 17: How Should I Portray Probability and Levels of Confidence?

Chapter 18. How Can Graphics Support My Analysis?

Did the analyst incorporate an appropriate number of graphics in the article?

Did any graphic summarize a large amount of information in a single chart?

Did any graphic add meaning or provide unique insights?

Chapter 19. How Do I Present My Message in the Most Compelling Way?

Do all the paragraphs begin with good topic sentences?

Is the key message clearly stated?

Chapter 20. How Do I Know When I Am Finished?

You might also like