High Quality
Open the downloaded document, and select print from the file menu (PDF reader required).
What Do Words Of Estimative Probability Mean?
An Exercise In Analyst Education
Kristan J. Wheaton
Mercyhurst College501 E. 38
th
Wheaton – What Do WEPs Mean?2
What Do Words Of Estimative Probability Mean?
An Exercise In Analyst Education
I was cleaning my office this week in anticipation of a new term (we are on a quarter system at Mercyhurst) and I ran across the results of a classroom exercise I conductregarding the meaning of words of estimative probability
1
(such as “likely” or “virtuallycertain”) or as they are commonly referred to around here, WEPs. I thought somediscussion of the exercise I use and the results of that exercise would be of interest tointelligence studies students and educators.
2
The value of WEPs is, of course, an ongoing question both within the intelligencecommunity and among its critics. At one end of the spectrum are those,like MichaelSchrage, who call for numeric estimates -- x has a 75% chance of happening plus or minus 10%, that sort of thing. At the other end of the spectrum are thosewho Sherman Kent called “poets”who believe that it doesn’t matter what an analyst says, policymakers and others will interpret the analysis however they wish. The intelligencecommunity (IC)has recently movedfurther in the direction of a position that, while notquite as extreme as Schrage’s, is clearly on that side of the spectrum as the “best practice” for effectively communicating the results of intelligence analysis todecisionmakers.Much of the reason for using WEPs instead of numbers centers around the imprecisenature of intelligence analysis in general, coupled with the misunderstandings that couldarise in the minds of decisionmakers if analysts used numbers to communicate their
1
This article started out a blog post (see details below). I have kept the hyperlinked endnotes for theconvenience of the reader in this draft version of the text.
2
Note: This is another attempt at what I call "experimental scholarship" (Seethis series for my first
attempt). The discussion regarding the use of blogs as a way to publish scholarly works (or, in my case,more-or-less scholarly works...)is pretty hot and heavyright now. However, I found writing an article inthe form of a series of blog posts extraordinarily useful the first time, if only for the comments that I received that I am sure will make any traditional journal article just that much better. It was the positive feedback I received from that experience that makes me want to give it another go.
Wheaton – What Do WEPs Mean?3estimative judgments. A large part of the argument against WEPs, on the other hand, hasto do with the imprecise meaning of the words themselves. In other words, what exactlydoes ‘likely” mean? That is where I intend to go next.
To Kent And Beyond!
The discussion of Words of Estimative Probability (WEPs) starts withSherman Kent’s seminal essayon the topic but hardly ends there. Linguistics experts have done a largenumber of studies on what they refer to (among other things) as “verbal expressions of probability”, “verbally expressed uncertainties” or “verbal probability expressions”.Others, in the fields of finance, health and meteorology have also wrestled with thisquestion.
3
Within the IC, though, there appears to be a limited number of studies on the topic.Steve Rieber, now with the DNI, presented his own paper on the meaning of WEPs a couple of years ago at the International Studies Association conference. At the time, he cited onlytwo studies as major researchfindings within the realm of intelligence analysis: One inDick Heuer’sclassic,The
Psychology Of Intelligence Analysis, and one (at least part of the basis for Rieber's paper) froma study of Kent Schoolanalysts.In the study cited from Heuer,analysts gave a single numerical probability for each word. For example, one analyst mightclaim that the word “likely”suggests a 75% probability whileanother might claim that itsuggests only a 60% probability.Kent School analysts, on theother hand, were asked to give arange of values for each word.The charts showing both resultsare below (Heuer'sis on this
3
Rachel Kesselman, a student at Mercyhurst, in her thesis, will address all these literatures at some length.She is scheduledto present her preliminary findings at the ISA conference at the end of March and will
likely complete her thesis (which focuses on the historical use of WEPs in National Intelligence Estimates)sometime in May or June, 2008. I won’t steal her thunder, then, but suffice it to say that this is a wellstudied topic outside the IC.
Add a Comment