kwheatonPage 32/8/2009 Nor was the US the only intelligence community to suffer such indignities. The Sovietshad their Operation RYAN, the Israelis their Yom Kippur War and the British their
Falklands Island. In each case, after the fact, senior government officials, the press andordinary citizens alike pinned the black rose of failure on their respective intelligencecommunities.To be honest, in some cases, the intelligence organization in question deserved thecriticism but, in many cases, it did not -- or at least not the full measure of fault itreceived. However, whether the blame was earned or not, in the aftermath of each of these cases, commissions were duly summoned, investigations into the causes of thefailure examined, recommendations made and changes, to one degree or another,ratified regarding the way intelligence was to be done in the future.While much of the record is still out of the public eye, I suspect it is safe to say thatintelligence successes rarely received such lavish attention.Why do intelligence professionals find intelligence so difficult, indeed impossible, toevaluate while decisionmakers do so routinely? Is there a practical model for thinkingabout the problem of evaluating intelligence? What are the logical consequences for both intelligence professionals and decisionmakers that derive from this model? Finally,is there a way to test the model using real world data?I intend to attempt to answer all of these questions but first I need to tell you a story…
A Tale Of Two Weathermen
I want to tell you a story about two weathermen; one good, competent and diligent andone bad, stupid and lazy. Why weathermen? Well, in the first place, they are
intelligence analysts, so I will not have to concern myself with all the meaninglessdistinctions that might arise if I use a real example. In the second place, they are enoughlike intelligence analysts that the lessons derived from this thought experiment – sorry, Imean “story” – will remain meaningful in the intelligence domain.Imagine first the good weatherman and imagine that he only knows one rule: If it issunny outside today, then it is likely to be sunny tomorrow (I have no idea why he onlyknows one rule. Maybe he just got hired.
Maybe he hasn’t finished weatherman schoolyet.
Whatever the reason, this is the only rule he knows). While the weatherman onlyknows this one rule, it is a good rule and has consistently been shown to be correct.His boss comes along and asks him what the weather is going to be like tomorrow. Thegood weatherman remembers his rule, looks outside and sees sun. He tells the boss, “Itis likely to be sunny tomorrow.”