A Tradecraft Primer: Structured Analytic Techinques for ...

1 A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis Prepared by the US Government March 2009. This primer highlights Structured Analytic techniques some widely used in the private sector and academia, some unique to the intelligence profession. It is not a comprehensive overview of how intelligence officers conduct analysis. Rather, the primer highlights how Structured Analytic techniques can help one challenge judgments, identify mental mindsets, stimulate creativity, and manage uncertainty. In short, incorporating regular use of techniques such as these can enable one to structure thinking for wrestling with difficult questions. T A B L E O F C O N T E N T S. Introduction .. 1. How To Use These Techniques .. 5. Diagnostic 7. Key Assumptions 7. Quality of Information Check .. 10. Indicators or Signposts of 12. Analysis of Competing Hypotheses (ACH) .. 14. Contrarian 17. Devil's 17. Team A/Team B .. 19. High-Impact/Low-Probability 22.

2 What If? 24. Imaginative Thinking Techniques .. 27. 27. Outside-In 30. Red Team 31. Alternative Futures 34. Strategies for Using Structured Analytic Techniques .. 38. Selective Bibliography .. 40. I N T R O D U C T I O N.. THE MIND-SET CHALLENGE Cognitive and perceptual biases in Intelligence human perception and judgment are analysts should Using the Analytic techniques contained another important reason for analysts to be self-conscious in this primer will assist analysts in consider alternatives. As Richards Heuer about their reasoning and others have argued, all individuals processes. They dealing with the perennial problems should think about of intelligence: the complexity of assimilate and evaluate information how they make international developments, incomplete through the medium of mental models judgments and reach and ambiguous information, and the (sometimes also called frames or conclusions, not just inherent limitations of the human mind.)

3 Mind-sets ). These are experience- about the judgments Understanding the intentions and based constructs of assumptions and and conclusions . capabilities of adversaries and other expectations both about the world in themselves. foreign actors is challenging, especially general and more specifc domains. when either or both are concealed. These constructs strongly infuence what Richards Heuer, Moreover, transnational threats today information analysts will accept that is, The Psychology of pose even greater complexity, in that data that are in accordance with analysts' Intelligence Analysis 1. they involve multiple actors including unconscious mental models are more nonstate entities that can adapt and likely to be perceived and remembered transform themselves faster than those than information that is at odds who seek to monitor and contain them. with them. Finally, globalization has increased the diversity of outcomes when complex, Mental models are critical to allowing interactive systems such as fnancial fows, individuals to process what otherwise regional economies or the international would be an incomprehensible volume of system as a whole are in information.

4 Yet, they can cause analysts to overlook, reject, or forget important The frst hurdle for analysts is identifying incoming or missing information that is the relevant and diagnostic information not in accord with their assumptions and from the increasing volume of ambiguous expectations. Seasoned analysts may and contradictory data that is acquired be more susceptible to these mind-set through open source and clandestine problems as a result of their expertise means. Analysts must also pierce the and past success in using time-tested shroud of secrecy and sometimes mental models. The key risks of mind- deception that state and nonstate actors sets are that: analysts perceive what they use to mislead. A systematic approach expect to perceive; once formed, they are that considers a range of alternative resistant to change; new information is explanations and outcomes offers one assimilated, sometimes erroneously, into way to ensure that analysts do not dismiss existing mental models; and conficting potentially relevant hypotheses and information is often dismissed or ignored.

5 Supporting information resulting in missed opportunities to warn. 1. Richards J. Heuer, Jr., The Psychology of Intelligence Analysis (Washington: Center for the Study of Intelligence, 1999). 2. These observations were drawn from a lengthier treatment of cognitive bias found in the Sherman Kent Center's Occasional Paper, Making Sense of Transnational Threats, Vol. 3, No. 1, October 2004. 1. Common Perceptual and Cognitive Biases Perceptual Biases Biases in Evaluating Evidence Expectations. We tend to perceive what we Consistency. Conclusions drawn from a expect to perceive. More (unambiguous) small body of consistent data engender information is needed to recognize an more confdence than ones drawn from a unexpected phenomenon. larger body of less consistent data. Resistance. Perceptions resist change even Missing Information. It is diffcult to in the face of new evidence. judge well the potential impact of missing Ambiguities. Initial exposure to ambiguous evidence, even if the information gap or blurred stimuli interferes with accurate is known.

6 Perception, even after more and better Discredited Evidence. Even though information becomes available. evidence supporting a perception may be proved wrong, the perception may not quickly change. Biases in Estimating Probabilities Biases in Perceiving Causality Availability. Probability estimates are Rationality. Events are seen as part of infuenced by how easily one can imagine an an orderly, causal pattern. Randomness, event or recall similar instances. accident and error tend to be rejected as explanations for observed events. For Anchoring. Probability estimates are example, the extent to which other people adjusted only incrementally in response to or countries pursue a coherent, rational, new information or further analysis. goal-maximizing policy is overestimated. Overconfdence. In translating feelings of Attribution. Behavior of others is attributed certainty into a probability estimate, people to some fxed nature of the person or are often overconfdent, especially if they country, while our own behavior is attributed have considerable expertise.

7 To the situation in which we fnd ourselves. Intelligence analysts must actively review Identifying indicators of change (or the accuracy of their mind-sets by applying signposts) that can reduce the chances Structured Analytic techniques that will of surprise. make those mental models more explicit and expose their key assumptions. Incorporating fndings derived from these The techniques found in this primer are techniques into our intelligence products designed to assist in this regard by: also serves the policymaker by: Instilling more structure into the Highlighting potential changes that intelligence analysis. would alter key assessments or predictions. Making Analytic arguments more transparent by articulating them and Identifying key assumptions, challenging key assumptions. uncertainties, intelligence gaps and disagreements that might illuminate Stimulating more creative, out-of-the- risks and costs associated with policy box thinking and examining alternative choices.

8 Outcomes, even those with low probability, to see if available data might Exploring alternative outcomes for which support these outcomes. policy actions might be necessary. 2. Strategic Assumptions That Were Not Challenged 1941 World War II. Japan would avoid all-out war because it recognized US military superiority. Given that US superiority would only increase, Japan might view a frst strike as the only way to knock America out of the war. 1950s Korean War China would not cross the Yalu River in support of the North Korean government. Red China could make good on its threats to counter US aggression against the North. 1962 Cuban Missile Crisis The Soviet Union would not introduce offensive nuclear weapons into Cuba. The Kremlin could miscalculate and believe it could create a fait accompli that a young US President would not be prepared to reverse. 1973 Yom Kippur War Arabs knew they could not win because they had failed to cooperate in the past and still lacked suffcient air defenses to counter Israeli airpower.

9 A surprise Arab attack, even if repelled, could wound Israel psychologically and prompt international calls for cease-fres and diplomatic negotiations. 1989 German Unifcation East Germany could not unify with the West Germany against the wishes of the Soviet Union. The Soviet Union under Gorbachev might not be prepared to intervene militarily in Eastern Europe as it had in the past. 1998 Indian Nuclear Test Conducting a nuclear test risked international condemnation and US sanctions and would threaten a newly elected coalition government. A successful and surprise nuclear test could boost Indian nationalist pride and solidify public support for a shaky coalition government. 2003 Iraq's WMD Programs Saddam failed to cooperate with UN inspectors because he was continuing to develop weapons of mass destruction. If Iraqi authorities had destroyed their WMD stocks and abandoned their programs, they might refuse to fully acknowledge this to the UN to maintain Iraq's regional status, deterrence, and internal regime stability.

10 3. 4. H O W T O U S E T H E S E T E C H N I Q U E S. The Analytic techniques in this primer techniques aim at developing new are designed to help individual analysts, insights, different perspectives and/or as well as teams, explore and challenge develop alternative outcomes. In fact, their analytical arguments and mind- many of the techniques will do some sets. Some techniques are fairly simple combination of these functions. However, to understand and employ such as analysts will want to select the tool that Brainstorming and Devil's Advocacy. best accomplishes the specifc task Others are more complex and demand a they set out for themselves. Although greater degree of analytical sophistication, application of these techniques alone resource commitment, and time. All the is no guarantee of Analytic precision or techniques are included because they accuracy of judgments, it does improve have helped other analysts avoid rigid the sophistication and credibility of ways of thinking or assisted them in intelligence assessments as well as their exploring new outcomes or implications usefulness to policymakers.