MY PHILOSOPHY OF INTELLIGENCE ANALYSIS
Before I make any additional comments on intelligence analysis, I should make you aware of my philosophy on the subject. Let me start first by how I define the term “national intelligence.” For me it is the intelligence that goes to the President and his top advisors. Sometimes it is strategic; other times it might be tactical. Some comes from extremely sensitive human and technical sources, while much of it is found in open sources. It can be presented orally, written down in a few lines, or come to senior officials in a multipage document. The information may arrive as news, background insights, or even best guesses. What makes it different is he audience. National Intelligence is intended specifically to increase the knowledge of senior officials and help them make better policy decisions.
I suspect most people may wonder why I’m making such a fuss about all of this because it is common knowledge that the President is already the Intelligence Community’s (IC’s) number one customer. True, but the IC dedicates the great majority of its time, money, and effort to meeting the needs of the US military. It is the military’s money that pays most of the bills, determines the targets, and sets the priorities. As far as I’m concerned, this is as it should be. National security issues are the number one focus of all our leaders, civilian and military alike. The President is as interested in “military” intelligence as the Secretary of Defense and the Joint Chiefs of Staff. What is not as well understood is that little of what is produced for the military is of direct value to anyone but the experts it was designed to support – those who depend on detailed bits and pieces of information to do their jobs – the warfighters, planners, logisticians, etc. This creates two sets of problems.
1. The first is easy to understand. Knowing the exact locations of radar sites and the heights of bridges is essential knowledge for military experts. But in its raw form it has few, if any, applications to national intelligence problems. On the other hand, however, many of the questions asked by senior officials are not relevant to the military expert. Accordingly, a large volume of the information gathered and analyzed for military use by the IC isn’t at all helpful in answering the questions of the President and other top officials.
2. The second is a bit less transparent. It deals mostly with the choices and practices of those responsible for national intelligence. Although the potentially valuable volume of knowledge left over after meeting the military’s requirements is quite extensive, it remains largely in the same format used by the military experts – detailed micro bits and pieces – something not immediately useful to senior officials in their raw form. This need not be an insurmountable problem, but most of those responsible for doing national intelligence choose not to work with all the microdetails available. They opt instead to develop their insights and observations from a much more limited number of bits in the data – the obvious pieces, those that are easiest to find.
In combination, both sets of problems result in intelligence that I believe lack the relevance and depth needed by the President and his top advisors to protect our national security.
Overlaying these two problems is the need for expertise in the production of quality intelligence. In Webster’s view an expert is, “...one who has acquired special skill in or knowledge of a particular subject through professional training and practical experience.” It is a simple, straightforward explanation that I think everyone can understand. Just defining it, however, does not help much when it comes to understanding what it means to be an expert, or to providing insights into what it takes – the expertise required – to achieve expert status.
Academic studies of expertise and professional practitioners highlight six ingredients that go into creating an expert:
1. It is accepted that in a wide range of fields from biology to history that eight to ten years of concentration on a subject is the norm for becoming an expert. But absent other ingredients in the process, seniority is a poor indicator of performance. Indeed, it is widely recognized that a senior analyst who has stopped learning can cause great harm by dismissing new ideas coming from less experienced colleagues or, on occasion, even block them from sending their work to others higher in the organization.
2. A narrow scope of study is as important in the equation as time on the job. It is a function of mental bandwidth. The human brain does not allow anyone to know a lot about everything. Accumulating the level of detail necessary for “expert status” greatly limits the number of subjects one can master. To create new knowledge in intelligence analysis or any other field of study, one must advance incrementally by bits and pieces, with any discoveries made only making sense mostly to other experts. An epiphany here or there might give a kickstart to the process, but no one has yet found a way to advance knowledge in the absence of the bits and pieces that only a narrowly focused expert can uncover.
3. Training is the third required element. Narrow, specialized work requires the proper formal training in that field. Academic study and organizational training immediately come to mind as examples of minimum requirements. What is not so obvious is that studies have found that these activities, by themselves, are rarely sufficient in gaining expertise. Because much of the work requires familiarity with obscure fragments within a narrow specialty, classroom preparation alone rarely prepares one for expert-level work. A mentor to guide the day-to-day activities of new entrants to the specialty is almost always an essential step in one becoming an expert.
4. Hands-on-practice is another important ingredient. Even the best mentor cannot substitute for an analyst doing the detailed, microwork of an expert. As in every field, expertise comes from practice. The more you do, the better you get.
5. Peer Review is also important. It need not be a formal requirement of the process, but it is an important step an expert must take in the quest for new knowledge. It fulfills the requirement for the reproducibility of one’s work and keeps the expert on his or her toes.
6. My experience suggests that state of mind and work style also set the expert apart from one’s colleagues, especially those whose primary claim to fame is time on the job. An expert’s top priority should be finding flaws and poking holes into their favorite hypothesis, something that is a never-ending process. Experts I have known also tell me the most important thing they have learned is how little they truly know about their field of concentration, and that even their most insightful findings are no more than a work in progress, certainly not a final answer.
Despite the Community’s tendency to deemphasize research over several decades, you cannot do quality analysis without it. Knowledge creation is the product of the microwork of many experts that is combined into a finished product by others. Both parties are necessary to produce useful intelligence. The raw findings of experts are almost always unintelligible and, therefore, of little or no direct use to policymakers, while the work of generalists, without the input from experts, is too superficial to be much more than a guess.
New knowledge comes from two key components:
· The human aspect – the thinking part, and
· Its raw material – the intelligence data.
Heuer’s classic work on the "Psychology of Intelligence Analysis" introduces the human aspect of intelligence analysis – how we as individuals think and more, importantly how we think we think, something that is often at odds with reality. Analysts that do not take his findings to heart quickly learn the pain that such an oversight can cause. But thinking is only one part of the analytic process. Analysts must also master the nature and essence of the raw materials they employ in their work. In other words, they must know what intelligence data looks like. Most analysts clearly think they know what it is, and they might take offense at my saying otherwise. They would argue that everyone knows that intelligence reports – the information processed by collection organizations – and open sources are the raw materials for producing all source analysis and assessments. I would agree, but only up to a point.
First, collectors routinely overlook important data in their regular processing procedures. Invariably, when a second look is taken, especially in response to their or other analysts’ questions, they find more useful data. You just must know what you are looking for. Second, intelligence reports typically contain numerous bits and pieces of data. Rarely, if ever, do you receive a report that contains only a single bit of information, and some can contain hundreds of pieces.
· Typically, however, on first reading we remember and index only a few, often those contained in the title or summary of the report. For want of a better term, I refer to these as “obvious pieces” of information, the kind that slap you in the face or poke you in the ribs when you encounter them. In short, they are the bits of information that are hard to miss.
· In a report, several other bits are clearly recognizable, if you are specifically looking for them. These are what I call the “less obvious pieces.” However, if these pieces are not on your radar at that moment, they can quickly and easily get lost in the haystack. Sometimes you may retain a number of these bits for a while, but often it’s not long before you’ve forgotten where you even saw them.
· Unfortunately, however, reports almost always contain a third type of data, what I call the “hidden pieces.” These slip by unnoticed to everyone, even the best of analysts. These pieces do not become intelligence data. They often don’t have any meaning until viewed or combined with other information weeks, months, or even years later.
All of this is to say that we should think of the nature and essence of intelligence data as more complex than just the information found in intelligence collection reports. It is important to remember that it takes all three types of data – “obvious,” “less obvious,” and “hidden” – to do high quality analysis. A fact all those who mistakenly think analysis is just about “connecting the dots” fail to take into account.
Just like there are different types of intelligence data, we need different kinds of analysis – current reporting, directed research, and basic research – to uncover and exploit all the intelligence data available. Each of these broad categories has its own unique characteristics, but like knowledge creation in any field, they are greatly influenced by each other and should be viewed more as elements of a process rather than as separate, independent activities. Although they do differ in the value of their contribution to the knowledge creation process, all are important. Each is necessary. The quality of intelligence products depends on how well we master these disciplines and the amount of intelligence data we glean in the process.
· Current Reporting is the easiest category to explain and is the one most visible to outsiders. It’s the news – the same sort of information we receive from the front page of the Washington Post and New York Times, or from 24/7 coverage on CNN and FOX, only it is classified. Current analysts depend largely on the “obvious” data contained in formal collection reports and open-source information. This is passed on daily to the policymakers in oral and written reports. In terms of sheer volume, current reporting constitutes the intelligence analysis policymakers see or hear. A prominent example of this category of analysis is the President’s Daily Brief (PDB).
· Directed or applied Research is less visible and, unlike current reporting, starts off with a question, not an event. A directed research study, for example, might examine in detail the strengths and weaknesses of the Russian military. A product of this sort requires the use of less obvious, and hidden data, not just the obvious pieces easily identified, and depends on the existence of a body of basic research performed by other experts.
· Basic Research is the foundation from which all sound current reporting and directed research products are built. It often focuses on micro issues and details of little interest to policymakers, but it provides the essential new knowledge for producing good analysis and is the key element in determining the quality of the intelligence they eventually receive. Basic researchers also produce the resource documents, such as biographic reports, orders-of-battle, and the technical manuals necessary for conducting other types of analysis.
The current reporter is the one who, after gathering obvious bits and pieces of knowledge, weaves them into a story containing prospects and implications, a language that policymakers speak and understand. If microstudies are available on relevant topics, the current reporter adds them to the mix as well. It is the current reporter who the policymaker most often interacts with, the one who they turn to for answers. They are important players in the process, but without the contributions of micro experts, what they pass on to the policymakers is necessarily based almost entirely on the obvious type of data they have found, using a methodology of: read – absorb – report (read and remember). Missing from this approach are the insights that would be gained from any experts doing directed and basic research using the methodology – study – pose a question – design a research strategy – populate the design with data – report the results (microstudies) – the source of the less obvious and hidden data. Without them military, technical, economic, and detailed political assessments are prone to errors and misjudgments. The complete lack of basic and directed research leads me to conclude that the IC’s support for national intelligence is broken. The state of tactical intelligence, however, based on single source collection – SIGINT, Imagery, and HUMINT – is I believe not nearly so dire.