Should your intuitions come before or after your analyses?
Danny Kahneman and I have different views on the way to use intuition.
We both agree that when informed by feedback and experience, in a reasonably coherent setting, intuitions can be valuable. My research with firefighters and other types of decision-makers has shown the importance of intuitive judgments. Kahneman’s early work with the Israel Defense Forces (IDF) convinced him of the value of intuitions. Kahneman’s experience is particularly relevant here, so let’s examine it in more detail.
In the 1950s, the IDF relied on trained evaluators to conduct a 15- to 20-minute interview to form a general impression of how well a recruit would do in the army. However, the IDF found that these predictions had little value for predicting a recruit’s future success. That’s why the IDF asked Kahneman to come up with something better. To replace the interview, Kahneman devised a more objective procedure that had the interviewers use a set of objective criteria to evaluate recruits on six different dimensions (e.g., responsibility and sociability). Kahneman made this interview as factual as possible to overcome any halo effect. He wanted the interviewers to turn off their intuition. When they resisted, Kahneman relented and added a final question, after all the objective data were collected, to use their intuition to imagine the recruit as a soldier and assign a global rating on a 5-point scale. As Kahneman expected, the new objective interview greatly increased predictive accuracy over the previous method.
But, to Kahneman’s surprise, that final global intuitive rating did just as well. The final method included both the objective and the intuitive parts. The IDF is still using Kahneman’s methodology today.
What is telling about this project is that when the interviewers just gave a global rating, their accuracy was terrible. But when they made global judgments after collecting the factual material, their global judgments were very good. Gathering the objective data improved their intuitions. That’s why Kahneman wants you to use intuitive judgments after you collect the objective data, not before.
My perspective, as described in the Recognition-Primed Decision model, is that we start with intuitions, in the form of pattern matching, and then step back and do a deliberate and conscious evaluation, perhaps going through a mental simulation of what might happen if we take an action.
Which of us is right?
I think that’s the wrong question. When faced with divergent views, a better question would be: Under what conditions should you start with your intuitions, and under what conditions should you delay your intuitions to the end?
From my perspective, if you want to get a good reading of your intuitions, you need to start with them.
I can see the benefits of doing the analyses first, breaking the choice down into smaller chunks, using objective markers where possible, and making judgments on those. You can be more accurate in making finer judgments about a set of candidates, looking at their experience level, their ability to manage stress, their performance on different test batteries, their readiness to show initiative, than you can in making an overall judgment about whether to promote the individual. If the global intuition came first, it would color the judgments on the individual dimensions. Once you decide a candidate is not suitable, you are going to have trouble making fair assessments of the individual features. In Kahneman’s terms, you are going to bias the judgments of the individual features.
From my perspective, if you want to get a good reading of your intuitions, you need to start with them. Once you have been decomposing the choice into its component dimensions and features, your first-impression intuition is going to be lost or at least distorted. Think of the exercise of flipping a coin to see what to do. You don’t want to put yourself at the mercy of the way the coin bounces. The purpose of the coin flip is to gauge how you feel about the result — are you relieved or disappointed. That’s how you can take stock of our intuitions.
What are the conditions for starting with intuitions or ending with them? Here are several dimensions to consider:
1. What is your experience level? If it is low, you should do it Kahneman’s way because you don’t have credible intuitions. Even if you have a lot of experience, you might still follow Kahneman’s advice, but if the experience is low, then your intuition won’t be very helpful, and you should bring in your intuitions at the end.
2. How much time pressure are you facing? If time is short (think of a firefighter), you won’t be able to do the decomposition Kahneman recommends.
3. How confident are you in the analytical framework? If your dimensions are time-tested and have demonstrated their value, no problem. But if the dimensions are ones you created on the fly, that’s a different story. Just being able to decompose a decision isn’t going to necessarily help you. The dimensions may overlap. They may miss important aspects of the choice. They may blind you to issues that aren’t reflected by your choice of dimensions.
4. What kind of decision are you making? For personnel selection decisions, I recommend going with Kahneman’s approach. In my own career, I can think of a number of times that I went with my gut about hiring a candidate who turned out to be a disaster or discarding a candidate who turned out to be a star — working for someone else. These kinds of decisions are sometimes referred to as “tree felling” because once you make the choice, you’re done. You don’t start to cut down a tree and then change your mind after you’ve cut halfway through the trunk. In contrast, if you are making a “hedge-trimming” decision you can adjust and adapt depending on how you like the results. You may be running a restaurant and planning out the menu, but you will be watching what your customers like and what they aren’t ordering. You will be changing the menu selections or descriptions or the way the dishes are prepared. Here, you are getting feedback and making discoveries. You don’t want to be trapped or biased by the initial analyses.
5. What kind of person are you? If you are open-minded, you should be more likely to adapt and revise your views. On the other hand, if you tend to be definitive and resist changing your mind, then you should definitely postpone making an intuitive judgment. You don’t want to fixate on that intuition.
6. Is the situation stable? If the conditions aren’t changing very rapidly, then you can safely start with your analysis. If the situation is very fluid, then your analysis might become obsolete before you are finished.
7. How clear are the goals? If you are working with an ill-structured task and ill-defined goals (e.g., a wicked problem), then you will need to rely on your intuition to adapt as you learn more. Your initial analyses won’t and shouldn’t guide you for very long.
8. Are you coordinating with others? If so, they’re likely to want some justification for your choice. Therefore, the analyses, based on decomposing the overall choice into smaller and more digestible chunks, are more likely to satisfy your team members than just saying, “This feels right to me.”
9. Can you put your intuition on hold? With personnel selection decisions that might mean reviewing the credentials for each candidate and making assessments on the fine-grained evaluation features before you interview them, not after.
Where does this leave us? I think we can all agree on the value of objective data and the value of intuitions. Your decision about how to take these nine factors into account isn’t going to be easy. The decision will take good judgment and careful analysis and experience and feedback. Hopefully, this essay has given you more to think about and more to analyze and has strengthened your intuitions.
Gary Klein is Senior Scientist at MacroCognition LLC and Chairman and Chief scientist at ShadowBox LLC. He is the author of “Seeing What Others Don’t,” “Streetlights and Shadows,” “Sources of Power,” and “Snapshots of the Mind,” from which this article is excerpted.
