Dianmica

Mem Cogn (2011) 39:1275–1289 DOI 10.3758/s13421-011-0104-1

The Cognitive Reflection Test as a predictor of performance on heuristics-and-biases tasks
Maggie E. Toplak & Richard F. West & Keith E. Stanovich

Published online: 4 May 2011 # Psychonomic Society, Inc. 2011

Abstract The Cognitive Reflection Test (CRT; Frederick, 2005) is designed to measure the tendency to override a prepotentresponse alternative that is incorrect and to engage in further reflection that leads to the correct response. In this study, we showed that the CRT is a more potent predictor of performance on a wide sample of tasks from the heuristics-and-biases literature than measures of cognitive ability, thinking dispositions, and executive functioning. Although the CRT has a substantial correlation withcognitive ability, a series of regression analyses indicated that the CRT was a unique predictor of performance on heuristics-and-biases tasks. It accounted for substantial additional variance after the other measures of individual differences had been statistically controlled. We conjecture that this is because neither intelligence tests nor measures of executive functioning assess the tendencytoward miserly processing in the way that the CRT does. We argue that the CRT is a particularly potent measure of the tendency toward miserly processing because it is a performance measure rather than a self-report measure. Keywords Cognitive reflection test . Rational thinking . Intelligence . Heuristics and biases . Thinking dispositions
M. E. Toplak (*) Department of Psychology, York University,4700 Keele Street, 126 BSB, Toronto, Ontario, Canada M3J 1P3 e-mail: mtoplak@yorku.ca R. F. West Department of Psychology, James Madison University, Harrisonburg, VA, USA K. E. Stanovich Department of Human Development and Applied Psychology, University of Toronto, Toronto, Ontario, Canada

The Cognitive Reflection Test (CRT) is a three-item measure introduced into the journal literature byFrederick (2005). The task is designed to measure the tendency to override a prepotent response alternative that is incorrect and to engage in further reflection that leads to the correct response. The quintessential item from the CRT was first discussed by Kahneman and Frederick (2002) in an article that reframed the heuristics-and-biases literature in terms of the concept of attribute substitution.The problem is as follows: A bat and a ball cost $1.10 in total. The bat costs $1 more than the ball. How much does the ball cost? When they answer this problem, many people show a characteristic that is common to many reasoning errors: They behave like cognitive misers (Dawes, 1976; Simon, 1955, 1956; Stanovich, 2009b; Taylor, 1981; Tversky & Kahneman, 1974). They give the first response thatcomes to mind—10 cents—without thinking further and realizing that this cannot be right. The bat would then have to cost $1.10, and the total cost would then be $1.20 rather than the required $1.10. People often do not think deeply enough to realize their error, and cognitive ability is no guarantee against making the error. Frederick (2005) found that large numbers of highly select universitystudents at MIT, Princeton, and Harvard were cognitive misers; they responded that the cost was 10 cents, rather than the correct answer... 5 cents. This problem and the two others (see the Method section below) on the CRT seem at first glance to be similar to the well-known insight problems in the problem-solving literature, but they in fact display a critical difference. Classic insight problems (seeGilhooly & Fioratou, 2009; Gilhooly & Murphy, 2005) do not usually trigger an attractive alternative response. Instead, the participant sits lost in thought trying to reframe the problem correctly—as in, for example, the classic nine-dot problem. The three

1276

Mem Cogn (2011) 39:1275–1289

problems on the CRT are of interest to researchers working in the heuristics-and-biases tradition...