In order to evaluate the success of simple heuristics we need to know more about how a relevant heuristic is chosen and how we learn which cues are relevant. These meta-abilities, we believe, are at the core of ecological rationality, rather than the individual heuristics.
Gigerenzer and Todd (G & T) focus on simple heuristics for decisions instead of optimization procedures that presume unbounded rationality. We agree that this is an important step towards an understanding of the cognitive processes underlying human (and animal) decision making. However, G & T mainly explain the success of simple heuristics as an exploitation of the structure of our natural environment. We wish to add that it is not the simple heuristics in themselves that make us smart. Knowing how to choose the right heuristic in the right context, and how to select relevant cues are just as important factors in the decision process (regardless of whether theses choices are conscious or unconscious). In brief, we are smart enough to make simple heuristics work and before we can evaluate the role of simple heuristics, we must know more about how people choose to apply a particular heuristic in a given decision situation.
A heuristic must be applied in a context where it can reliably utilize the world's natural structure. For instance, the recognition heuristic is most sensibly used when there is a (causal) connection between the fact that we recognize something, and whatever factor it is we are trying to determine. In the examples presented, the environmental criteria presumed by the heuristics are fulfilled by the selection of examples. However, there are plenty of real world situations where this is not the case. If these heuristics are applied in such situations, they may not be as successful as G & T claim. To repair this one must add to the description of the heuristics how they take advantage of the environmental structure through our ability to find and understand certain regularities.
In order to apply most heuristics successfully, it is also necessary to know the value of the cues that are utilized. Another feature of G & T's examples is that knowledge concerning the relevant cues is accessible to the decision maker. The selection and ecological ordering of cues had already been made in the context the examples came from (mostly statistical textbooks). The ecological rationality of a heuristic such as Take the Best, cannot be evaluated until we know more about how the cues are selected.
The value of a cue is judged by its ecological validity, which G & T define as the proportion of correct predictions generated by the cue. Knowledge about the ecological validity of different cues is necessary for successful application of several of the heuristics studied by G & T. However, in a practical decision situation, the agent must select the cues herself and has no guarantee that the most relevant ones have been found. And in such a situation, there is often no way of knowing whether the best decision was made. Hence there is a double difficulty in determining the validity of the cues.
We believe that ecological validity should be seen as only a secondary effect of the fact that a decision maker aims at forming hypothesis about causal connections between the cues and the decision variable. The causal reasoning involved in this process may be much more determining for how the decision makers act than the statistical correlations that are used in Take The Best and the other heuristics. Unfortunately, G & T do not discuss this kind of causal reasoning (Glymour 1998, Gopnik 1998).
Even if we stick to the ecological validity studied by G & T, it will be important to know how humans learn the correlations. One reassuring finding is that humans are very good at detecting covariations between multiple variables (Holland et al 1986, Billman & Heit 1988). (But we don't know how we do it.) This capacity is helpful in finding the relevant cues to be used by a heuristic. The ability can be seen as a more general version of "ecological validity" and it may thus be used to support G & T's arguments.
Another aspect of the role of the experience of the agent is that the agent has some meta-knowledge about the decision situation and its context which influences the attitude of uncertainty to the decision. If the type of situation is well-known, the agent may be confident in applying a particular heuristic (since it has worked well before). But the agent may also be aware of her own lack of relevant knowledge and thereby choose a different (less risk-prone) heuristic. The uncertainty pertaining to a particular decision situation will also lead the agent to greater attentiveness concerning which cues are relevant in that kind of situation.
We have focused on two problems that have been neglected by G & T: How the decision maker chooses the relevant heuristics and how the decision maker learns which cues are most relevant. We believe that these meta-abilities constitute the core of ecological rationality, rather than the specific heuristics that are used (whether simple or not). In other words, the important question concerning the role of heuristics is not whether the simple heuristics do their work, but rather whether we as humans possess the right expertise to use a heuristic principle successfully, and how we acquire that expertise.
Billman, D., & Heit, E. (1988) Observational learning from internal feedback: a simulation of an adaptive learning method. Cognitive Science 12: 587 &endash; 625.
Gigerenzer, G., Todd, P. K. & the ABC Research Group (1999) Simple Heuristics That Make Us Smart. Oxford: Oxford University Press.
Glymour, C. (1998) Learning causes: psychological explanations of causal explanation. Minds and Machines, 8: 39 &endash; 60.
Gopnik, A. (1998) Explanation as orgasm. Minds and Machines, 8: 101 &endash; 118.
Holland, J. H., Holyoak, K. J., Nisbett, R. E. & Thagard, P. R. (1986) Induction: Processes of Inference, Learning and Discovery. Cambridge Mass: MIT Press.