Probabilities and theories based on formal rules of inference

Illusory Inferences about Probabilities

Johnson-Laird, P.N. and Savary, F. (1996) Illusory Inferences about Probabilities. Acta Psychologica 93:pp. 69-90.

Abstract

The mental model theory postulates that reasoners build models of the situations described in premises. A conclusion is possible if it holds in at least one model of the premises; it is probable if it holds in most of the models; and it is necessary if it holds in all of the models. The theory also postulates that reasoners represent as little information as possible in explicit models and, in particular, that they represent only information about what is true. One consequence is that there should be a category of illusory inferences: they will have conclusions that seem obvious, but that are wholly erroneous. Experiment 1 established the existence of such illusory inferences about probabilities. Overall, 88% of the intelligent adult subjects chose as more probable an outcome that was impossible for at least one of the illusory problems. Experiment 2 corroborated the phenomenon and showed that illusory inferences include a wide variety of problems. Finally, the paper argues that current theories based on formal rules of inference are unlikely to be able to explain the illusions.

Johnson-Laird, P.N., & Savary, F. (1996) Illusory Inferences about Probabilities. Acta Psychologica 93 69-90

Illusory Inferences about Probabilities

August 12th 1995

Abstract

The mental model theory postulates that reasoners build models of the situations described in premises. A conclusion is possible if it holds in at least one model of the premises; it is probable if it holds in most of the models; and it is necessary if it holds in all of the models. The theory also postulates that reasoners represent as little information as possible in explicit models and, in particular, that they represent only information about what is true. One consequence is that there should be a category of illusory inferences: they will have conclusions that seem obvious, but that are wholly erroneous. Experiment 1 established the existence of such illusory inferences about probabilities. Overall, 88% of the intelligent adult subjects chose as more probable an outcome that was impossible for at least one of the illusory problems. Experiment 2 corroborated the phenomenon and showed that illusory inferences include a wide variety of problems. Finally, the paper argues that current theories based on formal rules of inference are unlikely to be able to explain the illusions.

Introduction

How do people reason? They can undoubtedly do so -- even without benefit of formal training -- and yet they can say little about how their mental processes lead them to conclusions. Psychologists, however, have been less silent, and they have proposed a variety of theories (for a review, see Evans, Newstead, and Byrne, 1993). The aim of the present paper is to advance our understanding of reasoning and, in particular, of reasoning about relative probabilities. In order for readers to have an intuitive grasp of this sort of reasoning, we invite them to make the following two inferences, and to write down their answers for future reference.

Problem 1. Suppose that only one of the following assertions is true about a specific hand of cards:

There is a king in the hand or there is an ace in the hand, or both.
There is a queen in the hand or there is an ace in the hand, or both.

Which is more likely to be in the hand: the king or the ace?

Problem 2. Suppose that only one of the following assertions is true about a specific hand of cards:

If there is a jack in the hand, then there is a queen in the hand.
If there is a ten in the hand, then there is a queen in the hand.

Which is more likely to be in the hand: the queen or the jack?

We will return to these two problems later; meanwhile, readers should remember to write down their answers to them.

The principal division in psychological theories of reasoning echoes a distinction in logic. On the one hand, syntactic theories propose that reasoners rely on formal rules of inference akin to those of a logical calculus; on the other hand, semantic theories propose that reasoners construct model-like entities as their interpretations of premises. In previous publications, we have defended a psychological theory based on mental models and argued that it accounts for reasoning in all the main domains of deduction (see e.g. Johnson-Laird and Byrne, 1991). In the present paper, we describe how the two main sorts of psychological theories -- formal rule theories and the mental model theory -- can be extended to deal with reasoning about probabilities. Next, we report two experiments on reasoning about relative probabilities. Their results corroborate the model theory. The decisive evidence is that, as the theory predicts, there are illusory inferences about probabilities, i.e. there are certain premises from which most people draw the same conclusions, which seem obvious, and yet which are egregious errors.

Probabilities and theories based on formal rules of inference

An inference is valid if its conclusion must be true given that its premises are true. Certain inferences are so obviously valid that many theorists have suggested that the mind is equipped with formal rules of inference that are used automatically to make these inferences. The paradigm case of such inferences is known as modus ponens, e.g.:

If there is a queen in the hand then there is a jack in the hand.
There is a queen in the hand.
Therefore, There is a jack in the hand.

Individuals draw this conclusion rapidly and without seeming to pause for thought. Rule theories accordingly postulate that the mind contains a formal rule of inference for modus ponens:

If p then q
p
Therefore, q

which is triggered by the premises and which leads at once to the derivation of the conclusion. Such theories were first proposed twenty years ago (see e.g. Osherson, 1975; Johnson-Laird, 1975; Braine, 1978), and they continue to flourish (see e.g. Braine and O'Brien, 1991; Rips, 1994). They predict that the difficulty of an inference depends on two main factors: the length of its formal derivation (using the rules postulated by the theory) and the ease of retrieving and using each of the required rules (which can be estimated from experimental data). These theories have had some success in fitting the results of experiments (Braine, Reiser, and Rumain, 1984; Rips, 1983, 1994).

Formal rule theories have so far been formulated to deal only with deductions, such as modus ponens, which lead to conclusions that are necessarily true. In contrast, the following problem concerns the relative probabilities of two events:

If there is a jack or queen in the hand, then there is an ace. Which is more likely to be in the hand: the jack or the ace? It has a valid answer, that is, an answer that must be true given that the premise is true: the ace is more likely to be in the hand than the jack. To the best of our knowledge, psychologists have not previously studied such problems, and it is not clear whether formal rule theories are intended to apply to them. Yet, they could be made to yield judgments of relative probability in the following way. Given the premise of the problem above:

If there is a jack or queen in the hand, then there is an ace. reasoners could proceed in the following way. They start by making a supposition, i.e. assumption for the sake or argument:

There is a jack in the hand

Next, according to certain formal rule theories (e.g. Braine and O'Brien, 1991), they could derive the conclusion:

There is an ace in the hand

using a rule of the form:

p
if p or q then r
Therefore, r

Similarly, from the supposition:

There is a queen in the hand

they can derive the same conclusion using a variant of the same rule. A system of "bookkeeping" could keep track of the respective possibilities, i.e., whenever there is a jack, there is an ace, but not vice versa, and hence conclude: an ace is more likely to be in the hand than a jack. No such theory has yet been proposed by any formal theorist, and so we will not pursue the details any further. For our purposes, it is sufficient to know that a rule theory for drawing conclusions about relative probabilities is at least feasible.