Design and materials

The subjects acted as their own controls and carried out two illusory inferences of the form:

1. Only one assertion is true about a specific hand of cards:
   King or ace, or both.
   Queen or ace, or both.
   Which is more likely: king or ace?
2. Only one assertion is true about a specific hand of cards:
   If king then ace.
   If queen then ace.
   Which is more likely: ace or king?

In fact, each problem concerned different cards, but for convenience we have stated the problems as though they were always about the same particular cards. The subjects also carried out two control problems:

1. If king then ace.
   Which is more likely: king or ace?
2. If king or queen then ace.
   Which is more likely: ace or king?

Control problem 3 calls for the initial models:

with a footnote to the effect that kings are exhausted in the explicit model, i.e. kings cannot occur in any other model, whereas aces can occur in the model signified by the ellipsis. Subjects should therefore infer that the ace is more likely to be in the hand than the king. If subjects make a biconditional interpretation, however, then they will treat the initial model as exhaustively representing both kings and aces, and so they will judge the two cards to be equiprobable.

Control problem 4 calls for the initial models:

The third of these models will be omitted by those individuals who interpret "or" as an exclusive disjunction. But, in either case, the models containing kings are a proper subset of the models containing aces, and so the subjects should judge that the ace is more likely to be in the hand than the king. These conclusions to the two control problems are correct, that is, even when the models are made completely explicit they still support the same conclusions.

Each subject carried out the four inferences in a different order, i.e. one of the 24 possible orders. The materials for each problem concerned a specific hand of cards, and four distinct hands of cards were assigned to each problem at random.

Procedure

The subjects were tested individually. They were told that their task was to make a series of judgments about how likely one card or another was to be in a hand of cards. They were to base their judgments solely on the information given to them. They could take as much time as they needed to make their response. The typed problems were presented on separate pages of paper, and the subjects wrote their answers beneath the problems.

Subjects

24 Princeton students carried out the experiment. None of them had received any training in logic or had participated in any experiments on reasoning. They were paid $4 per hour for participating in the experiment, which lasted for about ten minutes.

Results

The results are summarized in Table 1. There was a massive difference between the two sorts of inferences: the subjects were correct on 71% of the control inferences, but on only 17% of the illusory inferences. 20 out of the 24 subjects were correct on more of the control inferences than the illusory ones, and there were two ties (Sign test, p < .001). Overall, 21 out of the 24 subjects chose as more probable for one or both of the illusory problems a card that could not occur in the hand.

Insert Table 1 about here

Discussion

The experiment confirmed the existence of illusory inferences and it also established that individuals judge relative probabilities according to a risky principle (principle 2 above): they judge that one event is more probable than another if it occurs in more more models than the other event. The risk here is that one model may be much more probable than another, and so the procedure is sound only if the principle of "indifference" is correct, i.e. there is no reason to suppose that one model is more probable than another.

The model theory rests on the assumption that individuals focus on true states of affairs, which they represent explicitly, and rapidly forget, if they represent them at all, false states of affairs. This assumption led to the prediction of the existence of illusory inferences, but perhaps our results are merely a happy coincidence, and the true cause of the illusions is quite different from model theory's account. We will discuss the possibility of such alternative explanations later. However, we designed Experiment 2 in part to examine their plausibility.

Experiment 2

Experiment 2 was designed to examine a variety of possible illusions, including those that depend on two main sorts of major connective: biconditionals and exclusive disjunctions, and those that depend on a variety of minor connectives, including conditionals, biconditionals, and exclusive disjunctions. We were particularly interested in finding minimal illusions, that is, those based on the simplest possible premises. Hence, four of the illusory inferences were based on only two connectives, while the remaining two illusory inferences were based on three connectives. Unlike the previous experiment, the matched control problems in all cases had the same major connectives as the illusory problems, and, as far as possible, the same minor connectives. The procedure was also more sensitive, because for each problem the subjects made independent estimates of the probabilities of the two cards (making their responses by clicking a mouse to mark their estimates on separate scales presented on a computer screen). In a separate part of the experiment, the subjects were asked to consider each problem and to state what conclusion, if any, followed from the premises. The results of this part of the experiment, however, were not sufficiently revealing to merit a full discussion, and so we deal with them only briefly.

Design

Method

The subjects inferred probabilities for six illusory problems and six matched control problems. In order to make the problems easy to understand, there were two separate assertions in each problem, and the main connective in a problem was expressed in the following way:

• Exclusive disjunction: "Only one of the following assertions is true about a specific hand of cards."
• Biconditional: "If one of the following assertions is true about a specific hand of cards, then so is the other assertion."

Two of the matched pairs of problems depended on three sentential connectives, and four of the matched pairs of problems depended on two sentential connectives.

The illusory problem in the first pair of problems based on three connectives had an exclusive disjunction as its main connective:

1. Only one of the following assertions is true about a specific hand of cards:
If there is a jack in the hand then there is a queen in the hand.
If there isn't a jack in the hand then there is a queen in the hand.

The subjects' task was to estimate the probability that the jack was in the hand and to estimate separately the probability that the queen was in the hand. The subjects made both responses by indicating the relevant position on two separate computer-presented scales running from "impossible" to "certain". According to the model theory, this problem should elicit models of the following form:

and so subjects should assign a higher probability to the queen than to the jack. It is by no means certain whether individuals will represent the implicit model signified here by the ellipsis. The implicit model may be forgotten, or it may be omitted because the two antecedents of the conditionals exhaust the possibilities. In either case, the models support the same inference: the queen should be assigned a higher probability than the jack. This conclusion, of course, is an illusion: one of the premises must be false, and so there cannot be an queen. The correct models for the premise, as shown by a program that represents all possible contingencies in a fully explicit way, are as follows:

The correct response is accordingly to assign the jack a higher probability than the queen, whose presence in the hand is impossible.

The matched control problem took the form:

1'. Only one of the following assertions is true about a specific hand of cards:
If there is a jack in the hand then then there is a queen in the hand.
if there is a jack in the hand then there is not a queen in the hand.

We abbreviate the statement of this problem as follows:

Only one is true:
If J then Q.
If J then not Q.

using the conventions that "J" denotes "there is a jack", and "not Q" denotes "there is not a queen." This problem should elicit the following models:

and so subjects should assign a higher probability to the jack than to the queen. The fully explicit models are in this case:

Hence, the response is correct.

The second pair of problems with three connectives were based on biconditionals. According to the model theory, both problems yield only implicit models, that is, models with no explicit content (see problems 2 and 2' in Table 2 below), and so they were included in the experiment primarily to see how subjects would respond when a problem seemed not to have any explicit model. In the illusory case, the correct response is that the queen is more probable than the jack; in the control case, the correct response is unclear because the premises are self-contradictory.

We also used four illusory problems and four matched controls based on only two sentential connectives. For two of these pairs, the main connective was an exclusive disjunction, and for the other two of these pairs, it was a biconditional. One of the illusory problems based on a biconditional was:

3. If one of the following assertions is true about a specific hand of cards then so is the other assertion:
There is a jack if and only if there is a queen.
There is a jack.

We can again abbreviate this problem:

If one is true so is other:
J iff Q.
J.

It should elicit the models:

and so subjects should estimate that the two cards have equal probabilities of occurring in the hand. But, the fully explicit models of the problem are:

and so the correct answer is that the queen is certain to be in the hand whereas the jack is not.

The matching control problem was of the form:
3'. If one is true so is other:
If J then Q.
J.

which should elicit the single model:

and so subjects should again infer that the two cards have equal probabilities of occurring in the hand. In this case, the models are correct, and so the conclusion is, too. The full set of six illusory problems and their matched controls are shown in Table 2. The problems were presented in a different random order to each subject.

Insert Table 2 about here