A NEW THEORY OF VISUAL DETECTION* Wilson P. Tanner, Jr. and John A. Swets University of Michigan This paper is concerned with the human observer's behavior in detecting light signals in a uniform light background. Detection of these signals depends on information transmitted to cortical centers by way of the visual pathways. An analysis is made of the form of this information, and the types of decisions which can be based on information of this form. Based on this analysis, the expected form of data collected in "yes-no" and "forced-choice" psychophysical experiments is defined, and experiments demonstrating the internal consistency of the theory are presented. As the theory at first glance appears to be inconsistent with the large quantity of existing data on this subject, it is wise to review the form of this data. The general procedure is to hold signal size, duration, and certain other physical parameters constant, and to observe the way in which the frequency of detection varies as a function of intensity of the light signal. The way in which data of this form are handled implies certain underlying theoretical viewpoints. In Figure 1 the dotted 1.00 lines represent the form of the results of hypothetical //z. experiments. Consider first a single dotted line. Any point on the line might / / represent an experimental- / / ly determined point. This - - / point is corrected for chance by application of the usual formula: - n p=' - C (1) -' I where p' is the observed proportion of positive re- Figure 1. Conventional Seeing Frequency or Betting Curve. sponses, p is the corrected proportion of positive responses, and c is the intercept of the dotted curve at AI = 0. Justification of this correction depends on the validity of the assumption that a "falsealarm" is a guess, independent of any sensory activity upon which a decision might be based. For this to be the case it is necessary to have a mechanism which triggers when seeing occurs and becomes incapable of discriminating between quantities of neural attivity when seeing does not occur. Only under such a system would a guess be equally likely in the absence of seeing for all values of signal intensity. The application of the chance correction to data from both yes-no and forced-choice experiments is consistent with these assumptions. *This paper is based on work done for the U.S. Army Signal Corps under Contract No.DA-36-039 sc-15358. The experiments reported herein were conducted in the Vision Research Laboratory of the University of Michigan.

2 The solid curve represents a "true" curve onto which each of the dotted, or experimental, curves can be mapped by using the chance correction and proer estimates of "C." The parameters of the solid curve are assumed to be characteristic of the physiology of the individual's sensory system, independent of psychological control. The assumption carries with it the notion that if some threshold of neural activity is exceeded, phenomenal seeing results. To infer that the form of the curve representing the frequency of "seeing" is a function of light intensity is the same as the curve representing the frequency of "seeing" as a function of neural activity is to assume a linear relationship between neural activity and light intensity. Efforts to fit seeing frequency curves by normal probability functions suggest a predisposition toward accepting this assumption. A New Theory of Visual Detection The theory presented in this paper differs from conventional thinking with respect to these assumptions. First, it is assumed that false-alarm rate and correct detection vary together. Secondly, neural activity is assumed to be a monotone increasing function of light intensity, not necessarily linear. A more specific statement than this is left for experimental determination. DISPLAY RECEPTOR BIPOLAR OPTIC OPTIC VISUAL NERVE RADIATIONS CORTEX Figure 2. Block Diagram of the Visual Channel. Figure 2 is a block diagram of the visual pathways showing the major stages of transmission of visual information. All of the stages prior to the cortex are assumed to function only in the transmission of information, presenting to the cortex a representation of the environment. The function of interpreting this information is left to mechanisms at the cortical level. In this simplified presentation, the displayed information consists of neural impulse activity. In the case under consideration in which a signal is presented at a specified time in a known spatial location, the same restrictions are assumed to exist for the display. Thus, if the observer is asked to state whether a signal exists in location A at time B, he is assumed to consider only that information in the neural display which refers to location A at time B. A judgment on the existence of a signal is presumably based on a measure of neural activity. There exists a statistical relationship between the measure and signal intensity. That is, the more intense the signal, the greater is the average of the measures resulting. Thus, for any signal there is a universe distribution which is in fact a sampling distribution. It includes all measures which might result if the signal were repeated and measured an infinite number of times. The mean of this universe distribution is associated with the intensity level of the signal. The variance may be associated with other parameters of the signal such as duration or size, but this is beyond the scope of this paper.

3 Figure 3 shows two probability distributions: N representing the case where noise alone is sampled; that is, no signal exists, and S + N, the case where signal plus noise exists. The mean of N depends on background intensity; the mean of S + N on background plus signal intensity. The variance of N depends on signal parameters, not background x LL O X _ W G CD / CD \ A (M) Figure 3. Hypothetical Distributions of Noise and Signal Plus Noise. parameters, in the case considered here; that is, where the observer knowns "a priori" that if a signal exists, then it will be a particular signal. From the way the diagram is conceptualized, the greater the measure Y(M), the more likely it is that this sample represents a signal. But one can never be sure. Thus, if an observer is asked if a signal exists, he is assumed to base his judgment on the quantity of neural activity. He makes an observation, and then attempts to decide whether this observation is more representative of N or S + N. His task is, in fact, the task of testing a statistical hypothesis. The ideal behavior, that which makes optimum use of the information available in this task, is defined mathematically by Peterson and Birdsall in "The Theory of Signal Detectability." (2) The mathematics and symbols used are those of Peterson and Birdsall, unless otherwise stated. The first case considered is the yes-no psychophysical experiment in which a signal is presented at a known location during a well-defined interval in time. This corresponds to Peterson and Birdsall's case of the signal known exactly. For mathematical convenience, it is assumed that the distributions shown in Figure 6 are Gaussian, with variance equal for N and all values of S + N. Experimental results suggest that equal variance is not a true assumption, but that the deviations are not so great that the inconvenience of a more precise assumption is justified for the purpose of this analysis. It is also assumed that there is a cut-off point such that any measure of neural activity which exceeds that cut-off is in the criterion; that is, any value exceeding cut-off is accepted as representing the existence of a signal, and any value less than the cut-off represents noise alone. Again, for mathematical convenience, the cut-off point is assumed to be well defined and stable. The justification for accepting this convenience is twofold: First, such behavior is statistically optimum, and second, if absolute stability is physically impossible, any lack of definition or random instability throughout an experiment has the same effect mathematically as additional variance in the sampling distributions.

4 l.0. 0.9 0. 2 -......8. 0.8 -I.. 0.7 0.6 0.5 0.3 d 0.5 0 0 0o1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 P (A) 0 +OD +t I CRITERION SCALE -I -CO Figure 4. PN(A)'vs. PN(A). The Criterion Scale Shows the Corresponding Criteria Expressed in Terms of cN from MN. Now, consider the way in which the placing of the cut-off affects behavior in the case of a given signal. In the lower right-hand corner of Figure 4 the distributions N and S + N are reproduced for a value of d' = 1. The parameter d' is the square root of Peterson and Birdsall's d. The square root of d is more convenient here. d' is the difference between the means of N and S + N in terms of the standard deviation of N. The criterion scale is also calibrated in terms of the standard deviation of N. On the abscissa there is PN(A), the probability that, if no signal exists, the measure will be in the criterion, and, on the ordinate PSN(A), the probability that if a signal exists the measure will be in the criterion. If the cut-off is at -co, all measures are in the criterion: PN (A) = PSN(A) = 1. At minus one standard deviation PN(A) =.84, PsN(A) =.98. At zero, PN(A) =.5, PsN(A) =.84. At plus one PN(A) =.16 and PSN(A) =.5; and for plus oo PN(A) = PSN(A) = 0. Thus, for d' = 1 this is the curve showing possible detections for each false-alarm rate. The

5 curve represents the best that can be done with the information available, and the mirror image is the curve of worst possible behaviors. The maximum behavior in any given experiment is a point on this curve at which the slope is f where 3 1 - P(SN) (VN.CA + KN.A) P(SN) (V SNA + KSN.CA ) P(SN) is the "a priori" probability that the signal exists, VN.CA is the value of a correct rejection, KN.A the cost of a false-alarm, VSN.A the value of a correct detection, and KSN.CA is the cost of a miss. Thus, as P(SN) or VSN.A increase, or K^A decreases, /3 becomes smaller and it is worthwhile to accept a higher false-alarm rate in the interest of achieving a greater percentage of correct decisions. 1.0 --------- i/ 4 0.7 / 0. 0.2 03 0.4 0.5 0.6 0.7 0.8 0.9 1.0 2../ 0.5 10.4 01 1- l Criteria Expressed in Terms of a from.7 0.3 0.5 0.4 0.2 0.1 0 0.1 0.2 0:3 0.4 0.5 0.6 0.7 0.8 0,9 1.0 PN(A) + GO +1 0 -I -co CRITERION SCALE Figure 5. Signal Detector Curves. PN(A) vs. PSN(A). The Criterion Scale Shows the Corresponding Criteria Expressed in Terms'of aN from MN.

6 Figure 5 shows a family of curves of 1.0 PsN(A) vs. PN(A) with d' as a parameter. For 0 values of d' greater than 4, detection is very / good. This is to be compared with the pre- 08 dictions of the conventional theory shown in Figure 6 with PN(A) assumed to represent 0.7 guesses. For each value of d' it is assumed that there is a true value of PSN(A) either for 0.6 PN(A) = 0 or for some very small value. The / chance correction should transform each of these to horizontal lines. 0.4 Another way of comparing the predictions 0.3 of this theory with those of conventional theory EACH LINE IS FOR is to construct the so-called betting curves, 0.2 IEEN IN A DIFFEREN'T SIGNAL or curves showing the predicted shape of the I psychophysical function. These are shown in 01 INTENSITY Figure 7, where P(A), the probability of ac- / ceptance, is plotted as a function of d'. These 0 0.2 0.4 0.6 0.8 10 curves will not correct into the same curve by the application of the chance correction. N The shift is horizontal rather than vertical. Figure 6. Ps(A) vs. PN(A) as a Function of d' Assuming the The dotted portions of the curve show that Guessing Hypothesis. we are dealing with only a part of the curve, and thus, in the terms of this theory, it is improper to apply a normalizing procedure such as the chance-correction formula to that 1.0 part of the curve. / In the forced-choice psychophysical experiment, maximum behavior is defined in a different way. In the general forced-choice experi- a ment, the observer knows that the signal will occur in one of N inter-/ vals, and he is forced to choose in which of these intervals it occurs. The information upon which his de- cision is based is contained in the same display as in the case of the - yes-no experiment, and, presum- 0O ably, the value of d' for any given light intensity must be the same. While the solution of this problem Figure 7. P(A) as a Function of d' Assuming the Theory. is not contained in "The Theory of Signal Detectability," Peterson and Birdsall have assisted greatly in determining this solution. The probability that a correct answer P(C) will result for a given value of d' is the probability that one sample from the S + N distribution is greater than the greatest of N - 1 samples from the distribution of noise alone. The case in which four intervals are used is the basis for Figure 8. This figure shows the probability of one sample from S + N being greater than the greatest of three from N. For a given value of d' this is P(C) = / F(X)3g(x)dx (3) X = -0O

7 where F(x) is the area of N and 1.0 g(x) is the ordinate of S + N. In Figure 8 P(C), as determined by this integration, is plotted as a function of d' for the equal vari- 0.8 ance case. Criterion of Internal Consistency 0.6 These two sets of predictions are for the standard experi- 0.4 mental situations. They are based on the same neurological parameters. Thus, if the parameters, that is, d"s, are estimated 0. from one of the experiments, these estimates should furnish a basis for predicting the data for the other experiment if the theory is 0 0 2 3 4 5 6 internally consistent. An equiva- 2 3 4 5 6 lent criterion of internal consist- d ency is for both experiments to yield the same estimates of d'. Figure 8. P(C) as a Function of dl-A Theoretical Curve. Experimental Design Experiments were conducted to test this internal consistency, using three Michigan sophomores as observers. All of the experiments employed a circular target 30 minutes in diameter, 1/100 second in duration, on a ten foot-lambert background. Details of the experimental procedure and the laboratory have been published by Blackwell (1). The observers were trained in the temporal forced-choice experiment. The signal appeared in a known location at one of four specified times, and the observers were forced to choose the time at which they thought the signal occurred. Five light intensity increments were used here, with fifty observations per point per experimental session. The last two of these sessions were the test session so that each forced-choice point in the analysis is based on 100 experimental observations. Following the forced-choice experiments, there was a series of yes-no experiments under the same experimental conditions, except that only four light intensity increments were used. These were the same as the four greatest intensities used in the forced-choice experiments, reduced by adding a.1 fixed filter. In the first four of these sessions, two values of "a priori" probabilities, P(SN) equal to.8 and.4, were used. The observers were informed of the value of P(SN) before each experimental session. No values or costs were incorporated in these four sessions, which were excluded from the analysis as practice sessions. The test experiments consisted of twelve sessions in each of which all of the information necessary for the calculation of a /3 (the best possible decision level) was furnished the observers. While they did not know the formal calculation of 3, they knew the direction of cut-off change indicated by a change in any of these factors. The values and costs were made real to the observers, for they were actually paid off in cash. It was possible for them to earn as much as two dollars extra in a single experimental session as a result of this pay-off.

8 The first four sessions each carried the same value of 3 as P(SN) =.8 and the same pay-off was maintained. A high value of PN(A), or false-alarm rate, resulted. In the next four sessions with P(SN) held at.8, KN.A and VN.CA were gradually increased from session to session (not within sessions) until PN(A) dropped to a low value. Then P(SN) was dropped to.4, KNA and V NCA were reduced so that for the thirteenth session PN(A) stayed low. The last three sessions successively involved increases in VS.NA and KSN.CA again forcing PN(A) toward a higher value. Re sults Figures 9 and 10 show scatter diagrams of PsN(A) vs. PN(A) for a particular intensity of signal and for a single observer. These scatter diagrams can be used to estimate d'. In Figure 9 the estimate of d' is.7. In Figure 10, the estimate of d' is 1.3. Each d' estimated in this way is based on 560 observations. A procedure similar to this was used for the d"s for each of four signals for each of the four observers. In the forced-choice experiment the estimates of d' are made by entering our forcedchoice curve (Figure 8) using the observed percentage correct as an estimate of P(C). 1.0 0. 6d __ 4 -- _ —-— _-_ 0.4 Z L."^ ifly../. A.!1' r _": - _ ".t. _ =.' ".1.'' ~.. ____ _'" 0. 0.2 0.3 0.4 0.5 0.6 0.7.8 0.9 1.0 0.9 PN(A) 0.7 4. 0.6.', 0.2 I!i /. _ _...-. - -, —-: ----- ----- 0.2' 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0.. Figure 9.

9 1.0 0.8 o._ 1. __.:_..... 0.6Figure 10 observer, showing about the same picture. Figure 13 is for the third observer, show0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 P, (A) Figures 14, 15Figure 1.fo Figure 11 shows log d' as a function of log signal intensity for the first observer, the estimates of d' being from both forced-choice and yes-no experiments. In general the agreement is good. The deviation of the forced-choice point at the top can be explained on the basis of inadequate experimental data for the determination of the high probability involved. The. deviation of the low point is unexplained. Figure 12 is the same plot for the second observer, showing about the same picture. Figure 13 is for the third observer, showing not quite as good a fit, but nevertheless satisfactory for psychological experiments. For this observer, the lowest point for forced-choice is off the graph to the right of the line. Figures 14, 15, and 16 show the predictions for forced-choice data (when yes-no data are used to estimate d') for the three observers. Note that the lowest point is on the curve in both of the first two cases, suggesting that the deviation which appeared on the curves in Figures 11, 12, and 13 is not significant. Discussion with the vast amount of data in the literature, for, when the d' vs. AI function for any one of the observers s used to predict probability of detection as a function of A in terms of I~::/!~.....~

I-l 0 5. — 1- -5 5 - YN * 4 YN Y- FC 4 F FC/ FC YN YN 0-FC 0.9 -----— 04 0.9 09 00'8 0.7 0.7 0.7 YN FC Y N 0.1 - ----- 0. - ------ 0.4 — ----- -- _y - _ __-~ ---. --- -- ^ ^ _ ___or ------- -^ - _ _ ___04 0.1.2 3 4.5.6.7-.8-.9 1.0 -.- 2 3 A0 5.6.7-.8-.9 1.0 -0.5.2.3 4.5.6.7,8 91.0 Figure 11. Log d' vs. Log l for Observer 1. Figure 12. Log d' vs. Log t for Observer 2. Figure 13. Log d' vs. Log A for Observer 3. Figure 11. Log d' vs. Log /xI for Observer 1. Figure 12. Log d' vs. Log AI for Observer 2. Figure 13. Log d' vs. Log AI for Observer 3.

11 this theory, the result closely ap- 1.0 o proximates the type of curve fre- o quently reported. Shapes of curves thus furnish no basis for 0 8 selecting between the two theories and a decision must rest on the other arguments. 0.6 - According to conventional theory, application of the chance P correction should yield corrected 0 4 values of PSN (A) which are independent of PN(A), or should yield / corrected thresholds in the conventional sense which are inde- 0.2 G. 14 pendent of PN(A). Rank-order correlations for the three observers between PN(A) and corrected I thresholds (.30,.71,.67) are 0 I 2 3 4 5 6 highly significant; the combined Figure 14. Prediction of Forced-Choice Data from Yes-No Data for Observer 1. <<.001. This is a result consistent with theory presented here. 1.0'- o Another method of comparison is to fit the scatter diagrams (Figures 9 and 10) by straight 0.8 lines. According to the independence theory, these straight lines should intercept the point (1.00, 1.00). Sampling error would be 0.6 expected to send some of the lines P (C) to either side of this point. There are twelve of these scatter dia- 0 4 grams, and all twelve of these lines intersect the line PsN(A) = 1.00 at values of PN(A) between 0 and 1.00 in an order which 0.2 FIG. 15 would be predicted if these lines were arcs of the curves PsN(A) vs. PN(A) as defined by the theory I I of signal detectability. 0 2 3 4 5 6 d' Two additional sessions were Figure 15. Prediction of Forced-Choice Data from Yes-No Data for Observer 2. run in which the observers were permitted three categories of response (yes, no, and doubtful), in which the observers were told to be sure of being correct if they responded either yes or no. Again two "a priori" probabilities (.8 and.4) were employed, and again PN(A) was correlated with P(SN). The observers, interviewed after these sessions, reported that their "yes" responses were based on "phenomenal" seeing. This does not mean that the observers were abnormal because they hallucinated. It suggests, on the other hand, that phenomenal seeing develops through experience, and is subject to change with experience. Psychological as well as physiological factors are involved. Psychological "set" is a function of 13, and after experience with a given set one begins to see, or not to see, rather automatically. Change the set, and the level of seeing

UNIVERSITY OF MICHIGAN 123 9015 03526 8518 1.0 changes. The experiments re- o ported here were such that the observers learned to adjust rapidly to different sets. 0.8 Conclusions / 0.6 The following conclusions are advanced: 1) The conventional concept 0.4 of a threshold, or a threshold region, needs re-evaluating in the light of this theory. 0 2) The guessing hypothesis is rejected on the basis of statis- tical tests. 0 0 I 2 3 4 5 6 3) Change in neural activity is a power function of change in Figure 16. Prediction of Forced-Choice Data from Yes-No Data for Observer 3. light intensity. 4) The mathematical model of signal detection is applicable to the problems of visual detection. 5) The criterion of seeing depends on psychological as well as physiological factors. In these experiments the observers tended to use optimum criteria. 6) The experimental data support the logical connection between forced-choice and yesno techniques developed by the theory. List of References 1. Blackwell, H. R., Pritchard, B. S., and Ohmart, J. G. Automatic apparatus for stimulus presentation and recording in visual threshold experiments. J. Opt. Soc. Amer., in press. 2. Peterson, W. W., and Birdsall, T. G. The theory of signal detectability. Technical Report No. 13, Electronic Defense Group, University of Michigan.