ENGINEERING RESEARCH INSTITUTE THE UNIVERSITY OF MICHIGAN ANN ARBOR RELATIONSHIP BETWEEN TIME AND ERROR IN JAMMING TESTS Technical Report No. 51 Electronic Defense Group Department of Electrical Engineering By: D. R. Rothschild Approved by: _ _ __//_ A. B. Macnee Project 2262 TASK ORDER NO. EDG-7 CONTRACT NO. DA-36-039 sc-63203 SIGNAL CORPS, DEPARTMENT OF THE ARMY DEPARTMENT OF ARMY PROJECT NO. 3-99-04-042 SIGNAL CORPS PROJECT NO. 194B November, 1957

TABLE OF CONTENTS Page LIST OF ILLUSTRATIONS iii ABSTRACT iv ACKNOWLEDGEMENT v I. THE BASIC PROBLEM 1 1.1 Introduction and Purpose 1 1.2 Basic Property of a Laboratory Test 2 1.3 Variables in a Tactical Situation 2 1.4 Aspects of the Time Variable; Fixed and Sequential Time Tests 4 2. TESTS ON RADAR PRESENTATIONS 6 2.1 Probability of Error as the Primary Dependent Variable 6 2.2 Time and Probability of Error as Dependent Variables 7 3. VOICE COMMUNICATION JAMMING TESTS 8 3.1 Fixed-Time Tests 8 3.2 Sequential Time Tests 9 4. MATHEMATICAL EXAMPLE 10 4.1 General Mathematical Formulation 10 4.2 Specific Hypothetical Example 13 5. CONCLUDING REMARKS 19 APPENDIX 25 REFERENCES 27 DISTRIBUTION LIST 28 ii

LIST OF ILLUSTRATIONS Page Figure 1 General Relationship Between Average Time 12 and Probability Correct Figure 2 Relative Pay-Off, Po/cVT, as a Function of 15 P(C) and P(ip). Value-Cost Ratio as Parameter Figure 3 Optimum Probabilities of Correct and Error as 17 a Function of Value-Cost Ratio Figure 4 Optimum Average Communication Time, t as a 20 Function of J/S. V/K Ratio as Parameter Figure 5 Possible Plot of Jamming Test Results 22 Figure 6 P(C) vs t for Experimental Word Test 26 iii

ABSTRACT In studying the effects of jamming signals directed against communication systems which employ human operators, investigators have employed various laboratory testing methods. It is the thesis of this paper that usually a direct comparison of results obtained by various methods cannot be made. This is true becuase of a general lack of understanding of the basic nature of jamming tests. Some of the important factors which influence the results of different jamming tests are discussed in this report. It is shown that each testing method usually studies only one dimension of a multi-dimensional problem. The paper discusses two of these dimensions, time and probability of error. The role of the time and probability of error variables in a tactical situation is considered. Examples of various laboratory tests for measuring effects of jamming are considered. Included are tests for both radar and voice communication systems employing human observers; both fixed and variable (sequential) time tests are cited in which either probability of error or time is the primary dependent variable, or both. The importance of the values and costs of the particular situation is emphasized. A mathematical example is presented which shows the interplay between the time and probability of error variables as affected by values and costs. A general mathematical formulation is given, and a specific hypothetical example is worked out. Differences between various jamming tests are delineated in terms of the final results, a plot of Jamming Criterion vs. J/S. The way in which results can be influenced by such factors as the nature of the feedback channel and a "maximization of effort" design is considered. A preliminary experimental measurement of the influence of values and costs is given in the appendix. iv

ACKNOWLEDGEMENT The author wishes to thank Mr. T. G. Birdsall of the Electronic Defense Group for the valuable assistance and information which were obtained during private discussions. The comments of Mr. W. P. Tanner, Jr. are also greatly appreciated. or

ENGINEERING RESEARCH INSTITUTE ~ UNIVERSITY OF MICHIGAN RELATIONSHIP BETWEEN TIME AND ERROR IN JAMMING TESTS 1. THE BASIC PROBLEM 1.1 Introduction and Purpose In studying the effects of jamming signals directed against communication systems which employ human operators, investigators have employed various testing procedures. The natural question arises as to how one set of results compares with another, each obtained with a different testing procedure. It is often contended that some comparison between two valid, objective testsl should be 2 possible. However, it is the thesis of this paper that generally such a comparison should not be made. It is felt that there is a general lack of understanding of the basic nature of jamming tests and that each test may be studying only one dimension of a multi-dimensional problem. It is the purpose of this report to point out the various dimensions involved and to discuss two of them in particular, namely, time and probability of error.3 Although the discussion here will consider both radar and voice communication systems in which human 1. The designation "valid, objective tests" should be interpreted here in the same sense as used in reference 1 (p. 14). In particular, the word "objective" is used in contrast to the word "subjective". For the purposes of this paper in general, an "objective" test will imply a forced-choice experiment. 2. The desirability of such a comparison arises frequently in the voice communications jamming field. For example, a comparison of results from an articulation test and the Michigan Map Test (Reference 1). 3. Let P(E) = the probability of an error; then let P(C) = 1 - P(E), the probability of a correct choice.

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN observers1 are employed as the final link or decision mechanism, a stronger emphasis will be placed on voice communication systems as this is where difficulty and controversy have arisen most frequently. 1.2 Basic Property of a Laboratory Test The main purpose of this paper is to assist in the analysis and understanding of results obtained from laboratory jamming tests. Normally, the aim of a laboratory test is to provide answers to a particular field problem with the advantages of being able to control certain conditions and to suitably measure desired quantities. The basic requirement of the laboratory test is that it must be capable of providing measures from which valid predictions can be made in relation to the field situation. Therefore, in order to obtain meaningful measures, it is not necessary to set-up a test in the laboratory which is analogous to a field or tactical situation. In fact, it may be quite desirable to have the laboratory test quite different from the field situation, for any number of obvious reasons. The author and a few associates feel that this point should be strongly emphasized. Just as long as the basic requirement of the test stated above is met, there may be no need in the laboratory to duplicate, or even simulate, the actual field conditions. 1.3 Variables in a Tactical Situation In considering jamming of communication systems there are numerous parameters to examine when attempting to translate laboratory measures to a field situation. Of course there are the usual engineering quantities of power, frequency, propagation, modulation, etc. These will not be discussed here, as the main concern in this paper is an understanding of those factors affecting the In this paper the term "observer" will be used in a general sense to denote the human subject of the experiment, although he may have more to do than just "observe." _____________________________________ -2- ____________________________

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN type of test that can be conducted in the laboratory. Four factors which are basic to the communication process are: signal or message ensemble size, coding, time, and error. All four factors have been discussed in part elsewhere (including references 1 and 2); the significance of the first two has been treated more extensively and is more intuitively apparent perhaps than is the latter two. Consider a signal being transmitted over a communication link (either radar or voice) in a tactical situation in the face ofjamming. Assume that without any jamming present the signal could normally be received (by an operator) within a few seconds. In the presence of jamming, it is easy to conceive of two situations: (1) where it is essential to get the signal (or message) through no matter how long it takes (within a matter perhaps of many hours or even days), and (2) where it is desired to get as much of the signal (or information) through as quickly as possible, the information being of little value if delayed many seconds. Which of these conditions (or even possibly an intermediary condition) arises, depends upon the costs and values of the particular situation. On the basis of these costs and values 1, a decision must be made how to trade most efficiently (in the face of jamming) between time2 and probability of error. 1. Of course in a tactical situation, these costs and values are generally ndt clearly defined quantities, but are a composite of many factors which determine the "urgency" of any individual message. 2. Reference 3, Chapter 5, discusses a number of time factors which must be considered in the evaluation of electronic countermeasures in ground forces applications. These time factors are divided into two categories, depending upon whether one is jamming or being jammed. These factors are: Time Factors in Susceptibility Time Factors in ECM Action Allotment time Preparation time Communication time Reaction time Receiving time Transmission time Processing time Action time l_____________________________ -3- 1____________ — 3__,..

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN 1.4 Aspects of the Time Variable; Fixed and Sequential Time Tests As seen from the above possibilities in a tactical situation, it is possible in the laboratory to simulate any number of situations. Some experiments are designed so that time is held fixed (whether intentionally or not), others so that error probability is fixed (zero being the most likely fixed value), while other tests have both time and error as variables. Laboratory tests can be designed so that time enters in a number of ways. Some of these will become evident in the examples given in the following sections of this paper. However, a clearer understanding of their role can be had if some of the various time factors are first discussed here. Perhaps brief mention should be made of some of the "hidden" time factors, that is, factors which may not be the prime variables and may not be directly measured. These factors could include: signal duration, memory time, reaction time, preparation time, fatigue, etc. Their significance is apparent and some of these are discussed further in one of the examples of Section 2. An important distinction to be made, however, is that between a fixed time test and a sequential time test. In both tests there are two principal time quantities to be considered. One is a time; t, for the transmission of each signal (or message), and the other is a time T during which N signals (or messages) are transmitted. The time T for the transmission of N signals is usually long compared to the time, t, required for the transmission of any one signal. In either a fixed or a sequential time test the time T is a pre-set value; it is known prior to running the test. In practice, the time T can be either a definite termination time for the transmission of the N signals or may be merely determined by the "end of the day", in which one would normally try to run as many tests as possible in the designated working period. ______________________________________ -4- _________________________

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN In a fixed time test, the communication time, t, per signal is also fixed; it is a pre-determined quantity. That is, the transmission of each signal must be completed in a time, t; of course, there can be as many repetitions of the signal as desirable within the allotted time, t. This means that for the fixed time test the average communication time t is identical to the time per signal, t, and they are related to the total overall time T by T = Nt = NT (fixed-time). (1) Thus, N is also a pre-determined quantity. The measure obtained from the fixedtime test is the fraction of transmitted signals (or messages) correctly received in time T, or P(C). In a sequential time test,l the communication time, t,per signal is variable; thus there is actually a distribution of the time variable,t, and only the average communication time t is directly related to the total time T by T = Nt (sequential-time). (2) Here, N is not a pre-determined quantity. The decision to terminate the transmission of one signal (or message) and to proceed to the transmission of another is made by the observer in the process of performing the test. The reasons or procedures for stopping or not stopping the transmission of one signal after a time, t, can be numerous and perhaps somewhat nebulous. The considerations in such a decision may be: (1) a level of confidence; (2) the increase in pay-off expected of further time on that signal as compared to the expected value of time spent on another signal; (3) meeting some established criterion. With time, t, 1. The introductory discussion of sequential tests in this paper is intended only to be intuitive in nature. More formal presentations can be found elsewhere (References 7, 8 and 9). ____ -5

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN per signal variable in a sequential test, the measures obtained from the sequential time test are an average time t and a P(C). The time variability for individual signals permits the observer to spend less time on the "easy" signals and more time on the "difficult" signalsl. As a consequence of this, in general, an observer will receive a larger number of signals correctly in a sequential test than in a fixed time test, for the same average time t per signal. The above discussion has indicated several ways in which time, as well as error, may be an important factor in a particular laboratory test. A complete, yet possibly unattainable, picture can be obtained by collecting a complete set of data, with any one setting of jamming parameters, in which all values of time and error are permitted (by changing the costs and values). In the next two sections, tests on both radar and voice communications will be discussed from the above viewpoint. 2. TESTS ON RADAR PRESENTATIONS Many types of experiments can be conducted to study jamming effects on radar presentations. In many experiments time can be held fixed and quite often only enters the test in a subtle manner. Probability of error is the dependent or measured variable. In other experiments both time and probability of error can be measured variables. Examples of both types of tests will be presented in this section. 2.1 Probability of Error as the Primary Dependent Variable An experiment in which error probability (or more commonly, percentage correct responses) is measured is a four-alternative in space, forced-choice test 1. Of course, even with the same signal-to-noise ratio, the inherent statistical variations of the signals and the jamming will result in both "easy" and "difficult" signals. -6- _

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN on an A-scope. In the presence of jamming, a signal is presented at just one of four positions on the scope face and the observer is required to select the most likely position in which he thinks the signal appeared. After N such trials, the observer is scored on the number of correct detections, thus giving an estimate of the value of P(C). Outwardly, time does not appear in this particular test. Yet, although this is essentially a fixed-time test, time is actually involved in several ways. In addition to the repetition rate and pulse width, the number of successive sweeps on which the signal is present is important as each is related to the signal energy. These factors, as well as the integration obtained by the scope persistance, all contribute to detection ability. Reference 4 reports some results of this nature. Additional time effects are introduced by the length of period between two successive trials. The observer has only a finite time to think, to answer, and to prepare himself for the next trial. Memory time and reaction time may be hidden factors in such an experiment. 2.2 Time and Probability of Error as Dependent Variables In the previous experiment time entered in a very subtle way. A realistic test, in which time is measured directly, has been suggested by Mandel and Clarke for a PPI presentation (Reference 5). On the screen are, say, five targets scattered about and some interference. A sixth target signal is randomly introduoed onto the presentation and the observer is timed in the acquisition of the target. Of course, with interference present, the observer will in general not be able to locate the target correctly each time. There will be a measurable error, either in terms of coordinate distance or incorrect location (exceeding some criterion, such as a radial distance from the true target location). Therefore, from such a test, both a time measure and a probability of error (or correct choice) are obtained. Presumably, the longer the observer takes in trying to track the new -7

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN signal, the greater is the probability of correctly locating the target. Again, the decision at which instant to choose the target location is a matter of trading between time and probability of error and depends upon costs and values, or, perhaps, prior instruction given to the observer, such as a false alarm rate. The above test as described is a sequential time test. The time measure is determined by the observer's decision toectimate the target location. The test could be readily converted to a fixed-time test by merely allotting a fixed-time, say five seconds, for the observer to locate the target. That means that at the end of a five-second interval the observer must select a target location; the measure thus obtained from repeated trials in such a test is an estimate of the probability of correct. 3. VOICE COMMUNICATION JAMMING TESTS 3.1 Fixed-Time Tests The conventional articulation test is a well known example of a fixedtime (per word) test, although there is no effort to maximize the information rate or expected pay-off in the allotted time interval. Perhaps a more descriptive title for the articulation test is a "neglected time test." The matter of time in this test is discussed quite thoroughly in Section 2.3.4 and 2.3.6 of Reference 1. The essential point of that discussion is that the results of the tests can be varied by changing the input symbol rate, that is, the number of words per second presented to the observer. Hence, with a very slow input symbol rate, the conventional articulation test can provide one bound on the probability of error for the particular test situation.l Thus, the usual articulation test is a limiting case for a very restricted communication channel. T. The situation simulated by the articulation test is that of a one-way communication channel with no repetition and no feedback between receiver and transmitter; repetition and feedback information are generally devoted to the reduction or elimination of errors. -8

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN The Michigan Map Test (Reference 1) can be and has been rim as a fixedtime test by simply assigning a definite time period during which the observers must complete the transmission of each route. There may or may not be a feedback channel, and the measure obtained is an estimate of the probability of a correct route. 3.2 Sequential Time Tests Modification of any of the above tests, as well as others, will permit running them as sequential tests in which time is a dependent variable. An articulation test can be modified so as to be run as a sequential time test and to permit a maximization of effort (in accordance with either some bonus system2 or apriori instructions to the observers) within an overall time interval, T. The transmitting operator has a long list of words which would be impossible to transmit completely in the allotted time, say sixty seconds. The observers are instructed to try to send as many words correctly as possible in the sixty second interval. Repetitions are permitted and the receiving observer flashes a light when he desires the transmitting operator to send the next word. A measure obtainable from this experiment is the number of correct choices (words) per unit time, which is equivalent to a probability of a correct word per average word time3. Quite conceivably the results of this test are a function of the overall time interval, T. 1. In some tests, such as those run by Egan (Reference 6,) time enters indirectly in the form of number of repetitions of the transmitted word. 2. A bonus system is essentially a pay-off scheme which defines costs and values, usually in monetary terms. The observers then can be instructed to operate so as to attempt to maximize their pay-off. (See Section 4). 3. As an example, let the overall-time interval T = 60 seconds. Assume that 18 words are transmitted and 13 are correctly received. Then N = 18, Nc 13, and P(C) = 13/18; t 60/18 = average word time; Nc/T = 13/60 = correct choices per unit time, or P(C) _ 13/18 13 -t 60 18 60 -9

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN As another example of a sequential test, consider the Michigan Map Test which employs a light as the feedback channel. Each route consists of six towns and the transmitting operator keeps repeating each town of the route until the receiving observer flashes the light as a command to proceed to the next town. The time for transmission and the correctness of each route is noted. After running a number of maps, sufficient for statistical purpose, an estimate of the average time per route and of the probability of error is obtained. Here again, the confidence level achieved by the observer when he presses the light button depends upon the costs and values, which should be controlled by either a bonus system or instructions. 4. MATHEMATICAL EXAMPLE In order to tie together some of the thoughts expounded in this paper, a hypothetical mathematical example can serve to illustrate the interplay between the time and probability of error variables. First a general formulation will be presented and then a case with a specified relationship between time and P(C) will be treated. The mathematical formulation presented is not intended to be a rigorous treatment but only an initial intuitive-approach to the problem. 4.1 General Mathematical Formulation Consider the case of transmitting N signals, each of which is one out of a set of M orthogonal signals (discussed in Reference 1, Section 2.2.3), in a time period T. A bonus value, V, for receiving a signal correctly and a penalty cost, K, for receiving a signal incorrectly will be assigned. The object in this experiment is to maximize the pay-off Po, in the time T, in the face of jamming interference. The important quantities can be summarized as follows: -10-

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN Parameters M = signal ensemble size T = total transmission time for N signals V = the bonus value for a correct choice (or reception) K = the penalty cost for an incorrect choice (or reception) a = a parameter which depends upon the signal-to-noise ratio, the type of signal and equipment employed, and the particular mode of operation of the observer Variables (measured from test) N = number of signals transmitted in the time T t = average communication time per signal = T/N P(E) = probability of error = 1 - P(C) P(C) = probability of correct = 1 - P(E) PO = the expected pay-off in time T From the above formulation, it is apparent that the pay-off is Po = VN P(C) - KN P(E) = N [VP(C) - KP(E) ]. (3) Because N = T/ (4) and P(C) = 1 - P(E), (5) Equation 1 can be rearranged into Equation 6: = T [ - (V+ K) P(E)] (6) It is reasonable to assume that on the average, for any one signal, the longer the time spent in transmitting the signal, the greater the probability of correctly receiving it. In general, the relationship between average time and probability of correct will not be linear but will exhibit a characteristic similar to that of Figure 1, with a different curve for each value of the parameter a. Note that a as defined above, is a parameter depending upon signal-to-noise ratio, -11- ~

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN 1.00 -- - P(C) I/M t FIG. I. GENERAL RELATIONSHIP BETWEEN AVERAGE TIME AND PROBABILITY CORRECT. signal and equipment types, and the operating procedure of the observer. That is, each curve and value of a of Figure 1 represents a fairly fixed situation as regards to the operational features of the test. If the observer were to change his criterion, or method of decision, the value of a would be different and he would be operating on a different curve on the average. Therefore, different curves would be applicable for fixed time and sequential time tests. P(C) is also dependent on message ensemble size, as well as time and the paramter a. Thus, P(C) = fl t, M, a) (7) and for t = O, there is always chance probability, i.e., P(C)=. f (, M, a). (8) M 1 Because of Equation 5 P(E) - f(T, l) = 1 - fl(T,, a). (9) Substituting Equation 9 into Equation 6 P -= [V - (V+ K)f(t, M, ) ], (10) o T ------- -12

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN which is the pay-off equation. The basic question now is: for how many signals, N, should transmission be attempted in time T so as to maximize the pay-off? Or, in other words, what is the optimum time, T, (or optimum probability of error P(E)) that should, on the average, be spent in attempting to transmit each signal? The answer, of course, is to maximize Po of Equation 10 with respect to time, t, (M and a fixed), or = - [V - (V + K)f(t, M, )] = O (11) t t t Let the value of t that satisfies Equation 11 to be to. Then the maximum pay-off is Po (max) = [V - (V + K)f(:o, M, a) (12)'to 4.2 Specific Hypothetical Example With the basic formulation of Section 4.2 above, it will be instructive to consider a specific example further by hypothesizing the function f(T, M, a). Let a curve of Figure 1 have the equation: (C) = - M- e = fl(t, M, a); (13) (7)2 that is, assume an exponential variation of probability of correct with time, for 1. As Equation 9 is a relation among the four quantities t, M, a, and P(E), and if the inverse function has been found to be t = g [P(E), M, a,], (9a) then the pay-off, P can equally well be maximized with respect to the probability of error, P(E). Equation 9 would then become --- — aPo a fo T — [V-(V+K)P(E)] = 0 (lla) 3P(S) aP(E) g[P(E), M, a)] l Such a relation is employed in the next section. 2. The numbers in the margin along side the equations in this section refer to the more general analogous equations of Section 4.1. -13

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN fixed. c and M. Then P(E) = 1 - P(C) = f(7, M, a) = ME e -a (14) (9) M In many practical cases M>>1. With this assumption, the relation simplifies to P(E) = e<- (15) (9) and P = V [1 - (1 + K/V)e ] (16) (10) S 1 - n (17) (9a) a P(E) it is perhaps simpler algebraically to use poin the form P = VT [ (1 +K/V)P(E)] (18) 0 in z/p(E) A relative pay-off, which depends only on the value-cost ratio, may be defined as Relative Pay-off = 1 - (1 + K/V)P(E) (19) aVT In 1/P(E) This relative pay-off is plotted against P(E) and P(C) on probability paper in Figure 2 for three different value-cost ratios. It is seen that for each valuecost ratio there is an optimum (non-zero) probability of error (and consequently optimum average communication time o by Equation 17) which will yield the greatest pay-off. For example, for a value-cost ratio of unity, the optimum probability of error is 0.19, which means that the average transmission time per signal should be so adjusted that 81% of the signals are correctly received on the average. As expected for V/K = 1.0, it is necessary to receive 50% of the signals correctly at least to break even. For V/K = 0.50, the optimum P(E) is 0.10 and for V/K = 20, the optimum P(E) is 0.707. In this example, the optimum error probability for any value-cost ratio can be obtained mathematically by maximizing P Equation 18, with respect to P(E), aPo_ / VT Jl - (l+K/V)P(E7f 0 (20) (la) |P(E) _P(E) T L In 1/P(E) l -14-

PROBABILITY OF ERROR P(E).9999.999.99.90.50.10.01.001.0001 1.00 0.9o0 l _1 C O - 1 _ - 0.80 a 0.70C- f - __ — _ - v o.6o —------— y0 —-- -" ------- - 0.50 0.30 0.20 - - - 0T.20, 1rY-1 1!-osoX I 1! (1 / ",,, >; I o -1.9Oi ----- --— l - --- ------------ -20.60 _ - 0.70 - - - I -0.80 - 2.10 — 1.30 -.6 —-- -1.80 - 2.20 --- - 2.30 -2.40 o - 2.50 --- - - - - --- - - - - - -- - - - -- -- -- - - - - - - 0.60.0001.001.01.10.50.90.99.999.9999 PROBABILITY OF CORRECT, P(C) FIG. 2 RELATIVE PAY-OFF, P0/aVT, AS A FUNCTION OF P(C) AND P(E). VALUE-COST RATIO AS PARAMETER. [P(E):eT, MI]. 15

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN The result of this operation yields the relation: 1 + K 1 (21) V P(E) [1 + In 1/P(E)] This relation is plotted in Figure 3. Note that the equation and the curve follow intuitive notions. For high value-cost ratios, the optimum error probability increases and, in fact, approaches unity as V/K approaches infinity (or as K/V-+O), as seen by Equation 21. Also, by Equation 17, the optimum communication time per signal then approaches zero.1 Conversely, as the value-cost ratio approaches zero (or unity P(C)), the optimum communication time becomes infinite. By substituting Equation 21 into 19, a convenient relation which gives the optimum relative pay-off for each probability of error is PO1 1. (22) VTj opt. = 1 + in 1/P(E) 1. Actually, V/K need not become infinite for the optimum communication time to be zero, but only as large as M-l. The assumption of very large (or essentially infinite) signal ensemble size, M, leads to an infinite V/K ratio. If no assumption is made with respect to the size of M and the relation between time and error of Equation 14, i.e., P(E) = M-1 e -t (14) M is employed, then the relation between the optimum error probability and any value-cost ratio, similar to Equation 21, can be obtained as 1 + _K _ ----— 1 (21a) v P(E)1 + In M-1 ( MP(E) For t = 0 in Equation 14 P(E) = M-1 (21b) M which is the largest value of probability of error that can be actually obtained, as it is chance probability. Using this value of error in Equation 21a yields a value-cost ratio of V/K = M-1 Therefore, for V/K> M-l, the optimum communication time is zero and the payoff per unit of communication time is infinite. In other words, for this sufficiently high value-cost ratio, the best procedure for the observer to follow is just to employ guesses of the, signal and to use no communication time. -lo- --

(3)d 08I 3 -0 A.LI-8niVS d iflWnllidO rYX 0 LL 0.. _. __ Q 1 -- I'~~~~~~O IQ: I / _ _ _ _ I.-,,, ~o 0 0. o. -- /1 1- - QJ ]./ T s> N I 3 - - O O. 0 (\ ~ d' - - _____ o d 0 ^ ^__y 8 ----- ----— 40 N __/__ —-^ ~ w~~~~~^>1 wy —

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN This relation, although not significant in itself, is shown by the dashed curve of Figure 2. It is merely the locus of the optimum error probabilities and the fact that the relative pay-off increases as the error-probability increases is unimportant. Note that the relative pay-off, Po/aVT, is a function of the actual bonus value, V, as well as a and T. The actual expected pay-off, PO, (in time T) can only be determined (theoretically) when a, V, and T are known, not merely the ratio V/K. The main point to be emphasized by this example is that the values and costs,l which are normally fixed, will determine an optimum utilization of time and error so as to give the most profit or pay-off when transmitting N signals in some (long) time T. This hypothetical example can be pursued one step further by considerirg the relation between average communication time, t, and the J/S (jamming power to target signal power) ratio. It was pointed out above that signal-tonoise ratio (or J/S, equally as well) is one of the quantities upon which a depends. Let us assume, for purposes of illustration, that, in a particular testing situation in which J/S is varied, a will only be influenced by the changes in J/S. Since = 1 In 1/P(E), (17) and it is reasonable that the average communication time, t, will be a montonically increasing function of the J/S ratio, then, for example, assume that 1 = (1 + J/S)2 (23) This yields = (1 + J/S)21n 1/P(E). (24) 1. Although in many cases it may be impossible to assign specific numbers to V and K, there are usually numerous factors which contribute to each and thus, in one respect or another, values and costs determine any decision which is made..-18-,

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN For a fixed value-cost ratio, V/K, it has been shown that there is an optimum error-probability (Figure 3). Therefore, for V/K fixed, the optimum average communication time, to, is only a function of the J/S ratio, i.e., T = (1+ J/s)2 (25) where f = In 1/P(E) (25a) In Figure 4, to is plotted vs J/S for the same three value-cost ratio of the curves of Figure 2 and a segment of a curve for V/K = 0.10. At some criterion level, say to = 50, the difference in J/S ratios between curves may be noted, as is done on Figure 4. A difference in J/S of up to 5.72 db is seen at this criterion level. The important conclusion to be obtained from this hypothetical example is that, under the same conditions of jamming and operating in optimum manner (maximizing pay-off), it was possible to alter the final result by several decibels by merely changing the values and costs.l The example given in this section was, by Equation 13, based on an assumption of an exponential relation between probability of error and average time. Although this was a hypothetical assumption, it is not totally unrealistic. Several of the examples given in Section 2 and 3 could, to a first-order-approximation, be described by an exponential relationship between average time and error. 5. CONCLUDING REMARKS An attempt has been made to demonstrate the importance of the relationship between time and error, as influenced by the values and costs, in tests studying the effects of jamming signals directed against communication systems 1. Recall that it is the experimenter, not the observer, who controls the values and costs, -19

100 Optimum average I communication time, to / as a function of J / S ____ 90 V/ K ratio as parameter I I [P(E) e -t, a=(1+J / S)2], 80 - Z 70, o0. 60 -—,o r. ->..^ 1Criterion 0/ / =-30 level' ~ 20 > / /:,l 4.75db ~40 2 _ 4/6 85.72 db 40 0 2 4 6 8 10 12 14 J / S IN DECIBELS 20

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN which employ human observers at the terminal. The importance of these quantities in both the tactical situation and in the design of laboratory testing programs has been shown. The problem has been formulated mathematically with a specific example for further illustration. In essence the problem can be stated as follows: with fixed values and costs how should time and error-probability per signal be selected so as to maximize the total pay-off in some (long) total time T? A main purpose of this report is to assist in interpreting results from one test as compared to another. This has been done, in part, by exploring and discussing two particular dimensions involved in the jamming problem. However, in various jamming tests, there are certain other, often more subtle, differences which only have been implied here, but which will be delineated and summarized more clearly in this section. These other factors enter the picture because of the relation they bear with both time and error. In order to elucidate the importance of some of the factors, it is best to discuss first the type of comparison that can be made among the end results of several tests. In jamming tests against a communication channel, as defined in this report, one is usually interested in studying either the effect of several different jamming signals of the variation in signal or equipment parameters. Let us confine the discussion here to a comparison of several jamming signals, as this adequately will serve to illustrate the point. Normally the final results can be put into a plot of some criterion of jamming as a function of the J/S, the jamming power to target signal power ratio, such as in Figure 5. In considering results of two or more tests, one should examine not only the ordering of the signals, but also J/S values and, more important, the differences (or A J/S values) at some criterion level. (See Figure 5). ~~___~________________~ ~-21- _____________

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN n'^ l.01* j/-AJ/s IN // CRITERION SJ/S. -A J LEVEL ___m mmmmm -22m - -m m- m - — m m W C*~ J/S IN db FIG. 5. POSSIBLE PLOT OF JAMMING TEST RESULTS. Bearing in mind the notions of Figure 5, some of the other related factors can be considered. Tests vary as to the existance and type of return or feedback channel from receiver to transmitter. The articulation test has no feedback; tests employing a light as a command have a feedback channel with very limited capacity. It is quite possible to set-up a two way communication system with varying degrees of noise in the return channel, including the possibility of a noiseless (or clear-voice for communications) feedback channel having considerable capacity. In general, the use of feedback tends to reduce errors, so that the probability of error can vary markedly between tests, depending upon the nature of the feedback channel. As noted in some of the examples of Sections 2 and 3, and, in particular, the hypothetical mathematical example of Section 4, jamming tests can be designed such that there is an attempt at "maximization of effort" in the process of communication of the signals. Maximization can be accomplished in several ways. -22

ENGINEERING RESEARCH INSTITUTE ~ UNIVERSITY OF MICHIGAN For example, an attempt might be made to transmit the greatest number of correct signals in a given overall time period, or to adjust the average transmission time per signal so as to obtain the greatest pay-off, or to attempt to maximize the information rate. In other tests, such as the articulation test or the fouralternative forced-choice test on an A-scope, there is no apparent attempt at a "maximization of effort".1 For tests having a "maximization of effort" design, there is at least the implication that the observers are trying to "work through", or cope with, the jamming to the best of their ability. Certainly feedback can be a great asset toward this "maximization of effort" and often, when observers are trying to "maximize" or "work through" the jamming, surprising or unusual results may be obtained. The main point to be made here, then, is that tests designed with feedback and a "maximization of effort" intent may show very little jamming or a low value of the jamming criterion, whereas for tests designed withoit such factors may show very considerable jamming or a high value of the jamming criterion for the same J/S value. That is, the design of the experiment may influence considerably "how far into the jamming" the observers "can work" and thus the J/S values obtained at a criterion level. Thus, with a consideration of all of the above factors, it should be evident that in general it is difficult to make a fair comparison of results from different jamming tests. In particular, it has been shown that it is possible in some tests to obtain almost any result desired (for one variable) by suitably adjusting the costs and values (or prior instructions). Factors, other than values 1. Although not a principal topic of'this paper, "maximization of effort" can be affected by variation of the code in the presence of jamming. For example, in an actual voice communication channel it is possible to vary the code in order to fit the channel. In this way something "close" to channel capacity can be achieved with a low error rate. This is a different situation from either the A-scope test or an ordinary laboratory articulation test for which the code cannot be varied, and any maximization must be made with respect to the signal which exists. -23-

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN and costs, which are included in the design of the experiment can have a pronounced effect on end results. All of these may influence the range of the jamming variable involved thereby impairing direct comparisons of test results. This paper has discussed some important factors which should be considered in any comparison of results from different jamming tests. It is hoped that the comments here are not taken to show prejudice toward one type of test over another, but merely an effort to give some understanding and insight into the nature of various laboratory testing procedures and into a more proper interpretation of their results. In an appendix is included some experimental results which demonstrate a few of the ideas of this paper. I___________________________________ -24- _

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN APPENDIX A sequential-time word test (similar to that of section 3.2) was conducted for experimental verification of some of the ideas considered in this report. A long random list of words, chosen from an ensemble of twenty-five (M = 25) was available at the transmitting end of an audio voice communication channel.,White Gaussian noise of 20 kc bandwidth was added to the channel as the interference. A signal-to-noise ratio of approximately -20 decibels was maintained. In a 600 second interval (actually five two-minute periods) the observers were instructed to attempt transmission of as many words as desired so as to maximize their pay-off (or "make as much money as possible") within the allotted time. The receiving observer flashed a light when he desired the transmitting operator to send the next word. Three different value-cost ratios were employed; the results are summarized in Table 1. V/K T (secs.) N P c P(E) (secs.) Po (cents) 0.50 600 122.795.205 4.92 23.5 1.00 600 140.814.186 4.28 44 20.0 600 376.516.484 1.60 92.4 For all cases V = l/*, J/S - 20 db, M = 25 EXPERIMENTAL RESULTS OF WORD TEST TABLE 1 The results show the effect of changing the values and costs although they are probably not as marked as might be anticipated. This may be partly due to the relatively short period that the observers were exposed to this type of test. It would appear that more training might be necessary to make the observers sufficiently flexible in adjusting their mode of operation so as to accomodate a wide range of value-cost ratios. -25

ENGINEERING RESEARCH INSTITUTE * UNIVERSITY OF MICHIGAN From the data of Table 1, a rough curve (4 points) of probability correct P(C) vs. average time per word t can be drawn, with the added information that, for t = 0 p(c) l0 1 2 =o M 25 The curve is given in Figure 6. 1.00 P 0.8 - - --- U ad 0.6 0 u o o |/ FIG. 6 P (C) vs t > 0.4 experimental word test _. / M=25 J/ S: +20db / 0 0.2. 0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 AVERAGE TIME PER WORD, T ----------------------- -26

ENGINEERING RESEARCH INSTITUTE ~ UNIVERSITY OF MICHIGAN REFERENCES 1. Rothschild, D. R., "Laboratory Methods for Evaluation of Effectiveness of Jamming Signals Against Voice Communication Receivers," Technical Report No. 59, Electronic Defense Group, University of Michigan, Ann Arbor, Michigan, 1956. (Secret) 2. Tanner, W. P., Jr., "Method for Evaluating the Potential of a Signal to Jam Voice Communication," Technical Report No. 26, Electronic Defense Group, University of Michigan, Ann Arbor, Michigan, 1954. (Secret) 3. Welch, H. W., Butler, T. W., Jr., Bacon, F. R., Tanner, W. P., Jr., Hamilton, R. W., Norman, R.Z., "Methods for Evaluating Electronic Countermeasures in Ground Forces Applications," Technical Report No!;?, Electronic Defense Group, University of Michigan, Ann Arbor, Michigan, 1953. (Secret) 4. Lawson, J. L., and Uhlenbeck, G. E., Threshold Signals, McGraw-Hill Book Company, Inc., 1950. 5. I.N. Mandel and E. Clarke, Private Communication, Armour Research Foundation, Chicago, Illinois 6. J. P. Egan, F. R. Clarke, and E. C. Carterette, "On a Theory of the Transmission and Confirmation of Messages in Noise," Technical Report Contract No. AF 18(600)-571, Hearing and Communication Laboratory, Indiana University, October 1, 1955. 7. A. Wald, Sequential Analysis., John Wiley and Sons, 1947. 8. H. Blasbalg, "Theory of Sequential Filtering and Its Application to Signal Detection and Classification," Technical Report No. AF-8, Contract No. AF 33(616)-68, The Johns Hopkins University Radiation Laboratory, Baltimore, Maryland, October 18, 1954. 9. Fox, W., "Signal Detectability: A Unified Description of Statistical Methods Employing Fixed and Sequential Observation Processes", Technical Report No. 19, Electronic Defense Group, University of Michigan, Ann Arbor, Michigan, 1953. -27

DISTRIBUTION LIST 1 Copy Document Room Stanford Electronic Laboratories Stanford University Stanford, California 1 Copy Commanding General Army Electronic Proving Ground Fort Huachuca, Arizona Attn: Director, Electronic Warfare Department 1 Copy Chief, Research and Development Division Office of the Chief Signal Officer Department of the Army Washington 25, D. C. Attn: SIGEB 1 Copy Chief, Plans and Operations Division Office of the Chief Signal Officer Washington 25, D. C. Attn: SIGEW 1 Copy Countermeasures Laboratory Gilfillan Brothers, Inc. 1815 Venice Blvd. Los Angeles 6, California 1 Copy Commanding Officer White Sands Signal Copps Agency White Sands Proving Ground Las Cruces, New Mexico Attn: SIGWS-EW 1 Copy Commanding Officer White Sands Signal Corps Agency White Sands Proving Ground Las Cruces, New Mexico Attn: SIGWS-FC 1 Copy Commanding Officer Signal Corps Electronics Research Unit 9560th USASRU Mountain View, California 65 Copies Transportation Officer, SCEL Evans Signal Laboratory Building No. 42, Belmar, New Jersey FOR - SCEL Accountable Officer Inspect at Destination File No. 22824-PH-54-91(1701) 28

1 Copy J. A. Boyd Engineering Research Institute University of Michigan Ann Arbor, Michigan 1 Copy Document Room Willow Run Laboratories University of Michigan Willow Run, Michigan 12 Copies Electronic Defense Group Project File University of Michigan Ann Arbor, Michigan 1 Copy Engineering Research Institute Project File University of Michigan Ann Arbor, Michigan 29

UNIVERSITY OF MICHIGAN III 3illl i lll iI 3 9015 0369.5 6798