TE UNIVERSITY OF MICHIGAN INDUSTRY PROGRAM OF THE COLLEGE OF ENGINEERING.!-. - SURVEY OF OPERATIONS RESEARCH "-; ANMD / Y-T-E4S ENGINEERING, Harr H. Goode Harry H. Goode September 1957 IP-242

i^^ N 1

SURVEY OF OPERATIONS RESEARCH AND SYSTEMS ENGINEERING Harry H. Goode INTRODUCTION In Nuclear Engineering and Computer Engineering, which are also new fields of interest to this meeting, there is apparently no need to devote time to self-identification, as seems to be the case with Operations Research. Contrariwise, as recently as 1954, an Operations Research meeting which I attended devoted half of the meeting time to the question "What is OR?". In view of this uncertainty, it is remarkable that so important a group of educators as this one, concerned with the hard core of engineering education, should devote time to surveying a field whose practicioners keep nervously reassuring themselves of its existence. Still, I believe that the effort is worth undertaking because the OR phenomenon is part of a larger effect which touches all of engineering; because this effect must modify, perhaps profoundly, all engineering education; and because the causes yielding this effect are continuing ones which will lead repeatedly to more, and similar, developments in the future. I have indicated that OR is part of a larger effect. To deal with the latter, at the risk of overlapping my colleagues in the automation and automatic controls area and in the computer area, I have retitled this paper somewhat from the version given in the program. I have inserted "Engineering" in the place of "Analysis" in the title "Survey of Operations Research and Systems Analysis." In the first place, -1

-2systemsl analysis is only half of the analysis-synthesis process which constitutes engineering, and in the second, it has been so misused as to have lost meaning. This may controvert, somewhat, the intention of the session planners, but it will provide the supports necessary to set the subject firmly in place within the engineering structure. On the other hand, widening the boundaries in this manner makes the story somewhat more complicated to tell; but no simple outline would have served in any case. In fact, some of the OR practicioner's uncertainty concerning his existence follows, I believe, from a mistaken notion concerning this origins. But I shall reserve the exposition of my view of these origins until I have developed the main theme of this discussion: the survey of the field. To properly evaluate the material I put forth, you should know that:I consider myself a Systems Engineer) and that I have tried as nearly as possible to adopt a reportorial attitude. OUTLINE OF SURVEY Were I the specialist associated with any other assignment on this program, I should not have spent much time deciding how to cover the subject. State the main problem; give some definitions and develop some of the special jargon; exemplify some of the methods; state some of the frontier problems; say how the field fits into the engineering and scientific structure and give its mainpoints of contact and overlap; tell who has and what has been, accomplished; and guess something of the future. But this won't work with OR, or Systems Engineering. OR does not have a main problem, and that of Systems Engineering is diffuse. Most of the I I have objected, in the past, to this plural on the grounds that we do not say Radars, Mechanicals, Chemicals, Radios (etc.ad.inf.) Engineering but through slovenly speech habits we have taken to the plural in Systems Analysis, Engineering, Design, etc. In this paper I bow to the irresistible force of the human oral torrent.

-3definitions for both are obvious, or borrowed from other fields. The frontier problems for the most part are only slightly associated with each other and in most cases represent a solidly defined area in someone else's discipline. The methods for the most part have origins elsewhere and are still being developed there. And finally, to state the overlap and contact is to cover the entire field. OR and Systems Engineering are, in a sense, defined by overlap and contact with other fields. Like the several blind men and the elephant, the descriptions depend upon the part that is seized, and the elephants of OR and SE have many deformities in addition to all the normal parts. I am forced, therefore, to take another tack. I choose an operational one and ask: What action can a Dean of Engineering take as a result of the information I provide? And what information must he have had access in order to take such an action? Not being a Dean, and not being able to consult you before preparing this paper, I had to guess what actions you might take, and I may have erred in the usual two fashions and I tried to lean to the side of commission. Further, I know that a Dean of Engineering does not take actions; he "encourages" certain tendencies, events, and actions. One should, therefore, read the proper shading into the word at each point that "action" is mentioned. Accordingly, the following possible actions relative to OR and SE are listed and associated with the useful, or required, information. TIE DEAN MAY WANT TO THE INFORMATION REQUIRED IS 1. DO NOTHING WHAT ARE "OR" AND "SE"? 2. DEVELOP A COURSE TECHNICAL CONTENT OF FIELD

-43. ADD A NEW ENGINEERING CURRI- APPLICATIONS OF THE FIELD, CULUM SPECIALTY, OR DEPART- JOB DEMAND FOR STUDENTS MENT 4. ADD A SCHOOL, INCORPORATE OVERLAPS WITH OTHER DISCIPMATERIAL IN AN EXTANT COURSE, LINES DISCONTINUE A COURSE. 5. FORM A COMMITTEE, START AN WHO DOES IT? INSTITUTE, TAKE ON SPONSORED OR UNSPONSORED RESEARCH 6. KNOW WHETHER THE ACTION BE- DIFFICULTY LONGS AT GRADUATE OR UNDERGRADUATE LEVEL 7. HIMSELF BECOME AN OR OR SE WHAT IS THE FUTURE LIKELY WORKER TO BE? As usual this listing is oversimplified for discussion purposes and one or more of the actions may go with one or more of the pieces of information. But roughly, the column of items on the right hand side forms my survey outline. The ecstasies, or horrors, of taking the actions on the left must be delineated by my colleague. I concern myself with the required foundation of information. WHAT ARE THEY? The question of what is OR (and later, what is SE) is a formidable one. Were I to do more than to adopt an ad hoc answer for the purpose of this discussion, I should fall into the difficulty of the OR workers themselves as indicated by this extract from the presidential address in the August 1954 issue of their journal. The speaker is bemoaning the lack of uniformity of understanding of the field and says: "Consider the reaction of the neophyte to operations research when he is presented with the following collection of definitions of operations research: 2 Rinehart, R.F., J. Oper. Res. Soc. Am., Vol. 2, p. 230-231, Aug., 1954.

-5OR IS THE SCIENCE OF DECISION OR IS THE APPLICATION OF THE METHODS OF PHYSICAL SCIENCE TO PROVIDE QUANTITATIVE ANSWERS TO EXECUTIVES WITH REGARD TO OPERATIONS UNDER THEIR CONTROL OR IS QUANTITATIVE COMMON SENSE OR IS WHAT OPERATIONS-RESEARCH WORKERS DO OR IS SIMULTANEOUSLY INDUSTRIAL ENGINEERING, STATISTICS, QUALITY CONTROL, MARKET ANALYSIS, CIVIL ENGINEERING, APPLIED MATHEMATICS, APPLIED PHYSICS, APPLIED PSYCHOLOGY, AND ECONOMETRICS. OR IS AT PRESENT UNDEFINED, BUT IN TIME WILL BECOME DEFINED BY THE SUBJECT MATTER APPEARING IN JORSA. Each of these definitions has been written or stated by a member of ORSA". To this list, I add one which appears later in the Journal stating that "OR is a philosophy." As a non-operations researcher who feels certain that he has practiced it at one time or another in certain studies, I adopt for the purpose of this discussion the view that it consists of an attitude toward the solution of mankind's living problems which requires the employment of methodology of the physical scientist; using whatever tools are required from any field yielding them, certain ones turning out to be used more frequently than others; with the provisions that, because this type of problem solving generally requires groups of workers, the methodology must be made explicit so that the group will be a team.* I have neither fear nor hope that my definition will fail to vanish with this discussion. Systems Engineering does not have, and most of us feel that it does not require, a journal. There are some quasi-popular magazines published for profit, but these are not intended as journals of the * It is interesting to note that this exposition of method is a step which is usually undertaken by physical scientists in the later years of their lives.

-6discipline. However, I have talked with enough Systems Engineers to have collected a set of definitions similar to the ones above on OR. The difference in stating them will be that I am not at all disturbed by the varied opinions but I am impressed by the many impulses to define it.' These definitions are: SE DOES NOT EXIST IS PART OF "OR" IS "OR" IS COMMON SENSE ENGINEERING IS GOOD ENGINEERING (Apparently not the same). IS AN ATTITUDE REQUIRING EITHER/OR THE OVERALL VIEW, SKEPTICISM, THE MATHEMATICAL APPROACH, INGENUITY IS THE SEPARATION OF THE FUNCTIONS OF ANALYSIS AND SYSTHESIS IS TEAM EFFORT IS THE SCIENTIFIC METHOD APPLIED TO ENGINEERING PROBLEMS IS THE POLITICAL ACUMEN WHICH GETS THE PROBLEM TO THE ATTENTION OF PEOPLE AT THE PROPER LEVEL IS THE USE OF NEW TOOLS LIKE INFORMATION THEORY, HUMAN ENGINEERING, SERVOMECHANISM THEORY, ETC. IS THE NEW TOOLS OF INFORMATION THEORY, HUMAN ENGINEERING, SERVOMECHANISM THEORY, ETC. IS A NEW SCIENCE As a tentative base of operations, which I will enlarge upon later, I will assume that Systems Engineering came about, as did Operations Research, due to the increasing complexity of the problems we face in living with a rapidly expanding population, yielding a requirement for a complex approach to these problems. I will further assume that these problems arising in the operation of already existing complex

-7assemblies of things and people, leading to suggested changes of method or procedure -- the study method being used to discover the required changes, is operations research; further, it is frequently necessary to design new complexes of things and people to yield new assemblies of equipments -- the study method then being used is Systems Engineering. I believe it was Warren Weaver who said that dealing with a number of things two orders of magnitude greater than before induces a new science. If that is so, then OR and SE are new sciences. However, it is no matter whether OR and SE are new sciences or not: they exist, and are being used. Each of them is threaded on the same methodological skeleton; each uses any tool that turns out to be applicable. The problems of OR are so unlikely to have common parts that most of the subject deals with the analysis of the problem. Indeed, OR once had an alias: Operations Analysis. Equipments on the other hand, as the groupings of them get larger, tend to exhibit common characteristics among the collections chosen as solutions. Therefore, more attention can be paid to solution in Systems Engineering and synthesis enjoys discussion almost as much as analysis. Such discussion clusters about the parts likely to appear in the system, and the methodology will therefore exhibit these parts. Thus to make more explicit what OR and SE are, I must turn to methodology, and subsequently to the tools employed. And of course the discussion of the latter will fulfill the requirement of our survey outline for technical content. Before doing so, however, I cannot resist telling you of my mathematician friend with whom I was discussing the teaching of Systems Engineering. "Oh, Systems Engineering", he said, "I taught that last year. The first semester I taught Probability and Statistics, and the second the Operational Calculus."

-8METHODOLOGY As I indicated above, the need for an explicit statement of methodology follows from the fact that OR and SE workers pursue their professions in teams. It may be true that as Bridgeman3 says, "Science is what scientists do, and their are as many scientific methods as there are individual scientists", but if a group is to carry out the solution of a problem as a team, there had better be some common methodology. For this methodology, team workers have turned to a distillation of method from the physical sciences. And Bridgeman notwithstanding, in reviewing the Journal of the Operations Research Society of America I have found several expositions of such a method with a surprising amount of uniformity among them. Moreover, Systems Engineers seem to arrive at similar outlines of method. I do not guarantee the independence of all these attempts. It suffices that they share a large common ground. In the expositions and block diagrams for methodology which immediately follow, you may accuse me of using the techniques of the maker of women's foundations garments who forces a given amount of material into a preconceived form. However, in their case as in mine, this cannot be done with an arbitrary form. The material must roughly approximate the form being used. Moreover, in both cases it makes things quite a bit easier to look at. For OR methodology, I have chosen to paraphrase parts of a 4 paper by M. L. Hurni which, it seemed to me in a review of all the articles appearing in the OR journal since its inception, best described 3 Bridgman, P. W., Reflections of a Physicist, p. 83, Philosophical Library, New York, 1955. 4 Hurni, M. L., Observations on Operations Research, J. Op. Res. Soc. Am., August, 1954, Vol. 2, pp. 234-248.

-9what OR workers intend to do in carrying out their method. According to Hurni, the basic processes of OR consist of three major phases each of which involves a number of steps. For our purpose today, enough of the content as is necessary can be gathered from a mere listing of these phases and steps: PHASES STEPS I. JUDGMENT 1. DEFINING A FRAME OF REFERENCE 2. FINDING THE OPERATIONAL CHARACTERISTICS APPROPRIATE TO THE SITUATION 3. FEEDING BACK ACQUIRED INFORMATION II. RESEARCH AND SYNTHESIS 1. DETERMING THE METHODS AND UNITS OF MEASUREMENT 2. BUILDING THE CONCEPTUAL MODEL 3. USING MATHEMATICS 4. TESTING OF ASSUMPTIONS 5. MAKING THE MODEL UNDERSTOOD BY OTHERS 6. CLASSIFICATION OF ACTION ALTERNATIVES III. ACTION ALL ONE STEP. SELL THE ANSWER. More or less, most of the explicit statements of the process have the same content with different emphases, with slight additions or omissions. Some stress heavily the research features of the method, some the quantitative aspect, some the experimental approach, some the required objectivity; but with a little molding they all go together. There is usually a lack of emphasis on solutions and ways of finding alternatives, and heavy accent on analysis. For Systems Engineering, the problem always involves equipments, and the larger the number of peoples and equipments which have to fit

-10together, the more talk about Systems Engineering. The method may be practiced at almost any level of design from the vacuum tube up to an air traffic control system, or the entire defense network, but it is at the latter end of the spectrum that the employment of conscious method becomes essential. Further, as indicated above, large systems tend to have parts which are common such as communication networks, computers for logical centers, sensing organs such as radars or telephone instruments, and effector organs such as cutting tools or control surfaces. As a result, any methodological discussion of Systems Engineering must take account of these common elements. The large system contains many parts each of which constitutes a sub-system, almost as complicated as the original system. Further, the engineering process involves a series of approximations to the final answer. For example, in electrical engineering,their appropriations proceed through the laboratory breadboard, the development breadboard, the development prototype, the production prototype, etc. In chemical engineering, the answers pass through the laboratory process, the experimental setup, the pilot plant, the semi-works plant, etc. It is not to be expected, therefore, that any representation of the Systems Engineering process which is usually more complicated than the examples quoted, will yield a series of discrete phases. In the diagram below it is to be assumed that each of the steps shown will be gone through many times, with modification of the resulting output until a final phase is reached. Without further introduction, the following methodological diagram5 is exhibited representing the logical association of the steps 5 Goode, H.H. and Machol, R.E. System Engineering, McGraw-Hill, N.Y, 1957, p. 39.

-11in Systems Engineering and the system parts, the latter to be considered those steps in the design associated with the engineering of the sub-subsystems designated. INTERIOR EXTERIOR INTERIOR Statement of I problem - Single-thread Communication' ~ l design Analysis _ Mathematical high-traffic control Critical aspet ofi design t tequipment Experimental experiments s Cm desdeig nrn reeps t tReflexive design - control Field experiments Handling equipment Outputs I Figure 1. Steps in System Design. In this diagram, the words are not as self-explanatory as in the case of the OR outline and without going too deeply, the following explanations are offered: "exterior" is to be associated with problem statement and analysis, with the affairs exterior to the system, with the things beyond the control of the engineer. "interior" refers to the system itself, the solutions or syntheses, the things within the control of the engineer. "single-thread" refers to one of the things that the system is to do many of, as with a single telephone call in a telephone system, or a single interception of a bomber by a fighter in a defense system. Similarly, "high-traffic" designates the manyness aspect of the system; the 55 million telephones, the thousands of airplanes, the many weapons in terms of the classes and the members of a class. Competitive design refers to the special design aspects of systems whose inputs are intended to confound or destroy them. "logical control" refers to decision-making mechanisms at the center of a complex system

-12and "reflexive control" to the mechanisms at the periphery which respond on a reflexive basis -- i.e. without consulting the system center. "handling equipments" are mechanizations associated with the movement of physical objects such as airplanes in a transport system, or conveyors and feeders in automatic production systems. In this diagram there is much more emphasis on solution, much more synthesis, much more implication of ingenuity and inventiveness, along with all the analysis. In Systems Engineering teams it frequently develops that the analysts and the synthesists, to coin a word, are different people and complement one another. DEVELOPMENT OF METHOD There is little time here to go into the steps in these methods in any detail. But it would be remiss to fail to give some indication of the content of one or two of these steps, at least briefly. In all of the methodological approaches there is implied or explicitly stated an emphasis on the careful choice of a "measure of effectiveness". In the SE diagram it is in the statement of the problem, and in the OR list, in the discussion of measurement. In component engineering this concept is covered by the term "figure of merit". Now it usually turns out in component work that the figure of merit in a component is obvious or falls naturally out of analysis. Such quantities as "hardness", or "Q", or "time constant" do not usually involve deep concepts. In any event, if someone other than the maker of the original definition were to have done the work, he would probably arrive at something proportional to the presently used figure of merit. Even so, the figure of merit must frequently be supplemented by additional characteristics such as the effect of weight, or bulk, or

-13scarcity, etc. As the process or system under examination goes up in size, it becomes more and more equivocal as to what should be used as a measure of effectiveness, and fewer and fewer people arrive at the same conclusion. Moreover, the systems we are discussing include human beings, and the picture gets fuzzy. From whose viewpoint shall we design the Air Traffic System, or the Auto Traffic System, or even an automatic factory? What is a good result in a defense system becomes not at all clear as one examines how much a proposed system subtracts effort from all other systems, or how it interferes with the efforts of the State Department, or the Commerce Department, or the people on the South Sea Islands where bombs are tested. Evaluation depends on viewpoint. These difficulties have lead to investigations of the general nature of such questions which, while they have been thus far carried out by philosophers or economists or mathematicians, have been given impetus by the needs of OR and SE and have lead to at least one special term: suboptimization. The nonexistence of absolutes from which to start makes unresolved questions of this nature useful to any critic of any proposed system solution, no matter what the motivation of the critic. A portion of the literature of OR, as we shall see, is devoted to such questions. Another procedural step acknowledged to be important by OR and SE workers is the creation of a conceptual model, usually mathematical, for the process under study. These models have the same purpose, and structure, as in engineering and science. The mathematics useful for creating them and taught formally at undergraduate level has been almost entirely of the analytic-deterministic variety. It is almost the same in most graduate curricula. Yet, a large fraction of the problems

dealt with by the OR and SE worker are of the probablistic or numerical type, in various combinations. The types that can occur are displayed by example in this diagram: DETERMINISTIC STOCHASTIC gm mk F= Ma P(k) = k! May represent probability of ANALYTIC K particles reaching a counter in unit time-average arrival rate = m Oi NUMERICAL NUMERICAL MONTE CARLO INTEGRATION INTEGRATION Points are chosen to fall with equal likelihood anywhere in area - fraction of points below curve is estimate of 1 Jf(x)dx 0 Figure 2. ypes of Mathematical Models Used in Eniiineering

-15For the measured quantities of interest, the variances in the problems dealt with by OR and SE are a much larger fraction of their corresponding mean than is usually the case in engineering and science, generally. It is to be expected, therefore, that problems requiring stochastic considerations will occur much more frequently in the former. Further, the difficulty of getting analytic expressions which describe the behavior of various quantities leads the OR and SE worker (as it does the chemical engineer) to empirical expressions with great frequency. The high speed computer has made it easier to deal numerically (as opposed to analytically) with many of the.problems of manupulation in these instances. Thus, whereas the distribution of education for engineering mathematics may be somewhat as on the left in the diagram of Fig. 3, the requirement as indicated by the frequency of occurence of various types of models in practice is more like the distribution indicated on the right. The numbers represent my opinion. DETERMINISTIC STOCHASTIC DETERMINISTIC STOCHASTIC ANALYTIC 980% 1% ANALYTIC 55 % 25 % NUMERICAL 1% 0% NUMERICAL 15 %/ 5 % AS TAUGHT AS THEY OCCUR IN OR AND SE Figure 3. Distribution of Mathematical Model Types Certainly it is true that the OR and SE people have introduced an awareness of the requirement for a quantitative approach which for these problems is new in intensity, if not in kind. We are not surprised

-16then to find in the literature of OR and SE an emphasis on stochastic processes and probability generally, on the uses of computers, and on the combination of the two-the study of a problem which is carried on numerically but introduces quantities subject to probabilistic variation - which goes by the name of Monte Carlo. Even more strongly, the OR and SE workers have stressed the model itself as a subject of study. We shall find in the literature survey below a considerable emphasis in this area. We shall also not be unprepared to find mathematicians interested in, and helping to develop, both disciplines. In SE, we find evolving generalizations peculiar to the interest in large sets of equipments. When should the input to a large system first be standardized? At the periphery (as with the dial telephone which turns words into electrical pulses at the input), after entry into the system (as with the planning in the automatic postoffice which will encode letter addresses after the latter have entered the system), or in the logical control center (as with most business data processing systems where the information is coded just prior to manipulation in the central computing element). Again, what is the best choice of sub-system? Should the breakdown be functional as with the choice of communications, data processing and sensory apparatus for subsystems; or should it be geographical as with the choice of subsystems in air traffic control in New York, Chicago, etc; or shbuld it be vertical in the organizational structure as with the development of a data processing system at various echelons in the army structure. Each has advantages and disadvantages, but SE is beginning to provide experience and explicit discussion on which to base choices in particular cases. We find the Systems Engineer oriented to

-17equipments, and how they fit together. These discussions illustrate part of the developing methodology in both OR and SE. We have repeatedly referred to other sciences, and technologies, in this discussion. As I noted above, in a sense the overlap defines OR and SE. We shall expect the same borrowing, comingling, and dual identity for people and things as we discuss the technical content of the OR and SE disciplines. TECHNICAL CONTENT A Systems Engineer will use any tool that happens to turn out to be useful. Quite frequently,tools associated with electrical engineering turn out to be important. In the propulsion of missiles, chemical engineering and aeronautical engineering provide the substance and he will borrow from them. This does not mean that he practices these engineering disciplines, but it does mean that he will be required to understand many of their techniques. He will serve as interpreter among workers in various fields. This implies that he must know the languages of many but it does not require that he be a grammarian in any. As I have said, the Systems Engineer will use any tool, but certain tools, some of recent development, are turning out to be handmaiden to the practice in this field. In the figure below I have affixed some of these tools at the points on the methodological diagram at which they appear to be. of greatest value. Systems Engineering is proding development in these areas, and the workers in these areas are furthering understanding of the systems approach. So great is this interchange that one of the definitions above results: Systems Engineering is Information Theory, or Human Engineering, etc.

-18( Cybernetics o).yn omics EXTERIOR INTERIOR t System logic Inputs Statement' — ".... Information) of problem ~ theory ~1 ) Single-thread Communication -- oMathemmoaticoal I SnQueueg |_hog' A_ experim ents r~ design model I j_' theory_.., -. con —-- I,, i ~. ^~^ ~1 ~~~" ^ ~ -^ Servomechanism,\ i 1 Probabilit >-~ " Compeittive H J Field design~ Figure 4. Some of the Tools Associated With Systems Engineering6 It would be pleasant now to undertake a leisurely discussion of the content of each of these fields. As a Systems Engineer, I am not expert in any of them in the sense of being a frontier worker, but I feel sure I could give you a reasonably- understandable account of the central problem, the methods, the language, the relation to other fields, and is concerned with a problem which arises in any system whose handling of an input is slow enough to permit a new input to arrive which to "get on line" (form a queue) or "go away". The situation may be remedied by increasing system speed of response, providing parallel system handling capacity, or allowing new arrivals to pile up in some fashion (buffer storage), or be lost. What is best done will depend on the 6 Goode and Machol, loc. cit., p. 42

-19on the queue discipline (priorities, first come-first serve, etc.), the time distribution governing the arrival of inputs, the time distribution governing the handling of inputs (the service distribution), the number of service channels offered, etc. The discipline discards all knowledge of what is being done, and pays attention only to the time it takes to do it. It therefore proves most useful in the high-traffic aspects of systems. It has been applied in telephone systems, air traffic systems, service systems such as reservations, maintainance systems, and in weapons systems. Its frontier problems are concerned with channels serving a line with a changing average arrival rate (or with the average service time changing), with channels cascaded (the output of one being the input of the next), with paralleled channels. All of these problems are being attacked analytically. Methods of attacking the problem with computers, numerically and stochastically i.e., Monte Carlo fashion, are also under study. This is hardly a leisurely discussion but it has already absorbed too much of the available space. To do this for each tool would require far more than the present paper. It should be sufficient to indicate that this tool is not the major province of the Systems Engineer (one is more likely to find a paper on it in the journals of the statisticians than in the engineering journals), but it is an important part of the equipment of the Systems Engineer. For each of the tools indicated a similar discussion, and a similar qualification could be made. But it can be stated with reasonable certainty that the statistician is not interested, day-by-day in the developing theory of the servomechanism engineer (nor the reverse), while the Systems Engineer is in both.

-20In OR is not nearly so easy to assign particular tools to particular steps in methodology, since each problem is different, in contradiistinction to system design problems which have the parts as common elements. But since the methodology of the two disciplines is partially common, we must expect many of the tools to be common. Particularly those tools associated with problem statement and analysis will prove to be common. So it is that the OR worker stresses probability and statistics, computers, the mathematics of distribution of effort. And since he works with systems already in existence, some of the common tools applicable to system design such as game theory, queueing theory and human engineering will be of interest. Finally, since he is usually interested in keeping the system operating after design, the mathematics of programming, linear or noh-linear, or dynamic, will be given a special emphasis it does not receive in SE. To determine the content of the field of OR, I took a hint from one of the definitions given in the list above and reviewed the content of the Journal of the Operations Research Society of America from its inception in 1952 to its June 1957 issue. I reviewed in all, well over 400 papers which were either reproduced in the Journal in full, or abstracted prior to presentation at a National or Annual meeting. I tabulated these papers as to subject matter and source of authorship. Sometimes I ascribed a paper to two subjects and sometimes credited authorship to a single source twice when several authors were from the same organization. Since I would like to rough out for you the general outlines of the field, and since this was a concentrated personal endeavor over a relatively short period, the exactitude of each assignment of a paper down to hairsplitting accuracy of definition is not vouchsafed. Moreover, not all papers were contributed

-21Special sessions with a prechosen label, and invited papers, can force the content of the journal. Thus, OR is interested in attacking medical problems, a session is arranged yielding four papers on medical OR. However, I will underwrite the resulting picture set forth as being well beyond any accuracy required for action which may be taken with regard to the education of engineers. I have said that the field of Operations Research is mainly characterized by both methodology and borrowed tools, applied to the problems of living. The literature shows just such a split among the papers. DISTRIBUTION OF SUBJECT MATTER IN "OR" PAPERS No. Percent METHODOLOGY (INCLUDES HISTORY, WHAT IS "OR"? RELATIONS TO OTHER FIELDS, EXHORTATION, ETC.) 74 16 TECHNIQUES (MOSTLY SOME FORM OF MATH AS DETAILED LATER, A FEW IN PSYCH, ECONOMICS) 178 38 APPLICATIONS (BUS, SOCIAL PROBLEMS, ETC.) 104 22 PARTICULAR BUSINESS OPERATIONS, NOT REFERRED TO A SINGLE FIRM OR INDUSTRY (INVENTORY, SCHEDULING, ACCOUNTING, ETC.) 98 21 TRAINING AND ORGANIZATION FOR "OR" 20 4 474 101 This breakdown is enlightening to me. I must admit that as I read through the journal in review, I vacillated between the opinion that the operations researcher had rewritten the scientific book of Genesis, and the diametrically opposed opinion that OR was a new science. But its emphasis on methodology (16~/o - are other professions anywhere near this?), and the variety of its tools, bear out my impression that it is defined by overlap. OR gathers unto itself what is common to all

-22the scientific disciplines, and maintains tentacles, containing feeding tubes, buried in the many scientific and engineering endeavors. While one does not expect to find an article in the OR journal which presents a novel technical development for the first time (the scientific worker is likely to publish in his field of origin first), the publication of the technique in the OR journal may give it a special twist. Thus if accounting is discussed, it is with the idea of doing it by sampling techniques; if statistical sampling is discussed, it is embedded in a methodological discussion. It was noted before that OR has emphasized the need for study of the measure of effectiveness, or more basically of the concept of value, in order to properly solve problems involving human beings. 19 of the 74 articles on methodology were in this area. To further examine technical content it is well to break down the 178 articles concerned with technique. BREAKDOWN OF TECHNIQUE ARTICLES IN "OR" JOURNAL NUMBER PERCENT PROGRAMMING (LINEAR, DYNAMIC, WHY I LIKE PROGRAMMING, ETC.) 40 22 SIMULATION (INCLUDES MONTE CARLO, AND WAR GAMING) 35 20 QUEUEING THEORY 22 12 STATISTICS (WHICH WOULD BE ELIGIBLE FOR SOME STATISTICAL JOURNAL) 21 12 COMPUTING OR COMPUTERS 12 7 GAME THEORY (AND DECISION THEORY) 15 8 OPTIMUM DISTRIBUTION OF EFFORT AND MISC. OTHER MODELS 28 16 NON-MATH SCIENCES 5 3 178 100

-23Few of these techniques, it will be noted, have a basis which can be taught at the undergraduate level. All have aspects easily understood by undergraduates. APPLICATIONS The applications of SE will have been filled in to a large extent by my colleagues in discussing Automatic Control and Automation, and those discussing Computers. But in case they have prepared no organized presentation of the applications of Systems Engineering, I have gathered such a list. In order to give it a semblance of order, I have classified the systems in the following fashion: That system which has a multiplicity of possible inputs, whose order of arrival is not at the disposal of the system designer, will carry the designation M. Obviously a system which has a single type of input, such as a peanut vending machine, has its input ordered at the disposal of the designer. But even a complicated chemical process logging system, which has a large number of possible inputs, has its inputs ordered with regard to type by the designer. A system whose inputs arrive at instants in time which are not chosen by the designer will be designated S. Automatic factories are not such systems. A system whose inputs seek its confusion, or destruction, will be designated C. Finally the systems will be classified roughly according to speed of required response: min, sec, ms, microsec.: M,S,C, ms: MISSILE SYSTEMS; NIKE, BOMARC, IRBM, ICBM CONTINENTAL AIR DEFENSE; SAGE B 58, BOMBING AIRPLANE SYSTEM MISSILE MASTER, ANTI-AIRCRAFT CONTROL SYSTEM

-24M,S, micros AIRCRAFT FLIGHT SIMULATORS AND TRAINERS M,S, ms TELERAN, GROUND-CONTROLLING AIR TRAFFIC SYSTEM TELEPHONE SWITCHING SYSTEM, ELECTRONIC EXCHANGE AUTOMATIC POST OFFICE RAILROAD SWITCHING SYSTEM M,S, sec TRAIN AND AIRLINE RESERVATION SYSTEMS AIRPORT TIME UTILIZATION EQUIPMENT LYONS ELECTRONIC OFFICE CONNECTING UP BAKE SHOPS ON-LINE SALES RECORDING AND ACCOUNTING ERMA-AUTOMATIC BANKING SYSTEM AUTOMATIC WAREHOUSE FOR LEVER BROTHERS GOODRICH FOOTWEAR AND FLOORING AUTOMATIC INVENTORY TRANSACTION SYSTEM, CITY TO CITY, EP HUTTON (STOCKS) BILLING, INVENTORY, CONTROL AND ORDER PICKING, WHOLESALE DRUG M,S, min NAVY, ARMY AND AIR FORCE LOGISTIC SYSTEMS POWER ALLOCATION SYSTEM, KANSAS CITY POWER AND LIGHT CONTINENTAL PIPE LINE MICROWAVE CONTROL SYSTEM AUTOMATIC MAINTAINANCE, TELEPHONE SYSTEM S, ms TORONTO STOCK EXCHANGE TRANSACTION BOARD S, sec FREIGHT CAR ACCOUNT AND DISPATCHING, SOUTHERN PACIFIC AUTOMOTIVE TRAFFIC CONTROL IN DENVER AND BALTIMORE RADIOSONDE SYSTEM, AUTOMATIC METEOROLOGICAL SYSTEM WESTINGHOUSE AUTOMATIC ELEVATOR TRAFFIC SYSTEM AUTOMATIC RACETRACK TOTE AND BETTING BOARD S, min AUTOMATIC STEEL ROLLING MILL CONTROL M, micros VANGUARD-SATELLITE INSTRUMENTATION SYSTEM ANALOG-DIGITAL COMBINATION SIMULATORS M, ms ARNOLD ENGINEERING DEVELOPMENT CENTER, WIND-TUNNEL DATA REDUCTION SYSTEM M, sec AUTOMATIC MESSAGE ACCOUNTING SYSTEM FOR TELEPHONE SYSTEM M, min CENTRALIZED ACCOUNTING SYSTEM, G.E., LOUISVILLE ms BLIND LANDING SYSTEM AUTOMATIC COMPONENT ASSEMBLY SYSTEM MODULAR DESIGN-AUTOMATIC ELECTRONIC MANUFACTURING SYSTEM AUTOMATIC MILLING MACHINE AUTOMATIC LOGGING SYSTEM FOR PROCESS INDUSTRIES sec AUTOMATIC TELEVISION STATION OPERATING SYSTEM AUTOMATIC CONCRETE PREPARATION PLANT

-25In OR, the applications are a by-product of the tabulation mentioned earlier. It is interesting to note that the reporting of applications takes two forms: papers which discuss the solution of a problem in a particular firm or industry, and those which discuss a particular business function without reference to any particular company or industry. In other fields, we may usually characterize the state of affairs by thinking of a problem which awaits the arrival of an investigator. But in OR one gets the feeling that the investigator is out searching for a problem. Consider the following varied list which results from a breakdown of the 104 papers noted above as being concerned with applications: BREAKDOWN OF APPLICATION PAPERS IN PARTICULAR FIELDS FIELDS OF APPLICATION NUMBER PERCENT TRANSPORTATION (AUTO, SHIP, PLANE, TRAIN, BARGES) 27 26 MILITARY (ARMY, AIR FORCE, AMPH., CIVIL DEFENSE) 23 22 MANUFACTURING (CHEM., WATCH, CLOTHING, WIRE, STEEL, OIL, ELECTRICAL SUPPLY) 13 12 PUBLIC SERVICE (FIRE, WATER, CENSUS, GARBAGE, ETC.) 8 8 DESIGN (COMPUTERS, AIRPLANES) 7 7 MEDICINE AND BIOLOGY 7 7 COMMUNICATIONS (TELEPHONE, TELEVISION, PUBL.) 5 5 SOCIAL PROBLEMS (SCHOOL PLANNING, NEGRO MANPOWER, ATHLETIC GAMES, CONSUMER ACTION) 5 5 FINANCE, INSURANCE, STOCK MARKET 5 5 ODD (MINING, AGRICULTURE, FOOD DISTRIBUTION) 4 4 104 101

-26The occurrence of a paper by no means insures the successful attack on a problem. In fact, all that is reported in the list above is subject to a degradation factor which must be applied to take account of overstatement, hindsight, and misstatement. But the list of applications is certainly large. It is difficult to evaluate the outcome of these studies. But then it is in other fields, too, until the result is overwhelming. Perhaps this will be true of OR. It has been indicated that another way in which OR tackles application is in the study of particular business functions. The breakdown of the 98 papers devoted to these functions is as follows: BREAKDOWN OF 98 PAPERS WHICH STUDY BUSINESS FUNCTIONS WITHOUT REFERENCE TO A PARTICULAR FIRM OR INDUSTRY FUNCTION NUMBER PERCENT PLANNING (ALSO ORGANIZATION, PLANT LAYOUT, BUDGETING, REPLACEMENT THEORY) 9 9 SALES (USE OF EFFORT, ADVERTISING, MARKETING, FORECAST, BUYING AND SELLING) 9 9 FINANCE (PR CING, ACCOUNTING, COSTING, BIDDING) 19 19 OPERATING FOR PRODUCTION (INVENTORY, CONTROL, STANDARDS, MAINTAINANCE, SHIPPING, INSPECTION, TESTING, PLANT TRAFFIC) 53 54 PERSONNEL 3 3 DESIGN (RELIABILITY, WEAPON SYSTEM) 5 5 98 99 Certain of these functions have received a larger share of attention than others. Inventory has 25 papers associated with it. Production control has 18. It should be noted that inventory has been the subject of study for a good many years, even to the extent of mathematical modelling. Whitin7 has taken the trouble to point this out in an 7 Whitin, T. M., Erich Schneider's Inventory Control Analysis, JORSA, vol. 2, 329-334.

-27article devoted to the purpose. Even so, OR has certainly been the discipline practiced by the agricultural scientist" who solved the problem of uncertainty with regard to crop maturation times in the peagrowing industry. The impetus given to considerations of the effect of statistical uncertainty, and methods for taking account of it in operations, must be credited to OR. OVERLAPS WITH OTHER FIELDS It was clear in the lists of definitions that some think that SE and OR are not fields in their own right. A cynical anecdote has it that five percent have always thought that the other ninety-five percent didn't know much and in every age have adopted some title and gone around telling the others. But surely it is a contribution to put together many things to make a new whole. Whether a new discipline is so brought into being remains an open question, but my vote is in the affirmative. The Systems Engineer, is of course, a practicioner in research and development. He encounters and overlaps every form of engineering and every form of physics. He overlaps the OR worker in that part of his methodology associated with problem statement, and again when the system is already in operation but may need replacement. He overlaps the makers of new tools in information theory, in servocontrol systems, in psychology. He overlaps the project manager in the leadership, teamwork, and corrdination aspects of his job. And he overlaps the manager in the money, planning and human relations aspects of running a team to complete a given large-scale system design. Is his a descipline different from the rest? Is the telephone system the less a system because it is made up of components each of which has a name and an identifiable set of characteristics of its own? 8 Thornthwaite, C.W., OR in Agriculture, JORSA, Vol. 1, pp 33-38

-28The OR man does not bump into the components engineer, but he does collide with the man who makes or uses special tools; the statistician and the computer operator, the economic analyst and the industrial engineer. The latter is probably as close to what should have been an operations researcher in an earlier day as can be imagined, but he chose to concern himself with the narrow aspects of a particular phase of business endeavor - production techniques, and mostly with wage incentives and work standards, at that. There have also been skirmishes between managers and OR workers. Some silly statements have obscured in argument that usefulness of the OR man. It has even been written that we stand on the verge of Automatic Management -- that not a human will need to intervene to manage the operation of a complete industrial mechanism. These statements should not obscure reality. The hard-pressed manager hardly ever tackles a problem in a cold, objective, fashion. More than likely, the problem arises, and not understanding but suggestions for solution are sought. With such a demand, solutions are suggested. It then remains to sell the manager on the suggested solution. If sold, promotion. If not, try another solution. The good OR man offers an objective attempt to understand. He offers not a fire-fighting mechanism, but an attempt to understand the causes of fire. Finally, in OR overlaps, an event which must be noted is the formation, in 1954, of the Institute of Management Sciences. The origins of this organization are obscurely buried in the minds of the few individuals who formed it, the published versions of its beginnings notwithstanding. The reasons probably were complex including among them: a) the fact that the ORSA in its early stages was dominated by military operations researchers, b) a certain snobbishness among unfulfilled mathematicians which makes them

-29apply as an epithet the title "engineer" to the operations researcher, and c) an expressed sentiment among some of the founders that the purpose of the management scientist is to understand, not to apply (economists?). Whatever the cause, the following quote is of interest because it appeared in the February 1957 issue of the JORSA in a letter to the editor, calling for a merger of ORSA and TIMS: "On October 18 and 19, TIMS held its Third Annual Meeting in Los Angeles. On November 15 and 16 in San Francisco, a few hundred miles away, ORSA is holding its Tenth National Meeting. If the covers on the two programs were switched and, once in a while, the terms "Operations Research" and "Management Science" exchanged in the session and paper titles, no one would be able to tell the difference -- from content, format, or the names of those giving papers. Of the 35 or 40 organizations represented in each program, 16 are common to both. The president of ORSA was coauthor of a paper at the TIMS meeting. The TIMS Program Committee Chairman was on the Program Committee for the 1955 ORSA meeting in Los Angeles. Six persons appear on both programs as chairman or speakers. There was some grumbling among interested people unable to attend both meetings." He goes on to show that the journals differ little in paper content and the organizations differ little in membership. He continues, "...writes in Management Science, implying that OR is more engineering application to practical problems than it is research, that OR is problem-oriented whereas MS is 9 Lathrop, O. B. JORSA, Vol. 5, p 123.

-30knowledge-oriented. It has been suggested that management science embraces a much wider field -- theories of organization, of communications within groups, of "decisioning", of utility -- and that OR is a smaller part of management science. These possible distinctions, though worthy, are not yet reflected in the activities of the two Societies, nor are they generally recognized or accepted." Others have been aware of this call for science which would abstract the particular. In other scientific areas -- mathematics, physics, biology, -- the event has usually occurred after there existed a particular to abstract. However, the longing is, according to Royl0, understandable. He says of this claim to a new science. "In these arguments one may detect some measure of defensiveness on the part of operations researchers who desire academic ties and academic respectability. Per chance there is also a measure of intellectual snobbishness on the part of academicians who disdain contact with mundane action affairs." WHO DOES IT? Systems Engineering as a calling follows rather naturally with practicioners from other engineerings whose problems are embedded deeply in large systems. Thus the computer engineer is a natural to evolve into a Systems Engineer because of the appearance of the digital computer at the heart of many large systems. Indeed, the electronics engineer in any form is susceptible. Thus, the servo engineer, the communications engineer and the radar (or other sensing equipment) engineer find themselves drawn to the study of the entire system and all of its ramifications. Aeronautical engineers come in through the missiles and aircraft appearing 10 Boy, R.H., OR and Industrial Engrg: Contrast and Resemblance, Scientific Monthly, Sept. 1953, p 162

-31in war and transportation systems. Mechanical engineers enter occasionally through the servo mechanism, or the handling equipment. Chemical engineers enter via the process industries. Although industrial engineers would seem naturals for the purpose, their training is usually at such a low technical level that they fail in their attempts to meet the heavy technical challenge. All of these engineers enter by way of solution. But the mathematician, the physicist, and the psychologist frequently find themselves drawn into the melee by way of assignment to the clarification of some problem associated with a large system. And the rapidity with which they realize that the problem is different qualitatively, as well as quantitatively, from those previously encountered is testimony to the existence of a discipline. The fact that these Systems Engineers come from a specialized discipline does not imply that they remain specialists. Depending upon aptitude, they generalize more and more until it is sometimes difficult to identify the original background without a direct question. And yet there is hardly a Systems Engineer who does not hold membership in some specialized engineering, or scientific, society. This is perhaps a clue to the difficulty of putting ones finger on the Systems Engineering profession. The Operations Researcher is perhaps worse off in this respect. At least the Systems Engineers come from a scientific or engineering background. For OR, it is possible to join in conversation, work with a team, and even write a paper, without a technical background. All sorts of occupations give rise to Operations Researchers: problem solvers, tool makers, teachers, promotors and money makers. This leads to the type of worry about the entry of charlantans to the field illustrated by the

-32words of a retiring president2: "We have tried to have our membership committee serve as the professional standards committee. This has not been too effective. Perhaps it is the wrong way to tackle the problem. It may not protect the good name of the field against the unscrupulous and the unqualified." As with SE, Operation Researchers come from every field and in most cases retain their double identity. Physicists, mathematicians, statisticians, economists, biologists, agricultural researchers, historians and social science majors; some of each are specialists in their fields identifying themselves with OR, rather than OR workers with an interest in the specialty. It may perhaps give insight to examine the distribution of organizations employing the people who write the papers in the JORSA: BREAKDOWN OF SOURCES OF AUTHORSHIP OF JORSA PAPERS, INCEPTION TO PRESENT TYPE OF ORIGIN NUMBER PRECENT LEADERS NUMBER UNIVERSITIES 128 30 MIT 24 CASE INST. 18 COLUMBIA 17 NEXT LOWER 6 or 7 MILITARY OR AFFILIATE 126 29 ORO 39 RAND 29 OEG 20 MILITARY PROPER (A/F, ARMY,NAVY) 37 RESEARCH INSTITUTE 13 3 SRI 6 CONSULTANTS, ETC. 40 9 A.D. LITTLE 14 BOOZ-ALLEN-HAM. 4 BUSINESS AIRCRAFT 31 7 LOCKHEED 11 BOEING 4 Ibid Rinehart, R. F., J. Oper. Res. Soc. Am., Vol. 2, p. 230-231, Aug.,1954.

-33TYPE OF ORIGIN NUMBER PERCENT LEADERS NUMBER ELECTRONIC 42 10 G.E. 9 MELPAR 9 R-W 5 IBM 4 OTHER BUSINESS 26 6 FOOD 2 FOREIGN 15 3 BRITISH 7 CANADIAN 6 NON-MILITARY FED. GOV. 8 2 CENSUS 3 LOCAL GOVERNMENT 7 2 N.Y.PORT AUTH. 4 436 101 Notice that while the electronics and aircraft industries furnish a large fraction of the industrial authorship, it is not necessarily their own business that they are writing about as shown by the subject breakdown above. Moreover, in many cases the paper was obviously solicited. As one reads, it becomes evident that many people are reporting what they have been doing, and how they have been doing things for a long time prior to the invitation to write a paper in which they now assumed the role of Operations Researcher. A. D. Little has been doing OR for a long time, just as Bell Laboratories has been practicing Systems Engineering for a good many decades, without announcing it. This does not deny the existence of something different in OR and SE. It merely says that the needs were exhibited in some places earlier than others. And this brings us to the question of how it all came about. FORCES - PAST AND FUTURE Most Operations Researchers ascribe their origins, mistakenly I believe, to the war and the rise of the profession as OR techniques were used to solve military problems. However, I believe it came about through the increasing complexity of existence and the consequent needs for more sophisticated methods of solving the problems of living. The war is

-34responsible only insofar as it accelerated this growth of complexity, and insofar as it itself was an example of a complex living mechanism. To see this we must look at the increasing complexity, which may be measured by the rate at which man impinges upon man. Indicators are the increases in total population, the transportation speeds, and the communication volume. The estimated population of the globe has doubled once every hundred years for the past two hundred. In the U.S. a birth rate of 0.025 and a death rate of 0.010 yields an incremental population rate of 0.015 corresponding to a doubling of population every 46 years. The population per square mile in this country has increased from 17 to 24 and subsequently to 50 in the successive 100 year periods. Moreover, the urban population has increased from 17 percent of the total in 1850 to 63 percent in 1950. Transportation speeds have gone from 40 to 350 mph in 100 years and reasonably soon, this rate will be 1000. Telephones have increased from 1 to 55 million in 50 years. Productivity and power show the same picture. And the detailed histories do not show linear rises, but present curves which in every case do not contradict a hypothesis of exponential growth. This increasing complexity has led to increasingly complex problems. While solving the problems of his day da Vinci found it possible to encompass both science and art. But by 1800 science had separated from art and had itself broken down into physics, chemistry and mathematics. By 1900, civil engineering had not only broken off from the military, but had itself split into civil, mechanical and electrical, just as physics had broken into optics, mechanics and electricity. And so it went, until we have today an infrared spectroscopy, blood chemistry and servomechanism engineering, to name a few

-35specializations. Sooner or later with the problems getting more complex, some way had to be found to bridge the gap between specialists and we have witnessed the birth of the hybrids: physical chemistry, information theory, and human engineering to name a few. But these did not offer the required coordination for the use of many sciences in solving a single problem. To build a complex system such as a telephone system requires dozens of different specialists: engineers and scientists of every calling. And as these give rise to new systems based on complex technology, the operations of those systems, whether they be industrial, business, government or military, grows more complex and requires the coordinated efforts of many sciences. The pressure for a new method and a new concept grew greater. I believe that the method is embodied in the team approach and that the concept called for the generalist who talks many scientific languages. He is embodied in the Operations Researcher in the operational problem, and the Systems Engineer in the design problem. The pressure of the future is always underestimated. Greenwalt1 states: "When anyone in the past has attempted to predict the long term future, the forecast has turned out to be hopelessly shortsighted and pessimistic. Benjamin Franklin, for example, thought at the time of his life that it would perhaps take centuries to settle the American continent. The state of California stands as a monument to his error. Thomas Jefferson announcing the Louisiana Purchase felt that the territory might be fully occupied after twenty-five 11 Greenwalt, C.H., Challenge Magazine, March 1954

-36generations. The railroad and the steamboat opened it up to settlement within a few decades. In the early 1900's a gentleman from Philadelphia grew enthusiastic about private motor cars and foresaw the time when there would be a hundred or so in every city. He viewed this development as a boon to highway safety as it would free the country from drunken drivers and wild horses. The British socialists during the 19th century foresaw the period when it would be unnecessary for children to work more than 10 hours a day. I could go on with similar examples -- it should be evident that the progress of genius and inventiveness is something that is always underrated. All this follows from the growth rates examined above. From this discussion you have your answer about jobs, and the future. If further evidence is needed, the membership of ORSA has run April, 1953, 560; April, 1954, 892; April, 1956, 1204; December, 1956, 1383. Presumably, for the Systems Engineer, with no single professional society, the growth must be about the same The requirements for technical workers in these areas is part of the larger problem of the request for more technical workers in all areas. How you will provide the required members for these (or any other technical discipline for that matter), I do not know. It seems to me that we do a lot of proselytizing for the engineering professions these days, but there may be some doubt that the converts we make are truly engineers. Consider the following: human intelligence is distributed in some fashion. It does not matter how, but only that the distribution is relatively unchanging. Suppose that, as of now, the number of jobs in our civilization

-37requiring various levels of intelligence is precisely matched by the present distribution of intelligence of various sorts in the population. Since our complex living solutions are forcing the required percentages at higher intelligence levels upward rapidly, the balance is being upset due to the relatively unchanging nature of the populations intelligence distribution. If the two distributions are not yet matched, as assumed, the job requirement rate will soon overtake the intelligence distribution as indicated by the great rate of increasing complexity. Does talking youngsters into being engineers because of the opportunities presented solve the requirement for engineers? But of course, this is your problem, and that of the next speaker.

UNIVERSITY OF MICHIGAN 11111 11111111 1111 111 11111 111111111111 3 9015 03127 3298