Bureau of Business Research Graduate School of Business Administration University of Michigan December, 1970 vw BUREAU OF BUSINESS RESEARCH WORKING PAPER NO. 18 OPTIMIZATION IN COMPLEX MANAGEMENT SYSTEMS by W. Allen Spivey Professor of Statistics University of Michigan FOR DISCUSSION PURPOSES ONLY None of this material is to be quoted or reproduced without the express permission of the Bureau of Business Research I

BACKGROUND OF THIS PAPER This paper was given before the Centennial Meeting of The American Fisheries Society, New York City, September 14, 1970. It will be published in The Transactions of 7the Ameican Fisheries Society in 1971. A

Optimization in Complex Management Systems 1. Introduction. Optimization models have provided powerful tools of analysis in the fields comprehended by the words "management science;" they have been used successfully in wide classes of problems arising in industry, the physical sciences, and in governmental activities. An IBM publication recently listed twenty pages of applications, with an average of forty-three citations per page, which varied from archaelogy, anthropology, avionics, through medicine and nuclear physics to water pollution control in public utilities. Since fisheries management and other renewable resource management problems have much in common with management problems found outside these fields, it is reasonable to infer that optimization models will find increasing application in the fields that interest this audience. Indeed, several interesting studies in the field of environmental quality management have been published within the last four years which make use of optimization models. One of these, examined in detail below, is a water pollution study which uses a linear programming model to approximate a differential equation system describing the water quality of streams in terms of dissolved oxygen profile.

2 It thus seems appropriate to examine what optimization models are, to assess the present "state of the arts" with respect to them, and to consider the advantages and disadvantages that we have learned they have -- oftentimes "the hard way." This paper also presents a bibliography of items selected so as to indicate beginning, intermediate, and advanced level works on optimization as well as selected papers in applications. When reference is made to an item in the bibliography, the name of the author or authors will be indicated, followed by the number of the corresponding entry. 2. Optimization Models. An optimization model is one in which there is a function of decision variables which is to be optimized (i.e., either maximized or minimized). The decision variables may or may not be subject to constraints; if the former, the model is one of constrained optimization and we call the latter an unconstrained optimization model. Most optimization problems in the management sciences have constraints in them so that constrained optimization models have undergone extensive development in the last twenty years.* The function to be optimized is called an objective function. If the objective function is linear and each of fthe constraints is a linear equality or inequality, then the model is one of linear optimization, most frequently referred to as *The theoretical distinction between unconstrained and constrained optimization is not sharp; many constrained problems are best solved by transforming them into unconstrained problems in higher dimensional spaces.

3 -- 3 - linear programming (Dantzig [4], Gass [6], Hadley [7], Spivey [15], Spivey and Thrall [18]). An example of such a model is: (1) minimize z = c11 + + Cnxn subject to lXl + -** + alnn = b (2) aplx + *-* + apn = b p11 pnn p (3) x. > 0 (j = i, -, n), where the cj, aij, and b. are assumed to be known constants:1 and p and n are any positive integers. The x. are the decision variables and (2) and (3) express the constraints on the variables. If at least one of the variables in the objective function (1) is nonlinear or if at least one of the constraints (2) or (3) has a decision variable that is nonlinear (for example, 2 if we should have x2) the model becomes one of nonlinear optimization or of nonlinear programming (Boot [2], Hadley [10], Wilde and Beightler [23]). Moreover, if at least one of the variables in (1), (2) or (3) is a function of time, the model is then one of dynamic programming (see Wagner [22], Hadley [10]). Sometimes an optimization model is developed in which the values of some of the variables must be chosen in one time period and then choices for still other variables are subse quently made which depend on the decisions made earlier as well as other factors. These are multistage decision models and are

- 4 - regarded by some writers as special cases of dynamic programming (Dantzig [3], Spivey [16]). It should surprise no one that our ability to solve linear optimization problems is great. General theorems and solution procedures are known and in the management sciences linear optimization problems in which the number p of constraints is as large as 1,000 to 2,000 and the number n of decision variables is as large as 3,000 to 4,000 are fairly common and are solved routinely on large scale computers. Problems have been solved in which p and n are as large as 30,000 and 3,000,000 respectively. However, models of this kind can be solved only if they possess special properties which permit them to be broken down or decomposed into a sequence of smaller problems which can be solved. Our ability to manipulate and solve nonlinear optimization problems is limited. Nonlinear problems which are most readily solved are those for which linear approximations can be effectively utilized. In constrast to linear optimization problems, a nonlinear optimization problem may be regarded as a "large" one computationally if there are twelve decision variables and ten constraints. Fairly simple nonlinear problems have been developed for which an optimal solution is known to exist but which cannot be solved at all given the present state of our knowledge (dynamic programming problems oftentimes are nonlinear and solution procedures are available only for certain special classes of problems).

-5 - Sometimes an optimization problem will give rise to a model like (1), (2) and (3) above but which will have, in addition, a restriction that one or more of the decision variables be an integer. For example, we may have a model in which the decision variables x. represent the decision to invest or not to invest in a given water resources development project. Thus x. takes on the integer values of 0 and 1 only -- we.J either do not or we do undertake the project. A model giving us fractional values for x. is clearly inappropriate, nor do we "get out of the box" by "rounding" such a solution to the "nearest" integer, since such an integer is either 0 or 11 An interesting example of such a decision model is found in Spruill [19]. This class of problems is called - quite naturally — integer programming; it turns out that these problems are nonlinear too, so our ability to deal with models of this class is limited. Research in integer programming is very active, however, and there is a considerable literature (see Balinski [1]). We indicated earlier that the a.., the c. and b. are assumed to be known constants in the linear optimization model (1)-(3). Since in most applications of optimization models we do not really know the values of these "givens" exactly as the theory requires, we are led to inquire into the effects upon an optimal solution and the optimal value of the objective function (1) caused by changes in these givens. In many models, changing the givens corresponds to examining the effect of bad

6 data, since the alternate values of the givens can be regarded as alternate data points. It is perhaps the most significant feature of linear optimization models that not only can one quickly assess the nature of changes in givens, one can study the effects of continuous changes in them over ranges of values and one can calculate alternate optimal solutions where they are indicated. Letting the givens (the aij, bi, and cj) vary is called parametric programming because the givens then become parameters (Dantzig [4], Spivey and Thrall [18]). Parametric programming procedures can be used to determine the givens to which the model is sensitive and those to which it is insensitive. When a model contains a large number of variables, it is very useful to know that the model is relatively insensitive to certain givens. Among other things this indicates to the model builder the elements of the model that he should gather more data about and the elements that for the time being can be ignored. Moreover, the sensitivity studies made possible by parametric programming can be used as a basis for model refinement and development. One tries to model the sensitive elements more effectively and reduces or eliminates altogether the role played by the elements to which the model has "low sensitivity." 3. An Illustrative Exampe. major problem in water *This is a simplified discussion of concepts appearing in Loucks, ReVelle, and Lynn [14].

7 - pollution control is the release of organic wastes into streams. These wastes, called bio-degradable wastes, serve as food for many organisms in the streams, which in turn utilize dissolved oxygen. As organisms multiply, the dissolved oxygen may be depleted below a minimum level necessary to support a reasonable ecological balance in the streams. If there exists only a single source of pollution on a stream, the amount of waste that can be released without violating the stream quality standard (measured in terms of dissolved oxygen) can be determined by methods known to sanitary engineers. The problem becomes more complex, however, when there are two or more sources of pollution, since the wastes can mix to pollute areas downstream from both points. The capacity of a stream to assimilate bio-degradable wastes (and to this extent purify itself) is determined by such factors as stream flow, stream temperature, the waste concentration measured in terms of its biochemical oxygen demand, the dissolved oxygen concentration, and the physical and biological properties of the stream that affect settling rates, as well as other factors. We develop a model that can be applied to a variety of river basins with minimal alteration of the basic model. The latter is thus developed in modular form such that any river system can be examined by providing the appropriate number of modules or sections with the corresponding features modelled appropriately. The model we develop can be used to determine D

rP - 8 the least cost combination of waste water treatment facilities required in order to meet any set of dissolved oxygen standards. Furthermore, by means of parametric programming we can explore the sensitivity of the system cost to various dissolved oxygen control policies as well as the sensitivity of the model to changes in a variety of other factors. Let x. = number of gallons of organic wastes entering ] the stream in section or module j, j=l, **', n; R. = total number of gallons of organic wastes generated in section j, Cj = cost of removing one gallon of waste in section j. For each section j of the stream the waste removal cost is given by Cj(Rj - xj), so that the objective of minimizing the total waste removal cost over all n sections of the stream can be represented as n (4) minimize z = ~ C (R-xj). j=l j J The water quality index w. in any section is a measure of oxyJ gen level, quantity of pollutants, etc., for the section, and can be expressed as (5) w. = wj + kjx J J J where w. = water quality index at beginning of section j, J k. = rate of index change in section j per gallon of pollutant.

- 9 - For any section j there is a maximum allowable value for the water quality index set by governmental authorities, denoted by max w.. Thus for any j we have ] < max (6) wj w. To insure satisfactory assimilation of solid wastes we must have, as a result of extensive studies by sanitary engineers, (7) x.,9R (j = 1,, n), and various technological requirements place a lower bound on each x. of the following kind: (8) x. _.1R. (j = 1,, n). Thus for any n sections of a stream the linear programming model has the objective function (4) and the constraints (5), (6), (7) and (8). For a specific numerical example of this model, suppose we have a stream with a total of five sections and such that pollution sources are indicated as follows: 0 W o=0 (1) 0 W =W +W w=O 0 W= (0 (3) > ~w-"w3+wq ~/ w(4) (5) w=O W4 I

- 10 - The quality indices at the beginning of sections (1), (2) and (4) are zero, indicating no pollution at the beginning of these sections; the indices at the beginning of sections (3) and (5) are equal to the sum of the indices of the sections merging at these points respectively. Hence pollution can enter sections 3 and 5 depending upon conditions occurring in the tributary sections. Suppose further that sanitary engineers have studied the five sections of the stream and that the following data are available: max Section j R. (in gallons) C. (in dollars) k. max *__ ____ J -].a — - - 1.10,000 $1.00.1 900 2 2,000 1.50.2 800 3 4,000 4.00.2 300 4 6,000 1.00.2 200 5 3,000 8.00.3 700 The optimization model is then: (9) minimize z 59,000 - l.Ox - 1.5x2 - 4O.0.x4 - 8.0x 4.

- 11 -subject to x1 < 9,000 x1 > 1000 x2 < 1,800 x2 = 200 x3 < 3,600 (10) x3 = 400 x4 < 1,000 x4 600 4 = x5 2 2,700 x ' 300 <.lxl+.2x2+ 2x3 7 300 1x+.2x2+ 2x3+.2x4+.3x5 3 700 x. - 0 forall j. The problem (9) and (10) is a special case of the problem (1), (2), (3) introduced above and is a linear programming model. A few comments on how the constraints (10) were developed from the information above may be helpful. From the constraints (7) and (8) we have, when j = 1, x1 _.9R1 =.9(10,000) = 9,000 and x _.1R1 =.1(10,000) = 1,000. 1

- 12 - For the third constraint from the last in (10), we utilize the information provided in the diagram above as well as the numer0 0 ical data. From the former we see that wl = 0, w2 =0, 0 w3 = w + w2, where from (5) we have w = 7 + lx1 =. lx, 3 1 X - L' 0 w2 =2 +.2x.2x2, so that 0 w3 = w1 + w2 =.lxL +.2x2. Finally, from the constraint (6) and information in the table we have w~ <wmax =300 and the constraint becomes.lx1 +.2x2 < 300 as desired. A similar argument will indicate the form of the next to last inequality in (10). Solving the problem (9) and (10) results in an optimal solution in which x = 1,000 x4 = 600 x2 = 200 x5 = 1,200 x3 = 400 minimum cost = $45,900. This means that we would obtain a minimum system cost of $45,900 by having 1,000 gallons of organic waste entering section 1, 200 entering in section 2, 400 gallons in section 3, 600 gallons in section 4 and 1,200 gallons in section 5.

- 13 -If we parameterized this model we could examine the effects of -- changing the R. figures, -- changing the constants k., -- changing the costs Cj, -- changing the maximum allowable standards wma. ] Thus a wide range of alternative engineering possibilities could be examined as well as a wide range of alternative control decisions by governmental authorities. 4. Handling Goals in Optimization Models. The objective function in an optimization model can represent the goals in the problems and the numbers c. represents the weights that J are to be associated with the corresponding decision variables. We can, through parametric programming, regard these weights as variable, we can change them and study the effects upon optimal solutions. We can also examine "trade-offs" between goals by this approach if we can suitably model or specify the goals. An example is provided by goal programming (Spivey and Tamura [17]) in which one begins with a linear econometric model in reduced form (11) y* = Rx + s, where y* is an n by 1 vector of goal or target variables prescribed by a policy maker, x is an m by 1 vector of instrument variables,

- 14 -R is an n by m (real) matrix of impact multipliers, s is an n by 1 vector of constant or additive terms. The coefficients of system (11) can be, and oftentimes are, estimated from data. An example of a system of constraints which was developed from regression studies is given by Van Dyne [20]. A goal programming model can be developed with the traditional simultaneous equation system (11) as a base in the following way (this is taken from Spivey and Tamura [17]). We reformulate (11) as (12) y*- s = Rx - Iz+ + Itz x, z+, z- 0, where R is given in (11), I is an n by n identity matrix, and z+ and z- are n by 1 (unknown) deviation or discrepancy vectors. The ith components z+ of z+ and zT of z- measure the -'. -1 1 deviation upwards and downwards, respectively, of feasible yi values from the corresponding goal values y*; in other words, 1 the ith constraint of (12) can be written (13) s Rx + z (i = 1, -, n), where R. denotes the ith row vector of the matrix R and s. the.th i component of the vector s. If the policy maker can find a feasible instrument vector x for which both z+ and z7 in (13) are zero, then the goal 1 1 yi can be attained exactly. On the other hand, if there is no Ii

- 15 - feasible vector x for which zt = zT = 0 in (13) he cannot attain 1 1 the prespecified goal y. He can, however, find a feasible x that will allow him to come "as close as possible" to y*. The model of goal programming then is (14) minimize G = u z+ + vTZ subject to (15) y* - s = Rx - Iz+ + Iz Z+ > of x z, Z = 0, where I is an n by n identity matrix, T T u,v are nonnegative vectors representing weighting factors, z+, z are deviation vectors, It turns out that there are exactly three possibilities in an optimal solution to this problem: (i) zt = z- = 0, (ii) zt > 0, z = 0, (iii) zt = 0, zT > 0. Case (i) means that a solution has been found which permits the th goal y* to be attained exactly, and cases (ii) and (iii) indicate that a deviation, upwards and downwards respectively, must be accepted by the decision maker using the model. Moreover, if a policy maker desires to attain a given goal y* more than others he can attach larger weights u. and v. to the corresponding deviations zt and z, respectively. If he 1 1

I i I -16 -would accept an overattainment of y*, but cannot tolerate any underattainment, he could set u. = 0, but would make v. large J J enough to prevent zi from assuming a positive value in an optimal solution. Not only can we discover how close we can approach the prescribed goals y4, by using parametric programming on this optimization model we can parameterize the weighting vectors T T u and v to investigate how closely some goals Y. can be approached while holding other goal attainment levels fixed. We can also parameterize the goal vector y* itself. 5. Some Extensions. In some problems it is not satisfactory from an applied point of view to vary the given data parametrically; it may be more appealing to regard the given as a random variable having a known probability distribution or density function. When at least one of the ai., the bi, or the c. is a random variable, then the model (1), (2), and (3) becomes one of stochastic programming (Dantzig [4], Chap. 25; Dantzig [3]; Spivey [16]). This type of optimization model has disclosed analytical complexities of two basic types: the choice and treatment of optimization criteria and the time at which the random elements are to be observed and a decision is to be made. For example, if some bi is a random variable, then the value of the objective function becomes a random variable as well. It is then meaningless to speak of minimizing or maximizing z. We must instead adopt a criterion such as minimizing the expected value of z or

- 17 -minimizing the variance of z around some specified value. Moreover, if z is linear as in (1) but there are random elements in the problem and we choose to minimize the expected value of z, then it has been shown that the expected value of z need not be linear in the decision variables x. (a simple example appears -1 in Dantzig [3]). Although stochastic programming models are natural and appealing, there is little as yet that is operational and most of the work going on in the field can be regarded as basic research of the kind that precedes operational results by perhaps several years at least. Figure 1 presents a schematic representation of optimization models of the kind we have been discussing and it can be regarded as a visual summary of the many remarks made above. 6. Problems in Using and Interpreting Optimization Models. In order to develop an optimization model, one must choose a mathematical representation for the objective function. In order to do this in a given application it is necessary to know or to be able to agree upon what it is that "one wants to do." Specifically, one must know how the decision variables x. 1 are related (whether they are additive as in the linear case or multiplicative as in the nonlinear case, etc.) and what numerical weight each variable is to be assigned. It is often very difficult for practical persons to supply the analyst with sufficiently meaningful information to enable the latter to develop a suitable objective function. The practical man often does not wish to )l

p. % "CERTAINTY" APPROACH I" LINEAR PROGRAMMING (1) IS A LINEAR FUNCTION AND (2) ARE LINEAR 1l \ CONTINUOUS DISCRETE CASE CASE GENERAL OPTIMAL SOLUTION SOLUTION PROCEDURES MUST BE IN AVAILABLE INTEGERS 4 SOME SOLN. PROC. AVAILABLE -4 NONLINEAR PROGRAMMING EITHER (1) OR (2) COR BOTH] HAVE AT LEAST ONE NONLINEAR TERM CONTINUOUS DISCRETE CASE CASE, 4/ SOME FEW SOLUTION SOLUTION PROCEDURES PROCEDURES AVAILABLE AVAILABLE — rIF (1) IS GENE-RAL AND (2) ARE GENERAL EQUAL-! IITIES, THIS IS THE LAGRANGIAN CASE L, _ ~ _ DYNAMIC PROGRAMMING EITHER (1) OR (2) OR BOTH HAVE TERMS THAT DEPEND ON TIME V N CONTINUOUS DISCRETE CCASE CASE \ / SOME SOLUTION PROCEDURES 'AVAILABLE OPTIMIZE A FUNCTION (1) OF CERTAIN VARIABLES [i.e., MAX OR MIN] 4SU SUBJECT TO INEQUALITY OR EQUALITY CONSTRAINTS (2) ON VARIABLES I I I 00 _ I YN p. I I I 1 ~- --- I ~I \ l ----j — I -, -_. NON- PROBABILISTIC —1 PARA R / PROGRAMING SENSITIVITY ANALYSIS & HEURISTIC APPROXIMATION APPROXIMATIC COMES FROM THESE FIELDS [SOMETIMES] 'S If UNCERTAINTY APPROACH STOCHASTIC PROGRAMMING: EITHER (1) OR (2) OR BOTH HAVE "TERMS" WHICH ARE RANDOM VARIABLES HAVING KNOWN PROBABILITY DISTRIBUTIONS OR DENSITIES LINEAR / NONLINEAR DYNAMIC CERTAINTY EQUIVALENCE NEW ] DECISION CRITERIA' i FIGURE 1. Overview of Optimization Models

- 19 - specify which variable is the "more important" -- which should receive a large weight in the objective function. Many times the difficulty of the analyst is the reverse of this: the practical man specifies so many different variables to optimize and so many conditions to be satisfied that it is impossible to find a feasible solution. Again the value of parametrics in linear models manifests itself: the analyst can vary weights parametrically and engage in a dialog with the practical man which enables the latter to investigate the implications of assigning different weights in a model. In this way a model can be used as a learning device by the decision maker. On the other hand, one of the chief benefits of mathematical models is that they force applied persons to think through a problem carefully. The realization that some decision makers have great difficulty in clearly specifying objectives in a problem often has side benefits that are as great as those in the use of a model itself. 7. Data Problems in Optimization Models. Large optimization models generate a great demand for data; this is particularly true of linear models. Although we can easily solve models having 1,000 constraints and 3,000 variables, this does not mean that problems of this size are easily dealt with. If a problem has a constraint matrix of 1,000 by 3,000, then there are 3,000,000 data entries in this matrix and so there must be an effective information retrieval system which generates the required input data as well as efficient procedures for analyzing

- 20 -the output of these models. In many industrial applications it has been found that the cost of information inputs for optimization models exceeds by a factor of 20 the cost of solving the model when the relevant data are available. 8. Conclusion. We conclude this paper by emphasizing that the more successful our models and analytical tools become, the greater is the need for effective management of information at various points in the model building process. Actually, the models discussed in this paper, although powerful, are but one aspect of a collection of activities extending from formulation to analysis and decision as is suggested by the diagram shown in Figure 2. Successful model building requires that all these activities be carried out effectively......' 'IO -

P >< I 4 Data Updating and Revision -4 Data Bank Editor.h ol Data Bank `i/ Model Generator 1 I i-1 I Control Instructions Mathematical Programming Package I, Output Analyzer Analysis Report Figure 2. Interaction Between Data Handling and Model Building 1

Bibliography 1. Balinski, M. "Integer Programming: Methods, Uses, Computation," Management Science, vol. 12, no. 3, November, 1965, p. 253-313. 2. Boot, J.C.G., Quadratic Programming, Rand McNally Co., 1964. 3. Dantzig, G., "Linear Programming Under Uncertainty," Management Science, vol. 1, April-July, 1955, pp. 197-206. 4., Linear Programming and Extensions, Princeton University Press, 1963. 5. Deininger, R.A., Water Quality Management: The Planning of Economically Optimal Pollution Control Systems, Ph.D. dissertation, Northwestern University, 1965. 6. Gass, S., Linear Programming: Models and Applications, 2nd Edition, McGraw Hill Book Co., Inc., 1964. 7. Godfrey, J., G. Stillwagon, and W.A. Spivey, "Production and Market Planning With Parametric Programming," Industrial Management Review, vol. 10, no. 1, Fall, 1968, p. 61-76. 8. Gue, R.L. and M.E. Thomas, Mathematical Methods in Operations Research, The Macmillan Co., 1968. 9. Hadley, G., Linear Programming, Addison Wesley Publishing Co., 1962. 10., Nonlinear and Dynamic Programming, Addison Wesley Publishing Co., 1964. 11. Hinomoto, H., "Linear Programming Applied to Multi-Stage Capacity Expansion of a Water Treatment Distribution System," presented at the 38th national meeting of the Operations Research Society of America, Oct. 28, 1970. 12. Liebman, J.C. and W.R. Lynn, "The Optimal Allocation of Stream Dissolved Oxygen Resources," Water Resources Research, vol. 2, no. 3, Sept. 1966. 13. Llewellyn, R.W., Linear Programming, Holt, Rinehart and Winston Co., 1964. 14. Loucks, D.P., C.S. ReVelle, and W.R. Lynn, "Linear Programming Models for Water Pollution Control," Management Science, vol. 14, no. 4, p. B-166 -B-181. - 22 -

- 23 - 15. Spivey, W.A., Linear Programming: An Introduction, The Macmillan Co., 1963. 16., "Decision Making and Probabilistic Programming," Industrial Management Review, vol. 9, no. 2, Winter, 1968, p. 57-68. 17. Spivey, W.A. and H. Tamura, "Goal Programming in Econometrics," Naval Research Logistics Quarterly, vol. 17, no. 2, June, 1970, p. 183-192. 18. Spivey, W.A. and R.M. Thrall, Linear Optimization, Holt, Rinehart and Winston, Inc., 1970. 19. Spruill, M.L.,."An Alternative Method for Evaluating the Selection of Public Works Projects by Federal Agencies,"' to appear in the Proceedings of the Second National Meeting of the American Institute for the Decision Sciences, Dallas, Texas, Nov. 1970. 20. Van Dyne, G.M., "Application and Integration of Multiple Linear Regression and Linear Programming in Renewable Resource Analysis," Journal of Range Management, vol. 19, no. 6, 1966, p. 356-362. 21. Van Dyne, G.M., W.E. Frayer, and L.J. Bledsoe, "Some Optimization Techniques and Problems in the Natural Resource Sciences," in Studies in Optimization, 1, Society for Industrial and Applied Mathematics, Philadelphia, Penna., 1970. 22. Wagner, H., Operations Research, Prentice-Hall, Inc., 1969. 23. Wilde, D.J. and C.S. Beightler, Foundations ofOptimization, Prentice-Hall, Inc., 1968.

t l - -24 - OTHER WORKING PAPERS Working Paper Number 1 "Reflections on Evolving Competitive Aspects in Major Industries," by Sidney C. Sufrin 2 "A Scoring System to Aid in Evaluating a Large Number of Proposed Public Works Projects by a Federal Agency, " by M. Lynn Spruill 3 "The Delphi Method: ASystems Approach to the Utilization of Experts in Technological and Environmental Forecasting," by JohnD. Ludlow 4 "What Consumers of Fashion Want to Know," by Claude R. Martin, Jr. — Out of print. To be published in a forthcoming issue of the Journal of Retailing. 5 "Performance Issues of the Automobile Industry," by Sidney C. Sufrin, H. Paul Root, Roger L. Wright, and Fred R. Kaen 6 "Management Experience with Applications of Multidimensional Scaling Methods," by James R. Taylor 7 "Profitability and Industry Concentration, " by Daryl Winn 8 "Why Differences in Buying Time? A Multivariate Approach," by Joseph W. Newman and Richard Staelin —Out of print. To be published in a forthcoming issue of the Journal of Marketing Res earch. 9 "The Contribution of the Professional Buyer to the Success or Failure of a Store, " by Claude R. Martin 10 "An Empirical Comparison of Similarity and Preference Judgments in a Unidimensional Context," by James R. Taylor 11 "A Marketing Rationale for the Distribution of Automobiles," by H. O. Helmers 12 "Global Capital Markets," by Merwin H. Waterman 13 "The Theory of Double Jeopardy and Its Contribution to Understanding Consumer Behavior, " by Claude R. Martin y

r~ n Working Paper Number 14 "A Study of the Sources and Uses of Information in the Development of Minority Enterprise —A Proposal for Research on Entrepreneurship, " by Patricia L. Braden and Dr. H. Paul Root "Program Auditing," by Andrew M. McCosh "Time Series Forecasting Procedures for an Economic Simulation Model," by Kenneth 0. Cogger "Models for Cash Flow Estimation in Capital Budgeting, by James T. Godfrey and W. Allen Spivey 15 16 17