Bureau of Business Research Graduate School of Business Administration University of Michigan January, 1971 BUREAU OF BUSINESS RESEARCH WORKING PAPER NO. 22 SEA GRANT DELPHI EXERCISES: TECHNIQUES FOR UTILIZING INFORMED JUDGMENTS OF A MULTIDISCIPLINARY TEAM OF RESEARCHERS by John D. Ludlow, Research Associate Bureau of Business Research, University of Michigan Copyright Q) 1971 by The University of Michigan Printed in the United States of America All rights reserved

BACKGROUND OF THIS PAPER This paper is based on research done at the Bureau of Business Research, Graduate School of Business Administration, University of Michigan, in support of the Regional Economics and Water Resource Management project of the Sea Grant Program at the University of Michigan.

CONTENTS I. Introduction................... 1 II. Background of the Sea Grant Delphi Exercises.... 3 III. Research Needs..... 7 IV. Methodology........ 10 V. Evaluation.......... 20 FIGURES (2- 11) AND APPENDIXES....... 22 BIBLIOGRAPHY....... 75 -iii

10 FIGURES 1. Overview..... 2. Information package for Round 1 of Delphi Technical Panel 1... 0 0 3. Round 1 form for estimating important developments............ 4. Preliminary form for establishing matrix.............. 5. Definition of estimating factors and indexes —early rounds......... 6. Round 2 form for estimating important developments............. 7. Round 2 matrix for estimating pollution potentials......... 8. Events relegated to background information based on importance of concensus.... 9. Example of a statistical summary.... 10. Round 3 form for estimating important developments....... 11. Form for developing an ordered list of problems and opportunities...... 11 23 24-25 26 27-30 31 32 33 34 36 37-39 40-47 APPENDIXES A. Initial Memorandum to Technical Panel B. General Summary of Problems and Opportunities in Marine Resources and Alternatives in Waste Water Collection, Treatment, and Disposal 0 aI a. 10r -iv

0 Q4,11.R. I L c I C. Socioeconomic Development in the Grand Traverse Bay Region.......... 48-70 D. Participation Letter............. 71-74 EXHIBITS AND TABLES IN APPENDIXES Exhibits 1. Marine resource problems and opportunities... 44 2. Alternatives in waste collection, treatment and disposal................. 45 3. Impacts of waste water treatment........ 46 4. & 5. The Grand Traverse Bay Region........ 47&72 Tables 1. Comparison of Employment by Major Industry.. 55 2. Employment in Largest Manufacturing Industries in the Region................ 57 3. Employment in Natural-Resource-Oriented Industries, Grand Traverse Bay Region, 1959-601.............. 59 4. Average Earnings per Full-Time Employee, Michigan Upper Great Lakes Region... 60-61 5. Per Capita Personal Income and Per Capita Relatives................. 62 -v

I Introduction Forecasting is an integral part of planning. A forecast should: - Anticipate what occurrences are possible and assess their probabilities. - Assess the interactions (cros s-impacts) among these occurrences. -Identify the occurrences that can be controlled and the extent of such control. Evaluate alternative future possibilities, considering varying degrees of intervention that are within our power to control. -Convert these outcomes into displays that provide us with an assessment of the impact of the possible future. 1/ The objective of this paper is to present a plan for using modified Delphi techniques -essentially a method for securing informed judgments of a group of experts —to secure the use of the competences of a multidisciplinary team of researchers in forecasts related to planning for a representative region in the Great Lakes area. These forecasts will emphasize the consequences of regional planning on marine resources 1/ Selwyn Enzer, "A Case Study Using Forecasting as a Decisionmaking Aid," IFF Working Paper, WP-2, Middletown, Conn., Institute for the Future, Dec., 1969. -1 - A

-3 - 4. The improvement of communications among a multidisciplinary team of researchers, many of whom are accustomed to operating independently or within the boundaries of individual disciplines Experiments have shown that feedback and reassessment quite often result in convergence of opinions, as common elements of judgment are reinforced,. ambiguities resolved, extreme positions clarified, and the impact of related events assessed. The refinement and strengthening of a consensus are especially importantin regional planning, which is essentially a political process. But feedback and reassessment of informed judgments can be tremendously valuable even if it does not lead to conve rgence. 6 II Background of the Sea Grant Delphi Exercises The Sea Grant Program The Congressional mandate in the Sea Grant Act calls for both the development of marine resources for economic and social benefits to the nation and for the education and training of personnel to carry out such development. Integral to the program is the communication of useful information to the various marine communities with an interest in marine resources; The Sea Grant Program of the University of Michigan is designed to (1) establish standards of expertise and a center of knowledge for

-4 - oUI Great Lakes research, and (2) provide useful information as a service to regional planners and decision makers. Predictive models are to be used as a focusing device to bring together experts who are competent in diverse fields and organize their knowledge to provide analytical assistance in the design of a comprehensive planning system for dealing with marine resource problems and opportunities. The program strategy is to concentrate initially on a discrete subregion within the Great Lakes area, not only to avoid widely diffused research but to provide experience in coupling field research with systems analysis and model development. A feedback process is to be developed so that field research stimulates modeling, and the model in turn defines priorities for acquisition of additional field data. The Grand Traverse 2/ Bay Area — was selected because it is a reasonable physical analogue to Lake Michigan, the next subsystem of concern, and representative of many areas in the Great Lakes region where the economic development and quality of life are closely related to water resources. Although a considerable amount of empirical data is being developed in the Regional Economics program and other Sea Grant projects, judgments form the best source of insight into the future. The development of a method to obtain and refine informed judgments of knowledgeable people 2/ For this research the Grand Traverse Bay Region is considered to be the following ten counties which make up the Region 10 of the State of Michigan's planning and development regions. These counties are Emmet, Charlevoix, Antrim, Kalkaska, Grand Traverse, Benzie, Manistee, Wexford, and Missaukee.

-5 -is one of the most challenging problems in the task of improving decision making and planning. The need is particularly crucial in the Sea Grant Program, where the decision maker must consider the opinions of a multidisciplinary team of advisers —some of them experts in extremely specialized areas, while the competence and experience of others span a wide range of technical, economic, social, legal, and political matters — and where precise quantification and models for combining judgments are lacking. In spite of the importance of intuition and judgment there are inherent dangers in relying on the judgments of a single expert who will tend to view a problem in terms of the boundaries of his own expertise, even though his is only one of several disciplines bearing on the issue. The performance of the expert should improve when he can interact with other experts in the same or related fields. However,the use of a committee to effect the interaction introduces some serious administrative, logistical, sociological, and psychological barriers. The output is likely to be a compromise weighted toward the opinions of those in the group who are most articulate, most prominent, or better placed administratively. The Delphi method The Delphi techniques, developed by researchers of the RAND Corporation, provide an initial step toward a systematic use of expert opinion and they appear to have a flexibility that would allow for a much

-6 -3/ wider use than the published applications indicate.- The basic features of the method include anonymous responses, iteration, numerical estimates, statistical group summaries, controlled feedback, and reassessment. The means which they provide for obtaining informed judgments preserves the desirable characteristics of face-to-face group deliberations while overcoming some of the associated psychological and administrative barriers. The method yields a cumulative assessment of the group's anticipations without requiring elaborate investigations and support of each issue under consideration. The objective of the method is to obtain from a group of experts the most reliable consensus of opinion through anonymous responses to a carefully designed program of sequential interrogations. The techniques can be modified to exploit the talents of a well-informed multidisciplinary team of experts by securing judgments that are weighted to reflect specialized competence and, using the combined insight of the group, to interpret those judgments for regional planners and decision makers. The information exchange is accomplished through a series of 3/ Initially the studies were under the guidance of Dr. Norman Dalkey and Dr. Olaf Helmer. For background information on the Delphi techniques, modifications, and applications see: John D. Ludlow, "The Delphi Method: A Systems Approach to the Utilization of Experts in Technological and Environmental Forecasting," Working Paper No. 3, Bureau of Business Research, Graduate School of Business Administration, University of Michigan, Ann Arbor, Michigan, March 24, 1970.

-7 -information packages referred to as rounds. The opening rounds are controlled brainstorming sessions in which a respondent is encouraged to present developments in his area of expertise which he feels will be important for a specific issue or topic. The feedback of responses from other informed people serves to stimulate him to consider developments in his own or related areas that he may have inadvertently neglected. To facilitate the initial exchange of information and to focus on common measures of values in developing and presenting a viewpoint that may be difficult to articulate, respondents attach numerical estimates to a list of important events developed by the group. Self-appraisal indexes are provided to permit the respondent to indicate his relative competence in specific issues and his familiarity with the region. Desirability and feasibility indexes can be designed to account for the value judgments of the estimators and the relative influence of technical, social, economic, and political factors. These serve to help the expert in making his judgment and the rest of the panel in interpreting them. III Research Needs Although the initial focus of the Sea Grant Program is on a discrete subregion, its main concern is to develop techniques that will be useful in designing a comprehensive planning system for the management of the marine resources of the whole Great Lakes Basin. The Delphi method will be employed in several roles that represent essentially new applications. At a recent conference recognized experts ft ^ *.

4 -8 -4/ in forecasting and planning- were in agreement that the hierarchy of planning levels is best expressed by the basic three-level concept of policy planning, strategic planning, and tactical planning. Thus far the Delphi method has been employed primarily to obtain and refine the long-range forecasts associated with strategic planning. Sea Grant experts see their task not only as providing the basis for forecasting alternative futures for a region but as also assisting in "creating the future, " and their combined judgments should be brought to bear also at the normative (policy formulation) level and at the operational level of planning. This will be attempted in the Sea Grant Delphi exercises. The panels will be asked to make value judgments. Dalkey indicates that in making value judgments the validity of Delphi procedures —in the sense of the willingness of respondents to furnish lists of objectives or goals, to allocate weights, to accept a statistical aggregation of weights supplied by a group, and to reassess their judgments based upon feedback of information supplied by the group —is much more obscure than in 5/ factual judgments.Several techniques for improving the estimating process will also be tested. A concern in securing personal probability assessments is 4/ Erich Jantsch, Perspectives of Planning (Paris: Organization for Economic Cooperation and Development, 1969). 5/ Norman C. Dalkey, The Delphi Method: An Experimental Study of Group Opinion, Memorandum RM-5888-PR (Santa Monica, Calif.: RAND Corp., June, 1969). at ' e

/0 - |~-9 -that they correspond with the assessor's judgment (i.e., that he doesn't violate the postulates of coherence). The communication problem is compounded when individual distributions are combined into a single distribution representing a consensus of judgments and presented to a decision maker who must interpret it. To gain insight into the nature of the problem, numerical probabilities will be compared with associated verbal phrases. The Delphi techniques of numerical estimates, feedback of group responses, and reassessment will be employed to develop an ordered scale of verbal phrases, generally comparable to commonly used numerical probabilities. The verbal phrases may be more appropriate in estimating social developments where the use of numerical estimates tends to give an exaggerated, and consequently a somewhat less credible impression of precision. Another vexing problem associated with personal probability estimates is the assumptions the estimator makes about the future environment. There is a proposed remedy which is procedural: the attention of the panel is first focused on the technical environment and the political, social, institutional and other environmental factors are assumed to develop along present trends. Subsequently, a broader perspective is taken and the influences of the other environmental factors and the value judgments of the estimator are taken into account by using such measures as desirability, feasibility, and importance indexes, developed for various time periods and from a personal as well as a societal orientation. These indexes assist the estimator in p

-10 -making his judgments and the other panel members in interpreting and weighting them. Techniques for exploiting the use of conditional probability estimates will be examined by the program administrator. The administrative procedures are flexible enough to incorporate further refinements as the exercises progress. IV Methodology Overall research design The design of this exploratory research is depicted in Figure 1. In preparation for the Delphi exercises a study was made to determine which technical, social, economic, and political issues might have a significant impact on the region's marine resources, A progressive type of Delphi method will be attempted in which a list of pertinent developments in the technical environment will be generated and assessed by a technical panel before the important societal developments are considered by broader multidisciplinary panels. The prior consideration by a technical panel simplifies the problems of making estimates based on assumptions about the total future environment, avoids bogging down the panel members with data and directs their attention to areas where their interest and expertise lie. Many of the panel members selected should be experienced in the technology of waste water treatment and disposal and also familiar with the Grand Traverse Bay area. The self-appraisal and importance indexes

r ' I Select technical experts Technical panels Jj I. ---' --- -— ' - \ / Technical /'Waste- \ environ- / water \ ment treatment.... - _ 1.... r E7! Random access / It / 1 I - I I i I I V " 1 8 / \ ' I ' Seminar / /3egional \ /\I |Regional influen- Researcher I influen- Researchers tials / tials Combined Combined panel panel _.. _ --- - I. _ - I 1 - * 0 -- I I U I I' I.etters I I,, e 8 1 I 1 t I Seminar I I - I - ~~_ _'I_ _ _ _ _.~~~~~~~~~~~ \4 F wa Technological and societal developments Probability Timing Desirability Feasibility (economic, social, political) Relative importance Consequences for region Identify policies and programs i L-;i S imulat ion l/ \ / Delphi \ concepts / \ C~nC-ePT S / gaming Input / output (Feasibility) -ZI Goals Policy criteria C + Evaluat ion Hat) Figure 1. Overview. I, - I I. I

-1Zwill provide a mechanism for developing a subgroup, competent and interested in waste water treatment and disposal, and thus provide an opportunity for gaining a level of specificity consistent with operational planning while considering related factors in the larger technical environment Increased benefits from the exercises can be realized if a seminar is held to review the output of the technical panel. The broader panels will consider the judgments of the technical panels as well as other technical and societal developments. Regional planners and decision makers will be invited to participate along with Sea Grant researchers. The panels will be balanced so far as is practicable in terms of experience, age, sex, and so on, and several panels will be conducted concurrently to check the methodology for consistency. If the results justify it, a seminar will be held to review the output of the exercises. The Delphi exercises together with forecasts based on methods and information from other Sea Grant projects will provide information for developing alternative scenarios for the Grand Traverse Bay region. These scenarios can be subjected to gaming and simulation during which the Delphi concepts of dynamic feedback and reassessment can be used to evaluate the impact of various forms of intervention and to develop strategies as well as to generate exogenous technological and societal developments. The alternative futures developed could be tested for feasibility in relation to the state or national environment. Flexibility can be incorporated in the Delphi exercises so that the information exchange can be extended to other interested individuals

-13 -or groups on a random access basis. Evaluation and modification of the methodology to satisfy Sea Grant requirements will be conducted almost continuously. Technical panels There are expected to be about 25 experts participating in the technical panel: with a subpanel knowledgeable in waste water treatment and disposal formed from the group on the basis of self-appraisal and importance indexes. Candidates for the technical panel will be contacted personally to insure that objectives and procedures are understood and that all relevant disciplines are very likely to be represented throughout the period of information exchange. This direct contact is also desirable since it is intended that the initial administrative procedures will be quite flexible. The immediate objectives of the technical panel exercises are to use informed expert judgment to: 1. Establish consensus on the relative importance of pollution sources for Grand Traverse Bay and their potential for abatement 2. Obtain the insight of technical experts into the marine resource problems and opportunities of the Grand,Traverse Bay area 3. Generate a list of technological developments that would be important for the area i> ' '.~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

-14 -4, Determine the probability, timing, desirability, and feasibility of these developments 5. Establish a subpanel of experts knowledgeable in the technology of waste water treatment and disposal and use their judgments to develop: a. Weighted alternative approaches to waste water treatment and disposal b. Research and informational references c. Research and informational needs The essence of the Delphi method and modifications can best be illustrated by going through a series of sequential interrogations or rounds. Representative forms and responses are attached to this working paper (Figures 2-11). Round 1 Each participant is sent a package of general background information and estimating forms for (1) sources of pollution and specific pollutants, and (2) important technical developments relating to water resources. (See Figures 2, 3, and 4). It is desirable that the respondents volunteer most of the information that will be part of a combined list of events that will be fed back to them on subsequential rounds and narrowed on the basis of group judgments regarding importance and uncertainty. However, the administrator, with his technical consultants, should be prepared on subsequent rounds to interject some events for consideration by the

-15 -panel if an area of interest is not satisfactorily covered by unaided responses. The technical developments and sources of pollution are elicited on estimating forms designed to familiarize the panel with the estimating procedures and to minimize the inclusion of trivial events and those that cannot be subjected to the desired specific quantitative estimates. Definitions associated with the estimating factors for the early rounds are attached to the estimating forms (Figure 5). The questionnaires are returned to an administrative assistant (in the Sea Grant Advisory Office) who records their arrival and insure s that only a panel member's number is on the completed forms. He then turns them over to the chief investigator who, using technical consultants, will collate and edit them to eliminate duplications, combine similar events, and retain only discrete events and consistent technical references. Round 2 Each participant receives the edited list of events and pollution sources and is asked to make his assessments, bearing in mind the interrelationships of other events that the panel felt were important (Figures 6 and 7). He may also add other developments that he wishes the panel to consider. The first two rounds are designed as brainstorming procedures with the panel indicating acceptance or rejection of an idea by the numerical estimates which they assign. The responses are summarized to indicate the median estimate of the group —and selected subgroups —and the spread of opinions. The range of the consensus can be specifically designated but generally I

-16 -the interquartile range —the interval containing the middle 50 per cent of the responses —is considered to represent the thrust of the current consensus. Round 3 Starting with this round and on subsequent rounds, the statistical summary of the previous round is presented to the panel members (Figures 8 and 9). Each respondent is asked to reassess his position on those events that the panel has decided are important and on which there exists a wide range of opinions, considering carefully the statistical summaries, prerequisite technological improvements, and the internal consistency of interrelated events. If his revised estimate falls outside the ranges indicated as the consensus for the previous round, he is asked to provide brief supporting arguments for this extreme position. These minority opinions together with specific questions directed to specific panel members —based on an analysis of their previous responses —are included in the summaries of subsequent rounds. Respondents may be asked to revise their judgments on specific events as the result of the administrator's indicating a delay in prerequisite technology or proposing a specific type of intervention, such as pollution control regulations. (See Figure 10. ) Those panel members who have indicated a competence and an interest in waste water treatment and disposal will be asked to indicate an order of preference for waste water and disposal systems for a region similar to the Grand Traverse Bay Region, to provide pertinent research

-17 -and literature references, and to list information needs and research priorities. All respondents will be asked to indicate what problems and opportunities they foresee for the region (Figure 11). Fourth and subsequent rounds The information provided by the group is fed back to the respondents who are asked to reassess their position on important events on which there is still no satisfactory consensus and to provide a weighting of preferred approaches to waste water treatment and disposal and problems and opportunities of the region. If the results warrant, a seminar which could include regional planners and other interested people will be convened to discuss the results of the Delphi exercises. l p a Broad multidisciplinary panels Panels of approximately 25 members will be selected from a pool of names drawn up from the responses to a participation letter sent to all researchers in the Sea Grant program and to planners and decision makers in the Grand Traverse Bay Region. Biographical information provided by the participation questionnaire will be the basis for balancing the panels, The information exchange will be directed so that for the first two rounds participants from the region will constitute separate panels from the Sea Grant researchers. On subsequent rounds they will join together to comprise three combined panels. The output of the separate

-18 -and combined panels will be compared to check the consistency of the methodology. The immediate objectives of the broader panels are to utilize informed expert judgments to: 1. Identify important technological and societal developments —in addition to those identified by the technical panel —that could influence regional planning in the next 20 years 2. Determine the probability, timing, desirability, feasibility, relative importance, and trends of these developments 3. Anticipate the probable impact of these technological and societal developments on the region 4. Identify the problems and opportunities for the region in the next ten years The iterative feedback and reassessment techniques for the broader panels are similar to those employed by the technical panels. Some of the estimating procedures will be modified, however, to deal realistically with conceptual differences in anticipating future societal 6/ changes and forecasting technological progress.6/ The comparison of expected technological and societal developments is based on information contained in: Raul de Brigard and Olaf Helmer, "Some Potential Societal Developments, 1970-2000, IFF Report R-7, Middletown, Conn., Institute for the Future, Apr., 1970.

-19 -In technological forecasting, technology is generally treated as an irresistible force which can be accelerated or decelerated by society, but cannot be stopped or reversed. The occurrence or nonoccurrence of societal developments, however, depends greatly on the nature of human intervention and both progress and regression are possible. In addition, the process of social change is far more complex and the vocabulary far less precise in the social sciences than in the hard sciences. Some techniques will be explored to obtain and refine group judgments without giving an exaggerated impression of precision. One technique is to present graphically the historical trends of particular interest and ask the panel to extend the graphs through the forecasting period. Feasibility, desirability, and importance indexes, together with arguments advanced to support positions outside the median range, can be used to define a consensus which can be further refined through feedback and reassessment. Another similar technique, which researchers at the Institute for the Future have used in a series of studies dealing with long-range fore7/ casting of technological and societal events,- is to elicit from the panel possible future societal developments which might occur in the forecasting period and which would represent changes from current patterns. Two nominative scales were used in those studies to indicate the importance 7/ Ibid., p. 17. r

-20 -and the strength of the trends. This technique will be modified in an attempt to avoid some of the ambiguities in scale and timing that were encountered. The combined panels of planners and decision makers from the region and Sea Grant researchers will assess the impact on the Grand Traverse Bay region of the technological and societal developments which the technical panel or earlier rounds of the broader panel have described in terms of timing, desirability, feasibility (social, political, institutional, technical, economic), importance, and trend. And they will explore the feasibility of various forms of intervention to avoid undesirable consequences or to improve the likelihood of realizing opportunitie s. In the Sea Grant exercises the Delphi techniques will be used in conjunction with other methods of obtaining group judgments including interviews and conferences or seminars to discuss the results of exercises and the effectiveness of methodological modifications. V Evaluation Evaluating the effectiveness of the Delphi techniques in the Sea Grant exercises can be related conceptually to the test marketing of a new product- -in this case the new product is a group of management techniques for obtaining a consensus of informed judgments. Thus far, the Delphi method has received limited exposure which has been primarily among technically-oriented people. The Sea Grant

-21 -exercises will provide a unique opportunity for an evaluation of the method in an operational exercise with a much broader segment of talented people: noncaptive respondents and decision makers who have not had a key role in committing resources to the exercises or an active role in administering them. Even more important the participants will be particularly well qualified to criticize the techniques —they are not only concerned with the substantive results of the exercises but are dedicated to searching for improved methodologies and have a relatively keen awareness of the behavioral difficulties encountered in conventional approaches to group deliberations and assessments. Their recommendations could result in significant improvement and refinement of the method as well as wider applications.

I FIGURES (2-11) AND APPENDIXES These forms and figures are not necessarily those that will be used in the Sea Grant Delphi exercises but are given as examples of the type of information that will be contained in the early rounds. K

-23 -TECHNICAL PANEL 1 PANEL MEMBER 01 INFORMATION PACKAGE ONE This information package consists of: 1. A general socioeconomic summary of the area. 2. A general summary of marine resource problems and opportunities and alternatives in waste water collection, treatment and disposal. 3. Forms to be accomplished. a. sources of pollution b. important technical developments relating to water resources c. problems and opportunities for the region d. biographical information Please submit your response as early as practical but no later than To: Delphi Exercises Sea Grant Advisory Office Figure 2. Information package for Round 1 of Delphi Technical Panel 1.

W Round 1 Panel Member Estimating Form (A); Important Developments 1. 2. 3. 4. Self-evaluation (A). Familiarity with Grand Traverse Bay area (scale 1-5) --- List the developments that you feel will be important to the Marine resources of the area in the next 20 years. Indicate pre requisite technology associated with the developments. Indicate with a check mark (v) those developments pj CD CD Timing closely related to wastewater a, D l 7 I. t Evaluate all developments (: with respect to the factors on i o 0 the right. The definition of ^ 0I Year by which the factors and indexes are probability is x that attached. i 1 the event will have:.. _. _{occurred. I_ '.~,..,,,_,,,~, Remarks Group developed verbal phases associated with numerical probabilities will be available on subsequent rounds i _, _, _,,_._.,.~ ~, ~-~-.~ -Y..~-h~a.~6*Urs*.*I~HL~fl~~C ~~O~~-All Developments I5 I z I 1 -5 I I num0-10-0i ber(s) x=. 25 x=. 50 ~~~~~~~~~~~~-,, r ~"" ~~"". I x=. 75 Remarks _ -.- - ~ 5: ~ ~ --— ~- ~ -~- -II -+ - -, " — -... ----.,+._. _ , —. —. —. --- —-— P. ---.a;-aalJlOJIA., -~ ""I~l"~""V-"CCCI~~~~-!lr.. -.. _. i L i 1 e I t I i. i I I I i i I | I I J I I 6 I I N 4 i Figure 3. Round 1 form for estimating important developments.

Round 1 Panel Member Continuation of Estimating Form (A): Important Developments List additional developments and evaluate with respect to the factors on the right.. D -, (D w I.cort CDCD rrO CD Y-;c Co CD I0 i IC; 'p10 c r II 1 3 3 0 -0 - 1 1 2J S: < C n i: e *i num- D ber(s) 3. Timing Developments.:1-5, a.. I I I I A -=. 25: z i. x=. 50 x=. 75 Remarks i iI t II i.I I iI I i.. K II i k. I i i N) Ul Figure 3. (con't)

Round 1 Panel Member Estimating Form (B): Sources of Pollution Listed below are some potential sources of pollution to a body of water comparable to Grand Travers some specific pollutants. The purpose of this estimating form is to develop a list of pollution source pollutants that in subsequent rounds will be presented in a Matrix Form which will permit judgments made as to the relative importance of pollution sources, the social and economic activities affected, technical and economic feasibility of making the associated problems more tolerable. A tentative Ma be used in round 2 is attached for your guidance and consideration. Sources of Pollution 1. Effluent from Traverse City sewerage system 2. Septic tanks in the region 3. Storm water run off 4. Industrial wastes You are requested to add others e Bay and s and to be and the Ltrix to Specific Pollutants 11 1. Phosphorus compounds 2. Nitrogen compounds 3. Oxygen consuming Organic matter 4. Bacteria viruses 5. Turbidity 6. Suspended solids You are requested to add others Figure 4. Preliminary form for establishing estimating matrix.

-27 - a Definition of Estimating Factors and Indexes Self-evaluation indexes -- scale I to 5 (a) Familiarity with the Grand Traverse Bay Region Your estimate of your familiarity with the Grand Trave Region (the 10 county area shown in the information package) 1 = Totally unfamiliar with the region 2 = Casually acquainted with the region 3 = Well acquainted with a few aspects of the region 4 = Generally familiar with the region 5 = Actively studying the social and economic development of the region (b) Familarity with the item being considered rse Bay Your estimate of your knowledge and comprehension of the specific development being considered in comparison to other researchers who are participating in the Sea Grant Program at the University of Michigan 1 Unfamiliar 2 = Slightly familiar 3 Generally acquainted 4 = Well acquainted with most aspects 5 = Expert or researcher working in the area Feasibility indexes -- scale 1 to 5 o ~......... * 1*.,. (a) Technical feasibility Your estimate of the technical difficulty in accomplishing the event. Assume a political and social environment which might evolve normally from present circumstances I - Extremely difficult 2 Very difficult 3 = Moderately difficult 4 Slightly difficult 5 = Routine development Figure 5. Definition of estimating factors and indexes —early rounds,

-28 -Definition of Estimating Factors -- continued (b) Economic feasibility Your estimate of the economic reasonableness of accomplishing the event. Assume a political and social environment which might evolve normally from present circumstances 1 = Extreme subsidy required 2 = Moderate subsidy required 3 = Economic aspects are of minor concern 4 = Slightly attractive economically 5 = Economically attractive Although the main focus of this panel is on technical performance, some developments will be highly dependent on economic considerations. For some of these developments technically trained individuals can best make the economic feasibility assessment. Timing In an attempt to reconcile variations in personal probability estimating routines and to further refine the consensus of informed judgments, you are asked to: (a) Estimate the probability that the event will happen in the 1971-1980 time period (b) Estimate the dates by which the probability is.25, 50, and. 75 that the event will have occurred (c) Indicate —by remarks —any significant discontinuities in your probability estimates (d) Indicate numerical probability estimates for each of a list of probability-related words and phases. This is a one time requirement and is an attempt to develop verbal labels to a probability scale which can be used in estimating societal developments where numerical estimates could give an exaggerated impression of precision Importance Index -- scale 1 to 10 Starting with the second round the events will be rated according to their importance to the exercise as judged by the panel. In subsequent rounds only "important" issues on which there is no reasonable consensus Figure 5 (con't).

Definitions of Estimating Factors —continued will be considered by the panel. After examining all events on the estimating form make a judgment as to the most important and least important and assign a value of 10 and 1 respectively. Then assign each'of the other events values from 2 to 9 as you estimate their values relative to the most important and least important events. You may assign the same value of any number of events. Personal Probability Assessments Personal probability assessments are an important feature of the Delphi techniques. However, in estimating probabilities associated with social and political developments numerical scales give an exaggerated impression of precision. Therefore an attempt will be made to establish for each Delphi panel an ordered list of verbal phrases that corresponds generally to commonly used numerical probabilities. You are requested to assign a numerical probability from. 00 to 1. 00 to each of the following verbal phrases that intuitively express the notion of the likelihood of an event, for example: Likely =.70. ~... 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Highly probable = Very likely = Very probable Quite likely = Good chance = Likely - Rather likely Better than even = Slight odds in favor Tossup = --— TI I --—; 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. Slight odds against = Uncertain Somewhat unlikely - Fairly unlikely = Rather unlikely = Not much chance Improbable = Quite unlikely Very unlikely = Highly improbable = Figure 5 (con't)

-30. Select from the above list —or add your own —verbal phrases that best describe the following numerical probabilities..10.75.25 _ 90.50 Figure 5 (con't).

Roundc 2 Panel Member O X Estimating Form (A): Important Developments 1. Self-evaluation (A). Familiarity with Grand Traverse Bay Area (scale 1-5) ---2. Listed below are technical developments contributed by the panel as important to the marine resources of the area in the next 20 years. Add other events that you would like the panel to consider. C/I _ _ 3. Evaluate all developments withi respect to the factors on the right (economic feasibility only if pertinent). Indicate if an event is prerequisite to a listed event. ' cn. i. C; C 5 p-. 0 f 0. i1 4 ' i Feasib (D 0 0 p-a ility I ~ g {. '. oo CD c4, R CD. I~ ~ ~ ~ ~ ~ ~ ~~~~~~~~~~~~u 4c=ecarra-c ____________________ Timing Year'by which the probability is X that the event will have occurred. I I Probability Quantitative -verbal.25= 50= Remarks "'.75= ~~~.^^ i-Br *I<"? ~ i".~ <~ ' I ~: ' ~ tRem arksi Technical Development j-5-i1 1-5;o-. i R marks TechnicaliDvelopment 1-5ii-lI0t 1-5 1-5 D0-100. No. ix x=. x=.50 x =. 7= 75 - ~ i~~ u ~ PrsrSn- *~w~.'*~-ra~~L** u~u~r*- w ~r~auni ~ ~~P~E~,sh.~j~vl-sr~m~- -,i. -` — 11- - 1r1 I i -r 1.. i 4 63 Figure 6. Round 2 form for estimating important developments.

Round 2 Panel Member Estimating Form B: Sources of Pollution Listed below are some potential sources of pollution of Grand Traverse Bay and some specific pollutants. --------- * —rs- F I~l- ~, — ~ --- --— a ---- -- - — I-~~-~ — — 3$-~~~~ —~~ - 1. Add others which you would like the panel to consider 2. Evaluate with respect to the factors on the right. Assume a social and political environment which is consistent with present trends cn CD PS Vt~ 4 - Relative ImportariCe Abatement - Feasibility i Specific Pollutants Social and Economic Activities Affected,.o —, I 00 Ca i. 41 11 I ID - C CD n C.).P n 0 0 (3S ~-J ~.. A.L i i i | 1 -' 's Add Others 0 )I 0. a z.-i v It rI 0 CD I-: i If. 03 PS C) CD 03 '-A. i (B '1 I i ) I0 H C; II cr. Vtt en W S C U) a' Ii U) 0 CD -03 I PL. t03 1. Recreation 2. Commercial fishing 3. Indus. water quality 4. Aesthetic s 5. Public water supply List Others 6. 7. ~ 8. | I.J 1Source contributes what o of List (by Description of Source 5 1-10 1-10 i 1-5 1-5 specific pollutant in order o 4 c ol lSc number )' f importance fa.,-v,"; -Yci-. - -"-v~*,5"- u E^A t'.- -- s- - * -' I I 1. Effluent from Traverse City sewerage system (see definitions) 2. Septic tanks in the region1 3. Storm water run off 4. Industrial Wastes a. Food processing, b. Other estimates should sum to 100 r i i i i r ~~. 5 i r I r I I i Z t f C i i II t: 5; iq iI w Ij A 100 i 100 100 1 I 4aI 3 T.1 c PiI f PS%. t'; uN~I I t 100 l100 1 I 10.. -..f.l l j laooiooji ooj i. -, R 117..I. -- --- - li%:-, —l ----, c -A- --- Figure 7. Round 2 matrix for estimating pollution potentials.

-33 - SUMMARY OF GROUP RESPONSE 1. The following events will not be considered further by the panel because the panel has indicated that they are relatively unimportant. 2. The following events will not be considered further by the panel because a reasonably strong group consensus has been indicated. 3. You are asked to reassess your estimates of the remaining events on the basis of information supplied by your Delphi panel. Consider the interrelationships among events as well as the judgments of the other panel members. Figure 8. Events relegated to background information based on importance and consensus.

_ + 0Event 1 SUMMARY OF GROUP RESPONSE OPERATIONAL PHYSIC OC HE MICAL COAGULATION /AD SORPTION DIREC T TREATMEN T =,. ---.- -, -..._; F-r_ _.^ _ v..~ R... i -, - - o r. -. -- - * - - f I Panel and subpanels I Z D 0d U1h P) CD - X Famil- i iarity Impor tance 2 Ev't 3 R 'n 6 Your estimate - r 's. -, w, f, -,,,,,,~ -, s a. Total Panel 25 U M L 7 6 1. 7 4 1. 7 6 1. I I I I Feasibil-: robity ability Ib ( 1971 Tech Econ 1980) F-.........*. L0- 1M8 15 830 7 6.88 E 6 4. 72 t{2 1 i. 21 7 6.90 16 4. 75 4 2.25 zo s e who rated~~~~~~~~~! 1 Timing 1 9 (mediam.25,. 50, and 9 t7. 75 probabilities) 8 123456789012345 -, ---..~...... - *.... ~,,ya t.~. g-a. a,, r K'S-,1,E, _,_,_-.a, _ _.._ — _ — - j4. -^ _ _^ ~,.50 (88). 75 (92)_ Beyond 1985 (year in parenthesis).ose who rated )mpetence 4 or U M L 8 7 6 5.e.; 7 6 5 6 6 5 ;.-r~. ~- ~~ ~- r i,.-~ — ~~ -~ ----. ~r,:-. - i —...r~ r ____.... IU 7 7 6.7.90. ---- - Civil Engineer 5 i 66 5 72 L 6 5 55 5.47 _....,,.e. PREREQUISITE TECHNOLOGY: SUPPORT FOR EXTREME POSITIONS: DIRECTED QUESTIONS: -vaSmWOE vent 2 =X C~-~INI-a*I-~ CL*~ I ~~nl. _~u _ ar nlcari. as.. r; —''- Y C.? I~-E~ ' -ip >j - z ~~. 4La.b b b ei.- sfP 6 f *.-C g a' fp*.??? ~ -* i '-_i<-~ - _gUy;;_-a i6h'.i~i3iU3.QQT-(r~XeYSi-i~ '. Uri:4LcLL-"Ti-irCZur~i*C?bi9jhr~ n —Z - —r- r-tE;,- —.-;-~T.Us~:F4r~a'+s9-j Your Estimate t 1 -Total Panel: U= upper quartile M=median L=lower quartile. The interquartile range —the interval containing the middle of the responses —is considered the current consensus. i^ i*ri~au ~c.50 per cent.= 50 per cent 0 If reassessed estimate is still outside of consensus range, submit brief supporting arguement for your posit Figure 9. Example of a statistical summary. tion. I w

Round 3 Panel Member 4 Estimating Form (A) Important Developments 1. Self-evaluation (A). Familiarity with Grand Traverse Bay area (scale 1-5) — 2. Listed below are technical developments which the panel has judged important and on which a satisfactory consensus has not been obtained. You are asked to reassess your position, considering the attached statistical summaries and the internal consistency of inter- C Feasibility - Timing Remarks related events....... o Probabilit 3. Evaluate all developments with < e., I Year by which the Quantitative - c ~t t-t~ cr respect to the factors on the P H 2 - probability is x. 5= right. If any revised estimate, C C - o that the event will.50= falls outside of the consensus. -.have occurred.75= ranges, as indicated on the attached summary, submit brief; supporting arguement. V. Verb al Technical Development 1-5 1-10 1-5 1-5 0-100 x=. 25 x=. 50jx=. 75,~~~~~~~~~~~ Supporting Arguements i Probability and timing of event No. if event No. does not occur. Probability and timing of event No. if pollution control requirement becomes Figure. 10. Round 3 form for estimating important developments. I I

- 36 - Estimating Form C: Problems and Opportunities List what you feel are the most important problems and opportunities in the Grand Traverse Bay Watershed Area related to its water resources. Be as specific as practicable. A combined and edited list of problems and opportunities will be fed back to the panel on the next round and estimating proceedures will be employed to establish their relative importance and impact on the region. Figure 11. Form for developing an ordered list of problems and opportunities.

APPENDIX A INITIAL MEMORANDUM TO TECHNICAL PANEL MEMORANDUM TO: FROM: John D. Ludlow Regional Economics and Water Resource Management Sea Grant Program SUBJECT: Delphi Technical Panel 1 Thank you for agreeing to serve on the "Delphi" panel to consider technological developments that will have important influence on the water resources and related social and economic development of a region with the general physical and economic characteristics of the Grand Traverse Bay watershed area. This panel is made up of approximately 25 technical experts from the University of Michigan whose reputation and experience indicate that they can make a valuable contribution toward a definition of future technical environments for water resource management. This Delphi exercise is part of a program to develop regional forecasts which emphasize the interdependence of economic and social development and water management decisions. A summary of the panel's judgments will provide background information for broader based Delphi panels —including regional planners and decision makers —which will consider the social and political as well as the technical environment. Although a considerable amount of empirical data is being developed in the regional economics program and other Sea Grant projects, the best source of insight into the future is the judgment of well informed people. In the Sea Grant program, which employs the concepts of systems analysis and a multidisciplinary team of researchers, one of the most challenging issues is that of securing the consensus of those judgments on important issues. The Delphi techniques developed by researchers of the RAND Corporation provide a systematic method for obtaining and refining opinion and appear to have the flexibility to be effective in relating forecasting to regional planning. -37 -

-38 - The basic features of the Delphi method —anonymous response, numerical estimates, statistical group summaries, controlled feedback and reassessment —provide a systematic method for obtaining informed judgments that preserves the desirable characteristics of conventional methods of obtaining group judgments while minimizing some of the associated behavioral and administrative difficulties. The panel is a multidisciplinary group and it is expected that some members will be experts in specialized areas while the competences and experiences of others will span a wide range of disciplines. Self appraisal indexes will permit the respondent to indicate his relative competence on specific issues and his familiarity with the region. Experiments have shown that feedback and reassessment often result in the convergence of opinions as common elements of judgment are reinforced, ambiguities resolved, extreme positions clarified, and judgments formulated and refined as the interrelationships of events and the weighted opinions of experts in other areas are considered. However, the feedback and reassessment of informed judgments should be tremendously valuable even if it does not lead to convergence. The information package for the first of four rounds of questioning is enclosed with this letter. In each round after the second a panel member will receive a copy of his past response and a summary of the values assigned by the entire panel and by various subgroups within the panel. Special instructions will accompany each information package but you may want to retain this letter for future reference. To insure anonymity each respondent will be identified by a number —which will be linked to his name only for the purposes of mailing out information packages and checking in the responses. This will be done by an adminsitrative assistant in the Sea Grant office. Some biographical information will be requested to help develop statistical summaries for subgroups within the panel. The first information package is relatively large because it contains much background information. An important feature of the Delphi method is that it does not require elaborate development and support of positions taken on each issue. After the panel has established which items are to be considered first,these will be continuously narrowed until only those items are left judged important to the exercise and on which a satisfactory consensus has not been reached. The immediate objectives of the technical panel exercises are to use informed expert judgment to: 1. Establish the consensus on the relative importance of pollution sources for Grand Traverse Bay and their potential for abatement

-39 - 2. Obtain the insight of technical experts into the marine resource problems and opportunities of the Grand Traverse Bay area 3. Generate a list of technological developments that would be important for the area 4. Determine the probability, timing, desirability, and feasibility of these developments 5. Establish a subpanel of experts knowledgeable in the technology of waste water treatment and disposal,and use their judgments to develop: a. Weighted alternative approaches to waste water treatment and disposal b. Research and informational references c. Research and informational needs Please feel free to request any additional information —either by telephone (764-1366) or with your questionnaire response. A final summary anda complete analysis of the exercises will be sent to each respondent upon completion of the final round of information packages, expected to be round four.

APPENDIX B GENERAL SUMMARY OF PROBLEMS AND OPPORTUNITIES IN MARINE RESOURCES AND ALTERNATIVES IN WASTE WATER COLLECTION, TREATMENT, AND DISPOSAL The emphasis in the Delphi exercises will be on developing carefully formulated informed judgments. The panel will consider concurrently (1) interrelated regional development and marine resource problems and opportunities in the Grand Traverse Bay Watershed area, (2) the relative importance of sources of pollution and the economic and technical feasibility of controlling them, and (3) the technical environment relating to the treatment and disposal of wastewater. The identification and evaluation of discretely defined problems and opportunities is a fundamental phase of a longer-term research program designed to provide knowledge, information, and methods for more comprehensive management of regional marine resources. The attached Exhibit 1 is a summary of general marine resource problems and opportunities for the Grand Traverse Bay Watershed area. The consideration of these issues involves a relatively high level of technical expertise and would benefit from an information exchange by technical experts prior to consideration by a broader panel. The issue of wastewater treatment and disposal will be used to evaluate the effectiveness of the Delphi methodology in initiating the exchange of technical information and presenting the results in a form that can be used by regional planners and decision makers. In considering the issue, the objective is to gain the highest water quality -40 -

-41 - consistent with the maximum benefit to the people of the region. General alternatives in waste collection, treatment, and disposal are presented in Exhibit 2. Exhibit 3 illustrates the effects of domestic and industrial wastewater on water-based activities. Statement of the Problem - Domestic wastewater includes the liquid wastes originating in the sanitary conveniences of dwellings, business buildings, factories, and institutions. By comparison, industrial wastewater consists of the liquids discharged from commercial and industrial establishments including water used for processes, boiler feed, manufacturing, or cooling. Domestic wastewater is sometimes treated on an individual basis by means of septic tanks or cess pools and then discharged to the ground. Alternatively, domestic wastewater can be collected in sewers and transported to a central treatment plant; following treatment the.wastewater (or effluent) is discharged to the ground or into a water course. When a central treatment facility is employed, the waste is typically referred to as "municipal wastewater" and usually consists of a combination of domestic and industrial waste. 1/ The statement of the wastewater treatment and disposal problem and the associated illustrations are adopted from a Sea Grant project document: F.A. Smith, et al., Fourteen Selected Marine Resource Problems of Long Island New York: Descriptive Evaluations, Report to the Marine Resources Council, Nassau-Suffolk Regional Planning Board, January, 1970 (Hartford, Conn.: The Travelers Research Corporation, 1970).

-42 -Domestic wastewater consists of complex organic matter which is subject to attack by micro-organisms that transform it to a stable form. Domestic wastes are heavily laden with disease-producing bacteria and viruses excreted by persons suffering from or "carryingll infectious diseases. When domestic waste is treated using septic tanks or cess pools, it is subsequently discharged to a ground water table, and, consequently, such wastes may eventually reach a surface water body. Since the transmission of wastes to surface water via the ground water table is circuitous, it is difficult to forecast quantitatively the ultimate effects on surface water quality. In contrast, the impairment of surface water quality resulting from direct wastewater discharges is more evident. The impact of such a discharge depends on the characteristics of the waste stream before treatment, the type of treatment; and the location of the discharge. Available treatment methods can accomplish everything from the simple removal of large solid materials to the production of water of highest quality suitable for human consumption. Typically, however, treated municipal waste discharges affect the following surface water characteristics: pH, turbidity, suspended solids, bottom deposits, concentrations of bacteria and viruses, dissolved oxygen, phosphorous compounds, nitrogen compounds, and the concentration of oxygen-consuming organic matter. Industrial wastes (and municipal wastes with significant industrial waste compounds) may also influence a number of additional water quality characteristics, such as concentrations of metallic ions and other chemical

-43- elements, depending on the composition of the industrial waste sources. Changes in these water quality characteristics in turn can significantly affect the types and quantities of biological organisms inhabiting the affected environment. Changes in the levels of environmental characteristics noted above can influence a large number of water-related activities which, in the case of Grand Traverse Bay, include swimming, boating, water-skiing, and commercial and sport fishing. The specific groups of people affected quite naturally include all those associated with these activities as direct participants, and may even affect many individuals who do not make direct use of the Bay insofar as they feel "less good" when the environment is degraded. The substantive contribution of the Delphi exercises is expected to be planning-oriented information concerning an issue that is extremely important to regional decision makers together with the identification of informational needs and the development of priorities for additional research in the Sea Grant program.

l: Il -44 -MARINE RESOURCE PROBLEMS AND OPPORTUNITIES FOR THE GRAND TRAVERSE BAY AREA Sport and Commercial Fishing Waste from Marine Vessels a. oil and gas pollution b. sanitary wastewater discharges c. overboard trash disposal Water Recreation a. quality b. quantity c. ancillary facilities d. conflict between residents and non-residents e. seasonal and daily variations in demand f. conflict between recreational and non-recreational uses g. conflict among different recreational uses (e. g., water skiing vs. swimming and fishing) Solid Waste Disposal a. households b. industrial c. commercial d. government units Thermal Pollution and Thermal Enrichment Oil Spill Pollution Chemical Pesticides a. households b, agriculture Fertilizers a. households b. agriculture Domestic and Industrial Wastewater Treatment and Disposal Exhibit 1. Marine resource problems and opportunities.

l a~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A I I I 6i * * -45 - ALTERNATIVES IN WASTE COLLECTION AND TREATMENT 1. REGIONAL VS. LOCAL COLLECTION AND TREATMENT 2. MUNCIPAL-INDUSTRIAL Spectrum. of combined colle ction/treatment possibilities 3. SEPARATE VS. COMBINED SEWERAGE SYSTEMS (I. E., SEPARATE SANITARY AND STORM SEWERS). 4. TREATMENT ALTERNATIVES a. Primary b. Secondary (conventional) (1) activated sludge (2) trickling filter (3) lagoons c. Secondary (with phosphate removal) d. Secondary (physicochemical —high level treatment) e. Tertiary (for use with secondary but not needed if you have physicochemical) DISPOSAL xt a. Spray irrigation b. Industrial or municipal reuse c. Discharge to receiving water Exhibit 2. Alternatives in waste collection, treatment, and disposal.

TREATMENT DISPOSAL LOCATION WATER CHARACTERISTICS Domestic Waste.,. I N Cesspool/ Septic Tank Muncipal treatment I t F:A':'-+ v-; / Groun Watel \' /'..'/ / / AFFE d r. pH Turbidity Suspended solids Bottom deposits Bacteria viruses Dissolved oxygen Phosphorous compounds Nitrogen compounds Oxygen consuming organic matte r SECONDARY; EFFECTS &.-m_~j, AFFECTED ACTIVITIES N -7c Swimming Water skiing Boating Sport fishing Conmmercial fishing "Aesthetics" W ater Supply Aquatic plants Fish L; Surface Water 1 - Industrial treatment I ] OTI:CTII HER FACTORS NG CHARACTERISTICS 0' I. Storm Runoff \ Recharge \ Basins ~~~~~.0 I ~/ ~ Surface Precipi runoff.tation Wind Other Waste Sources &

LEFT. The Grand Traverse Bay (see inset) serves as a model for the University of Michigan's Sea Grant Pilot Program. It provides a microcosm of the problems and possibilities encountered in Lake Michigan and the Great Lakes region. 'IbEAVER r 'ISLA.. ~. r ` i ' t. ( U 1,NP / (17,518) E AET S! '\ -. < '10.^r 1 ^. ~ Petoskey ^'"^CH&ALEVOl, k --- —i / c ' O! L —, [Ii (16, 131) AR AIIJRMf i 0 o o t t j askas (5, 726) - i o IKAIA<1SKA RIGHT. The ten counties making up the Northwest Michigan S<' Development District approximate ( the Grand Traverse Bay Watershed area. It is representative of many areas in the Great Lakes region where economic development and quality of life are closely related to water resource. in parentheses are 1970 census popula s. 'The- numbers preliniinarv ir L \. /, \ O, 3JC)j ition fitures., j BENZIE M 1 - A(19,788) /MANISTEE Manistee I (38, 3~c GRAlD TRAVERSE a. I -.4s rtmo~nn~l I -. I 0 (20, 678) a 0 0 o (7,001) tt.... I C.tv; W XO D CAdilta fXFORD igC M1SSAUKEE! '! I gJ_ Exhibit 4. The Grand Traverse Bay Region

9 APPENDIX C SOCIOECONOMIC DEVELOPMENT IN THE GRAND TRAVERSE BAY REGION The Northwest Economic Development District is a state planning region made up of ten counties and approximates the Grand Traverse Bay watershed area. (See Exhibit 4.) The major natural assets of the region are related to its water resources. It has hundreds of miles of Lake Michigan shoreline and numerous clear water inland lakes and streams. It has thousands of acres of pine and hardwood forest abundant in wildlife, areas of clean fine sand, and deposits of gravel, limestone, and salt. The whole district is characterized by glaciated land with high rolling hills, broad valleys, and plateaus. It centers on the 45th parallel and has a seasonal climate with warm pleasant summers and 1/ cold snowy winters.It is part of the Upper Great Lakes Region. This is an area of 119 counties in the northern parts of Michigan, Minnesota, and Wisconsin that has been identified as socially and economically disadvantaged, and has been the -/"Prologue for Accelerated Growth of Economy," a report of the Northwest Michigan Development District, Traverse City, Michigan, Oct., 1968. -48 -

i I i.4 -49 -subject of intense study and special funding by the federal government in an effort to reestablish social and economic development at a rate comparable to the rest of the country. The weaknesses in the economy of the Upper Great Lakes Region include -Small population -Slow population growth - High outmigration - Slow structural change in industrial employment - High unemployment - Low per capita income - Large low-income class 2/ - Small high-income class2 The Upper Great Lakes Region is characterized as a rural area without much potential for agricultural production. It has lagged behind the nation and the remaining parts of Michigan, Minnesota, and Wisconsin in economic development. Manifestations of this lag are the income gap, the steady outmigration, and the persistently high levels of unemployment. The region has seemed inaccessible and unproductive to business and the environment hostile to it.- The Grand o 2o 2gattelle Memorial Institute, "Industries Suited for the Upper Great Lakes Region,t a report by David C. Sweet, John M. Griffin, and Hal S. Maggied, Columbus, Ohio, 1970. -/Ibid., and "Development Strategies-Upper Great Lakes Region,"? Annual Report of the Upper Great Lakes Regional Commission, Washington, D. C., Jan., 1969.

-50 -Traverse Bay area, a subregion in the Upper Great Lakes Region, has a socioeconomic development history that approximately parallels that of the larger region. However, in the past ten years there have been some significant changes in the trends of the social and economic development of this region and it may be helpful, as a frame of reference, to compare the development of the Grand Traverse Bay area with the whole Upper Great Lakes Region, as well as with the state of Michigan and the nation. The economic history of the region shows that its natural resources have been exploited and depleted in turn by fur traders, loggers, and various forms of agriculture. Subsequent experimentation and management have enabled wood products and agriculture to regain a significant but not leading role in the area's present economic structure. The area has long been a favorite summering place and with the development of winter recreational facilitiesskiing and snowmobiling are the two of the fastest growing forms of recreation in the United States -and the introduction of coho and king salmon for sport fishing, it has strengthened its claim to be the most popular tourist area in Michigan, which in 1967 ranked as the fourth most popular vacation state in the nation. It now serves as a vacation land for people in the metropolitan areas to the south, and tourism and recreation, together with supporting services, consti tutes the leading economic activity of the region.

-51 - The region's highest population level, 164,000, was reached in 1910. With the depletion of the forests and the cessation of related jobs the population dropped sharply. By 1920 it was only 132,000, and it reached a low of 120,000 in 1930. Preliminary 1970 census figures reveal that approximately 155,000 people now live in the region-a density of less than 35 per square mile. The population growth during the 1950-60 period was 2.6 per cent, far lower than that of the state (24 per cent) or the nation (18.5 per cent). However, in the 1960-70 period the gap narrowed as the region's growth increased to 11.4 per cent for the decade, while the growth of the state anad the nation eased to approximately 13 percent. Since the region's growth rate increased steadily throughout the period it is not unreasonable to assume that its annual growth rate at the present time exceeds that of the state and nation. However, growth and population density are not uniform throughout the region. Grand Traverse County currently has approximately 25 per cent of the region's population. Its population increased by 14.5 per cent between 1960 and 1970. The Traverse City metropolitan area has a base population of 32,000 and has been designated the primary growth center for the region in predictions based on the variety and quality 4/ of the business and industry already established there.Its geographical location with access to Grand Traverse Bay, and the size of its tourist, commercial, and manufacturing rologue for Accelerated Growth of Economy." " Prologue for Accelerated Growth of Economy.T

-52 - activities make it an excellent focal point from which growth can be generated to radiate throughout much of the region and be reinforced by the secondary growth centers of Manistee, Cadillac, and Petoskey. These secondary centers of growth have populations ranging from 6,000 to 11,000, are strategially located, and their social and economic development show trends similar to that of Traverse City. During the summer months, with the influx of vacationers, the population of the area approximately doubles. These summer people require services and their property comprises a substantial portion of the lal l tax base. Many of the natural resource development projects must have the support of these people to be effective. Immediately to the south of the region are some of the fastest growing areas in the United States and there are some indications that the population growth of the region will increase as expansion gradually spreads north. One important indicator of possible future population growth is the net migration of people in and out of the area. The prime motivating force of migration is economic in nature, with quality of life gaining importance as a secondary motive. Between 1950 and 1960 the region experienced a negative migration rate of 9.3 per cent, compared to a net positive rate of 2.5 per cent for the state. However, the migration rate of the area in the past ten years has gradually reached the neutral position which it holds at present. Some of the reduction in net outmigration is due to the movement into the area of people of retirement age. A negative migration rate

-53 - has an adverse effect on a region since the people most likely to move out are the young, relatively well-educated and productive workers who have the widest range of job opportunities open to them. The negative migration rates of the past are reflected by an imbalance in age distribution; the proportion of the region's people in the older age group is higher than that of the state or nation and the proportion in the most productive age bracket correspondingly lower. The youngest and oldest age groups make the greatest demands on a community for social and welfare services such as education, recreation, and medical care and are least able to contribute to meeting the cost of these programs. The financial burden falls on the age group most adversely affected by outmigration. Indians make up the regiods only significant minority group and represent less than 2 per cent of the population. Among the various measures for determining the relative importance of any particular industry to the economy of a region, the level of employment is particularly meaningful to this area. In 1968 all of the ten counties in the region were qualified for special grants, loans, and technical assistance under the Economic Development Act on the basis 5/ of high rates of unemployment.-/ Historically, the region has had a narrow-based economy, resulting in economic instability when the resource supporting the economy was depleted. Substantial growth and diversification 5/ - Ibid.

-54 - have taken place in recent years and the region's economy has become less reliant on seasonal and low-wage industries. In 1967 the five economic sectors employing over 10 per cent of the region's work force represented 84 per cent of the region's employment. These were manufacturing 27 per cent, wholesale and retail trade 16 per cent, self-employed 15 per cent, government 14 per cent, and services 12.5 per cent (Table I). When compared to 1960, the most significant gains were in services, government, and manufacturing. Agriculture employed a stable 7.5 per cent of the work force but represents a more important portion of the economy than employment figures indicate because it is resource based and has a high proportion of value added in the final product that is sold. In addition, approximately 45 per cent of the product produced is over and above the needs of the people in the region and is exported. These experts bring money into the region or are used. to import other products. The manufacturing sector offers the greatest opportunity for economic growth in the region but also the greatest threat to environmental quality and the dependent tourism and recreational activity. Manufacturing was the largest employer in 1960 and 1967 with a relatively strong growth rate during that period. The ratio of employment in manufacturing for the region is below that of the state, but above that of the Upper Great Lakes Region. Available regional employment data

TABLE 1 Comparison of Employment by Major Industry, 1967 Upper United Great * G.T.B. States Lakes Michigan Region Agriculture, Forestry, Fishing 4.7 13.2 2.8 7.5 Mining.8 2.5.4 *0 Construction 4.0 2.8 4.9 3.8 Manufacturing 26.5 20.0 36.8 26.8 Transportation and Utilities 5.8 5.0 4.4 3.5 Wholesale and Retail Trade 18.4 16.2 18.5 15.5 Finance, Real Estate, Insurance 4.4 2.3 3.5 2.0 Services 13.5 9.8 11.5 12.5 Government 15.9 15.3 13.2 13.9 Unclassified 6.0 12.9 4.0' 14.5 TOTAL 100.0 100.0 100.0 100.0 Sources: Battelle Memorial Institute, "Industries Suited for the Upper Great Lakes Region," a report by David C. Sweet, John M. Griffin, and Hal S. Maggied, Columbus, Ohio, 1970. *Michigan Employment Security Commission, Bureau of Business Research Calculations. I un

-56 - for 2-digit SIC manufacturing industries accounts for over 85 per cent of manufacturing employment used to develop the ratios by economic sectors. These data show important changes in the industrial structure of the region. In the 1959-67 time period total manufacturing employment gained 43 per cent, from 9,307 to 13,278. Grand Traverse County accounts for approximately 25 per cent of the region s employment in manufacturing and experienced a growth rate of 45 per cent in this area over the same time period. Those industries within the region, which deviated significantly from the average growth rate, are shown in Table 2. In some industries it is necessary to go to the 3 -digit SIC code to gain an accurate impression of changing demand or technology. For example an impressive gain from 387 to 889 (130 per cent) in motor vehicles and equipment (SIC-371) was overwhelmed by a sharp drop in ship and boat building and repairing (SIC-373) resulting in the apparent poor growth performance in transportation equipment (SIC-37). The Upper Great Lakes Region historically has relied on natural-resource based industries. If paper and allied products are included in the category of industries dependent on natural resources, the total employment in this area was 55 per cent in 1960 and 51 per cent in 1967 of total manufacturing employment. Natural-resource based industries are not expanding as rapidly nationally as some of the fabricating 6/ and specialty industries. 6-Battelle Memorial Institute, Industries Suited for the Upper Great Lakes Region." I

TABLE 2 Employment in Largest Manufacturing Industries in the Region SIC Code Industry Emp loyment 1959 1967 Employment Change Number Per Cent Employment in Faster Growth Industries 36 Electrical equipment and supplies 27 Printing and publishing 35 Machinery except electrical 30 Rubber and plastics 33 Primary metals industry 314 276 699 247 544 1,576 528 1,614 519 1,101 1,262 302 915 272 557 402 137 131 110 102 I -3.n! Employment in Slower Growth Industries 28 Chemical and allied products 32 Stone, clay and glass products 37 Transportation equipment 23 Apparel and other textile products 26 Paper and allied products 24 Lumber and wood products 462 421 497 467 1,391 -- 1,498 840 877 621 643 542 580 20 34 25 Emp loymer Food and kindred products Fabricated metal products Furniture and fixtures It in Average Growth Industries -41 -30 37 37 22 38 264 231 11.3 3,971 -9 -6 3 4 4 7 1,452 848 370 1,716 1,079 433 19 27 35 43 Total Manufacturing Employment* 9,307 13,278 *Not the sum of individual columns since smaller industries in eluded in the total but not in the categories. Source: Bureau of Business Research, University of Michigan. the region are in

-58 -By comparison, this group of industries is much less important in the Grand Traverse Bay Region, where it constituted 32.5 per cent of total manufacturing employment in 1960 and 25.6 per cent in 1967. (See Table 3.) Agriculture (7.5 compared to 13.2 per cent) and mining (negligible compared to 2.5 percent) are other resource based segments which are less important in the Grand Traverse Bay area from the standpoint of employment than in the Upper Great Lakes Region. Table 4 indicates that the industries in which employment opportunities in the region are increasing most rapidly are generally those in which the wages are relatively higher. Table 5 shows that per capita income for the region was approximately half that of the national average in 1940, but that the income trend as a per cent of United States average per capita income has subsequently steadily improved relative to the nation, the Upper Great Lakes Region, and state of Michigan. Supporting factors for industrial development include transportation, recreation, environmental quality, communications, energy-generating utilities, water and sewer facilities, housing quality, investment capital, and institutional resources. Only a brief word about the most apparent problems and opportunities in these areas will be offered. Transportation includes both the facilities and processes involved in distribution of a product and acquisition of the factors involved in production. Except for natural resource based industries the regional manufacturers are at

a. I TABLE 3 Employment in Natural-Resource-Oriented Industries, Grand Traverse Bay Region 1959-60 Employment. Percent of Total Industry 1959 1967 1959 1967 Food and Kindred Products 1,452 1,716 15.7 12.9 Lumber and Wood Products 542 580 15.8 4.4 Stone, Clay, and Glass 497 467 5.3 3.5 Paper and Allied Products 621 643 6.7 4.8 Total Natural-Resource-Oriented Industries 3,112 3,406 32.5 25.6 All Manufacturing Industries 9,307 13,278 100.0 100.0 Source: Table 2. ~n s!

TABLE 4 Average Earnings Per Full-Time Employee Michigan Upper Great Lakes Region - _ _ _ - - -~ — 1 ^1 V -.~~-.111-~-.1 1 ~ P~19~-I___ --- 4W1 _Ls 1962.. 1967 Per Cent Per Cent Average Ind. U.S. Wage Average Average Average. Wage _ _ I Per Cent Per Cent Ind. U.S. Average Average Industry Agriculture Mining Con. Const. Manufacturing Food and Kindred Products Textiles Apparel Lumber and Wood Products Furniture Paper Printing* Chemicals Petroleum Rubber* Leather Stone and Clay Primary Metals* Fabricated Metals Machinery* Electrical Machinery* Transportation Equipment Instruments and Miscellaneous Transportation Transportation Services Utilities $1,544 7,012 4,708 4,433 0 2,719 4,672 3,769 6,358 7,310 6,888 4,500 4,735 3,949 6,112 5,517 5,051 5,704 3,824 4,864 4,150 6,890 6,598 88 116 80 85 0 76 114 81 106 123 100 60 82 104 107 80 80 87 62 68 75 101 107 30 137 92 86 0 53 91 73 124 142 134 87 92 77 119 107 98 111 74 95 81 134 128 $2,902 8,346 7,613 5,492 0 4,694 4,169 4,334 7,819 6,364 7,130 7,541 5,603 4,678 7,004 6,888 6,012 6,868 4,182 0 5,005 8,300 8,735 117 110 102 87 0 110 80 79 109 90 87 84 84 101 103 86 80 87 59 0 75 102 117 46 133 121 87 0 74 66 69 124 101 113 120 74 111 109 95 109 66 0 79 132 139 1 CD

i. * ~ Table 4 (Continued) -~ ---- iB~eE~WP~t~ ---- r 0ICC _!~ 1962 1967...____.. ^.. ^.u ^^ ^ ^ ^ ^ ^ u. -... ----f ~~~m *^4.^. ---------- ^,^, *.^ Industry Average Wage Per Cent Per Cent Ind. U.S. Ave rage Average Per Cent Average. Ind. Wage Average Per Cent eU.S. g Average - --- 4 R~r Finance, Real Estate, Insurance Services Hotels Personal Services Business Services Amusement Services Professional Services Government Federal Civilian State and Local TOTAL $4,758 2,898 2,028 4,955 3,438 2,387 7,318 4,296 $4,400 87 85 70 90 73 64 121 85 86 93 56 39 96 67 46 143 84 86 5,791 $3 583 2,060 5,233 4,219 2,690 7,893 5,417 $5,419 86 88 56 81 73 57 105 88 86 92 57 32 83 67 42 125 86 86 ______ ________________________________________________________________ A. - -----— ~ --- ——; --- —-------------- _? *Faster growing industries in the Grand Traverse Bay Region.

A v~ 1929 TABLE 5 Per Capita Personal Income 1940 1950 1959 313 993 1,466 683 1,714 2,264 382 1,090 1,608 592 1,496 2,161 1962 1965 1967 Region Michigan Upper Great Lakes United States 366 806 432 705 1,627 2,449 1,779 2,368 2,079 3,054 2,076 2,760 2,378 3,395 2,419 3,159 aO! Per Capita Relatives (Per Cent of National Average) Region 52 53 66 68 69 75 75 Michigan 114 115 113 105 103 110 107 Upper Great Lakes 62 65 73 75 75 75 77 Source: "Industries Suited for the Upper Great Lakes Region."

-63 - a disadvantage in competing with firms located cser to the major markets-disadvantages which may be offset by advantages in other factors such as cost and availability of labor. However, products with some combination of high value to unit weight, short transit time, and special service requirements are amenable to air transportation, in which case locational advantages are related to airport accessibility.: In such cases, the integrated industrial park-airport concept at Traverse City gives an area which is otherwise relatively poorly located from the standpoint of transportation definite advantages, for distribution of products and acquisition of the materials for production. Another important advantage is the convenience of workers and management to fly to and from business engagements and social, cultural, and recreational centers which is an inducement in recruiting personnel. Prospective industries and businesses are concerned not only with plant location advantages but also are placing an increased emphasis on community services and the quality of living conditions. The environmental quality of the region and its recreational opportunities not only provide the basis for the leading economic activity-tourism and recreation-but also aid in attracting other businesses and industries. The region faces some critical decisions in determining the tradeoffs between economic development and environmental quality.

I t -64 - One of the more insidious problems of the region is a deficiency of adequate housing. In some communities the shortage is severe enough to prevent workmen from moving into the area and to restrict business expansion. The 1960 census report indicated that only 25 per cent of the housing units were connected to public sewers and only 76 per cent had hot and cold running water. The 1970 figures have not been analyzed as yet but the difficulties in obtaining home loans throughout the nation in recent years together with the lack of individual security to gain loans, the limited resources, and the conservative lending practices of financial institutions within the region make it unlikely that the situation has improved dramatically. Hospital facilities are generally adequate, and at Traverse City and Petoskey the medical complexes are exceptional, both in quality and diversity of their staff and 7/ equipment.The lack of venture capital from financial institutions within the region is a restricting factor. Many potentially successful operations are never started and those that are financed and operated by "outsiders" are often treated with skepticism by the local people. Because educational institutions provide much of the training needed to create a productive labor force and are a factor in attracting desirable industries and services the - "Prologue for Accelerated Growth of Economy." ~:":

-65 -quality of its education is vital to the development of an area. In general, schools in the Grand Traverse Bay Area are well cared for and adequately staffed. Beyond the high school level there is Northwestern Michigan College at Traverse City and North Central Michigan College at Petoskey, both basically two-year vocational-technical schools. Northwestern Michigan College has recently established a Great Lakes Maritime Academy with a three-year program to train seamen specifically for Great Lakes shipping. Perhaps the most critical issue facing. the communities in the region is that of waste water treatment and disposal. Only half of the communities provide a sewer system. The use of inadequate septic tanks and outdoor privies in close proximity to lakes and streams present an added hazard to the environment when the population is more than doubled during the warm summer months. The present sewage system at Traverse City has a primary treatment plant able to pass peak flows of only 13 MGD (million gallons per day). Plant efficiencies are less than 20 per cent removal of suspended solids. During 1969 the state of Michigan's Water Resource Commission rated 64 industrial establishments in the area, of which 31 had less than adequate waste water controls. One of these was the Morgan-McCool cherry and apple canning plant which was reportedly dumping into the bay 547,000 gallons of waste every day during the canning season in July and August.8/ 8/"Traverse City's 'Shaggy' Waters Endanger Tourism," Chicago Tribune, Oct. 26, 1970.

-66 -New Opportunities Important technological and sociological changes are providing new opportunities for rural areas with limited agricultural potential in the Upper Great Lakes Region. Industries and workers are recognizing increasing diseconomies associated with urban locations. Many highly skilled people are giving added importance to the quality of the environment when making vocational and locational decisions. Interstate highway systems, better employment information systems, and a more mobile labor force make it easier for industries to locate away from urban areas. In an increasingly mobile and affluent society people are willing to travel greater distances for both work and recreational purposes. With imaginative planning and preparation, the Grand Traverse Bay Region can benefit socially and economically from these trends. Many of the region's characteristics that were once considered sources of weaknesses, such as low population density and remoteness from metropolitan areas, could become assets. If the younger segments of the population can be properly educated and trained and can be induced to remain in the region as the nucleous of a productive labor force, and if ways can be found to utilize the experience and talents of the older segments, the problems associated with present imbalances in the age structure of the region may become opportunities.

-67 - There is considerable evidence of pollution and deterioration in the water resources of the region. However, the people of Traverse City, in cooperation with five surrounding townships, have assumed the leadership in seeking solutions to the area's water pollution problems by initiating the development of a regional sewage treatment plan. The present sewage treatment plant will be expanded to meet flow capacities predicted for the year 1990 and will be modified to provide secondary treatment capable of removing 90 per cent of the impurities from sewage and industrial wastes, including 85 per cent phosphate removal. Once the plant is built, industries will be required to divert their wastes into the municiple treatment system or to treat it themselves to meet pollution control standards. Since 1967, the Economic Development Assistance Program has provided funds for, among other projects, the expansion of Antrim County and Traverse City Airports, water and sewer facilities in the village of Kalkaska, marinas in the cities of Petroskey, Manistee, and Northport, and dock facilities for the Great Lakes Maritime Academy at Northwestern Michigan College. There are many indications that the people in the region are acting to enhance the positive aspects of the region. Hunting, skiing, and snowmobiling are being added to tourist and recreational areas to complement their summertime attractions and provide a year-round business. There appears to be widespread public support of the excellent health care

-68 -facilities at Petoskey and Traverse City and the money brought into the region by outsiders who use these facilities could provide an important source of strength for social and economic development throughout the region. Short-field landing and takeoff capability in general and commercial aircraft could ease the problems associated with inadequate access roads to recreational areas. Mobile and manufactured homes could be a partial solution to the shortage of adequate housing in the region. Summary This region has approximately 155,000 residents and is Michigan's most popular area for recreation and tourism. However, in recent years it has been considered by the federal government as part of a socially and economically depressed area. Early in- this century the region was characterized by vigorous economic activity and growth which, however, suffered a sharp decline shortly after 1910. Only recently has the region recovered a population and industrial growth comparable to the rest of the state and the nation. The following have been major causes of the economic decline of the region: -The displacement of labor in resource based industries because of technological change - The deterioration of resources-fur,- lumber, and agriculture-and increased competition

-69 - from other areas that supply the same resources - The small size of the region's enterprises - Remoteness from metropolitan markets As a direct result of the economic decline, there has been a migration from the region of people in the most productive age group and of those with the most marketable skills. This has created an imbalance in the age and skill levels of the population. Chronically high unemployment rates have been experienced due to the lack of match between job opportunities and the characteristics of the labor force, the scarcity of job opportunies, and reliance on seasonal economic activities. The long period of lagging economic opportunity and growth and the low density of population has tended to separate the region from the mainstream of national, social, and economic development. This has had a cumulative effect on the supporting activities needed to sustain economic growth and has been reflected in such things as inadequate housing, lack of investment funds, outdated waste water treatment and disposal systems. In recent years there have been encouraging signs of favorable trends in some of the key factors used. in measuring the region's economic and social health and its potential for development. Based upon the region's strong growth in employment between 1959 and 1967 and a shift to faster growing industries, it is likely that employment growth rates will exceed that of the Upper Great Lakes Region and the United States in the next decade. A

-70 -There are also some important new technical and social forces that will tend to make the Grand Traverse Bay Region increasingly attractive for new businesses, industries, homes, and recreational activity. At the same time there is encouraging evidence that the people of the area are actively concerned not only with the economic and social development of their region but also with the management of the water resources and environmental qualities which are the region's main source of economic potential. They are fortunate to live in a region where flexibility is still possible in making the critical decisions concerning the trade-offs between economic development and environmental quality.

APPENDIX D PARTIC IPATION LETTER Dear (Regional Influential): You are invited to participate in an exchange of information with the researchers in the Sea Grant Program of the University of Michigan. From this exchange, forecasts of the impact of technological and social changes on the Grand Traverse Bay area (see the attached Exhibit 5 for a more precise definition of the area) will be developed. The aims of the Sea Grant Program of the University of Michigan are: (1) to establish standards of expertise and a center of knowledge for Great Lakes research, and (2) to provide useful information as a service to regional planners and decision makers. The Grand Traverse Bay area was selected because it is representative of many areas in the Great Lakes region where economic development and the quality of life are closely related to water resources. Although a considerable amount of empirical and scientific data is being gathered and used as a basis for projections, the judgments of informed people form the best source of insight into the future. To obtain informed judgments systematically, the Sea Grant Program will use the Delphi techniques developed by the researchers of the Rand Corporation as part of their study of methods for improving decision making. The method is designed to obtain the most reliable consensus of opinion from anonymous responses by a group of experts to a carefully designed program of sequential interrogations. The exchange of information is accomplished through a series of information packages referred to as rounds. The early rounds are controlled brainstorming sessions in which the respondent is encouraged to present what he feels will be important technical and social developments in the future. The responses are edited to eliminate duplication or ambiguous items, and are then combined and fed back to the group. In subsequent rounds the respondents will be asked to attach numerical estimates to each development indicating its probability, timing, importance and impact on the Grand Traverse Bay area. Self-appraisal indexes will permit the respondent to indicate his relative competence on specific issues and his familiarity with the region. -71 -

LEFT. The Grand Traverse Bay (see inset) serves as a model for the University of Michigan's Sea Grant Pilot Program. It provides a microcosm of the problems and possibilities-encountered in Lake Michigan and the Great Lakes region.!i ^ / *i I t bEAVER SA EEM-ET!..~~~~~~~~~~~~~6... \. RIGHT. The ten counties making up the Northwest Michigan s'^ ' Development District approximate ( the Grand Traverse Bay Watershed area. It is representative of many areas in the Great Lakes region where economnic development and quality of life are closely related to water resources. p. M[ANISTEE Manistee..- '.' E.., -r -. u s g8Betlare 0 o AHTRIM KALKASKA i 0 0 0 C. I i I ct~ I E WEXFORD MISSAUKEE - - — I Exhibit 5. The Grand Traverse Bay Region

-73 -After the second round, each participant will receive a copy of his past response with a statistical summary of the values assigned by the entire panel and by various subgroups within the panel. Information obtained from research in the Sea Grant Program and from a Delphi panel of technical experts will also be provided as background information to assist the participants in the formulation of their individual judgments. Experiments have shown that feedback and reassessment often result in the convergence of opinions as common elements of judgments are reinforced, ambiguities resolved, extreme positions clarified, and the judgments formulated and refined as the interrelationships of events and the weighted opinions of experts in other areas are considered. However, the feedback and reassessment of informed judgments should be tremendously valuable even if they do not lead to convergence. An important feature of the Delphi method is that it does not require elaborate development and support of positions taken on each issue. After the panel has established which items are to be considered these will be continuously narrowed until only those items are left which have been judged important to the exercise and on which a satisfactory consensus has not been reached. In assessing the likelihood of social developments, quantitative probability estimates give an exaggerated impression of precision. You are requested to complete the form on probability assessments so that an ordered list of verbal phases that intuitively express the notion of numerical probability can be developed for your panel. To insure anonymity each respondent will be identified by a number —which will be linked to his name only for the purposes of mailing out information packages and checking in the responses. Please feel free to request any additional informationeither by telephone (313/764-1366) or with your response. A final summary and a complete analysis of the exercises will be sent to each respondent upon completion of the final round of information packages —expected to be round four. A seminar will be held at the University of Michigan to review the output of the Delphi exercises.

-74 -It is requested that you respond to this letter as soon as practical but not later than The procedures are sufficiently flexible that you may miss a round and still be represented in the final results. John D. Ludlow Research Associate Bureau of Business Research Graduate School of Business University of Michigan

BIBLIOGRAPHY Books Brech, Ronald Britain 1984: Unilever's Forecast —An Experiment in Economic History of the Future. London: Darton, Longman and Todd, Ltd., 1963. Bright, James R,, ed. Technological Forecasting for Industry and Government, Englewood Cliffs, N. J.: Prentice-Hall, Inc., 1968. The following articles were particularly useful: Gordon, T.J. "New Approaches to Delphi, pp. 134-43. Jantsch, Erich. "Integrating Forecasting and Planning through a Function-Oriented Approach," pp. 426-48. North, Harper Q., and Pyke, Donald L. "Technology, the Chicken —Corporate Goals, the Egg," pp. 412 -25. Jantsch, Erich. Technological Forecasting in Perspective. Paris: Organization for Economic Cooperation and Development, Apr., 1967. Kahn, H., and Wiener, A. J, The Year 2000: A Framework for Speculation on the Next 33 Years. London: CollierMacmillan, Ltd., 1967. McGregor, Douglas. The Professional Manager. New York: McGrawHill Book Company, 1967. Journal Articles Bright, James R. "Evaluating Signals of Technological Change." Harvard Business Review (Jan. -Feb., 1970), pp. 62-70. -75 -

-76 -Journal Articles -- Continued Helmer, Olaf, and Dalkey, Norman. "An Experimental Application of the Delphi Method to the Use of Experts." Management Science, IX (Apr., 1963). Helmer, Olaf, and Rescher, Nicholas. "On the Epistemology of the Inexact Sciences. " Management Science, VI (Oct., 1959). Martino, Joseph P. "An Experiment with the Delphi Procedure for Long-Range Forecasting." IEEE Transactions on Engineering Management, EM-15 (Sept., 1968), 138-44. North, Harper Q., and Pyke, Donald L. "Technological Forecasting in Planning for Company Growth. " IEEE Spectrum, VI (Jan., 1969), 30-36. Winkler, Robert L. "The Quantification of Judgment: Some Methodological Suggestions." Journal of the American Statistical Association, LXII (Dec., 1967). _ "The Consensus of Subjective Probability Distributions." Management Science, XV (Oct., 1968), 61-75. ~___ __... "The Assessment of Prior Distribution in Bayesian Analysis." Journal of the American Statistical Association, LXII (Sept., 1967). Papers Reproduced by the RAND Corporation as a Courtesy to Its Staff Brown, Bernice, and Helmer, Olaf. "Improving the Reliability of Estimates Obtained from a Consensus of Experts." P-2986, Sept., 1964. Dalkey, Norman C. The Delphi Method: An Experimental Study of Group Opinion. Memorandum RM-5888-PR. Santa Monica, Calif.: RAND Corp., June, 1969. Gordon, T. J., and Helmer, Olaf. "Prospects of Technological Progress." P-3643, Aug., 1967.

-77 - Papers Reproduced by the RAND Corporation as a Courtesy to Its Staff —Continued "Systematic Use of Expert Opinions." P-3721, Nov., 1967. Helmer, __. "Report on a Long-Range Forecasting Study." P-2982, Sept., 1964. Olaf. "Analysis of the Future: The Delphi Method." P-2558, March, 1967. Series of Reports Issued by the Institute for the Future Presenting the Findings of Studies on the Development of Long-Range Forecasts of Technological and Social Events de Brigard, Raul, and Helmer, Olaf. "Some Potential Societal Developments:.1970-2000." IFF Report R-7. (To be published. ) Enzer, Selwyn, and de Brigard, Raul. "Issues and Opportunities in the State of Connecticut: 1970-2000." IFF Report R-8, March, 1970. Enzer, Selwyn; Gordon, Theodore J.; Rochberg, Richard; and Buchele, Robert. "A Simulation Game for the Study of State Policies. IFF Report R-9, Sept., 1969. Gordon, Theodore J., and Ament, Robert H. "Forecasts of Some Technological and Scientific Developments and Their Societal Consequences. IFF Report R-6, Sept., 1969. Helmer, Olaf; Gordon, Theodore J.; Enzer, Selwyn; de Brigard, Raul; and Rochberg, Richard. "Development of Long-Range Forecasting Methods for Connecticut: A Summary. IFF Report R-5, Sept., 1969.

-78 - Working Paper Numbe r 1 2 3 4 5 - 6 7 8 9 10 OTHER WORKING PAPERS "Reflections on Evolving Competitive Aspects in Major Industries," by Sidney C. Sufrin "A Scoring System to Aid in Evaluating a Large Number of Proposed Public Works Projects by a Federal Agency," by M. Lynn Spruill "The Delphi Method: A Systems Approach to the Utilization of Experts in Technological and Environmental Forecasting," by John D. Ludlow "What Consumers of Fashion Want to Know," by Claude R. Martin, Jr. —Out of print. To be published in a forthcoming issue of the Journal of Retailing. "Performance Issues of the Automobile Industry," by Sidney C. Sufrin, H. Paul Root, Roger L. Wright, and Fred R. Kaen "Management Experience with Applications of Multidimensional Scaling Methods," by James R. Taylor "Profitability r-nd Industry Concentration," by Daryl Winn "Why Differences in Buying Time? A Multivariate Approach, " by Joseph W. Newman and Richard Staelin —Out of print. To be published in a forthcoming issue of the Journal of Marketing Research. "The Contribution of the Professional Buyer to the Success or Failure of a Store, " by Claude R. Martin "An Empirical Comparison of Similarity and Preference Judgments in a Unidimensional Context, " by James R. Taylor

-79 - Working Paper:Number 11 1 - "A Marketing Rationale for the Distribution of Automobiles," by H. O. Helmers 12 ": Global Capital Markets, by Merwin H. Waterman 13 "The Theory of Double Jeopardy and Its Contribution to Understanding Consumer Behavior, by Claude R. Martin 14 "A Study of the Sources and Uses of Information in the Development of Minority Enterprise —A Proposal for Research on Entrepreneurship, " by Patricia L. Braden and Dr. H. Paul Root 15 "Program Auditing," by Andrew M. McCosh 16 "Time Series Forecasting Procedures for an Economic Simulation Model," by Kenneth 0. Cogger 17 "Models for Cash Flow Estimation in Capital Budgeting," by James T. Godfrey and W. Allen Spivey 18 "Optimization in Complex Management Systems," by W. Allen Spivey 19 "Support for Women's Lib: Management Performance, " by Claude R. Martin 20 "Innovations in the Economics of Project Investment, " by Donald G. Simonson 21 "Corporate Financial Modeling: Systems Analysis in Action," by Donn C. Achtenberg and William J. Wrobleski