T 'I, Division of Research Graduate School of Business Administration The University of Michigan July 1981 FORECASTING EXPERIENCES OF MAJOR U.S. ECONOMETRIC MODELS IN THE 1970's Working Paper No. 271 W. Allen Spivey William J. Wrobleski The University of Michigan FOR DISCUSSION PURPOSES ONLY None of this material is to be quoted or reproduced without the express permission of the Division of Research.

I INTRODUCTION Many of the difficulties arising in the appraisal of econometric model performance have been highlighted by a number of researchers. During the last 25 years many papers and books have been written by both individuals and teams of econometric modellers in which attempts have been made to evaluate the forecasting performance of econometric models. Unfortunately, the practitioner, whether economist, government official, or businessman, is still up in the air as to the accuracy of econometric models in short-term forecasting. Why is this true? Problems of evaluating short-term forecasts have persisted for two reasons. First, the many evaluative studies have provided only simple descriptive statistics of forecast errors (such as root mean squared errors and mean absolute errors) calculated from small data sets. The largest data set of ex ante forecasts used in the literature consists of at most 20 one-quarter ahead forecast observations (McNees, 1975). Second, we lack an adequate framework for the necessary step of going beyond these purely descriptive measures to statistical inferences made from the small samples of time series of correlated forecast errors of econometric models that are currently available. The importance of the second point has not been fully appreciated even by economists, and an understanding of it has been ill-served by econometric modellers, who have been content to serve up large masses of root mean squared error calculations supported only by

- 2 - brief narrative comments. The Fromm and Klein paper (1976), which presented forecasting results from 11 econometric models, is an example of a study which presents a large number of root mean squared error calculations. In Table 1 we reproduce root mean squared errors from this paper for ex post forecasts of two econometric models which were calculated for a 1 through 6 quarter ahead extrapolation period. Table 1 Root Mean Squared Errors for Forecasts of Real Gross National Product Quarter Ahead of Extrapolation Model 1 2 3 4 5 6 Wharton Mark III 5.02 12.93 17.96 19.35 21.24 21.55 BEA 3.51 9.05 11.54 11.02 8.42 6.83 Source: Fromm and Klein (1976), Table 2. Looking at the 5 quarter ahead root mean squared errors for the Wharton Mark III and the U.S. Department of Commerce's BEA model, to take an example, one sees that the difference between the corresponding error measures is 21.24 - 8.42 = 12.82. If a different extrapolation period were to be examined for these models and the same number of 5 quarter ahead forecasts were to be generated, it should be apparent that the difference between the new root mean squared errors might be larger or smaller than 12.82. The latter includes the possibility of a negative difference which would mean, of course, that the Wharton model would have a smaller root mean squared error than the BEA model for the alter

I - 3 - nate period. Because differences would be expected to vary in numerical magnitude over different forecast periods of a given number of quarters ahead, a satisfactory assessment of forecasting performance clearly requires that we assess the significance of an observed difference such as 12.82. This in turn requires that one know the sampling distribution of such differences. One could then test the null hypothesis that the expected root mean squared errors for the two models are the same against an alternative that they differ, and still other inference procedures could be employed. It could be the case, for example, that a difference of 12.82 is less than one standard error, or that all the pairwise differences in the paper of Fromm and Klein are not significant. Practically speaking this would mean that there is no way of choosing between the 11 econometric models examined in the paper, at least in terms of forecasting performance. Attempts to compare root mean squared errors of the kind appearing in the forecast evaluation literature without recourse to statistical inference corresponds to attempts of experimenters of 100 years ago and earlier to assess the difference between sample means in the absence of knowledge of the sampling distributions of the statistics involved. The unhappy fact is that forecast error results vary from one economic variable to another, from one time period of given length to another, and from model to model. At the present time we have no analytical framework for assessing in probability terms the magnitude of observed differences in root mean squared errors.

- 4 - The difficulty of developing the requisite sampling distribution theory for root mean squared errors is deeper than that for sample means which faced yesterday's experimenters because forecast errors of econometric models are by their nature correlated observations. Econometric model forecasts are generated by a sequential process which induces various dependencies in the forecasts. Forecasts of the endogenous variables for a given quarter t + T in the future, for example, depend on the forecasts of the endogenous variables made by the model for quarters t, t+l, -, t+T-l as well as on the forecasted future values of the exogenous variables for these quarters made by the model builder, the latter being determined outside the model itself. In addition, in ex ante forecasting subjectively determined "add factors" are also used. These are modifications in the constant terms of the various equations made by model builders when an examination of forecast errors generated by the model appears to indicate the presence of serial correlation in the errors or a change in the mean of the errors. Either of these influences may indicate that the model should be respecified and/or the entire model reestimated. Because these preferred appraoches may be too difficult or too costly, model builders often temporarily resort to the expedient of partially compensating for these influences by adjusting constant terms by means of add factors. Finally, the actual ex ante forecasts that are released to clients are further modified subjectively by model builders in ways which may also contribute to dependence between forecasts for

- 5 successive periods. In any case, model builders often override the model forecasts and incorporate their own assessments of future economic developments into them. All of these influences produce a forecast error history in which complicated correlation structures are present. Such structures, together with the small sample size of forecast error data sets currently available, require that an exact small sample distribution theory for the correlated forecast errors be used to interpret root mean squared errors. Moreover, for the very small sample sizes of econometric forecasts currently available, any estimators of forecast accuracy based on such a theory must have low precision and tests of hypotheses will have low power. At the present time, we lack the necessary small sample theory and the small forecast error data bases prevent one from using existing large sample (asymptotic) inference procedures. Thus, any analysis we engage in must be descriptive in nature and refer only to given time periods, given economic variables, and models, and we must keep in mind that if variables or time periods are changed, then measures of forecast performance will also change. NEW LOOKS AT ECONOMETRIC FORECASTS FOR THE 1970-75 PERIOD What does this mean for the practitioner who wants to gain some understanding of comparative econometric model forecasting performance? One must use descriptive measures and avoid making unwarranted generalizations. Unfortunately, even when using such measures as root mean squared errors or mean absolute errors in a

- 6 - limited "track record" sense, these measures can be difficult to interpret. Moreover, because these measures are averages, they can mask time dependent variations in forecast errors which can be of importance in assessing forecast performance descriptively. We introduce a simple graphical procedure which enables one not only to visualize individual forecast errors easily but to form impressions of what we call the composite forecast path through time of an econometric model. This procedure supplements numerical descriptive measures and can have important uses in descriptive analyses of the forecasting performance of econometric models. We now consider the forecasting performance over 1 through 8 quarter ahead forecast horizons in the 1970's of the Data Resources Inc. (DRI) model and of the Wharton Mark III and Chase Econometric models. Figs. l(a), l(b), and l(c) respectively display 1, 4, and 8 quarter ahead forecasts of current dollar or nominal GNP made by the DRI model. The forecast period extends from the third quarter of 1970, denoted 1970.3, through the second quarter of 1975, denoted 1975.2. In other words, the first of the 1 quarter ahead forecasts was made in 1970.2 for the quarter 1970.3 and the last of these was made in 1975.1 for the quarter 1975.2. There are 20 such 1 quarter ahead forecasts for each of the three models, although Fig. 1 displays forecasts for the DRI model only. It is important to realize that Fig. l(a) shows a sequence of 1 quarter ahead forecasts each of which has a different base in terms of its quarter of origin. That is, Fig. l(a) shows a sequence

I I I - II I I II I r I 11 I I/! i xja.1 -0,/ / /.4 t.1 J* I I I II /! I I -- fc.a,- - rwoCMt _- - CARTL --- nnccftis _ rCTfL -_. -fontW n 71 72 73 yiwt 74 7 7i n n l(b). 4 Quarter Forecasts 74 7a Ahead 1 (a). 1 Quarter Ahead Forecasts 71 72 n 74 71 l(c). 8 Quarter Ahead Forecasts Fig. 1. Forecasts of Current Dollar GNP Made by the DRI Model

- 8 - of 20 different, 1 quarter ahead forecasts originating from a sequence of advancing bases. The three graphs in Fig. 1 thus display forecast performance across time for the corresponding number of quarters ahead. Because each successive forecast is made on the basis of all information available to the model builder as the base quarter advances, the forecasts reflect the accumulation of information and hence the sequence of forecasts can be called a composite forecast path. Fig. l(b) shows 4 quarter ahead forecasts of current dollar GNP. Here the first forecast is for 1971.2 and was made in 1970.2. The last 4 quarter ahead forecast was made for 1975.2 and was made 4 quarters earlier, i.e., in 1974.2. There are 17 such 4 quarter ahead forecasts in our data base. Finally, Fig. l(c) displays 8 quarter ahead forecasts. Again, the first of these is for the quarter 1972.2 and this forecast was made 8 quarters earlier, or in 1970.2. The last of these forecasts was made in the quarter 1973.2 and is, of course, a forecast for 1975.2. There are 13 forecasts for 8 quarters ahead for each of the 3 models. It should be emphasized that all forecasts here are the ex ante forecasts actually released to clients. These forecasts required not only the use of forecasts or subjectively determined values of exogenous variables in the respective models, but they also incorporated further and possibly substantial judgmental modifications of an ad hoc nature by model builders. Although these forecasts resulted from the interactions of forecasters with their models and could be regarded as forecasts of the model builders

- 9 - themselves, we will refer to them simply as model forecasts. Our data base of forecasts includes 1 through 8 quarter ahead forecasts of 10 macroeconomic variables for the DRI, Wharton, and Chase models. The macroeconomic variables are: current dollar and real or constant (1958) dollar GNP, the implicit GNP price deflator (in 1958 dollars), nonresidential fixed investment, residential fixed investment, change in business inventories, personal consumption expenditures for durable goods and for nondurable goods and services, net exports, and unemployment rate. This data base consists of a total of 3960 forecasts and for convenience in exposition we must select only a small number of results to present graphically. Moreover, the choice of the models discussed for the variables selected for discussion has been done without design, and as we point out below, similar results would hold for other model choices. Returning to Fig. l(a), one sees that current dollar GNP is forecasted well for 1 quarter ahead by the DRI model. These forecasts show a slight tendency to underforecast then overforecast, with the under and over estimates tending to alternate with one another. A shallow turning point in actual current dollar GNP occurred in 1974.4, which was followed by a sharp reversal in 1975.2. Both of these developments were missed by the 1 quarter ahead forecasts for these periods. The 4 quarter ahead forecasts, displayed in Fig. l(b), show a persistent tendency to underforecast the actual values until early 1975, when they overshoot the actual values by large amounts. The

- 10 - underforecasting not only continues for the 8 quarter ahead forecasts in Fig. l(c) but is of much larger amounts. Because of the turning point and its subsequent reversal in 1975.1, the strong underforecasting tendency of the 8 quarter ahead forecasts appears to result fortuitously in smaller overforecasting errors for 1975 than is the case for the 4 quarter ahead forecasts. Thus while the 8 quarter ahead forecasts continued to rise almost linearly throughout 1974 and the first 2 quarters of 1975, the turning point occurring in 1975.1 produced a no growth or "sideways" displacement in actual GNP. The timing and duration of the latter as well as the sharp recovery of GNP after 1975.1 almost compensated for the large and persistent underforecasting that had been occurring previously, thus bringing the actual values of GNP for 1975.1 and 1975.2 closer to the forecasted values for these 2 quarters. An examination of the graphs for each of the 1 through 8 quarter ahead forecasts -- of which only three are shown in Fig. 1 in order to save space -- indicates that the tendency to persistently underforecast current dollar GNP begins with the 3 quarter ahead forecasts. In addition, the amount by which GNP is underforecasted increases with the number of quarters ahead of the forecasts. These findings on current dollar GNP forecasting are consistent with those of Zarnowitz (1978) mentioned earlier. Figs. 2(a), 2(b), and 2(c) display 1, 4, and 8 quarter ahead forecasts of GNP in 1958 dollars -- constant dollar or "real" GNP -- for the DRI model. This figure affords an interesting contrast to Fig. 1. The 1 quarter ahead forecasts of real GNP are good through 1973 but, unlike similar forecasts of current dollar GNP,

1-01tft-ORl I-Glo! u Ui II It I \ \\ I b i; I rl it iI t -_ IUIL - I- rorC CA1 T ___ CUHL - -- rftu= S T1 T - ACTLP - - - f TUMCLR 71 72 7) TCM 74 71 71 72 7) TYM 2(b). 4 Quarter Forecasts h4 ad Ahead 71 72 7 74 71 Ti[M 2(c). 8 Quarter Ahead Forecasts 2(a). 1 Quarter Ahead Forecasts Fig. 2. Forecasts of Constant Dollar or Real GNP Made by the DRI Model

- 12 - begin deteriorating after 1973.1, with a tendency to overforecasting beginning after 1974.1. The 4 quarter ahead forecasts of real GNP consistently underestimate through 1973.2, at which time they begin to overestimate by large and varying amounts. In Fig. 2(c) the 8 quarter ahead forecasts of real GNP show a continuing and nearly linear increase until 1975.1, when a downturn is forecasted, although an actual increase in real GNP occurred for that quarter. Thus large overforecasting errors result for 1973 and beyond and, it should be noted, the 8 quarter ahead forecasts miss 5 turning points. Indeed, these forecasts of real GNP show little correspondence to the actual values. It may be recalled that the period from 1974.4 through 1975.1 was part of a severe recession in the United States, in the course of which a decline of 6.6 percent in constant dollar or real GNP occurred from the period extending from 1973.4 through 1975.1. In contrast, current dollar GNP first began to decline in 1974.4 as can be seen from Fig. 1. Forecasts of the GNP implicit price deflator for the Wharton Mark III model are displayed in Figs. 3(a), 3(b), and 3(c). The 1 quarter ahead forecasts appear to be fairly accurate with some tendency toward underforecasting, but Figs. 3(b) and 3(c) show sharp increases in amounts of underforecasting of inflation effects with variations in the amount of underforecasting over time. The 8 quarter ahead forecasts in particular show such a pattern with considerable wandering or drifting away from the actual values on the

I-tr5s4lMW W-OFTS TDW 5"-4Ufilunow 8 81 IT I I - [1 I E b II I I b It i. 2' 2 I I i f I I iJ 23 T ___TLL -. rwftIws - -UL _ --- rwows T - fK7cfl - - - FOMEDW I 71 72 71 XtMf 24 7n 71 12 7 n n~~~~TU 74 75 In 72 n 74 w 3(a). 1 Quarter Ahead Forecasts 3 (b). 4 Quarter Ahead Forecasts 3(c). 8 Quarter Ahead Forecasts Fig. 3. Forecasts of the GNP Implicit Price Deflator Made by the Wharton Mark III Model

- 14 - low side. This is accompanied by erratic, turning point type movements. These graphs also confirm Zarnowitz's findings concerning the persistent tendency of econometric models to underforecast inflation (price) effects. Some economists interpret such results as further evidence of the shortcomings in capturing price influences in an economic system of the Keynesian type ISLM theoretical framework, on which these econometric models are based. In any case, it should be noted that 1973 was a year of rapid acceleration in the U.S. price level, resulting from sharp increases in demand for output and from the repercussions from world commodity price movements and the oil embargo. How well do these models perform in forecasting another variable of importance in policy deliberations, the unemployment rate? Forecasts of this variable by the Chase model are shown in Figs. 4(a), 4(b), and 4(c). The 1 quarter ahead forecasts follow the shape of the movements in the actual unemployment rate fairly well, although the actual rate is slightly underestimated most of the time. The model forecasts are, by and large, too optimistic even on a 1 quarter ahead basis. Passing to the 4 quarter ahead forecasts, one sees that these do not track the actual rates well; the 8 quarter ahead forecasts in Fig. 4(c) "flip-flop" back and forth and appear to wander aimlessly, unrelated to the actual unemployment rates. If one were to examine the 1 through 8 step ahead forecasts of the unemployment rate for each of the three models (only the 1, 4, and 8 quarter ahead forecasts for the Wharton model are shown here), one would find that sharp deteriorations in forecast quality begin after two quarters for each model. This feature may suggest that

I-ft'J-COLS I I 4 i? i I I i I,~~~~~~~~~~~~~~! / "" ",,._': / '~~~~~~~~~/ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I —:-l.-\/ ( 0 1 1 II I I I I I I f I~~~~~~I % —. I -~~~~ i I % I I I I % I r \ / I \ I % I \ % I _ '_\.. Ii I I h L W) -___ CTUf -—.. rWtcA51 la sArsbM '-V-.. -- 71 72 7n TEn 74 7 i - - 7) 72 72 T(M 74 75 7; 72 72 trm 74 71 4(a). Quarter Ahead Forecasts 4(b). 4 Quarter Ahead Forecasts 4(c). 8 Quarter Ahead Forecasts Fig. 4. Forecasts of Unemployment Rate Made by the Chase Model

- 16 - the present econometric modelling approach to unemployment, which places considerable emphasis on the use of the Phillips curve assessment of the trade-off between inflation and unemployment, is inadequate for other than the very short-term, capturing at best only the temporary trade-offs between these variables. Friedman (1967) asserted, for example, that there is only a temporary trade-off between inflation and unemployment and that there is no permanent trade-off. Moreover, he stated that the temporary trade-off comes not from inflation, but rather from unanticipated inflation, which usually takes the form of unanticipated increases in inflation rates. Thus to improve the dismal forecasting of unemployment rates by the major U.S. econometric models for more than 2 quarters into the future, it may be necessary to supplement the use of a Phillips curve by an approach based on the natural rate of unemployment hypothesis advanced by Friedman and others in which rational expectations about future inflation effects play a central role. Readers interested in international economic problems may be curious about the forecasts of U.S. net exports made by these econometric models (net exports are defined as the difference between total exports and total imports). The 1, 4, and.8 quarter ahead forecasts of net exports made by the Chase model are shown in Figs. 5(a), 5(b), and 5(c). The 1 quarter ahead forecasts, unlike those for current dollar and real GNP, are not good and they appear to be "out of phase" by 1 or 2 quarters, leading the actual values for the initial years of the forecast horizon and then lagging them for the later years. Again the 4 and 8 quarter ahead forecasts show "I

I-4ox - tl" 1l '4 R ' I I I i 8, I. A. /" I it n 7/ IE - rCRflA. - - tICfCR3i --- CL.... fsvIe?!I - 4flAIL..t ) 't, tl,~l 1',' 5(a). 1 Quarter Ahead Forecasts 5(b). 4 Quarter Ahead Forecas ts 5(c). 8 Quarter Ahead Fig. 5. Forecasts of Net Exports Made by the Chase Model

- 18 - the "wandering" movements observed earlier with very little correspondence to actual net exports being shown for either forecast set. This performance may be an indication that the international sector of these econometric models was given less attention over this period by the model builders than other sectors, international movements perhaps being regarded as having much less importance for the U.S. economy. The growing economic interdependence of western nations and the increasing relative importance of the international sector in the U.S. economy can be expected to cause domestic model builders to reassess their approach to modelling this sector. As previously indicated, we have chosen to present discussions of only a small subcollection of forecasts from the parent set of 3960 forecasts that are available for the period under consideration. If one were to examine all these forecasts for the 10 macroeconomic variables in the data base that we have available for the 3 models, one would see that forecast accuracy deteriorates for forecasts of 3 or more quarters into the future for each of the models and for each of the variables. This deterioration in accuracy also increases, of course, with the increase in the number of quarters ahead of the forecast. We thus conclude that for these models and for the time period studied, forecasts were reliable only for 1 or 2 quarters into the future. SIMILARITIES IN FORECASTS OF THE MAJOR U.S. ECONOMETRIC MODELS A question that is often of interest to the consumer of forecasts is: are there similarities in forecasts across econometric

- 19 - models for a given quarter ahead and for given macroeconomic variables? Previous studies have attempted to deal with this question by presenting large sets of root mean square errors for individual models. These are difficult to interpret when one wishes to assess forecasting performance across models and they can be helpfully supplemented by the simple graphical analyses to which we now turn. Figs. 6(a), (b), (c), and (d) display the 1, 4, 5, and 8 quarter ahead forecasts of the 3 models for current dollar or nominal GNP. In Fig. 6(a) one sees that the 1 quarter ahead forecasts of the models are surprisingly similar. The 4 and 5 step ahead forecasts are also similar, but differences appear in the 8 quarter ahead forecasts. The parts of Fig. 7 show the same selection of quarters ahead for the variable, real GNP. There is again a remarkable similarity in the 1 quarter ahead forecasts. Differences in forecasts appear for 4 and 5 quarters ahead, in contrast to the corresponding situation for current dollar GNP, and differences are largest for the 8 quarter ahead forecasts of real GNP. Turning to the forecasts of unemployment rate shown in Fig. 8, one sees that the 1 quarter ahead forecasts are similar to one another and that those for 4, 5 and 8 quarters are dissimilar. For this variable the differences between the forecasts tend to increase with the increase in the number of quarters ahead of the forecasts. These similarities among forecasts, which also appear for other variables as well, may be regarded by readers as surprising given

U: I bi 61 -I a I 1 q l a I 4 t! o I H -- HASmtTO ___MTON.__. 0 O - _ OF1SE __ RWWTON I -1.a- I a I _ - I 71 72 73 TEAM 74 75 71 72 73 TEF.I 74 75 6 (a). Forecasts of 1 Quarter Ahead 6(b). Forecasts of 4 Quarters Ahead Fig. 6. Forecasts of Current Dollar GNP for the DRI, Chase, and Wharton Mark III Models

5-GM -Ga I,, I bi 03 Is b IA e S K II V o. ice Ton~S 71 72 73 FItC 74 75 71 72 73 TEf. 74 75 6(c). Forecasts of 5 Quarters Ahead 6 (d). Forecasts of 8 Quarters Ahead Fig. 6. Forecasts of Current Dollar GNP for the DRI, Chase, and Wharton Mark III Models

I-G54 B-a6t It l. fa i It m'U i N) NJ ___ - -WSE _- MITON T - - CHRSE -_ Hf1mtIO II - - 3 ~~ I- s C s - - - 71 72 73 TERR 74 75 71 72 73 74 75 7 (a). Forecasts of 1 Quarter Ahead 7(b). Forecasts of 4 Quarters Ahead Fig. 7. Forecasts of Real GNP for the DRI, Chase and Wharton Mark III Models

5m58 6-Gt8 aI I I/!.J IW m';c5 I N) uJ... W17 SEI = streiw,.- *OR -. - OiRSE ___ WHTON _._ Ng'~TON 71 72 73 74 75 71 72 73 TERR 74 75 7(c). Forecasts of 5 Quarters Ahead Fig. 7. Forecasts of Real GNP for the and Wharton Mark III Models 7 (d). Forecasts of 8 Quarters Ahead DRI, Chase

I-gt q-m aO - r-t j0. tQ 4:\J Uo+ U)+ - - ORSQ _: MWTO.. TDPIO 71 72 73 TEffl 74 75 71 72 73 TEAR 74 75 8 (a). Forecasts of 1 Quarter Ahead 8(b). Forecasts of 4 Quarters Ahead Fig. 8. Forecasts of Unemployment Rate for the DRI, Chase, and Wharton Mark III Models

a ac E 1) I i i 10. i i i I I A. \ 'S - r f ' // / I p %%jI NO ' a. I I O-t 'I- ' ' --- — _ ---.-.. - _- ' _4.._ --- - _n~~~~~~.. ~ _ t,-,- CTRI ___ WfmRTON 71 72 73 TEM 74 - 75 71 72 73 TEWm 74 75 8(c). Forecasts of 5 Quarters Ahead 8(d). Forecasts of 8 Quarters Ahead Fig. 8. Forecasts of Unemployment Rate for the DRI, Chase, and Wharton Mark III Models

4 l - 26 - that the 3 models differ so greatly in many respects. For example, there are considerable differences in the number of equations in the models, in types of nonlinearities employed, in the selection and specification of exogenous variables needed for forecasts, in the selection of estimation procedures and in the choice of the sample period for which estimates are made, and in the selection of initial values and of equation normalizations required in the use of the Gauss-Seidel algorithm to develop forecast solutions from the models. Perhaps the forecasts owe their similarities, in view of these many differences in structural details, to the many subjective and judgmental modifications that the respective model builders employed before their release to clients. CONCLUDING COMMENTS The forecasting record of the U.S. econometric models discussed here for the first half of the 1970's has been unreliable for the periods of 3 or more quarters into the future. Particularly disturbing is the apparent inability of these models to capture price and supply-side effects well or to provide reliable insights into investment anticipations. Econometric models will have to assimilate these vitally important effects in a more satisfactory way or the interest of practitioners in them may begin to wane and they may be justified in replacing them by other forecasting approaches. A case for econometric models is frequently made on the basis of their use in economic policy assessment rather than in short-term forecasting. However, the magnitudes of the forecast errors of the

L v - 27 - models for periods beyond 1 year would appear to cast considerable doubt on their use in assessing policy alternatives -- a mode in which they appear to be used by governmental organizations some of which frequently seek to assess policy effects over a 10 to 20 year horizon. Such policy assessments of econometric models are carried out by means of policy multipliers calculated from the models. The latter are in essence generalized forecasts which require the use of judgmental choices of future values of the exogenous variables for each period of the horizon over which policy assessments are being made. If model forecasts deteriorate in accuracy over a 1 to 2 year horizon, as is the case for every variable and model considered here, it is difficult to give credibility to generalized forecasts made by the models for still longer periods of time. On the other hand, the short time horizons of 1 or 2 quarters ahead for which econometric forecasts appear to be relatively accurate are much too short to be of interest in policy assessment. A further difficulty in the use of econometric models in policy assessment has been raised by Lucas (1976) and others. These writers, interpreting models from the point of view of rational expectations, have called attention to the problem of changes in economic structure caused by the policy changes one is using an econometric model to examine. Such structural changes, in addition to the problems posed by longer-term forecasting with an econometric model of the kind we have explored here, further compound the difficulties of interpreting the long-run policy multipliers of econometric models.

- 28 - Some economists have referred to the 1950's and 1960's as the age of Keynes and the 1970's as the age of the econometricians. A far better forecasting performance for the latter half of the 1970's -- which now seems unlikely -- will be necessary to justify this view.

u' ) -29 -Duesenberry, J., G. Fromm, L. Klein, and E. Kuh, eds. The Brookings Quarterly Econometric Model of the United States. Chicago: Rand McNally, 1965. Duesenberry, et. al., eds. The Brookings Model: Some Further Results. Chicago: Rand McNally, 1969. Friedman, M. "The Role of Monetary Policy." American Economic Review 58 (1967): 1-17. Fromm, G., and L. Klein. "A Comparison of Eleven Econometric Models of the United States." American Economic Review 63 (1973): 385-393. Fromm, G., and L. Klein. "The NBER/NSF Model Comparison Seminar: An Analysis of Results." Annals of Economic and Social Measurement 5 (1976): 1-27. Haitovsky, H., G. Treyz, and V. Su. Forecasts with Quarterly Macroeconomic Models. New York: National Bureau of Economic Research, 1974. Hickman, B.G., ed. Econometric Models of Cyclical Behavior. Vols. 1 and 2. New York: Columbia University Press, 1972. Klein, L. An Essay on the Theory of Economic Prediction, Chicago: Markham, 1971, Lucas, R.E. "Econometric Policy Evaluation; A Critique." In K, Brunner and A. Meltzer, eds. Carnegie-Rochester Conference Series on Public Policy. Vol. 1. New York; North Holland, 1976, pp. 19-46. McNees, S.K. "An Evaluation of Economic Forecasts," New England Economic Review. Boston: Federal Reserve Bank of Boston, Nov/Dec. 1975, pp. 3-39. Spivey, W.A., and W.J. Wrobleski. Econometric Model Performance in Forecasting and Policy Assessment, Washington, D.C.: American Enterprise Institute for Public Policy Research, 1979. Zarnowitz. V. An Appraisal of Short Term Economic Forecasts. New York: National Bureau of Economic Research, 1967. Zarnowitz, V. "How Well Do Economists Forecast Growth, Recession, and Inflation?" Economic Outlook USA. Ann Arbor, Mich.; Survey Research Center of the University of Michigan, Spring, 1978, pp. 22-25. Zarnowitz, V. "An Analysis of Annual and Multiperiod Quarterly Forecasts of Aggregate Income, Output, and the Price Level." Journal of Business 52 (1979): 1-33.