Research Support School of Business Administration December 1993 CORPORATE DISCLOSURE POLICY AND ANALYSTS Working Paper #726 Mark Lang Stanford University and Russell Lundholm University of Michigan

I I / ~~~~~~~I~~~~~~~~~~~~~~~~~~~~~ I i{ Ii

CORPORATE DISCLOSURE POLICY AND ANALYSTS Mark Lang* and Russell Lundholm** December 1993 * Stanford University ** University of Michigan We thank Mary Barth, Amy Sweeney, Katherine Schipper, Tom Stober and workshop participants at Stanford University, the University of California-Berkeley, the University of Arizona, the University of Oregon, Pennsylvania State University and Harvard University for helpful comments and I/B/E/S Inc. for providing the analyst forecast data.

2 Introduction In this paper we examine the relation between the disclosure practices of firms, the number of analysts following each firm, and properties of the analysts' forecasts. Financial analysts are an integral part of the capital market, providing earnings forecasts, buy/sell recommendations and other information to brokers, money managers and institutional investors. Further, much of the information analysts use in their evaluations is provided directly by the firm. Given their role as information intermediaries, it is important to understand the relation between a firm's disclosures and analysts' activities in order to understand the flow of information from the firm to the capital market. We provide evidence that more analysts follow firms that are perceived to have more forthcoming disclosure policies (as measured by the Financial Analysts Federation's Corporate Information Committee Report) and that more forthcoming firms experience more accurate forecasts, less dispersion among individual analyst forecasts and less variability in forecast revisions.1 Survey research supports the view that analysts are information intermediaries between the firm and the capital market. Lees [1981] finds that analysts rely heavily on firms for information in evaluating companies. Their sources of information, in order of importance, are: (1) interviews with company executives, (2) 10-K's and other reports to the SEC, (3) annual and interim stockholders' reports, (4) management forecasts (if disclosed), and (5) formal presentations by company executives. Knutson [1992] also argues that companies are a primary source of information to analysts and identifies various communication channels. Other survey evidence finds that analysts are second only to annual reports as a source of information to buy-side professional investors (SRI International [1987]). Thus, many capital market participants receive information from analysts, who in turn receive information from the firm. A considerable body of accounting research has focused on the link between the capital market and the analyst,

3 particularly the stock market response to earnings forecasts, but considerably less research has considered the link between the firm's disclosure policies and the analyst. Although all publicly traded firms must meet minimum disclosure requirements set by the SEC, they vary greatly in the amount of additional information they provide to the capital market Even for mandatory disclosures such as annual and quarterly financial statements, firms have substantial discretion in the clarity of the presentation and the level of detail provided. Differences in disclosure policies are even more pronounced for voluntary disclosures such as press releases and direct contact with analysts. Given the variety of disclosure media available to firms and the fact that much of the information is qualitative in nature, it is difficult to quantify the informativeness of a firm's disclosures. We measure the forthcomingness of a firm's disclosure practices using the evaluations in the Financial Analysts Federation Corporate Information Committee Report (FAF report). These data have the advantage of capturing all facets of a firm's disclosure to the capital markets as measured by a primary group of users of the disclosed information, financial analysts. In the FAF report, analysts evaluate the complete range of a firm's disclosures, summarizing their evaluations by a score in three categories: annual published information, quarterly and other published information, and investor relations. A typical year's report covers approximately 460 firms in 27 industries, with an average of 13 analysts in each industry providing the evaluations. Most previous empirical research on the determinants of analyst following and the characteristics of analyst forecasts has focused on either a particular mandatory disclosure or firm characteristics other than their disclosure policies. In a cross-sectional study, Bhushan [1989] finds that the number of analysts following a firm is increasing in firm size, institutional ownership, and return variability. In a time-series study, O'Brien and Bhushan [1990] find that the number of analysts following a firm increases when a firm's return volatility has declined, increases more for firms with smaller prior analyst following, and increases more for firms in industries with more stringent disclosure requirements and

4 increasing numbers of firms. Brennan and Hughes [1991] find that analyst following is greater for firms with lower share prices, after controlling for size, return variability and past returns, and that the number of analysts increases after a stock split Perhaps closest to our study, Byrd, Johnson and Johnson [1993] document an increase in analyst following after CEO presentations to analyst societies. Similarly, there is a substantial literature on the properties of analyst forecasts (see Brown, Foster and Noreen [1985] for a review), but relatively few papers have considered how a firm's disclosures affect forecasts, particularly voluntary disclosures. In terms of mandatory disclosures, Brown and Han [1992] and Morse, Stephan and Stice [1991] find that the consensus of analyst long-range forecasts increases around earnings announcements, and Swaminathan [1991] finds that consensus increases following the adoption of segmental reporting requirements. Using inches of print in the Wall Street Journal, Kross, Ro and Schroeder [1990] find that the accuracy of analyst forecasts increases, relative to a time-series model, as the amount of coverage increases.2 This research makes several contributions. First, it provides insight into the role of analysts in the capital market and the importance of firm-provided information. The role of analysts in securities markets is not well understood. While some survey and behavioral evidence exists on the activities of analysts, relatively little empirical evidence exists on the role of analysts in capital markets, particularly with respect to how different sources of information affect their decision-making.3 For example, there is little evidence on the effect of firms' disclosure policies on the accuracy of analyst forecasts. Further, depending on whether cross-analyst forecast variation reflects differences in access to information or differences in interpretation of shared information, increased disclosure could either increase or decrease consensus. Finally, depending on whether analysts view firm disclosure as preempting their analysis or aiding in it, incentives to follow the firm could be increasing or decreasing in the informativeness of firm disclosures. The results of this

5 paper suggest that a firm's disclosure policy is an important explanatory variable for analyst following and various characteristics of their forecasts. This research also provides insight into the role of information in capital markets and managerial incentives for disclosure. Theoretical research suggests that firm price is increasing in the number of investors following the firm (e.g., Fishman and Hagerty [1989] and Merton [1987]) and how accurately the firm's future prospects can be estimated (e.g., Barry and Brown [1985]), and decreasing in the degree of information asymmetry between market participants (e.g., Glosten and Milgrom [1985]). Further, it is often asserted that investors do not like surprises (e.g., Cooper [1989]) and that large revisions in expectations, and the accompanying large share price revisions, can attract lawsuits (e.g., Alexander [1991]). Therefore, potential motives for more forthcoming disclosures include increasing the investor following and knowledge of the firm, and decreasing investors' information asymmetry and the volatility of their revisions in expectations. These benefits are also echoed in the popular press (see, for example, Marcus and Wallace [1991] and Mahoney [1991]).4 Finally, to the extent that analysts may be viewed as either influencing or representing investor beliefs, perspectives suggested by Nichols [1989] and Schipper [1991], an analysis of the relation between disclosure and analysts provides evidence on the effects of disclosure on capital markets. Hypotheses To structure our empirical analysis we focus on two aspects of analysts' behavior -- their choice of which firms to study, as measured by the number of analysts providing earnings forecasts for a firm, and characteristics of their resulting forecasts, specifically the forecast accuracy, the degree of dispersion among forecasts and the variability of forecast revisions during the year.5 The remainder of this section establishes the relation between the informativeness of a firm's disclosures and these dependent variables.

6 Disclosure and Analyst Following The predicted relation between analyst following and disclosure depends on the role of analysts in the security markets and the nature of the information provided by firms. Assume that analysts have access to both firm-provided information and private information, that they can follow only a limited number of firms, and they follow firms that will generate the greatest demand for their services net of the cost of supplying the service. If the demand for analysts is primarily due to their comparative advantage over investors in receiving and interpreting firm disclosures then increased disclosure to analysts will increase the value of their reports and, hence, the demand for their services. Similarly, if the cost of acquiring company-provided disclosure is less than the cost of acquiring the information privately, an increase in disclosure will increase analysts' willingness to follow the firm. For either reason, increased disclosure will attract more analysts. These arguments are consistent with the investor relations literature, which lists an increased analyst following as a benefit of a more forthcoming disclosure policy (see Mahoney [1991] and Marcus and Wallace [1991]) and with analysts' stated desire for increased disclosure (see, for example, FAF [1988]).6 Alternatively, if the demand for analysts is primarily due to their ability to discover private information not disclosed by the firm then, assuming diminishing returns to additional information, a more informative disclosure policy provides investors with a substitute for an analyst report. Similarly, firm disclosure may reduce the set of available private information, thus increasing the analysts' information acquisition cost. For either reason, increased disclosure will lead to a smaller analyst following. While this effect of disclosure on analysts is not frequently cited, it is consistent with the stated opposition by some analysts to disclosure of management earnings forecasts (Lees [1981] and Knutson [1992]). Of course, it is likely that the true role of analysts in capital markets lies somewhere between the two extremes we have described here. We offer these two views

7 only to show that the relation between disclosure and the number of analysts may be either positive or negative.7 Disclosure Policy and Characteristics of Analyst Forecasts In addition to affecting the number of analysts following the firm, additional disclosure is also likely to affect the properties of their earnings forecasts. First, to the extent that firm disclosures are informative about future earnings, analysts' forecast accuracy is likely to increase with the informativeness of the firm's disclosures. It is possible that a firm's disclosures are useful to analysts without providing information about next period's earnings, but it is difficult to imagine scenarios where additional disclosure systematically reduces the accuracy of earnings forecasts. However, the extent to which the firm's disclosure policy affects forecast accuracy is an empirical issue. Second, the dispersion among the analysts' forecasts may be either increasing or decreasing in the informativeness of the firm's disclosure policies. For example, suppose that each analyst has the same forecasting model and each observes the firm's disclosures, but different analysts observe different private information. As the informativeness of the commonly observed firm disclosures increases, analysts place less weight on their private information, thus increasing the consensus in their forecasts. Alternatively, suppose that all analysts have the same information but analysts differ in their forecasting models.8 For example, if analysts place similar weights on aggregate disclosures, such as income, but place different weights on the components of the aggregate, such as segmental profits, then a more informative disclosure practice, such as reporting more detailed segmental data, can lead to more heterogeneous analyst forecasts.9 Third, additional disclosure is likely to affect the variability of earnings forecast revisions during the year. In assessing the informativeness of a firm's disclosure practices, the FAF considers the timeliness of information releases, particularly for the investor relations and other publications categories of disclosure -- firms that disclose information

8 on an ongoing basis are judged to have a more informative disclosure practice than those that withhold information until a mandatory disclosure is required. The volatility of forecast revisions may either increase or decrease with more informative disclosure practices. At the time a disclosure is made, a more precise disclosure causes analyst forecasts to change more severely in response to the disclosure, but there is less remaining uncertainty to resolve, so subsequent disclosures create less volatile revisions. But to the extent that more informative disclosure practices shift precision from later to earlier disclosures, both effects work to reduce the volatility of forecast revisions; the early forecasts become better informed and there is less information remaining to cause later forecasts to fluctuate.10 Empirical Analysis The Disclosure Data The data used to measure disclosure are the ratings of firm disclosure in the annual FAF Report for 1985 to 1989. The analysis in the FAF report is conducted by industryspecific subcommittees composed of leading analysts,11 and contains evaluations of a firm's forthcomingness in disclosures along three dimensions: annual published information, quarterly and other published information, and investor relations and related aspects.12 The subcommittees meet first to agree on the firms to be reviewed and the criteria and weights to be applied in evaluating companies in the industry.13 Individual members later complete evaluations of each firm, which are returned to the subcommittee chairperson. The subcommittees generally meet a second time to summarize the scores, prepare a written explanation, and decide whether to recommend any companies for an Award for Excellence in Corporate Reporting. Each subcommittee report is reviewed by the Corporate Information Committee for consistency and fairness and a final report is prepared containing a detailed evaluation of each firm, summarized with a score in each category and a total company score which is a weighted combination of the three category

9 scores. The report is mailed to all the firms surveyed as well as to the FASB, the SEC and other interested parties, and in many cases the subcommittee meets individually with company representatives to discuss the results. A copy of a representative subcommittee report is included as Appendix A. Briefly, in the "Annual Published Information" category, analysts assess the clarity and candidness of the financial highlights and president's letter; the amount of detail about the corporate officers, the corporation's goals and product and geographic segments; and the overall level of detail in the financial statements and footnotes. In the "Quarterly and Other Published Information" category, analysts consider the depth of coverage in quarterly reports and the availability and timeliness of other written material, such as press releases, proxy statements, summaries of the annual meeting proceedings and presentations to analyst groups, and statistical supplements. In the "Investor Relations" category analysts assess how knowledgeable and responsive the company contact is to analyst questions, the accessibility of management and their candor in discussing corporate developments, and the frequency and content of presentations to analysts. Approximately 19% of the industry subcommittees (covering 38% of the firms) report only the overall company scores and approximately 5% of the industries (covering 5% of the firms) report only rank values, either category and overall ranks or overall ranks alone. Thus, the composition of the sample varies depending on whether the total score or individual category scores are used in the analysis. The FAF data represent a cross-section of industries, including service, manufacturing, financial, transportation and extraction.14 Banking provides 20% of the total score observations, although this industry subcommittee does not report individual category scores. Although disclosure policy is likely to be correlated across the three categories, we include each separately, as well as the total score, because firms may vary in their choice of disclosure media and because the effect of disclosure policy may vary by the media employed.15 For example, because much of the disclosure included in the investor relations category is targeted specifically at analysts, this

10 category may have the most influence on the willingness of analysts to follow the firm but, to the extent that periodic press releases are important sources of information on material developments for the firm, the other publications category may be an important determinant of forecast accuracy. The FAF data contain a potential sample of 751 firms (732 with COMPUSTAT data) that are rated in at least 1 of the 5 FAF reports considered. In total, there are 2272 firm-years in the sample, with each firm evaluated approximately three times in the five year period. Panel A of table 1 provides descriptive statistics for the three category scores and the total company score. The score for each category is a percent (points received for the category divided by the number of points assigned to the category). This allows us to aggregate across industries with different weights assigned to the different categories. The percent of total points, however, preserves the weights assigned by the different industry subcommittees. As seen in the table, all categories and the total score average slightly over 70% of the possible points. Each category has considerable variation; the difference between the 1st and 99th percentiles is highest for the investor relations variable (75%) and lowest for the annual report variable (58%), which may suggest that firms can more clearly differentiate their investor relations efforts than their annual report disclosures. There are advantages and disadvantages of the data for purposes of investigating the effects of disclosure policy on analysts. As discussed in the introduction, a primary advantage is that the scores capture all facets of disclosure which are viewed as important by analysts, rather than focusing on a single aspect like management earnings forecasts or press releases, and include both quantitative and qualitative aspects of disclosure. In addition, the measures are prepared by analysts who are primary users of financial statements and are familiar with the firms' disclosures, having used them in their analysis during the year, and appear to reflect a substantial investment of time and effort.16 A potential problem with the data is that they reflect analysts' perceptions of firms' disclosure policies rather than the disclosure policies themselves and are, therefore,

11 subjective measures.17 The FAF's evaluation process includes several factors specifically designed to enhance the objectivity of the evaluations. First, only summary measures for a subcommittee are presented, thereby reducing the opportunity for individual analysts to curry management's favor by providing a more optimistic evaluation than is warranted. Second, a primary motive for having subcommittee members meet both before and after the evaluation process is to reduce subjectivity and ensure that all members use the same basic criteria in their evaluations. Further, because each subcommittee report includes a written justification of each firm's score, and the report is frequently presented to the firm orally, subcommittee members cannot capriciously make their evaluations. Finally, the Corporate Information Committee's review of the subcommittee reports helps ensure fairness and consistency. However, one cannot completely dismiss the possibility of noise or bias in the data. Dependent Variables As discussed earlier, we consider four dependent variables in our analysis which we believe are likely to be affected by a firm's disclosure policy, all taken from the IBES Summary Tape. The dependent variables are: Number of Analysts = the number of analysts providing an annual earnings forecast as of each month's IBES report date, Std. Dev. of Forecasts = the inter-analyst standard deviation of forecasts deflated by stock price as of the each month's IBES report date,

12 Forecast Accuracy Revision Volatility the absolute value of the forecast error from a random walk model of earnings (actual earnings less last year's annual earnings per share) less the absolute value of the analyst forecast error (actual earnings per share less the median analyst forecast of earnings), deflated by stock price as of each month's IBES report date, and the standard deviation of the change in the median forecast from the preceding month, deflated by the stock price as of the beginning of the fiscal year. Each variable is based on the forecasts of annual earnings for the current fiscal year. The number of analysts, the standard deviation of forecasts and forecast accuracy are computed as the simple average of the measure across the twelve monthly reporting periods on the IBES tape during the company's fiscal year.l8 Results are very similar using measures computed based on data for a single month computed at fiscal year-end, six months prior to fiscal year-end, or twelve months prior to fiscal year-end. We deflate the forecast measures by the price of the firm to present the variables on a comparable basis across firms. Forecast accuracy is measured relative to a time series forecast to capture the notion that disclosure should increase accuracy relative to that achieved by simply extrapolating from last year's earnings.19 The revision volatility is intended to capture the smoothness with which analysts' expectations change during the period.20 Panel B of table 1 provides descriptive statistics for the dependent variables. The median firm-year in the sample is followed by 16.7 analysts, has a standard deviation of forecasts equal to.5% of its share price, has a forecast accuracy that is.8% of its share price greater than the random walk forecast, and a standard deviation of forecast revisions

13 equal to.2% of its share price. Each variable also exhibits considerable variation across the sample, as evidenced by the inter-quartile ranges. The Control Variables We consider five control variables based on the results of previous research. The first three proxy for features of a firm's information environment that might affect the number of analysts following a firm and the properties of their forecasts. The variables, computed from COMPUSTAT, are Market Value = the market value of the firm's equity at fiscal year-end, Std. Dev. of ROE = the historical standard deviation of return on equity computed over the preceding ten years. Return-Earnings Correlation = the historical correlation between annual returns and earnings computed over the preceding ten years, and Bhushan [1989] and Brennan and Hughes [1991] find that firm size and various measures of performance volatility are related to the number of analysts following a firm. As in Lang and Lundholm [1993], we include the return-earnings correlation as a proxy for the amount of value-relevant information conveyed by earnings (for example, a lower correlation implies that more of the variation in returns is due to non-earnings information). The idea is that analysts may have a smaller comparative advantage over investors for firms whose earnings are particularly informative. In addition, King, Pownall and Waymire [1992] suggest that the benefits to private information acquisition may be higher for larger firms, for firms with relatively volatile performance and for firms with a relatively high correlation

14 between earnings and returns. They hypothesize that the benefit to private knowledge of future earnings is greatest for firms with these characteristics.21 For the regressions of the forecast properties (i.e. the standard deviation of forecasts, the forecast accuracy and the revision volatility) we use two additional control variables. The variables, taken from the IBES Summary Tape, are Earnings Surprise = the absolute value of the difference between the current year's earnings per share and last year's earnings per share, divided by the price at the beginning of the fiscal year, and Percent New Forecasts = the number of forecasts revised during the month divided by the number of forecasts at the month end, averaged over the twelve months in the firm's fiscal year. Earnings surprise is included as an exogenous control for cross-sectional and intertemporal variation in the potential information that may be conveyed during the period. We include the percent of new forecasts to control for cross-sectional variation in the "staleness" of forecasts. This is necessary because the median forecast and the standard deviation of forecasts on the IBES summary tape are based on all available forecasts, regardless of whether the individual forecast was updated during the month. Panel C of table 1 presents descriptive statistics on the control variables. The median firm-year has a market value in excess of $1 billion, indicating that the sample firms are generally large. However, size ranges from $33 million in the 1st percentile to over $20 billion in the 99th percentile, so that very large and small firms are represented in the sample.22 For the median firm-year, the standard deviation of return on equity is 1.5%,

15 the earnings/returns correlation is 36%, the absolute value of the change in earnings per share from the previous year is 1.9% of the share price, and 25% of the forecasts are revised each month. Design and Descriptive Statistics Because each industry is evaluated by a separate set of analysts, we industry-adjust the dependent and independent variables; consequently, the statistical tests that follow should be interpreted as explaining intra-industry variation in the number of analysts and their forecast properties by the intra-industry variation in the disclosure and control variables. This approach has the advantage of controlling potential cross-industry variation in scores due to differences in subcommittee composition, but has the disadvantage of ignoring cross-industry variation in scores due to legitimate differences in disclosure policy across industries. However, the results for measures that are not industry-adjusted are consistent with, and generally strong than, those reported in the paper. Because we do not have a model that predicts a particular functional form between the dependent and independent variables, we base our analysis on ranked data. In particular, we rank the dependent and independent variables within their industry-year and, because there are different numbers of firms in each industry, we convert the ranks to rank - 1 percentiles by the expression # of fi-ms - 1 The conversion yields the percentile of the firm's rank within its industry-year, so that the lowest-ranking firm receives a zero and the highest-ranking firm receives a one.23 Table 2 presents the simple correlations between the independent variables considered in this study. As one would expect if firms coordinate disclosure policy across media, the three disclosure categories are all significantly correlated with one another at levels ranging from 0.43 for the annual report - investor relations correlation to 0.62 for the annual report - other publications correlation.24 However, the fact that the correlations are considerably less than one suggests that the categories may capture different aspects of

16 disclosure. Among the control variables, the strongest association is between size and the standard deviation of return on equity, which are negatively correlated. Finally, the correlations between the disclosure variables and the control variables are generally significant, although weaker than the correlations within either set of variables. As reported in Lang and Lundholm [1993], firm-years with higher disclosure scores are generally characterized by larger market values, lower standard deviations of past return on equity, and lower past return/earnings correlations. Further, higher disclosure scores are associated with a larger number of forecasts revision, suggesting that a more forthcoming disclosure policy is associated with more frequent forecast revisions. Table 3 presents simple correlations between the analyst variables, and between the analyst variables and the independent variables. The highest correlation is between the forecast revision volatility and the standard deviation of forecasts (.76); the correlations between other pairs of variables are weaker. It is somewhat surprising that forecast accuracy is not significantly correlated with the number of analysts given, under fairly standard assumptions, that the median forecast asymptotically becomes more accurate as the number of forecasts increases. With respect to the correlations between the analyst variables and the control variables, the number of analysts has the largest association with market value (0.70), as we might expect. The standard deviation of forecasts and the revision volatility are both highly correlated with the earnings surprise and the standard deviation of past return on equity suggesting that, for firm-years with high past and present uncertainty, analysts have less consensus among them and their forecasts change more dramatically through the year. In terms of the primary relations of interest in the paper, table 3 displays consistent patterns across all three disclosure categories. The number of analysts is positively correlated with the informativeness of a firm's disclosure practices, and firms with more informative disclosure practices have less dispersion in analyst forecasts, the forecasts are more accurate and there is less volatility in the sequence of forecast revisions. However,

17 given the correlations between the analyst variables and the control variables which may also affect disclosure policy, simple correlations may be misleading. In the next section we assess the marginal affect of disclosure on the analyst variables by estimating multiple regressions using the ranked data. Number of Analysts and Disclosure The results of regressions of the number of analysts on the control and disclosure variables are given in table 4.25 As expected, firm size is positive and significant. The standard deviation of past return on equity and the orrelation o pstbetween annual returns and earnings have negative coefficients that are generally significant, suggesting that analysts are more likely to follow firms with low earnings variability and firms for which earnings are not highly correlated with returns.26 The fact that analysts are attracted to firms whose returns are not highly correlated with their earnings suggests that other information is driving the variation in returns. Presumably, this is the information that the analysts are providing. The relations between disclosure and number of analysts following the firm are consistently significant and positive. Investor relations has the strongest results for any category, consistent with the finding in Lees [1981] that analysts rely most heavily on direct contact with the firm for information. The positive relation between the number of analysts and the informativeness of a firm's disclosure policy suggests that firm disclosure complements rather than substitutes for analyst activities. It is consistent with the notion that analysts are not soley private information collectors; processing firm-provided information is also a significant aspect of their function in capital markets. The Direction of Causality We developed our hypotheses under the assumption that a firm chooses its disclosure policies partly to influence the activities of analysts. Thus, we view the

18 informativeness of disclosure as causing the observed level of analyst following. It is possible, however, that the direction of causality runs the opposite direction -- that the presence of a large number of analysts causes the firm to increase its disclosures. Or both analyst following and the informativeness of disclosures may be caused simultaneously by a set of purely exogenous variables. We investigate these possibilities next One approach to establishing the direction of causality is to examine the association between changes in disclosure scores with changes in lead and lag number of analysts. There are several problems with this approach. First, the disclosure evaluation process is primarily designed to make comparisons between firms, not comparisons of the same firm through time. As such, the composition of the industry subcommittees and the specific criteria and weights applied in the evaluation vary from year to year. Therefore, a significant proportion of the variation in changes in disclosure scores may reflect changes in subcommittees or criteria over time. Second, the disclosure policy of a firm and the number of analysts following it are likely to change only slowly over time because of the fixed costs of substantially changing disclosure policies or adding or deleting a firm from an analyst's following. Therefore, we include changes computed over time periods of up to four years. However, because a given firm is evaluated only intermittently, the number of observations for longer intervals is drastically reduced. Subject to these caveats, we compute the rank-order correlations between changes in disclosure scores and changes in both lead and lag analyst following. If changes in the number of analysts following the firm tend to precede changes in disclosure policy, then the relation for lagged change in number of analyst following the firm should be significantly positive. As seen in table 5, all but one of the lag correlations are negative while all the lead correlations are all positive. Subject to the caveats discussed above, this evidence suggests that changes in disclosure tend to precede, rather than follow, changes in number of analysts following the firm.

19 A second approach is to estimate a simultaneous equations model. This method assumes that a firm's disclosure policy and the number of analysts following the firm are determined simultaneously by other exogenous variables. The benefit of a simultaneous equations approach is that it potentially controls for the bias in the OLS estimates when the error term is correlated with the independent variables, as is the case when a truly endogenous variable is treated as an exogenous (independent) variable. With this method, we estimate a first stage regression of the disclosure score on a set of instrumental variables that we believe are exogenous and then use the predicted value from this regression in place of the actual disclosure score in the number-of-analysts regression. Whether the coefficient estimates from this approach are more or less biased than the OLS estimates depends crucially on whether the instrument variables are indeed exogenous (i.e. independent of the error term) and whether the first stage regression captures most of the variation in the disclosure score. As discussed in Bound, Jaeger and Baker [1993], "if the correlation between the instrument and the endogenous explanatory variable is weak, even a very small correlation between the instrument and the error will produce a larger inconsistency in the instrumental variable estimate of P than in the OLS estimate." In our context, therefore, the difficulty is identifying instrumental variables that are highly correlated with disclosure, but are uncorrelated with the error from the number of analysts regression. One source of instruments is Lang and Lundholm [1993], who model disclosure scores as a function of the firm's structural environment (market value and standard deviation of past return on equity), performance (market-adjusted returns and analysts' earnings forecast errors), and activity in the public offering market (security offering in the current or following two years). While the explanatory variables are statistically significant, the resulting regressions have adjusted R2's of at most 0.09, suggesting relatively weak power as potential instruments. Given the limited explanatory power, it is important that the potential instruments not be correlated with the error in the number of analysts regression. However, it is difficult to argue with certainty that any of the

20 instrumental variables are uncorrelated with the error from the number of analyst regression. Therefore, results from this approach should be interpreted with caution. The results of the second stage regressions are presented in table 6, along with the R2's from the first stage regressions. As seen in the table, the significance of the disclosure scores is reduced relative to the results in table 4, but all remain significant at less than the 0.10 level. While the reduced significance may reflect the effects of simultaneity, it is also quite possible that it reflects the low R2 in the first stage regressions. Overall, the results clearly indicate a significant relation between the firm's disclosure policy and the number of analysts following the firm. Further, there is limited evidence that differences in firms' disclosure policies cause differences in analyst following, rather than the other way around. The next three regressions address the relation between disclosure and forecast properties. As previously discussed, we add earnings surprise and the percent of new forecasts as control variables.27 Forecast Consensus and Disclosure The results from the standard deviation of forecasts regressions are presented in table 7, panel A. The relations between the standard deviation of forecasts and the earnings surprise, percentage of new forecasts, standard deviation of past ROE and the past returnsearnings correlation are all significantly positive, suggesting that consensus among analysts tends to be lower when there is a large earnings surprise, the past earnings series has been volatile and unrelated to past returns, and there have been frequent forecast revisions. The relation between the standard deviation of analyst forecasts and market value is negative, suggesting that analyst consensus tends to be higher for larger firms. The results for all four disclosure variables suggest a negative relation between the standard deviation of forecasts and disclosure, consistent with the notion that additional

21 disclosure on average increases consensus among analysts. The relation is strongest between the standard deviation of analyst forecasts and the annual report and investor relations variables. This suggests that analysts use both firm-provided and private information, so that more firm-provided information increases the weight on this common source of information and decreases the weight on private information.28 Accuracy of Forecasts Relative to a Random Walk and Disclosure Results of the regression for forecast accuracy are presented in table 7, panel B. Because accuracy is measured relative to a random walk, the variable should be interpreted as capturing analysts' forecast superiority relative to this benchmark. The accuracy variable is positively correlated with the earnings surprise (although partly by construction) and market value, suggesting the analysts' advantage relative to a random walk is greatest when there is a relatively large change in earnings and for larger firms. Accuracy is negatively associated with the percent of new forecasts, which is inconsistent with our original motivation for including this variable (we expected a low percent of new forecasts to imply many stale and therefore inaccurate forecasts). Accuracy is significantly positively correlated with all three categories of disclosure and with the total disclosure score, confirming that firm disclosure is an important determinant of the accuracy of analyst forecasts. Further, although all three disclosure variables are significant, the relation is strongest for the other publications category. This result is confirmed in the regressions including all three categories of disclosure where other publications is significant, investor relations is only marginally significant, and the annual report variable is insignificant. This suggests that firms who report significant firm events with press releases etc. allow analysts to improve their forecasts relative to a mechanical time-series model.

22 Volatility of Forecast Revisions and Disclosure The standard deviation of forecast revisions regressions are reported in table 7, panel C. The variability of forecast revisions is decreasing in the market value and increasing in the earnings surprise, percentage of new forecasts and the standard deviation of past ROE, suggesting that forecast revision volatility tends to be larger for smaller firms, for firms with a large earnings surprise and relatively volatile earnings processes, and for firms with frequent forecast revisions. The relation between each of the disclosure variables and the standard deviation of forecast revisions is negative and statistically significant. The relation is strongest for the investor relations variable, consistent with the notion that investor relations provides information on an ongoing basis thereby shifting the resolution of uncertainty to earlier points in time and, consequently, reducing the variability in the forecast revision process. Conclusions and Extensions The results of this paper suggest that firms with more forthcoming disclosure practices in their industry have a greater analyst following, more consensus among analysts, more accurate analyst forecasts, and less variable forecast revisions. Further, these results hold after controlling for size, the earnings surprise and other attributes of the firm's information environment. The following description of the relation between firm disclosures and analysts is consistent with the results of this paper. Firms that have more forthcoming disclosure policies attract more analysts, either because this increases the value of analyst reports or because it reduces their costs. Different analysts still make different forecasts, so each must believe that they possess some private information, but the fact that more firmprovided information attracts more analysts implies that analysts are not primarily providers of private information (otherwise the firm disclosure would be a substitute for an analyst report). Further, more forthcoming disclosures cause the analysts to place more weight on

23 the common firm-provided information, thereby increasing the consensus in their forecasts. This also suggests that analysts have similar forecasting models, or that the firm disclosures serve to homogenize as well as inform their forecasting models. To the extent that analysts, through their role as information intermediaries, are representative of the market as a whole and subject to the caveats discussed earlier, the results suggest potential motivations for disclosure. Firms with more forthcoming disclosure policies have a larger pool of potential investors, and these investors have more accurate beliefs about the firm's future performance, they have greater consensus in their expectations, and their expectations change in a smoother manner over time. All of these attributes may serve to reduce the firm's cost of capital.29 While the link between the results from this paper and these final conclusions are admittedly speculative, developing the connection suggests a promising direction for future research.

24 Table 1 Descriptive Statistics for Unadjusted Dependent and Independent Variables Panel A: Disclosure Variables percentile Disclosure Variable n mean 1% 25% 50% 75% 99% Annual Report 1324.73.37.64.75.83.95 Other Publications 1392.70.30.60.72.83.96 Investor Relations 1392.74.25.64.77.87 1.0 Total Score 2272.70.32.62.72.80.95 Panel B: Dependent Variables percentile Dependent Variable n mean 1% 25% 50% 75% 99% Number of Analysts 2175 17.6 2.4 9.9 16.7 24.3 39.8 Standard Deviation of Forecasts 2141.014.001.003.005.010.150 Accuracy vs. Random Walk 2163.015 -.102 -.001.008.017.347 Revision Volatility 2211.010.000.001.002.005.137 Panel C: Control Variables percentile Control Variable n mean 1% 25% 50% 75% 99% Market Value ($ millions) 2215 2438 33 433 1072 2538 20794 Std(Return on Equity) 2022.021.001.005.015.028.111 Return-Earnings Correlation 1952.316 -.590.090.360.590.895 Earnings Surprise 2166.076.000.009.019.044.842 Percentage New Forecasts 2175.253.092.196.248.304.447

25 Table 1, Notes The four disclosure variables are the analysts' ratings of disclosure measured as the percent of possible points received in each category. The first three dependent variables and the percentage of forecasts revised are all calculated as averages of monthly values over the twelve months of the fiscal year using data from the IBES Summary Tape. Number of Analysts is the number of analysts providing forecasts, Standard Deviation of Forecasts is the standard deviation of earnings per share forecasts divided by share price, Accuracy Versus Random Walk is the absolute value of the difference between realized annual earnings per share and earnings per share from the prior year less the absolute value of the difference between realized earnings and the median analyst forecast of earnings, divided by beginning of year share price. Revision Volatility is standard deviation over the twelve months of the fiscal year of the change in the median forecast each month, taken from the IBES Summary Tape, divided by the beginning of year share price. Market Value is of outstanding equity at the beginning of the fiscal year. Std(Return on Equity) is the standard deviation of net income divided by shareholders' equity and the Return-Earnings Correlation is the correlation between annual earnings and annual stock returns, each computed for the ten years preceding the current FAF report year. Earnings surprise is the absolute value of the difference between realized annual earnings per share and earnings per share from the prior year divided by beginning of year share price. Percent New Forecasts is the number of forecasts revised during the month divided by the number of forecasts at the end of the month. In total there are 751 firms with at least one year of FAF data during the years 1985 through 1989.

26 Table 2 Correlation between the Percentiles of the Independent Variables Ranked within Industry-Year (Listed below each correlation is the p value. The number of observations ranges from 1192 to 2272) Other Inv. Total Mkt. Pub. Rel. Score Value SD of R/E Earnings ROE Corr Surprise Percent New Annual Report Other Publications Investor Relations.623.463.825.0001.0001.0001.494.804.0001.0001.737.0001.230.0001.169.0001 -.113 -.115 -.073.065.0001.0001.0100.0207 -.079 -.106 -.017.084.0050.0002.5451.0022.188 -.101.0001.0003.245 -.085.0001.0002 -.099 -.031.0005.2657 -.083 -.018.0003.4079.028.3092.049.0225 Total Score Market Value Std. Dev. of ROE -.247 -.113 -.125.086.0001.0001.0001.0001.111.186.057.0001.0001.0112 Returns-Earnings Correlation.126.027.0001.2354 Earnings Surprise.253.0001 Annual Report, Other Publications, Investor Relations and Total Score are the analysts' ratings of disclosure measured as the percent of possible points received in each category. Market Value is of outstanding equity at the beginning of the fiscal year. Std. Dev of ROE is the standard deviation of net income divided by shareholders' equity and the ReturnEarnings Correlation is the correlation between annual earnings and annual stock returns, each computed for the ten years preceding the current FAF report year. Earnings surprise is the absolute value of the difference between realized annual earnings per share and earnings per share from the prior year divided by beginning of year share price. Percent New is the 12 month average of the number of forecasts revised during the month divided by the number of forecasts at the end of the month.

27 Table 3 Correlations between Percentiles of Dependent Variables and Disclosure and Control Variables Ranked within Industry-Year. (The p value is listed below each correlation. The number of observations ranges from 1253 to 2125) Number of Std. Dev. Forecast Revision Analysts of Forecasts Accuracy Volatility The Dependent Variables Revision Volatility -.197.764.025.0001.0001.2488 Forecast Accuracy.024.118.2687.0001 Std. Dev. of Forecasts -.149.0001 The Disclosure Variables Annual Report.281 -.178.016 -.175.0001.0001.5789.0001 Other Publications.228 -.095.088 -.113.0001.0005.0014.0001 Investor Relations.267 -.177.084 -.200.0001.0001.0023.0001 Total Score.331 -.123.067 -.146.0001.0001.0021.0001 The Control Variables Market Value.701 -.255.018 -.270.0001.0001.3836.0001 Std. Dev. of ROE -.190.303.110.267.0001.0001.0001.0001 Return-Earnings Correlation -.116.152.038.107.0001.0001.0994.0001 Earnings Surprise -.071.454.625.404.0009.0001.0001.0001 Percent New Forecasts.115.385.012.477.0001.0001.5797.0001 The variable definitions are given in table 1.

Table 4 Regression for Number of Analysts Int Mkt Val SD ROE Corr AnnRDt OPub InvRel Total N R2 0.19 0.61 -0.05 -0.06 0.12 1164 0.45 (7.4) (26.0) (-2.1) (-2.9) (5.6) 0.18 0.63 -0.05 -0.06 0.12 1211 0.46 (7.4) (27.7) (-2.3) (-3.0) (5.7) 0.18 0.62 -0.05 -0.06 0.15 1211 0.46 (7.3) (27.1) (-2.1) (-3.0) (6.7) 0.11 0.66 -0.01 -0.02 0.17 1868 0.52 (6.2) (38.6) (-0.8) (-1.5) (10.1) 0.16 0.60 -0.05 -0.06 0.05 0.04 0.10 1164 0.46 (6.0) (25.6) (-2.0) (-2.9) (1.8) (1.4) (3.9) Regressions are estimated using ordinary least squares, with all variables ranked and then computed as percentiles within the industry-year. T-statistics are noted in parentheses below each coefficient estimate. Number of Analysts is the average number of analysts on IBES providing an estimate of annual earnings over the twelve months of the company's fiscal year. Int is the intercept from the regression. Mkt Val is the market value of outstanding equity at the beginning of the fiscal year. SD ROE is standard deviation of net income divided by shareholders' equity and Corr is the correlation between annual returns and earnings, each calculated over the 10 years prior to the observation. AnnRpt, OPub, InvRel and Total are the annual report, other publications, investor relations and total score, respectively, based on the FAF scores for each category of disclosure. N is the number of observations and R2 is the adjusted R2 for the regression. =

Table 5 Rank-Order Correlations between Changes in Disclosure Scores and Lag 1 and Lead 1 Changes in the Number of Analysts, Computed over Window Lengths of 1 to 4 Years. 4 Year Window AAnnual AOther AInvestor ATotal Report Pubs Relations Score A in lead 1 0.11908 0.29018 0.11076 0.23833 correlation # of analysts 0.1494 0.0003 0.1802 0.0001 p value 148 148 148 265 number of obs. A in lag 1 -0.00767 0.05087 -0.13693 -0.01944 # of analysts 0.9270 0.5435 0.1005 0.7569 145 145 145 256 3 Year Window AAnnual AOther AInvestor ATotal Report Pubs Relations Score A in lead 1 0.02421 0.14201 0.04560 0.05948 # of analysts 0.6578 0.0083 0.3985 0.1594 337 345 345 561 A in lag 1 -0.04320 -0.02268 -0.11085 -0.08039 # of analysts 0.4300 0.6751 0.0399 0.0607 336 344 344 545 2 Year Window AAnnual AOther AInvestor ATotal Report Pubs Relations Score A in lead 1 0.11373 0.17181 0.05293 0.07938 # of analysts 0.0091 0.0001 0.2135 0.0171 525 554 554 902 A in lag 1 -0.03703 -0.01245 -0.10952 -0.05556 # of analysts 0.3972 0.7708 0.0102 0.0999 525 550 550 878 1 Year Window AAnnual AOther AInvestor ATotal Report Pubs Relations Score A in lead 1 0.05136 0.06131 0.00597 0.02076 # of analysts 0.1461 0.0715 0.8608 0.4361 802 865 865 1409 A in lag 1 -0.04702 -0.02555 -0.03657 -0.03731 # of analysts 0.1846 0.4561 0.2861 0.1680 798 853 853 1367

Table 6 Regression for Number of Analysts using Instruments for the Disclosure Scores First Stage R2 Int Mkt Val SD ROE Corr AnnRpt OPub InvRel Total N 0.09 0.57 (1.2) (14.7) 0.09 0.60 (1.2) (20.3) 0.15 0.60 (2.3) (18.0) 0.09 0.65 (2.1) (22.7) -0.04 -0.05 0.33 (-1.6) (-1.8) (2.1) -0.05 -0.05 (-2.0) (-1.8) 0.31 (2.2) 1154 0.42 1201 0.43 1201 0.45 1845 0.51.07.05.08.09 -0.04 -0.06 (-1.8) (-2.7) 0.22 (1.7) -0.01 -0.02 (-0.7) (-1.4) 0.20 (2.1)

Table 6, Notes Regressions are estimated using two-stage least squares, with all variables ranked and then computed as percentiles within the industry-year. The first stage regression is disclosure score = bO + blMktVal + b2SDROE + b3Corr + b4PFE + b5MRET + b60ffer, and the predicted value of the disclosure score is used as an instrument for the disclosure scot in the second stage regressions reported above. T-statistics are noted in parentheses below eac coefficient estimate. Number of Analysts is the average number of analysts on IBES providing ar estimate of annual earnings over the twelve months of the company's fiscal year. Int is the intercept from the regression. Mkt Val is the market value of outstanding equity at the beginning of the fiscal year. SD ROE is standard deviation of net income divided by shareholders' equity and Corr is the correlation between annual returns and earnings, each calculated over the 10 years prior to the observation. AnnRpt, OPub, InvRel and Total are the annual report, other publications, investor relations and total score, respectively, based on th FAF scores for each category of disclosure. N is the number of observations and R2 is the adjusted R2 for the second stage regression The first stage R2 is the adjusted R2 from the firs stage regression.,e;h le;t

Table 7, Panel A for Standard Deviation of Analyst Forecasts Recression -- -- -- -- I II... Int 0.28 (9.2) 0.25 (8.5) 0.27 (9.2) 0.20 (9.0) 0.29 (9.3) Surp 0.28 (11.4) 0.28 (11.8) 0.28 (11.9) 0.32 (16.3) 0.28 (11.4) % New 0.31 (13.0) 0.30 (12.9) 0.30 (12.9) 0.32 (16.7) 0.30 (12.8) Mkt Val -0.25 (-9.6) -0.26 (-10.4) -0.25 (-9.8) -0.19 (-9.4) -0.24 (-9.3) SD ROE 0.17 (6.9) 0.17 (7.0) 0.17 (6.9) 0.17 (8.6) 0.17 (6.8) Corr AnnRpt 0.06 -0.11 (2.5) (-4.4) 0.07 (3.0) 0.07 (3.0) 0.06 (3.4) 0.06 -0.09 (2.5) (-3.0) OPub InvRel Total N R2 1153 0.38 -0.06 (-2.5) -0.10 (-4.4) 1200 0.37 1200 0.38 -0.07 1850 0.38 (-3.8) 1153 0.38 0.04 -0.07 (1.2) (-2.6)

Table 7 Continued, Panel B Regression for Accuracy of Analyst Forecasts Relative to a Random W Jl. 1 1 11 II I ralk Int Surr % New Mkt Val SD ROE Corr AnnRpt OPub InvRel Total N R2 0.16 0.67 (5.5) (29.0) 0.15 0.66 (5.3) (28.7) 0.16 0.66 (5.6) (28.7) 0.15 0.67 (7.2) (36.6) 0.13 0.67 (4.6) (29.0) -0.18 0.13 0.04 -0.03 0.05 (-8.2) (5.2) (1.8) (-1.5) (2.4) -0.18 0.12 0.03 -0.04 (-8.2) (5.1) (1.5) (-1.7) 0.10 (4.4) 1159 0.43 1206 0.42 1206 0.42 -0.18 0.12 0.04 -0.04 (-7.8) (4.9) (1.6) (-1.8) 0.08 (3.6) -0.17 0.12 0.03 -0.02 (-9.2) (6.4) (1.7) (-1.1) 0.05 1858 0.43 (2.7) -0.19 0.12 0.04 -0.03 -0.01 0.08 (-8.3) (5.0) (1.8) (-1.4) (-0.4) (2.6) 0.04 (1.6) 1159 0.43

Table 7 Continued, Panel C Regression for Revision Volatility, ",. ", ~, " ' ' ",,., '?,'. '"'. " ". lift,. ",. ",.. ' ', ',.., "..~~~~~~~~ Int Surp % New Mkt Val SD ROE 0.10 (4.3) Corr AnnRpt OPub InvRel Total N R2 0.32 0.21 0.41 -0.27 (10.9) (8.5) (17.6) (-10.7) 0.01 -0.11 (0.5) (-4.5) 1159 0.38 0.31 0.21 0.41 -0.29 (10.5) (9.0) (17.8) (-11.6) 0.33 0.21 0.41 -0.27 (11.5) (9.1) (17.8) (-10.9) 0.25 0.24 0.44 -0.22 (11.4) (12.6) (23.3) (-11.1) 0.11 (4.4) 0.10 (4.3) 0.12 (6.3) 0.02 (1.0) -0.08 (-3.4) 1206 0.38 0.02 (1.0) -0.14 (-6.0) 1206 0.39 0.02 (1.1) -0.10 1858 0.40 (-5.2) 0.34 0.21 0.41 -0.26 (11.4) (8.6) (17.5) (-10.2) 0.10 (4.2) 0.01 -0.07 0.03 -0.11 (0.4) (-2.3) (0.9) (-4.1) 1159 0.39

Table 7, Notes Regressions are estimated using ordinary least squares, with all variables ranked and then computed as percentiles within the industry-year. T-statistics are noted in parentheses below each coefficient estimate. Standard Deviation of Analyst Forecasts is the average standard deviation of analyst forecasts of current year earnings divided by the reporting date price computed over the twelve months of the company's fiscal year. Accuracy of Analyst Forecast Relative to a Random Walk is the average of the absolute value of the forecast error from a random walk model of annual earnings less the absolute value of the analyst forecast error divided by price as of the IBES report date computed as the average over the twelve months of the company's fiscal year. Revision Volatility is the standard deviation of change in median analyst forecast divide by price at the IBES report date computed over the twelve months of the company's fiscal year. Int is the intercept from the regression. Surp is the absolute value of the change in earnings per share from the previous year divided by beginning of fiscal year price. % New is the average of the number of analysts on IBES revising their annual earnings forecast divided by the total number of analysts providing an earnings forecast for the month computed over the twelve months of the company's fiscal year. Mkt Val is the market value of outstanding equity at the beginning of the fiscal year. Sd ROE is the standard deviation of net income divided by shareholders' equity and Corr is the correlation between annual returns and earnings, each calculated over the 10 years prior to the observation.Annual Report, Other Publications and Investor Relations and Total Score are based on the FAF scores for each category of disclosure. N is the number of observations and R2 is the adjusted R2 for the regression.

APPENDIX A 1988 AEROSPACE SUBCOMMJITEE REPORT Aerospace Recommendation For Award For the first time since 1983, the Aerospace Subcommittee recommended that only one award for corporate reporting excellence be granted. General Dynamics was the nearly unanimous selection to receive the award because of its responsiveness and consistency in reporting financial information. Overall Comments The subcommittee's purpose is to evaluate the financial reporting practices of aerospace-related companies so that investors have the information necessary to make informed judgments. The survey is designed to assess the comparability of corporate results, earnings quality, and the disclosure of information relevant for the projection of future results. To fairly evaluate the companies in the survey, the subcommittee used both quantitative and qualitative data. The industry as a whole makes a superior effort to keep investors abreast of the changing markets, regulations, and laws. With the exception of Lockheed, which fell to number two, the top four companies repeated their previous year's rankings. The differences separating the top four from the others narrowed, but each of the companies at the top went beyond required disclosures by providing better access to management, substantial relevant operating data, and a complete discussion of its product/market performance and outlook. Evaluation and Review The communications and reporting practices of 11 companies were reviewed by eight subcommittee members. Two previous subcommittee members resigned and three were added. The companies induded in the survey were the same as last year: Boeing, General Dynamics, Grumman, Lockheed, Loral, Martin Marietta, McDonnell Douglas, Northrop, Raytheon, Rohr, and United Technologies. The evaluation criteria were grouped into three major categories and 21 subcategories. The major groups were annual reports and 10-Ks (45 points), quarterly and other published material (30 points), and analyst relations (25 points). The maximum number of points was 100. The number of evaluators is indicated in brackets next to each company's name in the summary table that follows the narrative portion of the subcommittee report. Four of the companies were reviewed by fewer than all members of the subcommittee. Summary of Results Overall, the scores for the 11 companies narrowed modestly, with the average score somewhat lower than the year before. The mean score was 69.5 in 1989 and 70.8 in 1987. Only two companies produced better scores than in the year-ago period, but five companies had scores better than the average, as compared with four in 1987. General Dynamics captured the top spot on the strength of its annual report and its analyst relations. The company was first in each of those categories but fourth in quarterly reporting. Lockheed took the runner-up spot in both the annual report and the quarterly categories. Raytheon captured the second place spot for analyst relations, and United Technologies again captured the top spot for quarterly reporting. As was the case in 1987, the top three annual reports significantly outdistanced the others. Northrop, the third-place holder, finished a mere fourtenths of a point behind General Dynamics. Each company disclosed significant program data and each provided meaningful discussion of factors that influenced results. The next two (United Technologies and McDonnell Douglas) produced reports that were considered above average. Quarterly reports tended to fall into two categories: those that provide segment data and those that fail to do so. Seven of the 11 companies provide fairly complete quarterly statements and are generally given high marks for it. The top five go beyond the requirement and provide a substantive variance analysis or explanation of factors influencing performance during the quarter. The subcommittee again ranked General Dynamics as having the best analyst relations. Raytheon and

lk- - ~ — r — ~ - dmbmft m - - ' - - -- Y --.C —~~F ~~- y~P~l~iBa J Ae-om I I 1 i United Technologies rose one spot, and Grumman ranked fourth. Lockheed fell to number seven, and both Northrop and McDonnell Douglas fell by one position. Company Comments Boeing: Boeing earned the last place spot in the survey because of its inadequate disclosure, lack of quarterly segment data, and low rating for analyst relations. The subcommittee recognizes the companys need to guard proprietary data and does not wish to receive such information but does seek a better discussion of the factors that influence profitability. The information that Boeing provides, however, is sometimes overlooked or misunderstood. The company deserves substantial credit for disclosing projected deliveries, sales, research and development, and capital expenditures. Several subcommittee members commented favorably on the meeting format used by the chairman to address the investment community. The company is also willing to attend conferences and is generally considered evenhanded in its dealing with investors. Its Current Market Outlook is considered an invaluable tool. Quarterly press releases covering earnings are considered useful as well. General Dynamics: The company received high marks from all subcommittee members because it was willing to communicate both its strategies and its operating outlook. The annual report contained delivery data by program, indicated profitability of significant programs, and provided an outlook of the upcoming year for selected programs. A timely restatement for FAS 96 was provided. The annual also had an excellent ten-year summary containing backlog, head count, and employee compensation. The stock price range could be added. Quarterly segment data are available in the press release; the quarterly report contains a funds flow statement. The 10-Qs and 10-Ks are mailed as part of the ordinary course of communicating with analysts, which is helpful. Analyst relations were considered superior because the company willingly discusses problems as well as opportunities, provides above-average follow-up to questions, and offers analysts opportunities to meet with operating officers. The company's analysts and investors meeting was considered worthwhile. The subcommittee was pleased to recommend that General Dynamics receive an Award for Excellence in Corporate Reporting. Grumman: The company maintained its overall ranking because it provided a better annual report and its analysts relations program was recogr&i. Among Grumman's positive annual report featusnn were disclosure of delivery schedules and projetledl capital spending, a solid discussion of the swiings:i program sales, and a review of new contract awards. The letter to shareholders was considered urmealistic, on the other hand, and it failed to address tlw real issues facing the company. The summary table txnldd benefit by the inclusion of salary and wal;e data. Management is considered accessible, a nd the company-sponsored meeting enhances invesi on;' understanding of the company's outlook. Lockheed: The subcommittee offered signiiiant praise for the excellent level of program dltail. Its Behind the Numbers section provides the t'esl: discussion of any company's operations. It not clrdiy covers items that have occurred but also touches upon events that might develop, giving hivestors sufficient information to formulate appropriate: eqNc.:tations. The company deserves substantial a edit for carrying this commentary to its quarterly report; as well. A dear discussion of strategies and objectives:is carried in both the annual and the quarterly reports. Several members commented favorably upon the accessibility of management. On the other side of the ledger, a better explanation of the ESOP program and its future financial ramifications could have been provided as could more complete coverage of the company's electronics and technology segments. The company's quarterly statements could also contain research and development expenditures, the number of shares outstanding at the end of the quarter, iud book value. Loral: Several subcommittee members xcommented that the annual report had improved over the previous year's because it discussed major programs more completely. However, few details existed in the annual to support the chairman's forecast. On the other hand the potential contribution from the ccrnmpanys most recent acquisition was provided. The summary of financial history could contain additional details such as book value, head count, andt sallr: and wages. Coverage of the charge for early rertiement would have been better discussed under mtnnagement's review of operations as opposed to a Folint in the pension footnote. Restated data subsequent: bt an acquisition were quickly available. The qlarterly reports, on the other hand, could contain addiitional data such as a cash flow statement. The conlpany's chairman continued to accommodate analyst re.uests and provided a rapid response to questiors. The meeting held for analysts subsequent to the cxmpan/s annual meeting is considered a valwuibhl forum. I I

Martin Marietta: To its credit Martin Marietta was one of the two companies that earned higher scores in 1988 as compared with 1987. Improvement was recognized in all three categories but overall the reporting unfortunately still fell short of a superior effort. The company flirts with increasing its level of discosure but appears reluctant to complete the task. The 10-Qs contain a wealth of information; more of it should be incorporated into the quarterly report. The summary balance sheet in Martin Marietta's quarterly reports hurt its score; it was the only company to abbreviate its statements this way. The annual reports operational review section provided substantial and useful program-related information covering both revenues and profits. Although the footnotes contained a wealth of data about the environment, the subcommittee felt that the shareowners' letter could have benefited from a better discussion of the topic. On the other hand, several subcommittee members praised Martin Marietta's willingness to meet with analysts and to introduce operating management to investors. McDonnell Douglas: The subcommittee generally agreed that the company's quarterly reports and annual reports provide a superior level of financial reporting. The company discloses its mix of military and commercial revenues in the 10-K and produces above-average discussion of the factors influencing operations. Several subcommittee members remarked that the chairman's letter provided an excellent discussion of the potential earnings risks for the upcoming year as well as a frank assessment of the operating environment. The company also provided an indication of the potential growth in the Transport Aircraft segment and an indication of the future of the Information Systems Group. Like most of the companies in the survey, however, the company is reluctant to disclose the spending associated with the Advanced Tactical Fighter. McDonnell Douglas deserved high marks for its willingness to disclose the unusual items that affected its results. Delivery schedules for the upcoming year would be helpful, however. A timely restatement for the adoption of FAS 94 was provided. In the area of analyst relations, the relative newness of the company's investor relations team may have been one factor hurting its overall score. In addition providing only one primary contact might be more helpful. The company is willing to permit analysts to meet with senior management and appears willing to increase its contact with investors; the latter could serve to improve the company's analyst relations score in the upcoming year. Northrop: The company continues to produce one of the best annuals and series of quarterly reports in the industry. Investors are given substantial program data that enable the investor to evaluate performance. In addition to providing a capital spending forecast, the company provides a basis to estimate spending on the Advanced Tactical Fighter. Projected deliveries for its major unclassified programs re disclosed, and the mix between development and production contracts is indicated. The receivables for the Advanced Tactical Fighter program are also disclosed, providing an indication of cash flow. Given its recent operating performance, it is not surprising that the company has suffered a bit for its analyst relations. However, the subcommittee appreciates the contact with the top financial officer, and several members complimented the company for its cooperation in arranging individual visits. Raytheon: Despite its superior analyst relations, the company ranked near the bottom of the industry. Both the annual report and the quarterly reports lack sufficient data to enable comparison of the company's operating performance with that of others. Several subcommittee members again remarked that the lack of segment data in the quarterly reports and the inclusion of interest and other income in the segment results tended to mask the various units' performance. Although the annual report provided good coverage of the company's businesses, it could have provided a better review of segment strategy. There also could have been a more complete discussion of the company's prospects. The company's analyst contact is considered helpful, objective, and honest. Rohr: The company's reporting practices were considered satisfactory. The shareholder letter covered the company's prospects and strategies. The annual report also did an excellent job of explaining the company's business products. Sufficient quarterly information was given to indicate the company's direction, and the company was willing to provide substantial supplemental data. A better discussion of the individual factors influencing operating performance could be disclosed, however, and the company could discuss unit shipments more fully. The summary of historical data could also contain head count and salary and wages. The company's meetings with investors were considered helpful. United Technologies: The company scores well in all three of the categories used to evaluate financal reporting. Subcommittee members found the wealth of quarterly data provided as a supplement to the quarterly reports useful and cted the efforts made to

10 Aerspace provide quarterly reports that disclosed segment operating profits. The company's annual report contained sufficient data to gauge the size of the company's many operating businesses and calibrate operating performance excluding unusual charges. The footnotes were of superior quality, especially with regard to foreign business exposure. Several subcommittee members praised the analyst relations effort but noted that both the fact book, an unusually valuable tool, and quarterly meetings with investors were curtailed. The subcommittee expressed its hope that other forums will be employed to maintain the high level of credibility built over the past year or two. SUMMARY OF 1988 AEROSPACE EVALUATIONS Company (# of Responses) Boeing (8) General Dynamics (8) Grumman (6) Lockheed (8) Loral (8) Martin Marietta (8) McDonnell Douglas (7) Northrop (5) Raytheon (8) Rohr (7) United Technologies (8) Mean Score Maximum Score Annual & 10-K 30.5 39.0 32.2 38.8 27.5 Quarterly & Other Published Material 18.0 22.6 20.2 24.5 15.3 Analyst Relations 11.0 22.4 17.5 16.4 17.4 1988 Total Points 59.5 84.0 69.8 79.6 60.1 1987 Total Points 61.3 84.3 69.8 84.4 63.7 Ranking (1987 Rank) Annual Quarterly Relations Total 7 (6) 9 (8) 10 (11) 11 (10) 1986 Rank 11 1985 1984 Rank Rank 9 11 1 (2) 6 (8) 2 (1) 10 (9) 4 (4) 7 (6) 2 (2) 11 (10) 1 (1) 4 (7) 7 (2) 5 (5) 1 (2) 5 (5) 2 (1) 10 (9) 29.5 15.8 17.1 62.4 57.9 8 (11) 10 (11) 6 (9) 9 (11) 3 6 6 5 11 7 1 2 1 9 - - 10 8 8 8 3 3 1 1 2 7 7 9 4 10 33.6 38.6 27.0 28.3 22.3 23.4 18.4 21.0 9.7 13.2 18.8 16.3 65.6 75.2 64.1 65.6 66.7 77.8 66.6 68.6 5 (5) 3 (3) 11 (10) 9 (7) 5 (2) 3 (5) 8 (9) 6(7) 11 (10) 9 (8) 2 (3) 8 (6) 6 (7) 4 (4) 8 (8) 6 (6) 35.0 25.8 18.1 78.9 78.1 4 (4) 32.7 20.6 16.2 69.5 70.8 1 (1) 3 (4) 3 (3) 5 4 4 45.0 30.0 25.0 100.0 100.0 AEROSPACE SUBCOMMITTEE Howard A. Rubel, Chairman, New York C. J. Lawrence, Morgan Grenfell Inc. Steve Binder, CFA, New York Bear, Stearns & Co. Inc. Judith L. Comeau, New York Goldman, Sachs & Co. William N. Deatherage, CFA, New York Dean Witter Reynolds Inc. Thomas O. Lloyd-Butler, San Francisco Montgomery Securities Gregory A. McCrickard, CFA, Baltimore T. Rowe Price Associates, Inc. John A. Modzelewski, CFA, New York PaineWebber, Inc. Philip W. Friedman, New York Drexel Burnham Lambert Inc.

35 References Alexander, J. "Do the Merits Matter? A Study of Settlements in Securities Class Actions." Stanford Law Review (1991) 43:497-598. Baginski, S. and J. Hassell. "The Market Interpretation of Management Earnings Forecasts as a Predictor of Subsequent Financial Analyst Forecast Revision." Accounting Review (January 1990) 65:175-90. Barry, C. and S. Brown. "Differential Information and Security Market Equilibrium." Journal of Financial and Ouantitative Analysis (1985) 20:407-22. Bhushan, R."Firm Characteristics and Analyst Following." Journal of Accounting and Economics" Vol. (1989) 11: 255-74. Bound, J., D. Jaeger and R. Baker. "The Cure Can Be Worse Than The Disease: A Cautionary Tale Regarding Instrumental Variables." NBER technical paper # 137 (June 1993). Brennan, M., and P. Hughes. "Stock Prices and the Supply of Information." Journal of Finance (December 1991) 46:1665-91 Brown, P., G. Foster, and E. Noreen. Security Analyst Multi-Year Forecasts and the Capital Market, American Accounting Association. Sarasota, Florida. (1985) Brown, L. and J. Han. "The Impact of Annual Earnings Announcements on Convergence of Beliefs." Accounting Review (1992) 67: 862-75.

36 Brown, L., G. Richardson, and S. Schwager. "An Informational Interpretation of Financial Analyst Superiority in Forecasting Earnings." Journal of Accounting Research (Spring 1987) 25:49-67. Byrd, J., M. Johnson, and M. Johnson. "Investor Relations and the Cost of Capital." University of Michigan working paper (1993) Cooper, W. "Innocents Abroad." Institutional Investor (February 1989): 189-92. Financial Analysts Federation. Report of the Financial Analysts Federation Corporate Information Committee. New York, NY, 1985, 1986, 1987, 1988, 1989. Fishman, M. and K. Hagerty. "Disclosure Decisions by Firms and the Competition for Price Efficiency." Journal of Finance (1989) 44:633-46. Foster, G. Financial Statement Analysis. New Jersey. Prentice-Hall. 1986. Francis, J., D. Philbrick and K. Schipper. "Shareholder Litigation and Corporate Disclosure Policies." working paper, University of Chicago (1993). Glosten, L. and P. Milgrom. "Bid, Ask, and Transaction Prices in a Specialist Market with Heterogeneously Informed Traders." Journal of Financial Economics (1985):71-100. Iman, R.L. and W.J. Conover. "The Use of Rank Transformation in Regression." Technometrics (1979) 21:499-509.

37 Jennings, R. "Unsystematic Security Price Movements, Managerial Earnings Forecasts, and Revisions in Consensus Analyst Earnings Forecasts." Journal of Accounting Research (1987) 25: 90-110. King, R., G. Pownall, and G. Waymire. "Expectations Adjustments via Timely Management Forecasts: Review, Synthesis, and Suggestions for Future Research." Journal of Accounting Literature (1990) 9:113-44. Klein, R. and V. Bawa,, "The Effect of Estimation Risk on Optimal Portfolio Choice." Journal of Financial Economics (1976) 3:215-31. Knutson, P. "Financial Reporting in the 1990's and Beyond: A Position Paper of the Association for Investment Management and Research." University of Pennsylvania working paper (1992). Kross, W., B. Ro, and D. Schroeder. "Earnings Expectations: The Analysts' Information Advantage." Accounting Review (1991) 65: 461-76. Lang, M., and R. Lundholm. "Cross-Sectional Determinants of Analyst Ratings of Corporate Disclosures." Journal of Accounting Research (1993) 31:246-71. Lees, F. Public Disclosure of Corporate Earnings Forecasts. The Conference Board: New York (1981). Mahoney, W. Investor Relations: The Professional's Guide to Financial Marketing and Communications. New York Institute of Finance: New York (1991).

38 Marcus, B. and S. Wallace. Competing in the New Capital Markets: Investor Relations Strategies for the 1990s. HarperBusiness: New York (1991). Merton, R. "Presidential Address: A Simple Model of Capital Market Equilibrium with Incomplete Information." Journal of Finance (1987) 42:483-510. Morse, D., J. Stephan, and E. Stice. "Earnings Announcements and the Convergence (or Divergence) of Beliefs." Accounting Review (1991) 66:376-88. Nichols, D. The Handbook of Investor Relations. Dow Jones-Irwin: Homewood, IL. (1989). O'Brien, P. and R. Bhushan. "Analyst Following and Institutional Ownership." Journal of Accounting Research (Supplement 1990) 28: 55-82. Schipper, K. "Information Transfers." Accounting Horizons. (December 1991) 4:105-21 SRI International. Investor Information Needs and the Annual Report. Financial Executives Research Foundation: Morristown, NJ. (1987). Swaminathan, S. "The Impact of SEC Mandated Segment Data on Price Variability and Divergence of Beliefs." Accounting Review (1991) 66:23-41. "The 1988 All-America Research Team." Institutional Investor (October 1988):98-184. Verrecchia, R. "Discretionary Disclosure." Journal of Accounting and Economics (1983) 5:179-94.

39 Endnotes lAs we use the term, "disclosure policy" includes both the amount of information disclosed and the medium of disclosure chosen by the firm. Further, "informative" or "forthcoming" disclosure policies, as we use the terms, are informative and effective in communicating with investors. This is the construct that the Financial Analysts Federation's Corporate Information Committee Report purports to measure. 2Note, however, that Wall Street Journal coverage also captures the impact of exogenous events and the paper's editorial policy. Other related research includes Baginski and Hassell [1990] and Jennings [1987] who provide evidence that analysts revise their forecasts in response to management earnings forecasts. Given that management forecasts tend to be more accurate than analyst forecasts, all else equal, analyst forecast accuracy should be higher for firms issuing management forecasts. In addition, Brown, Richardson and Schwager [1987] show that forecast accuracy (relative to a random walk) is positively associated with firm size and negatively associated with the dispersion of analyst forecasts. 3See Schipper [1991] for a discussion of research on analysts and directions for new research. 40f course, there may well be costs associated with a forthcoming disclosure policy. These include potential litigation (see Francis, Philbrick and Schipper [1993]), revealing information to competitors or simply the direct cost of preparing and communicating the disclosures. See Lang and Lundholm [1993] for a summary of the costs and benefits of a forthcoming disclosure policy. 5By focusing on earnings forecasts, our analysis ignores other functions of analysts such as issuing buy/sell recommendations and generating trades. It is unclear how an analyst weights each of these different activities, but we assume that the choice of which firms to provide forecasts for and then making the forecasts are significant parts of the analyst's job.

40 6As an example, Robert Dunlap of Irving Trust is quoted in Nichols [1989] as stating, "I don't follow Pullman because they won't tell you enough about their business to allow you to get a handle on it.... If they change and become more open with the street, there is no doubt that I'd take more of an interest in Pullman." 7Although we think it is unlikely, it is possible that the number of analysts determines the disclosure policy, rather than the other way around, as we have hypothesized. How analysts would pressure firms into more disclosure, other than choosing to quit following the firm, is unclear. They could issue sell recommendations or unfavorable forecasts, but if these actions do not reflect their underlying beliefs, but are simply punitive, then they are quite costly to the analysts. We have not incountered any anecdotal evidence of this nature; to the contrary, see footnote 6. 8This possibility is considered in Kandel and Pearson [1992]. 9A formal example of decreasing consensus with more informative disclosure is as follows. Suppose that a more informative disclosure policy reported Yi and Y2 while a less informative policy reported only Y1+Y2. Further, suppose that Yi = F+ei, i=1,2, where F is next period's earnings and Ej(FI.) is analyst j's forecast; V(F)-l=v, and V(Ei)-l=sij, so that each analyst j has a potentially different assessment of the variance of each signal's error term; and F, e1 and 62 are independent and normally distributed. Finally, suppose that s12=s21 and sll=s22; analysts 1 and 2 agree on the sum of the signal precisions but they disagree about which signal has which precision level. In this case, El(FIYI+Y2) - E2(FIYl+Y2) = 0 (because the analysts agree on the sum of the signal precisions) but the absolute value of El(FIY1,Y2) - E2(FIY1,Y2) is positive for all but a set of measure zero. Thus, a more informative disclosure leads to less consensus in forecasts. 10Formally, denote the realized earnings per share by Z and consider two information signals Xl and X2. The sequence of forecasts is E(Z), E(ZIXi), E(ZIX1,X2) and E(ZIZ). With normally distributed random variables it can be shown that, as precision is subtracted

41 from X2 and added to X1, the variance of the forecast revision E(ZIX1,X2) - E(ZIX1) decreases. 11A comparison of the analysts included on the subcommittees with those selected for the Institutional Investor All-American Research Team suggests a substantial overlap. For example, in 1988, 3 of the 5 analysts in the airline subcommittee and 10 of the 18 analysts in chemical subcommittee were members of the 1988 All-American Research Team. In discussions with analysts, it appears that the primary motive for serving on a subcommittee is visibility and an indication of status in the analyst community. 12The stated construct being measured in the evaluation process is the forthcomingness and effectiveness of communicating with investors. However, given the evaluation is conducted largely by analysts, the criteria may really be effectiveness in communicating with analysts. 13The larger firms in the industry are generally reviewed, and the set remains relatively constant over time. A complete list of the general disclosure criteria is given in the appendix of the FAF report and is available from the authors on request. 14See table 1 in Lang and Lundholm [1993] for a summary of the industries in the FAF data. 1SSee Marcus and Wallace [1991] and Mahoney [1991] for a discussion the relative advantages and disadvantages of various disclosure media 16Given that the dependent variables ine the analysis are the number of analysts following the firm and characteristics of the analysts' forecasts, it might appear that the same analysts in the FAF subcommittees are included in the IBES data. However, the potential overlap is limited because the number of analysts on the FAF subcommittee is typically fewer than the number in the IBES database and FAF subcommittees include buy and sell-side analysts and portfolio managers while the IBES data is limited to sell-side analysts.

42 17A related issue is that, in addition to measuring current disclosure, a goal of the FAF is to increase the forthcomingness of firms' disclosure. To the extent that the evaluation process is effective in achieving this goal, the firm's current disclosure policy is not independent of past scores. 18For example, the number of analysts following a December 31 year-end firm for 1985 is computed as the sum of the number of analysts following the firm at the IBES reporting date for each of the months January through December 1985, divided by twelve. 19Brown, Richardson and Schwager [1987] and Kross, Ro and Schroeder [1991] both use a similar measure. Results are very similar if accuracy is measured as the absolute value of the analyst forecast error deflated by price with no adjustment for the random walk forecast error. 20Disclosure policies might also affect the bias in analyst forecasts. For example, analysts might reward firms who were particularly forthcoming with biased earnings forecasts. However, Lang and Lundholm [1993] provide evidence that positive forecast errors are associated with more disclosure. Although this result could imply that analysts bias their forecasts downward for firms who disclose more, we feel the more likely explanation is consistent with the notion that, when disclosure is costly, firms are more willing to report good news than bad news (see, for example, Verrecchia [1983]). 21We also performed three specification checks for alternative control variables in the number-of-analysts regressions. First, we included the number of institutional investors and the percent of shares outstanding held by institutions in 1987, reflecting Bhushan's [1989] evidence that analyst following is increasing in institutional ownership. Second, we included the firm's share price, based on Brennan and Hughes [1991] result that the number of analysts is higher for firms with lower share prices. Finally, we included the other three dependent variables as independent variables in each regression. The results are not sensitive to any of these variations.

43 22Because the sample firms are large on average, care should be taken in generalizing the results to other populations. Marcus and Wallace [1991] and Mahoney [1991] suggest that, because of the competition for analyst attention and limited alternate sources of information, disclosure policy is likely to have a greater impact for small firms than for large firms. 23See Iman and Conover [1979], for a discussion of rank regression. 2Significance levels for the simple correlations should be interpreted with caution as the same firm may appear multiple times in the sample, potentially inducing dependence in the data. A more detailed discussion of this issue is presented in the next section in the context of the regression results. 25Significance levels for the multiple regressions should be interpreted with some caution because some of firms are included in the analysis for consecutive years, which may induce autocorrelation in the residuals. We have conducted specification checks which specifically take first order autocorrelation into account with essentially identical results. As a more extreme check, we also conducted the analysis using only one observation per firm with very similar results. Finally, a firm occurs in our sample on average three times, so at most the number of independent observations is overstated by three, causing the t-statistics to be overstated by at most a factor of S3 = 1.7. Unless otherwise stated, we refer to a result as "significant" if the p-value for a two-tailed test is less than.01. 26The results for firm size are consistent with the hypotheses of King, Pownall and Waymire [1990], but the results for the earnings variability and the returns/earnings correlation are not 27Because the choice of the most relevant control variables is not clear, we include the same set of control variables in each regression for consistency, but the results are not particularly sensitive to the choice of control variables.

44 28This result is also consistent with the interpretation that analysts have no private information, but a more informative disclosure practice serves to homogenize analysts' forecast models. 290f course, the firm must trade off these benefits against the costs of a more forthcoming disclosure practice.

5 I f l I r, ~ ~ ~ ~ ~~ ~ ~ ~~ ~ ~ ~~ ~ ~ ~~ ~ ~ ~~ ~ ~ ~~ ~ ~ ~~ ~ ~~~ ~ ~~~ ~ ~~~ ~ ~~~ ~ ~~~ ~ ~~~ ~ ~~~ ~ ~~~ ~ ~~ ~ ~~~ ~ ~~ ~ ~~ ~ ~~ ~ ~~ ~ ~~ ~ ~~ ~~~ ~~~ ~~~ ~~~ ~~~ ~~~ ~~~ ~~~ ~~~ ~~~ ~~~ ~~~ ~~~ ~~~ ~~~ ~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~,