Division of Research School of Business Administration November 1987 TOWARD A CONCEPTUAL USAGE MODEL OF MARKETING RESEARCH Working Paper #536 James R. Taylor Michael J. Ryan The University of Michigan Nancy A. Haskell Laval University FOR DISCUSSION PURPOSES ONLY None of this material is to be quoted or reproduced without the expressed permission of the Division of Research. The authors are indebted to Donald Barclay for his help in data analysis, Gerry Tellis for his aid in constructing the experimental design, and Richard Bagozzi, Claes Fornell, Thomas Kinnear, and Rohit Deshpande for their comments on an earlier draft. Copyright 1987 University of Michigan School of Business Administration Ann Arbor Michigan 48109

I

TOWARD A CONCEPTUAL USAGE MODEL OF MARKETING RESEARCH ABSTRACT A model is proposed to organize antecedents and outcomes from the conceptual use of marketing research. Relationships among price, options, attitude, competency, decision quality, and performance were examined in an eight-year simulated marketing environment. Dynamic variable relationships supported conceptual usage. For example, the information usage-decision quality relationship developed over time after being initially overshadowed by the user's competency. Suggestions are made regarding the need to explore the user's learning process and the need for appropriate information delivery and support systems.

I

TOWARD A CONCEPTUAL USAGE MODEL OF MARKETING RESEARCH INTRODUCTION The use of marketing research information is generally thought to improve decisions through the reduction of uncertainty (c.f., Buzzell, Cox, and Brown 1969; Kinnear and Taylor 1979) thereby leading to better performance. However, there is considerable evidence that both basic (Myers, Massy, Greyser, 1980) and problem-oriented research (Adler and Mayer 1977; Bellenger, 1979; Dyer and Shimp 1977; Holbert 1974; Krum 1978) is under-utilized. Deshpande and his associates (Deshpande, 1982; Deshpande and Jeffries, 1981; Deshpande and Zaltman, 1982, 1983, 1984) and Lee, Acito, and Day (1987) found that information is less likely to be used if it fails to correspond to the decision maker's prior beliefs. Deshpande and Zaltman (1982) also report that the higher the perceived technical quality and the greater the perceived actionability of the information, the greater the information usage. In addition to their individual level variables, Deshpande and Zaltman (1982) also report that organizational structure and manager-researcher interaction relate to research usage. Deshpande and Zaltman (1984) found that researchers and managers differed in the relative importance of these variables. Deshpande, et al., and Lee, et al., investigated instrumental or short term information usage. That is, they examined usage within the context of a specific decision rather than a long term or series of decisions. Lee, et al., considered usage to be reflected in changes in the decision maker's confidence and/or demonstratable, variable effects on the decision itself. Deshpande, et al., examined perceived usage. This research will investigate the long term or conceptual use of research information. We will also extend the scope of previous studies by examining the effects of usage on decision quality and performance. The paper will proceed in four steps.

-2 - First, we will specify our dependent variable, information use. Second, we will specify an overall approach or general model. Third, we will specify variable relationships. Fourth, we will operationalize and test parts of the model in an experimental simulation. THE NATURE OF CONCEPTUAL INFORMATION USAGE Empirical studies of public policy decision makers (cf. Rich, 1975; Weiss and Buculavas, 1977, 1980) have found that research information did not influence specific program decisions, yet decision makers report that they value the information. In an effort to explain this state of affairs, Rich (1977) coined the following terms: Instrumental use refers to those cases where respondents cited and could document (or it could be cross-validated) the specific way in which the information was being used for decision-making or problem-solving purposes. Conceptual use refers to influencing a policy-maker's thinking about an issue without putting information to any specific documentable use. It also refers to planned uses of information in the future. None of the marketing or public policy research has examined conceptual use. As Deshpande and Zaltman (1982) note, it is difficult to investigate conceptual usage in a managerial setting. Obviously, to do so would involve some sort of longitudinal design. More recently, however, Wilton and Myers (1986) carried out a consumer behavior study in which they found that information utility and use were higher in conceptual rather than instrumental tasks. Consequently, one is left with the nagging notion that there is more to managerial use of information than may be reflected in one particular outcome. It seems that information would be stored and retrieved from memory for use in subsequent decisions. The general model that follows will reflect information usage and performance over time.

-3 - PROPOSED GENERAL MODEL Our proposed general model attempts to reach multiple objectives. We take as a given that the research information is relevant and actionable. The problem, then, is influencing managers to use the information. Exogenous variables were chosen on the basis that they are easily subject to managerial control in a "real world" environment. In this regard, individual rather than organizational level variables were employed. Variable relationships were based on analogous conceptual frameworks and empirical support. Finally, the model conformed to the environment available to us within which we could simulate ongoing marketing decisions and manipulate the exogenous variables. The structure of our proposed model is shown in Figure 1. Our approach Insert Figure 1 About Here views marketing research as a product/service and managers as consumers and follows the structure common to general models of buyer behavior (c.f., Howard & Sheth 1969). Thus, antecedents, behavioral moderators, and outcomes are shown as three blocks of causally linked variables. A brief explanation of why variables were chosen for inclusion follows, after which a detailed rationale for the proposed relationships is developed. Whereas a number of individual level variables (including some of those mentioned above from earlier studies), could serve as antecedents to attitude and be subject to managerial control, three variables seemed appropriate both in terms of value cues and our ability to manipulate them. These variables are as follows: availability of options, the degree to which the user can selectively choose rather than having to purchase all available information; price, the monetary cost of the information to the user; and availability of

-4 - funds, the amount of budgeted resources available to purchase information. Our reasoning is that, similar to any service purchase, these attributes serve as value cues. Deshpande and Zaltman define usage as the perceived value of the information as it pertained to the decision at hand. Since we focus on conceptual rather than instrumental usage, we separate perceived value from a specific decision. Lee, et al., view information usage only as a behavior since they define it as a change in the particular decision. We separate value and usage into attitudinal and perceived behavior components, which we call attitude toward research information and perceived information usage. Thus, attitude is viewed in the traditional cognitive sense as a precursor to behavior. Both of these variables are perceptual indicators in the model and are purposefully defined in very general terms. The failure to demonstrate instrumental usage of information in the public policy literature has led to discussions of the various meanings that the term information use may contain (c.f. Machlup 1979). Since we are attempting to show linkages to outcomes developing over time rather than a specific, short-term linkage general definitions seem appropriate. In this regard, we follow Ajzen and Fishbein's (1975) presumption that in order to be linked, variables in a model should be defined at the same level of specificity. Consistent with motivational theories (e.g., Atkinson 1964), use of information is probably dependent on both willingness, as reflected in attitude, and ability. Consequently, we have included competency in the model. Decisions and performance variables represent the first attempt to empirically link information usage to outcomes. Previous marketing studies used usage as the dependent variable. Those who have discussed the underutilization issue assume an information use-decision quality link. Research on

-5 - decision making in general, however, (c.f., Hogarth 1981) does not always support this relationship, even in tightly controlled laboratory studies where the decisions are well defined and the number of alternatives are few and obvious. Marketing problems occur in situations that are often not well defined and have no clear-cut alternatives. Such unstructured decisions are characterized by a high degree of uncertainty and often are not made in a formal clear-cut sense, but rather "accrete" (Weiss 1980). Direct and immediate impact of the use of research on decisions of this type is rare; use may, rather, be characterized as having "cumulative consequences" (Weiss and Buculavas 1980). Therefore, we propose that use of marketing research information affects long-term performance (conceptual usage) and is not totally captured in short-term (instrumental usage) results. Having outlined our approach, we now turn to support for the hypothesized relationships and a tentative model of information use, its antecedents, and its impact on decision quality and performance. ATTITUDE AND BEHAVIOR REGARDING INFORMATION Attitude Toward Information Although marketing research information may be pertinent, that is, provide the means for reducing decision uncertainty, it does not follow that decision makers will use it. As with any product, use is probably preceded by evaluation, which must be viewed from the eye of the user, not the producer. Following the attitude literature, we use "overall evaluation," "affect" and "attitude" synonymously (c.f., Fishbein and Ajzen 1975). Those who subscribe to the traditional affective definition of attitude generally view it as a predisposition to behave (c.f., Osgood, Suci, and Tannenbaum, 1957, p. 190), a view that is once again becoming popular (Cooper and Croyle 1984). There is also considerable evidence that attitude influences buyer behavior (Ryan and

-6 - Bonfield 1975). Thus, we propose that perceived information usage is influenced by attitude toward the research information. A pilot study, primarily used to sort out operational issues discussed below, reveals that a good deal of variance in firm performance is accounted for by individual ability. It may be, as shown by Crockett (1965), that these differences play a role in one's ability to process information. It follows, then, that competency would allow greater use and thereby be linked to perceived information usage. Whereas perceived information usage should support decisions, thereby leading to decision quality, competency may also affect decision quality directly. That is, raw ability, even if not combined with higher degrees of perceived information usage, should impact decision quality. ANTECEDENTS TO ATTITUDE Whereas numerous antecedent candidates exist in the behavioral literature, we have chosen a set apt to be present and controllable in a marketing environment. These antecedents reflect our view that attitude arises from characteristics of the decison-making situation (c.f., Isen 1984) as well as the research information itself. These characteristics may be hardly noticeable to a decision maker yet may have a pronounced effect on thought and behavior. Price Both the marketing and social psychology literature support the notion that price tends to affect evaluation. The price/quality relationship has a long history of investigation in marketing literature and it is widely believed that buyers impute more value to highly priced goods than to low-priced goods. The empirical evidence concerning the price/quality relationship, however, is mixed. In an extensive review, Monroe (1973) found support for this relationship over some range of prices for some product categories. He concluded that

-7 - the relationship seems to hold where other evaluative criteria are unavailable. For example, Leavitt (1954) and Tull et al. (1964) found a positive pricequality relationship where price was the only available product information. Monroe (1976), however, found that price as a value cue was mitigated by previous use experience. More closely related to the perceived value of market research information, Medlin (1979) reports a sealed bidding experiment in which there was a positive linear relationship between the perceived value of information and the maximum price offered for it. Based on Monroe and Medlin's work, we propose that price will influence attitude, and that the relationship will weaken over time as users gain experience with the information. Availability of Funds If funding is restricted, this would seem to enhance attitude in two ways. First, a small budget would enhance value perceptions, as a constant research price would become a greater percentage of the total available funds. Thus, availability of funds should be directly related to price. Commodity theory (Brock 1968; Zellinger, et al., 1975) holds that increasing the unavailability of an item increases its value. Thus, we also predict that as funds decline, attitude toward the information will improve. That is, funding impacts attitude directly and through the mediating effects of price. Availability of Options With the advent of computers and the resulting information explosion, the disadvantages of providing managers with more information than they can process have been recognized. Consumer researchers believe that consumers can be overloaded, but there is a controversy over whether this actually occurs (Jacoby, et al., 1974; Jacoby 1974, 1984; Malhotra 1984). Oskamp (1965) found that

-8 - providing people with more information increases subjective confidence in the correctness of a judgment without a corresponding increase in predictive accuracy. One explanation for this finding is that, due to cognitive overload, information is simply lost in the mire (McGuire 1976). From the results of a decision experiment, Pitz (1980) suggests that as people integrate successively more complex levels of information, sensitivity to relevant variables diminishes. Zif (1976) reported that subjects complained about buying mandatory information and questioned its value. One would expect, then, that being able to select a subset of available information would lead to a more positive attitude than receiving a forced set of all available information. It seems that both price and availability of funds would sensitize options. If one is expending large amounts from a limited budget, that is, the greater the price and the lower the funding, the importance of selection of a subset of available information becomes. Thus, in addition to their direct effects, price and availability of funds are both hypothesized to impact attitude through the mediating effects of availability of options. DECISIONS AND PERFORMANCE Evidence concerning the relationships among these variables is scarce and mixed. Zif (1976) found that subjects receiving a fixed set of informationin a marketing simulation outperformed those receiving information on an optional basis. Biggs and Greenlaw (1976) found, in an experiment with three levels of information, that while higher levels of information were associated with increased performance, decision quality remained constant. Martin and Sell (1980) concluded that the utility of information is a function of amount and evaluation of information. Whether more highly valued information leads to greater performance, however, appears to be an open question. It is well known

-9 - that information is sought for a variety of reasons unrelated to decisionmaking, such as safeguarding the decision-maker from future criticisms of poor results (Hogarth 1980). It is also thought, as mentioned earlier, that information aids decision-makers in a general sense over time. Such aids include helping the decision-maker to set out explicit assumptions, to understand the environment, its relevant variables, and their interactions. Therefore, we propose that perceived information use considered broadly —i.e., going beyond instrumental use —impacts performance through decision quality on a cumulative basis. That is, due to the strategic nature of marketing decisions and the inability to directly relate information use to specific decision quality and specific performance, as well as the cumulative consequence associated with information use, as described above, information usage is expected to have more of a long-term than short-term impact on performance. A major difficulty in this type of study is the complex nature of performance, alluded to above. A good deal of decision making research utilizes simulation games which employ probabilistic response functions and competitive interactions. Thus, as in the real world, sound strategic moves do not guarantee performance outcomes, typically measured by return on investment or market share. For example, a company may begin making quality decisions, that is, decisions which successfully integrate research information and judgment, that follow sound marketing principles. However, budget and market position are typically a function of cumulative results from past decisions. Thus, future performance may be partly a function of poor past performance, regardless of the soundness of current decisions. Correct decisions in a strong competitive environment would also less positively affect performance than the same decisions made in the absence of able competitors. Counter-intuitive findings such as those of Biggs and Greenlaw, cited above, together with the

-10 - complex nature of information use and performance, suggest that an accurate assessment of information usage necessitates a simultaneous examination of attitude, decision quality, and performance. The propositions embedded in Figure 1 should be regarded tentatively, as the evidence from which they are drawn is mixed, the background for the propositions comes from disciplines with divergent interests and methodological standards, and the measures have yet to be refined. Thus, we propose it as a first step in imposing a theoretical structure containing inputs, behavioral moderators, and outputs on the conceptual use of marketing research information. METHOD Overview The research setting used to test the above propositions employed MARKSTRAT, a ten-period strategic marketing decision simulation, and teams of MBA students at a leading Mid-Western business school. An experimental design was employed to manipulate the three antecedent variables, price, availability of funds, and availability of options. Analysis was carried out over eight periods: the first two of the ten periods were used to acclimate players to the game and thereby minimize procedural errors, and to equalize starting positions. The eight-period analysis allowed an examination of cumulative performance. Partial least squares (PLS) analysis (Wold 1980 b,c), using a modified version of Lohmoller's (1981) program, was employed to analyze the data, allowing simultaneous estimation of structural and measurement model parameters. Research Setting and Sample MARKSTRAT (Larreche and Gatignon 1977) is a computer-simulated marketing environment that encompasses competitive reaction and environmental uncertainty.

-11 - It has been judged as one of the most realistic simulation games available in marketing (Lambert 1980) and is used extensively by major industrial, consumer, and service firms to improve the decision-making skills of marketing managers and by major business schools to educate MBAs. It has also been used previously as an academic research environment (Hogarth and Makridakis 1981; Ross 1987). Key strategic decisions in the MARKSTRAT environment involve cost dynamics, segmentation and positioning, market evolution and repositioning, and new product development. Marketing objectives are attained through decisions affecting production levels, R & D, mass advertising, advertising research, and channel management. An industry is composed of five competing multi-brand companies which manufacture and market a consumer durable good comparable to a consumer electronic entertainment product. In this study, each company was managed by a three-member team. A company normally can purchase up to 15 marketing research studies in each period to assist in decision-making. These range from traditional consumer surveys, product attribute ratings, market forecasts, and the like, to more sophisticated information such as perceptual maps, sales force experiments, and advertising experiments. At the beginning of each period, each company is given a budget based upon the previous period's net marketing contribution. Two hundred and ten second-year MBA students participated in MARKSTRAT as a marketing planning and strategy course requirement. There were 70 firms assigned to 14 independent industries which were spread across four course sections and two instructors. In order to familiarize students with the game environment and procedural details, and to minimize computer input errors, subjects obtained information about initial starting positions and observed the results of the first two period moves made by the instructors. Inputs and results for these periods were provided, but actual market research reports

-12 - were not made available. Thus, teams, which had access only to dummy reports in the MARKSTRAT manual, began making strategic decisions for Period 3 with the benefit of observing two prior period decisions and results but without marketing research information. As mentioned above, initial starting positions in MARKSTRAT normally gave some firms greater potential for success. Consequently, the first two moves ensured that each of the five firms within a given industry started on a relatively equal footing in regard to budget and product positions at the beginning of the third period. The third period then constituted the first of eight periods during which the teams managed their respective firms. Operational Measures In a previous study, Hogarth and Makridakis (1981) operationalized performance in the MARKSTRAT environment as market share and cumulative net marketing contribution, since these may be monitored for each period of play. Hogarth and Makridakis considered these measures to be conceptually distinct, objective measures of different aspects of performance. Other measures of performance have been utilized within the context of testing the ability of computer models to improve performance, including: decision improvement, number of large errors, resource volatility, and profit volatility (McIntyre 1979; Chakravarti, Mitchell and Staelin 1979). However, resource and profit volatility are appropriate measures only within a closed, stable environment with no competitive reaction or environmental trends which may affect the measures. Decision improvement and number of large errors are only measurable where optimal outcomes are known to the researcher. Based upon Hogarth and Makridakis' research and upon a pilot study of the MARKSTRAT environment, we constructed the initial model with five measures of marketing performance:

-13 - market share in units for each of two distinct product markets, market share in dollars for each market, and cumulative net marketing contribution in dollars. Since these measures obviously overlap, we developed a multiple-item uni-dimensional instrument for performance. Consequently, we employed an analytical procedure, described below, which estimated both model parameters and psychometric characteristics. Decision Quality is necessarily a subjective component in a dynamic, competitive environment such as that of the MARKSTRAT game. It was measured by two scores based on instructors' evaluations of final team strategy reports. Scores were based on three issues: the depth of a team's evaluation of its situation in the market, the analysis supporting the evaluation, and adherence to marketing principles. Guidelines and procedures were furnished in course lecture materials as well as material contained in Bernhardt and Kinnear (1981, Chapter 1), Buzzell (1978), Corey (1978), and Abell and Hammond (1979). This assured that a common approach was used by students in developing team strategy reports and by instructors in evaluating them. This procedure separated decision quality from the objective performance measures. Each instructor independently graded each team's strategy report. Points were assigned to each of the three issues; the highest possible score was 48, and scores ranged from 21 to 47. Kendall's coefficient of concordance (.96) indicated a high degree of interrater reliability. Information Usage was operationalized as the amount of information which the subjects perceived that they used. In order to capture the cumulative consequence, we did not specify the decision for which it was used or what type of use was made of it. Amount of information was used to cover the two experimental conditions for availability of options described below. In one condition subjects were required to purchase all available research information, in

-14 - the other condition they could select from what was available. Consequently, the question was phrased so that perceived usefulness could be judged regardless of how much information was actually received. To simply ask how much information was used would seem to foster an upward bias toward those who received larger amounts. Consistent with the earlier discussion we also wanted to separate "usage" and "useful". The term "useful" would seem to bias the results toward a positive relationship with quality of decision and performance as, consistent with attribution theory, teams performing well may attribute it to rational (use of research information) behavior. Respondents indicated on a 7-point scale how much of the information which they had received was used (most-some). Attitude was measured toward the research information itself rather than toward any particular use. Again, this was done to avoid emphasis on any specific decision. Teams were asked for their agreement or disagreement with three statements describing the research information using commonly accepted evaluative adjectives (dislike-like, bad-good, foolish-wise) with responses scored 1 to 7. The teams reported the group's consensus opinion concerning both perceived information usage and attitude toward the information together with their strategic decisions for each period. This procedure was followed so that questionnaire items would blend with MARKSTRAT operating procedures which require team decisions as input. We felt that alternatives, such as averaging individual team member's scores, would be different enough from other required input to draw attention to the questionnaire items and lead to a bias. Competency was operationalized by three measures. The first two indicators were scores given by the two instructors, based on a team's written situation analysis, following observation of the first two periods' activities

-15 - and results. It seems likely that a firm's ability to synthesize and predict from past information would influence its ability to capitalize on forthcoming research information. The grading procedures followed the format described above for decision quality. Scores ranged from 4 to 20 out of a possible 20 points. The interrater reliability index was.93. These two measures were named Skill 1 and Skill 2. The third measure was an average of principal component scores for each firm member. Variable inputs, based on the aforementioned pilot study, were cumulative grade point average and GMAT total, verbal, and quantitative test scores. When submitted to a principal components analysis, along with undergraduate grade point average, these four variables all loaded above.57 on the first component and explained 46% of the total variance. This component was named ability. Students were ranked on the ability scores within each class section, and each set of three contiguously ranked students composed a team. Two-member teams, where necessary, were formed on the basis of the two most average student scores. This procedure resulted in 67 three-person and 3 two-person teams, for a total of 70 teams. An average of the members' ability scores was then used as the indicator for each team. The method of assignment guaranteed small withinteam relative to across-team ability variance. The initial operational model is shown in Figure 2. Measures are desigI I Insert Figure 2 about here I I nated as y and are shown in the boxes. The Is represent standardized weights for formative indicators (arrows toward the construct) and the Xs represent component loadings for reflective indicators (arrows toward the measure). The constructs, from Figure 1, are shown in circles; theoretical relationships are represented by the as.

-16 - Experimental Design Setting up the experiment involved three considerations. First, experimental procedures were developed to manipulate the availability of options, the cost of information, and availability of funds. These manipulations are shown in the dashed boxes in Figure 2. Second, firms were assigned to experimental conditions so that an even distribution of firms, industries and class sections was obtained across conditions. Thirdly, management teams were assigned to firms so there was no correlation between the team's level of competency (the average of the three members' ability component individual scores) and the experimental condition to which they were assigned. These three steps will be discussed in detail below. The three experimental conditions were operationalized through the use of company memos specifying changes within the company. These memos were distributed after each team had studied the MARKSTRAT manual and believed that they would be able to select the information desired at certain prices and that they would operate within a specified budget limitation. Firms assigned to the group having no option concerning information availability were informed that commencing with the third period —their first period of managing the company — they would not have the freedom to purchase the information they wished, but rather would be obligated to purchase all available marketing research reports for each subsequent period. Firms in the group having the option of purchasing any of the marketing research reports they chose received no such message. In addition, each firm received two messages, one concerning a change in the cost of information and another concerning a change in the budget limitation. All communications stated that changes were companyspecific. Firms in the highprice condition were notified that a surcharge of 35% was being added to each marketing research report's listed price, whereas those in the low-price

-17 - condition were notified of a 35% reduction. Firms in the high-funds condition were notified of a $60 million budget ceiling for the duration of the game, whereas those in the low-funds condition were given a $40 million ceiling (in contrast to the $50 million budget which all teams assumed they would have). Firms were randomly assigned to conditions. In theory, this procedure should have evenly dispersed class sections, teams, and industries across the eight conditions. However, random assignment of the small number of firms —9 firms to each of 6 conditions and 8 firms to each of 2 conditions —produced patterns that were apparent by inspection. Subsequent randomizations were carried out until inspection revealed no apparent patterns. Finally, teams were assigned to firms so that there was no correlation between competency scores and the firm's location in the design. This involved three objectives: 1) even allocation of teams across firms within an industry within a class section. 2) even allocation of teams across firms and industries within a section. 3) even allocation of teams across firms, industries and sections. A three-step systematic approach was adopted, as the team-to-cell ratio was too small to rely solely on randomization procedures. First, teams were rank ordered within class sections on the competency scale and were assigned an identification number. Second, teams from class section 1 were randomly allocated to firm by industry. The team ID's were then arranged in a firm-byindustry matrix which was duplicated for sections 2, 3, and 4. This procedure achieved the first two objectives. To achieve the third objective, teams within a class section were reassigned in the matrix such that each held the position of the team before it along a partial diagonal in the previous class section and the first team held the position of the last firm in the previous section and so forth (exhibit 2). An inspection of the scores by firm, section, and industry revealed no patterns in the competency distribution. This procedure is detailed in the Appendix.

-18 - ANALYSIS Partial least squares (PLS) analysis (Wold 1980 b,c), a structural equations methodology for estimating path models with indirectly observed latent variables, was judged appropriate for the analysis. While the LISREL latent variable model has become popular in the marketing literature,1 its objective of maximizing overall model fit requires a more solid theoretical base than is possible in this study, since a number of models may fit the data equally well under LISREL estimation. In addition, its restrictive multi-normality assumptions and dependency on large sample size and reflective indicators are not met in this design. In contrast, PLS is a prediction-oriented system, the objective of which is to explain observed variance. It is appropriate when theory is weaker due to the infancy of the research area or to multiple origins from various research traditions, both of which apply here. It is restricted only by ordinary least squares assumptions and operates well with small sample sizes. The algorithm uses a form of fixed point and least squares estimation. The variables representing the experimental treatments —option/no option to choose information, high/low price, and high/low funds —as well as information usage and firm performance, were specified as defined constructs and measured by formative indicators in the PLS model; thus, these constructs are weighted linear combinations of their indicators. The remaining variables were specified as imperfectly defined constructs and measured by reflective indicators. The analysis that follows has three major components. First, the initial model is specified in regard to measures and structure. In regard to measures, we followed the "purification" approach advocated by Churchill (1979), except that we employed simultaneous psychometric and structural parameter estimation. Structural model specification followed the traditional "trimming" approach (c.f., Kerlinger and Pedhazur 1973, Chapter 11).

-19 - Measurement Model Specification Figure 2 presented the hypothesized relationships, including the proposed measurement model. Although performance measures from the MARKSTRAT simulation had been previously used by Hogarth and Makridakis (1981), it was not clear whether they used market share figures for a particular market or for both markets (since MARKSTRAT involves 2 markets, market S and market V). We found that the market share units and market share dollars gained in different markets (market V and market S) were inappropriate measures which together did not accurately reflect total performance. Since the program calculates market share in a particular market based upon the number of entrants (of five), the first firm to enter market V (the newly evolving market) is attributed 100 percent of market share. This does not represent an accurate view of performance, since a firm doing poorly in market S which was the first to enter market V would be attributed the highest overall performance. Therefore, market share units and market share dollars for markets S and V (four measures total) were replaced by two measures: total market share (units) and total market share (dollars) which were calculated from total firm and market sales. Next, we examined the behavior of cumulative net marketing contribution, the final measure of performance. To do this, we ran PLS analyses for all eight periods using cumulative net marketing contribution up to the period in question and then re-analyzed the eight periods using total net marketing contribution at period 8. This improved the decision quality-performance relationship and explained variance in performance. However, since the measures of decision quality are static, this path became stable over all periods, and the R2 reflected explained variance for the last period only, as expected. Therefore, we did not replace the cumulative net marketing measures by the period 8 measure. However, we then eliminated cumulative net

-20 - marketing contribution from the model due to strong multi-collinearity with other performance measures with no significant changes in explained variance or structural relationships except a reduction in R2 in performance of 3% for period 1 only. This result does not support Hogarth and Makridakis' (1981) assumption that cumulative net market contribution and market share were independent. Since net marketing contribution is also not expected to reflect long-term decisions in period 1, this change was dictated in the interest of parsimony. Residual co-variance of the ability measure with the two skill indicators was high. When this measure was dropped from the analysis, explained variance in decision quality was reduced by 18% and model parameters were not significantly affected. Consequently, ability was retained as a measure in the final model due to its contribution to explained variance and its strong impact in the aforementioned pilot study. At this point, the measures well represent their respective constructs which are shown with their interrelationships in Figure 2. Structural model specification was conducted, as described in the following section, before final assessment of psychometric characteristics since PLS simultaneously models measurement and structural relationships. Structural Model Specification It appeared that periods 1 and 2 constituted a learning process for the firms, as coefficients stabilized in periods 3 through 7. Period 8 results were again erratic at times, indicating perhaps a "throw-away" period when subjects realized that the end of the game had been reached and may have, therefore, attempted "irrational' moves to make a "last-ditch" attempt to gain share (debriefing interviews at the end of the experiment supported these suspicions). Period 5 was then chosen as a representative period upon which subsequent model modifications, discussed below, were principally based. The final

-21 - model shown in Figure 3 reflects the elimination of one path modeled in Figure 2 to be zero (11,9), several other insignificant paths (864' 854' 874' 8989 875), where parsimoniously appropriate, as well as the addition of one path not predicted by previous research, (B96), and the retention of one path modeled to be zero (811,8). The analysis and rationale upon which these modifications were based are discussed below. The probability levels shown in Figure 3 were obtained from t-values computed with a jacknife procedure which Insert Figure 3 about here avoided parametric statistical assumptions which could not be met.2 Thus, the model does not rely on classical assumptions for derivative or inferential statistics. As expected, the direct effects of perceived information usage on performance ( 11,9) were insignificant, and deletion of this path did not significantly alter structural relationships or explained variance. However, the competency-performance path ( 11,8), which was hypothesized to be zero, was significant in the first three periods. Consequently, it was maintained in the model. Contrary to predictions, the competency-perceived information usage path (B98) was insignificant and negligible (coefficients ranged from -.13 to.05) and consequently was dropped from the model. The remaining paths that were dropped from the model were due to an unsuccessful experimental manipulation of availability of funds. The high/ low funding to availability of funds path (B43) was insignificant (paths ranged from -.18 to -.05) and only accounted for an average of 3% of the variance. This result was expected once the experiment was underway since a firm's allocated budget was based on net marketing contribution for the

-22 - preceding period and many firms did not approach the experimentally imposed budget ceiling. Therefore, the operationalization was judged unsuccessful and this variable and its associated paths (064' 854, and 074) were dropped. Unexpectedly, price was not significantly related to attitude for any of the eight periods. It was, however, related to availability of options for each period. This result may have been because the manipulation effects on options were stronger (R2 =.53) than those obtained on price (R2 =.14). Although dropping price did not change the explained variance in attitude, removing it did reduce the variance in options (R2 from.61 to.50). Consequently, it was retained in the model and the price-attitude path (875) was dropped. Having specified the structural form of the model, we proceeded with a simultaneous evaluation of structural model and measurement model fit. Structural Relationships The model shown in Figure 3 was well-behaved as residual co-variances were moderate and structural-measurement residual covariances were near zero. The model explained an increasing amount of performance variance over time, ranging from 17% in period 1 to 65% in period 8, averaging 42% across all periods. The average performance explained variance is due mostly to the direct effects of decision quality (39%) and much less to the direct effects of competency (2%). The competency-performance relationship (811,8) pattern was the opposite of the information usage-decision quality relationship (B10 9) as moderate relationships in the first three periods disappeared in the last five periods. The explained variance in decision quality was relatively constant over the eight periods (varying from 14% to 18%). Whereas most of the decision quality variance is directly attributable to competency (average of 14%)

-23 - rather than information usage (2%), the information usage/decision quality path (B 109) appeared as a step function going from insignificant in the first four periods to significant in the last four periods. Information usage explained variance averaged 23% over the eight periods, ranging from 13% to 32%, becoming steady and increasing considerably for the last four periods as compared to periods 2, 3, and 4. While availability of options only explained 10% of this variance, there was a strong optionsattitude interaction which accounted for 34%, and the attitude direct effect accounted for the remaining 56% of information usage variance. The manipulation effects on attitude accounted for an average of 12% of explained variance, none of which was present in the first two periods, with an average of 15% accounted for in the last six periods. All of the variance came from the direct effects of availability of options which, as mentioned above, may have overshadowed the weaker experimental effects obtained for the price manipulation. Psychometric Characteristics Figure 4 presents the results obtained for the measurement model. All loadings and most weights are significant at the.01 level across all eight periods. The insignificant weights associated with market share units in I I Insert Figure 4 about here periods 6, 7, and 8, and the weights of opposite sign in periods 2 and 7 are most likely due to a dependent relationship between performance indicators. For example, market share dollars is normally dependent upon market share units; however, a negative relationship may occur if a firm attempts to buy unit share. Communalities are high, and residual co-variances are negligible, with the noted exception of Ability with Skill 1 and Skill 2.

-24 - Average variance extracted (pvc (n)) captures the reliability of a construct by assessing the amount of variance in a construct relative to the amount of variance due to measurement error. At a minimum, values greater than.5 should be obtained; i.e., the variance due to the construct should outweigh measurement error variance (Fornell and Larcker 1982). Average variance extracted is much greater than.5 for all constructs (Table 1); therefore model Insert Table 1 about here constructs may be regarded as reliable. Discriminant validity was assessed by 2 s2 comparing (c (n)) with R2 Since PLS standardizes variables, R is equal to the squared multiple correlation associated with a construct (Fornell and Larcker 1982). The final model satisfies the requirement for discriminant validity that the variance shared between a construct and its indicators be significantly greater than the variance shared between the construct and neighboring constructs (pv(n) > R2) Overall, the psychometric characteristics of the adjusted model in Figure 3 appear satisfactory. DISCUSSION There is consistent support for learning effects and the cumulative impact of variables on performance. Performance appears to be primarily a function of competency during early periods, with decision quality becoming steadily stronger and replacing competency over time. Therefore, it appears that decision quality pays off in the long-run cumulative effects. On the other hand, common ability may pay off in the short-term only to be replaced by decision quality (based on marketing principles) in the long run. The role played by information usage in the model appears to support either of these

-25 - speculations, as it did not significantly impact decision quality until the last four periods. This finding could be due to learning in that four periods of experience may have been necessary before information usage actually resulted in higher decision quality. While there was a slight decline in the attitude information usage relationship over time, it was not as pronounced as expected. The stronger relationships occurred in the first two periods where there were also no effects on attitude from availability of options. The moderate effects from attitude on information usage occurred during the last six periods when availability of options did have an impact on attitude. Again, there appear to be learning effects regarding the experimental effects. It may be that these effects, during the later periods, offset a possible decline in the natural (unexperimentally induced) attitude-information usage relationship. Regardless of specific variable relationships, the above discussion strongly supports the dynamic nature of this system. An interesting issue for future research is the nature of learning effects arising from experience with information and its adaptation to decision making. In addition to dynamic implications, there were a number of static implications as well. Although there was a relationship between information usage and decision quality for the later periods, it was weak. This finding is at variance with current marketing thought but is consistent with decision theory research (c.f., Hogarth 1980). This finding is surprising, however, given the previously mentioned analytic orientation of the decision-makers as well as the relevance and actionability of the information itself. In regard to the latter issue, the information was specifically tailored to reduce uncertainty regarding key strategic decisions. For example, future shifts in segment ideal points could be predicted accurately once obvious patterns were spotted in past

-26 - reports. It appears, as with most products, that after-sales service and follow-up may be necessary to ensure that potential benefits are realized. In the case of market research information, as with many industrial services, custom tailoring may be necessary across individuals and over time as they gain experience. This view is consistent with the position of "evolutionary design" proponents who argue that individual decision-makers require different information in different forms at different times to be more effective and, therefore, require systems flexible enough to respond to individual needs (Keen and Scott-Morton 1978). At any rate, there appear to be one or more missing links which tie information use to quality of decision making. The computer-based marketing decision support system may be such a missing link. By allowing the rapid manipulation of data and testing of assumptions, such a system allows important information within the data to emerge and permits more time for creative thinking and learning. Casual observation of MARKSTRAT participants by course instructors over (the past years) suggests this idea. Those students who utilized available computer resources appeared to develop a better understanding of the MARKSTRAT environment and make better long-term decisions which affect performance. To date, no research has delved into this fascinating area. Only a few studies (McIntyre 1979; Chakravarti, et al., 1979) have attempted to assess the impact of computerized systems on decision quality; none have attempted to do so in a dynamic environment. We suggest, therefore, that future research investigate the impact of computer-based decision support on information evaluation and use and on decision quality within a dynamic environment. On the other hand, casual observation from MARKSTRAT use suggests that firms which develop a core strategy in the first few periods which is then adjusted in subsequent periods produce consistently good performance. To put

-27 - it simply, we propose that decision-makers must know what they are looking for before they can find it or before support systems can be designed to help them find it. If our proposed causal ordering is correct, then the inexperience of our subjects would account for many of our findings. While the same may hold for managers in new or changing environments or those recently exposed to sophisticated research tools, such generalizations are tenuous for reasons discussed in the Limitations section below. The study produced some interesting findings regarding the antecedents of information usage. Optional information was more favorably evaluated than unrequested information. There are implications here for data processing and marketing research departments currently supplying routine and/or unrequested reports to decision-makers. Forcing information on decision-makers does not appear to encourage use; rather it appears to decrease it, mainly due to the moderating effects of attitude. Price had an impact only on options and did not directly impact attitude. However, findings regarding specific antecedents to attitude are very tentative, since the funding manipulation was unsuccessful and the price manipulation was not strong. Taken as a whole, however, the findings suggest that marketing research usage is susceptible to influences under managerial control that are not necessarily related to the information itself. In this sense, information appears marketable using standard well-known methods rooted in socio-psychological theory. Future research should investigate the viability of alternative motivational approaches for fostering information usage. LIMITATIONS The limitations of this research include the use of single indicators and static indicators for some of the variables in the model. Future research

-28 - should incorporate the use of multiple indicators wherever possible and measure all indicators on a more frequent basis when several periods of data are available. The use of successive periods of data allowed us to trace the effects of experimental treatments and other hypothesized relationships over time. Our results indicate that it is inappropriate to utilize a single or only a few periods of data from a strategic environment, since the results change over time. The use of second-year MBA students may introduce some bias, since these subjects are trained to be high information users and have less experience, on the average, than marketing decision-makers. That we did not find a strong perceived information usage/decision quality link for these analytically trained decision-makers suggests a strong test since a sampling bias would more likely induce a positive finding. The necessity for obtaining enough team members to achieve the number of observations necessary to test the model necessitated the use of available MBA classes. In reviewing the relevant literature, Ashton and Kramer (1980) reported that studies which have focused on decision-making have found considerable similarities in the decisions and apparent underlying informationprocessing behavior of student and non-student groups. Hence, MBA students, the decision-makers of the near future, may be typical of inexperienced managers. On the other hand, learning and socialization factors for a new manager would be quite different from that of a second year MBA student playing MARKSTRAT with known colleagues. Managers also have the very real incentives of promotions, bonuses, or loss of job, motivators to make high quality decisions somewhat different from classroom grades. Thus, MBA students are only approximations of managers. The number of observations in this study was small. This does not, however, cause estimation problems in PLS (Wold 1980a). In fact, parameter

-29 - value changes over time appeared to be systematic, thus suggesting a low possibility of unstable estimation. The results from this study raise both methodological and substantive issues. The use of simulations such as MARKSTRAT allow tests of strategic marketing issues previously not amenable to empirical research. In the present research we were able to analyze changes over a simulated eight-year period. The study only approximates an actual marketing environment as the simulation of ten years into a 10 week game may provide an upward bias to learning effects. It seems memory decay would be more likely for information gathered in a less compressed and more complex "real world" marketing environment. The "real world" also contains factors such as changing environmental uncertainty, political risks, and socialization issues not considered in this study. Nonetheless, the simulation provided control not possible in a "real" logitudinal study. The lack of ecological validity in a management simulation environment is a necessary research tradeoff between realism and control. When using simulations such as MARKSTRAT researchers must take pains to understand the workings of the simulated environment since it may, similar to real situations, contain artifacts that would invalidate findings. In the present case, failure to correct the advantages certain teams had as a result of MARKSTRAT starting positions, together with failure of subjects to learn procedural matters, could have affected performance regardless of information usage or decision quality. As Ross (1987) points out, the other study (Hogarth and Makridakis 1981) known to us that used the MARKSTRAT simulation did not control for initial financial and market share positions. It also appeared to neglect consideration of the biased impact which accompanies being first-toenter in the second major MARKSTRAT market. An inappropriate assumption was also made regarding cumulative net marketing contribution and market share

-30 - measure independence. These problems render their results tenuous. Thus, future researchers should be sensitive to the possibility of simulation artifacts. In addition to controlling possible artifacts, there seems little to be gained from manipulating inputs and examining game outputs, as is commonly done in the simulation literature, unless intervening variables are included that attempt to capture the nature of decisions. In the present study, intervening variables such as attitude, information usage, individual competency, and decision quality played key roles and gave some insights into the how and why of information usage. A final limitation arises from the way in which the model was fitted. More specifically, the model was initially fitted and subsequently tested on the same data. A hold-out sample was precluded by the limited availability of subjects. Replication was not possible as modifications to the simulation and its administration are underway to introduce delivery and usage support systems, a need strongly suggested from this study. Consequently, the findings regarding the total model fit should be regarded as exploratory. This limitation is offset to some degree by the consistent and explainable patterns of relationships found over the eight time periods which suggests model stability, at least within the present sample. CONCLUSIONS Knowledge concerning the use of marketing research information is still in its infancy. Prior investigations have focused on types of marketing research requested or made available by marketing research departments and factors affecting use. However, while information usage is a multi-dimensional concept which has been characterized as a process of cumulative consequence, these

-31 - investigations have focused on instrumental use. Likewise, while it seems reasonable that the value of information lies in its impact on performance, previous research did not examine the use-performance relationship. This study attempted to add to previous knowledge by presenting an organizing model of the correlates of attitude toward information and of information usage, as well as the impact of information usage on performance over time. Results suggest that, while availability of options impacts attitude and information usage, both directly and indirectly, information usage is only weakly linked to decision quality. Support for conceptual usage was also found. Perceived information usagedecision quality linkages only developed over time, the decision qualityperformance relationship became stronger once these linkages were established, and competency, which had driven performance before these linkages were established, stopped impacting performance. This finding may lessen the considerable brow-beating that accompanies the frequent failure to empirically support instrumental use. By its very nature, conceptual usage is a difficult concept to study. The multiple objectives sought in this research in order to do so resulted in several limitations. Consequently, the findings from this initial effort should be regarded tentatively. Replications and future research solutions to the limitations are obviously needed. For example, the use of real managers, possibly participants in a large scale executive program, would be one direction for future research. It appears that a marketing research information marketing program is a viable concept. In their pioneering study, Deshpande and Zaltman (1982) suggest that organizational and informational attributes impact research information application to a particular decision. In addition, our research suggests

-32 -that traditional variables such as price and attitude are also relevant and that general information usage may impact cumulative performance over time. Thus, information usage should not be judged only in terms of the decision at hand. There also appears to be a need to develop delivery and usage support systems that would strengthen the information usage/performance relationship. The dynamic relationships found in this research are suggestive of learning. Future research should also explore the nature of this learning as such knowledge is fundamental in designing effective information support systems.

-33 -Footnotes 1. See Special Edition of the Journal of Marketing Research (November 1982). 2. Lohmollor's (1981) program does not compute t statistics associated with the path coefficients. Consequently, the program was modified to include familiar jacknifing procedures (e.g., Crask and Perreault 1977; Fenwick 1979; Wildt, et al., 1982). Whereas jacknifed t values may be larger or smaller than classical values (cf., Fenwick 1979), it is not known, within the context of econometric models, if this is due to assumptions underlying the standard formulas (Bergstrtm and Wold 1983, p. 113). Consequently, classical t statistics obtained with Noonan's (1981) program were obtained for purposes of comparison, which did not change the results. 3. Since both of these constructs were measured by static indicators, only one beta estimate is shown in Figure 3.

-34 -TABLE 1 Reliability and Validity Assessment Construct Attitude (n7) Competency (n8) Decision Quality (n 1) Reliability range.73 -.96 N/A N/A (p ) Vc mean.84.63.96 Discriminant Validity (R ) range mean.00 -.27.13 N/A N/A.14 -.17.15 'Assessment of convergent and discriminant validity are inappropriate for constructs defined by formative or single indicators.

-35 -FIGURE 1 Proposed General Model

FI(tCKi 2 Initiaal Operaional Model 11, 21) 76. — A I I I 11, Note: The experimental variables contain both observed (y) and unobserved (n) notation as they were treated. for computational purposes, as perfectly observed variables.

FIGURE 3 Adjusted Model, Structural Results R2 rl 6 61 Est. P.64.01.62.01.71.01.70.01.67.01.76.01 1).49 2).39 3).58 4).60 5).61 6).68 7).64 8).28 876 Eat. P.05 NS.15 NS.37.01.29.01.39.01.42.01.32.01.52.01 R2Rn72n R21 R _ 7 R,9 Ru2,10 I 1 1).00 1).32 1).14 1).17 2).02 97 2).15 2).14 2).20 3) t. 3).17 3).14 3).31 4).08 4).13 4).15 4).31 5).15 1).57.01 ).28 5).18 5).46 6).17 2).40.01 6).23 6).15 6).60 7).10 3).25.01 7).25 7).15 7).64 8).27 4).34.01 8).27 8).17 8).65 5).18.01 6).42.01 7).31.01 r- -- jOption/ \No Opt lone I:Y (n) i ^65 Eat. P 1) -.32.01 2).12.05 3) -.21.01 4) -.19.01 5).33.01 6) -.30.01 7) -.41.01 r- —. ---' 8) -.34.01 B10 9 Eat. P 896 Eat. P 1) -.07.05 2) -.05 NS 3).24.04 4).04 NS 5).43.01 6).12 NS 7).31.01 8).21.05 1) -.06 NS 2) -.05 NS 3) -.01 NS 4).10 NS 5).20.01 6).10.01 7).09.01 8).17.01 1).19.01 2).28.05 3).49.01 4).56.01 5).69.01 6).79.01 7).81.01 11,8 Eat. P 1).31.01 2).26.01 3).13.01 4).01 NS 5) -.04 NS 6) -.05 NS 7) -.03 NS 8).08 NS I I i~ i^ 52 Eat. P 1).40.01 2).28.01 3).41.01 4).42.01 5).40.01 6).32.01 7).39.01 8).40.01 a2n "5 1).16 2).08 3).17 4).17 5).16 6).10 7).15 8).16 E 0t..37.01.37.01 Significant tets were based on Jackknifing estimates; therefore equivalent path estlates ma y not reflect.imilar t values. Only significant levels.05 a*d.01 are given. NOTE; The numbers I through 8 In the vectors of coefficients refer to time periods. (1) refers to relationships at the end of the first year, (2) at end of the second year, and so forth. Performance la cumulative up to the year Indicated. of the

FIGURE 4 Adjusted Model —Measurement Results I wo 00 I A77 1).98 2).85 3).92 4).94 5).93 6).90 7).95 8).95 A78.98.89.92.96.83.90.91.94 A79.98.83.88.98.91.88.94.92 A 416 1).45 2 -.44 4).49 5).39 6).10* 7) -.09* 8).02* A1117.69 1.36.86.59.62.91 1.08.98 Note: Measurement and structural model results were obtained simultaneously. have been omitted for simplicity. Structural model paths

-39 - REFERENCES Abell, D. F., and J. S. Hammond (1979), Strategic Market Planning: Problems and Analytical Approaches, Englewood Cliffs: Prentice-Hall, Inc. Adler, L., and C. S. Mayer (1977), Managing the Market Research Function, AMA Monograph No. 5., Chicago: American Marketing Association. Ashton, R. H. and S. S. Kramer (1980), "Students as Surrogates in Behavioral Accounting Research: Some Evidence," Journal of Accounting Research, 18 (Spring), 1-15. Atkinson, J. W. (1964), An Introduction to Motivation, Princeton, New Jersey: Van Nostrand. BergstrOm, R., and H. Wold (1983), Fixed-Point Estimation in Theory and Practice, Gottingen, Germany: Vandenhoeck and Ruprecht. Bellenger, Danny N. (1979), "The Marketing Manager's View of Marketing Research," Business Horizons, 22 (June), 59-65. Bernhardt, K. L., and T. C. Kinnear (1981), Cases in Marketing Management, Rev. Edition, Plano, Texas: Business Publications, Inc. Biggs, W. D., and P. S. Greenlaw (1976), The Role of Information in a Functional Business Game," Simulation and Games, Vol. 7, March, p. 53-64. Brock, T. C. (1968), "Implications of Commodity Theory for Value Change," in Psychological Foundation of Attitudes, A. G. Greenwald, T. C. Brock, T. M. Ostrom, Eds., New York: Academic Press. Buzzell, D. (1978), "Note on Market Definition and Segmentation," Boston: President and Fellows of Harvard College. Buzzell, R. D., D. F. Cox, and R. V. Brown (1969), Marketing Research and Information Systems, New York: McGraw-Hill Book Company. Chakravarti, D., A. Mitchell, and R. Staelin (1979), "Judgment Based Marketing Decision Models: An Experimental Investigation of the Decision Calculus Approach," Management Science, Vol. 25, No. 3, March, 251-263. Churchill, G. (1979), "A Paradigm for Developing Better Measures of Marketing Constructs," Journal of Marketing Research, 16, 64-73. Cooper, J. and R. T. Croyle (1984), "Attitudes and Attitude Change," Annual Review of Psychology, Vol. 35, 395-426. Corey, E. (1978), "Marketing Strategy-An Overview," Boston: President and Fellows of Harvard College. Crask, M. R., and W. D. Perreault, Jr. (1977), "Validation of Discriminant Analysis in Marketing Research," Journal of Marketing Research, 14 (February), 60-68.

-40 - Crockett, W. H. (1965), "Cognitive Complexity and Impression Formation," in Progress in Experimental Personality Research, Vol. 2, B. A. Maher, Ed., New York: Academic Press. Deshpande, R. (1982), "The Organizational Context of Market Research Use," Journal of Marketing, Vol. 46, No. 4 (Fall), 91-101. Deshpande, R. and S. Jeffries (1981), "Attitude Affecting the Use of Marketing Research in Decision Making: An Empirical Investigation," in Educators' Conference Proceedings, Series #47, K. L. Bernhardt, et al., eds. Chicago: American Marketing Association, 1-4. Deshpande, R., and G. Zaltman (1982), "Factors Affecting the Use of Market Research Information: A Path Analysis," Journal of Marketing Research, 14 (February), 14-31. Deshpande, R. and G. Zaltman (1983), "Patterns of Research Use in Private and Public Sectors," Knowledge: Creation, Diffusion, Utilization, 4 (June), 561-75. Deshpande, R. and G. Zaltman (1984), "A Comparison of Factors Affecting Researcher and Manager Perceptions of Market Researcher and Manager Perceptions of Market Research Use," Journal of Marketing Research, 21 (February), 32-8. Dyer, R. F., and T. A. Shimp (1977), "Enhancing the Role of Marketing Research in Public Policy Decision Making," Journal of Marketing, 41 (January), 63-67. Fenwick, I. (1979), "Techniques in Market Measurement: The Jacknife," Journal of Marketing Research, 16 (August), 410-440. Fishbein, M., and I. Ajzen (1975), Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research, Reading Massachusetts: AldisonWesley Publishing Company. Fornell, C., and D. F. Larcker (1982), "Evaluating Structural Equation Models with Unobservable Variables and Measurement Error," A Second Generation of Multivariate Analysis, Vol. 2: Measurement and Evaluation, C. Fornell, Ed., New York: Praeger Publishers. Hogarth, R. M. (1980), Judgement and Choice: The Psychology of Decision, Wiley, Chichester, England. Hogarth, R. M. (1981), "Beyond Discrete Biases: Functional and Dysfunctional Aspects of Judgemental Heuristics," Psychological Bulletin, Vol. 90, No. 2, 197-217. Hogarth, R. M., and S. Makridakis (1981), "The Value of Decision Making in a Complex Environment: An Experimental Approach," Management Science, 27 (January), 93-107.

-41 - Holbert, N. A. (1974), "How Managers See Marketing Research, Journal of Advertising Research, 14 (December), 41-46. Howard, J. A., and J. N. Sheth (1969), The Theory of Buyer Behavior, New York: Wiley. Isen, A. M. (1984), "Toward Understanding the Role of Affect in Cognition," in Handbook of Social Cognition, R. Wyer and T. Srull, Eds., Hillsdale, NJ: Erlbaum, 1984. Jacoby, J. (1974), "Brand Choice Behavior as a Function of Information Load: Replication and Extension," The Journal of Consumer Research, 1 (June), 33-42. Jacoby, J. (1984), "Perspectives on Information Overload," Journal of Consumer Research, Vol. 10 (March), 432-435. Jacoby, J., D. F. Speller, and C. A. Kohn (1974), "Brand Choice as a Function of Information Load," Journal of Marketing Research, 10 (February), 63-69. Keen, P. G. W., and M. S. Scott-Morton (1978), Decison Support Systems: An Organizational Perspective, Reading, MA: Addison Valley. Kerlinger, N., and J. Pedhazur (1973), Multiple Regression in Behavior Research, New York: Holt, Rinehart, and Winton. Kinnear, T. C., and J. R. Taylor (1979), Marketing Research: An Applied Approach, New York: McGraw-Hill Book Company. Krum, R. (1978), "B for Marketing Research Departments," Journal of Marketing, 42 (October), 8-12. Lambert, D. R. (1980), "On Compensatory Demand Functions in Marketing Simulations," in D. C. Brenenstuhl and W. D. Biggs, Eds., Experimental Learning Enters the Eighties, Bureau of Business and Economic Research, College of Business Administration, Arizona State University, Tempe, Arizona. Larreche, J. C., and H. Gatignon (1977), Markstrat: A Marketing Strategy Game, Palo Alto: Scientific Press. Leavitt, H. J. (1954), "A Note on Some Experimental Findings About the Meaning of Price," Journal of Business, 27 (July), 205-210. Lee, H., F. Acito, and R. L. Day (1987), "Evaluation and Use of Marketing Research by Decision Makers: A Behavioral Simulation," Journal of Marketing Research, 24 (May), 187-96. Lohmoller, J. B. (1981), "LVPLS 1.6 Program Manual: Latent Variables Path Analysis with Partial Least-Squares Estmation, MAnchen: Hochschule der Burdesivehr. Machlup, F. (1979), "Uses, Value and Benefits of Knowledge," Knowledge: Creation, Diffusion, Utilization, 1 (September), 62-81.

-42 - Malhotra, K. (1984), "Reflections on the Information Overload Paradigm in Consumer Decision Making," Journal of Consumer Research, Vol. 10 (March), 436-440. Martin, M. W., and J. Sell (1980), "The Marginal Utility of Information —Its Effect Upon Decision-Making," Sociological Quarterly, No. 2, 233-242. McIntyre, S. H. (1979) "The Leverage Impact of Judgment Based Marketing Models," Unpublished doctoral dissertation, Stanford University. McGuire, W. J. (1976), "Some Internal Factors Influencing Consumer Choice," The Journal of Consumer Research, 2 (March), 302-319. Medlin, M. (1979), "The Value of Information in a Sealed Bidding Experiment," Organizational Behavior and Human Performance, 24, 269-286. Monroe, K. B. (1973), "Buyer's Subjective Perceptions of Price," Journal of Marketing Research, 10 (February), 70-80. Monroe, K. B. (1976), "The Influence of Price Differences and Brand Familiarity on Brand Preference," Journal of Consumer Research, 3 (June), 42-49. Myers, J. G., W. F. Massy, and S. A. Greyser (1980), Marketing Research and Knowledge Development: An Assessment for Marketing Management. Englewood Cliffs, New Jersey: Prentice-Hall, Inc. Noonon, R. (1981), A Manual for the PLS Program, Institute of International Education, Stockholm, Sweden. Osgood, C. E., G. J. Suci, and P. H. Tannenbaum (1957), The Measurement of Meaning, Urbana: University of Illinois Press. Oskamp, S. (1965), "Overconfidence in Care-Study Judgements," Journal of Consulting Psychology, 29, 261-265. Pitz, G. (1980), Sensitivity of Direct and Derived Judgments to Probabilistic Information," Journal of Applied Psychology, Vol. 65 (2) (April), 164-171. Rich, R. F. (1975), "Selective Utilization of Social Science Related Information by Federal Policy Makers," in Inquiry, 8, 239-45. Rich, R. F. (1977), "Use of Social Science Information by Federal Bureaucrats: Knowledge for Action vs. Knowledge for Understanding," in Using Social Research in Public Policy Making, Carol H. Weiss, ed., Lexington, MA: Lexington-Heath, 199-211. Rich, R. F. (1979), "Editor's Introduction," American Behavioral Scientist, Vol. 22, No. 3, J-F, 327-338. Ross, W. T. (1987), "A Re-examination of the Results of Hogarth and Makridakis. The Value of Decision Making in a Complex Environment: An Experimental Approach," Management Science, 33 (February), 288-96.

-43 - Ryan, M. J., and E. H. Bonfield (1975), "The Fishbein Extended Model and Consumer Behavior," Journal of Consumer Research, 2 (September), 118-136. Tull, D. S., R. A. Boring, and M. H. Gonsior (1964), "A Note on the Relationship of Price and Imputed Quality," Journal of Business, 37 (April), 186-191. Weiss, C. H. (1980), "Knowledge Creep and Decision Accretion," Knowledge: Creation, Diffusion, Utilization, 1 (March), 381-404. Weiss, C. H., and M. J. Bucuvalas (1980), "The Challenge of Social Research to Decision Making," in Using Social Research in Public Policy Making, Carol H. Weiss, ed., Lexington, MA: Lexington-Heath, 199-211. Weiss, C. H., and M. J. Bucuvalas (1980), "Truth Tests and Utility Tests: Decision Makers' Frames of Reference for Social Science Research," American Sociological Review, Vol. 45 (April), 303-312. Wildt, A. R., Z. V. Lambert, and R. M. Durand (1982), "Applying The Jacknife Statistic in Testing and Interpreting Canonical Weights, Loadings, and Cross-Loadings," Journal of Marketing Research, 19 (February), 99-107. Wilton, P. C. and J. G. Myers (1986), "Task, Expectation, and Information Assessment Effects in Information Utilization Processes," Journal of Consumer Research, 12 (March), 469-86. Wold, H. (1980a), "Factors Influencing the Outcome of Economic Sanctions, An Application of Soft Modeling," paper presented at Fourth World Congress of Econometric Society, February, Aix-en-Provence, France. Wold, H. (1980b), "Model Construction and Evaluation When Theoretical Knowledge is Scarce —Theory and Application of Partial Least Squares," in Evaluation of Econometric Models, J. Kmenta and J. G. Ramsey, Eds., New York: Academic Press. Wold, H. (1980c), "Soft Modelling: Intermediate Between Traditional Model Building and Data Analysis," Mathematical Statistics, 6, 333-346. Zellinger, 0. A., H. L. Fromkin, D. E. Speller, and C. A. Kohn (1975), "A Commodity Theory Analysis of the Effect of Age Restrictions Upon Pornographic Materials," Journal of Applied Psychology, 60, 94-99. Zif, J. (1976), "Optional Versus Fixed Information System in a Simulation Game," Simulation of Games, 7 (March), 35-52.

-44 - APPENDIX Method for Assigning Teams to Firms Problem: Students were in four sections, thus sections Si, i 1....4 Each section had industries, as follows industries =1ij j = 1....4 for i < 4 j = 1, 2 for i = 4 Each industry had five firms, thus firms Fijk k = 1....5 Each section had groups, such that groups = Gig = 1....20 for i < 4 2 = 1.... 10 for i = 4 The groups were ordered by a competence scale such that order - Gi + 1 > G on the scale Groups had to be assigned to firms such that there was no correlation between the competence scale and firms. The experiment can be described as an asymmetric rectangular design with si (Iij x Fijk) 3 x (4 x 5) + 1 x (2 x 5) for which 70 groups had to be assigned with Si (Gi ) 3 x (20) + 1 x (10) Objectives: The problem (see figure 1) can be translated into the following three hierarchical objectives: 01: Random allocation of groups across firms within an industry, within a section.

-45 - 02: Random allocation of groups across firms and industries within a section. 03: Random allocation of groups across firms, industries and sections. Method: A systematic approach was adopted as the cells were too few to achieve the objectives by random number. The groups were ordered on the competence scale from 1-20. These groups were then allocated to the rectangular matrix along a partial diagonal route (see figure 2) for each section which would best achieve objectives 1 and 2. To achieve objective 3, the above allocation rule was repeated for sections S., i > 1, but the firms were reassigned to the cells of the matrix such that each firm held the row of the firm before it in the previous section, and the first firm held the name of the last firm in the previous section, i.e., in general. Fij (k + 1) = F(i + )jk, with a wrap around.

-46 - FIGURE 1 THE EXPERIMENTAL DESIGN A. For Si, i < 4: Industry 1 2 3 4 Firms _____ 1 2 3 4 5 B. For S4: 40 Industry 1 2 Firms 1 2. 3 4

-47 -FIGURE 2 THE ASSIGNMENT ROUTE A. For Si. i < 4 *m doww- " 10% ft.0 do B. For S4: 4 Code: -4 Route Direction O Assignment Rank